ElasticSearch Connector for C#Read / write ElasticSearch data inside your app; perform many ElasticSearch operations without coding, just use easy to use high performance API Connector for ElasticSearch In this article you will learn how to quickly and efficiently integrate ElasticSearch data in C#. We will use high-performance ElasticSearch Connector to easily connect to ElasticSearch and then access the data inside C#. Let's follow the steps below to see how we can accomplish that! ElasticSearch Connector for C# is based on ZappySys API Driver which is part of ODBC PowerPack. It is a collection of high-performance ODBC drivers that enable you to integrate data in SQL Server, SSIS, a programming language, or any other ODBC-compatible application. ODBC PowerPack supports various file formats, sources and destinations, including REST/SOAP API, SFTP/FTP, storage services, and plain files, to mention a few. |
Connect to ElasticSearch in other apps
|
Create ODBC Data Source (DSN) based on ZappySys API Driver
Step-by-step instructions
To get data from ElasticSearch using C# we first need to create a DSN (Data Source) which will access data from ElasticSearch. We will later be able to read data using C#. Perform these steps:
-
Download and install ODBC PowerPack.
-
Open ODBC Data Sources (x64):
-
Create a User data source (User DSN) based on ZappySys API Driver
ZappySys API Driver-
Create and use User DSN
if the client application is run under a User Account.
This is an ideal option
in design-time , when developing a solution, e.g. in Visual Studio 2019. Use it for both type of applications - 64-bit and 32-bit. -
Create and use System DSN
if the client application is launched under a System Account, e.g. as a Windows Service.
Usually, this is an ideal option to use
in a production environment . Use ODBC Data Source Administrator (32-bit), instead of 64-bit version, if Windows Service is a 32-bit application.
-
Create and use User DSN
if the client application is run under a User Account.
This is an ideal option
-
When the Configuration window appears give your data source a name if you haven't done that already, then select "ElasticSearch" from the list of Popular Connectors. If "ElasticSearch" is not present in the list, then click "Search Online" and download it. Then set the path to the location where you downloaded it. Finally, click Continue >> to proceed with configuring the DSN:
ElasticsearchDSNElasticSearch -
Now it's time to configure the Connection Manager. Select Authentication Type, e.g. Token Authentication. Then select API Base URL (in most cases, the default one is the right one). More info is available in the Authentication section.
Steps how to get and use ElasticSearch credentials
For Local / Hosted Instance by you
- Get your userid / password and enter on the connection UI
For Managed Instance (By Bonsai search)
If your instance is hosted by bonsai then perform these steps to get your credentials for API call- Go to https://app.bonsai.io/clusters/{your-instance-id}/tokens
- Copy Access Key and Access Secret and enter on the connection UI. Click Test connection.
- If your Cluster has no data you can generate sample data by visiting this URL and click Add Sample Data https://{your-cluster-id}.apps.bonsaisearch.net/app/home#/tutorial_directory
Fill in all required parameters and set optional parameters if needed:
ElasticsearchDSNElasticSearchBasic Authentication (UserId/Password) [Http]http://localhost:9200Optional Parameters User Name (or Access Key) Password (or Access Secret) Ignore certificate related errors Fill in all required parameters and set optional parameters if needed:
ElasticsearchDSNElasticSearchWindows Authentication (No Password) [Http]http://localhost:9200Optional Parameters Ignore certificate related errors -
Once the data source connection has been configured, it's time to configure the SQL query. Select the Preview tab and then click Query Builder button to configure the SQL query:
ZappySys API Driver - ElasticSearchRead / write ElasticSearch data inside your app; perform many ElasticSearch operations without coding, just use easy to use high performance API Connector for ElasticSearchElasticsearchDSN -
Start by selecting the Table or Endpoint you are interested in and then configure the parameters. This will generate a query that we will use in C# to retrieve data from ElasticSearch. Hit OK button to use this query in the next step.
SELECT * FROM Indexes
Some parameters configured in this window will be passed to the ElasticSearch API, e.g. filtering parameters. It means that filtering will be done on the server side (instead of the client side), enabling you to get only the meaningful datamuch faster . -
Now hit Preview Data button to preview the data using the generated SQL query. If you are satisfied with the result, use this query in C#:
ZappySys API Driver - ElasticSearchRead / write ElasticSearch data inside your app; perform many ElasticSearch operations without coding, just use easy to use high performance API Connector for ElasticSearchElasticsearchDSNSELECT * FROM Indexes
You can also access data quickly from the tables dropdown by selecting <Select table>.AWHERE
clause,LIMIT
keyword will be performed on the client side, meaning that thewhole result set will be retrieved from the ElasticSearch API first, and only then the filtering will be applied to the data. If possible, it is recommended to use parameters in Query Builder to filter the data on the server side (in ElasticSearch servers). -
Click OK to finish creating the data source.
Video Tutorial
Read data in C#
Using ODBC DSN
-
Create a new Console project and use this code to read the data:
"ElasticsearchDSN"; -
Press F5 to run the code and read the data:
-
Here is the code in text format:
using System; using System.Data.Odbc; namespace ConsoleApp { class Program { static void Main() { var dsn = "ElasticsearchDSN"; using (var conn = new OdbcConnection(String.Format("DSN={0}", dsn))) { conn.Open(); var cmd = new OdbcCommand("SELECT * FROM Products", conn); //Increase the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios cmd.CommandTimeout=600; // 600-Seconds var rdr = cmd.ExecuteReader(); while (rdr.Read()) { for (int i = 0; i < rdr.FieldCount; i++) { Console.Write("{0}\t", rdr[i]); } Console.WriteLine(); } } Console.ReadKey(); } } }
Using a full ODBC connection string
If you want to avoid being dependent on a DSN and creating multiple DSNs for each platform (x86, x64), then you can use a fully qualified connection string. Simply go to your DSN and copy the Connection String:
-
Open ODBC data source configuration and click Copy settings:
ZappySys API Driver - ElasticSearchRead / write ElasticSearch data inside your app; perform many ElasticSearch operations without coding, just use easy to use high performance API Connector for ElasticSearchElasticsearchDSN
-
The window opens, telling us the connection string was successfully copied to the clipboard:
-
Then in your C# code, pass the connection string as an argument when calling the OdbcConnection object's constructor, for example:
var connectionString = @"DRIVER={ZappySys API Driver};ServiceUrl=http://localhost:9200;UserName=[$UserName$];Password=[$Password$]"; using (var conn = new OdbcConnection(connectionString)) { // your code goes here }
Limitations of using a full connection string
Despite using a full ODBC connection string may be very convenient it comes with a limitation: it's length is limited to 1024 symbols (or sometimes more). It usually happens when API provider generates a very long Refresh Token when OAuth is at play. If you are using such a long ODBC connection string, you may get this error:
"Connection string exceeds maximum allowed length of 1024"
But there is a solution to this by storing the full connection string in a file. Follow the steps below to achieve this:
- Open your ODBC data source.
- Click Copy settings button to copy a full connection string (see the previous section on how to accomplish that).
- Then create a new file, let's say, in C:\temp\odbc-connection-string.txt.
- Continue by pasting the copied connection string into a newly created file and save it.
-
Finally, the last step! Just construct a shorter ODBC connection string using this format:
DRIVER={ZappySys API Driver};SettingsFile=C:\temp\odbc-connection-string.txt
- Our troubles are over! Now you should be able to use this connection string in C# with no problems.
Actions supported by ElasticSearch Connector
Learn how to perform common ElasticSearch actions directly in C# with these how-to guides:
- Count documents
- Create Index
- Delete documents
- Delete Index
- Get document by ID from Index or Alias
- Get documents from Index or Alias
- Get Index or Alias metadata
- Insert documents
- List aliases
- List indexes
- Search / Query documents
- Update documents
- Upsert documents
- Make Generic API Request
- Make Generic API Request (Bulk Write)
Conclusion
In this article we showed you how to connect to ElasticSearch in C# and integrate data without any coding, saving you time and effort.
We encourage you to download ElasticSearch Connector for C# and see how easy it is to use it for yourself or your team.
If you have any questions, feel free to contact ZappySys support team. You can also open a live chat immediately by clicking on the chat icon below.
Download ElasticSearch Connector for C# Documentation