Video tutorial

Watch this quick video to see the integration in action. It walks you through the end-to-end setup, including:

  • Installing the SSIS PowerPack
  • Configuring a secure connection to ElasticSearch
  • Working with ElasticSearch data directly inside SSIS
  • Exploring advanced API Source features
While this video uses the OData Connector as an example, the core concepts and setup process are exactly the same for the ElasticSearch Connector.

Ready to dive in? Download the product to jump right in, or follow the step-by-step guide below to see how it works.

Prerequisites

Before we begin, make sure the following prerequisites are met:

  1. SQL Server Data Tools (SSDT) designer installed for Visual Studio.
  2. SQL Server Integration Services Projects 2022+ Visual Studio extension installed.
  3. SSIS PowerPack is installed.

List aliases in SSIS

  1. Open Visual Studio and click Create a new project.

  2. Select Integration Services Project. Enter a name and location for your project, then click OK.

  3. From the SSIS Toolbox, drag and drop a Data Flow Task onto the Control Flow surface, and double-click it:

    Drag Data Flow Task onto Control Flow to use SSIS PowerPack Data Flow components
  4. Make sure you are in the Data Flow Task designer:

    Make sure you are in Data Flow designer in SSIS package
  5. From the SSIS toolbox drag and API Source (Predefined Templates) on the data flow designer surface, and double click on it to edit it:

    SSIS API Source (Predefined Templates) - Drag and Drop
  6. Select New Connection to create a new connection:

    API Source - New Connection
  7. Use a preinstalled ElasticSearch Connector from Popular Connector List or press Search Online radio button to download ElasticSearch Connector. Once downloaded simply use it in the configuration:

    ElasticSearch
    ElasticSearch Connector Selection
  8. Select your authentication scenario below to expand connection configuration steps to:

    • Configure the authentication in ElasticSearch.
    • Enter those details into the API Connection Manager configuration.
    Basic Authentication (UserId/Password)

    ElasticSearch authentication

    Use HTTP Basic Authentication with any Elasticsearch-compatible cluster (self-hosted, Elastic Cloud, OpenSearch, or other managed hosts).

    1. Set Service URL to your cluster base URL (for example https://your-host:9200 or the HTTPS URL from your provider).
    2. Enter User Name and Password for a user that can call the Elasticsearch REST API. Some managed hosts label these as Access Key / Access Secret — use those values in the same fields.
    3. If you use a self-signed certificate and see a trust error, set Ignore certificate related errors to True (only if you trust the server).
    4. Click Test connection.

    Optional — Bonsai example: On Bonsai, open https://app.bonsai.io/clusters/{your-instance-id}/tokens, copy Access Key and Access Secret into User Name and Password, then test. Sample data (if needed): https://{your-cluster-id}.apps.bonsaisearch.net/app/home#/tutorial_directory.

    API Connection Manager configuration

    Just perform these simple steps to finish authentication configuration:

    1. Set Authentication Type to Basic Authentication (UserId/Password) [Http]
    2. Optional step. Modify API Base URL if needed (in most cases default will work).
    3. Fill in all the required parameters and set optional parameters if needed.
    4. Finally, hit OK button:
    ElasticSearch
    Basic Authentication (UserId/Password) [Http]
    http://localhost:9200
    Optional Parameters
    User Name (or Access Key)
    Password (or Access Secret)
    Ignore certificate related errors
    ZappySys Http Connection
    Find full details in the ElasticSearch Connector authentication reference.
    Windows Authentication (No Password)

    ElasticSearch authentication

    No instructions available.

    API Connection Manager configuration

    Just perform these simple steps to finish authentication configuration:

    1. Set Authentication Type to Windows Authentication (No Password) [Http]
    2. Optional step. Modify API Base URL if needed (in most cases default will work).
    3. Fill in all the required parameters and set optional parameters if needed.
    4. Finally, hit OK button:
    ElasticSearch
    Windows Authentication (No Password) [Http]
    http://localhost:9200
    Optional Parameters
    Ignore certificate related errors
    ZappySys Http Connection
    Find full details in the ElasticSearch Connector authentication reference.
  9. Select List aliases endpoint from the dropdown and hit Preview Data:

    API Source - ElasticSearch
    Read and write Elasticsearch data effortlessly. Integrate, manage, and automate indexes and documents — almost no coding required.
    ElasticSearch
    List aliases
    Optional Parameters
    EnableCustomReplace True
    SearchFor "(\w+(?<!aliases))"\s*:\s*\{\s*\}--regex
    ReplaceWith "$1": {fake:1}
    EnablePivot True
    IncludePivotPath True
    EnablePivotPathSearchReplace True
    PivotPathSearchFor (.*?).aliases.(.*)--regex
    PivotPathReplaceWith $1
    SSIS API Source - Read from table or endpoint
  10. That's it! We are done! Just in a few clicks we configured the call to ElasticSearch using ElasticSearch Connector.

    You can load the source data into your desired destination using the Upsert Destination , which supports SQL Server, PostgreSQL, and Amazon Redshift. We also offer other destinations such as CSV , Excel , Azure Table , Salesforce , and more . You can check out our SSIS PowerPack Tasks and components for more options. (*loaded in Trash Destination)

    Execute Package - Reading data from ElasticSearch and load into target

ElasticSearch Connector actions

Need another use case? Pick the next ElasticSearch action in SSIS below.

Conclusion

You now know how to list aliases in SSIS without writing complex code. ElasticSearch SSIS Connector handled pagination and authentication automatically.

Ready to get started? Download the trial or ping us via chat if you need help: