Video tutorial

Watch this quick video to see the integration in action. It walks you through the end-to-end setup, including:

  • Installing the SSIS PowerPack
  • Configuring a secure connection to ElasticSearch
  • Working with ElasticSearch data directly inside SSIS
  • Exploring advanced API Source features
While this video uses the OData Connector as an example, the core concepts and setup process are exactly the same for the ElasticSearch Connector.

Video doesn't load? Watch this video on YouTube.

Ready to dive in? Download the product to jump right in, or follow the step-by-step guide below to see how it works.

Prerequisites

Before we begin, make sure the following prerequisites are met:

  1. SQL Server Data Tools (SSDT) designer installed for Visual Studio.
  2. SQL Server Integration Services Projects 2022+ Visual Studio extension installed.
  3. SSIS PowerPack is installed.

Update documents in SSIS

  1. Open Visual Studio and click Create a new project.

  2. Select Integration Services Project. Enter a name and location for your project, then click OK.

  3. From the SSIS Toolbox, drag and drop a Data Flow Task onto the Control Flow surface, and double-click it:

    Drag Data Flow Task onto Control Flow to use SSIS PowerPack Data Flow components

  4. Make sure you are in the Data Flow Task designer:

    Make sure you are in Data Flow designer in SSIS package

  5. Read the data from the source, using any desired source component. You can even make an API call using the ZappySys JSON/XML/API Source and read data from there. In this example, we will use an OLE DB Source component to read real-time data from a SQL Server database.

  6. From the SSIS Toolbox drag and drop API Destination (Predefined Templates) on the Data Flow Designer surface and connect source component with it, and double click to edit it.
    SSIS API Destination (Predefined Templates) - Drag and Drop

  7. Select New Connection to create a new connection:

    API Destination - ElasticSearch
    Read and write Elasticsearch data effortlessly. Integrate, manage, and automate indexes and documents — almost no coding required.
    API Destination - ElasticSearch

  8. To configure the ElasticSearch connector, choose one of the following methods:

    • Choose from Popular Connector List: Select a pre-installed service directly from the dropdown menu.
    • Search Online: Use this to find and download a new connector file to your computer.
    • Use Saved/Downloaded File: Once the file is downloaded, browse your local drive to load it into the configuration.

    After that, just click Continue >>:

    ElasticSearch
    API Destination -

  9. Proceed with selecting the desired Authentication Type. Then select API Base URL (in most cases default one is the right one). Finally, fill in all the required parameters and set optional parameters if needed. You may press a link Steps to Configure which will help set certain parameters. More info is available in Authentication section.

    Basic Authentication (UserId/Password)

    ElasticSearch authentication

    Use HTTP Basic Authentication with any Elasticsearch-compatible cluster (self-hosted, Elastic Cloud, OpenSearch, or other managed hosts).

    1. Set Service URL to your cluster base URL (for example https://your-host:9200 or the HTTPS URL from your provider).
    2. Enter User Name and Password for a user that can call the Elasticsearch REST API. Some managed hosts label these as Access Key / Access Secret — use those values in the same fields.
    3. If you use a self-signed certificate and see a trust error, set Ignore certificate related errors to True (only if you trust the server).
    4. Click Test connection.

    Optional — Bonsai example: On Bonsai, open https://app.bonsai.io/clusters/{your-instance-id}/tokens, copy Access Key and Access Secret into User Name and Password, then test. Sample data (if needed): https://{your-cluster-id}.apps.bonsaisearch.net/app/home#/tutorial_directory.

    API Connection Manager configuration

    Just perform these simple steps to finish authentication configuration:

    1. Set Authentication Type to Basic Authentication (UserId/Password) [Http]
    2. Optional step. Modify API Base URL if needed (in most cases default will work).
    3. Fill in all the required parameters and set optional parameters if needed.
    4. Finally, hit OK button:
    ElasticSearch
    Basic Authentication (UserId/Password) [Http]
    http://localhost:9200
    Optional Parameters
    User Name (or Access Key)
    Password (or Access Secret)
    Ignore certificate related errors
    ZappySys Http Connection
    Find full details in the ElasticSearch Connector authentication reference.
    Windows Authentication (No Password)

    ElasticSearch authentication

    No instructions available.

    API Connection Manager configuration

    Just perform these simple steps to finish authentication configuration:

    1. Set Authentication Type to Windows Authentication (No Password) [Http]
    2. Optional step. Modify API Base URL if needed (in most cases default will work).
    3. Fill in all the required parameters and set optional parameters if needed.
    4. Finally, hit OK button:
    ElasticSearch
    Windows Authentication (No Password) [Http]
    http://localhost:9200
    Optional Parameters
    Ignore certificate related errors
    ZappySys Http Connection
    Find full details in the ElasticSearch Connector authentication reference.

  10. Select [Dynamic Table] table from the dropdown, then select Update as operation, and hit Preview Data:

    API Destination - ElasticSearch
    Read and write Elasticsearch data effortlessly. Integrate, manage, and automate indexes and documents — almost no coding required.
    ElasticSearch
    [Dynamic Table]
    Update
    Optional Parameters
    DoNotAddMultiContentSeparator True
    RowHeaderFooterContinueOnError True
    RowHeader
    SSIS API Destination - Access table operation

  11. Finally, map the desired columns:

    API Destination - ElasticSearch
    Read and write Elasticsearch data effortlessly. Integrate, manage, and automate indexes and documents — almost no coding required.
    API Destination - ElasticSearch

  12. That's it; we successfully configured the POST API Call. In a few clicks we configured the ElasticSearch API call using ZappySys ElasticSearch Connector

    Execute Package

ElasticSearch Connector actions

Need another use case? Pick the next ElasticSearch action in SSIS below.

Conclusion

You now know how to update documents in SSIS without writing complex code. ElasticSearch SSIS Connector handled pagination and authentication automatically.

Ready to get started? Download the trial or ping us via chat if you need help:

Download SSIS PowerPack
Chat with an Expert