Cosmos DB Connector for Azure Data Factory (SSIS) : Get table partition key ranges

Learn how to get table partition key ranges using the Cosmos DB Connector for Azure Data Factory (SSIS). This connector enables you to read and write Azure Cosmos DB data effortlessly. Query, integrate, and manage databases, containers, documents, and users — almost no coding required. We'll walk you through the exact setup.

Let's dive in!

Prerequisites

Before we begin, make sure the following prerequisites are met:

1. SSIS designer installed. Sometimes it is referred as BIDS or SSDT (download it from Microsoft).
2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
3. SSIS PowerPack is installed (if you are new to SSIS PowerPack, then get started!).

Get table partition key ranges in SSIS

1. Begin with opening Visual Studio and Create a New Project.

2. Select Integration Service Project and in new project window set the appropriate name and location for project. And click OK.

   In the new SSIS project screen you will find the following:

   • SSIS ToolBox on left side bar
   • Solution Explorer and Property Window on right bar
   • Control flow, data flow, event Handlers, Package Explorer in tab windows
   • Connection Manager Window in the bottom
   
   Note: If you don't see ZappySys SSIS PowerPack Task or Components in SSIS Toolbox, please refer to this help link.

3. Now, Drag and Drop SSIS Data Flow Task from SSIS Toolbox. Double click on the Data Flow Task to see Data Flow designer.

   

4. From the SSIS toolbox drag and API Source (Predefined Templates) on the data flow designer surface, and double click on it to edit it:
   

   

5. Select New Connection to create a new connection:
   

   

6. Use a preinstalled Cosmos DB Connector from Popular Connector List or press Search Online radio button to download Cosmos DB Connector. Once downloaded simply use it in the configuration:
   

   Cosmos DB

   

7. Now it's time to configure authentication. Firstly, configure authentication settings in Cosmos DB service and then proceed by configuring API Connection Manager. Start by expanding an authentication type:

   API Key

   Cosmos DB authentication

   Connecting to your Azure Cosmos DB data requires you to authenticate your REST API access. Follow the instructions below:

   1. Go to your Azure portal homepage: https://portal.azure.com/.
   2. In the search bar at the top of the homepage, enter Azure Cosmos DB. In the dropdown that appears, select Azure Cosmos DB.
   3. Click on the name of the database account you want to connect to (also copy and paste the name of the database account for later use).
   4. On the next page where you can see all of the database account information, look along the left side and select Keys:
   5. On the Keys page, you will have two tabs: Read-write Keys and Read-only Keys. If you are going to write data to your database, you need to remain on the Read-write Keys tab. If you are only going to read data from your database, you should select the Read-only Keys tab.
   6. On the Keys page, copy the PRIMARY KEY value and paste it somewhere for later use (the SECONDARY KEY value may also be copied and used).
   7. Now go to SSIS package or ODBC data source and use this PRIMARY KEY in API Key authentication configuration.
   8. Enter the primary or secondary key you recorded in step 6 into the Primary or Secondary Key field.
   9. Then enter the database account you recorded in step 3 into the Database Account field.
   10. Next, enter or select the default database you want to connect to using the Default Database field.
   11. Continue by entering or selecting the default table (i.e. container/collection) you want to connect to using the Default Table (Container/Collection) field.
   12. Select the Test Connection button at the bottom of the window to verify proper connectivity with your Azure Cosmos DB account.
   13. If the connection test succeeds, select OK.
   14. Done! Now you are ready to use Cosmos DB Connector!

   API Connection Manager configuration

   Just perform these simple steps to finish authentication configuration:

   1. Set Authentication Type to API Key [Http]
   2. Optional step. Modify API Base URL if needed (in most cases default will work).
   3. Fill in all the required parameters and set optional parameters if needed.
   4. Finally, hit OK button:

   Cosmos DB

   API Key [Http]

   https://[$Account$].documents.azure.com

   Required Parameters
   Primary or Secondary Key	Fill-in the parameter...
   Account Name (Case-Sensitive)	Fill-in the parameter...
   Database Name (keep blank to use default) Case-Sensitive	Fill-in the parameter...
   API Version	Fill-in the parameter...
   Optional Parameters
   Default Table (needed to invoke #DirectSQL)

   

8. Select Get table partition key ranges endpoint from the dropdown and hit Preview Data:

   

   API Source - Cosmos DB

   Read and write Azure Cosmos DB data effortlessly. Query, integrate, and manage databases, containers, documents, and users — almost no coding required.

   Cosmos DB

   Get table partition key ranges

   Required Parameters
   Table Name (Case-Sensitive)	Fill-in the parameter...
   Optional Parameters
   Database Name (keep blank to use default) Case-Sensitive

   

9. That's it! We are done! Just in a few clicks we configured the call to Cosmos DB using Cosmos DB Connector.

   You can load the source data into your desired destination using the Upsert Destination , which supports SQL Server, PostgreSQL, and Amazon Redshift. We also offer other destinations such as CSV , Excel , Azure Table , Salesforce , and more . You can check out our SSIS PowerPack Tasks and components for more options. (*loaded in Trash Destination)

   

Deploy SSIS package to Azure Data Factory (ADF)

Once your SSIS package is complete, deploy it to the Azure-SSIS runtime within Azure Data Factory. The setup process requires you to upload the SSIS PowerPack installer to Azure Blob Storage and then customize the runtime configuration using the main.cmd file. For a complete walkthrough of these steps, see our detailed guide on the Azure Data Factory (SSIS) and Cosmos DB integration.

Conclusion

And there you have it — a complete guide on how to get table partition key ranges in Azure Data Factory (SSIS) without writing complex code. All of this was powered by Cosmos DB Connector, which handled the REST API pagination and authentication for us automatically.

Download the trial now or ping us via chat if you have any questions or are looking for a specific feature (you can also reach out to us by submitting a ticket):

More actions supported by Cosmos DB Connector

Got another use case in mind? We've documented the exact setups for a variety of essential Cosmos DB operations directly in Azure Data Factory (SSIS), so you can skip the trial and error. Find your next step-by-step guide below:

• Create a document in the container
• Create Permission Token for a User (One Table)
• Create User for Database
• Delete a Document by Id
• Get All Documents for a Table
• Get All Users for a Database
• Get Database Information by Id or Name
• Get Document by Id
• Get List of Databases
• Get List of Tables
• Get table information by Id or Name
• Get User by Id or Name
• Query documents using Cosmos DB SQL query language
• Update Document in the Container
• Upsert a document in the container
• Make Generic REST API Request
• Make Generic REST API Request (Bulk Write)

More Cosmos DB integrations

SSIS

SQL Server

Microsoft Fabric

Power BI

SSRS

Informatica PowerCenter

MS Access

Excel

SSAS

C#

Python

Java

Tableau

SAP Crystal Reports

Azure Data Factory (Pipeline)

Talend Studio

UiPath

PowerShell

Alteryx

ODBC