Dropbox Connector for Azure Data Factory (Pipeline)

Read / write Dropbox data inside your app; perform many Dropbox operations without coding, just use easy to use high performance API Connector for Dropbox

In this article you will learn how to quickly and efficiently integrate Dropbox data in Azure Data Factory (Pipeline) without coding. We will use high-performance Dropbox Connector to easily connect to Dropbox and then access the data inside Azure Data Factory (Pipeline).

Let's follow the steps below to see how we can accomplish that!

Download Documentation

Create ODBC Data Source (DSN) based on ZappySys API Driver

Step-by-step instructions

To get data from Dropbox using Azure Data Factory (Pipeline) we first need to create a DSN (Data Source) which will access data from Dropbox. We will later be able to read data using Azure Data Factory (Pipeline). Perform these steps:

  1. Download and install ODBC PowerPack.

  2. Open ODBC Data Sources (x64):

    Open ODBC Data Source

  3. Create a User data source (User DSN) based on ZappySys API Driver:

    ZappySys API Driver
    Create new User DSN for ZappySys API Driver
    • Create and use User DSN if the client application is run under a User Account. This is an ideal option in design-time, when developing a solution, e.g. in Visual Studio 2019. Use it for both type of applications - 64-bit and 32-bit.
    • Create and use System DSN if the client application is launched under a System Account, e.g. as a Windows Service. Usually, this is an ideal option to use in a production environment. Use ODBC Data Source Administrator (32-bit), instead of 64-bit version, if Windows Service is a 32-bit application.
    Azure Data Factory (Pipeline) uses a Service Account, when a solution is deployed to production environment, therefore for production environment you have to create and use a System DSN.

  4. When the Configuration window appears give your data source a name if you haven't done that already, then select "Dropbox" from the list of Popular Connectors. If "Dropbox" is not present in the list, then click "Search Online" and download it. Then set the path to the location where you downloaded it. Finally, click Continue >> to proceed with configuring the DSN:

    DropboxDSN
    Dropbox
    ODBC DSN Template Selection

  5. Now it's time to configure the Connection Manager. Select Authentication Type, e.g. Token Authentication. Then select API Base URL (in most cases, the default one is the right one). More info is available in the Authentication section.

    User Account

    Dropbox authentication

    To use OAuth authentication, firstly, you need to create OAuth application:

    1. Log into your Dropbox account.
    2. Go to Dropbox My Apps.
    3. Then press Create app button to create a new app: My OAuth apps in Dropbox
    4. Once a new page opens, select Scoped access option: Create OAuth app to get Dropbox REST API data
    5. Next, select Full Dropbox to access all files and folders or App folder to access specific folder's files and folders option.
    6. Continue by giving your app a name.
    7. Then check I agree to Dropbox API Terms and Conditions checkbox.
    8. Click Create app button.
    9. Once a new page opens, click Enable additional teams and Enable additional users buttons: Configure OAuth app in Dropbox
    10. Then copy App key and App secret and copy paste them into your favorite text editor (you will need them later).
    11. Proceed by setting a Redirect URI and clicking Add button.

      NOTE: If you don't have a working Redirect URI, you can use https://zappysys.com/oauth (it's safe).

    12. Then click on Permissions tab and select application scopes: Set OAuth app permissions in Dropbox
    13. Select all Individual Scopes and Team Scopes if you want to manage team data.
    14. Click Submit button.
    15. Now go to SSIS package or ODBC data source and in User Account authentication set these parameters:
    1. For ClientId field use App key value.
    2. For ClientSecret field use App secret value.
    3. For ReturnUrl field use Redirect URI value.
  6. Done! Now you are ready to use Dropbox Connector!
NOTE: If you are planning to use your current data connection/token for automated processes, we recommend that you use a generic account for token generation when the login box appears (e.g. sales_automation@mycompany.com instead of bob_smith@mycompany.com). When you use a personal account which is tied to a specific employee profile and that employee leaves the company, the token may become invalid and any automated processes using that token will fail. Another potentially unwanted effect of using a personal token is incorrect logging; the API calls (e.g. Read, Edit, Delete, Upload) made with that token will record the specific user as performing the calls instead of an automated process.
API Connection Manager configuration

Just perform these simple steps to finish authentication configuration:

  1. Set Authentication Type to User Account [OAuth]
  2. Optional step. Modify API Base URL if needed (in most cases default will work).
  3. Fill in all the required parameters and set optional parameters if needed.
  4. Press Generate Token button to generate the tokens.
  5. Finally, hit OK button:
DropboxDSN
Dropbox
User Account [OAuth]
https://api.dropboxapi.com/2/
Required Parameters
UseCustomApp Fill-in the parameter...
ReturnUrl Fill-in the parameter...
Optional Parameters
ClientId
ClientSecret
Scope
RetryMode RetryWhenStatusCodeMatch
RetryStatusCodeList 429
RetryCountMax 5
RetryMultiplyWaitTime True
ODBC DSN OAuth Connection Configuration