How to call Zoom REST API using SSIS (OAuth / JWT)

Introduction

Tasks / Components used in this article.

	REST API Web Service Task
	JSON Source (REST API Source)

Prerequisites

Before we look into Step-By-Step section to extract and load data from Zoom to SQL Server let’s make sure you met the following requirements.

1. SSIS designer installed. Sometimes it is referred as BIDS or SSDT (download it from Microsoft site).
2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
3. ZappySys SSIS PowerPack installed. Click on the link to download a FREE trial.
4. You have the basic familiarity with REST API concepts and Zoom API.

Zoom API Authentication

Now let’s look at step by step how to read data from Zoom API or POST data to Zoom using SSIS.

Zoom Provides 3 ways to authenticate API calls. In this article we will cover only JWT and OAuth 2.0 connection approach.

1. Use JWT Token
2. Use OAuth 2.0 Token (3-legged OAuth)
3. Use API Key / Secret directly in API call (Mostly used by older API like this one)

Method-1 : Call Zoom API in SSIS using JWT Token (RFC 7519)

In this section we will learn how to use JWT Token method to call Zoom API (i.e. JWT as per RFC 7519 and here ).

JWT method is simple compared to OAuth 2.0 (next section) so many people prefer it compared to OAuth.

Now let’s look at how to use JWT token approach to authenticate your Zoom API calls.

Obtain Zoom API Key / Secret for JWT Auth

1. First Visit your Zoom Developer account here
2. Click on API Tab and copy your API Key and Secret (We will use it later)
   

   Obtain Zoom API Key and Secret (Used to generate JWT Token)

3. Whitelist the Redirect URL
   

   Whitelist the Redirect URL

Once we have API key / Secret we can move on to SSIS part.

Configure SSIS HTTP Connection with JWT

Now lets get started with SSIS part. We will first create HTTP Connection for Zoom API access using JWT method.

1. Open SSIS Package and drag ZS REST API Task from SSIS Control Flow Toolbox
2. Change URL Access Mode to Url From Connection
3. Enter URL as https://api.zoom.us/v2/users/me/settings
4. Select New ZS-HTTP connection type from the connection dropdown
5. Configure HTTP Connection as below. Select Credentials Type as JWT Token. Enter API Key and Secret.
   

   Configure HTTP Connection Manager (for API using JWT Token Authentication)

6. Once HTTP connection is configured you can click OK to save connection.
7. On the REST API Task click Test to confirm see its working.
   

   Calling Zoom REST API using SSIS REST API Task

Debug Web API call using Fiddler

To test how things looking behind the scene we strongly suggest to use tool like  fiddler. You can double click the URL entry (Right side) to see Request and Response Panels. Top panel is Request (URL, Headers, Body) and Bottom Panel is Response.

Debugging Web API call using Fiddler in SSIS

Loading data from REST API to SQL Server

Once you setup HTTP / OAUTH connection we can use JSON Source or XML Source to extract data from API and  load into SQL Server. If you are API is XML based API then use XML Source in below examples else use JSON Source. For example purpose we will load data into SQL Server but you can load into any Target (e.g. Flat file, Oracle, Excel) using Microsoft or ZappySys Destination connectors To load REST API data into SQL Server perform the following steps (Screenshots are generic and used for demonstration for purpose, your values can be different than below)

1. Go to SSIS Data Flow tab
2. Drag ZS JSON Source from the SSIS Toolbox (Use ZS XML Source if your API is XML based API)
3. Double click to edit Source
4. Set API URL you like to call, Check Use Credentials if you need to use HTTP or OAuth connection
5. Set Method / Body if its other than GET. For method other than GET make sure you select correct Content-Type from Dropdown (e.g. application/json). This indicate input data format in Body.
6. Set HTTP Headers if needed (in Most cases you won't need to set this)
7. Select Filter (many times this is not needed). If your API response has array node then select it here. For example if your response looks like this { rows: [ {...}, {...}, {...} ] } then filter would be $.rows[*]. Select Array node you like to extract in the Filter selection UI.
8. Configure Pagination if needed on the pagination tab. Check your API documentation see it supports pagination.
9. Click OK to Save component settings and generate desired metadata / columns for the output.
10. Now drag OLEDB destination from the SSIS toolbox and drop on the Data Flow surface (technically you can use any valid Destination but for demo we will use OLEDB Destination)
11. Connect Source to Destination Component
12. On OLEDB destination select / create new SQL Connection and then Click "New Table"
13. Click on Mappings tab to map columns by names and click OK to save
14. Execute the Package

Configure SSIS OLEDB Destination – Loading REST API Data into SQL Server Table

REST API to SQL Server Column Mappings for OLEDB Destination

Loading REST API to SQL Server in SSIS

POST / Write Data to API (Insert, Update or Delete)

There will be a time you like to automate POST actions (e.g. create new incident via API call). Check this article to learn more on how to POST , DELETE, PUT data using API calls

Common Errors

There are many error can occur during runtime. Here are most common errors you may face at runtime.

Truncation related error

The most common error you may face when you run a SSIS package is truncation error. During the design time only 300 rows are scanned from a source (a file or a REST API call response) to detect datatypes but at runtime it is likely you will retrieve far more records. So it is possible that you will get longer strings than initially expected. For detailed instructions how to fix common metadata related errors read an article "How to handle SSIS errors (truncation, metadata issues)".

Authentication related error

Another frequent error you may get is authentication error, which happens when you deploy/copy a package to another machine and run it there. Check Deployment to Production below to see why it happens and how to solve this problem.

Deployment to Production

In SSIS package sensitive data such as tokens and passwords are by default encrypted by SSIS with your Windows account which you use to create a package. So SSIS will fail to decrypt tokens/passwords, when you run it from another machine using another Windows account. To circumvent this when you are creating a SSIS package which uses authentication components (e.g. an OAuth Connection Manager or an HTTP Connection Manager with credentials, etc.), consider using parameters/variables to pass tokens/passwords. In this way you won’t face authentication related errors when package is deployed to a production server. Check our article on how to configure packages with sensitive data on your production or development server.

Reading Zoom API data using JSON Source (with Pagination)

Now let’s look at how to use ZappySys JSON Source to read API data and parse into rows/column

1. Drag and drop data flow task from SSIS Toolbox
2. Double click task to go to data flow designer
3. Drag SSIS ZS JSON Source from toolbox
4. Edit JSON Source like below
5. Enter URL like below (Use different URL if needed.)
   https://api.zoom.us/v2/users/me/meetings?type=upcoming&page_size=100&page_number=1
6. Select connection
7. Goto pagination tab and enter information as below screenshot.
8. Click on Select Filter
9. Click Preview to test

Read Zoom API data using JSON Source (Configure Pagination)

Method-2 : Call Zoom API in SSIS using OAuth 2.0

Now let’s look at OAuth 2.0 approach to authenticate with Zoom API.

Register OAuth App

Very first step to authenticate your Zoom API call using OAuth method is to register App. Go through steps  listed here to register your App. Below are high level steps.

1. Visit http://marketplace.zoom.us/
2. Click Build menu item at the top
3. Select App Type (i.e. Account Level or User Level) and Click Begin
4. Once App credentials displayed enter Redirect URL (use https://zappysys.com/oauth2 if you are not sure what to enter here). This URL never receives data from API but its only used to get authentication code during OAuth Process so don’t worry.
5. Click on Scopes tab and select one or more permissions you like to assign to your App.

Configure SSIS OAuth Connection

Now lets get started with SSIS part. We will first create OAuth Connection for Zoom API access.

1. Open SSIS Package and drag ZS REST API Task from SSIS Control Flow Toolbox
2. Change URL Access Mode to Url From Connection
3. Enter URL as https://api.zoom.us/v2/users/me/settings
4. Select New ZS-OAUTH connection type from the connection dropdown
5. Configure OAuth connection as below. Enter Token file path, enter Redirect URL, ClientID, Client Secret and then click Generate Token (You may have to login and then click Accept button to grant permission).
   

   Configure OAuth Connection for Zoom API (Enter Token File Path – This is needed to handle changing refresh token scenario)

   

   Configure OAuth Connection for Zoom API (Enter Redirect URL / Callback URL)

   

   Generate Token for Zoom API using OAuth 2.0

6. Once you get token is generated you can click OK to save connection.
7. On the REST API Task click Test to confirm its working.
   

   Calling Zoom REST API using SSIS REST API Task

Debug Web API call using Fiddler

To test how things looking behind the scene we strongly suggest to use tool like  fiddler. You can double click the URL entry (Right side) to see Request and Response Panels. Top panel is Request (URL, Headers, Body) and Bottom Panel is Response.

Debugging Web API call using Fiddler in SSIS

Loading data from REST API to SQL Server

Once you setup HTTP / OAUTH connection we can use JSON Source or XML Source to extract data from API and  load into SQL Server. If you are API is XML based API then use XML Source in below examples else use JSON Source. For example purpose we will load data into SQL Server but you can load into any Target (e.g. Flat file, Oracle, Excel) using Microsoft or ZappySys Destination connectors To load REST API data into SQL Server perform the following steps (Screenshots are generic and used for demonstration for purpose, your values can be different than below)

1. Go to SSIS Data Flow tab
2. Drag ZS JSON Source from the SSIS Toolbox (Use ZS XML Source if your API is XML based API)
3. Double click to edit Source
4. Set API URL you like to call, Check Use Credentials if you need to use HTTP or OAuth connection
5. Set Method / Body if its other than GET. For method other than GET make sure you select correct Content-Type from Dropdown (e.g. application/json). This indicate input data format in Body.
6. Set HTTP Headers if needed (in Most cases you won't need to set this)
7. Select Filter (many times this is not needed). If your API response has array node then select it here. For example if your response looks like this { rows: [ {...}, {...}, {...} ] } then filter would be $.rows[*]. Select Array node you like to extract in the Filter selection UI.
8. Configure Pagination if needed on the pagination tab. Check your API documentation see it supports pagination.
9. Click OK to Save component settings and generate desired metadata / columns for the output.
10. Now drag OLEDB destination from the SSIS toolbox and drop on the Data Flow surface (technically you can use any valid Destination but for demo we will use OLEDB Destination)
11. Connect Source to Destination Component
12. On OLEDB destination select / create new SQL Connection and then Click "New Table"
13. Click on Mappings tab to map columns by names and click OK to save
14. Execute the Package

Configure SSIS OLEDB Destination – Loading REST API Data into SQL Server Table

REST API to SQL Server Column Mappings for OLEDB Destination

Loading REST API to SQL Server in SSIS

POST / Write Data to API (Insert, Update or Delete)

There will be a time you like to automate POST actions (e.g. create new incident via API call). Check this article to learn more on how to POST , DELETE, PUT data using API calls

Common Errors

There are many error can occur during runtime. Here are most common errors you may face at runtime.

Truncation related error

The most common error you may face when you run a SSIS package is truncation error. During the design time only 300 rows are scanned from a source (a file or a REST API call response) to detect datatypes but at runtime it is likely you will retrieve far more records. So it is possible that you will get longer strings than initially expected. For detailed instructions how to fix common metadata related errors read an article "How to handle SSIS errors (truncation, metadata issues)".

Authentication related error

Another frequent error you may get is authentication error, which happens when you deploy/copy a package to another machine and run it there. Check Deployment to Production below to see why it happens and how to solve this problem.

Deployment to Production

In SSIS package sensitive data such as tokens and passwords are by default encrypted by SSIS with your Windows account which you use to create a package. So SSIS will fail to decrypt tokens/passwords, when you run it from another machine using another Windows account. To circumvent this when you are creating a SSIS package which uses authentication components (e.g. an OAuth Connection Manager or an HTTP Connection Manager with credentials, etc.), consider using parameters/variables to pass tokens/passwords. In this way you won’t face authentication related errors when package is deployed to a production server. Check our article on how to configure packages with sensitive data on your production or development server.

Conclusion

In this post you have seen how easy it is to call Zoom API in SSIS. Just register OAuth App and start calling your API. Download SSIS PowerPack and explore many other API integration uses.