In this tutorial, we will use the REST API Task to call some API. We will use the OAuth connection along the REST API Task to write / update data to Google Spreadsheet.

Prerequisites

Before we perform the steps listed in this article, you will need to make sure the following prerequisites are met:

1. SSIS designer installed. Sometimes it is referred to as BIDS or SSDT (download it from Microsoft site).
2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
3. Make sure ZappySys SSIS PowerPack is installed (download it).
4. Credentials of Google API.

Component Mention in this article

	JSON Generator Transform
	Web API Destination
	REST API Task

Create a Google API Project

The first step to access any Google API is to create an API Project in Google Console. If you don’t want to go through this then Skip this Step-1 and in the next section select the Default OAuth App option on OAth Connection Manager (This is the easiest option for now unless you want to use your own OAuth App).

Check this article for step-by-step instructions. When you follow these instructions make sure you enable Google Sheet API.

Create OAuth Connection Manager in SSIS

Once you create a Google API project and obtained a Client ID and Client Secret your next step is to create OAuth Connection Manager in SSIS. ZappySys OAuth connection manager comes with many predefined OAuth Providers (e.g. Facebook, Twitter, Google, etc) but you can also define custom OAuth settings for any OAuth enabled API.

To create SSIS OAuth 2.0 Connection for Google API perform the following steps.

1. Download and Install SSIS PowerPack.
2. Create a new SSIS Package.
3. Right-click in Connection Manager Area and Click “New Connection”.
4. When prompted select ZS-OAUTH connection type, On the OAuth Connection Manager Select Provider=Google.
5. Click here to create credentials. Then select the “Use Custom OAuth App” option and enter your ClientID, ClientSecret.
6. In the Scopes enter below scopes (or click Select Scopes). This will allow read/write access to Drive Files or Sheet (E.g. Reading file content / Export file to CSV). You can use the Scope browser to see many other available permissions. For more information click here.
   

   https://www.googleapis.com/auth/drive
   https://www.googleapis.com/auth/drive.file
   https://www.googleapis.com/auth/drive.readonly
   https://www.googleapis.com/auth/spreadsheets
   https://www.googleapis.com/auth/spreadsheets.readonly

7. Click on the Generate Token button.
   

   SSIS OAuth Connection Manager – Access Google Drive API using OAuth 2.0

8. You will see UI as below… Click Allow (You may have to scroll to see that button sometimes).
9. Click on Test Connection to see the connection is working and click on the OK button to save configure setting UI.

Find Google Sheet ID for API calls

Before we can write or update Sheet data in SSIS we need to know Google Driver File ID. We will use this ID in the next few sections.

The easiest way to find File ID is to look at the URL like below. If you need to know ID at runtime dynamically then refer to the last section.

Get Google Drive File ID for API call (Sheet ID)

Create Variables to make dynamic URL

1. Here, we are going to make a Dynamic URL value.
2. In Visual Studio Project Right Click on Design Panel, Select Variables.
   

   Create Variables

3. Let’s Create Variables with correct DataType and Value. In variable “varRange” set only the starting point(Sheet1!A2) of data if you don’t know the ending point and leave the first-row for giving header row later or click here.
   

Write SQL Server data to Google Spreadsheet

Now let’s look at a real-world example. How to write data from SQL Server table or other source and send to Google Sheet using Web API Destination.

1. Drag and Drop SSIS Data Flow Task from SSIS Toolbox.
   

   Dragging and dropping Data Flow Task into Control Flow

2. Double click on the Data Flow task to see the Data Flow designer surface.
3. Here, In Visual Studio, drag and drop the OLE DB Source, JSON Generator Transform and Web API Destination in the design surface and join the components with the blue arrow.
   

   Drag and Drop OLE DB Source, JSON Generator Transform, and Web API Destination

4. Here, you can use our Free ZS Dummy Data Source too. Click here for more information.
5. But, we are going through OLE DB Source so we need OLEDB Connection.

Create OLE DB Connection

1. Lets, create the OLE DB Connection.
2. Let’s, Right-click on Connection Managers Panel to Create OLEDB Connection, so you can use Source and Context Menu will appear, Select New OLEDB Connection from the Context Menu.
   

   Create OLE DB Connection

3. Now, click on New Button to create Connection.
   

   Create New Connection

4. Let’s Configure Connection Manager, just Follow steps one by one as we have created.
   

   Configure OLE DB Connection

5. Click on Test Connection to see correct configure it.
6. Click on the OK button to save connection configure setting UI.

Configure JSON Generator Transform

1. Double click on JSON Generator Transform to configure it.
2. Set Output Mode to Single Dataset Array. Right-click on Mappings, select Add Static Element then enter Name: “range” and Select variable we have created for value.
   

   Configure JSON Generator Transform – Add Static Value

3. Again, Right-click on Mappings, select Add Static Element then enter Name: “majorDimension” and Value: “ROWS“.
   

   Configure JSON Generator Transform – Add Static Value

4. Now, Right-click on Mappings, select Add Document Array, Check to Treat as 2D array option and enter Element Name as “values“.
   

   Configure JSON Generator Transform – Add Document Array

5. Then, right-click on Document Array, select Add elements > Select Add Multiple Columns and check on columns you like to add.
   

   Configure JSON Generator Transform – Add Element

6. Click on the OK button to save JSON Generator Transform configure setting UI.

Configure Web API Destination

1. Double click on Web API Destination to configure it.
2. Select the OAuth connection manager created in an earlier section (Assuming you had all scopes with write permission).
3. In the Setting tab,
   1. In URL, Generate Dynamic URL like below. Change Sheet ID, Range designator(including Sheet name) and API URL Parameters as per your need.
      

      https://sheets.googleapis.com/v4/spreadsheets/{{User::varGoogleSpreadsheetID}}/values/{{User::varRange}}:{{User::varSpreadsheetValues}}?valueInputOption={{User::varValueInputOption}}

      Example URL without variables (some part masked with xxxxxxxx):
      

      https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6xxxxxxxxCHrORJc/values/Sheet1!A2:B4:append?valueInputOption=USER_ENTERED

   2. Change Request Method to POST.
   3. Select JSON Generator variable into Body (Request Data) or you can select Input column for Body (JSON Generator Output).
   4. Set application/JSON to Body Content-type.
      

      Configure Web API Destination

4. Click on the OK button to save Web API Destination configure setting UI.
5. That’s all, You are ready to Run or Execute Task and verify data into Google spreadsheet. Now update header row values for click here.
   

   Run or Execute Package and verify data into Google Spreadsheet

Append data to Google SpreadSheet using REST API Task

Now let’s look at how to write to Google SpreadSheet. To write multiple rows in a single request. Also, check this link to learn more.

For more information, you can see the example here. For parameter, detail review this API documentation.

You can use SSIS REST API Task or Web API Destination to achieve the Spreadsheet write scenario. For a simple example, let’s use the REST API Task.

1. Drag and Drop ZS REST API Task on the control flow designer.
   

   Drag and Drop REST API Task

2. Double click on ZS REST API Task to configure it.
3. In the Request Settings tab,
   1. set Request URL Access Mode to Url from a connection. Select OAuth Connection from the dropdown (Same connection we created in the previous section).
   2. Enter the URL below. Change Sheet ID, Range designator(including Sheet name) and API URL Parameters as per your need.
      

      https://sheets.googleapis.com/v4/spreadsheets/{{User::varGoogleSpreadsheetID}}/values/{{User::varRange}}:{{User::varSpreadsheetValues}}?valueInputOption={{User::varValueInputOption}}

      Example URL without variables (some part masked with xxxxxxxx):
      

      https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6xxxxxxxxCHrORJc/values/Sheet1!A1:append?valueInputOption=USER_ENTERED

   3. Change Request Method to POST.
   4. Enter Request Body as below
      
      Note: For example purpose, we included header line in data cells but in the real-world header may already exist in the sheet.

      {
        "range": "{{User::varRange}}",
        "majorDimension": "ROWS",
        "values": [
          ["Item", "Cost", "Stocked", "Ship Date"],
          ["Wheel", "$20.50", "4", "3/1/2016"],
          ["Door", "$15", "2", "3/15/2016"],
          ["Engine", "$100", "1", "3/20/2016"],
          ["Totals", "=SUM(B2:B4)", "=SUM(C2:C4)", "=MAX(D2:D4)"]
        ]
      }

   5. Change Request Content type to application/JSON.
      

      Write data to Google SpreadSheet using REST API Task (OAuth 2.0 Connection)

4. Click on the OK button to save REST API Task configure setting UI.
5. That’s all, You are ready to Run or Execute Task and verify data into Google spreadsheet.
   

   Execute REST API Task and verify Google Spreadsheet data

Update Google Sheet Cells

Now let’s look at the below example of updating multiple cells in google Spreadsheet. For more information, you can see the example here. For parameter, detail review this API documentation.

You can use SSIS REST API Task or Web API Destination to achieve the Spreadsheet Update scenario. For a simple example, let’s use the REST API Task.

Update Google Sheet header cells manually

1. Drag and drop ZS REST API Tasks on the control flow designer.
2. Double click on the First REST API Task to configure it.
3. In the Request Settings tab,
   1. Enter the URL below. Change Range designator(including Sheet name: Sheet1!A1) and API URL Parameters as per your need.
      

      https://sheets.googleapis.com/v4/spreadsheets/{{User::varGoogleSpreadsheetID}}/values/{{User::varRange}}?valueInputOption={{User::varValueInputOption}}

      Example URL without variables (some part masked with xxxxxxxx):
      

      https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6xxxxxxxxCHrORJc/values/Sheet1!A1?valueInputOption=USER_ENTERED

   2. Change Request Method to PUT.
   3. Enter Request Body as below
      

      {
        "range": "{{User::varRange}}",
        "majorDimension": "ROWS",
        "values": [
          ["CustomerID", "CustomerName"]
        ]
      }

4. Click on the OK button to save REST API Task configure setting UI.
5. That’s all, You are ready to Run or Execute Task and verify data into Google spreadsheet.
   

Single Range (Set Values for multiple cells)

1. Follow the previous section(write data to Google Spreadsheet) to update sheet data.
2. In the Request Settings tab,
   1. Enter the URL below. Change Sheet ID, Range designator(including Sheet name) and API URL Parameters as per your need.
      

      https://sheets.googleapis.com/v4/spreadsheets/{{User::varGoogleSpreadsheetID}}/values/{{User::varRange}}?valueInputOption={{User::varValueInputOption}}

      Example URL without variables (some part masked with xxxxxxxx):
      

      https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6xxxxxxxxCHrORJc/values/Sheet1!A2:B4?valueInputOption=USER_ENTERED

   2. Change Request Method to PUT.
   3. Enter Request Body as below
      

      {
        "range": "{{User::varRange}}",
        "majorDimension": "ROWS",
        "values": [
          ["Laptop", "$2000.50"],
          ["Mobile", "$1150"]
        ]
      }

3. That’s all, click on the Test Request/Response button and verify data into Google spreadsheet.
   

   Calling Google Sheet API to Update Multiple Cell Values

Clear data to Google SpreadSheet using REST API Task

Clear a sheet of all values while preserving formats

Lets clear a sheet of all values by REST API Task, Before processed we need sheet Id. So let’s get the SheetId by API Call. For more information about click here.

1. Drag and Drop two ZS REST API Tasks on the control flow designer and rename(Get Sheet ID and Clear Google Sheet of all values) it.
2. Double click on the First REST API(Get Sheet ID) Task to configure it.
3. In the Request Settings tab,
   1. Enter the URL below. Change Sheet ID and API URL Parameters as per your need.
      

      https://sheets.googleapis.com/v4/spreadsheets/{{User::varGoogleSpreadsheetID}}

      Example URL without variables (some part masked with xxxxxxxx):
      

      https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6xxxxxxxxCHrORJc

   2. Change Request Method to GET.
      REST API Task – Get Google Sheet ID
4. Now go to Response Settings Tab, Select Response content Type Json and Write Response Content Filter Expression. Check on Save Response Content. Select Save Mode to Variable and Create a New variable or select it.
   Example of Response Content Filter Expression for the first Sheet : 
   

   $.sheets.[0].properties.sheetId

   

   REST API Task – Save Google Sheet ID into Variable

5. Now, click on the Test Request/Response to check configures are ok.
6. Click on the OK button to save REST API Task configure setting UI.
7. Double click on the second REST API(Clear Google Sheet of all values) Task to configure it.
8. In the Request Settings tab,
   1. Enter the URL below. Change Sheet ID and API URL Parameters as per your need.
      

      https://sheets.googleapis.com/v4/spreadsheets/{{User::varGoogleSpreadsheetID}}:batchUpdate

      Example URL without variables (some part masked with xxxxxxxx):
      

      https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6xxxxxxxxCHrORJc:batchUpdate

   2. Change Request Method to POST. Enter Request Body as below and change a variable parameter as per your need.
      

      {
        "requests": [
          {
            "updateCells": {
              "range": {
                "sheetId": {{User::varGoogleSheetID}}
              },
              "fields": "userEnteredValue"
            }
          }
        ]
      }

   3. Change Request Content type to application/JSON.
      
      

      REST API Task Configure – Clear sheet of all values

9. Click on the OK button to save REST API Task configure setting UI.
10. That’s all, You are ready to Run or Execute Task and verify data into Google spreadsheet.

Clears range cell values from a spreadsheet

1. Double click on REST API Task to configure it. Then, follow the way just like the previous section configures REST API Task.
2. Here you need to set the value of Variable “varSpreadsheetValues” to “clear” or you can change it directly in URL append to clear and value of variable “varRange” (Change cell range and sheet name) as per your need. For more information about it click here.
   1. Enter the URL below. Change Sheet ID, Range designator(including Sheet name) and API URL Parameters as per your need.
      

      https://sheets.googleapis.com/v4/spreadsheets/{{User::varGoogleSpreadsheetID}}/values/{{User::varRange}}:{{User::varSpreadsheetValues}}?valueInputOption={{User::varValueInputOption}}

      Example URL without variables (some part masked with xxxxxxxx):
      

      https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6xxxxxxxxCHrORJc/values/Sheet1!A2:B4:clear?valueInputOption=USER_ENTERED

3. Change Request Method to POST.
4. Enter Request Body as below
   

   {}

5. Change Request Content type to application/JSON.
6. That’s all, click on the Test Request/Response button and verify data into Google spreadsheet.

Conclusion

Google Drive API and Google Sheets API provide a great way to automate file-related functionality. However, to call Google API  you have to use the SDK / coding approach (e.g. C#, Java, Python, Ruby). Luckily ZappySys SSIS PowerPack provides a great way to integrate any Google API call via simple drag and drop approach without coding. Try out yourself see how long it takes to call virtually any REST API.

References

Finally, you can use the following links for more information:

• JSON Generator Transform
• Web API Destination
• REST API Task
• Help File:
  • JSON Generator Transform
  • Web API Destination
  • REST API Task

The post How to write/update Google Sheet in SSIS appeared first on ZappySys Blog.

In our previous article we saw how to call REST API or SOAP Web Service in SSIS. In this article we will only focus how to perform HTTP POST in SSIS using RESTful approach.

HTTP Protocol supports many methods also referred as verb (e.g. GET, PUT, POST, HEAD, DELETE). Based on this verb, target server (API server) can perform desired action. For example GET is usually used for read only data fetch (similar to select query in SQL). POST is usually used to create new record or update existing data.

For any HTTP operation you can submit parameters/data in 3 ways.

via URL query string
via HTTP Headers
via Body (Only valid for operations such as POST, PUT)

HTTP POST in SSIS

Traditionally whenever SSIS developers wanted to perform HTTP POST operation in SSIS they had to write C# Code using Script Task. This is no more case after ZappySys developed easy to use drag and drop REST API Tasks and Connectors. As of now there 4 main connectors/tasks which supports HTTP POST

1. SSIS Web API Destination
2. SSIS REST API Task
3. SSIS JSON Source
4. SSIS XML Source

SSIS PowerPack offers following two methods to POST data from your database to target API URL (e.g. SQL Server to API POST in JSON format).

Method-1 : HTTP POST using Web API destination (In Data Flow)

This method allows you to read data from any source and generate POST data and then submit to API endpoint using SSIS Web API Destination. Using this destination you can configure URL and API Body (e.g. JSON/XML document you want to submit) for each request. Check following screenshot to get idea how API POST works

Step-By-Step : Video Tutorial

You can also extract http response (i.e. JSON or XML) from Web API Destination and parse it into Rows and Columns using JSON Parser Transform like below. If you want to handle errors gracefully without stopping the execution then use error output (Drag red arrow from Web API destination and attach to downstream (Make sure you set Redirect for Error and Truncate both when prompted on attach)

Method-2 : HTTP POST using SSIS REST API Task (In Control Flow)

REST API Task support many advanced options too incase you want to tweak certain aspects of your POST request.

Here is the screenshot of HTTP POST in JSON format using ZappySys REST API Task

Perform HTTP POST operation using SSIS REST API Task – Send data in JSON content type

Parse response into rows/columns using SSIS JSON Source

If you wish to POST data and parse JSON response into rows/columns then you can use SSIS JSON Source for this purpose. JSON Source has great JSON Parser which supports parsing very large JSON  (stored in File or API URL or Direct string) into Rows and Columns. This is useful if you wish to save response into target such as SQL Server or CSV file.

Post JSON data to HTTP URL and parse response into columns and rows

Parse response into rows/columns using SSIS XML Source

If you wish to POST data and parse XML response into rows/columns then you can use SSIS XML Source for this purpose. XML Source supports parsing XML data (stored in File or API URL or Direct string) into Rows and Columns. This is useful if you wish to save response into target such as SQL Server or CSV file.

Passing credentials for Web Request for HTTP POST

Unfortunately there is no one standard for passing credentials to server but good thing is ZappySys REST API Task comes with many options to secure your request and pass credentials (Userid/Password or API Key or OAuth Token).

Basic Authentication

Most common approach of passing credentials is userid/password. Check this article how to use Url from connection if you wish to pass credentials using Basic authentication (i.e. Userid/password).

OAuth Authentication (1.0 or 2.0)

If your API support OAuth protocol then check these articles how to use OAuth with ZappySys REST connectors

Passing Parameters in POST Body (Make it dynamic)

SSIS PowerPack support variable placeholders for many editable text fields (usually indicated by variable icon or pencil icon).

If you wish to pass parameters in your data used in Body then use variable placeholder (e.g. {{User::varMyDate}}. When you edit POST Body you will see <<Insert Variable>> option.

Here is the complete list of available placeholder functions

Upload file using Multi-Part POST

If you have need to submit data to server from file then check this link (its shows few ways to upload file using HTTP POST)

Most common Content-Type for POST operation

There are many data formats which you can submit to server using HTTP POST but below is the most common data formats you will ever use for most of scenarios.

Data format is usually determined by server based on Content-Type header you include along with your request.

HTTP POST Form data – submit key/value pairs (application/x-www-form-urlencoded)

This is the most common data format for HTTP POST. Its also famous because any type you submit form on any website, most likely it will use below method for submitting your data to server. As you see its key/value pairs separated by ampersand [&]. Also if some special character found then it needs to be URL encoded (e.g. @ is replaced by %40)

POST https://myserver.com/submit
Content-Type: application/x-www-form-urlencoded

firstname=bob&lastname=smith&email=bob%40mycompany.com

HTTP POST JSON data (application/json content-type)

JSON is becoming very popular data format so if you are dealing with newer API then most likely you will have to submit your POST request using this content type like below. Assume that you creating new customer record then your request may look like below.

POST https://myserver.com/api/customer
Content-Type: application/json

{
 "firstname" : "bob",
 "lastname" : "smith",
 "email" : "bob@mycompany.com"
}

HTTP POST CSV data (text/plain content-type)

When API POST required data in CSV format then you have to specify this content type. Below is the example of HTTP POST with CSV data.

POST https://myserver.com/api/customer
Content-Type: text/plain

firstname,lastname,email
bob,smith,bob@mycompany.com

HTTP POST XML data (application/xml content-type)

When API POST required data in XML format then you have to specify (application/xml) content type. Below is the example of HTTP POST with XML data.

POST https://myserver.com/api/customer
Content-Type: application/xml

<?xml version="1.0"?>
<data>
 <firstname>bob</firstname>
 <lastname>smith</lastname>
 <email>bob@mycompany.com</email>
</data>

HTTP POST Binary data (Upload file)

If you are POSTing large amount of data from file (i.e. uploading file) to your API endpoint then check this article about How to Upload file using Multi-Part Upload

HTTP POST Raw data (no content type)

Do not pass any content-type header to send raw request

Advanced Read / Write Scenarios / API Error Handling

Now let’s look at some advanced scenario where ýou first have to call one API (GET or POST) using JSON Source or XML Source and then POST data to some other URL using Web API Destination.

Once you get response after POST/GET parse those records for further processing along with error handling. Such API calls can be achieved using following pattern.

Using Template Transform to build JSON or XML for POST Body

Check this article to learn more about some techniques to build JSON or XML for API POST Call (e.g. SOAP )

Encoding for POST Body

When using application/x-www-form-urlencoded content type (POST Form data) your server may require you to encode certain characters in property name or value using URL Encoding technique. As you see below key/value pairs separated by ampersand [&]. Also if some special character found then it needs to be URL encoded (e.g. @ is replaced by %40). There are many online tools found to encode or decode data using URL encoding technique.

firstname=bob&lastname=smith&email=bob%40mycompany.com

Troubleshooting / Debugging HTTP POST issues using Fiddler

If you wish to see what’s being passed to server via any client tool or when you perform POST request ? Use tool called fiddler. Check this article to learn how to use Fiddler for debugging purpose.

Conclusion

As we enter into cloud first world you will face more and more need to integrate various RESTful API into your ETL workflows. SSIS PowerPack comes with complete toolset to automate any API calls (Read, Write or Parse). Try it for free here to discover many hidden features not discussed in this article.

The post HTTP POST in SSIS – Send data to Web API url appeared first on ZappySys Blog.