In today’s data-driven world, the ability to seamlessly integrate and transfer files between different platforms is crucial for efficient business operations. If you’re a user of SQL Server Integration Services (SSIS) and looking to integrate Google Drive into your data workflow, you’re in luck. ZappySys SSIS PowerPack offers a powerful Google Drive Connector that makes downloading files from Google Drive a breeze. In this guide, we’ll walk you through the process of downloading files from Google Drive using SSIS and the ZappySys SSIS PowerPack Google Drive Connector.

Download files from Google Drive with SSIS:

In this section, we will learn how to configure and use Google Drive Connector in the API Source & API Destination to download files from Google Drive.

1. Begin with opening Visual Studio and Creating a New Project.
2. Select Integration Service Project and in the new project window set the appropriate name and location for the project. Click OK.
3. In the new SSIS project screen you will find the following:
   1. SSIS ToolBox on the left sidebar
   2. Solution Explorer and Property Window on the right bar
   3. Control flow, data flow, event Handlers, Package Explorer in tab windows
   4. Connection Manager Window at the bottom

   -`
   

   NOTE: If you don’t see ZappySys SSIS PowerPack Task or Components in the SSIS Toolbox, please refer to this help link.
4. Now, Drag and Drop the SSIS Data Flow Task from the SSIS Toolbox. Double-click on the Data Flow Task to see the Data Flow Designer.
   
5. From the SSIS toolbox drag and drop API Source (Predefined Templates) on the data flow designer surface, and double-click on it to edit it:
   
6. Select New Connection to create a new connection:
   

Configuring the Google Drive Connector

1. Use a preinstalled Google Drive Connector from the Popular Connector List or press the Search Online radio button to download Google Drive Connector. Once downloaded simply use it in the configuration:
   
2. Proceed with selecting the desired Authentication Type. Then select API Base URL (in most cases default one is the right one). Finally, fill in all the required parameters and set optional parameters if needed.
3. After configuring all the required properties in the API Connection Manager, please click on the “Test Connection” button to confirm that the connection is established successfully.
   

   Method1: Connect to Google Drive using User Account [OAuth]

   You may press a link Steps to Configure which will help set certain parameters.

   

   SSIS API Source – Generate Token – Google Drive with User Account

   Method2: Connect to Google Drive using Service Account [OAuth]

   You may press a link Steps to Configure which will help set certain parameters.

   

   SSIS API Source – Generate Token – Google Drive with Service Account

How to download a single file from Google Drive

In the API Source Component, we’ve chosen the API Connection Manager. Next, we need to configure the remaining properties of the component to download the single file from Google Drive.

1. Choose the Download a file endpoint from the “Select Table/Endpoint” dropdown menu.
2. To access detailed API documentation for downloading files, you can click on the following link: Help for: Download a File.
3. Please choose a File ID from the drop-down menu to indicate which file you wish to download. This is an essential parameter for the download process.
4. Please indicate the desired filename for the file being downloaded by setting the FileName parameter.
5. If you intend to replace the local file with the one being downloaded, please select the ‘AlwaysOverwrite‘ option from the dropdown menu labeled ‘FileOverwriteMode‘. Alternatively, you can choose ‘FileIfExists’ or ‘SkipIfExists’ based on your specific needs.

Please refer to the sample screenshot below.

SSIS API Source – Google Drive – Download Single File

Once you have configured all the mentioned properties above, please click on the “Preview Data” button to initiate the API call. With just a few clicks, we have successfully configured the call to Google Drive using the ZappySys Google Drive Connector for downloading a Single file from Google Drive.

How to download multiple files from Google Drive

In the API Destination Component, we have chosen the API Connection Manager. Now, we must proceed to configure the remaining properties of the component to facilitate the download of multiple files from Google Drive. Additionally, we should make use of an OLE DB Source and Derived Column component to map the files for the download process.

1. Drag and drop the OLE DB Source Component from the SSIS Toolbox. Double-click on the OLE DB Source Component to initiate its configuration.
   

   OLE DB Source – Drag and Drop

2. Create a new OLE DB Connection Manager and configure your SQL Server settings for the connection. Next, select the Data access mode, specify the name of the table or view, or write an SQL Command. Afterward, click on the ‘Preview’ button to view a preview and check the ‘Columns’ tab for the source columns.
   

   SSIS OLE DB Source – Configure SQL Connection and fetch the table data

3. Next, drag and drop the Derived Column Component from the SSIS Toolbox. Double-click on the Derived Column Component to begin its configuration.
   

   Drag and drop the Derived Column

4. Now, add a new Derived Column by setting the expression to concatenate FilePath and FileName. Click ‘OK’ to save the configuration.
   

   SSIS Derived Column – Add New Column & Set Expression

5. Now, drag and drop the API Destination Component from the SSIS Toolbox. Double-click on the API Destination Component to begin its configuration.
   

   Drag and Drop the SSIS API Destination Component

6. Double-click on the API Destination component to configure it for uploading multiple files to Google Drive.
   1. Choose the Download a file endpoint from the “Select Table/Endpoint” dropdown menu.
   2. To access detailed API documentation for downloading files, you can click on the following link: Help for: Download File.
   3. Please specify the ‘FileOverwriteMode’ option as ‘AlwaysOverwrite,’ or you can choose ‘FailIfExists’ or ‘SkipIfExists’ options based on your requirements.
      Please refer to the sample screenshot below.
      

      SSIS API Destination Component – Configure Settings Tab

   4. Now, go to the Mappings tab and specify the source columns with the corresponding target properties.
      Note: Here We’re mapping FileID with $$Id and FileNameWithPath with $$ResponseDataFile. You can create mappings based on your Source columns
      Please refer to the sample screenshot below
      

      SSIS API Destination Component – Configure Mappings Tab

7. Once you have configured all the mentioned properties above, please click on the “Preview Data” button from the Settings tab to initiate the API call. With just a few clicks, we have successfully configured the call to Google Drive using the ZappySys Google Drive Connector for downloading multiple files from Google Drive.
   Please refer to the sample screenshot below.
   

   Download Multiple files from Google Drive

Conclusion

In this blog, we learned how to connect with Google Drive using SSIS API Connection Manager and download single or multiple files from Google Drive using a combination of SSIS components, including the SSIS API Source component, SSIS API Destination component, OLE DB Source component, and Derived Column component.

The post How to download files from Google Drive using SSIS appeared first on ZappySys Blog.

In today’s data-driven world, the ability to seamlessly integrate and transfer files between different platforms is crucial for efficient business operations. If you’re a user of SQL Server Integration Services (SSIS) and looking to integrate Google Drive into your data workflow, you’re in luck. ZappySys SSIS PowerPack offers a powerful Google Drive Connector that makes uploading files to Google Drive a breeze. In this guide, we’ll walk you through the process of uploading files to Google Drive using SSIS and the ZappySys SSIS PowerPack Google Drive Connector.

Upload files to Google Drive with SSIS:

In this section, we will learn how to configure and use Google Drive Connector in the API Source to upload files on Google Drive.

1. Begin with opening Visual Studio and Creating a New Project.
2. Select Integration Service Project and in the new project window set the appropriate name and location for the project. Click OK.
3. In the new SSIS project screen you will find the following:
   1. SSIS ToolBox on the left sidebar
   2. Solution Explorer and Property Window on the right bar
   3. Control flow, data flow, event Handlers, Package Explorer in tab windows
   4. Connection Manager Window at the bottom
   

   SSIS Toolbox – ZappySys Tasks

   NOTE: If you don’t see ZappySys SSIS PowerPack Task or Components in the SSIS Toolbox, please refer to this help link.
4. Now, Drag and Drop the SSIS Data Flow Task from the SSIS Toolbox. Double-click on the Data Flow Task to see the Data Flow Designer.
   
5. From the SSIS toolbox drag and drop API Source (Predefined Templates) on the data flow designer surface, and double-click on it to edit it:
   
6. Select New Connection to create a new connection:
   

Configuring the Google Drive Connector

1. Use a preinstalled Google Drive Connector from the Popular Connector List or press the Search Online radio button to download Google Drive Connector. Once downloaded simply use it in the configuration:
   
2. Proceed with selecting the desired Authentication Type. Then select API Base URL (in most cases default one is the right one). Finally, fill in all the required parameters and set optional parameters if needed.
3. After configuring all the required properties in the API Connection Manager, please click on the “Test Connection” button to confirm that the connection is established successfully.
   

   Method1: Connect to Google Drive using User Account [OAuth]

   You may press a link Steps to Configure which will help set certain parameters.

   

   SSIS API Source – Generate Token – Google Drive with User Account

   Method2: Connect to Google Drive using Service Account [OAuth]

   You may press a link Steps to Configure which will help set certain parameters.

   

   SSIS API Source – Generate Token – Google Drive with Service Account

How to upload a local file to Google Drive

In the API Source Component, we’ve chosen the API Connection Manager. Next, we need to configure the remaining properties of the component to upload a local file to Google Drive.

1. Choose the Upload a file endpoint from the “Select Table/Endpoint” dropdown menu.
2. To access detailed API documentation for uploading files, you can click on the following link: Help for: Upload File.
3. Please specify the desired filename for the uploaded file by setting the FileName parameter.
4. Specify the desired local file path by setting the DiskFilePath parameter.
5. If you wish to upload the local file to a specific folder in Google Drive, please choose the folder ID from the dropdown menu labeled ParentFolderId. Otherwise, you can skip this parameter as it’s optional.

Please refer to the sample screenshot below.

SSIS API Source – Google Drive – Upload Single File

Once you have configured all the mentioned properties above, please click on the “Preview Data” button to initiate the API call. With just a few clicks, we have successfully configured the call to Google Drive using the ZappySys Google Drive Connector for uploading a local file to Google Drive.

How to upload multiple files to Google Drive

In the API Source Component, we’ve selected the API Connection Manager. Next, we need to configure the remaining properties of the component to upload multiple files to Google Drive. Additionally, we should utilize a Foreach Loop Container to iterate through each file one by one in a loop.

1. Create 3 SSIS variables as described below.
   

   Variable Name	Datatype	Value
   varFileLists	Object	System.Object
   varFileName	String	fake.extension
   varFullFilePath	String	C:/fake/fake.extension

2. Go to Control Flow, From the SSIS toolbox drag and drop ZS Advanced File System Task on the control flow designer surface, and double click on it to edit it:
   
3. Select the “Get file list as ADO.NET Data Table” action from the Action drop-down menu. Set the file path in the Path textbox. Choose the variable (Object type) where you want to store the file list, and then click the OK button to save the settings.
   

   SSIS Advanced File System Task – Get file list as ADO.net Data Table

4. Now, drag and drop the “Foreach Loop Container” from the SSIS Toolbox and connect it with the “ZS Advanced File System Task.” Double-click on the “Foreach Loop Container” to edit its properties.
   

   SSIS Foreach Loop Container – Drag & Drop

5. In the Foreach Loop Editor’s “Collection” tab, choose the Enumerator as “Foreach ADO Enumerator.” In the “ADO object source variable” field, select your SSIS object-typed variable. Select the “Rows in the first table” radio option.
   

   SSIS Foreach Loop Editor – Collection – Select Variable & Enumeration mode

6. In the Foreach Loop Editor’s “Variable Mappings” tab, select the variables that you want to map to the values from the collection, and then click the “OK” button to save your selections.
   

   SSIS Foreach Loop Editor – Variable Mappings

7. Inside the Foreach Loop Container, drag and drop the Data Flow Task, and then rename it according to your preference.
   

   SSIS Data Flow Task – Drag & Drop inside Foreach Loop Container

8. Double-click on the Data Flow Task to see the Data Flow Designer, From the SSIS toolbox drag and drop API Source (Predefined Templates) on the data flow designer surface, and double click on it to edit it:
   
9. Double-click on the API Source component to configure it for uploading multiple files to Google Drive.
   1. Choose the Upload a file endpoint from the “Select Table/Endpoint” dropdown menu.
   2. To access detailed API documentation for uploading files, you can click on the following link: Help for: Upload File.
   3. Please specify the desired SSIS Variable [varFileName] which contains file name for the uploaded file by setting the FileName parameter.
   4. To specify the desired local SSIS variable [varFullFilePath], which holds the file path, set the DiskFilePath parameter.
      When configuring the SSIS variable directly in the Raw Edit, JSON may become invalid due to the requirement of slashes within the local file path. To address this, we use a JSONENC Placeholder Function for proper escaping.
      

      <<{{User::varUserFullFilePath}},FUN_JSONENC>>

   5. If you wish to upload the local file to a specific folder in Google Drive, please choose the folder ID from the dropdown menu labeled ParentFolderId. Otherwise, you can skip this parameter as it’s optional.

   Please refer to the sample screenshot below.

   

   SSIS API Source – Google Drive – Upload Multiple files configuration

   To configure dynamic filenames and file paths, you can set variables. Click on “Raw Edit,” and in the “Edit Parameters” popup, you can use SSIS variables, as shown in the image below.

   Once you have configured all the mentioned properties above, please click on the “Preview Data” button to initiate the API call. With just a few clicks, we have successfully configured the call to Google Drive using the ZappySys Google Drive Connector for uploading multiple files to Google Drive.

Conclusion

In this blog, we learned how to connect with Google Drive using SSIS API Connection Manager and upload single or multiple files to Google Drive using a combination of SSIS components, including the SSIS API Source component, SSIS ZS Advanced File System Task, Foreach Loop Container, and SSIS variables.

The post How to upload files on Google Drive using SSIS appeared first on ZappySys Blog.

In this tutorial we will use REST API Task to call some ad-hoc API (e.g. get File List from Google Drive) and save output into Variable or File. We will use OAuth connection along with ZappySys JSON Source connector to read data from Google SpreadSheet (Use Drive API) and load into SQL Server (Export Google SpreadSheet to CSV).

Create Google API Project

First step to access any Google API is create an API Project in Google Console. If you don’t want to go through this then Skip this Step-1 and in the next section select Default OAuth App option on OAth Connection Manager (This is the most easiest option for now unless you want to use your own OAuth App).

Check this article for step-by-step instructions. When you follow these instructions make sure you enable Google Drive API (In the article it shows how to enable YouTube API as an example but you will need to enable Drive API for this article).

Create OAuth Connection Manager in SSIS

Once you create Google API project and obtained Client ID and Client Secret your next step is to create OAuth Connection Manager in SSIS. ZappySys OAuth connection manager comes with many predefined OAuth Providers (e.g. Facebook, Twitter, Google etc) but you can also define custom OAuth settings for any OAuth enabled API.

To create SSIS OAuth 2.0 Connection for Google API perform following steps.

1. Download and Install SSIS PowerPack
2. Create new SSIS Package
3. Right click in Connection Manager Area and Click “New Connection”
4. When prompted select ZS-OAUTH connection type
5. On the OAuth Connection Manager Select Provider=Google Sheets + Drive.
6. You can keep “Use Default App” selected or choose Custom App. Click here to learn – how to register Custom Google App
7. If you choose the Custom App option then enter the below scopes (or click Select Scopes). If you choose the default app then it’s not required.

   This will allow read / write access to Drive Files or Sheet (E.g. Reading file content / Export file to CSV). You can use Scope browser to see many other available permissions.

   https://www.googleapis.com/auth/drive
   https://www.googleapis.com/auth/drive.file
   https://www.googleapis.com/auth/drive.readonly
   https://www.googleapis.com/auth/spreadsheets
   https://www.googleapis.com/auth/spreadsheets.readonly

   Click Generate Token button.

   

   SSIS OAuth Connection Manager – Access Google Drive API using OAuth 2.0

8. You will see UI as below… Click Accept (You may have to scroll to see that button sometimes)
9. Click Test to see connection is working.

NOTE: If you don’t want to use Default OAuth App provided by ZappySys then select “Use Custom OAuth App” option (App created in previous section) specify your ClientID, ClientSecret.

Find Google Sheet ID for API call

Before we can read or write Sheet data in SSIS we need to know Google Driver File ID. We will use this ID in next few sections.

Easiest way to find File ID is look at the URL like below. If you need to know ID at runtime dynamically then refer last section.

Get Google Drive File ID for API call (Sheet ID)

Reading Google SpreadSheet Data in SSIS

Basically there are two ways you can export / read Google SpreadSheet data.

• Method-1 : Read Google SpreadSheet using SSIS JSON / REST API Source
  • In this approach we directly pull data from SpreadSheet using Core Google SpreadSheet.Values API to read particular Tab from Sheet and you can also specify Range. This is most effective way to read. Make sure you have v2.5.0.10807 or higher version (released after Aug 03 2017). New version added 2D array transformation options to parse multi-dimensional arrays found in SpreadSheet API JSON response.
• Method-2 : Read Google SpreadSheet using Flat File Source (as CSV)
  • Another approach is export SpreadSheet as CSV format. Then use Flat File Source to read that data and load into Target (E.g. SQL Server)

Method-1 : Read Google SpreadSheet using SSIS JSON / REST API Source

Now lets look at real steps to read Google SpreadSheet. To get values from specific tab and specific cell-row range you have to use Google SpreadSheet.Values API

In this example we will load below Google Spreadsheet into SQL Server

Step-By-Step : Loading Google SpreadSheet into SQL Server using SSIS

1. Drag and drop data flow on the control flow designer surface
2. Double click data flow and you will see SSIS Toolbox refreshed with new components
3. Drag ZS JSON Source from SSIS Toolbox onto Data flow designer surface
4. Double click JSON SOurce and configure below setting
   1. Enter your Spreadsheet URL in the following format
      

      https://sheets.googleapis.com/v4/spreadsheets/{your-file-id}/values/{cell-range}

      For example purpose we will use this public SpreadSheet provided by Google. If spreadsheet fileid is 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms (see previous section how to get fileid) and your tab name is Class Data and you like to extract data from A to F columns starting from 2nd row then your Cell range in the URL can be ‘Class Data’!A2:F
      Here is the actual URL we will use for demo
      

      https://sheets.googleapis.com/v4/spreadsheets/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/values/'Class Data'!A2:F

      — OR —
      You can also enter like below (If you omit Tab name then first table is used)
      

      https://sheets.googleapis.com/v4/spreadsheets/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/values/A1:F
      
      --OR-- Use below URL for to supply Column names manually (Start from A2)
      
      https://sheets.googleapis.com/v4/spreadsheets/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/values/A2:F

      You can also enter like below, specify first row and last row
      

      https://sheets.googleapis.com/v4/spreadsheets/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/values/A1:F101

   2. Check Use credentials and select OAuth connection we created in previous section
   3. Click on Select Filter and select values node and click OK  or you can directly enter following filter.
      

      $.values[*]

      

      Configure SSIS JSON/REST API Source – Read data from Google SpreadSheet API

   4. Now click on array Transformation tab. Now we will show you two different ways to configure column names.
   5. Method-1: Auto Detect Column Names (added in version 3.1.2)
      In recent version we added column less array transformation where we have option to detect first record as column name.
      1. Select Transform Type as Column less array
      2. Check option First line has column names like below example (Ignore direct Sample JSON – We provided for demo only. In your case it will be URL)
         

         JSON With 2D Array Pattern – Column names in First Row (Google Sheets API Pattern)

   6. Method-2: Manually Enter Column Names (For old version)
      1. Select Transform simple 2D array option from dropdown
      2. Check Specify columns list manually
      3. Enter column names (Must match the order and count based on range you selected)
         

         Student Name,Gender,Class Level,Home State,Major,Extracurricular Activity

         

         Configure SSIS JSON/REST API Source 2D Array Transformation for Google SpreadSheet API

   7. Now click Preview to see the data
      

      Configure SSIS JSON/REST API Source –
      Preview Google SpreadSheet Data

   8. Click OK to save UI
   9. Attach your JSON Source to target (e.g. OLEDB Destination for SQL Server Table).
   10. Configure Destination
       

       Configure SQL Server destination for Google SpreadSheet to SQL Data load

   11. Execute SSIS Package to load Google SpreadSheet data int SQL Server
       

       SSIS Example -Loading data from Google SpreadSheet into SQL Server Table

Method-2 : Read Google SpreadSheet using Flat File Source (as CSV)

Once we have Google Drive fileId we can perform following actions to export data into CSV file format. After that you can easily read CSV using SSIS FlatFile Source and load into SQL Server or other target.

Export Google SpreadSheet as CSV file using SSIS REST API Task

Here are the steps to export Google Drive SpreadSheet to CSV file

1. Just like previous section configure REST API Task
2. Enter following URL (See we used FileId from SSIS variable. You can hardcode too.
   

   https://www.googleapis.com/drive/v3/files/{{User::varFileId}}/export?mimeType=text/csv

3. Goto response tab. Check Save response and select “Save to file option”.
4. Enter file path.
5. Response character set select utf-8 (This will allow Unicode characters in response)
6. Click Test to see preview
   

   Export Google SpreadSheet to CSV file format using SSIS – Call Google Drive API to Export

7. Click OK to save

Load Google SpreadSheet data into SQL Server using SSIS (REST API Call)

Once you export SpreadSheet to CSV file you can easily consume it to load into SQL Server or any Target (e.g. Oracle, MySQL) using Native SSIS FlatFile Source and OLEDB/ADO.net Destination.

Write data to Google Spread Sheet

Now lets look at how to write to Google SpreadSheet. To write multiple rows in a single request you may use batchUpdate API. Also check this link to learn more.

Below is sample request to write data to columns and some data to rows (set majorDimension = COLUMNS to write vertical and use ROWS to write horizontally. You can mix multiple ways just like below example)

POST 
https://sheets.googleapis.com/v4/spreadsheets/1lkMEgu0zm2Q-cnPeoNZp4hqeIuASxXSpsrxtDDV89tg/values:batchUpdate
Authorization: Bearer ya29.Gl3WBFb3xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Content-Type: application/json
Accept: */*

{
  "valueInputOption": "USER_ENTERED",
  "data": [
    {
      "range": "Sheet2!A1:A4",
      "majorDimension": "COLUMNS",
      "values": [
        ["Item", "Wheel", "Door", "Engine"]
      ]
    },
    {
      "range": "Sheet2!B1:D2",
      "majorDimension": "ROWS",
      "values": [
        ["Cost", "Stocked", "Ship Date"],
        ["$20.50", "4", "3/1/2016"]
      ]
    }
  ]
}

Write data to Google SpreadSheet using REST API Task (OAuth 2.0 Connection)

Update Google Sheet Cells – Single Range (Set Values for multiple cells)

Now let’s look at the below example of updating multiple cells in google Sheet. For more information, you can see the example here. For parameter detail review this API documentation.

You can use SSIS REST API Task or Web API Destination to achieve Spread Sheet Update scenario. For simple example lets use REST API Task.

1. Drag and Drop ZS REST API Task on the control flow designer
2. Edit Task and select URL from Connection Method
3. Select OAuth Connection from the dropdown (Same connection we created in the previous section)
4. Enter the URL as below. Change Sheet ID, Range designator and API URL Parameters as per your need.
   https://sheets.googleapis.com/v4/spreadsheets/[Sheet-ID-Here]/values/[Range-Here]?[Parameters-Here] Example URL (some part masked with xxxxxxxx):  
   

   https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6xxxxxxxxCHrORJc/values/Sheet1!A2:B4?valueInputOption=USER_ENTERED

5. Change Request Method to PUT
6. Change Request Content type to application/json
7. Enter Request Body as below (Assuming we need to update 4 cells)
   

   {
   "majorDimension": "ROWS",
     "values": [
      ["row1_cellA","row1_cellB"],
      ["row2_cellA","row2_cellB"],
     ]
   }

8. Click Test Request see it works.
   

   Calling Google Sheet API to Update Multiple Cell Values

Update Google Sheet Cells – Multiple Ranges

If you like to update values which needs to define multiple ranges then you need to use batchUpdate API. Import changes are you can’t include Range in the URL like previous example of Single Range Update. Also you have to use POST method and body with multiple ranges.

Example of sheet update with multiple range

POST https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHxxxxxxxxxxxxxxxc/values:batchUpdate?valueInputOption=USER_ENTERED
Content-Type: application/json

>>>> BODY <<<<<

{
  "data": [
    {
      "range": "Sheet1!A2:B3",
      "majorDimension": "ROWS",
      "values": [
        [
          "row1_cellA",
          "row1_cellB"
        ],
        [
          "row2_cellA",
          "row2_cellB"
        ]
      ]
    },
    {
      "range": "Sheet1!D2:E3",
      "majorDimension": "ROWS",
      "values": [
        [
          "row1_cellD",
          "row1_cellE"
        ],
        [
          "row2_cellD",
          "row2_cellE"
        ]
      ]
    }
  ]
}

Write SQL Server data to Google Sheet

Now let’s look at real world example. How to write data from SQL Server table or other source and send to Google Sheet.

1. Drag and Drop SSIS Data Flow Task from SSIS Toolbox
   
2. Drag source (e.g. OLEDB Source) and configure it to read from source like SQL Server Table
   

   OLE DB Source – Drag and Drop

3. Drag ZS JSON Generator.
   1. Connect Source to JSON Generator (blue arrow)
   2. Double click it to configure
   3. Select Output Mode as Single Dataset Array option for Mode. Enter Some Batch value (e.g. 500) this will make sure we send 500 rows at a time to Google API call. This way if we have many rows its not rejected by API call.
   4. Click Add element (Select Static Element) . Name: majorDimension and Value: ROWS
   5. Now click Add Document Array icon, name as values
   6. check Treat as 2D array option and click OK to save
   7. Now under values node click Add elements > Select Multiple Columns and Add columns you like to add.
   8. Click OK to save JSON GeneratorBelow is just an example How you can use Batch Settings to create JSON with N rows in each JSON document.
      
4. Now drag ZS Web API destination
   1. Connect JSON Generator to Web API destination (blue arrow)
   2. Double click it to configure
   3. Select same OAuth connection manager created in earlier section (Assuming you had all scopes with write permission)
   4. Now select Input column for Body (JSON Generator Output)
   5. Enter API call URL as below (Replace ID with your own id)
      

      https://sheets.googleapis.com/v4/spreadsheets/{your-file-id}/values/Sheet1!A2?valueInputOption=USER_ENTERED

   6. Select Request Method as PUT
   7. Select content type as application/json
   8. You can enter sample Body As below.
      

      {
      "majorDimension": "ROWS",
        "values": [
         ["Bob","bob@mycompany.com",55],
         ["Sam","sam@mycompany.com",57]
        ]
      }

   9. Click Test Request / Response button. Check your Sheet in browser after it see it updates?
   10. If you see 200 OK response means we are good to go.
   11. Run package and see your real data updated in Sheet.

Format Cells using Google Sheet API

Now let’s look at an example to format cell using Google Sheet API (e.g. Change Font, Color, Border, Background). For more information review this API documentation.

To format Sheet you first need to get SheetID  (keep in mind this is not same as spreadsheetId which you used in URL). SheetId is basically numeric ID of your Tab in WorkSheet.

Get SheetID

Request

GET https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHtxxxxxxxxxxxxxx

Response

{
  "spreadsheetId": "1tuGO3_-2JlSmyiHwX6bRFUWqqrXlt4BRoX8rCHrORJc",
  "properties": {
    "title": "Test.xlsx",
    "locale": "en",
    "autoRecalc": "ON_CHANGE",
    "timeZone": "America/Los_Angeles",
    "defaultFormat": {
      "backgroundColor": {
        "red": 1,
        "green": 1,
        "blue": 1
      },
      "padding": {
        "right": 3,
        "left": 3
      },
      "verticalAlignment": "BOTTOM",
      "wrapStrategy": "OVERFLOW_CELL",
      "textFormat": {
        "foregroundColor": {},
        "fontFamily": "Calibri",
        "fontSize": 11,
        "bold": false,
        "italic": false,
        "strikethrough": false,
        "underline": false
      }
    }
  },
  "sheets": [
    {
      "properties": {
        "sheetId": 1896331083,
        "title": "Sheet1",
        "index": 0,
        "sheetType": "GRID",
        "gridProperties": {
          "rowCount": 1000,
          "columnCount": 26,
          "rowGroupControlAfter": true,
          "columnGroupControlAfter": true
        }
      }
    }
  ],
  "spreadsheetUrl": "https://docs.google.com/spreadsheets/d/1tuGO3_-2JlSmyiHwX6bRFUWqqrXlt4BRoX8rCHrORJc/edit"
}

Notice that in above response we see “sheetId”: 1896331083. This ID we will use in the Update API to format Cells.

Update Borders Example

Here is the example of Google Sheet Formatting API updateBorders Request

POST https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSxxxxlt4BRoX8rCHrORJc:batchUpdate
Content-Type: application/json

>>>> BODY <<<<<

{
  "requests": [
    {
      "updateBorders": {
        "range": {
          "sheetId": 1896331083,
          "startRowIndex": 0,
          "endRowIndex": 10,
          "startColumnIndex": 0,
          "endColumnIndex": 6
        },
        "top": {
          "style": "DASHED",
          "width": 1,
          "color": {
            "blue": 1.0
          },
        },
        "bottom": {
          "style": "DASHED",
          "width": 1,
          "color": {
            "blue": 1.0
          },
        },
        "innerHorizontal": {
          "style": "DASHED",
          "width": 1,
          "color": {
            "blue": 1.0
          },
        },
      }
    }
  ]
}

Response

{
  "spreadsheetId": "1tuGO3_-2JlSmyiHwX6bRFUWqqrXlt4BRoX8rCHrORJc",
  "replies": [
    {}
  ]
}

Update Cell Font Style (Bold, Style, Color, Background)

Now let’s look at how to use repeateCellRequest to apply certain type of formatting to multiple cells. This may be color, font style, background color.

POST https://sheets.googleapis.com/v4/spreadsheets/1tuGO3xxxxxx4BRoX8rCHrORJc:batchUpdate
Content-Type: application/json

>>>> BODY <<<<<

{
  "requests": [
    {
      "repeatCell": {
        "range": {
          "sheetId": 1896331083,
          "startRowIndex": 0,
          "endRowIndex": 1
        },
        "cell": {
          "userEnteredFormat": {
            "backgroundColor": {
              "red": 0,
              "green": 0.5,
              "blue": 0.7,
              "alpha": 1.3
            },
            "textFormat": {
              "foregroundColor": {
                "red": 1
              },
              "bold": true,
              "italic": true,
              "strikethrough": true,
              "underline": true
            }
          }
        },
        "fields": "userEnteredFormat"
      }
    }
  ]
}

In above example we have applied textFormat and backgroundColor but you can use any of below sections.

Set Currency Formatting

Here is an example request to set currency formatting

POST https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6bRFUWqqrXlt4BRoX8rCHrORJc:batchUpdate
Content-Type: application/json

>>>> BODY <<<<<

{
  "requests": [
    {
      "repeatCell": {
        "range": {
          "sheetId": 1896331083,
          "startRowIndex": 1,
          "startColumnIndex": 4,
          "endColumnIndex": 5
        },
        "cell": {
          "userEnteredFormat": {
            "numberFormat": {
              "type": "CURRENCY",
              "pattern": "$#,##0.00"
            }
          }
        },
        "fields": "userEnteredFormat.numberFormat"
      }
    }
  ]
}

Set Cell Validation

You can also set cell validation by calling following API call. This will restrict user’s ability to enter wrong data in certain cells.

POST https://sheets.googleapis.com/v4/spreadsheets/1tuGO3_-2JlSmyiHwX6bRFUWqqrXlt4BRoX8rCHrORJc:batchUpdate
Content-Type: application/json

>>>> BODY <<<<<

{
  "requests": [
    {
      "setDataValidation": {
        "range": {
          "sheetId": 1896331083,
          "startRowIndex": 1,
          "startColumnIndex": 0,
          "endColumnIndex": 6
        },
        "rule": {
          "condition": {
            "type": "ONE_OF_LIST",
            "values": [
              {
                "userEnteredValue": "Red"
              },
              {
                "userEnteredValue": "Green"
              },
              {
                "userEnteredValue": "Yellow"
              }
            ]
          },
          "inputMessage": "Color must be Red, Yellow or Green",
          "strict": true,
          "showCustomUi": true
        }
      }
    }
  ]
}

Dynamically obtain File ID for Google Drive REST API call

Once OAuth connection is defined you can now call any valid API from Google based on Permission you requested in OAuth Connection. OAuth connection takes care of renewing you Token and passing Authorization as per OAuth standard. All these protocol complexity is hidden from you.

First step to get data from Google SpreadSheet using REST API call is get fileId of the Google Drive File you want to read. The easiest way to know file ID is call following API. Below API call returns name, ID and other information about file(s). You can get only fileID and use that in Next Section to export Google SpreadSheet to CSV format.

https://www.googleapis.com/drive/v3/files

OR

https://www.googleapis.com/drive/v3/files/?q=name%3D'Customers'

NOTE: For more information about searching files in Google drive / getting list of files using search expression language check this link. Make sure expression is URL encoded (e.g. name=’Customers’ should be passed as name%3D’Customers’)

Above API returns response as below. Notice the name and fileId. We will use fileId when calling export API in next section.

{
 "kind": "drive#fileList",
 "files": [
  {
   "kind": "drive#file",
   "id": "1lkMEgu0zm2Q-cnPeoNZp4hqeIuASxXSpsrxtDDV89tg",
   "name": "Customers",
   "mimeType": "application/vnd.google-apps.spreadsheet"
  },
  {
   "kind": "drive#file",
   "id": "0B6UyXr6ZlIFTc3RhcnRlcl9maWxl",
   "name": "How to get started with Drive",
   "mimeType": "application/pdf"
  }
 ]
}

If you want to automate retrieval of fileId from document name then perform the following steps.

1. In the SSIS package Control flow. Drag and drop ZS REST API Task from SSIS toolbox
2. In the Request Access mode, select “URL from Connection” option
3. Select OAuth connection manager (Created in Section-2)
4. Enter URL as below (It will return any file name equals to Customers (%3D is encoded value for = sign)
   

   https://www.googleapis.com/drive/v3/files/?q=name%3D'Customers'

5. Click Test (You will see file information)
   

   Get Google Drive File List in SSIS – Call Google Drive API to search file by name or expression

6. If you want to extract only fileID from full JSON then go to response tab
7. Select Format=JSON, Enter following JSONPath expression to extract only fileId from first record
   

   $.files[0].id

8. Check save response content, select SSIS variable where you want to save fileId as below. Now click Test to see preview of response. You will notice now only fileId will be returned from full JSON response.
   

   Get Google Drive fileid from file name using API call in SSIS

9. Click OK to save task

Conclusion

Google Drive API and Google Sheets API provides a great way to automate file related functionality. However, to call Google API  you have to use SDK / coding approach (e.g. C#, Java, Python, Ruby). Luckily ZappySys SSIS PowerPack provides a great way to integrate any Google API call via simple drag and drop approach without coding. Try out yourself see how long it takes to call virtually any REST API.

The post How to read / write Google SpreadSheet using SSIS appeared first on ZappySys Blog.