When you create a linked server, the connection succeeds, but the query fails. By default, non-admin users do not have access to a Linked Server in SQL Server. When such users attempt to query a Linked Server, they may encounter errors similar to the following:

OLE DB provider "SQLNCLI11" 
for linked server "ZappySysLink" returned message "Unable to open a logical session". 

Msg -1, Level 16, State 1, Line 0 

SMux Provider: Physical connection is not usable [xFFFFFFFF].

This issue occurs because the Linked Server does not have a login mapping configured for the user executing the query.

Cause

1. Non-admin users do not inherit access to Linked Servers by default.
2. No login mapping exists between the local SQL Server login and the remote Linked Server credentials.
3. SQL Server cannot determine which remote credentials to use when the user connects.

For admin users, SQL Server often uses the default option “Be made using this security context”, which allows access without additional configuration.

Solution

The post How to add non-admin user access for linked servers appeared first on ZappySys Blog.

In our previous article we explored several powerful features of the ZappySys Data Gateway. This follow‑up post focuses on keeping the gateway secure. You’ll learn about the new options available in the Network Settings tab, how to restrict access by IP address, and how to grant dataset‑level permissions to non‑admin users.

Network Settings Overview

The Network Settings tab contains filters that control which clients can connect to the gateway. Two common scenarios are addressed below: allowing a range of IP addresses, or permitting only a single IP address.

If your connection still fails even after opening port 5000, a third-party firewall is very likely blocking it.

The “Add Firewall Rule” option only creates a rule in Windows Defender Firewall and does not configure rules in third-party security software.

Common symptoms of this issue include:

• Login timeout or connection timeout errors
• OPENQUERY or Linked Server failing to connect
• Port appears open in Windows Firewall but connection still fails

Third-Party Firewall Blocking Gateway Connection

When you click “Add Firewall Rule” in ZappySys Data Gateway, the rule is added to Windows Defender Firewall to allow inbound traffic on the configured port (default: 5000).

In environments with third-party security software, this alone may not be sufficient.

Why?

If your system is using third-party security software (such as ESET Endpoint Security, McAfee, Symantec, etc.), Many enterprise antivirus tools include their own firewall, which takes priority over or replaces Windows Defender Firewall.
For example, in one case, the rule was added successfully in Windows Firewall, but traffic was still blocked by ESET Endpoint Security. Once the ESET firewall was paused, the connection worked immediately.
In such cases:

• Windows Defender rules may be ignored
• Traffic can still be blocked even if the port is open in Windows Firewall
• Connection attempts (e.g., Linked Server, ODBC queries) may fail with timeout errors

What to check

If you are unable to connect even after adding the firewall rule:

• Verify whether any third-party firewall/antivirus is installed
• Temporarily disable or pause it for testing
• If the connection works after disabling, then the traffic is being blocked by that software

How to fix

To resolve this issue:

• Add an inbound allow rule in your third-party firewall for the Gateway port (default: 5000)
• Ensure the rule allows traffic from the required source machines
• Ensure both inbound and outbound rules are allowed if required by your network policy
• Ensure rules are applied under the correct network profile (Domain / Private / Public)
• Contact your network/security administrator if needed

Test connectivity

You can verify if the port is reachable using PowerShell:

Test-NetConnection -ComputerName <GatewayHost> -Port 5000

If TcpTestSucceeded = False, the port is still blocked by a firewall or network security layer.

Note

⚠️ Important: Opening a port in Windows Defender Firewall alone does not guarantee connectivity if a third-party firewall is active.

IP Range Filter

1. Ensure you have installed the ZappySys ODBC PowerPack with the default options (this also installs the Data Gateway Service).
2. Open the gateway configuration app by typing “Gateway” in the Start menu and selecting ZappySys Data Gateway Configuration.
   

   Open ZappySys Data Gateway

3. Assuming the other tabs are already configured the way you want, switch to Network Settings.
4. Click the Edit IP Filters button (see screenshot below):
   

   Range IP Filters Network Settings for Data Gateway

5. In the IP Filters dialog, set Enabled to true, then supply the From and To addresses that define the permitted range:

Single range example:

[
  {
    "Name": "Network_1",
    "Enabled": true,
    "From": "192.168.1.1",
    "To": "192.168.1.255"
  }
]

Multiple ranges example:

[
  {
    "Name": "Network_1",
    "Enabled": true,
    "From": "192.168.1.1",
    "To": "192.168.1.255"
  },
  {
    "Name": "Network_2",
    "Enabled": true,
    "From": "192.167.1.1",
    "To": "192.167.1.255"
  }
]

6. Save the dialog, then click OK to apply the settings. Only clients whose IP addresses fall within the defined ranges will be allowed.

Single IP Filter

1. Repeat steps 1‑3 from the previous section.
2. Open the IP Filters dialog and enable it.
3. Enter the same value for the From and To fields to restrict access to one address.
4. Save and close the dialog; only that address may connect to the gateway.

Specific IP Filter Network Settings for Data Gateway.

Note: once any filter rule is enabled, all other addresses are automatically blocked.

User‑Based Permissions

You can further limit access by granting specific users permission to individual data sources. This is useful when you want to allow non‑admin users to consume only certain datasets.

1. Make sure the user(s) exist on the Users tab and that they are not assigned administrator rights.
2. Switch to the Datasets tab.
3. Find the row for the data source you wish to secure and click Edit in the Users column.
4. In the dialog that appears, select one or more users on the left pane and click the [ >> ] button to move them to the right pane.
5. Choose the desired permission type (for example, Full permission).
   

   Set Dataset User Permissions – ZappySys Data Gateway

6. Click OK to save the changes.
7. Restart the Data Gateway service to ensure the new permissions take effect.

Conclusion

In this article we reviewed several ways to harden the ZappySys Data Gateway. The network filters let you restrict connections to either a specific IP address or a range of addresses. The new user‑based permissions allow you to grant dataset access to individual users. Together these features make your gateway more secure and easier to manage.

If you haven’t yet installed the gateway, download it here to get started.

The post How to Secure ZappySys Data Gateway (Network Settings) appeared first on ZappySys Blog.

In our previous few posts we saw how to call various google apis in SSIS.  In this post lets learn how to call Google Search Console API in SSIS or other ODBC Compatible Apps such as Power BI, Informatica, SSRS using API Drivers for ODBC

About Google Search Console API (Google Webmaster API)

If you are new to Google Search Console API (i.e. Webmaster API) then start from this link. You can also check API Explorer here to test API requests.

Search API can be called by supplying either API key in URL  or use OAuth

Search Console V3 Covers following APIs (May add more in future)

Search Console API Call Example

Here is simple example of Search Console API call. Below example returns all sites you added under your search console account.

Request

GET https://www.googleapis.com/webmasters/v3/sites?key={YOUR_API_KEY}

Response

{
 "siteEntry": [
  {
   "siteUrl": "https://my-google-search-api-1.com/",
   "permissionLevel": "siteOwner"
  },
  {
   "siteUrl": "https://www.my-google-search-api-1.com/",
   "permissionLevel": "siteOwner"
  },
  {
   "siteUrl": "https://my-google-search-api-2.com/",
   "permissionLevel": "siteOwner"
  },
  {
   "siteUrl": "https://www.my-google-search-api-2.com/",
   "permissionLevel": "siteOwner"
  }
 ]
}

Step-By-Step – Call Search Console API in SSIS

Now lets look at very simple API call to list search console sites

1. Open SSIS Package
2. Drag and drop ZS REST API Task from SSIS tool box
   

   Drag and Drop ZS REST API Task from SSIS Toolbox

3. Now double click REST API Task and configure (Skip Step 5 if you use API Key instead of OAuth).
4. Enter API URL you like to call. For This example we will use simple URL as below
   OAuth based Authentication
   

   https://www.googleapis.com/webmasters/v3/sites/

   API Key based Authentication
   

   https://www.googleapis.com/webmasters/v3/sites/?key={YOUR_API_KEY}

5. If you want to use OAuth based credentials then perform the following steps
   1. Select URL from Connection 
   2. Click New ZS-OAUTH connection from Dropdown
   3. On the OAuth UI select Google as Provider
   4. Enter Scopes as below
      

      https://www.googleapis.com/auth/webmasters
      https://www.googleapis.com/auth/webmasters.readonly

   5. Click Generate Token. When you get Login Prompt enter your Google Account information and click Accept
   6. Click OK to Save OAuth UI
6. This is how it will look like if you use OAuth Connection
   

   Call Google Search Console API using SSIS REST API Task (OAuth Authentication)

7. Now click Test Request / Response to check if its working. You should get response like below

Read from Search Console Analytics API (query)

Now let’s look at how to call Search Console Analytics API (i.e. query) for Specified Site and extract data in tabular format using ZS JSON Source. In below example we will obtain search Impression, clicks, position etc for specified site. We will aggregate this over date and country (dimensions).

In this section you will learn how to use JSON Source Adapter to extract data from JSON file (In this case its Web URL).

1. Firstly, You need to Download and Install SSIS ZappySys PowerPack.
2. Once you finished first step, Open Visual Studio and Create New SSIS Package Project.
3. Now, Drag and Drop SSIS Data Flow Task from SSIS Toolbox.
   
4. Double click on the Data Flow task to see Data Flow designer surface.
5. From the SSIS toolbox drag and drop JSON Source on the Data Flow designer surface.
   
6. Double click JSON Source and configure like below
   1. Enter URL as below. Notice two things, first we used <<somedata,FUN_URLENC>> placeholder function to encode : and // in the URL so it become %2F %3A. We use

      --dont-escape--

        suffix at the end.
      For OAuth based Credentials
      

      https://www.googleapis.com/webmasters/v3/sites/<<https://mysite.com,FUN_URLENC>>/searchAnalytics/query--dont-escape--

      For API Key based Credentials
      

      https://www.googleapis.com/webmasters/v3/sites/<<https://mysite.com,FUN_URLENC>>/searchAnalytics/query/?key={YOUR_API_KEY}--dont-escape--

   2. Check Use Credentials and select OAuth connection (created earlier) (if you are using API key in URL then skip this step).
   3. Select POST as HTTP Method
   4. Enter Body for Query. Here are some examples for various queries you can use. For our case we want to group by date and country so use below
      

      {
      "startDate": "2019-01-01",
      "endDate": "2019-01-10",
      "dimensions" : ["date","country"]
      }

   5. Select Content Type as

      application/json

        from Content Type drop down.
   6. Select Filter or enter as 

      $.rows[*]

   7. Now go to Extract Multiple Arrays Tab and Check Array Flattening Option (on older version it was on the different Tab)
7. Click Preview and you will see your Google Search Console API data as below.
   

   Read from Google Search API Console (Analytics Query Endpoint) – JSON Source

Loading Google Search Console API data into SQL Server / Other Target

ZappySys SSIS PowerPack makes it easy to load data from various sources such as REST, SOAP, JSON, XML, CSV or from other source into SQL Server, or PostgreSQL, or Amazon Redshift, or other targets. The Upsert Destination component allows you to automatically insert new records and update existing ones based on key columns. Below are the detailed steps to configure it.

Step 1: Add Upsert Destination to Data Flow

1. Drag and drop the Upsert Destination component from the SSIS Toolbox.
2. Connect your source component (e.g., JSON / REST / Other Source) to the Upsert Destination.

SSIS - Data Flow - Drang and Drop Upsert Destination Component

Step 2: Configure Target Connection

1. Double-click the Upsert Destination component to open the configuration window.
2. Under Connection, select an existing target connection or click NEW to create a new connection.
   • Example: SQL Server, or PostgreSQL, or Amazon Redshift.

Step 3: Select or Create Target Table

1. In the Target Table dropdown, select the table where you want to load data.
2. Optionally, click NEW to create a new table based on the source columns.

Configure SSIS Upsert Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS

Step 4: Map Columns

1. Go to the Mappings tab.
2. Click Auto Map to map source columns to target columns by name.
3. Ensure you check the Primary key column(s) that will determine whether a record is inserted or updated.
4. You can manually adjust the mappings if necessary.

SSIS Upsert Destination - Columns Mappings

Step 5: Save Settings

• Click OK to save the Upsert Destination configuration.

Step 6: Optional: Add Logging or Analysis

• You may add extra destination components to log the number of inserted vs. updated records for monitoring or auditing purposes.

Step 7: Execute the Package

• Run your SSIS package and verify that the data is correctly inserted and updated in the target table.

SSIS Upsert Destination Execution

Import Google Search Console API data in Reporting / other ETL tools (ODBC Usecase)

There will be a time when you dont want to use SSIS connector like mentioned in previous sections but extract Google Search API data in Reporting Tools / Other ETL Platform.  Good news is you can use ZappySys ODBC Driver for JSON / REST API in ODBC Apps. See few popular ODBC Apps below.

• Power BI connection for Google Search Console API
• Excel connection for Google Search Console API
• Informatica connection for Google Search Console API
• MS Access connection for Google Search Console API
• C# , PowerShell, VB.net connection for Google Search Console API

Preview / Generate Query in JSON Driver / XML Driver

Load Google Search Console API in SQL Server without any ETL

Now let’s look at even more interesting integration scenario. If you have SQL Server and like to load Google Search Console API without any ETL then you can use Data Gateway

Read this article to learn how to load REST API in SQL Server without any coding (Use just T-SQL) .

You can write query like below to load data inside SQL Table.

SELECT * FROM OPENQUERY([MY_LINKED_SERVER]
, 'SELECT * FROM $
WITH(
	 Src=''https://www.googleapis.com/webmasters/v3/sites/<<https://mysite.com,FUN_URLENC>>/searchAnalytics/query/?key={YOUR_API_KEY}--dont-escape--''
	,Filter=''$.rows[*]''
	,RequestData=''{
"startDate": "2019-01-01",
"endDate": "2019-01-10",
"dimensions" : ["date","country"]
}''
	,RequestContentTypeCode=''ApplicationJson''
	,Header=''cache-control: no-cache || Accept: */*''
	,RequestMethod=''POST''
	,EnableArrayFlattening=''True''
)')

SSMS Output – Query REST API via Linked Server OPENQUERY statement (Connect to ZappySys Data Gateway)

Conclusion

So in this post we saw how easy it is to achieve total REST API integration in SSIS using JSON Connector. We also saw how to use ODBC Drivers to query REST API data inside Apps like Excel, MS Access, Informatica and other ODBC Apps using ODBC JSON / REST API driver. If you are SSIS User download SSIS PowerPack and try for FREE and if you don’t have SSIS in house and want to try more generic approach then Download ODBC PowerPack Drivers for FREE Trial.

The post Get data from Google Search Console API in SSIS and ODBC Apps appeared first on ZappySys Blog.

There might be a case when you have many CSV, JSON or XML files in Amazon S3 bucket and you want them to be imported straight into a SQL Server table. Here come ZappySys ODBC PowerPack and ZappySys Data Gateway (part of ODBC PowerPack) which will enable you to accomplish that. ZappySys ODBC PowerPack includes powerful Amazon S3 CSV, Amazon S3 JSON and Amazon S3 XML drivers that let you connect to an S3 bucket and read the contents from the files in the bucket. Bringing in ZappySys Data Gateway allows doing that right from a SQL Server. Let’s begin and see how to import Amazon S3 files into SQL Server.

These drivers are related to this article:

Amazon S3 Driver (for XML Files)

Amazon S3 Driver (for JSON Files)

Amazon S3 Driver (for CSV Files)

Prerequisites

1. Have an AWS account.
2. Existing Amazon S3 bucket.
3. SQL Server instance installed (can be a SQL Server Express LocalDB instance).
4. SQL Server Management Studio (SSMS) installed.
5. ZappySys ODBC PowerPack installed (must be installed on a Windows machine; can be a different machine than SQL Server is installed on).

Getting started

We will start from a simple example of how to load contacts from a JSON that are located in Amazon S3 bucket, then we will proceed by loading contacts from many XML files that are compressed in ZIP, and finally, we will traverse many folders and subfolders to gather CSVs and load contacts from them.

Step-by-Step – Import JSON file located at Amazon S3 into SQL Server

Open and configure ZappySys Data Gateway

Create an Amazon S3 JSON data source in ZappySys Data Gateway

1. The first thing you will have to do is to create a data source in ZappySys Data Gateway. Just click Add button, give the data source a name, e.g. “MyContactsJSON“, and then select Native – ZappySys Amazon S3 JSON Driver:
   

   Adding a JSON data source in ZappySys Data Gateway to load it from Amazon S3 bucket into SQL Server

2. Then click Edit and add the Data Gateway user you created in Users tab. We will use this user later when adding a Linked Server to the Data Gateway to authenticate:
   

   Adding a user to the newly created Amazon S3 JSON data source

3. Once you do that, then click Edit to configure the data source:
   

   Preparing to configure the Amazon S3 JSON data source to load a JSON from Amazon S3 bucket to SQL Server

4. When a window opens, click Click here to Configure the Connection and enter your Access Key and Secret Key:
   

   Configuring the authentication to Amazon S3 bucket in ZappySys Data Gateway

5. After that, select a JSON file you want to load, and then click Select Filter button to choose data you want to be displayed in SQL Server:
   

   Configuring the JSON data source to load a JSON from Amazon S3 bucket into SQL Server

6. Go to the Preview tab and click the Preview Data button to make sure everything is configured correctly and preview the results:
   

   Previewing JSON data in the data source based on Amazon S3 JSON Driver

Now you are ready to create a Linked Server and connect to the data source you just created. If you used “MyContactsJSON” as the data source name, make sure to use the same name when creating a Linked Server.

Set up a SQL Server Linked Server

Execute the SQL query

Once you created the Linked Server to ZappySys Data Gateway, you are ready to execute the SQL query and load data into SQL Server. Supposedly, you created the Linked Server with the name “GATEWAY”, then open SSMS and execute this query:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see similar results after you execute the query:

The results of getting JSON data straight from Amazon S3 bucket using ZappySys Amazon S3 JSON driver

Now we are ready to proceed to the next section and import many XML files at once.

Step-by-Step – Import many XML files located at Amazon S3 into SQL Server

Overview

Supposedly, you have many XMLs where each one is zipped and you want to load them all into a SQL Server table:

Zipped XML files located in Amazon S3 bucket to be loaded into SQL Server

The first thing you will have to do is to create a data source, based on ZappySys Amazon S3 XML Driver. Let’s proceed and just do that.

Create an Amazon S3 XML data source in ZappySys Data Gateway

Follow the same steps as in Create an Amazon S3 JSON data source in ZappySys Data Gateway section when adding a new data source, except for these two steps:

1. When adding a data source select Native – ZappySys Amazon S3 XML Driver as Connector Type:
   

   Choosing “ZappySys Amazon S3 XML Driver” to load XMLs to SQL Server

2. Then click Edit and edit the data source similarly like in this window:
   

   Configuring data source based on ZappySys Amazon S3 XML Driver to load XMLs into SQL Server

Now we are ready to set up a Linked Server to this newly created data source.

Set up a SQL Server Linked Server

Follow the very same steps from the section above where we added the Linked Server to a JSON data source, except that change Datasource property of the Linked Server to match the name of the Amazon S3 XML data source you created. Once you do that, you are ready to load XMLs into your SQL Server.

Execute the SQL query

Again, if you created the Linked Server with name “GATEWAY”, execute this SQL query in SSMS:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see a similar view once you execute the query:

The data of many XMLs loaded from Amazon S3 bucket into SQL Server

Now we are ready to move to the next section and recursively scan CSVs and load them into SQL Server.

Step-by-Step – Import CSV file located at Amazon S3 into SQL Server

Overview

Let’s say you have many CSVs that are located in a folder tree, including subfolders and you want them all in your SQL Server:

Many CSV files located in Amazon S3 bucket folders and subfolders to be loaded into SQL Server

We will follow the same steps to accomplish that as we did for loading JSON and XMLs.

Create an Amazon S3 CSV data source in ZappySys Data Gateway

Follow the steps as in Create an Amazon S3 JSON data source in ZappySys Data Gateway section when adding a new data source, except for these two steps:

1. When adding a CSV data source select Native – ZappySys Amazon S3 CSV Driver as Connector Type:
   

   Choosing “ZappySys Amazon S3 CSV Driver” to load CSVs to SQL Server

2. Then click Edit and edit the data source similarly:
   

   Configuring data source based on ZappySys Amazon S3 CSV Driver to load CSVs into SQL Server

Now we are ready to proceed and add a Linked Server to this data source.

Set up a SQL Server Linked Server

Again, follow the very same steps from the section above where we added the Linked Server to a JSON data source, except that change Datasource property of the Linked Server to match the name of the Amazon S3 CSV data source we created. Once you do that, you are ready to load CSVs into your SQL Server.

Execute the SQL query

Once again, if you created the Linked Server with name “GATEWAY”, execute this SQL query in SSMS:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see a similar view, once you execute the query:

The results of loading many CSVs from Amazon S3 bucket into SQL Server

Conclusion

We had a goal to import Amazon S3 files into SQL Server from SQL Server itself. We learned how to load JSON, XML and CSV files into SQL Server, which some of them were zipped and were located at different folders and subfolders. It was possible to accomplish that using ODBC PowerPack, ZappySys Data Gateway and ODBC PowerPack drivers – Amazon S3 JSON Driver, Amazon S3 XML Driver, and Amazon S3 CSV Driver. At first, we created data sources in the Data Gateway, then we created Linked Servers in SQL Server and connected to ZappySys Data Gateway, which finally made it possible to load data from Amazon S3 bucket straight from SQL queries in SSMS.

Check Amazon S3 Integration with Other BI Tools (Power BI, Excel, SSRS, MS Access, etc.)

ZappySys ODBC Drivers built using ODBC standard which is widely adopted by industry for a long time. Which mean the majority of BI Tools / Database Engines / ETL Tools already there will support native / 3rd party ODBC Drivers. Below is the small list of most popular tools / programming languages our Drivers support. If your tool / programming language doesn't appear in the below list, which means we have not documented use case but as long as your tool supports ODBC Standard, our drivers should work fine.  

BI / Reporting Tools Integration	ETL Tools Integration	Programming Languages Integration
Microsoft Power BI Tableau SSRS (SQL Reporting Services) QlikView /Qlik Sense MicroStrategy Google Sheet Microsoft Excel Microsoft Access Oracle OBIEE Many more (not in this list).....	Informatica PowerCenter (Windows) Informatica Cloud SSIS (SQL Integration Services) SQL Server Talend Data Studio Pentaho Kettle Oracle OBIEE Many more (not in this list).....	Visual Studio C# C++ JAVA Python PHP PowerShell T-SQL (Using Linked Server)

 

The post Import Amazon S3 files into SQL Server (CSV/JSON/XML Driver) appeared first on ZappySys Blog.

There might be a case when you have many CSV, JSON or XML files in Azure Blob and you want them to be imported straight into a SQL Server table. Here come ZappySys ODBC PowerPack and ZappySys Data Gateway (part of ODBC PowerPack) which will enable you to accomplish that. ZappySys ODBC PowerPack includes powerful Azure Blob CSV, Azure Blob JSON and Azure Blob XML drivers that let you connect to an Azure Storage Explorer and read the contents from the files in the Container. Bringing in ZappySys Data Gateway allows doing that right from a SQL Server. Let’s begin and see how to Import Azure Blob files into SQL Server (CSV/JSON/XML Driver).

These drivers are related to this article:

Azure Blob Driver (for CSV Files)

Azure Blob Driver (for JSON Files)

Azure Blob Driver (for XML Files)

Prerequisites

1. Firstly, Download and install Microsoft Azure Storage Emulator and Microsoft Azure Storage Explorer.
2. SQL Server instance installed (can be a SQL Server Express LocalDB instance).
3. SQL Server Management Studio (SSMS) installed.
4. ZappySys ODBC PowerPack installed (must be installed on a Windows machine; can be a different machine than SQL Server is installed on).

NOTE: If you want to use Live account (Azure Storage) then you can skip Step #3

Getting started

In order to start, we will show several examples. ZappySys includes Data Gateway that will help you in reading data of Azure Blob files. Here we are showing you is, how to Import Azure Blob files into SQL Server (CSV/JSON/XML Driver).

You can connect to Azure Storage Service from SSIS, you will need Storage Account Name and Access Key. Ask your SysAdmin or responsible person to provide that information to you. Click here to read more about how to get your Storage Account Name and Access Key. Here are sample Credentials.

Account Name: mystorageaccount
Access Key: Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==

If you don’t have Azure Storage account then you can try offline mode on your local machine. You can just download Azure Storage Emulator and start testing.

Setup Azure Storage Explorer

Setup Azure Storage client tools we will start from a simple example of how to load Invoice Data from a CSV that is located in Azure Blob, then we will load JSON contacts data from them and finally, we will proceed by loading contacts from many XML files those are compressed in ZIP.

1. Once you have downloaded and installed storage emulator You can launch Microsoft Azure Storage Emulator from its Physical location or from the desktop or start menu shortcut.

   

   Azure Storage Emulator Physical Location

2. If you can see the below-attached Command Prompt screen after Emulator started. Then you can proceed to start Microsoft Azure Storage Explorer as the Azure Storage Emulator is started successfully.

   

   Command Prompt Screen after Microsoft Azure Storage Emulator Started

3. Now, you have to download and install Microsoft Azure Storage Explorer and then you can launch Microsoft Azure Storage Explorer from its Physical location or from the desktop or start menu shortcut.

   

   Microsoft Azure Storage Explorer Location.

4. For Creating a Blob Container, First of all, you need to go to Microsoft Storage Explorer Window. Then you can go through like this way (Storage Accounts –> (Development) –> Blob Containers).
   

   Microsoft Azure Storage Explorer: Create a Blob Container

Open and Configure ZappySys Data Gateway

Import CSV file located at Azure Blob into SQL Server

Overview

Let’s say you have CSV file that is located in Azure Blob Container, and you want them all data into your SQL Server. We will follow the same steps for that we will do for import JSON and XML file into SQL Server.

Azure Blob Container – CSV File Located

Create an Azure Blob CSV data source in ZappySys Data Gateway

1. The first thing you will have to do is to create a data source in ZappySys Data Gateway. In the Data Source tab, Just click on Add button, give the data source a name, e.g. “MyInvoiceCSV“, and then select Native – ZappySys Azure Blob CSV Driver.
   

   Create Data Source – Azure Blob CSV Driver

2. Then click on Edit and add the Data Gateway user you created in the Users tab. We will use this user later when adding a Linked Server to the Data Gateway to authenticate:
3. Now, click on the Edit button to configure “MyInvoiceCSV” Azure Blob CSV Data Source.
   

   Configure Data Gateway

4. When a window open, click on Click here to Configure the Connection and select Use the Microsoft Azure Storage Emulator. If you have Online Storage Account and Account Key then Select Enter Storage Account Credentials and enter it.
   
   

   Create Azure Storage Connection

5. Now, Select Azure Blob Container and file from it and In the Data Format / Compression (Zip / GZip) tab set suitable file Compression Format (Zip or GZip).
   

   ZappySys Azure Blob CSV – Configure Driver

6. Go to the Preview tab and click on the Preview Data button to make sure everything is configured correctly and preview the results.
   

   ZappySys Azure Blob CSV – Preview Data

7. Now, Click on OK button also Save and Restart Service.
   

   ZappySys Data Gateway Service – Save and Restart

8. That’s all, now we are ready to proceed and add a Linked Server to this data source.

Set up a SQL Server Linked Server

Execute the SQL query

Once again, if you created the Linked Server with name “GATEWAY”, execute this SQL query in SSMS:

SELECT * INTO MyInvoices FROM OPENQUERY([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyInvoices

The results

You should see a similar view, once you execute the query:

The results of loading many CSVs from Azure Blob into SQL Server

Import JSON file located at Azure Blob into SQL Server

Overview

Let’s say you have JSON file that is located in Azure Blob Container, and you want them all data into your SQL Server. We will follow the same steps of the import CSV file into SQL Server and we will do for XML file into SQL Server.

Create an Azure Blob JSON data source in ZappySys Data Gateway

Follow the steps as in Create an Azure Blob CSV data source in ZappySys Data Gateway section when adding a new data source, except for these two steps:

1. The first thing you will have to do is to create a data source in ZappySys Data Gateway. Just click Add button, give the data source a name, e.g. “MyContactsJSON“, and then select Native – ZappySys Azure Blob JSON Driver.
   

   ZappySys Data Gateway – Create New Data Source
2. After that, select a JSON file from container you want to load, and then click Select Filter button to choose data you want to be displayed in SQL Server.
   

   Configuring the JSON data source to load a JSON from Azure Blob container into SQL Server

Now you are ready to create a Linked Server and connect to the data source you just created. If you used “MyContactsJSON” as the data source name, make sure to use the same name when creating a Linked Server.

Set up a SQL Server Linked Server

Again, follow the very same steps from the section above where we added the Linked Server to a CSV data source, except that change Catalog property of the Linked Server to match the name of the Azure Blob JSON data source we created. Once you do that, you are ready to load JSON into your SQL Server.

Execute the SQL query

Once you created the Linked Server to ZappySys Data Gateway, you are ready to execute the SQL query and load data into SQL Server. Supposedly, you created the Linked Server with the name “GATEWAY”, then open SSMS and execute the following query:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see similar results after you execute the query:

The results of getting JSON data straight from Azure Blob JSON

Now we are ready to proceed to the next section and import many XML files at once.

Import many XML files located at Azure Blob into SQL Server

Overview

Supposedly, you have many XMLs where each one is zipped and you want to load them all into a SQL Server table.

Zipped XML files located in Azure Blob container to be loaded into SQL Server

The first thing you will have to do is to create a data source, based on ZappySys Azure Blob XML Driver. Let’s proceed and just do that.

Create an Azure Blob XML data source in ZappySys Data Gateway

Follow the same steps as in Create an Azure Blob JSON data source in ZappySys Data Gateway section when adding a new data source, except for these two steps:

1. When adding a data source select Native – ZappySys Azure Blob XML Driver as Connector Type.
   

   Choosing “ZappySys Azure Blob XML Driver” to load XMLs to SQL Server
2. Then click on Edit and select Azure Blob Container and file from it and In the Data Format / Compression (Zip / GZip) tab set suitable file Compression Format (Zip or GZip).
   

   Configuring data source based on ZappySys Azure Blob XML Driver to load XMLs into SQL Server

3. Now we are ready to set up a Linked Server to this newly created data source.

Set up a SQL Server Linked Server

Follow the very same steps from the section above where we added the Linked Server to a CSV data source, except that change Catalog property of the Linked Server to match the name of the Azure Blob XML data source you created. Once you do that, you are ready to load XMLs into your SQL Server.

Execute the SQL query

Again, if you created the Linked Server with name “GATEWAY”, execute the following SQL query in SSMS:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see a similar view once you execute the query:

The data of many XMLs loaded from Azure Blob Container into SQL Server

Now we are ready to move to the next section and recursively scan CSVs and load them into SQL Server.

Conclusion

We had a goal to import Azure Blob files into SQL Server from SQL Server itself. We learned how to Import Azure Blob files into SQL Server (CSV/JSON/XML Driver), which some of them were zipped and were located at different folders and subfolders. It was possible to accomplish that using ODBC PowerPack, ZappySys Data Gateway, and ODBC PowerPack drivers – Azure Blob JSON Driver, Azure Blob XML Driver, and Azure Blob CSV Driver. At first, we created data sources in the Data Gateway, then we created Linked Servers in SQL Server and connected to ZappySys Data Gateway, which finally made it possible to load data from Azure Blob straight from SQL queries in SSMS.

References

Finally, you can use the following links for more information:

• Azure Blob CSV Driver
• Azure Blob JSON Driver
• Azure Blob XML Driver

The post Import Azure Blob files into SQL Server (CSV/JSON/XML Driver) appeared first on ZappySys Blog.

In our previous blog we saw how to read JIRA data in SQL Server. Now let’s learn how to Import SAP S/4HANA OData Service Data Into Sql Server. SAP S/4HANA provides OData REST API interface to access data in your application using HTTP Protocol. We will use ODBC XML Driver to read SAP data and load into SQL Server.

So, let’s get started.

About SAP HANA / OData REST API Service

You can expose your SAP Data using ODATA REST API Service. Here is a good article which shows how to expose data as OData Service.   For more information on SAP OData Service feature check this SAP help page. If you are new to OData Standard then read here to know more how OData can facilitate data extraction using HTTP REST API. If you need Sample XML based OData Service then use below test URLs.

https://services.odata.org/Northwind/Northwind.svc/
https://services.odata.org/Northwind/Northwind.svc/Customers
https://services.odata.org/Northwind/Northwind.svc/Orders
https://services.odata.org/Northwind/Northwind.svc/Invoices
https://services.odata.org/Northwind/Northwind.svc/Products

Now let’s look at how to read SAP Data using ODBC Driver. At this point we assume you have exposed your data as OData Endpoint.

Read SAP S/4HANA data using XML Driver

First let’s look at steps to configure XML Driver. We will read data from SAP S/4HANA OData Service and then in next section we will look at how to load data into SQL Server or other target.

1. To do this, first of all, we will open the ZappySys Data Gateway Configuration:
   

   Open ZappySys Data Gateway

2. Add the Native – ZappySys XML Driver Data source.
   

   Add Gateway Data Source

3. Now edit that XML data source to configure it. Enter your OData Service URL its typically like below. Replace 3 parts with your own value (i.e. replace {MY-INSTANCE},  {MY-PROJECT}, {MY-TABLE})
   

   https://{MY-INSTANCE}/sap/opu/odata/sap/{MY-PROJECT}/{MY-TABLE}

   For example if you are hosting SAP HANA in Cloud Instance then your URL may look like below
   

   https://myXXXXXX-api.s4hana.ondemand.com/sap/opu/odata/sap/MyTestProject/PurchaseOrders

   And create New ZS-HTTP connection in it select Basic Authentication and enter your SAP HANA UserID / Password to call OData Service and select the desire filter.
   

   XML Driver : SAP S/4HANA OData Service Configuration

4. Now go to Data Format / Compression (Zip/GZip) tab and select Data Format as OData to get all the records.
   

   Data Format : Odata

5. Finally, now using Query Builder and Code Generator we will generate the query.
   

   Generate Query

6. That’s it we are ready to load SAP S/4HANA OData Service data to SQL Server.

Load SAP S/4HANA OData Service in MS SQL Server

SAP S/4HANA Integration with Other BI Tools (Power BI, Excel, SSRS, MS Access…)

ZappySys ODBC Drivers built using ODBC standard which is widely adopted by industry for a long time. Which mean the majority of BI Tools / Database Engines / ETL Tools already there will support native / 3rd party ODBC Drivers. Below is the small list of most popular tools / programming languages our Drivers support. If your tool / programming language doesn't appear in the below list, which means we have not documented use case but as long as your tool supports ODBC Standard, our drivers should work fine.  

BI / Reporting Tools Integration	ETL Tools Integration	Programming Languages Integration
Microsoft Power BI Tableau SSRS (SQL Reporting Services) QlikView /Qlik Sense MicroStrategy Google Sheet Microsoft Excel Microsoft Access Oracle OBIEE Many more (not in this list).....	Informatica PowerCenter (Windows) Informatica Cloud SSIS (SQL Integration Services) SQL Server Talend Data Studio Pentaho Kettle Oracle OBIEE Many more (not in this list).....	Visual Studio C# C++ JAVA Python PHP PowerShell T-SQL (Using Linked Server)

 

Conclusion

So in this blog, we learned how to Import SAP S/4HANA OData Service Data in MS SQL Server using ODBC XML / SOAP API Driver in a very simple way. You can achieve many more functionalities with this tool. Check our blogs/articles on XML File / REST API Driver https://zappysys.com/blog/category/odbc-powerpack/odbc-drivers/xml-soap-api-driver/ to find out what this tool is capable of more.

References

Finally, you can use the following links for more information about the use of SAP S/4HANA OData Service with our tools:

• SAP API Business Hub
• Landing Page for ODBC XML / SOAP API Driver, you can also find Tutorial Video here.
• Help File: Documentation of XML Driver.
• Blog/articles link: https://zappysys.com/blog/category/odbc-powerpack/odbc-drivers/xml-soap-api-driver/

The post Import SAP S/4HANA OData Service Data Into Sql Server via ODBC Driver appeared first on ZappySys Blog.

In this release we focused on adding URL JOIN Features which may allow many multi steps scenarios in one query. Bing Ads, Amazon MWS, Google BigQuery, Amazon Athena all these supports Job style queries which needs 3 step process (Create job, keep checking status, once done read report data). We also improved compatibility with Informatica and MS Access.

Version 1.1.3.10621 [Jun 24, 2019]

New Features/Improvements

• NEW: Add option to replace placeholders in File content if you use IsMultiPart = True
• NEW: All API Drivers – Add option EnableBodyPlaceholderForMultiPart to support replacing placeholders in file content in IsMultiPart mode (E.g. replace <%access_token%> or [$token$] or [$alias.MyJoinColumn$] )
• NEW: All API Drivers – Add option OutputAsRawDocumentSingleRow (Returns __DOCUMENT__ field with entire File / Response string in one field)
• NEW: All API Drivers – Add Options to parse from any unstructured data – RawOutputExtractMode, RawOutputDataRowTemplate, RawOutputFilterExpr
• NEW: All API Drivers – Add Status Check feature for all API drivers (Keep sending request until desired value found) – Options added EnableStatusCheck, StatucCheckMaxWaitSeconds, StatucCheckIterationWaitSeconds, StatusSuccessValue, StatusFailureValue
• NEW: All API Drivers – Add support for ConnectionType in URL Join
• NEW: All API Drivers – Add Support for HMAC Authentication
• NEW: All ODBC Drivers – Add support for Parameters
• NEW: Data Gateway – Add Code Generator UI for SQL / Java
• NEW: Data Gateway – Allow to add firewall rule via UI
• NEW: Data Gateway – Better logging about connected devices
• NEW: Salesforce Driver – Add support for Select * FROM

Bug fixes

• FIX: All API Drivers – joinXX_DataConnectionType Option not supported in JOIN URL
• FIX: All API Drivers- Show meaningful error when same option used twice in WITH clause (Right now you see odd error – An item with the same key has already been added)
• FIX: Apps – Informatica – Driver may output many extra empty rows in some cases
• FIX: Apps – MS Access – Linked table throwing call fail error (due to missing support for parameters in current driver)
• FIX: Data Gateway – Certain errors not reported correctly back to caller
• FIX: Data Gateway – Enabling caching options may hang query in some case
• FIX: Data Gateway – Generic ODBC Connection Type fails with error Connection is Busy
• FIX: Data Gateway – When you use Meta option in WITH clause you may see empty rowset rather error in some cases
• FIX: Salesforce Driver – Can not load table list in Excel / Power BI
• FIX: Salesforce Driver – Failed to preview data in Power BI
• FIX: Salesforce Driver – Load ConnectionString doesnt set Username
• FIX: Salesforce Driver – Nested queries not supported – You may get syntax error
• FIX: Salesforce Driver – SSIS ODBC destination showing bad Table names in list

The post ODBC PowerPack v1.1.3 Released appeared first on ZappySys Blog.

In our previous blog post we saw how to import REST / SOAP API in SQL Server. Using same concepts let’s look at how to import Bing Ads data into SQL Server. We will explore many techniques to call Bing Ads API and learn how to automate data extraction without doing any ETL. You can call Bing Ads API just using T-SQL code (Yes you heard it right). At the end of this article you will learn how to Download Performance Reports from Bing Ads Account without any coding (See sample screenshot below). Please go through full article carefully.

NOTE: Bing Ads now renamed as Microsoft Advertising but this article will use popular name for time being.

Import Bing Ads data into SQL Server (Using SSMS T-SQL Code)

About Bing Ads API / SOAP Web Service

If you are new to Bing Ads API then start from here. Bing Ads APIs are SOAP XML APIs. You can call it to automate many scenarios for Bing Ads. In this article we will mainly focus on read scenario (e.g. Read Performance Data) but you can use these techniques to write / update (e.g. Create New Campaign).

If you are new to SOAP WebService then Check this article to learn more. It will explain you how to use tools like SoapUI tool to craft SOAP Requests for Bing API or any other SOAP API.

For Bing API you can use Service WSDL files found here. Use URL ending ?wsdl to import in SoapUI.

Example Bing API Request

Here is raw API request for Bing API. This sample assumes you have completed 3-Legged OAuth Authorization to obtain AccessToken and RefreshToken (Explained later in this article). Below request uses AccessToken you might have already obtained via process like this. If you are using ZappySys tools then this will be generated automatically for you.

Request Method / URL:

POST https://clientcenter.api.bingads.microsoft.com/Api/CustomerManagement/v13/CustomerManagementService.svc

Headers:
Notice that for each Api you may need correct SOAPAction in Header. If you are not sure then use SoapUI Tool. Execute request in SoapUI and then go to Raw tab like this one.

Content-Type: text/xml; charset=utf-8
SOAPAction: "GetUser"

Body:
Notice that you have to replace AuthenticationToken and DeveloperToken in below request. AccessToken is placed inside AuthenticationToken tag. This Auth token is short lived and extracted via OAuth Process and DeveloperToken you can extract as mentioned in the previous section or get it from here.

  
    GetUser
    <%access_token%>
    1052Bxxxxxxxxxxxxxx
  
  
    
      
    
  

Response:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Header>
    <h:TrackingId xmlns:h="https://bingads.microsoft.com/Customer/v13">fddf3cb9-e4c2-4e75-9546-f3bc20ea91a8</h:TrackingId>
  </s:Header>
  <s:Body>
    <GetUserResponse xmlns="https://bingads.microsoft.com/Customer/v13">
      <User xmlns:a="https://bingads.microsoft.com/Customer/v13/Entities" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
        <a:ContactInfo>
          <a:Address>
            <a:City i:nil="true" />
            <a:CountryCode>US</a:CountryCode>
            <a:Id i:nil="true" />
            <a:Line1 i:nil="true" />
            <a:Line2 i:nil="true" />
            <a:Line3 i:nil="true" />
            <a:Line4 i:nil="true" />
            <a:PostalCode i:nil="true" />
            <a:StateOrProvince i:nil="true" />
            <a:TimeStamp i:nil="true" />
            <a:BusinessName i:nil="true" />
          </a:Address>
          <a:ContactByPhone>false</a:ContactByPhone>
          <a:ContactByPostalMail>false</a:ContactByPostalMail>
          <a:Email>someone@zappysys.com</a:Email>
          <a:EmailFormat i:nil="true" />
          <a:Fax i:nil="true" />
          <a:HomePhone i:nil="true" />
          <a:Id>48045678</a:Id>
          <a:Mobile i:nil="true" />
          <a:Phone1>111-222-3333</a:Phone1>
          <a:Phone2 i:nil="true" />
        </a:ContactInfo>
        <a:CustomerId>19112345</a:CustomerId>
        <a:Id>48012345</a:Id>
        <a:JobTitle i:nil="true" />
        <a:LastModifiedByUserId>48012345</a:LastModifiedByUserId>
        <a:LastModifiedTime>2019-06-04T14:22:35.38</a:LastModifiedTime>
        <a:Lcid>EnglishUS</a:Lcid>
        <a:Name>
          <a:FirstName>Someone</a:FirstName>
          <a:LastName>Good</a:LastName>
          <a:MiddleInitial i:nil="true" />
        </a:Name>
        <a:Password i:nil="true" />
        <a:SecretAnswer i:nil="true" />
        <a:SecretQuestion>None</a:SecretQuestion>
        <a:UserLifeCycleStatus>Active</a:UserLifeCycleStatus>
        <a:TimeStamp>AAAAAJiX/cU=</a:TimeStamp>
        <a:UserName>someone@zappysys.com</a:UserName>
        <a:ForwardCompatibilityMap i:nil="true" xmlns:b="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
      </User>
      <CustomerRoles xmlns:a="https://bingads.microsoft.com/Customer/v13/Entities" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
        <a:CustomerRole>
          <a:RoleId>41</a:RoleId>
          <a:CustomerId>19160000</a:CustomerId>
          <a:AccountIds i:nil="true" xmlns:b="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />
          <a:LinkedAccountIds xmlns:b="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />
          <a:CustomerLinkPermission i:nil="true" />
        </a:CustomerRole>
      </CustomerRoles>
    </GetUserResponse>
  </s:Body>
</s:Envelope>

POST http://myhost/api/v2/customers/new
X-SecretKey: Abcdxxxxxxxxxxxxxx
Content-Type: application/json

{ firstname: "Someone", lastname: "Good" }

{
 request_status: "CreatedOK",
 new_customer_id: "1234566"
}

Step-By-Step: How to Import Bing Ads Data into SQL Server

Now let’s look at how to import Bing Ads Data into SQL Server Table. This tutorial will require 10-15 mins of your time. At the end of this tutorial you will be able to Read Bing Ads Performance Report and Import data into SQL Server. Step listed in this tutorial may help you to call Bing API in other tools like SSIS, Power BI, SSRS, Tableau and so on.

So let’s get started.

Register OAuth App for Bing Ads API (Azure Active Directory App)

Very first thing to call any Bing API is to register OAuth App (Under Azure Active Directory Tab).

Here is how to register OAuth App which can be used to call Bing Ads API later on.

1. Login to Azure Portal:
   • Navigate to the Azure Portal and log in using your credentials.
2. Access Azure Active Directory:
   • In the left-hand menu, click on Azure Active Directory.
3. Register a New Application:
   • Go to App registrations and click on New registration.
     
   • Application Name: Enter a name for your application.
     
   • Supported Account Types: Choose the account types your app will support. For example, select Accounts in this organizational directory only if you need access to data in your organization only.
   • Redirect URI:
     • Set the type to Web.
     • In the textbox enter https://login.microsoftonline.com/common/oauth2/nativeclient as the Redirect URI or any other valid redirect URL, e.g., https://zappysys.com/oauth.
     • Use this Redirect URI in the Redirect URL grid row.
4. Save Client ID:
   • After registering the app, copy the Application (client) ID and paste it into the Client ID field in the API Connection Manager configuration.
5. Set Authorization & Token URLs:
   • Click on the Endpoints link in the App registration overview.
   • Authorization URL: Copy the OAuth 2.0 authorization endpoint (v2) URL (e.g., https://login.microsoftonline.com/{your-tenant-id}/oauth2/v2.0/authorize) and paste it into the Authorization URL field in the configuration grid.
   • Token URL: Copy the OAuth 2.0 token endpoint (v2) URL (e.g., https://login.microsoftonline.com/{your-tenant-id}/oauth2/v2.0/token) and paste it into the Token URL field.
6. Create a Client Secret:
   • In the Certificates & secrets tab, click New client secret.
   • Set an expiration period for the secret.
   • Copy the generated client secret and paste it into the Client Secret field in the API Connection Manager configuration.
7. Configure API Permissions:
   • Go to the API Permissions section.
   • Click on Add a permission, select Microsoft Graph, and choose Delegated Permissions.
   • Add the required permissions:
     • offline_access
     • openid
     • profile
     • Sites.Read.All
     • Sites.ReadWrite.All
     • User.Read
     • email
     

     Azure AD App – Required API Permissions

   • Grant Admin Consent for the permissions that require it.
8. Generate Tokens:
   • Use the Generate Token feature in the API Connection Manager to generate authentication tokens.
9. Use a Generic Account for Automation:
   NOTE: If you are planning to use your current data connection/token for automated processes, we recommend that you use a generic account for token generation when the login box appears (e.g. sales_automation@mycompany.com instead of bob_smith@mycompany.com). When you use a personal account which is tied to a specific employee profile and that employee leaves the company, the token may become invalid and any automated processes using that token will fail. Another potentially unwanted effect of using a personal token is incorrect logging; the API calls (e.g. Read, Edit, Delete, Upload) made with that token will record the specific user as performing the calls instead of an automated process.
10. That’s it!

Get Bing API Developer Token

Once we have OAuth App registered next thing is to obtain Developer Token for Bing Ads API. Click here to request new token or view existing Developer token.

Create new Bing Ads API Data Source in Gateway

Once you setup Data Gateway user we can move to next step. To call Bing API we need to setup Connection for XML Driver in ZappySys Data Gateway.

1. Click on Add New Data source. Name as BingAds and select Native – ZappySys XML Driver option.
   

   Add Gateway Data Source

2. Now click Edit to Configure Settings. For now we will only care OAuth Connection settings. All other options are overwritten in SQL Query.
   

   Edit Gateway Data Source Settings

3. When UI opens Click on Load ConnectionString and Paste below connection string after changing some items (Screenshot may be differ). Make sure to replace following attributes.RefreshTokenFilePath – Enter some valid file path which will hold refresh token
   ClientId – get Application Id from Azure Portal under App we created earlier)
   DeveloperToken – Replace 1052Bxxxxxxxxxxxx with your own devloper token.
   

   Load ZappySys Driver ConnectionString to configure UI

   Use below string and replace parameters 

   DRIVER={ZappySys XML Driver};
   DataPath='https://clientcenter.api.bingads.microsoft.com/Api/CustomerManagement/v13/CustomerManagementService.svc';
   DataConnectionType=OAuth;
   AuthUrl='https://login.microsoftonline.com/common/oauth2/v2.0/authorize';
   TokenUrl='https://login.microsoftonline.com/common/oauth2/v2.0/token';
   ReturnUrl='https://login.microsoftonline.com/common/oauth2/nativeclient';
   RefreshTokenFilePath='c:\xxxxxx_some_folder_xxxxxxx\BingAds_RefreshToken.txt';
   ScopeSeparator='{space}';
   Scope='https://ads.microsoft.com/ads.manage offline_access';
   ClientId='xxxxxxxxxxxxxx';
   UseCustomApp=True;
   RequestData='<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
     <s:Header>
       <h:ApplicationToken i:nil="true" xmlns:h="https://bingads.microsoft.com/Customer/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
       <h:AuthenticationToken xmlns:h="https://bingads.microsoft.com/Customer/v13"><%access_token%></h:AuthenticationToken>
       <h:DeveloperToken xmlns:h="https://bingads.microsoft.com/Customer/v13">1052Bxxxxxxxxxxxx</h:DeveloperToken>
       <h:Password i:nil="true" xmlns:h="https://bingads.microsoft.com/Customer/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
       <h:UserName i:nil="true" xmlns:h="https://bingads.microsoft.com/Customer/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
     </s:Header>
     <s:Body>
       <GetUserRequest xmlns="https://bingads.microsoft.com/Customer/v13">
         <UserId i:nil="true" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
       </GetUserRequest>
     </s:Body>
   </s:Envelope>';
   RequestContentTypeCode=TextXml;
   RequestMethod='POST';
   RequestHeaders='SOAPAction: "GetUser"'

4. Next to OAuth Click on Configure Settings
5. By default most of settings on OAuth connections should be pre-populated for you but still lets Notice few things. This is how to setup redirect URI
6. 

   Configure Redirect URI for Bing Ads API OAuth Connection

7. Here is where you setup Changing Refresh Token File path. if you dont supply this path then your refresh token extracted first time will expire in 60 days.
   

   Refresh Token File path (Only needed if API issues new Refresh Token when you request AccessToken)

8. Once you verify all settings click Generate Token and Finish the process. It will populate AccessToken and RefreshToken Fields if everything goes well.
   

   Configure Bing Ads API Connection (OAuth 2.0)

9. Click OK to save OAuth Connection UI to go back to main UI
10. You can now Click Test Connection to confirm everything
11. If you need to access Gateway from Other machine then go to Firewall tab and Click Add Rule
12. Click Save button on Gateway UI to save and restart the service.

Create a new Linked Server for Bing Ads API

Once we setup setup Bing Ads Data Source in Gateway we can move to next step which is create a Linked Server which points to Data gateway Data source we created. This will create a bridge between ZappySys Driver and SQL Server so you can call API via T-SQL.

1. If you are using Latest ODBC PowerPack then on Data gateway UI you will See App Integration Tab. Copy Code and run it in SSMS to create a new linked server.
   

   ZappySys Data Gateway code generator for Linked Server

2. If things go well when you execute code you may see some result in grid.

Run sample Queries to call Bing API in SQL Server / Import Data

Once Linked Server is created we can run various queries like below.

Example-1

--use all default settings from gateway data source UI
--Read
Select * from OPENQUERY( [YourLinkedServer] , 'select * from $') 

/*
--import
Select * into #tempTable from OPENQUERY( [YourLinkedServer] , 'select * from $') 
Select * from #tempTable
*/

Example-2

--use Filter to extract specific Node from response XML
Select * from OPENQUERY( [YourLinkedServer] , 
'select * from $
WITH(Filter=''$.s:Envelope.s:Body.GetUserRequest[*]'', 
  ElementsToTreatAsArray=''GetUserResponse'' 
   )'
)

Example-3

--Override URL, Headers and Body Settings. Change DeveloperToken in below request
--Header, Filter, URL, Request data is different for each Endpoint

SELECT * FROM OPENQUERY([YourLinkedServer]
, 'SELECT * FROM $
WITH(
	 ElementsToTreatAsArray=''GetUserResponse''
	,Src=''https://clientcenter.api.bingads.microsoft.com/Api/CustomerManagement/v13/CustomerManagementService.svc''
	,Filter=''$.s:Envelope.s:Body.GetUserResponse[*]''	
	,Header=''SOAPAction: "GetUser"''	
	,RequestData=''<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:v13="https://bingads.microsoft.com/Customer/v13">
<soapenv:Header>
   <v13:DeveloperToken>1052xxxxxxxxxxxxx</v13:DeveloperToken>
   <v13:AuthenticationToken><%access_token%></v13:AuthenticationToken>
</soapenv:Header>
<soapenv:Body>
   <v13:GetUserRequest>
      <v13:UserId xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
   </v13:GetUserRequest>
</soapenv:Body>
</soapenv:Envelope>''
)')

Download Bing Ads Performance Data in SQL Server Table

Now lets look at more complex example to read data from Bing Performance Report. Calling This API is tricky due to many steps involved in data extraction. But not to worry if you use ZappySys Drivers. This is API is somewhat similar as Amazon MWS Report Request (See last section).

Here are high level steps needs to be performed to call API for Bing Ads Report (e.g. Performance Report)

1. Call SubmitGenerateReportResponse API to request new Report. This returns ReportRequestId
2. Now we have to keep checking every few seconds see report is ready or not. Call PollGenerateReport until Status field is Success from Pending
3. Once previous step returns Success extract Report Download URL (Its Zip File which may contain data in CSV or XML)

So lets get started to implement above login in a single T-SQL Query and load data into Table. There is a known limitation with Query Size we can send to data gateway from SSMS so we can use some tricks to read Body from File rather than hard coding in SQL. We can use IsMultiPart=True and then supply Filepath in RequestBody (Path must start with @ symbol)

1. Bing Report API needs 2 additional values along with DeveloperToken we got earlier. You need to supply AccountId and CustomerId part of Body so obtain it by visiting below page in your bing account.
   Visit https://ads.microsoft.com/cc/accounts
   

   Obtain Bing Ads AccountId, ClientId for APi calls

2. Create new XML file for first step in a Notepad. Enter below content and save as C:\Requests\BingAds\sql_body1.xml
   Replace Following Values as per your settingsCustomerAccountId (Found 2 places)
   CustomerId
   DeveloperToken
   

   <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
     <s:Header>
       <h:ApplicationToken i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
       <h:AuthenticationToken xmlns:h="https://bingads.microsoft.com/Reporting/v13"><%access_token%></h:AuthenticationToken>
       <h:CustomerAccountId xmlns:h="https://bingads.microsoft.com/Reporting/v13">481xxxxxxxxxxx</h:CustomerAccountId>
       <h:CustomerId xmlns:h="https://bingads.microsoft.com/Reporting/v13">191xxxxxxxxxxx</h:CustomerId>
       <h:DeveloperToken xmlns:h="https://bingads.microsoft.com/Reporting/v13">1052Bxxxxxxxxxxx</h:DeveloperToken>
       <h:Password i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
       <h:UserName i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
     </s:Header>
     <s:Body>
       <SubmitGenerateReportRequest xmlns="https://bingads.microsoft.com/Reporting/v13">
         <ReportRequest i:type="CampaignPerformanceReportRequest" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
           <ExcludeColumnHeaders>false</ExcludeColumnHeaders>
           <ExcludeReportFooter>false</ExcludeReportFooter>
           <ExcludeReportHeader>false</ExcludeReportHeader>
           <Format>Xml</Format>
           <ReportName>My Campaign Performance Report</ReportName>
           <ReturnOnlyCompleteData>false</ReturnOnlyCompleteData>
           <Aggregation>Hourly</Aggregation>
           <Columns>
             <CampaignPerformanceReportColumn>TimePeriod</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>AccountId</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>CampaignId</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>CampaignName</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>CampaignStatus</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>DeviceType</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>BidMatchType</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>QualityScore</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>AdRelevance</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>LandingPageExperience</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>Revenue</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>Assists</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>ExpectedCtr</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>DeliveredMatchType</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>AveragePosition</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>Conversions</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>AdDistribution</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>Network</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>Clicks</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>Impressions</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>Ctr</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>AverageCpc</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>Spend</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>LowQualityClicks</CampaignPerformanceReportColumn>
             <CampaignPerformanceReportColumn>LowQualityConversionRate</CampaignPerformanceReportColumn>
           </Columns>
           <Filter>
             <AccountStatus i:nil="true" />
             <AdDistribution i:nil="true" />
             <DeviceOS i:nil="true" />
             <DeviceType i:nil="true" />
             <Status i:nil="true" />
           </Filter>
           <Scope>
             <AccountIds xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
               <a:long>481xxxxxxxxxxx</a:long>
             </AccountIds>
             <Campaigns i:nil="true" />
           </Scope>
           <Time>
             <CustomDateRangeEnd i:nil="true" />
             <CustomDateRangeStart i:nil="true" />
             <PredefinedTime>Yesterday</PredefinedTime>
             <ReportTimeZone>PacificTimeUSCanadaTijuana</ReportTimeZone>
           </Time>
         </ReportRequest>
       </SubmitGenerateReportRequest>
     </s:Body>
   </s:Envelope>

    
3. Create another XML file for second step in a Notepad. Enter below content and save as C:\Requests\BingAds\sql_body2.xml
   Replace following values as per your need.
   

   <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
     <s:Header>
       <h:ApplicationToken i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
       <h:AuthenticationToken xmlns:h="https://bingads.microsoft.com/Reporting/v13"><%access_token%></h:AuthenticationToken>
       <h:CustomerAccountId xmlns:h="https://bingads.microsoft.com/Reporting/v13">480xxxxxxxxxxx</h:CustomerAccountId>
       <h:CustomerId xmlns:h="https://bingads.microsoft.com/Reporting/v13">191xxxxxxxxxxx</h:CustomerId>
       <h:DeveloperToken xmlns:h="https://bingads.microsoft.com/Reporting/v13">1052Bxxxxxxxxxxx</h:DeveloperToken>
       <h:Password i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
       <h:UserName i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
     </s:Header>
     <s:Body>
       <PollGenerateReportRequest xmlns="https://bingads.microsoft.com/Reporting/v13">
         <ReportRequestId>[$a.ReportRequestId$]</ReportRequestId>
       </PollGenerateReportRequest>
     </s:Body>
   </s:Envelope>

    
4. Now try to execute below query. This should generate Performance report and download it in a SQL table for you.
   

   select * into tmpBingAdsReport from OPENQUERY(Zs_BingAds,
   'SELECT c.* FROM $
   WITH(
   	--submit report request for XML format
   
   	 alias=''a'' 
   	,Src=''https://reporting.api.bingads.microsoft.com/Api/Advertiser/Reporting/v13/ReportingService.svc''
   	,Filter=''$.s:Envelope.s:Body.SubmitGenerateReportResponse''
   	,RequestData=''@C:\Requests\BingAds\sql_body1.xml''
   	,IsMultiPart=''True''
   	,RequestContentTypeCode=''TextXml''
   	,Header=''SOAPAction: "SubmitGenerateReport"''
   	,RequestMethod=''POST''
   	,EnableBodyPlaceholderForMultiPart=''True''
   	,Meta=''[{"Name": "ReportRequestId","Type": "Int64"}]''
   
   	--keep checking status until report is ready - returns final url when done
   
   	,join1_alias=''b''
   	,join1_Filter=''$.s:Envelope.s:Body.PollGenerateReportResponse.ReportRequestStatus''
   	,join1_RequestData=''@C:\Requests\BingAds\sql_body2.xml''
   	,join1_IsMultiPart=''True''
   	,join1_Src=''https://reporting.api.bingads.microsoft.com/Api/Advertiser/Reporting/v13/ReportingService.svc''
   	,join1_RequestContentTypeCode=''TextXml''
   	,join1_Header=''SOAPAction: "PollGenerateReport"''
   	,join1_RequestMethod=''POST''
   	,join1_EnableBodyPlaceholderForMultiPart=''True''
   	,join1_EnableStatusCheck=''True'', 
   	,join1_StatucCheckMaxWaitSeconds=300, --wait max 5 mins?
   	,join1_StatucCheckIterationWaitSeconds=5, --check every 5 sec?
   	,join1_StatusSuccessValue=''Success'', --look for success word in response
   
   	--Download report (Zip file) and Parse rows
   
   	,join2_alias=''c''
   	,join2_DataConnectionType=''Default''
   	,join2_Src=''[$b.ReportDownloadUrl$]''
   	,join2_Filter=''$.Report.Table.Row[*]''
   	,join2_ElementsToTreatAsArray=''Row''	
   	,join2_RequestMethod=''GET''
   	,join2_IsMultiPart=''False''
   	,join2_FileCompressionType=''Zip''	
   )')
   
   select * from tmpBingAdsReport

   

   Import Bing Ads data into SQL Server

Multi Step Query to download Report

So in previous section we saw how to write a single query to get your Performance Report from Bing Ads. However there will be time you want to split 3 steps into seperate queries.

Here is how to do.

/*
RPC and RPC OUT options must be turned on to run this proc

EXEC master.dbo.sp_serveroption @server=N'Zs_BingAds', @optname=N'rpc', @optvalue=N'true'
EXEC master.dbo.sp_serveroption @server=N'Zs_BingAds', @optname=N'rpc out', @optvalue=N'true'
--Below needed to support EXEC + INSERT (dynamic query)
EXEC master.dbo.sp_serveroption @server=N'Zs_BingAds, @optname=N'remote proc transaction promotion', @optvalue=N'false'
Also 
1. Change Body file path
2. Change CustomerAccountId, CustomerId, DeveloperToken
*/

ALTER proc usp_Api_BingAds_Reports
as
--///////////////////////////////////////////////////////////////////////////////
--Step-1 - submit report request for XML format
--///////////////////////////////////////////////////////////////////////////////
create table #tmpBingAdsReport_Step1(ReportRequestId bigint)
INSERT into #tmpBingAdsReport_Step1
EXEC(
'SELECT * FROM $
WITH(
 Src=''https://reporting.api.bingads.microsoft.com/Api/Advertiser/Reporting/v13/ReportingService.svc''
,Filter=''$.s:Envelope.s:Body.SubmitGenerateReportResponse''
,RequestData=''@c:\BWProjects\Requests\BingAds\sql_body1.xml''
,IsMultiPart=''True''
,RequestContentTypeCode=''TextXml''
,Header=''SOAPAction: "SubmitGenerateReport"''
,RequestMethod=''POST''
,EnableBodyPlaceholderForMultiPart=''True''
,Meta=''[{"Name": "ReportRequestId","Type": "Int64"}]''
)') AT zs_BingAds


--///////////////////////////////////////////////////////////////////////////////
--Step2 - keep checking status until report is ready - returns final url when done
--///////////////////////////////////////////////////////////////////////////////

declare @reportid varchar(100)
select top 1 @reportid=cast(ReportRequestId as varchar(100)) from #tmpBingAdsReport_Step1

create table #tmpBingAdsReport_Step2(ReportDownloadUrl varchar(500))
INSERT into #tmpBingAdsReport_Step2 
EXEC(
'SELECT * FROM $
WITH(
 Filter=''$.s:Envelope.s:Body.PollGenerateReportResponse.ReportRequestStatus''
,RequestData=''<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Header>
    <h:ApplicationToken i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
    <h:AuthenticationToken xmlns:h="https://bingads.microsoft.com/Reporting/v13"><%access_token%></h:AuthenticationToken>
    <h:CustomerAccountId xmlns:h="https://bingads.microsoft.com/Reporting/v13">480xxxxxxxxx</h:CustomerAccountId>
    <h:CustomerId xmlns:h="https://bingads.microsoft.com/Reporting/v13">191xxxxxxxxx</h:CustomerId>
    <h:DeveloperToken xmlns:h="https://bingads.microsoft.com/Reporting/v13">105xxxxxxxxxxx</h:DeveloperToken>
    <h:Password i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
    <h:UserName i:nil="true" xmlns:h="https://bingads.microsoft.com/Reporting/v13" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" />
  </s:Header>
  <s:Body>
    <PollGenerateReportRequest xmlns="https://bingads.microsoft.com/Reporting/v13">
      <ReportRequestId>'+ @reportid +'</ReportRequestId>
    </PollGenerateReportRequest>
  </s:Body>
</s:Envelope>''
--,IsMultiPart=''True''
,Src=''https://reporting.api.bingads.microsoft.com/Api/Advertiser/Reporting/v13/ReportingService.svc''
,RequestContentTypeCode=''TextXml''
,Header=''SOAPAction: "PollGenerateReport"''
,RequestMethod=''POST''
--,EnableBodyPlaceholderForMultiPart=''True''
,EnableStatusCheck=''True'', 
,StatucCheckMaxWaitSeconds=300, --wait max 5 mins?
,StatucCheckIterationWaitSeconds=5, --check every 5 sec?
,StatusSuccessValue=''Success'', --look for success word in response
,Meta=''[{"Name": "ReportDownloadUrl","Type": "String",Length:"300"}]''
)') AT zs_BingAds


declare @reportpath varchar(500)
select top 1 @reportpath=ReportDownloadUrl from #tmpBingAdsReport_Step2

select * from #tmpBingAdsReport_Step1
select * from #tmpBingAdsReport_Step2

--///////////////////////////////////////////////////////////////////////////////
--Step-3 - Get report file
--///////////////////////////////////////////////////////////////////////////////
--Static table
--//INSERT INTO BingAdsReport
EXEC(
'SELECT * FROM $
WITH(
 DataConnectionType=''Default''
,Src='''+ @reportpath +'''
,Filter=''$.Report.Table.Row[*]''
,ElementsToTreatAsArray=''Row''	
,RequestMethod=''GET''
,IsMultiPart=''False''
,FileCompressionType=''Zip''	
)') AT zs_BingAds


--select * from tmpBingAdsReport_Step3
--select * from openquery(Bing, 'select * from $')


/*
select * into tmpBingAdsReport from OPENQUERY(zs_BingAds,
'SELECT c.* FROM $
WITH(
--submit report request for XML format

alias=''a'' 
,Src=''https://reporting.api.bingads.microsoft.com/Api/Advertiser/Reporting/v13/ReportingService.svc''
,Filter=''$.s:Envelope.s:Body.SubmitGenerateReportResponse''
,RequestData=''@c:\BWProjects\Requests\BingAds\sql_body1.xml''
,IsMultiPart=''True''
,RequestContentTypeCode=''TextXml''
,Header=''SOAPAction: "SubmitGenerateReport"''
,RequestMethod=''POST''
,EnableBodyPlaceholderForMultiPart=''True''
,Meta=''[{"Name": "ReportRequestId","Type": "Int64"}]''

--keep checking status until report is ready - returns final url when done

,join1_alias=''b''
,join1_Filter=''$.s:Envelope.s:Body.PollGenerateReportResponse.ReportRequestStatus''
,join1_RequestData=''@c:\BWProjects\Requests\BingAds\sql_body2.xml''
,join1_IsMultiPart=''True''
,join1_Src=''https://reporting.api.bingads.microsoft.com/Api/Advertiser/Reporting/v13/ReportingService.svc''
,join1_RequestContentTypeCode=''TextXml''
,join1_Header=''SOAPAction: "PollGenerateReport"''
,join1_RequestMethod=''POST''
,join1_EnableBodyPlaceholderForMultiPart=''True''
,join1_EnableStatusCheck=''True'', 
,join1_StatucCheckMaxWaitSeconds=300, --wait max 5 mins?
,join1_StatucCheckIterationWaitSeconds=5, --check every 5 sec?
,join1_StatusSuccessValue=''Success'', --look for success word in response

--Download report (Zip file) and Parse rows

,join2_alias=''c''
,join2_DataConnectionType=''Default''
,join2_Src=''[$b.ReportDownloadUrl$]''
,join2_Filter=''$.Report.Table.Row[*]''
,join2_ElementsToTreatAsArray=''Row''	
,join2_RequestMethod=''GET''
,join2_IsMultiPart=''False''
,join2_FileCompressionType=''Zip''	
)')

select * from tmpBingAdsReport

select * from openquery(Bing, 'select * from $')
*/
go

Conclusion

In this article we saw how to import Bing Ads API data in SQL Server without any coding using ZappySys ODBC PowerPack (SOAP / XML Driver). Download it it and try it for FREE to explore many API integration features not discussed in this article.

The post Import Bing Ads data into SQL Server (Performance Reports) appeared first on ZappySys Blog.

In our previous blog post we saw how to import rest API in SQL Server. Using same concepts lets look at how to import Amazon MWS Data into SQL Server. We will explore many techniques to call Amazon MWS API and learn how to automate Amazon MWS data extraction without doing any ETL. We will call MWS XML or CSV API just using T-SQL code. We will also learn how to Download Reports from by calling MWS API.

You can also refer other articles here (Power BI) and here (SSIS)

About Amazon MWS  API

If you want to sell something on Amazon you can use their e-commerce platform with some monthly fee. You can setup your entire online store / inventory using admin interface.

Amazon Marketplace Web Service (Amazon MWS) is an integrated web service API that helps Amazon sellers to programmatically exchange data on listings, orders, payments, reports, and more. Using these API you can read or write data from your Seller account and integrate it inside your own Systems (e.g. Reporting / ETL / BI tools).

Obtain MWS API Access Key / Secret and Seller ID

Very first thing to call any MWS API is to obtain Access Key, Secret Key and Seller ID (i.e. Merchant ID). To obtain this you first have to register a developer.

Here is where to look for your Merchant ID and Key / Secret.  Here is direct link to User Permissions page.

How to get Amazon MWS developer keys and know your Merchant ID (i.e. Seller ID)

Getting Started

Now let’s look at step by step instructions on how to call Amazon MWS API and then import into Power BI. In below steps we will use  ZappySys ODBC PowerPack so make sure it’s installed first.

Using ScratchPad to Test Amazon MWS API

Each MWS API has set of required and optional parameters. You can refer each API page for details. For example ListMatchingProducts API which we will use as an example. To make MWS API testing / learning easy, Amazon provides online testing tool called MWS ScratchPad. We will use this tool to craft API requests and use that information in ODBC driver SQL queries and later import data in Power BI.

1. Open MWS ScratchPad  by vising https://mws.amazonservices.com/scratchpad/index.html
2. Select Products API category and pick ListMatchingProducts API
3. Enter your SellerID (i.e. Merchant ID), Developer AccessKey (i.e. AWSAccessKeyId) and Secret Key we obtained in the previous section. Enter MarketId (e.g. ATVPDKIKX0DER for USA Market) and other parameters as below and Click Submit to see response.
   

   Using Amazon MWS ScratchPad (API Testing Tool – Created by Amazon)

4. Click on the Response Details Tab to extract some important information we will use for ODBC configuration. Notice attributes in red rectangles. They are the ones which we will need in the POST request Body. ZappySys Drivers supply many common parameters automatically but any API endpoint specific parameter must be supplied in the Request Body. For example you dont have to supply SignatureVersion, Timestamp, Signature or SignatureMethod
   

   Calling Amazon MWS API in Scratchpad Testing Tool (Response Details Tab) – ListMatchingProducts Example

This API requires Market ID parameter so enter it as per your Market where you selling. For example USA marketplaceid = ATVPDKIKX0DER. You can lookup correct MarketplaceId by checking this help (endpoints / marketplaces).

Setup Amazon MWS API Connections

Create XML Driver Connection for Amazon MWS API (e.g. RequestReport )

Some APIs we need to call in this article is XML format and some API (e.g. GetReport) is CSV format – Tab delimited so we need to create 2 different connections using CSV driver and XML driver. So lets get started with Amazon MWS Connection for XML API.

1. Click on Add New Data source. Name as AMAZON-MWS-XML and select Native – ZappySys XML Driver option.
   

   Add Gateway Data Source

2. Now click Edit to Configure Settings. For now we will only care OAuth Connection settings. All other options are overwritten in SQL Query.
   

   Edit Gateway Data Source Settings

3. When UI opens for Driver Enter MWS API Service URL as below. Notice that we added /Products/2011-10-01 (This was extracted from previous section). You can check full list of Amazon MWS Endpoints here. Because we are connecting to USA region we will use below URL.
   

   https://mws.amazonservices.com/Products/2011-10-01

4. Select OAuth from the Connection dropdown and Click Configure Settings
5. On the Connection UI, Select OAuth Provider Amazon Market Web Service (MWS)
6. Enter your API Access Key and Secret Key (obtained from here)
7. Your connection will look like below.
   

   ODBC Connection Credentials for Amazon MWS API (Access Key, Secret Key, URL Endpoint)

8. Also configure Retry options to handle API limit reached error when you call API too often. For Amazon MWS we get Status code 503 when request is throttled so lets narrow down on which error we want to retry.
   

   Retry Options

9. Click OK to save OAuth Connection UI to go back to main UI
10. Now Go to Throttling Tab and enter some delay (e.g. enter 20000 for 20sec delay after each request). This will slow down API calls so we don’t face errors about API Limit reached.
11. Click OK to Save UI
12. Click Save button on Gateway UI to save and restart

Create CSV Driver Connection for Amazon MWS API (e.g. GetReport )

Now lets create a connection CSV API (e.g. GetReport ).

Follow almost same steps as previous section except use CSV Driver when you click on Add new Data source select Native – ZappySys CSV Driver and name data source as AMAZON-MWS-CSV . As we mentioned for now we mostly want to setup Connection and not worry about Request Body, Pagination etc.

Create Linked Server for SQL Server

Run following Script in SSMS to create Linked servers. Change ######## with Your gateway password. Change localhost with machine name or IP where ZappySys Gateway Is running. Use localhost if SQL Server and Gateway both on the same machine.

USE [master]
GO

EXEC master.dbo.sp_addlinkedserver @server = N'AMAZON-MWS-XML', @srvproduct=N'', @provider=N'SQLNCLI', @datasrc=N'localhost,5000', @catalog=N'AMAZON-MWS-XML'
EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'AMAZON-MWS-XML',@useself=N'False',@locallogin=NULL,@rmtuser=N'tdsuser',@rmtpassword='########'
GO

EXEC master.dbo.sp_addlinkedserver @server = N'AMAZON-MWS-CSV', @srvproduct=N'', @provider=N'SQLNCLI', @datasrc=N'localhost,5000', @catalog=N'AMAZON-MWS-CSV'
EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'AMAZON-MWS-CSV',@useself=N'False',@locallogin=NULL,@rmtuser=N'tdsuser',@rmtpassword='########'
GO

If you want to know how to configure Linked Server using UI then check this article.

Amazon MWS ListOrders Example

Now lets look at how to call ListOrders request with pagination in SQL Server. Create a following Stored Procedure to List MWS Orders. If you want to extract many orders then call Report rather than this proc (See next section for generating reports)

USE MyDatabase
GO
/*
Example: 
 exec usp_MWS_ListOrders @SellerId='Axxxxxxx', @MarketplaceId ='Bxxxxxxx',@CreatedAfter='2019-01-01'  
*/
Create proc [dbo].[usp_MWS_ListOrders]
	@SellerId varchar(100),
	@MarketplaceId varchar(100),
	@CreatedAfter varchar(100),
	@CreatedBefore varchar(100)=null,
	@MaxRows int=0
as
declare @sql varchar(max) 
declare @limitClase varchar(100)=''
if(@MaxRows>0)
	set @limitClase	=' LIMIT ' + cast(@MaxRows as varchar(20))

--//Confirm ISO Date format 2019-06-01T04%3A00%3A00Z  rather 2019-06-01T04:00:00Z
if(isnull(@CreatedAfter,'')<>'')
begin
	set @CreatedAfter='&CreatedAfter=' + @CreatedAfter
	set @CreatedAfter=replace(@CreatedAfter,':','%3A')
	if(@CreatedAfter NOT LIKE @CreatedAfter + 'T%')
		set @CreatedAfter=@CreatedAfter + 'T00%3A00%3A00Z'
	if(@CreatedAfter NOT LIKE @CreatedAfter + '%Z')
		set @CreatedAfter=@CreatedAfter + 'Z'
	--//T04%3A00%3A00Z
end
if(isnull(@CreatedBefore,'')<>'')
begin
	set @CreatedBefore='&CreatedBefore=' + @CreatedBefore
	set @CreatedBefore=replace(@CreatedBefore,':','%3A')
	if(@CreatedBefore NOT LIKE @CreatedBefore + 'T%')
		set @CreatedBefore=@CreatedBefore + 'T00%3A00%3A00Z'
	if(@CreatedBefore NOT LIKE @CreatedBefore + '%Z')
		set @CreatedBefore=@CreatedBefore + 'Z'
end

set @CreatedAfter = isnull(@CreatedAfter,'')
set @CreatedBefore = isnull(@CreatedBefore,'')


set @sql = 'SELECT * FROM OPENQUERY([AMAZON-MWS-XML]
, ''
SELECT ReportRequestId FROM $
' + @limitClase +' 
WITH(
     Src=''''https://mws.amazonservices.com/Orders/2013-09-01''''
	,Filter=''''$.RequestReportResponse.RequestReportResult.ReportRequestInfo''''
	,ElementsToTreatAsArray=''''Order''''
	,RequestData=''''Action=ListOrders[$tag$]' + @CreatedAfter + @CreatedBefore + '&MarketplaceId.Id.1='+ @MarketplaceId +'&SellerId='+ @SellerId +'''''
	,RequestMethod=''''POST''''
	,NextUrlAttributeOrExpr=''''$.ListOrders[$tag$]Response.ListOrders[$tag$]Result.NextToken''''
	,NextUrlSuffix=''''&NextToken=<%nextlink_encoded%>''''
	,WaitTimeMs=20000 -- slow down to avoid throttling. 6 requests per min
	,HasDifferentNextPageInfo=''''True''''
	,EnablePageTokenForBody=''''True''''
	,PagePlaceholders=''''body=|ByNextToken;filter=|ByNextToken''''
	--,DataConnectionType=''''OAuth''''
	--,ScopeSeparator=''''{space}''''
	--,ServiceProvider=''''AmazonMWS''''
	--,ClientId=''''AKIxxxxxx''''
	--,ClientSecret=''''AKIxxxxxx''''
	--,UseCustomApp=''''True''''
	,CacheFileLocation=''''c:\temp\mws-orders.cache''''
	,CacheEntryTtl=300 --send every 5 mins else use from cache
	,IncludeParentColumns=''''False''''
)
'')'
print @sql

EXECUTE( @sql)

Here is how to call this proc. Change SellerId, MarketplaceId and Date.

exec usp_MWS_ListOrders @SellerId='Axxxxxxx', @MarketplaceId ='Bxxxxxxx',@CreatedAfter='2019-01-01'

Download Amazon MWS Report data in SQL Server (CSV / XML Format)

Now lets look at how to extract data from Custom Report API (You must need ODBC PowerPack v1.1.1 or higher). However reading data from Reports not single step process because report generation is Job style API. Which means you send report request and wait  until its done. Once Report ready you have to read in CSV format or XML. Some Reports only available in CSV format. Check this post to understand how complex it can be to get data. We will make it simple for you to understand this in 3 steps. Basically calling reports requires minimum 3 API calls (see below)

1. Create a new report request – Cal RequestReport API . It returns ReportRequestId (You can use it in the next step)
2. Check Report Status see its done – Call GetReportRequestList API (Pass ReportRequestId got in the previous step to get data for only one Report request we care). Keep checking Report Status every few seconds until you get _DONE_  or _DONE_NO_DATA_ status in response.
3. Read Report Data – Call GetReport. This API call returns CSV or XML data. So use correct Driver for this step. If CSV data is returned then we must use CSV Driver rather than XML Driver.

Method 1 – Single query approach (New Driver)

In new driver ZappySys added few new features which makes it possible to call above 3 steps process in a single query. New query syntax offers StatusCheck and Mixing Content Formats using same driver. Because Most MWS Reports are in CSV Format we have to use CSV Driver but oddly first 2 steps listed in above sections are in XML Format so we have to use XML driver. Technically we need atleast 2 steps (i.e 2 queries) but we will show you how to achieve using single query as below.

1. Assuming you have followed steps to create a linked server AMAZON-MWS-CSV (see previous section)
2. Run below query. Replace necessary parameters as per your need. (i.e MarketplaceId , SellerId, CreatedAfter date)
   

   --- //////////////////////////////////////////////////////////////////////////
   -- Change _GET_MERCHANT_LISTINGS_DATA_ to something else if you need different report type (all 3 steps) -- https://docs.developer.amazonservices.com/en_US/reports/Reports_ReportType.html
   -- Replace MarketplaceId in all 3 steps (i.e for USA use ATVPDKIKX0DER) --https://docs.developer.amazonservices.com/en_US/dev_guide/DG_Endpoints.html
   -- Replace SellerId in all 3 steps
   -- Remove or Change CreatedAfter in first step (or use CreatedBefore parameter instread of CreatedAfter)
   --- //////////////////////////////////////////////////////////////////////////
   select * into tmpMWS_Report from OPENQUERY([AMAZON-MWS-CSV],
   '
   SELECT s3.* FROM $
   WITH(
   Src=''https://mws.amazonservices.com/Reports/2009-01-01'',RequestMethod=''POST''
   ,RequestData=''Action=RequestReport&ReportType=_GET_MERCHANT_LISTINGS_DATA_&MarketplaceId=ATVPDKIKX0DER&SellerId=Axxxxxxxxxx&CreatedAfter=2017-12-31T04%3A00%3A00Z''
   ,IncludeParentColumns=''False''
   ,EnableRawOutputModeSingleRow=''True''
   ,RawOutputFilterExpr=''//*[local-name()="ReportRequestId"]'' 
   ,RawOutputDataRowTemplate=''{ReportRequestId:"[$1]"}'' 
   ,RawOutputExtractMode=''Xml'' 
   ,WaitTimeMs=5000
   ,Meta=''[{"Name": "ReportRequestId","Type": "Int64"}]''
   ,Alias=''s1''
   
   ,Join1_alias=''s2''	
   ,Join1_Src=''https://mws.amazonservices.com/Reports/2009-01-01'',Join1_RequestMethod=''POST''
   ,Join1_RequestData=''Action=GetReportRequestList&ReportRequestIdList.Id.1=[$s1.ReportRequestId$]&MarketplaceId=ATVPDKIKX0DER&SellerId=Axxxxxxxxxxxxx''
   ,Join1_EnableRawOutputModeSingleRow=''True''
   ,Join1_RawOutputFilterExpr=''//*[local-name()="ReportProcessingStatus"]||//*[local-name()="GeneratedReportId"]'' 
   ,Join1_RawOutputDataRowTemplate=''{ReportProcessingStatus:"[$1]",GeneratedReportId:"[$2]"}'' 
   ,Join1_RawOutputExtractMode=''Xml'' 
   ,Join1_EnableStatusCheck=''True'', 
   ,Join1_StatucCheckMaxWaitSeconds=300,
   ,Join1_StatucCheckIterationWaitSeconds=25,
   ,Join1_StatusSuccessValue=''_DONE_''
   ,Join1_StatusFailedValue=''_DONE_NO_DATA_|_CANCELLED_''
   
   ,Join2_Alias=''s3''
   ,Join2_Src=''https://mws.amazonservices.com/Reports/2009-01-01'',Join2_RequestMethod=''POST''
   ,Join2_RequestData=''Action=GetReport&ReportId=[$s2.GeneratedReportId$]&MarketplaceId=ATVPDKIKX0DER&SellerId=Axxxxxxxxxxxxxx''	
   ,Join2_ColumnDelimiter=''{TAB}''
   ,Join2_ResponseCharset=''Windows-1252''	 
   ,Join2_EnableStatusCheck=''False'' 
   ,Join2_EnableRawOutputModeSingleRow=''False''
   ,Join2_WaitTimeMs=10000
   )') 
   go
   select * from tmpMWS_Report

    

Performance Tip

If you like to improve performance and dont want to use SELECT INTO approach to load into temp table as above then you can use EXEC (….) AT YourLinkedServerName  as below.

Advantage of this method is your Query speed will increase because system calls API only once when you call EXEC AT. In OPENROWSET it needs to call above query twice (Once to obtain metadata and once to get data).

INSERT INTO tmpMWS_Report
EXEC('select s3.* .................. ') AT [AMAZON-MWS-CSV]

Method 2 – Multiple query approach (For old driver)

Now lets see how to load Amazon MWS data in SQL Server using old driver (v1.1.2 or older) which didnt support single query approach like previous section. If you are using older driver or for some reason you like to split steps into multiple procedures then use below approach.

usp_MWS_Report  Stored Proc ( Generate ReportRequest, Wait until done, Read data )

USE MyDatabase
GO

/*
Examples: 
Get ReportType from here https://docs.developer.amazonservices.com/en_UK/reports/Reports_ReportType.html

exec usp_MWS_Report @MarketplaceId ='ATVPDKIKX0DER', @SellerId='AZxxxxxxxx', @ReportType = '_GET_MERCHANT_LISTINGS_DATA_'
exec usp_MWS_Report @MarketplaceId ='ATVPDKIKX0DER', @SellerId='AZxxxxxxxx', @ReportType = '_GET_FLAT_FILE_ORDER_REPORT_DATA_'

--Do not Throw Error if no data found (return empty resultset or run custom sql)
exec usp_MWS_Report @MarketplaceId ='ATVPDKIKX0DER', @SellerId='AZxxxxxxxx', @ReportType = '_GET_FLAT_FILE_ORDER_REPORT_DATA_',@ThrowErrIfNoData=0,@EmptyRowSql='select top 0 * from Orders'
--Supply date range
exec usp_MWS_Report @MarketplaceId ='ATVPDKIKX0DER', @SellerId='AZxxxxxxxx', @ReportType = '_GET_MERCHANT_LISTINGS_DATA_', @StartDate='2019-01-01T00:00:00Z', @EndDate='2019-01-31T00:00:00Z'

--loading into table 

--//////////////////////////////////
— Step-1 : Create Local Linked server to point to self
--//////////////////////////////////
USE [master]
GO
--change default catalog
--For MSSQL 2012, 2014, 2016, 2017, and 2019 use @provider=N'MSOLEDBSQL' below (SQL Server Native Client 11.0)---
EXEC master.dbo.sp_addlinkedserver @server = N'ME', @srvproduct=N'', @provider=N'SQLNCLI11', @datasrc=N'localhost', @catalog=N'Northwind'
--For MSSQL 2022 or higher use @provider=N'MSOLEDBSQL' below (Microsoft OLE DB Driver for SQL Server)---
--EXEC master.dbo.sp_addlinkedserver @server = N'ME', @srvproduct=N'', @provider=N'MSOLEDBSQL', @datasrc=N'localhost', @catalog=N'Northwind'

EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'ME',@useself=N'True',@locallogin=NULL,@rmtuser=NULL,@rmtpassword=NULL
GO
--//////////////////////////////////
— Step-2 : Enable RPC OUT and Disable MSDTC
--//////////////////////////////////
EXEC sp_serveroption 'ME', 'rpc out', true;  
GO
--Below needed to support EXEC + INSERT (dynamic query)
EXEC sp_serveroption 'ME', 'remote proc transaction promotion', false;
GO
--test simple query
select * from OPENQUERY(ME,'select 1')
GO

--//////////////////////////////////
— Step-3 : Load data into table from MWS Report
--//////////////////////////////////
SELECT * INTO #amazon_mws_report
FROM OPENQUERY(me,'
SET NOCOUNT ON;
EXEC usp_MWS_Report 
   @Columns=''[item-name], [item-description], [listing-id]'',
   @MarketplaceId =''ATVPDKIKX0DER'', 
   @SellerId=''Axxxxxxxxxxxxxxxxx'', 
   @ReportType = ''_GET_MERCHANT_LISTINGS_DATA_''
WITH RESULT SETS 
(
	(itmname varchar(1000),itmdesc varchar(1000),itmid varchar(1000))
)
')

select * from #amazon_mws_report


*/
CREATE proc [dbo].[usp_MWS_Report]
	@MarketplaceId varchar(100),
	@SellerId varchar(100),
	@ReportType varchar(100)='_GET_MERCHANT_LISTINGS_DATA_',
	@StartDate varchar(100)=null,
	@EndDate varchar(100)=null,
	@ReportId bigint=null,  ---i.e. 14783134522018025
	@ThrowErrIfNoData bit=1,
	@EmptyRowSql varchar(1000)=null, --SQL to run if no data found
	@Columns varchar(8000)='*'  --report columns which goes in final SELECT  e.g. [item-name], [item-description], [listing-id] ....
as


declare @retGeneratedReportId bigint
set @retGeneratedReportId=@ReportId
---////// STEP 1 /////////
if(@ReportId is null)
begin
	create table #step1(ReportRequestId bigint)
	insert into #step1
	exec usp_MWS_RequestReport @ReportType,@MarketplaceId,@SellerId,@StartDate,@EndDate

	--select * from #step1 --debug

	declare @retReportRequestId bigint
	select @retReportRequestId=ReportRequestId from #step1

	---////// STEP 2 /////////
	declare @status varchar(100)
	declare @cnt int
	create table #step2(ReportProcessingStatus varchar(100),GeneratedReportId bigint)
	
	--Keep looping until status is done or we waited too long
	while(isnull(@status,'') IN ('','_SUBMITTED_','_IN_PROGRESS_') )
	begin
		truncate table #step2
		insert into #step2
		exec usp_MWS_GetReportIdAndStatus @retReportRequestId,@MarketplaceId,@SellerId 
	
		select @status=ReportProcessingStatus, @retGeneratedReportId=GeneratedReportId from #step2
		if(isnull(@status,'') IN ('','_SUBMITTED_','_IN_PROGRESS_') )			
		begin
			--amazon says dont check often https://docs.developer.amazonservices.com/en_UK/reports/Reports_GetReportRequestList.html
			Print 'Status='+  isnull(@status,'') +'. Checking after 45 sec... #' + cast(@cnt as varchar(10))
			WAITFOR DELAY '00:00:45'			
		end
		else
			print @status

		set @cnt=@cnt+1

		--if(@cnt>50)
			--//throw err?
	end
	--select * from #step2 --debug
end

if(@retGeneratedReportId is null and @status='_DONE_NO_DATA_')
begin
	if(@ThrowErrIfNoData=1)
		RAISERROR('Cannot generate report - No data found for specified criteria',16,1); 
	else if (isnull(@EmptyRowSql,'')<>'')
		execute(@EmptyRowSql)
end
else if(@retGeneratedReportId is null)
	RAISERROR('Cannot generate report - Unknown error (either report got cancelled or other error occurred)',16,1); 

	
---////// STEP 3 /////////
--If xml format of report data then use XML driver else CSV (most are Tab delimited reports in CSV format)
--exec usp_MWS_GetReportDataXml @retGeneratedReportId,@MarketplaceId,@SellerId
exec usp_MWS_GetReportDataCsv @retGeneratedReportId,@MarketplaceId,@SellerId,@Columns

drop table #step1
drop table #step2

GO

usp_MWS_RequestReport  Stored Proc ( Step1 – Create Report Request )

USE [MyDatabase]
GO

CREATE proc [dbo].[usp_MWS_RequestReport]
	@ReportType varchar(100),
	@MarketplaceId varchar(100),
	@SellerId varchar(100),
	@StartDate varchar(100)=null,
	@EndDate varchar(100)=null
as

/*Learn more : https://docs.developer.amazonservices.com/en_UK/reports/Reports_RequestReport.html */

--//Confirm ISO Date format 2019-06-01T04%3A00%3A00Z  rather 2019-06-01T04:00:00Z
if(isnull(@StartDate,'')<>'')
begin
	set @StartDate='&StartDate=' + @StartDate
	set @StartDate=replace(@StartDate,':','%3A')
	if(@StartDate NOT LIKE @StartDate + 'T%')
		set @StartDate=@StartDate + 'T00%3A00%3A00Z'
	if(@StartDate NOT LIKE @StartDate + '%Z')
		set @StartDate=@StartDate + 'Z'
	--//T04%3A00%3A00Z
end
if(isnull(@EndDate,'')<>'')
begin
	set @EndDate='&EndDate=' + @EndDate
	set @EndDate=replace(@EndDate,':','%3A')
	if(@EndDate NOT LIKE @EndDate + 'T%')
		set @EndDate=@EndDate + 'T00%3A00%3A00Z'
	if(@EndDate NOT LIKE @EndDate + '%Z')
		set @EndDate=@EndDate + 'Z'
end

set @StartDate = isnull(@StartDate,'')
set @EndDate = isnull(@EndDate,'')

declare @sql varchar(max) 
set @sql = 'SELECT * FROM OPENQUERY([AMAZON-MWS-XML]
, ''
SELECT ReportRequestId FROM $
WITH(
	Src=''''https://mws.amazonservices.com/Reports/2009-01-01'''',RequestMethod=''''POST''''
	--,ElementsToTreatAsArray=''''Product''''
	,Filter=''''$.RequestReportResponse.RequestReportResult.ReportRequestInfo''''
	,RequestData=''''Action=RequestReport&ReportType=' + @ReportType  + '&MarketplaceId='+ @MarketplaceId +'&SellerId='+ @SellerId + @StartDate + @EndDate +'''''
	,CacheFileLocation=''''c:\temp\mws-cache-step-1.cache''''
	,CacheEntryTtl=300 --send every 5 mins else use from cache
	,WaitTimeMs=''''300''''
	,IncludeParentColumns=''''False''''
	,Meta=''''[{"Name": "ReportRequestId","Type": "Int64"}]''''
)'')'
--print @sql

EXECUTE( @sql)
--sleep 25 seconds after this - hope report is ready by that time else We have to loop
WAITFOR DELAY '00:00:25'  --lets wait 30 sec... incase we get luck and report is done?
GO

usp_MWS_GetReportIdAndStatus Stored Proc ( Step2 – Get Report Id and Report Status )

USE MyDatabase
GO


CREATE proc [dbo].[usp_MWS_GetReportIdAndStatus]
	@ReportRequestId bigint,
	@MarketplaceId varchar(100),
	@SellerId varchar(100)
as

declare @sql varchar(max) 
set @sql = 'SELECT * FROM OPENQUERY([AMAZON-MWS-XML]
, ''SELECT *  FROM $
WITH(
	 Src=''''https://mws.amazonservices.com/Reports/2009-01-01'''',RequestMethod=''''POST''''
	,ElementsToTreatAsArray=''''ReportRequestInfo''''
	,Filter=''''$.GetReportRequestListResponse.GetReportRequestListResult.ReportRequestInfo''''
	,RequestData=''''Action=GetReportRequestList&ReportRequestIdList.Id.1=' + cast(@ReportRequestId as varchar(20)) + '&MarketplaceId='+ @MarketplaceId +'&SellerId='+ @SellerId +'''''
	,WaitTimeMs=''''300''''
	,IncludeParentColumns=''''False''''
	--,CacheFileLocation=''''c:\temp\mws-cache-step-2.cache''''
	--,CacheEntryTtl=10  --do not use from cache too old.. but cache still prevents too many requests
	,Meta=''''[{"Name": "ReportProcessingStatus","Type": "String", Length: 100},{"Name": "GeneratedReportId","Type": "Int64"}]''''	
)'')'

--print @sql

EXECUTE( @sql)

GO

usp_MWS_GetReportDataCsv Stored Proc (Step3 – Get Report Data in CSV format )

Last step in report generation is download the report data in CSV format.

USE MyDatabase
GO

CREATE proc [dbo].[usp_MWS_GetReportDataCsv]
	@ReportId bigint,
	@MarketplaceId varchar(100),
	@SellerId varchar(100),
	@Columns varchar(8000)='*'

as

declare @sql varchar(max) 
set @sql = 'SELECT '+ @Columns +' FROM OPENQUERY([AMAZON-MWS-CSV], 
''SELECT * FROM $
WITH(
    Src=''''https://mws.amazonservices.com/Reports/2009-01-01'''',RequestMethod=''''POST''''
	,RequestData=''''Action=GetReport&ReportId=' + cast(@ReportId as varchar(20)) + '&MarketplaceId='+ @MarketplaceId +'&SellerId='+ @SellerId +'''''
	,WaitTimeMs=''''300''''
	,ColumnDelimiter=''''{TAB}''''
	,ResponseCharset=''''Windows-1252''''	
	,CacheFileLocation=''''c:\temp\mws-cache-step-3.cache''''
	,CacheEntryTtl=300 --once report is generated lets try to read from cache 
	)''
)'

print @sql

EXECUTE( @sql)
GO

Example – Import data from Amazon MWS into SQL Server Table

Finally when we have all stored procs created we can load into target table as below. In our previous article we saw how to use SELECT INTO along with OPENQUERY to load data into new temp table or insert into some existing table. However when you want to load Stored Proc output into table its not straight forward specially when we calling dynamic SQL.

Three tricks worth mentioning here which makes it possible to load data from stored proc into temp table dynamically.

1. Create a linked server to Self so we can use OPENQUERY against to call stored proc so its easy to use SELECT INTO.
2. Use WITH RESULT SETS to describe metadata of output. This is needed because we are using dynamic SQL inside proc and WITH RESULT SETS describes the metadata.
3. Use of SET NO COUNT ON to avoid error like below
   

   "SOMESERVERNAME" indicates that either the object has no columns 
   or the current user does not have permissions on that object

4. If you dont do above tricks then you may see errors like this (INSERT is not allowed when we use DYNAMIC SQL inside nested Stored Procs)

To load output into table you must know column names from the output.  If you dont know you can try to run stored proc without @Columns like below.

EXEC usp_MWS_Report @MarketplaceId =''ATVPDKIKX0DER'', @SellerId=''Axxxxxxxxxxxxx'', @ReportType = ''_GET_MERCHANT_LISTINGS_DATA_''

Once we know columns from report, you can run following script to insert into table.

Full Script – Load MWS Report Output into Table

--//////////////////////////////////
— Step-1 : Create Local Linked server to point to self
--//////////////////////////////////
USE [master]
GO
--change default catalog
--For MSSQL 2012, 2014, 2016, 2017, and 2019 use @provider=N'MSOLEDBSQL' below (SQL Server Native Client 11.0)---
EXEC master.dbo.sp_addlinkedserver @server = N'ME', @srvproduct=N'', @provider=N'SQLNCLI11', @datasrc=N'localhost', @catalog=N'Northwind'
--For MSSQL 2022 or higher use @provider=N'MSOLEDBSQL' below (Microsoft OLE DB Driver for SQL Server)---
--EXEC master.dbo.sp_addlinkedserver @server = N'ME', @srvproduct=N'', @provider=N'MSOLEDBSQL', @datasrc=N'localhost', @catalog=N'Northwind'

EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'ME',@useself=N'True',@locallogin=NULL,@rmtuser=NULL,@rmtpassword=NULL
GO
--//////////////////////////////////
— Step-2 : Enable RPC OUT and Disable MSDTC
--//////////////////////////////////
EXEC sp_serveroption 'ME', 'rpc out', true;  
GO
--Below needed to support EXEC + INSERT (dynamic query)
EXEC sp_serveroption 'ME', 'remote proc transaction promotion', false;
GO
--test simple query
select * from OPENQUERY(ME,'select 1')
GO

--//////////////////////////////////
— Step-3 : Load data into temp table from Amazon MWS Report
--//////////////////////////////////
SELECT * INTO #amazon_mws_report
FROM OPENQUERY(me,'
SET NOCOUNT ON;
EXEC usp_MWS_Report 
   @Columns=''[item-name], [item-description], [listing-id]'',
   @MarketplaceId =''ATVPDKIKX0DER'', 
   @SellerId=''Axxxxxxxxxxxxx'', 
   @ReportType = ''_GET_MERCHANT_LISTINGS_DATA_''
WITH RESULT SETS 
(
	(itmname varchar(1000),itmdesc varchar(1000),itmid varchar(1000))
)
')

select * from #amazon_mws_report

Import Amazon MWS Data into SQL Table (Load Report output into temp table)

Conclusion

In this article, we saw how to load Amazon MWS Data into SQL Server. We used simple SQL queries with ZappySys XML / CSV Driver to pull data from Amazon MWS API. ZappySys ODBC PowerPack makes it super easy to consume data from virtually any API (JSON / XML or CSV Web Services), no coding required. Download here to get started with ZappySys ODBC drivers.

The post How to import Amazon MWS data into SQL Server (T-SQL) appeared first on ZappySys Blog.

In our previous article we saw how to read Google Analytics data using SSIS. But what if you don’t use SSIS and you still like to Import Google Analytics data into SQL Server or Read inside other Reporting / ETL Apps (e.g. Excel, Power BI, MS Access … and many more). Not all BI tools comes with native driver / connectivity for Google Analytics. Sometimes they do offer Google Analytics connectivity but don’t offer feature you looking for (e.g. Sort, Segment Filter). In such case you have to look for solution outside.

Well not to worry we will show you how simple it is to connect to Google Analytics using ZappySys JSON Driver (For REST API / Files) in few mins. This driver is one of the best REST API drivers you can find in the market. It is so generic that it can virtually access any REST API you may find internally or on public sites (e.g. Facebook, Salesforce, Azure, AWS). Check these articles to see many more use cases of JSON Driver.

At the end of this article you will learn how to Query Google Analytics data in SSMS like below (Yes that’s correct .. Pretty Awesome!!! ).

Import Google Analytics Data into SQL Server Table (using T-SQL Code)

Testing Google Analytics API

If you are new to Google Analytics API then read little bit about Google Analytics REST API here. Google also offers a really good way to test Analytics API using Query Explorer Tool here. So please check that and understand how to craft correct REST API URL which will be used in the next section.

Using Google Analytics Query Explorer Tool – Create REST API Url (Dimension / Metrics Selection Browser)

Sample Google Analytics API Response

Behind the scene here is how API Request and Response Looks like for Google Analytics REST API. Notice that Authorization Header in below request is automatically added by ZappySys Driver or Query Testing tool above. Also notice nextLink attribute in response, its used to fetch more data. By default each response contains upto 10000 rows.

Request

GET https://www.googleapis.com/analytics/v3/data/ga?ids=ga:185737326&start-date=30daysAgo&end-date=yesterday&metrics=ga:avgSessionDuration&dimensions=ga:date
Authorization: Bearer ya29.GlwUxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Response

{
    "kind": "analytics#gaData",
    "id": "https://www.googleapis.com/analytics/v3/data/ga?ids=ga:123456789&dimensions=ga:date&metrics=ga:avgSessionDuration&start-date=30daysAgo&end-date=yesterday",
    "query": {
        "start-date": "30daysAgo",
        "end-date": "yesterday",
        "ids": "ga:123456789",
        "dimensions": "ga:date",
        "metrics": ["ga:avgSessionDuration"],
        "start-index": 1,
        "max-results": 1000
    },
    "itemsPerPage": 1000,
    "totalResults": 30,
    "selfLink": "https://www.googleapis.com/analytics/v3/data/ga?ids=ga:123456789&dimensions=ga:date&metrics=ga:avgSessionDuration&start-date=30daysAgo&end-date=yesterday",
    "nextLink": "https://www.googleapis.com/analytics/v3/data/ga?ids=ga:123456789&dimensions=ga:date&metrics=ga:avgSessionDuration&start-date=30daysAgo&end-date=yesterday&start-index=10001&max-results=10000",
    "profileInfo": {
        "profileId": "123456789",
        "accountId": "33445566",
        "webPropertyId": "UA-33445566-3",
        "internalWebPropertyId": "454545454",
        "profileName": "All Web Site Data",
        "tableId": "ga:123456789"
    },
    "containsSampledData": false,
    "columnHeaders": [
        {
            "name": "ga:date",
            "columnType": "DIMENSION",
            "dataType": "STRING"
        },
        {
            "name": "ga:avgSessionDuration",
            "columnType": "METRIC",
            "dataType": "TIME"
        }
    ], "totalsForAllResults": {
        "ga:avgSessionDuration": "49.3779488037477"
    },
    "rows": [ 
		["20190425", "401.38912133891213"], 
		["20190426", "512.49723756906077"], 
		["20190427", "11.53488372093024"], 
		["20190428", "432.148648648648646"], 
		["20190429", "234.650735294117645"],
		.......
		.......
		.......
		.......
		["20190525", "74.123435294117645"]
	]
}

Creating Google API Project (Obtain Client ID / Secret)

Very first step to call any Google API including Google Analytics is to create Google API Project and register OAuth App to obtain Client ID and Client Secret. If you dont want to go through this hassle and start easy way then ZappySys offers Inbuilt Default App on OAuth Connection UI but we strongly recommend you create your own app rather than using Default App because in Default App API call limit is shared by many. Your data is never shared with ZappySys in any case even you use Default App.

Here is how to create API Project to call Google API. Once you create Google API Project make sure you enable Google Analytics API.

So once you have Client ID and Client Secret we can move forward to the next step.

Loading Google Analytics data into SQL Server

In our previous article we saw in depth general idea on how to import REST API data in SQL Server (T-SQL Script) without relying on any ETL tools.  Now lets look at step by step on how to load Google Analytics data into SQL Server.

In this example, our goal is to extract a Google Analytics report which shows average session duration and new users count by date (For last 30 days).

Setup ZappySys Data Gateway

Very first step to access any REST API Data inside SQL Server is to configure ZappySys Data Gateway. We covered this in previous article.  But here are high level steps for initial setup of Data Gateway.

1. Search for Gateway in Start menu and  Select ZappySys Data Gateway
   

   Open ZappySys Data Gateway

2. Create new User in Data gateway on Users tab. Enter username and password (we will use this when we create Linked Server) . Check Admin Option
   

   Add Data Gateway User

Setup Google Analytics API Data Source in Gateway / ODBC

Once gateway user is setup, now lets create a new Data Source for Google Analytics API. In this section we will talk how to create data source in gateway but most instructions can be used to create data source in ODBC too. At the end of this article we explained how to Launch ODBC Datasource UI. Step#1 and 2 are only different, all other steps same in ODBC DSN creation.

1. Click Add Data Source option (Select Native – JSON Driver )
   

   Add Gateway Data Source (Native JSON Driver)

2. Click Edit to configure data source
   

   Edit Gateway Data Source Settings

3. Now lets configure Driver settings.  Click on Load Connection String button.
   

   Load ZappySys Driver ConnectionString to configure UI

4. Enter the following ConnectionString to get started with predefined settings.
   Change ids (111223344 to your own Profile ID in DataPath URL). Other URL Parameters explained in next section.
   

   DRIVER={ZappySys JSON Driver};
   DataPath='https://www.googleapis.com/analytics/v3/data/ga?ids=ga:11223344&start-date=30daysAgo&end-date=yesterday&metrics=ga:avgSessionDuration,ga:newUsers&dimensions=ga:date';
   DataConnectionType=OAuth;
   ScopeSeparator='{space}';
   Scope='https://www.googleapis.com/auth/analytics https://www.googleapis.com/auth/analytics.readonly';
   ServiceProvider=Google;
   Filter='$.rows[*]';
   ArrayTransformType=TransformSimpleTwoDimensionalArray;
   ArrayTransColumnNameFilter='$.columnHeaders[*].name';
   RequestMethod='GET';
   NextUrlAttributeOrExpr='$.nextLink';
   ClientId='xxxxxxxxxxxx';
   ClientSecret='yyyyyyyyyyyyyy';
   UseCustomApp='True'

    
5. URL used in DataPath is most important (parameters explained below) . You can use   Query Explorer Tool here to build same URL.
   

   https://www.googleapis.com/analytics/v3/data/ga?ids=ga:11223344&start-date=30daysAgo&end-date=yesterday&metrics=ga:avgSessionDuration,ga:newUsers&dimensions=ga:date

   Parameters Explained 
   
   Here is some required parameters used in above REST API URL.
   ids=ga:11223344
   This is your Profile ID which you like to extract.  Read more
   start-date=30daysAgo
   This can be yyyy-MM-dd or some other some predefined date keywords (e.g today, yesterday 10daysAgo). Read more.
   end-date=yesterday
   Same as above
   metrics=ga:avgSessionDuration,ga:newUsers
   This can be comma separated list of metrics you like to extract (max 10). Read more.
   dimensions=ga:date
   This can be comma separated list of metrics you like to extract (max 7). Read more.
6. Now its time to configure OAuth settings. Click on Configure Connection. Enter Client ID and Client Secret Obtained in the previous section (see here) and then click Generate Token as below.
   

   Configure Google API Data Source / DSN (Google Analytics API Example)

7. Thats it now can click on Preview Tab and run sample query like below.
   

   select * from $

   –OR–
   (Select Table name from dropdown and generate default query and then you can add alias , remove unwanted columns)
   

   SELECT 
   "ga:date" date, 
   "ga:avgSessionDuration" avgSessionDuration,
   "ga:newUsers" newUsers
   FROM "rows"

   –OR–
   (Type custom query with your own URL generated from here, Change Profile ID – from 11223344 to your own id)
   

   SELECT * FROM $
   WITH(
   	 Src='https://www.googleapis.com/analytics/v3/data/ga?ids=ga:11223344&start-date=30daysAgo&end-date=yesterday&metrics=ga:avgSessionDuration,ga:newUsers&dimensions=ga:date'
   	,IncludeParentColumns='False'
   )

   

Create Linked Server in SQL Server using T-SQL Script

Once our Gateway Data Source is configured we can now move to SQL Server Part to define Linked Server.

Here is how you can define linked server to access Google Analytics Data inside SQL Server. There are two ways to create Linked Server (1)  via T-SQL script (2) via SSMS UI)

Here is how to create using Script.

USE [master]
GO

--drop existing linked server for same name if already found
EXEC master.dbo.sp_dropserver @server=N'GOOGLE_ANALYTICS_LS', @droplogins='droplogins'
GO

EXEC master.dbo.sp_addlinkedserver 
	@server = N'GOOGLE_ANALYTICS_LS', 
	@srvproduct=N'', 
	@provider=N'SQLNCLI', 
	@datasrc=N'localhost,5000', --this is host name and port number where  ZappySys Data Gateway is running
	@catalog=N'ZS-GA' --this must be same name as defined on Data Source tab Grid on ZappySys Data Gateway

EXEC master.dbo.sp_addlinkedsrvlogin 
	@rmtsrvname=N'GOOGLE_ANALYTICS_LS',
	@useself=N'False',
	@locallogin=NULL,
	@rmtuser=N'SOME-GATEWAY-USER', -- user name created on ZappySys Data Gateway
	@rmtpassword='SOME-GATEWAY-PASSWORD' -- password for ZappySys Data Gateway user
GO

Create Microsoft SQL Server Linked Server using SSMS UI

Import Google Analytics Data into SQL Server Table (T-SQL Code)

Now lets see how to run sample query to import Google Analytics Data into SQL Server. Notice that how we have escaped quotes in OPENQUERY.
NOTE: Change id 11223344 with your own Id (use this tool to create URL)

select * into GaApiLoad from OPENQUERY([GOOGLE_ANALYTICS_LS] ,
'SELECT * FROM "rows"
WITH 
 (
   SRC=''https://www.googleapis.com/analytics/v3/data/ga?ids=ga:11223344&start-date=30daysAgo&end-date=yesterday&metrics=ga:avgSessionDuration,ga:newUsers&dimensions=ga:date''
  ,IncludeParentColumns=''False''
 )
')
select * from GaApiLoad

Import Google Analytics Data into SQL Server Table (using T-SQL Code)

Expose Google Analytics Data as View

To make it simple you can expose above query as SQL Server views so you can access them from any Reporting / ETL Tools.

CREATE VIEW vw_Google_Analytics_BounceRate_PageViews_Last30Days
AS
--change ids to your own Profile ID
select * from OPENQUERY([GOOGLE_ANALYTICS_LS] ,
'SELECT * FROM "rows"
WITH 
 (
   SRC=''https://www.googleapis.com/analytics/v3/data/ga?ids=ga:11223344&start-date=2019-04-01&end-date=2019-04-30&metrics=ga:bounceRate,ga:uniquePageviews&dimensions=ga:date''
  ,IncludeParentColumns=''False''
 )
')
GO

Making things Dynamic using Stored Procedure with Parameters

Now let’s make few things dynamic. We will change above query so we can pass Id, Dimensions and Metrics as parameters.

Run below script to Create a stored proc and fetch google analytics data in SQL Server.

use MyDatabase
go
create proc usp_GoogleAnalyticsData
	@ids varchar(1000),
	@startDate varchar(50)='30daysAgo',
	@endDate varchar(50)='yesterday',
	@metrics varchar(200)='ga:avgSessionDuration,ga:newUsers',
	@dimensions varchar(200)='ga:date'	
as

declare @sql varchar(4000)

set @sql='select * from OPENQUERY([GOOGLE_ANALYTICS_LS] ,
''SELECT * FROM "rows"
WITH 
 (
   SRC=''''https://www.googleapis.com/analytics/v3/data/ga?ids=ga:' + @ids 
   +'&start-date='+ @startDate 
   +'&end-date='+ @endDate 
   +'&metrics='+ @metrics 
   +'&dimensions='+ @dimensions 
   +'''''
  ,IncludeParentColumns=''''False''''
 )
'')'

print @sql
execute(@sql)

go

--Example: query google analytics data
exec usp_GoogleAnalyticsData 
	@ids='11223344',  --Change this ID - Obtain from here https://ga-dev-tools.appspot.com/query-explorer/
	@startDate ='30daysAgo',  --or like this 2019-04-01
	@endDate   ='yesterday', --or like this 2019-04-30
	@dimensions='ga:date', 
	@metrics   ='ga:bounceRate,ga:uniquePageviews'

Insert Google Analytics Data into table (Save Stored Procedure Output)

Now what if you like to save Stored Proc output into a table? Well for that you must create table with same structure as stored proc output and then use INSERT INTO sql like below.

--import google analytics data in a table
if(object_id('GaDataLoad') is not null)
	drop table GaDataLoad
create table GaDataLoad([Date] date, BounceRate numeric(10,5), PageViews numeric(10,5))  

insert into GaDataLoad
exec usp_GoogleAnalyticsData 
	@ids='11223344',  --Change this ID - Obtain from here https://ga-dev-tools.appspot.com/query-explorer/
	@startDate ='30daysAgo',  --or like this 2019-04-01
	@endDate   ='yesterday', --or like this 2019-04-30
	@dimensions='ga:date', 
	@metrics='ga:bounceRate,ga:uniquePageviews'

select * from GaDataLoad

Performance Tips / Handling data errors due to missing rows

By default ZappySys API Drivers sends minimum 2 API requests. First one to fetch metadata and second, third… for Data. There will be a time when you wont have any data and it may throw error about no records found because it fails to parse metadata. You can avoid such issue by supplying metadata before hand so you can avoid expensive API calls. Check this article see how you can supply cached metadata in your SQL Query as below. This will avoid Metadata call and speedup significantly. It will also avoid errors due to no rows found for specified criteria.

select * from OPENQUERY([GOOGLE_ANALYTICS_LS] ,
'SELECT * FROM "rows"
WITH 
 (
   SRC=''https://www.googleapis.com/analytics/v3/data/ga?ids=ga:11223344&start-date=30daysAgo&end-date=yesterday&metrics=ga:bounceRate,ga:uniquePageviews&dimensions=ga:date''
  ,IncludeParentColumns=''False''
  ,Meta=''[ 
     {"Name": "ga:date","Type": "String", "Length": 16 },
     {"Name": "ga:bounceRate", "Type": "String", "Length": 36 },
     {"Name": "ga:uniquePageviews", "Type": "String", "Length": 10 }
    ]''
 )
')

ODBC Connection – Google Analytics Data in Other Apps (e.g. SSRS / Power BI / Excel / Tableau)

So far we have talked accessing data inside SQL Server using Data Gateway Approach but what if you like to access in other apps ? Well you have three options.

1. Use Microsoft SQL Server Driver to call Linked Server Queries (OPENQUERY approach we saw earlier)
2. Access Google Analytics Data using ODBC connectivity
3. Use Microsoft SQL Server Driver to send direct SQL queries to Gateway (Bypass Linked Server – Use inner SQL query without OPENQUERY in this case)

Method-1 : Linked Server Approach

First approach most likely works in all cases because most apps will support connecting to SQL Server using OLEDB / ADO.net / ODBC / JDBC Drivers. So calling SQL Queries which uses Linked Server (i.e. OPENQUERY statement)  is advisable.

Method-2 : ODBC Driver Approach

However in some cases this may not be possible (e.g. You don’t have SQL Server inhouse or you don’t want to rely on SQL Server to access API). In such case you can use ODBC connectivity in your app. Since ODBC is widely adopted standard most app out there (Except JAVA apps) should support ODBC Drivers. If you like to use this approach then create ODBC DSN rather than Data Gateway Data source and use it in your Reporting / ETL / Custom Apps.

Open ODBC Data Sources (Create DSN)

ODBC User DSN Tab: Add new Driver Screen

Method-3 : Direct connection to Data Gateway (By pass Linked Server)

Last option we suggest for cases like JAVA Apps / Linux / Mac machines where you cannot install ZappySys ODBC Drivers and you dont have option to use SQL Server Linked Server either. In such case you can try to send SQL Queries to Gateway directly using Microsoft SQL Server Compatible Drivers (i.e. ODBC /JDBC). See this example how we called API Queries inside JAVA Apps.

Google Analytics / REST API Integration in various apps

ZappySys ODBC Drivers built using ODBC standard which is widely adopted by industry for a long time. Which mean the majority of BI Tools / Database Engines / ETL Tools already there will support native / 3rd party ODBC Drivers. Below is the small list of most popular tools / programming languages our Drivers support. If your tool / programming language doesn't appear in the below list, which means we have not documented use case but as long as your tool supports ODBC Standard, our drivers should work fine.  

BI / Reporting Tools Integration	ETL Tools Integration	Programming Languages Integration
Microsoft Power BI Tableau SSRS (SQL Reporting Services) QlikView /Qlik Sense MicroStrategy Google Sheet Microsoft Excel Microsoft Access Oracle OBIEE Many more (not in this list).....	Informatica PowerCenter (Windows) Informatica Cloud SSIS (SQL Integration Services) SQL Server Talend Data Studio Pentaho Kettle Oracle OBIEE Many more (not in this list).....	Visual Studio C# C++ JAVA Python PHP PowerShell T-SQL (Using Linked Server)

 

Conclusion

In this article we explored many ideas of JSON / API integration in SQL Server.  We saw how to create OAuth App for Google API and import Google Analytics data into SQL Server Table without doing any ETL using pure T-SQL code (Query / Views /Stored Procs). You can Download  FREE Trial of ODBC PowerPack and try yourself see how easy it is to query any API inside SQL Server and avoid expensive ETL processes.

The post Import Google Analytics data into SQL Server / Reporting / ETL appeared first on ZappySys Blog.