There might be a case when you have many CSV, JSON or XML files in Azure Blob and you want them to be imported straight into a SQL Server table. Here come ZappySys ODBC PowerPack and ZappySys Data Gateway (part of ODBC PowerPack) which will enable you to accomplish that. ZappySys ODBC PowerPack includes powerful Azure Blob CSV, Azure Blob JSON and Azure Blob XML drivers that let you connect to an Azure Storage Explorer and read the contents from the files in the Container. Bringing in ZappySys Data Gateway allows doing that right from a SQL Server. Let’s begin and see how to Import Azure Blob files into SQL Server (CSV/JSON/XML Driver).

These drivers are related to this article:

Azure Blob Driver (for CSV Files)

Azure Blob Driver (for JSON Files)

Azure Blob Driver (for XML Files)

Prerequisites

1. Firstly, Download and install Microsoft Azure Storage Emulator and Microsoft Azure Storage Explorer.
2. SQL Server instance installed (can be a SQL Server Express LocalDB instance).
3. SQL Server Management Studio (SSMS) installed.
4. ZappySys ODBC PowerPack installed (must be installed on a Windows machine; can be a different machine than SQL Server is installed on).

NOTE: If you want to use Live account (Azure Storage) then you can skip Step #3

Getting started

In order to start, we will show several examples. ZappySys includes Data Gateway that will help you in reading data of Azure Blob files. Here we are showing you is, how to Import Azure Blob files into SQL Server (CSV/JSON/XML Driver).

You can connect to Azure Storage Service from SSIS, you will need Storage Account Name and Access Key. Ask your SysAdmin or responsible person to provide that information to you. Click here to read more about how to get your Storage Account Name and Access Key. Here are sample Credentials.

Account Name: mystorageaccount
Access Key: Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==

If you don’t have Azure Storage account then you can try offline mode on your local machine. You can just download Azure Storage Emulator and start testing.

Setup Azure Storage Explorer

Setup Azure Storage client tools we will start from a simple example of how to load Invoice Data from a CSV that is located in Azure Blob, then we will load JSON contacts data from them and finally, we will proceed by loading contacts from many XML files those are compressed in ZIP.

1. Once you have downloaded and installed storage emulator You can launch Microsoft Azure Storage Emulator from its Physical location or from the desktop or start menu shortcut.

   

   Azure Storage Emulator Physical Location

2. If you can see the below-attached Command Prompt screen after Emulator started. Then you can proceed to start Microsoft Azure Storage Explorer as the Azure Storage Emulator is started successfully.

   

   Command Prompt Screen after Microsoft Azure Storage Emulator Started

3. Now, you have to download and install Microsoft Azure Storage Explorer and then you can launch Microsoft Azure Storage Explorer from its Physical location or from the desktop or start menu shortcut.

   

   Microsoft Azure Storage Explorer Location.

4. For Creating a Blob Container, First of all, you need to go to Microsoft Storage Explorer Window. Then you can go through like this way (Storage Accounts –> (Development) –> Blob Containers).
   

   Microsoft Azure Storage Explorer: Create a Blob Container

Open and Configure ZappySys Data Gateway

Import CSV file located at Azure Blob into SQL Server

Overview

Let’s say you have CSV file that is located in Azure Blob Container, and you want them all data into your SQL Server. We will follow the same steps for that we will do for import JSON and XML file into SQL Server.

Azure Blob Container – CSV File Located

Create an Azure Blob CSV data source in ZappySys Data Gateway

1. The first thing you will have to do is to create a data source in ZappySys Data Gateway. In the Data Source tab, Just click on Add button, give the data source a name, e.g. “MyInvoiceCSV“, and then select Native – ZappySys Azure Blob CSV Driver.
   

   Create Data Source – Azure Blob CSV Driver

2. Then click on Edit and add the Data Gateway user you created in the Users tab. We will use this user later when adding a Linked Server to the Data Gateway to authenticate:
3. Now, click on the Edit button to configure “MyInvoiceCSV” Azure Blob CSV Data Source.
   

   Configure Data Gateway

4. When a window open, click on Click here to Configure the Connection and select Use the Microsoft Azure Storage Emulator. If you have Online Storage Account and Account Key then Select Enter Storage Account Credentials and enter it.
   
   

   Create Azure Storage Connection

5. Now, Select Azure Blob Container and file from it and In the Data Format / Compression (Zip / GZip) tab set suitable file Compression Format (Zip or GZip).
   

   ZappySys Azure Blob CSV – Configure Driver

6. Go to the Preview tab and click on the Preview Data button to make sure everything is configured correctly and preview the results.
   

   ZappySys Azure Blob CSV – Preview Data

7. Now, Click on OK button also Save and Restart Service.
   

   ZappySys Data Gateway Service – Save and Restart

8. That’s all, now we are ready to proceed and add a Linked Server to this data source.

Set up a SQL Server Linked Server

Execute the SQL query

Once again, if you created the Linked Server with name “GATEWAY”, execute this SQL query in SSMS:

SELECT * INTO MyInvoices FROM OPENQUERY([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyInvoices

The results

You should see a similar view, once you execute the query:

The results of loading many CSVs from Azure Blob into SQL Server

Import JSON file located at Azure Blob into SQL Server

Overview

Let’s say you have JSON file that is located in Azure Blob Container, and you want them all data into your SQL Server. We will follow the same steps of the import CSV file into SQL Server and we will do for XML file into SQL Server.

Create an Azure Blob JSON data source in ZappySys Data Gateway

Follow the steps as in Create an Azure Blob CSV data source in ZappySys Data Gateway section when adding a new data source, except for these two steps:

1. The first thing you will have to do is to create a data source in ZappySys Data Gateway. Just click Add button, give the data source a name, e.g. “MyContactsJSON“, and then select Native – ZappySys Azure Blob JSON Driver.
   

   ZappySys Data Gateway – Create New Data Source
2. After that, select a JSON file from container you want to load, and then click Select Filter button to choose data you want to be displayed in SQL Server.
   

   Configuring the JSON data source to load a JSON from Azure Blob container into SQL Server

Now you are ready to create a Linked Server and connect to the data source you just created. If you used “MyContactsJSON” as the data source name, make sure to use the same name when creating a Linked Server.

Set up a SQL Server Linked Server

Again, follow the very same steps from the section above where we added the Linked Server to a CSV data source, except that change Catalog property of the Linked Server to match the name of the Azure Blob JSON data source we created. Once you do that, you are ready to load JSON into your SQL Server.

Execute the SQL query

Once you created the Linked Server to ZappySys Data Gateway, you are ready to execute the SQL query and load data into SQL Server. Supposedly, you created the Linked Server with the name “GATEWAY”, then open SSMS and execute the following query:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see similar results after you execute the query:

The results of getting JSON data straight from Azure Blob JSON

Now we are ready to proceed to the next section and import many XML files at once.

Import many XML files located at Azure Blob into SQL Server

Overview

Supposedly, you have many XMLs where each one is zipped and you want to load them all into a SQL Server table.

Zipped XML files located in Azure Blob container to be loaded into SQL Server

The first thing you will have to do is to create a data source, based on ZappySys Azure Blob XML Driver. Let’s proceed and just do that.

Create an Azure Blob XML data source in ZappySys Data Gateway

Follow the same steps as in Create an Azure Blob JSON data source in ZappySys Data Gateway section when adding a new data source, except for these two steps:

1. When adding a data source select Native – ZappySys Azure Blob XML Driver as Connector Type.
   

   Choosing “ZappySys Azure Blob XML Driver” to load XMLs to SQL Server
2. Then click on Edit and select Azure Blob Container and file from it and In the Data Format / Compression (Zip / GZip) tab set suitable file Compression Format (Zip or GZip).
   

   Configuring data source based on ZappySys Azure Blob XML Driver to load XMLs into SQL Server

3. Now we are ready to set up a Linked Server to this newly created data source.

Set up a SQL Server Linked Server

Follow the very same steps from the section above where we added the Linked Server to a CSV data source, except that change Catalog property of the Linked Server to match the name of the Azure Blob XML data source you created. Once you do that, you are ready to load XMLs into your SQL Server.

Execute the SQL query

Again, if you created the Linked Server with name “GATEWAY”, execute the following SQL query in SSMS:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see a similar view once you execute the query:

The data of many XMLs loaded from Azure Blob Container into SQL Server

Now we are ready to move to the next section and recursively scan CSVs and load them into SQL Server.

Conclusion

We had a goal to import Azure Blob files into SQL Server from SQL Server itself. We learned how to Import Azure Blob files into SQL Server (CSV/JSON/XML Driver), which some of them were zipped and were located at different folders and subfolders. It was possible to accomplish that using ODBC PowerPack, ZappySys Data Gateway, and ODBC PowerPack drivers – Azure Blob JSON Driver, Azure Blob XML Driver, and Azure Blob CSV Driver. At first, we created data sources in the Data Gateway, then we created Linked Servers in SQL Server and connected to ZappySys Data Gateway, which finally made it possible to load data from Azure Blob straight from SQL queries in SSMS.

References

Finally, you can use the following links for more information:

• Azure Blob CSV Driver
• Azure Blob JSON Driver
• Azure Blob XML Driver

The post Import Azure Blob files into SQL Server (CSV/JSON/XML Driver) appeared first on ZappySys Blog.

In our previous blog post we saw how to import rest API in SQL Server. Using same concepts lets look at how to import Amazon MWS Data into SQL Server. We will explore many techniques to call Amazon MWS API and learn how to automate Amazon MWS data extraction without doing any ETL. We will call MWS XML or CSV API just using T-SQL code. We will also learn how to Download Reports from by calling MWS API.

You can also refer other articles here (Power BI) and here (SSIS)

About Amazon MWS  API

If you want to sell something on Amazon you can use their e-commerce platform with some monthly fee. You can setup your entire online store / inventory using admin interface.

Amazon Marketplace Web Service (Amazon MWS) is an integrated web service API that helps Amazon sellers to programmatically exchange data on listings, orders, payments, reports, and more. Using these API you can read or write data from your Seller account and integrate it inside your own Systems (e.g. Reporting / ETL / BI tools).

Obtain MWS API Access Key / Secret and Seller ID

Very first thing to call any MWS API is to obtain Access Key, Secret Key and Seller ID (i.e. Merchant ID). To obtain this you first have to register a developer.

Here is where to look for your Merchant ID and Key / Secret.  Here is direct link to User Permissions page.

How to get Amazon MWS developer keys and know your Merchant ID (i.e. Seller ID)

Getting Started

Now let’s look at step by step instructions on how to call Amazon MWS API and then import into Power BI. In below steps we will use  ZappySys ODBC PowerPack so make sure it’s installed first.

Using ScratchPad to Test Amazon MWS API

Each MWS API has set of required and optional parameters. You can refer each API page for details. For example ListMatchingProducts API which we will use as an example. To make MWS API testing / learning easy, Amazon provides online testing tool called MWS ScratchPad. We will use this tool to craft API requests and use that information in ODBC driver SQL queries and later import data in Power BI.

1. Open MWS ScratchPad  by vising https://mws.amazonservices.com/scratchpad/index.html
2. Select Products API category and pick ListMatchingProducts API
3. Enter your SellerID (i.e. Merchant ID), Developer AccessKey (i.e. AWSAccessKeyId) and Secret Key we obtained in the previous section. Enter MarketId (e.g. ATVPDKIKX0DER for USA Market) and other parameters as below and Click Submit to see response.
   

   Using Amazon MWS ScratchPad (API Testing Tool – Created by Amazon)

4. Click on the Response Details Tab to extract some important information we will use for ODBC configuration. Notice attributes in red rectangles. They are the ones which we will need in the POST request Body. ZappySys Drivers supply many common parameters automatically but any API endpoint specific parameter must be supplied in the Request Body. For example you dont have to supply SignatureVersion, Timestamp, Signature or SignatureMethod
   

   Calling Amazon MWS API in Scratchpad Testing Tool (Response Details Tab) – ListMatchingProducts Example

This API requires Market ID parameter so enter it as per your Market where you selling. For example USA marketplaceid = ATVPDKIKX0DER. You can lookup correct MarketplaceId by checking this help (endpoints / marketplaces).

Setup Amazon MWS API Connections

Create XML Driver Connection for Amazon MWS API (e.g. RequestReport )

Some APIs we need to call in this article is XML format and some API (e.g. GetReport) is CSV format – Tab delimited so we need to create 2 different connections using CSV driver and XML driver. So lets get started with Amazon MWS Connection for XML API.

1. Click on Add New Data source. Name as AMAZON-MWS-XML and select Native – ZappySys XML Driver option.
   

   Add Gateway Data Source

2. Now click Edit to Configure Settings. For now we will only care OAuth Connection settings. All other options are overwritten in SQL Query.
   

   Edit Gateway Data Source Settings

3. When UI opens for Driver Enter MWS API Service URL as below. Notice that we added /Products/2011-10-01 (This was extracted from previous section). You can check full list of Amazon MWS Endpoints here. Because we are connecting to USA region we will use below URL.
   

   https://mws.amazonservices.com/Products/2011-10-01

4. Select OAuth from the Connection dropdown and Click Configure Settings
5. On the Connection UI, Select OAuth Provider Amazon Market Web Service (MWS)
6. Enter your API Access Key and Secret Key (obtained from here)
7. Your connection will look like below.
   

   ODBC Connection Credentials for Amazon MWS API (Access Key, Secret Key, URL Endpoint)

8. Also configure Retry options to handle API limit reached error when you call API too often. For Amazon MWS we get Status code 503 when request is throttled so lets narrow down on which error we want to retry.
   

   Retry Options

9. Click OK to save OAuth Connection UI to go back to main UI
10. Now Go to Throttling Tab and enter some delay (e.g. enter 20000 for 20sec delay after each request). This will slow down API calls so we don’t face errors about API Limit reached.
11. Click OK to Save UI
12. Click Save button on Gateway UI to save and restart

Create CSV Driver Connection for Amazon MWS API (e.g. GetReport )

Now lets create a connection CSV API (e.g. GetReport ).

Follow almost same steps as previous section except use CSV Driver when you click on Add new Data source select Native – ZappySys CSV Driver and name data source as AMAZON-MWS-CSV . As we mentioned for now we mostly want to setup Connection and not worry about Request Body, Pagination etc.

Create Linked Server for SQL Server

Run following Script in SSMS to create Linked servers. Change ######## with Your gateway password. Change localhost with machine name or IP where ZappySys Gateway Is running. Use localhost if SQL Server and Gateway both on the same machine.

USE [master]
GO

EXEC master.dbo.sp_addlinkedserver @server = N'AMAZON-MWS-XML', @srvproduct=N'', @provider=N'SQLNCLI', @datasrc=N'localhost,5000', @catalog=N'AMAZON-MWS-XML'
EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'AMAZON-MWS-XML',@useself=N'False',@locallogin=NULL,@rmtuser=N'tdsuser',@rmtpassword='########'
GO

EXEC master.dbo.sp_addlinkedserver @server = N'AMAZON-MWS-CSV', @srvproduct=N'', @provider=N'SQLNCLI', @datasrc=N'localhost,5000', @catalog=N'AMAZON-MWS-CSV'
EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'AMAZON-MWS-CSV',@useself=N'False',@locallogin=NULL,@rmtuser=N'tdsuser',@rmtpassword='########'
GO

If you want to know how to configure Linked Server using UI then check this article.

Amazon MWS ListOrders Example

Now lets look at how to call ListOrders request with pagination in SQL Server. Create a following Stored Procedure to List MWS Orders. If you want to extract many orders then call Report rather than this proc (See next section for generating reports)

USE MyDatabase
GO
/*
Example: 
 exec usp_MWS_ListOrders @SellerId='Axxxxxxx', @MarketplaceId ='Bxxxxxxx',@CreatedAfter='2019-01-01'  
*/
Create proc [dbo].[usp_MWS_ListOrders]
	@SellerId varchar(100),
	@MarketplaceId varchar(100),
	@CreatedAfter varchar(100),
	@CreatedBefore varchar(100)=null,
	@MaxRows int=0
as
declare @sql varchar(max) 
declare @limitClase varchar(100)=''
if(@MaxRows>0)
	set @limitClase	=' LIMIT ' + cast(@MaxRows as varchar(20))

--//Confirm ISO Date format 2019-06-01T04%3A00%3A00Z  rather 2019-06-01T04:00:00Z
if(isnull(@CreatedAfter,'')<>'')
begin
	set @CreatedAfter='&CreatedAfter=' + @CreatedAfter
	set @CreatedAfter=replace(@CreatedAfter,':','%3A')
	if(@CreatedAfter NOT LIKE @CreatedAfter + 'T%')
		set @CreatedAfter=@CreatedAfter + 'T00%3A00%3A00Z'
	if(@CreatedAfter NOT LIKE @CreatedAfter + '%Z')
		set @CreatedAfter=@CreatedAfter + 'Z'
	--//T04%3A00%3A00Z
end
if(isnull(@CreatedBefore,'')<>'')
begin
	set @CreatedBefore='&CreatedBefore=' + @CreatedBefore
	set @CreatedBefore=replace(@CreatedBefore,':','%3A')
	if(@CreatedBefore NOT LIKE @CreatedBefore + 'T%')
		set @CreatedBefore=@CreatedBefore + 'T00%3A00%3A00Z'
	if(@CreatedBefore NOT LIKE @CreatedBefore + '%Z')
		set @CreatedBefore=@CreatedBefore + 'Z'
end

set @CreatedAfter = isnull(@CreatedAfter,'')
set @CreatedBefore = isnull(@CreatedBefore,'')


set @sql = 'SELECT * FROM OPENQUERY([AMAZON-MWS-XML]
, ''
SELECT ReportRequestId FROM $
' + @limitClase +' 
WITH(
     Src=''''https://mws.amazonservices.com/Orders/2013-09-01''''
	,Filter=''''$.RequestReportResponse.RequestReportResult.ReportRequestInfo''''
	,ElementsToTreatAsArray=''''Order''''
	,RequestData=''''Action=ListOrders[$tag$]' + @CreatedAfter + @CreatedBefore + '&MarketplaceId.Id.1='+ @MarketplaceId +'&SellerId='+ @SellerId +'''''
	,RequestMethod=''''POST''''
	,NextUrlAttributeOrExpr=''''$.ListOrders[$tag$]Response.ListOrders[$tag$]Result.NextToken''''
	,NextUrlSuffix=''''&NextToken=<%nextlink_encoded%>''''
	,WaitTimeMs=20000 -- slow down to avoid throttling. 6 requests per min
	,HasDifferentNextPageInfo=''''True''''
	,EnablePageTokenForBody=''''True''''
	,PagePlaceholders=''''body=|ByNextToken;filter=|ByNextToken''''
	--,DataConnectionType=''''OAuth''''
	--,ScopeSeparator=''''{space}''''
	--,ServiceProvider=''''AmazonMWS''''
	--,ClientId=''''AKIxxxxxx''''
	--,ClientSecret=''''AKIxxxxxx''''
	--,UseCustomApp=''''True''''
	,CacheFileLocation=''''c:\temp\mws-orders.cache''''
	,CacheEntryTtl=300 --send every 5 mins else use from cache
	,IncludeParentColumns=''''False''''
)
'')'
print @sql

EXECUTE( @sql)

Here is how to call this proc. Change SellerId, MarketplaceId and Date.

exec usp_MWS_ListOrders @SellerId='Axxxxxxx', @MarketplaceId ='Bxxxxxxx',@CreatedAfter='2019-01-01'

Download Amazon MWS Report data in SQL Server (CSV / XML Format)

Now lets look at how to extract data from Custom Report API (You must need ODBC PowerPack v1.1.1 or higher). However reading data from Reports not single step process because report generation is Job style API. Which means you send report request and wait  until its done. Once Report ready you have to read in CSV format or XML. Some Reports only available in CSV format. Check this post to understand how complex it can be to get data. We will make it simple for you to understand this in 3 steps. Basically calling reports requires minimum 3 API calls (see below)

1. Create a new report request – Cal RequestReport API . It returns ReportRequestId (You can use it in the next step)
2. Check Report Status see its done – Call GetReportRequestList API (Pass ReportRequestId got in the previous step to get data for only one Report request we care). Keep checking Report Status every few seconds until you get _DONE_  or _DONE_NO_DATA_ status in response.
3. Read Report Data – Call GetReport. This API call returns CSV or XML data. So use correct Driver for this step. If CSV data is returned then we must use CSV Driver rather than XML Driver.

Method 1 – Single query approach (New Driver)

In new driver ZappySys added few new features which makes it possible to call above 3 steps process in a single query. New query syntax offers StatusCheck and Mixing Content Formats using same driver. Because Most MWS Reports are in CSV Format we have to use CSV Driver but oddly first 2 steps listed in above sections are in XML Format so we have to use XML driver. Technically we need atleast 2 steps (i.e 2 queries) but we will show you how to achieve using single query as below.

1. Assuming you have followed steps to create a linked server AMAZON-MWS-CSV (see previous section)
2. Run below query. Replace necessary parameters as per your need. (i.e MarketplaceId , SellerId, CreatedAfter date)
   

   --- //////////////////////////////////////////////////////////////////////////
   -- Change _GET_MERCHANT_LISTINGS_DATA_ to something else if you need different report type (all 3 steps) -- https://docs.developer.amazonservices.com/en_US/reports/Reports_ReportType.html
   -- Replace MarketplaceId in all 3 steps (i.e for USA use ATVPDKIKX0DER) --https://docs.developer.amazonservices.com/en_US/dev_guide/DG_Endpoints.html
   -- Replace SellerId in all 3 steps
   -- Remove or Change CreatedAfter in first step (or use CreatedBefore parameter instread of CreatedAfter)
   --- //////////////////////////////////////////////////////////////////////////
   select * into tmpMWS_Report from OPENQUERY([AMAZON-MWS-CSV],
   '
   SELECT s3.* FROM $
   WITH(
   Src=''https://mws.amazonservices.com/Reports/2009-01-01'',RequestMethod=''POST''
   ,RequestData=''Action=RequestReport&ReportType=_GET_MERCHANT_LISTINGS_DATA_&MarketplaceId=ATVPDKIKX0DER&SellerId=Axxxxxxxxxx&CreatedAfter=2017-12-31T04%3A00%3A00Z''
   ,IncludeParentColumns=''False''
   ,EnableRawOutputModeSingleRow=''True''
   ,RawOutputFilterExpr=''//*[local-name()="ReportRequestId"]'' 
   ,RawOutputDataRowTemplate=''{ReportRequestId:"[$1]"}'' 
   ,RawOutputExtractMode=''Xml'' 
   ,WaitTimeMs=5000
   ,Meta=''[{"Name": "ReportRequestId","Type": "Int64"}]''
   ,Alias=''s1''
   
   ,Join1_alias=''s2''	
   ,Join1_Src=''https://mws.amazonservices.com/Reports/2009-01-01'',Join1_RequestMethod=''POST''
   ,Join1_RequestData=''Action=GetReportRequestList&ReportRequestIdList.Id.1=[$s1.ReportRequestId$]&MarketplaceId=ATVPDKIKX0DER&SellerId=Axxxxxxxxxxxxx''
   ,Join1_EnableRawOutputModeSingleRow=''True''
   ,Join1_RawOutputFilterExpr=''//*[local-name()="ReportProcessingStatus"]||//*[local-name()="GeneratedReportId"]'' 
   ,Join1_RawOutputDataRowTemplate=''{ReportProcessingStatus:"[$1]",GeneratedReportId:"[$2]"}'' 
   ,Join1_RawOutputExtractMode=''Xml'' 
   ,Join1_EnableStatusCheck=''True'', 
   ,Join1_StatucCheckMaxWaitSeconds=300,
   ,Join1_StatucCheckIterationWaitSeconds=25,
   ,Join1_StatusSuccessValue=''_DONE_''
   ,Join1_StatusFailedValue=''_DONE_NO_DATA_|_CANCELLED_''
   
   ,Join2_Alias=''s3''
   ,Join2_Src=''https://mws.amazonservices.com/Reports/2009-01-01'',Join2_RequestMethod=''POST''
   ,Join2_RequestData=''Action=GetReport&ReportId=[$s2.GeneratedReportId$]&MarketplaceId=ATVPDKIKX0DER&SellerId=Axxxxxxxxxxxxxx''	
   ,Join2_ColumnDelimiter=''{TAB}''
   ,Join2_ResponseCharset=''Windows-1252''	 
   ,Join2_EnableStatusCheck=''False'' 
   ,Join2_EnableRawOutputModeSingleRow=''False''
   ,Join2_WaitTimeMs=10000
   )') 
   go
   select * from tmpMWS_Report

    

Performance Tip

If you like to improve performance and dont want to use SELECT INTO approach to load into temp table as above then you can use EXEC (….) AT YourLinkedServerName  as below.

Advantage of this method is your Query speed will increase because system calls API only once when you call EXEC AT. In OPENROWSET it needs to call above query twice (Once to obtain metadata and once to get data).

INSERT INTO tmpMWS_Report
EXEC('select s3.* .................. ') AT [AMAZON-MWS-CSV]

Method 2 – Multiple query approach (For old driver)

Now lets see how to load Amazon MWS data in SQL Server using old driver (v1.1.2 or older) which didnt support single query approach like previous section. If you are using older driver or for some reason you like to split steps into multiple procedures then use below approach.

usp_MWS_Report  Stored Proc ( Generate ReportRequest, Wait until done, Read data )

USE MyDatabase
GO

/*
Examples: 
Get ReportType from here https://docs.developer.amazonservices.com/en_UK/reports/Reports_ReportType.html

exec usp_MWS_Report @MarketplaceId ='ATVPDKIKX0DER', @SellerId='AZxxxxxxxx', @ReportType = '_GET_MERCHANT_LISTINGS_DATA_'
exec usp_MWS_Report @MarketplaceId ='ATVPDKIKX0DER', @SellerId='AZxxxxxxxx', @ReportType = '_GET_FLAT_FILE_ORDER_REPORT_DATA_'

--Do not Throw Error if no data found (return empty resultset or run custom sql)
exec usp_MWS_Report @MarketplaceId ='ATVPDKIKX0DER', @SellerId='AZxxxxxxxx', @ReportType = '_GET_FLAT_FILE_ORDER_REPORT_DATA_',@ThrowErrIfNoData=0,@EmptyRowSql='select top 0 * from Orders'
--Supply date range
exec usp_MWS_Report @MarketplaceId ='ATVPDKIKX0DER', @SellerId='AZxxxxxxxx', @ReportType = '_GET_MERCHANT_LISTINGS_DATA_', @StartDate='2019-01-01T00:00:00Z', @EndDate='2019-01-31T00:00:00Z'

--loading into table 

--//////////////////////////////////
— Step-1 : Create Local Linked server to point to self
--//////////////////////////////////
USE [master]
GO
--change default catalog
--For MSSQL 2012, 2014, 2016, 2017, and 2019 use @provider=N'MSOLEDBSQL' below (SQL Server Native Client 11.0)---
EXEC master.dbo.sp_addlinkedserver @server = N'ME', @srvproduct=N'', @provider=N'SQLNCLI11', @datasrc=N'localhost', @catalog=N'Northwind'
--For MSSQL 2022 or higher use @provider=N'MSOLEDBSQL' below (Microsoft OLE DB Driver for SQL Server)---
--EXEC master.dbo.sp_addlinkedserver @server = N'ME', @srvproduct=N'', @provider=N'MSOLEDBSQL', @datasrc=N'localhost', @catalog=N'Northwind'

EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'ME',@useself=N'True',@locallogin=NULL,@rmtuser=NULL,@rmtpassword=NULL
GO
--//////////////////////////////////
— Step-2 : Enable RPC OUT and Disable MSDTC
--//////////////////////////////////
EXEC sp_serveroption 'ME', 'rpc out', true;  
GO
--Below needed to support EXEC + INSERT (dynamic query)
EXEC sp_serveroption 'ME', 'remote proc transaction promotion', false;
GO
--test simple query
select * from OPENQUERY(ME,'select 1')
GO

--//////////////////////////////////
— Step-3 : Load data into table from MWS Report
--//////////////////////////////////
SELECT * INTO #amazon_mws_report
FROM OPENQUERY(me,'
SET NOCOUNT ON;
EXEC usp_MWS_Report 
   @Columns=''[item-name], [item-description], [listing-id]'',
   @MarketplaceId =''ATVPDKIKX0DER'', 
   @SellerId=''Axxxxxxxxxxxxxxxxx'', 
   @ReportType = ''_GET_MERCHANT_LISTINGS_DATA_''
WITH RESULT SETS 
(
	(itmname varchar(1000),itmdesc varchar(1000),itmid varchar(1000))
)
')

select * from #amazon_mws_report


*/
CREATE proc [dbo].[usp_MWS_Report]
	@MarketplaceId varchar(100),
	@SellerId varchar(100),
	@ReportType varchar(100)='_GET_MERCHANT_LISTINGS_DATA_',
	@StartDate varchar(100)=null,
	@EndDate varchar(100)=null,
	@ReportId bigint=null,  ---i.e. 14783134522018025
	@ThrowErrIfNoData bit=1,
	@EmptyRowSql varchar(1000)=null, --SQL to run if no data found
	@Columns varchar(8000)='*'  --report columns which goes in final SELECT  e.g. [item-name], [item-description], [listing-id] ....
as


declare @retGeneratedReportId bigint
set @retGeneratedReportId=@ReportId
---////// STEP 1 /////////
if(@ReportId is null)
begin
	create table #step1(ReportRequestId bigint)
	insert into #step1
	exec usp_MWS_RequestReport @ReportType,@MarketplaceId,@SellerId,@StartDate,@EndDate

	--select * from #step1 --debug

	declare @retReportRequestId bigint
	select @retReportRequestId=ReportRequestId from #step1

	---////// STEP 2 /////////
	declare @status varchar(100)
	declare @cnt int
	create table #step2(ReportProcessingStatus varchar(100),GeneratedReportId bigint)
	
	--Keep looping until status is done or we waited too long
	while(isnull(@status,'') IN ('','_SUBMITTED_','_IN_PROGRESS_') )
	begin
		truncate table #step2
		insert into #step2
		exec usp_MWS_GetReportIdAndStatus @retReportRequestId,@MarketplaceId,@SellerId 
	
		select @status=ReportProcessingStatus, @retGeneratedReportId=GeneratedReportId from #step2
		if(isnull(@status,'') IN ('','_SUBMITTED_','_IN_PROGRESS_') )			
		begin
			--amazon says dont check often https://docs.developer.amazonservices.com/en_UK/reports/Reports_GetReportRequestList.html
			Print 'Status='+  isnull(@status,'') +'. Checking after 45 sec... #' + cast(@cnt as varchar(10))
			WAITFOR DELAY '00:00:45'			
		end
		else
			print @status

		set @cnt=@cnt+1

		--if(@cnt>50)
			--//throw err?
	end
	--select * from #step2 --debug
end

if(@retGeneratedReportId is null and @status='_DONE_NO_DATA_')
begin
	if(@ThrowErrIfNoData=1)
		RAISERROR('Cannot generate report - No data found for specified criteria',16,1); 
	else if (isnull(@EmptyRowSql,'')<>'')
		execute(@EmptyRowSql)
end
else if(@retGeneratedReportId is null)
	RAISERROR('Cannot generate report - Unknown error (either report got cancelled or other error occurred)',16,1); 

	
---////// STEP 3 /////////
--If xml format of report data then use XML driver else CSV (most are Tab delimited reports in CSV format)
--exec usp_MWS_GetReportDataXml @retGeneratedReportId,@MarketplaceId,@SellerId
exec usp_MWS_GetReportDataCsv @retGeneratedReportId,@MarketplaceId,@SellerId,@Columns

drop table #step1
drop table #step2

GO

usp_MWS_RequestReport  Stored Proc ( Step1 – Create Report Request )

USE [MyDatabase]
GO

CREATE proc [dbo].[usp_MWS_RequestReport]
	@ReportType varchar(100),
	@MarketplaceId varchar(100),
	@SellerId varchar(100),
	@StartDate varchar(100)=null,
	@EndDate varchar(100)=null
as

/*Learn more : https://docs.developer.amazonservices.com/en_UK/reports/Reports_RequestReport.html */

--//Confirm ISO Date format 2019-06-01T04%3A00%3A00Z  rather 2019-06-01T04:00:00Z
if(isnull(@StartDate,'')<>'')
begin
	set @StartDate='&StartDate=' + @StartDate
	set @StartDate=replace(@StartDate,':','%3A')
	if(@StartDate NOT LIKE @StartDate + 'T%')
		set @StartDate=@StartDate + 'T00%3A00%3A00Z'
	if(@StartDate NOT LIKE @StartDate + '%Z')
		set @StartDate=@StartDate + 'Z'
	--//T04%3A00%3A00Z
end
if(isnull(@EndDate,'')<>'')
begin
	set @EndDate='&EndDate=' + @EndDate
	set @EndDate=replace(@EndDate,':','%3A')
	if(@EndDate NOT LIKE @EndDate + 'T%')
		set @EndDate=@EndDate + 'T00%3A00%3A00Z'
	if(@EndDate NOT LIKE @EndDate + '%Z')
		set @EndDate=@EndDate + 'Z'
end

set @StartDate = isnull(@StartDate,'')
set @EndDate = isnull(@EndDate,'')

declare @sql varchar(max) 
set @sql = 'SELECT * FROM OPENQUERY([AMAZON-MWS-XML]
, ''
SELECT ReportRequestId FROM $
WITH(
	Src=''''https://mws.amazonservices.com/Reports/2009-01-01'''',RequestMethod=''''POST''''
	--,ElementsToTreatAsArray=''''Product''''
	,Filter=''''$.RequestReportResponse.RequestReportResult.ReportRequestInfo''''
	,RequestData=''''Action=RequestReport&ReportType=' + @ReportType  + '&MarketplaceId='+ @MarketplaceId +'&SellerId='+ @SellerId + @StartDate + @EndDate +'''''
	,CacheFileLocation=''''c:\temp\mws-cache-step-1.cache''''
	,CacheEntryTtl=300 --send every 5 mins else use from cache
	,WaitTimeMs=''''300''''
	,IncludeParentColumns=''''False''''
	,Meta=''''[{"Name": "ReportRequestId","Type": "Int64"}]''''
)'')'
--print @sql

EXECUTE( @sql)
--sleep 25 seconds after this - hope report is ready by that time else We have to loop
WAITFOR DELAY '00:00:25'  --lets wait 30 sec... incase we get luck and report is done?
GO

usp_MWS_GetReportIdAndStatus Stored Proc ( Step2 – Get Report Id and Report Status )

USE MyDatabase
GO


CREATE proc [dbo].[usp_MWS_GetReportIdAndStatus]
	@ReportRequestId bigint,
	@MarketplaceId varchar(100),
	@SellerId varchar(100)
as

declare @sql varchar(max) 
set @sql = 'SELECT * FROM OPENQUERY([AMAZON-MWS-XML]
, ''SELECT *  FROM $
WITH(
	 Src=''''https://mws.amazonservices.com/Reports/2009-01-01'''',RequestMethod=''''POST''''
	,ElementsToTreatAsArray=''''ReportRequestInfo''''
	,Filter=''''$.GetReportRequestListResponse.GetReportRequestListResult.ReportRequestInfo''''
	,RequestData=''''Action=GetReportRequestList&ReportRequestIdList.Id.1=' + cast(@ReportRequestId as varchar(20)) + '&MarketplaceId='+ @MarketplaceId +'&SellerId='+ @SellerId +'''''
	,WaitTimeMs=''''300''''
	,IncludeParentColumns=''''False''''
	--,CacheFileLocation=''''c:\temp\mws-cache-step-2.cache''''
	--,CacheEntryTtl=10  --do not use from cache too old.. but cache still prevents too many requests
	,Meta=''''[{"Name": "ReportProcessingStatus","Type": "String", Length: 100},{"Name": "GeneratedReportId","Type": "Int64"}]''''	
)'')'

--print @sql

EXECUTE( @sql)

GO

usp_MWS_GetReportDataCsv Stored Proc (Step3 – Get Report Data in CSV format )

Last step in report generation is download the report data in CSV format.

USE MyDatabase
GO

CREATE proc [dbo].[usp_MWS_GetReportDataCsv]
	@ReportId bigint,
	@MarketplaceId varchar(100),
	@SellerId varchar(100),
	@Columns varchar(8000)='*'

as

declare @sql varchar(max) 
set @sql = 'SELECT '+ @Columns +' FROM OPENQUERY([AMAZON-MWS-CSV], 
''SELECT * FROM $
WITH(
    Src=''''https://mws.amazonservices.com/Reports/2009-01-01'''',RequestMethod=''''POST''''
	,RequestData=''''Action=GetReport&ReportId=' + cast(@ReportId as varchar(20)) + '&MarketplaceId='+ @MarketplaceId +'&SellerId='+ @SellerId +'''''
	,WaitTimeMs=''''300''''
	,ColumnDelimiter=''''{TAB}''''
	,ResponseCharset=''''Windows-1252''''	
	,CacheFileLocation=''''c:\temp\mws-cache-step-3.cache''''
	,CacheEntryTtl=300 --once report is generated lets try to read from cache 
	)''
)'

print @sql

EXECUTE( @sql)
GO

Example – Import data from Amazon MWS into SQL Server Table

Finally when we have all stored procs created we can load into target table as below. In our previous article we saw how to use SELECT INTO along with OPENQUERY to load data into new temp table or insert into some existing table. However when you want to load Stored Proc output into table its not straight forward specially when we calling dynamic SQL.

Three tricks worth mentioning here which makes it possible to load data from stored proc into temp table dynamically.

1. Create a linked server to Self so we can use OPENQUERY against to call stored proc so its easy to use SELECT INTO.
2. Use WITH RESULT SETS to describe metadata of output. This is needed because we are using dynamic SQL inside proc and WITH RESULT SETS describes the metadata.
3. Use of SET NO COUNT ON to avoid error like below
   

   "SOMESERVERNAME" indicates that either the object has no columns 
   or the current user does not have permissions on that object

4. If you dont do above tricks then you may see errors like this (INSERT is not allowed when we use DYNAMIC SQL inside nested Stored Procs)

To load output into table you must know column names from the output.  If you dont know you can try to run stored proc without @Columns like below.

EXEC usp_MWS_Report @MarketplaceId =''ATVPDKIKX0DER'', @SellerId=''Axxxxxxxxxxxxx'', @ReportType = ''_GET_MERCHANT_LISTINGS_DATA_''

Once we know columns from report, you can run following script to insert into table.

Full Script – Load MWS Report Output into Table

--//////////////////////////////////
— Step-1 : Create Local Linked server to point to self
--//////////////////////////////////
USE [master]
GO
--change default catalog
--For MSSQL 2012, 2014, 2016, 2017, and 2019 use @provider=N'MSOLEDBSQL' below (SQL Server Native Client 11.0)---
EXEC master.dbo.sp_addlinkedserver @server = N'ME', @srvproduct=N'', @provider=N'SQLNCLI11', @datasrc=N'localhost', @catalog=N'Northwind'
--For MSSQL 2022 or higher use @provider=N'MSOLEDBSQL' below (Microsoft OLE DB Driver for SQL Server)---
--EXEC master.dbo.sp_addlinkedserver @server = N'ME', @srvproduct=N'', @provider=N'MSOLEDBSQL', @datasrc=N'localhost', @catalog=N'Northwind'

EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'ME',@useself=N'True',@locallogin=NULL,@rmtuser=NULL,@rmtpassword=NULL
GO
--//////////////////////////////////
— Step-2 : Enable RPC OUT and Disable MSDTC
--//////////////////////////////////
EXEC sp_serveroption 'ME', 'rpc out', true;  
GO
--Below needed to support EXEC + INSERT (dynamic query)
EXEC sp_serveroption 'ME', 'remote proc transaction promotion', false;
GO
--test simple query
select * from OPENQUERY(ME,'select 1')
GO

--//////////////////////////////////
— Step-3 : Load data into temp table from Amazon MWS Report
--//////////////////////////////////
SELECT * INTO #amazon_mws_report
FROM OPENQUERY(me,'
SET NOCOUNT ON;
EXEC usp_MWS_Report 
   @Columns=''[item-name], [item-description], [listing-id]'',
   @MarketplaceId =''ATVPDKIKX0DER'', 
   @SellerId=''Axxxxxxxxxxxxx'', 
   @ReportType = ''_GET_MERCHANT_LISTINGS_DATA_''
WITH RESULT SETS 
(
	(itmname varchar(1000),itmdesc varchar(1000),itmid varchar(1000))
)
')

select * from #amazon_mws_report

Import Amazon MWS Data into SQL Table (Load Report output into temp table)

Conclusion

In this article, we saw how to load Amazon MWS Data into SQL Server. We used simple SQL queries with ZappySys XML / CSV Driver to pull data from Amazon MWS API. ZappySys ODBC PowerPack makes it super easy to consume data from virtually any API (JSON / XML or CSV Web Services), no coding required. Download here to get started with ZappySys ODBC drivers.

The post How to import Amazon MWS data into SQL Server (T-SQL) appeared first on ZappySys Blog.

In this article, you will see few Examples to import REST API in SQL Server Table (Call JSON / XML SOAP API).  You will also learn how to load JSON Files into SQL Server Table using T-SQL Stored procedure code.  So let’s get ready for fun 🙂

JSON File format and REST API (i.e. RESTful API ) is becoming more and more popular each day. You may already have some data integration scenarios to read/write From REST API services in SQL Server or maybe API integration in other Apps such as Power BI, Tableau, SSIS, Informatica.

REST API integration can be challenging without the right set of tools. Even for highly skilled programmer sometimes its difficult to achieve the desired result.  But not to worry at the end of this article you will find out how easy it is to pull data from JSON / XML based web service using familiar SQL Query Language.  In the next few sections, you will learn how to call REST API inside SQL Server using familiar T-SQL code.

Steps to Import JSON / XML REST API in SQL Server

Here is the summary of steps to query REST API / JSON File in SQL Server (Using T-SQL code). For detailed steps read the full article and watch the video tutorial.

1. Install ZappySys ODBC PowerPack  from here (it contains drivers for JSON / XML API)
2. Create ODBC DSN for JSON / XML API Source
3. Configure ZappySys Data Gateway Service so SQL Server can use ZappySys Drivers in T-SQL code
4. Create and Configure Linked Server to connect to JSON / REST API Service
5. Write some T-SQL code in your stored procedure like below to import data from JSON / REST Service
   

   Example – Import JSON / XMLFiles / REST API Data into SQL Server Table ( Call API using T-SQL Stored Procedure)

Video Tutorial – Read from JSON / XML / REST API in SQL Server (T-SQL code)

If you want to save time and go quick then check out following video tutorial on calling JSON / REST API Service in SQL Server code. Steps The following video will be useful to configure your Linked Server and query REST API. It shows how to configure ODBC DSN, ZappySys Data Gateway Server (Used for Linked Server). To see other use cases of ZappySys Data Gateway click here

Requirements

1. A first requirement will be to SQL Server Database Engine Installed
2. The second requirement will be SSMS installed
3. Make sure to have ZappySys ODBC PowerPack installed.
4.  Another important requirement is the internet connection to connect to REST API information using a URL.

Step-By-Step : Load REST API in SQL Server

In next few sections, you will see step-by-step approach to consume REST API / JSON data in SQL Server using T-SQL code.

Configure ZappySys Data Gateway Service

To call external Drivers (e.g. ZappySys JSON/REST API Driver) in SQL Server we have to use Linked Server feature. We will use ZappySys Data Gateway Service along with SQL Linked Server to call ZappySys Drivers (e.g. JSON / XML API Drivers) using T-SQL Code in Stored Procedures.

Why use ZappySys Data Gateway Service?

Gateway Service is useful to call ZappySys driver from any other machine without installing ZappySys driver. For example your Unix server can consume API data by connecting to ZappySys Data Gateway (which can be running on Remote Windows OS). Many times you cannot access remote server but like to Query API data (Google SpreadSheet Use case). To learn more about other use cases click here.

Call REST API in SQL Server (T-SQL Code) using Linked Server

Optionally, use these lines of code to create the linked server using T-SQL instead of using the UI:

/* Add Linked Server called RESTAPI-SV (change if needed) to access REST API via Data Gateway - JSON/REST Driver */
/* customer_api is just an example data source name - created on Data gateway UI */

EXEC master.dbo.sp_addlinkedserver 
	@server = N'RESTAPI-SV', 
	@srvproduct=N'', 
--- For MSSQL 2012, 2014, 2016, 2017, and 2019 use below (SQL Server Native Client 11.0)---
    @provider=N'SQLNCLI11',
--- For MSSQL 2022 or higher use below (Microsoft OLE DB Driver for SQL Server)---
--- @provider=N'MSOLEDBSQL',
	@datasrc=N'localhost,5000', 
	@catalog=N'customer_api'

/* Add login for Linked Server - This is same user account and password which you define on Users tab on ZappySys Data Gateway Config Tool */
EXEC master.dbo.sp_addlinkedsrvlogin 
	@rmtsrvname=N'RESTAPI-SV',
	@useself=N'False',
	@locallogin=NULL,
	@rmtuser=N'admin',
	@rmtpassword='adminP@ss!23'
GO
--//OTHER IMPORTANT OPTIONS
EXEC master.dbo.sp_serveroption @server=N'RESTAPI-SV', @optname=N'rpc', @optvalue=N'true'
GO

EXEC master.dbo.sp_serveroption @server=N'RESTAPI-SV', @optname=N'rpc out', @optvalue=N'true'
GO

--Increase default timeout for long running query (default is 600 sec)
--EXEC master.dbo.sp_serveroption @server=N'RESTAPI-SV', @optname=N'query timeout', @optvalue=N'1200'
GO

--Below needed to support EXEC + INSERT (dynamic query)
EXEC master.dbo.sp_serveroption @server=N'RESTAPI-SV', @optname=N'remote proc transaction promotion', @optvalue=N'false'
GO

1. Now it is time to create some queries. Let’s run a simple query to REST API:
   

   select * from OPENQUERY([RESTAPI-SV] , 'SELECT * FROM value' )

2. The query will show all the data from REST API:
   

   REST API QUERY IN SQL SERVER

    

3. Also, the following query can be used when you need to use filters (Notice that we escaped single tick with double tick)

   select * from OPENQUERY([RESTAPI-SV] , 'SELECT * FROM value where ContactTitle=''Owner''' )

4. Store the data in a temporary table using the following query:

   select * INTO #tmpcustomer from OPENQUERY([RESTAPI-SV] , 'SELECT * FROM value' )

5. And now you can query and use the SQL table:

   SELECT * FROM #tmpcustomer t

6. You can also check the metadata, values and odata_next link:

   select * from OPENQUERY([RESTAPI-SV] , 'SELECT * FROM $' )

    

Call REST API in SQL Server (T-SQL Code) using Polybase (For SQL 2019 or higher)

How to use Dynamic SQL with parameters

So far we saw how to use OPENQUERY statement with a static SQL query. Now let’s look at the dynamic Query approach to Read REST API data and Load into SQL Table. For Dynamic SQL you have to use EXEC rather than OPENQUERY.

--MSDTC option must be off for INSERT + EXEC pattern (Find Linked Server Properties | Server Options | Enable Promotion of Distributed Transaction - [Set to false]
--OR-- run via code like below
--EXEC master.dbo.sp_serveroption 'MyLinkedServer', 'remote proc transaction promotion', false;

declare @tbl nvarchar(100) = 'value'
--Just output the query data (no insert)
EXEC('select col1,col2 from ' + @tbl) AT [MyLinkedServer]

--Load into table (dont output)
insert into _tmp_invoices(col1,col2) 
EXEC('select col1,col2 from ' + @tbl) AT [MyLinkedServer]

--We recommend using column names in the list rather than * (all) to avoid metdata issue
--Use Data Source > Preview UI > Table dropdown to generate column names

How to parameterize API calls in SQL Server

Now you know how to write simple SQL queries to fetch data from API. In real world you will have a need to pass parameters to your API. You can pass parameters to your API following ways.  Notice that how we have escaped single tick. In this example we have passed Header, Body and URL using Dynamic approach. In your case you may need to pass only one way (e.g. Pass parameter via URL only)

declare @url varchar(100)='http://httpbin.org/post' 
declare @body varchar(100)='{id:1,notes:"Line1\r\nLine2"}'
declare @header varchar(100)='Content-Type:text/plain||x-hdr1:AAA'

declare @sql varchar(1000)
declare @sqlForLs  varchar(1000) = 
'SELECT * FROM $
WITH  (
SRC='''+ replace(@url,'''', '''''')  +''' ,
METHOD=''POST'' ,
HEADER='''+ replace(@header,'''', '''''')  +''' ,
BODY='''+ replace(@body,'''', '''''')  +''' 
)'
 
set @sql = 'SELECT * into #tmp from OPENQUERY(LS,''' + replace(@sqlForLs,'''', '''''') + ''')'
print @sql

EXECUTE(@sql);

How to Import JSON File to SQL Server (Single or Multiple)

Not only you can query REST API in SQL Server but also you can query a JSON file. The steps are all the same except in one step. Instead of specifying a URL in the ZappySys ODBC driver (see Add the ZappySys driver in the Windows ODBC Administrator section, step 7) and specify the file path and set the format to Default:

Add properties JSON file

If you dont want to hard code file name in the DSN then you can also use following Query syntax to specify URL or File Path inside SQL Query

select * into #tmp from OPENQUERY([JSON-SV], 'SELECT * FROM value WITH (SRC=''C:\test\cust2012.json'')' )

How to load multiple JSON files into SQL Server Table

Let’s say that you have multiple JSON files in a folder (similar structure). The file names are customer_2012.json, customer_2013.json, customer_2014.json and they are all in the same folder. Is it possible to query all of them in SQL Server without any extra looping code?

The answer is yes. You just need to create the linked server to a single JSON file and then run this query (JSON-SV is the name of the linked server):

select * into #tmp from OPENQUERY([JSON-SV], 'SELECT * FROM value WITH (SRC=''C:\test\cust*.json'')' )

We are just using this code:

(SRC=''C:\test\cust*.json'')' )

SRC means source and it will search all the files with json extension starting with the word “cust”.

How to read and parse JSON string in T-SQL

In SQL Server 2016 and later, it is possible to read JSON data. However in older versions, it was not possible. The following example will show how to parse JSON data using a Linked Server named JSON-SV.

1. 1. We will first store JSON values in a SQL Server table in a column named MyData:
      

      create table #tmp(MyData nvarchar(max))
      
      insert into #tmp values(
      '{ 
      rows:[ 
      {id:1, name:"Sam''s Club"}, 
      {id:2, name:"Walmart"}, 
      {id:3, name: "Facebook"} 
      ]
      }')

   2. We just created a temporary table named tmp and in the column MyData, we stored JSON values. In the next step, we will store the values of the temporary table in a variable named json and store and parse the values en the variable sql:
      

      declare @json nvarchar(max)
      declare @sql nvarchar(max)
      
      select @json = MyData from #tmp
      
      
      set @sql='select * from OPENQUERY([JSON-SV] , ''SELECT * FROM rows WITH
      (DATA=@'''''+ Replace(@json ,'''','''''''''') +''''')'' );'
      print(@sql)
      execute(@sql)

   3. If everything is OK, you will be able to see the JSON values parsed in a nice table format:
      

      JSON values parsed

How to Call REST API from a SQL Stored Procedure

How can we call REST API using a stored procedure? We need to use dynamic queries to pass parameters. The following example will run a query where the city is specified in a variable.

1. We will create the stored procedure first:
   

   CREATE PROCEDURE getcity 
   @City NVARCHAR(50) 
   AS 
   DECLARE @sql NVARCHAR(500)='select * from OPENQUERY([RESTAPI-SV] , 
   ''SELECT * FROM value where City=''''' + @City + ''''''')' 
   EXECUTE(@sql)

2. The stored procedure name is getcity. We will call the RESTAPI-SV that calls the REST API and pass the parameter City in the query. We finally execute the query stored in the variable using the Execute statement.
3. Finally, we just invoke the stored procedure:
   

   EXECUTE getcity @City = N'Berlin'

   The query will display all the data with the city equal to Berlin.

How to POST data to REST API in SQL Server (Send data to URL)

The following example shows how to POST data to API service where data is coming from an embedded string in SQL Server using a Linked Server. I am assuming that you already created a linked server following previews steps explained before.

POST data to URL (embedded Body)

In SSMS run this query:

select * from OPENQUERY([RESTAPI-SV] , 'SELECT * FROM $ 
WITH (
 METHOD=''POST''
,HEADER=''Content-Type: application/json || x-hdr1: AAA''
,SRC=''http://httpbin.org/post''
,BODY=''{ id:2,  notes2:"Line1\r\nLine2" }''
)' 
)

The query will invoke the Linked Server for REST API, created before and use the POST Method with the embedded strings.

POST data to URL (read body from file)

Here is an example to POST data to URL where Body is read from Local file. Notice two additional attributes in below.  We set IsMultiPart=true and also Body has @ symbol before file path.

select * from OPENQUERY([RESTAPI-SV] , 'SELECT * FROM $ 
WITH (
 METHOD=''POST''
,HEADER=''Content-Type: application/json || x-hdr1: AAA''
,SRC=''http://httpbin.org/post''
,BODY=''@c:\data\api_call_body.json''
,IsMultiPart='True'
)' 
)

Avoiding errors with POST / PUT / DELETE  requests to create / modify /delete data

There will be a time you may get error when you issue POST requests because same request sent multiple times and duplicate records got created. To avoid such situation refer to this article (Read Handling POST request section)

Pagination Concepts for REST API / SOAP XML  in SQL Server

SELECT * FROM $
WITH(
SRC=@'https://zappysys.com/downloads/files/test/pagination_nextlink_inarray_1.json'
,NextUrlAttributeOrExpr = '$.nextlink'  --keep reading until this attribute is missing. If attribute name contains dot then use brackets like this $.['my.attr.name']
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/page-xml.aspx?page=1&mode=DetectBasedOnResponseStatusCode'
,PagingMode='ByUrlParameter'
,PagingByUrlAttributeName='page'
,PagingByUrlEndStrategy='DetectBasedOnResponseStatusCode'
,PagingByUrlCheckResponseStatusCode=401
,IncrementBy=1
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/cust-<%page%>.xml'
,PagingMode='ByUrlPath'
,PagingByUrlAttributeName='<%page%>'
,PagingByUrlEndStrategy='DetectBasedOnRecordCount'
,IncrementBy=1
)

SELECT * FROM $
LIMIT 25
WITH(
	 Src='https://wordpress.org/news/wp-json/wp/v2/categories?per_page=10'
	,PagingMode='ByResponseHeaderRfc5988'
	,WaitTimeMs='200' --//wait 200 ms after each request
)

Error Handling for REST API / SOAP XML in SQL Server

Sometimes errors occur... they just do and there is nothing you can do! Or can you? Actually, in ODBC PowerPack you can handle them in two ways.

METHOD 1 - Using Error Handling Options

When to use?

You may want to use them when your source is a resource located on the Internet; e.g. a file on a website, a file on an FTP server or just a plain API HTTP response. By default, when a remote server returns an error, data retrieval is stopped, an error is raised and no data is given back to you. This might not be always desirable.

Scenario 1

Imagine a scenario, that there is a web server which each day at 12 AM releases a new JSON file with that day's date as filename, e.g. http://www.some-server.com/data/2018-06-20.json. And, of course, you want to download it and use it daily in your Power BI report. But you have a problem: Power BI report data sources are refreshed each hour and you may get HTTP 404 status code (no file was found) when a file is not released yet. Which consequentially means other data sources won't be updated as well and you will see old and cached data on the report. That's where you could use Continue on any error or Continue when Url is invalid or missing (404 Errors) to avoid an error being raised and let other data sources to be updated.

Scenario 2

Another scenario is when you expect a web server to raise some kind of HTTP error when accessing a URL. You don't want ODBC Data Source to raise an error but instead, you want to get response data. That's where you can use Continue on any error or alike together with  Get response data on error to continue on an error and get the data:

METHOD 2 - Using Connection [Retry Settings]

Another scenario you may run into is a buggy web server. You ask it to give you some file or data and it, like a snotty kid, just doesn't give it to you! You have to ask twice or thrice before it does its job. If that's the case, you have to retry HTTP requests using Connection:

OAuth / HTTP Connection in SQL Server

If you need to authenticate or authorize your user to access a web resource, you will need to use one of the Connections:

• HTTP
• OAuth

HTTP Connection

Use HTTP Connection for simple Windows, Basic, NTLM or Kerberos authentication. Just fill in a username and a password and you are good to go!

You can also use HTTP Connection for more sophisticated authentication like:

• SOAP WSS (when accessing a SOAP WebService)
• Static Token / API Key (when need to pass an API key in HTTP header)
• Dynamic Token (same as Static Token method except that each time you need to log in and retrieve a fresh API key)
• JWT Token (As per RFC 7519)

OAuth

If you are trying to access REST API resource, it is a huge chance, you will need to use OAuth Connection. Read this article to understand how OAuth authentication and authorization works and how to use it (article originally was written for SSIS PowerPack, but the concepts and UI stay the same): 
https://zappysys.com/blog/rest-api-authentication-with-oauth-2-0-using-ssis/

Performance Tips for REST API / XML SOAP Calls

SELECT * FROM value
WITH(
	 Src='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json&$filter=Country eq ''USA'''
	,DataFormat='Odata'
)

-- Slow query - Client-side filtering
SELECT * FROM value
WHERE Country ='USA'
WITH(
	 Src='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json'
	,DataFormat='Odata'
)

SELECT * FROM $
WITH 
(  SRC='https://myhost.com/some-api'
  ,CachingMode='All'  --cache metadata and data rows both
  ,CacheStorage='File' --or Memory
  ,CacheFileLocation='c:\temp\myquery.cache'
  ,CacheEntryTtl=300 --cache for 300 seconds
)

SELECT * FROM $ 
LIMIT 10 --//add this just to test how fast you can get 10 rows
WITH(
  Filter='$.LargeArray[*]--FAST' --//Adding --FAST option turn on STREAM mode (large files)
 ,SRC='https://zappysys.com/downloads/files/test/large_file_100k_largearray_prop.json.gz'
 --,SRC='c:\data\large_file.json.gz'
 ,IncludeParentColumns='False'  --//This Must be OFF for STREAM mode (read very large files)
 ,FileCompressionType='GZip' --Zip or None (Zip format only available for Local files)
)

Other Considerations for Calling Web API in SQL Server

Calling XML SOAP Web Service in SQL Server

So far we have looked at examples to consume data using JSON driver. Now let’s look at an example, to call XML SOAP Web Service.

To call SOAP API you need to know Request XML Body Structure. If you are not sure how to create SOAP Request body then no worries. Check this article to learn how to generate SOAP Request body using the Free tool SoapUI. Basically, you have to use SoapUI to generate Request XML and after that, you can replace parameters as needed in the generated body.

What is SOAP Web Service?

If you are new to SOAP Web Service sometimes referred as XML Web Service then please read some concept about SOAP Web service standard from this link There are two important aspects in SOAP Web service.

1. Getting WSDL file or URL
2. Knowing exact Web Service URL

What is WSDL

In very simple term WSDL (often pronounced as whiz-dull) is nothing but a document which describes Service metadata (e.g. Functions you can call, Request parameters, response structure etc). Some service simply give you WSDL as xml file you can download on local machine and then analyze or sometimes you may get direct URL (e.g. http://api.mycompany.com/hr-soap-service/?wsdl )

Example SQL Query for SOAP API call using ZappySys XML Driver

Here is an example SQL query you can write to call SOAP API. If you not sure about many details then check next few sections on how to use XML Driver User Interface to build desired SQL query to POST data to XML SOAP Web Service without any coding.

SELECT * FROM $
WITH(
	 Src='http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx'
	,DataConnectionType='HTTP'
	,CredentialType='Basic' --OR SoapWss
	,SoapWssPasswordType='PasswordText'
	,UserName='myuser'
	,Password='pass$$w123'
	,Filter='$.soap:Envelope.soap:Body.GetHolidaysAvailableResponse.GetHolidaysAvailableResult.HolidayCode[*]'
	,ElementsToTreatAsArray='HolidayCode'	
	,RequestMethod='POST'	
	,Header='Content-Type: text/xml;charset=UTF-8 || SOAPAction: "http://www.holidaywebservice.com/HolidayService_v2/GetHolidaysAvailable"'
	,RequestData='
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:hol="http://www.holidaywebservice.com/HolidayService_v2/">
   <soapenv:Header/>
   <soapenv:Body>
      <hol:GetHolidaysAvailable>
         <!--type: Country - enumeration: [Canada,GreatBritain,IrelandNorthern,IrelandRepublicOf,Scotland,UnitedStates]-->
         <hol:countryCode>UnitedStates</hol:countryCode>
      </hol:GetHolidaysAvailable>
   </soapenv:Body>
</soapenv:Envelope>'
)

Now let's look at steps to create SQL query to call SOAP API. Later we will see how to generate code for your desired programming language (e.g. C# or SQL Server)

Video Tutorial - Introduction to SOAP Web Service and SoapUI tool

Before we dive into details about calling SOAP API using ZappySys XML Driver, lets first understand what is SOAP API and how to create SOAP requests using SoapUI tool. You will learn more about this process in the later section. The video contains some fragment about using SOAP API in SSIS but just ignore that part because we will be calling Soap API using ZappySys ODBC Driver rather than SSIS Components.  

Using SoapUI to test SOAP API call / Create Request Body XML

Assuming you have downloaded and installed SoapUI from here, now we are ready to use WSDL for your SOAP Web Service Calls. If you do not have WSDL file or URL handy then contact your API provider (sometimes you just have to add ?wsdl at the end of your Service URL to get WSDL so try that. Example: http://mycompany/myservice?wsdl ). If you don't know what is WSDL then in short, WSDL is Web service Description Language (i.e. XML file which describes your SOAP Service). WSDL helps to craft SOAP API request Body for ZappySys XML Driver. So Let's get started.

1. Open SoapUI and click SOAP button to create new SOAP Project
2. Enter WSDL URL or File Path of WSDLFor example WSDL for our sample service can be accessed via this URL

   http://www.dneonline.com/calculator.asmx?wsdl

   
   Create new SOAP API Project in SoapUI tool for SOAP API Testing
3. Once WSDL is loaded you will see possible operations you can call for your SOAP Web Service.
4. If your web service requires credentials then you have to configure it. There are two common credential types for public services (SOAP WSS or BASIC )
   1. To use SOAP WSS Credentials select request node and enter UserId, Password, and WSS-PasswordType (PasswordText or PasswordHash)
      
      Configure SOAP WSS Credentials for SoapUI (SOAP API Testing Tool)
   2. To use BASIC Auth Credentials select request node and double-click it. At the bottom click on Auth (Basic) and From Authorization dropdown click Add New and Select Basic.
      Configure Basic Authorization for SoapUI (SOAP API Testing Tool)
5. Now you can test your request first Double-click on the request node to open request editor.
6. Change necessary parameters, remove optional or unwanted parameters. If you want to regenerate request you can click on Recreate default request toolbar icon.
   Create SOAP Request XML (With Optional Parameters)
7. Once your SOAP Request XML is ready, Click the Play button in the toolbar to execute SOAP API Request and Response will appear in Right side panel. Test SOAP API using SoapUI Tool (Change Default XML Body / Parameters, Execute and See Response)

Create DSN using ZappySys XML Driver to call SOAP API

Once you have tested your SOAP API in SoapUI tool, we are ready to use ZappySys XML driver to call SOAP API in your preferred BI tool or Programming language.

1. First open ODBC Data Sources (search ODBC in your start menu or go under ZappySys > ODBC PowerPack > ODBC 64 bit)
2. Goto System DSN Tab (or User DSN which is not used by Service account)
3. Click Add and Select ZappySys XML Driver ZappySys ODBC Driver for XML / SOAP API
4. Configure API URL, Request Method and Request Body as below ZappySys XML Driver - Calling SOAP API - Configure URL, Method, Body
5. (This step is Optional) If your SOAP API requires credentials then Select Connection Type to HTTP and configure as below.
   ZappySys XML Driver - Configure SOAP WSS Credentials or Basic Authorization (Userid, Password)
6. Configure-Request Headers as below (You can get it from Request > Raw tab from SoapUI after you test the request by clicking the Play button) Configure SOAP API Request Headers - ZappySys XML Driver
7. Once credentials entered you can select Filter to extract data from the desired node. Make sure to select array node (see special icon) or select the node which contains all necessary columns if you don't have array node. Select Filter - Extract data from nested XML / SOAP API Response (Denormalize Hierarchy)
8. If prompted select yes to treat selected node as Array (This is helpful when you expect one or more record for selected node) Treat selected node as XML Array Option for SOAP API Response XML

Preview SOAP API Response / Generate SQL Code for SOAP API Call

Once you configure settings for XML Driver now you can preview data or generate example code for desired language (e.g. C#, Python, Java, SQL Server). Go to Preview tab and you will see default query generated based on settings you entered in previous sections. Attributes listed in WITH clause are optional. If you omit attribute in WITH clause it will use it from Properties tab.

Preview Data

Preview SOAP API Response in ZappySys XML Driver

Generate Code Option

Generate Example Code for ZappySys Driver

OPENQUERY vs EXEC AT (Handling Larger SQL Text)

So far we have seen examples of using OPENQUERY. It allows us to send pass-through query at remote server. The biggest limitation of OPENQUERY is it doesnt allow you to use variables inside SQL so often we have to use unpleasant looking dynamic SQL (Lots of tick, tick ….  and escape  hell). Well there is good news. With SQL 2005 and later you can use

EXEC( your_sql ) AT your_linked_server syntax

Disadvantage of EXEC AT is you cannot do SELECT INTO like OPENQUERY. Also you cannot perform JOIN like below in EXEC AT

SELECT a.* FROM OPENQUERY(ls_json,'select * from $') a 
JOIN  OPENQUERY(ls_json,'select * from $') b ON a.id=b.id

However you can always do INSERT INTO MyTable EXEC(…) AT LINKEDSRV. So table must exists when you do that way.

Here is how to use it. To use EXEC AT you must turn on RPC OUT option. Notice how we used variable in SQL to make it dynamic. This is much cleaner than previous approach we saw.

USE [master]
GO

EXEC master.dbo.sp_addlinkedserver @server = N'ls_Json', @srvproduct=N'', @provider=N'SQLNCLI', @datasrc=N'localhost,5000', @provstr=N'Network Library=DBMSSOCN;', @catalog=N'JsonApi'
EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'ls_Json',@useself=N'False',@locallogin=NULL,@rmtuser=N'tdsuser',@rmtpassword='########'
GO

EXEC sp_serveroption 'ls_Json', 'rpc out', true;  
go

declare @tbl varchar(100)='$'
EXEC('select * from ' + @tbl ) AT ls_Json

Here is the difference between OPENQUERY vs EXEC approaches.

SQL Server OPENQUERY vs EXEC for Linked Server – Handling Larger SQL Text (More than 8000 chars)

If you decide to use EXEC instead of OPENQUERY then make sure below setting is ON else you will get an error.

Under the Linked Server Options, Enable RPC and RPC Out and Disable Promotion of Distributed Transactions(MSDTC).

RPC OUT setting for EXEC AT statement in SQL Server Linked Server

Configure Firewall Settings

So far we have assumed that Gateway is running on the same machine as SQL Server. However there will be a case when ZappySys ODBC PowerPack is installed on a different machine than SQL Server.  In such case you may have to perform additional Firewall configurations. On most computers firewall settings wont allow outside traffic to ZappySys Data Gateway. In such case perform following steps to allow other machines to connect to Gateway.

1. Search for Windows Firewall Advanced Security in start menu.
2. Under Inbound Rules > Right click and click [New Rule] >> Click Next
3. Select Port on Rule Type >> Click Next
4. Click on TCP and enter port number under specified local port as 5000 (use different one if you changed Default port) >> Click Next
5. Select Profile (i.e. Private, Public) >> Click Next
6. Enter Rule name [i.e. ZappySys Data Gateway – Allow Inbound ] >> Click Next
7. Click OK to save the rule

ZappySys Datagateway – Allow Inbound Traffic on Port 5000 Firewall Rule

Known Issues

In this section we will discuss some known issues when using OPENQUERY / Data Gateway Connectivity

Error: The data is invalid

There will be a time when you start facing odd errors such as below. Possible

OLE DB provider "SQLNCLI11/MSOLEDBSQL" for linked server "Zs_Csv" returned message "Deferred prepare could not be completed.".
OLE DB provider "SQLNCLI11/MSOLEDBSQL" for linked server "Zs_Csv" returned message "Communication link failure".
Msg 13, Level 16, State 1, Line 0

Session Provider: The data is invalid.

Possible Cause:

There are few reasons for such error but below are two main reasons

• If query length is more than 2000 Characters long such as below then you might get such error
  

  SELECT * FROM OPENQUERY(LS, '--some really long text more than 2000 chars--')

• If query contains multiple OPENQUERY statements for JOIN / UNION like below then also it might fail (Gateway doesnt support parallel queries on a single connection – MARS compatibility issue).
  

  select a.id, b.name from OPENQUERY(LS, 'select * from tbl1') a join OPENQUERY(LS, 'select * from tbl2') b on a.id=b.id

Possible Fix:

There are few ways to fix above error based on reason why you getting this error (i.e. Query Length issue OR JOIN/UNION in the same statement)

• If your query has long SQL (more than 2000 chars ) then reduce SQL length using different techniques
  • e.g. use SELECT * FROM MyTable rather than SELECT col1,col2… FROM MyTable
  • Use Meta Option in WITH clause if you must use column name. (e.g. SELECT * FROM MyTable WITH(META=’c:\meta.txt’) this way you can define column in Meta file rather than SELECT query. Check this article
  • Consider using EXECT (….) AT [Linked_Server_name] option rather than OPENQUERY so you can use very long SQL (See next section on EXEC..AT usecase)
  • Consider using Virtual Table / Stored Proc to wrap long SQL so your call is very small (where usp_GetOrdersByYear is custom proc created on ZappySys Driver UI)
    

    SELECT * FROM OPENQUERY(LS, 'EXEC usp_GetOrdersByYear 2021')

• If your query uses JOIN  / UNION with multiple OPENQUERY in same SQL then use multiple Linked servers (one for each OPENQUERY clause) as below.
  

  select a.id, b.name from OPENQUERY(LS_1, 'select * from tbl1') a join OPENQUERY(LS_2, 'select * from tbl2') b on a.id=b.id

Error: Unable to begin a distributed transaction (When INSERT + EXEC used)

If you try to use EXEC statement and try to insert data into table (like below) you might get below error unless MSDTC option is turned off.

INSERT INTO MyTable EXEC('select * from tbl') AT MyLinkedServer

"Protocol error in TDS stream"
The operation could not be performed because OLE DB provider "SQLNCLI11" for linked server "ls_Json2" was unable to begin a distributed transaction.
--OR--
The operation could not be performed because OLE DB provider "MSOLEDBSQL" for linked server "ls_Json" was unable to begin a distributed transaction.

Solution:
Method-1: Go to linked server properties | Server Options | Enable Promotion of Distributed Transaction | Change to false (Default is true)
Now your try your INSERT with EXEC AT and it should work

Method-2: Run the below command if you dont want to use UI

EXEC master.dbo.sp_serveroption @server=N'My_Linked_Server', @optname=N'remote proc transaction promotion', @optvalue=N'false'

Error: Cannot use OPENQUERY with JOIN / UNION

When you perform JOIN  / UNION ALL on the same Linked Server it may fail to process sometimes because Datagateway doesnt support parallel query requests on the same connection. Workaround for that would be create multiple linked servers for the same datasource. See above section for the same workaround.

Error: Truncation errors due to data length mismatch

Many time you may receive truncation errors if Table column is length is less than actual column size from query column. To solve this issue use New version of Data gateway ( check – Use nvarchar(max) for string options – Found on General Tab)

Performance Tips

Now lets look at few performance tips

Use INSERT INTO rather than SELECT INTO to avoid extra META request

We discussed some Pros and Cons of OPENQUERY vs EXEC (…) AT in previous section. One obvious advantage of EXEC (….) AT is it reduces number of requests to driver (It sends pass through query). With EXEC you cannot load data dynamically like SELECT INTO tmp FROM OPENQUERY. Table must exist before hand if you use EXEC.

INSERT INTO tmp_API_Report_Load(col1,col2)
EXEC('select col1,col2 from some_api_table') AT [API-LINKED-SERVER]
--OR--
INSERT INTO tmp_API_Report_Load(col1,col2)
select col1,col2 from OPENQUERY([API-LINKED-SERVER], 'select col1,col2 from some_api_table')

Advantage of this method is your Query speed will increase because system calls API only once when you call EXEC AT. In OPENROWSET it needs to call above query twice (Once to obtain metadata and once to get data).

Use Cached Metadata if possible

By default, most of the SQL sent to Data gateway need to invoke two phases. First, get metadata and second fetch data. You can bypass meta API call by supplying static metadata. Use META property in WITH clause as per this article to speed up your SQL.

Conclusion

To conclude, we can say that using ZappySys JSON / XML Drivers we can query JSON / XML Files and call REST API or SOAP Web Service using familiar SQL Language. All you need to do is install ZappySys ODBC Drivers, configure it in the ODBC DSN and use ZappySys Gateway as a Proxy. Finally, you add a Linked Server in SQL Server to connect to ZappySys Data Gateway. Feel free to download  ZappySys Drivers here. You can contact ZappySys Team here if you need any API integration help.

References

• ZappySys ODBC PowerPack online help
• JSON data in SQL Server
• Linked Servers (Database Engine)

Keywords: Call REST API, SQL SERVER, JSON

The post How to import REST API in SQL Server (Call JSON / XML SOAP Service) appeared first on ZappySys Blog.