Export REST API to CSV is in some cases necessary to process the data because many tools can handle CSV files. In this new article, we will show different ways to export the data. The first example will do it using C#. The second example with use Python. Let’s take a look at these examples.

Requirements to export REST API to CSV

1. First of all, you will need ZappySys ODBC PowerPack installed.
2. Secondly, you will require Visual Studio installed for the C# example.
3. Finally, it is necessary Python installed for the Python example.

Export REST API to CSV using C#

C# is a pretty popular programing language. In the first example, we will show how to display the REST API information to a CSV file named. Let’s take a look at the code:

1. First of all, we will connect to REST API using a connection to the following Data Path:
   

   https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json

   using (OdbcConnection conn = 
         new OdbcConnection("Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))

2. Secondly, we create a connection using a file stream. We will save the results to a file in the c:\sql\sample.csv:
   

   FileStream fs = new FileStream("C:\\sql\\sample.txt", FileMode.Create);
   StreamWriter writer = new StreamWriter(fs);
   StringBuilder output = new StringBuilder();

3. Also, we will add a query to REST API. Note that with the ZappySys ODBC PowerPack, you can do a simple SQL query to get REST API data. This is pretty simple and intuitive:
   

   OdbcCommand cmd = new OdbcCommand(
        @"SELECT  CustomerID,CompanyName FROM value", conn);

4. In addition, we will read the data and close the connections:
   

   conn.Close();  
   writer.Close(); 
   fs.Close();

5. The complete code will be the following:
   

   using System;
   using System.IO;
   using System.Data.Odbc;
   
   public class Program
   {
       public static void Main()
       {
           var outpath = @"C:\temp\sample.txt";
   
           using (var conn = new OdbcConnection("Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))
           {
               conn.Open();
               var cmd = new OdbcCommand(@"SELECT CustomerID,CompanyName FROM $ WITH(Filter='$.value[*]')", conn);
               
               //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
               cmd.CommandTimeout=600; // 600-seconds
               
               var rdr = cmd.ExecuteReader();
   
               using (var fs = new FileStream(outpath, FileMode.Create))
               {
                   using (var writer = new StreamWriter(fs))
                   {
                       //write file header
                       writer.WriteLine("CustomerID,CompanyName");
   
                       while (rdr.Read())
                       {
                           //write file row
                           writer.WriteLine("{0},{1}", rdr["CustomerID"], rdr["CompanyName"]);
                       }
                       conn.Close(); //close connection
                       writer.Close();
                       fs.Close();
                   }
               }
           }
   
           //Read from file and display the content
           Console.Write(File.ReadAllText(outpath));
   
           Console.WriteLine("\r\n===== Press any key to end the program =====\r\n");
           Console.Read();
       }
   }

    
6. Finally, you will be able to see the file created:
   

   REST API into CSV

Export REST API to CSV using Python

Python is another really popular programming language. The popularity is growing a lot. In this example, we will learn how to Export REST API to CSV using Python.

1. First of all, you will need to install Pip if not included in Python. Pip is Package Installer.
   For instructions about the installation, refer to this link.
2. Secondly, you will also need the pyodbc. The pyodbc allows connecting to ODBC using Python. To install it go to the scripts folder of Python where Python is installed and run this command:pip install pyodbc.
3. Once that pyodbc is installed, we will run the following code:Full Code
   

   import csv
   import pyodbc
   
   conn = pyodbc.connect(
       r'DRIVER={ZappySys JSON Driver};'
       )
   cursor = conn.cursor()	
   rows = cursor.execute("SELECT CustomerID,CompanyName FROM value WHERE COUNTRY='Germany' WITH (SRC='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json')") 
   with open(r'C:\sql\cus2.csv', 'w', newline='') as csvfile:
       writer = csv.writer(csvfile)
       writer.writerow([x[0] for x in cursor.description])  
       for row in rows:
           writer.writerow(row)

4. Now lets’s understand parts of above code. We have used the csv and pyodbc modules in the code:
   

   import csv
   import pyodbc

5. Also, we connect to the ZappySys ODBC Driver:
   

   conn = pyodbc.connect(
   r'DRIVER={ZappySys JSON Driver};'
   )

6. In addition, we have used a cursor to get the rows and send a SQL query to get data from the REST API:
   

   cursor = conn.cursor() 
   rows = cursor.execute("SELECT CustomerID,CompanyName FROM value 
   WHERE Country='Germany' WITH 
   (SRC='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json')")

7. Following code is to open the CSV file stream:
   

   with open(r'C:\sql\customer.csv', 'w', newline='') as csvfile:

8. Finally, we will write the data from REST API into the CSV file:
   

   writer = csv.writer(csvfile)
   writer.writerow([x[0] for x in cursor.description]) 
   for row in rows:
   writer.writerow(row)

9. To conclude, if everything is OK, you will be able to see the created CSV file:
   

   Python code REST API

Using ODBC DSN in Connection String

So far we have seen DSN less connection string approach for ODBC Driver but now lets look at another way to use ODBC Driver in your C# or Python code. You can define many settings on DSN Datasource rather than setting in the ConnectionString.

Configure DSN for REST API Connection

1. First of all, we will access the following URL:
   

   https://services.odata.org/V3/Northwind/Northwind.svc/Orders?$format=json

2. Secondly, in the windows start menu, Search for “ODBC” open the ODBC Data Sources.
3. Also, in the ODBC Administrator, press Add and select the ZappySys JSON
   Driver:
   

   Add ODBC JSON

4. Finally, specify the URL of step 1 and save the configuration:
   

   URL ODBC

Using ODBC DSN in C# Code

Now to use ODBC DSN you created simply change our previous C# Code as below (Just one line)

using (var conn = new OdbcConnection("DSN=Your-DSN-Name-Goes-Here"))
{
    conn.Open();
    ...........
    ...........
    ...........

Using ODBC DSN in C# code to call REST API

Using ODBC DSN in Python Code

Now to use ODBC DSN you created simply change our previous C# Code as below (Just one line)

conn = pyodbc.connect(
    r'DSN=Your-DSN-name-Goes-Here;'
    )
	
	...........
	...........
	...........

ZappySys JSON /REST API Driver Query Examples

Reading from XML files or API can be done using the same way as previous sections except you have to use ZappySys XML Driver. Read help file here to see json query examples.

ZappySys XML / SOAP Driver Query Examples

Reading from XML files or API can be done using the same way as previous sections except you have to use ZappySys XML Driver. Read help file here to see xml query examples.

Conclusion

To conclude, in this article, we show how to access REST API using C# and Python. We used the ZappySys ODBC PowerPack that allows accessing to REST API data and JSON files using SQL queries. It is possible to create simple SQL queries and access the data. It is also possible to access to XML files and Web API with this tool. If you liked this tool you can test the ZappySys ODBC PowerPack here.

References

Finally, if you want to read more about this topic, refer to these links:

• Download ZappySys ODBC PowerPack Installer
• How To Use the ODBC .NET Managed Provider in Visual C# .NET and Connection Strings
• Python ODBC Wiki

The post How to export REST API to CSV using c# or Python appeared first on ZappySys Blog.

In this article, we will learn how to import REST API in Power BI.

Power BI is a very popular Business Analytic tool used to get business information. It is very popular because it is easy to install, simple to learn and very intuitive. Also, REST API is very popular these days and we wanted to show you a way to integrate them in Power BI with the help of the ZappySys ODBC PowerPack. This software includes powerful drivers to query REST API URL, Local XML / JSON files and XML SOAP Web Service with simple SQL queries.

The following video will show how to import REST API in Power BI (JSON or XML Data):
[youtube https://www.youtube.com/watch?v=-3OPfhMY1aI&w=720&h=405]

Requirements

1. First, you will need to have Power BI Desktop (FREE) installed
2. In the second place, you will need to have internet access
3. On third place, our ZappySys ODBC Power Pack

Step-by-Step: Import REST API into Power BI

We’ll walk through the steps to import a REST API into Power BI. The steps outlined below focus on working with JSON APIs, but the same principles can be applied when importing SOAP XML Web Services or local JSON/XML files.

For demonstration purposes, we will utilize a REST API to retrieve data in JSON format. The example URL we’ll use is:

https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json

This URL leverages OData and presents information in JSON format. Our objective is to use the ZappySys ODBC Power Pack to establish a connection to this URL and retrieve the information seamlessly into Power BI.

Create ODBC DSN – JSON Driver

DRIVER={DRIVER-NAME-GOES-HERE}[;Prop1=xxxxxx][;Prop2=xxxxxx] .... [;PropN=xxxxxx]

DRIVER={ZappySys JSON Driver}
DRIVER={ZappySys JSON Driver};DataPath='c:\mydata.json'
DRIVER={ZappySys XML Driver};DataPath='http://myserver/api/xml/getOrders'
DRIVER={ZappySys XML Driver};DataPath='c:\mydata.xml'

DRIVER={DRIVER-NAME-GOES-HERE}[;Prop1=xxxxxx][;Prop2=xxxxxx] .... [;PropN=xxxxxx]

DSN=MyJsonAPI_DSN
DSN=MyJsonAPI_DSN;DataPath='http://myserver/api/json/getOrders'

Connect to REST API data source in Power BI (Connect JSON / XML data)

Publish Power BI dashboard

1. In order to publish the report, press Publish
2. Press Save
   

   Save changes in Power BI:

    

3. Select a workspace for the report
   

   Select Power BI workplace.

4. And that is all. You have now your report ready.

Import data using ODBC ConnectionString (DSN-less)

Let’s explore the process of importing data using the ODBC Driver without the need for configuring an ODBC DSN.

When importing data, you have the option to either use an ODBC DSN or a Direct ConnectionString. This section will guide you through the steps of utilizing the ODBC ConnectionString.

After opting for the ODBC option to retrieve data, you will encounter the “Select ODBC Data Source” screen. To proceed without a DSN, follow these steps:

1. Change the DSN dropdown to “(none).”
2. Enter the full ODBC ConnectionString, adhering to the syntax below. You can construct your own connection string using the ODBC Driver UI and utilize the “Copy ConnectionString” option.
   Syntax: 

   Driver={Your Driver Name}[;Property1=value][;Property2=value]......

3. Here is a screenshot with ConnectionString Setting
   

   Import Data from ODBC Driver – DSN less mode (Use Connection String)

4. Click “Next” to select the authentication method. Choose “Windows” and input the password value if necessary. If your connection string already includes the Password attribute, there’s no need to worry.
   

   Import Data from ODBC Driver – using Connection String – With Password Attribute

POST data to REST API URL

Up until now, we’ve explored the process of fetching data from URLs and importing it into Power BI. However, in certain scenarios, APIs may necessitate the use of the PUT or POST method. These methods involve submitting parameters in the body and receiving a response.

For a more in-depth understanding of how to seamlessly POST data to a URL in Power BI, watch the informative video below.

Calling XML SOAP Web Service in Power BI

So far, we have looked at examples to consume data using JSON driver. Now let’s look at an example, to call XML SOAP Web Service in Power BI.

To call SOAP API you need to know Request XML Body Structure. If you are not sure how to create SOAP Request body then no worries. Check this article to learn how to generate SOAP Request body using the Free tool SoapUI. Basically, you have to use SoapUI to generate Request XML and after that, you can replace parameters as needed in the generated body.

What is SOAP Web Service?

If you are new to SOAP Web Service sometimes referred as XML Web Service then please read some concept about SOAP Web service standard from this link There are two important aspects in SOAP Web service.

1. Getting WSDL file or URL
2. Knowing exact Web Service URL

What is WSDL

In very simple term WSDL (often pronounced as whiz-dull) is nothing but a document which describes Service metadata (e.g. Functions you can call, Request parameters, response structure etc). Some service simply give you WSDL as xml file you can download on local machine and then analyze or sometimes you may get direct URL (e.g. http://api.mycompany.com/hr-soap-service/?wsdl )

Example SQL Query for SOAP API call using ZappySys XML Driver

Here is an example SQL query you can write to call SOAP API. If you not sure about many details then check next few sections on how to use XML Driver User Interface to build desired SQL query to POST data to XML SOAP Web Service without any coding.

SELECT * FROM $
WITH(
	 Src='http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx'
	,DataConnectionType='HTTP'
	,CredentialType='Basic' --OR SoapWss
	,SoapWssPasswordType='PasswordText'
	,UserName='myuser'
	,Password='pass$$w123'
	,Filter='$.soap:Envelope.soap:Body.GetHolidaysAvailableResponse.GetHolidaysAvailableResult.HolidayCode[*]'
	,ElementsToTreatAsArray='HolidayCode'	
	,RequestMethod='POST'	
	,Header='Content-Type: text/xml;charset=UTF-8 || SOAPAction: "http://www.holidaywebservice.com/HolidayService_v2/GetHolidaysAvailable"'
	,RequestData='
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:hol="http://www.holidaywebservice.com/HolidayService_v2/">
   <soapenv:Header/>
   <soapenv:Body>
      <hol:GetHolidaysAvailable>
         <!--type: Country - enumeration: [Canada,GreatBritain,IrelandNorthern,IrelandRepublicOf,Scotland,UnitedStates]-->
         <hol:countryCode>UnitedStates</hol:countryCode>
      </hol:GetHolidaysAvailable>
   </soapenv:Body>
</soapenv:Envelope>'
)

Now let's look at steps to create SQL query to call SOAP API. Later we will see how to generate code for your desired programming language (e.g. C# or SQL Server)

Video Tutorial - Introduction to SOAP Web Service and SoapUI tool

Before we dive into details about calling SOAP API using ZappySys XML Driver, lets first understand what is SOAP API and how to create SOAP requests using SoapUI tool. You will learn more about this process in the later section. The video contains some fragment about using SOAP API in SSIS but just ignore that part because we will be calling Soap API using ZappySys ODBC Driver rather than SSIS Components.  

Using SoapUI to test SOAP API call / Create Request Body XML

Assuming you have downloaded and installed SoapUI from here, now we are ready to use WSDL for your SOAP Web Service Calls. If you do not have WSDL file or URL handy then contact your API provider (sometimes you just have to add ?wsdl at the end of your Service URL to get WSDL so try that. Example: http://mycompany/myservice?wsdl ). If you don't know what is WSDL then in short, WSDL is Web service Description Language (i.e. XML file which describes your SOAP Service). WSDL helps to craft SOAP API request Body for ZappySys XML Driver. So Let's get started.

1. Open SoapUI and click SOAP button to create new SOAP Project
2. Enter WSDL URL or File Path of WSDLFor example WSDL for our sample service can be accessed via this URL

   http://www.dneonline.com/calculator.asmx?wsdl

   
   Create new SOAP API Project in SoapUI tool for SOAP API Testing
3. Once WSDL is loaded you will see possible operations you can call for your SOAP Web Service.
4. If your web service requires credentials then you have to configure it. There are two common credential types for public services (SOAP WSS or BASIC )
   1. To use SOAP WSS Credentials select request node and enter UserId, Password, and WSS-PasswordType (PasswordText or PasswordHash)
      
      Configure SOAP WSS Credentials for SoapUI (SOAP API Testing Tool)
   2. To use BASIC Auth Credentials select request node and double-click it. At the bottom click on Auth (Basic) and From Authorization dropdown click Add New and Select Basic.
      Configure Basic Authorization for SoapUI (SOAP API Testing Tool)
5. Now you can test your request first Double-click on the request node to open request editor.
6. Change necessary parameters, remove optional or unwanted parameters. If you want to regenerate request you can click on Recreate default request toolbar icon.
   Create SOAP Request XML (With Optional Parameters)
7. Once your SOAP Request XML is ready, Click the Play button in the toolbar to execute SOAP API Request and Response will appear in Right side panel. Test SOAP API using SoapUI Tool (Change Default XML Body / Parameters, Execute and See Response)

Create DSN using ZappySys XML Driver to call SOAP API

Once you have tested your SOAP API in SoapUI tool, we are ready to use ZappySys XML driver to call SOAP API in your preferred BI tool or Programming language.

1. First open ODBC Data Sources (search ODBC in your start menu or go under ZappySys > ODBC PowerPack > ODBC 64 bit)
2. Goto System DSN Tab (or User DSN which is not used by Service account)
3. Click Add and Select ZappySys XML Driver ZappySys ODBC Driver for XML / SOAP API
4. Configure API URL, Request Method and Request Body as below ZappySys XML Driver - Calling SOAP API - Configure URL, Method, Body
5. (This step is Optional) If your SOAP API requires credentials then Select Connection Type to HTTP and configure as below.
   ZappySys XML Driver - Configure SOAP WSS Credentials or Basic Authorization (Userid, Password)
6. Configure-Request Headers as below (You can get it from Request > Raw tab from SoapUI after you test the request by clicking the Play button) Configure SOAP API Request Headers - ZappySys XML Driver
7. Once credentials entered you can select Filter to extract data from the desired node. Make sure to select array node (see special icon) or select the node which contains all necessary columns if you don't have array node. Select Filter - Extract data from nested XML / SOAP API Response (Denormalize Hierarchy)
8. If prompted select yes to treat selected node as Array (This is helpful when you expect one or more record for selected node) Treat selected node as XML Array Option for SOAP API Response XML

Preview SOAP API Response / Generate SQL Code for SOAP API Call

Once you configure settings for XML Driver now you can preview data or generate example code for desired language (e.g. C#, Python, Java, SQL Server). Go to Preview tab and you will see default query generated based on settings you entered in previous sections. Attributes listed in WITH clause are optional. If you omit attribute in WITH clause it will use it from Properties tab.

Preview Data

Preview SOAP API Response in ZappySys XML Driver

Generate Code Option

Generate Example Code for ZappySys Driver

SOAP / REST API pagination in Power BI

SELECT * FROM $
WITH(
SRC=@'https://zappysys.com/downloads/files/test/pagination_nextlink_inarray_1.json'
,NextUrlAttributeOrExpr = '$.nextlink'  --keep reading until this attribute is missing. If attribute name contains dot then use brackets like this $.['my.attr.name']
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/page-xml.aspx?page=1&mode=DetectBasedOnResponseStatusCode'
,PagingMode='ByUrlParameter'
,PagingByUrlAttributeName='page'
,PagingByUrlEndStrategy='DetectBasedOnResponseStatusCode'
,PagingByUrlCheckResponseStatusCode=401
,IncrementBy=1
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/cust-<%page%>.xml'
,PagingMode='ByUrlPath'
,PagingByUrlAttributeName='<%page%>'
,PagingByUrlEndStrategy='DetectBasedOnRecordCount'
,IncrementBy=1
)

SELECT * FROM $
LIMIT 25
WITH(
	 Src='https://wordpress.org/news/wp-json/wp/v2/categories?per_page=10'
	,PagingMode='ByResponseHeaderRfc5988'
	,WaitTimeMs='200' --//wait 200 ms after each request
)

SOAP / REST API Error Handling in Power BI

Sometimes errors occur... they just do and there is nothing you can do! Or can you? Actually, in ODBC PowerPack you can handle them in two ways.

METHOD 1 - Using Error Handling Options

When to use?

You may want to use them when your source is a resource located on the Internet; e.g. a file on a website, a file on an FTP server or just a plain API HTTP response. By default, when a remote server returns an error, data retrieval is stopped, an error is raised and no data is given back to you. This might not be always desirable.

Scenario 1

Imagine a scenario, that there is a web server which each day at 12 AM releases a new JSON file with that day's date as filename, e.g. http://www.some-server.com/data/2018-06-20.json. And, of course, you want to download it and use it daily in your Power BI report. But you have a problem: Power BI report data sources are refreshed each hour and you may get HTTP 404 status code (no file was found) when a file is not released yet. Which consequentially means other data sources won't be updated as well and you will see old and cached data on the report. That's where you could use Continue on any error or Continue when Url is invalid or missing (404 Errors) to avoid an error being raised and let other data sources to be updated.

Scenario 2

Another scenario is when you expect a web server to raise some kind of HTTP error when accessing a URL. You don't want ODBC Data Source to raise an error but instead, you want to get response data. That's where you can use Continue on any error or alike together with  Get response data on error to continue on an error and get the data:

METHOD 2 - Using Connection [Retry Settings]

Another scenario you may run into is a buggy web server. You ask it to give you some file or data and it, like a snotty kid, just doesn't give it to you! You have to ask twice or thrice before it does its job. If that's the case, you have to retry HTTP requests using Connection:

Security / Connection Types (Options for HTTP, OAuth, SOAP) in Power BI

If you need to authenticate or authorize your user to access a web resource, you will need to use one of the Connections:

• HTTP
• OAuth

HTTP Connection

Use HTTP Connection for simple Windows, Basic, NTLM or Kerberos authentication. Just fill in a username and a password and you are good to go!

You can also use HTTP Connection for more sophisticated authentication like:

• SOAP WSS (when accessing a SOAP WebService)
• Static Token / API Key (when need to pass an API key in HTTP header)
• Dynamic Token (same as Static Token method except that each time you need to log in and retrieve a fresh API key)
• JWT Token (As per RFC 7519)

OAuth

If you are trying to access REST API resource, it is a huge chance, you will need to use OAuth Connection. Read this article to understand how OAuth authentication and authorization works and how to use it (article originally was written for SSIS PowerPack, but the concepts and UI stay the same): 
https://zappysys.com/blog/rest-api-authentication-with-oauth-2-0-using-ssis/

Import data into Power BI from SQL Query

If you prefer to import data from an SQL query instead of selecting a table name, you can leverage advanced options during the import steps. Here’s a step-by-step guide:

1. After selecting the DSN, proceed to the import steps.
2. Click on the “Advanced Options” to access the SQL Query editor.

By utilizing these advanced options, you gain the flexibility to tailor your import process by specifying custom SQL queries to retrieve the exact data you need.

Import data into Power BI using SQL Query (ODBC Data source)

Edit Query / Using Parameters in Power BI (Dynamic Query)

let
    vKey=paraAPIKey,
    Source = Odbc.Query(
"dsn=ZS - OData Customers", 
"SELECT * FROM $ WITH (SRC='http://httpbin.org/post'," 
& "METHOD='POST'," 
& "HEADER='Content-Type:application/json'," 
& "BODY=@'{""CallerId"":1111, ""ApiKey"":""" & vKey & """}')")
in
    Source

Edit Query Settings after Import

There will be a time you need to change initial Query after dataset import in Power BI. Not to worry, just follow these steps to edit your SQL.

Edit Power BI Data Source SQL after REST / SOAP Web Service Import

Performance Tips for REST API / XML SOAP Calls

SELECT * FROM value
WITH(
	 Src='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json&$filter=Country eq ''USA'''
	,DataFormat='Odata'
)

-- Slow query - Client-side filtering
SELECT * FROM value
WHERE Country ='USA'
WITH(
	 Src='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json'
	,DataFormat='Odata'
)

SELECT * FROM $
WITH 
(  SRC='https://myhost.com/some-api'
  ,CachingMode='All'  --cache metadata and data rows both
  ,CacheStorage='File' --or Memory
  ,CacheFileLocation='c:\temp\myquery.cache'
  ,CacheEntryTtl=300 --cache for 300 seconds
)

SELECT * FROM $ 
LIMIT 10 --//add this just to test how fast you can get 10 rows
WITH(
  Filter='$.LargeArray[*]--FAST' --//Adding --FAST option turn on STREAM mode (large files)
 ,SRC='https://zappysys.com/downloads/files/test/large_file_100k_largearray_prop.json.gz'
 --,SRC='c:\data\large_file.json.gz'
 ,IncludeParentColumns='False'  --//This Must be OFF for STREAM mode (read very large files)
 ,FileCompressionType='GZip' --Zip or None (Zip format only available for Local files)
)

Conclusion

In this article, we guide you through the process of configuring the ZappySys Power Pack, focusing on the ZappySys driver for JSON. This driver is instrumental in extracting data from REST APIs or JSON files. Follow these steps to seamlessly add the extracted data to Power BI and generate insightful reports.

Additionally, we’ll walk you through the configuration of the Power BI gateway, ensuring that the data is refreshed on a specified schedule for up-to-date and accurate reporting. To try this configuration yourself, download ODBC PowerPack and embark on a journey of efficient data extraction and reporting.

References

• What is Power BI?
• Keep your dashboards and reports up-to-date with your on-premises data sources
• Power BI – Read REST API / JSON File / XML File / SOAP (Pagination, OAuth, OData)

The post How to import REST API in Power BI (Load JSON / SOAP XML) appeared first on ZappySys Blog.

In this post, we will learn how to import JSON to Excel.  We will use ODBC PowerPack to connect and query a JSON file. This article also covers creating Excel from SOAP XML Web Service so read full article to learn about SOAP API Export.

JSON stands for Java Script Object Notation and it is an Open and Standard format to read object with attributes and values.  JSON is replacing XML because it is faster to read data, it is easier to parse data, it does not require end tags, it is shorter and it can use arrays.

We will load the data using the ZappySys JSON driver and then upload the data in Excel.

Requirements

This article assumes following requirements are met before you can follow the steps listed in this article.

1. Make sure that Microsoft Excel installed
2. Also, the driver ODBC PowerPack installed

Video Tutorial

Let’s take a look to this step by step tutorial.

An introduction to Rest API and OData

In this example, we will use OData (Open Data Protocol) to consume REST API. REST API (Representational State Transfer Application Program Interface) allows to handle the interoperability betwee computers and internet.

In REST API we can handle web services in different formats. In this example, we will work with the Northwind example. The Northwind example is available in this URL:

https://services.odata.org/V3/Northwind/Northwind.svc

1. By default the data is displayed in XML format. To show the data in JSON use this URL:

   https://services.odata.org/V3/Northwind/Northwind.svc/?$format=json

2. There are collections of data like Categories, CustomerDemographic, Customers, etc. For example the following URL will show the data of the categories collection:

   https://services.odata.org/V3/Northwind/Northwind.svc/Categories?$format=json

3. In the next steps, we will use ZappySys drivers to connect to this URL and query using OData.

Configure ODBC DSN for ZappySys JSON Driver

ODBC driver can be accessed in two modes.

1. Using DSN
2. Without DSN (Supply direct Connection String e.g. DRIVER={ZappySys JSON Driver}; …….. )

In this article, we will use DSN approach (User DSN). We will first add the ZappySys JSON Driver in the ODBC Data source Administrator.

Follow these steps to accomplish the task:

1. First, Windows search, write ODBC and select the ODBC Data sources (32 bits)
   

   Open ODBC Data Source

2. As a second step, in ODBC Data source Administrator press the Add button.
   

   Add ZappySys

    

3. In this step, create new data source, select ZappySys ODBC Driver.
   

   Add new zappysys json driver

4. Here we have several properties, write a data source name. In this example, the name will be ZappySys JSON to Excel.
5. The Data Source (URL or file path) can specify the URL of the source or if it is a local file, you can specify the local path. In this example, the URL is:
   

   https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json

   You can also specify local file path as Data Source

   For single file:  c:\data\myfile_1.json
   For multiple files: c:\data\myfile_*.json

6. Expand Other settings and in Data Format, select OData. Press OK.
   

   OData properties json driver

How to import REST API data to Excel

1. In Excel, go to Data Ribbon and select From Other Sources and From Microsoft Query.
   

   Excel Data Ribbon

2. In Choose Data Source, select ZappySys JSON for Excel and press OK.
   

   Select zappysys data source

3. Click Value and press the > to display all the attributes and press next.
   

   Create json query in excel

4. You can filter data and select columns and check if a columns is equal to, greater than or less than a specific value. In this example, we will not apply filters. Press next.
   

   filter json data in excel

5. Now if you want to export data to sheet without using custom Query then select first option. If you like to enter custom query then select  Microsoft Query (second option) once you close the wizard you may get an option to enter custom SQL (see the toolbar of Graphical Designer)We will use first option for now.
   

   Return data in Excel

6. In this step, Excel will show the Import Data window. Excel will let you select the sheet to insert the data. We will choose the existing sheet.
   

   Choose excel sheet

7. If everything is OK, you will be able to see the data.
   

   Excel rows with data

How to import data from REST API to CSV

If you already have your data in Excel with the previous steps, converting the data extracted from Rest API to Excel to a CSV is a straight forward process.

1. Using Excel, go to the File menú and select Save as and select a folder to store it.
   

   Save as in Excel

2. In order to save the data in CSV, save as type select CSV (Comma delimited)
   

   convert excel to csv

How to import JSON to Excel (From Local file)

1. To import data in Excel, the steps are the same for Rest API that for a physical json file. The only difference is that a local path is used instead of a URL. In order to get JSON data into Excel, you need to repeat the steps to add the ZappySys ODBC Driver in the ODBC Data source, but instead of specifying a URL, we just need to specify the local path.
2. To test the driver, let’s say that we have the following named sample.json file:
   

   {
     "odata.metadata": "https://services.odata.org/V3/Northwind/Northwind.svc/$metadata#Customers",
     "odata.nextLink": "Customers?$skiptoken='ERNSH'",
     "value": [
       {
         "CustomerID": "ALFKI",
         "CompanyName": "Alfreds Futterkiste",
         "ContactName": "Maria Anders",
         "ContactTitle": "Sales Representative",
         "Address": "Obere Str. 57",
         "City": "Berlin",
         "Region": null,
         "PostalCode": "12209",
         "Country": "Germany",
         "Phone": "030-0074321",
         "Fax": "030-0076545"
       },
       {
         "CustomerID": "ANATR",
         "CompanyName": "Ana Trujillo Emparedados y helados",
         "ContactName": "Ana Trujillo",
         "ContactTitle": "Owner",
         "Address": "Avda. de la Constitución 2222",
         "City": "México D.F.",
         "Region": null,
         "PostalCode": "05021",
         "Country": "Mexico",
         "Phone": "(5) 555-4729",
         "Fax": "(5) 555-3745"
       },
       {
         "CustomerID": "ANTON",
         "CompanyName": "Antonio Moreno Taquería",
         "ContactName": "Antonio Moreno",
         "ContactTitle": "Owner",
         "Address": "Mataderos 2312",
         "City": "México D.F.",
         "Region": null,
         "PostalCode": "05023",
         "Country": "Mexico",
         "Phone": "(5) 555-3932",
         "Fax": null
       },
       {
         "CustomerID": "AROUT",
         "CompanyName": "Around the Horn",
         "ContactName": "Thomas Hardy",
         "ContactTitle": "Sales Representative",
         "Address": "120 Hanover Sq.",
         "City": "London",
         "Region": null,
         "PostalCode": "WA1 1DP",
         "Country": "UK",
         "Phone": "(171) 555-7788",
         "Fax": "(171) 555-6750"
       },
       {
         "CustomerID": "BERGS",
         "CompanyName": "Berglunds snabbköp",
         "ContactName": "Christina Berglund",
         "ContactTitle": "Order Administrator",
         "Address": "Berguvsvägen 8",
         "City": "Luleå",
         "Region": null,
         "PostalCode": "S-958 22",
         "Country": "Sweden",
         "Phone": "0921-12 34 65",
         "Fax": "0921-12 34 67"
       },
       {
         "CustomerID": "BLAUS",
         "CompanyName": "Blauer See Delikatessen",
         "ContactName": "Hanna Moos",
         "ContactTitle": "Sales Representative",
         "Address": "Forsterstr. 57",
         "City": "Mannheim",
         "Region": null,
         "PostalCode": "68306",
         "Country": "Germany",
         "Phone": "0621-08460",
         "Fax": "0621-08924"
       },
       {
         "CustomerID": "BLONP",
         "CompanyName": "Blondesddsl père et fils",
         "ContactName": "Frédérique Citeaux",
         "ContactTitle": "Marketing Manager",
         "Address": "24, place Kléber",
         "City": "Strasbourg",
         "Region": null,
         "PostalCode": "67000",
         "Country": "France",
         "Phone": "88.60.15.31",
         "Fax": "88.60.15.32"
       },
       {
         "CustomerID": "BOLID",
         "CompanyName": "Bólido Comidas preparadas",
         "ContactName": "Martín Sommer",
         "ContactTitle": "Owner",
         "Address": "C/ Araquil, 67",
         "City": "Madrid",
         "Region": null,
         "PostalCode": "28023",
         "Country": "Spain",
         "Phone": "(91) 555 22 82",
         "Fax": "(91) 555 91 99"
       },
       {
         "CustomerID": "BONAP",
         "CompanyName": "Bon app'",
         "ContactName": "Laurence Lebihan",
         "ContactTitle": "Owner",
         "Address": "12, rue des Bouchers",
         "City": "Marseille",
         "Region": null,
         "PostalCode": "13008",
         "Country": "France",
         "Phone": "91.24.45.40",
         "Fax": "91.24.45.41"
       },
       {
         "CustomerID": "BOTTM",
         "CompanyName": "Bottom-Dollar Markets",
         "ContactName": "Elizabeth Lincoln",
         "ContactTitle": "Accounting Manager",
         "Address": "23 Tsawassen Blvd.",
         "City": "Tsawassen",
         "Region": "BC",
         "PostalCode": "T2F 8M4",
         "Country": "Canada",
         "Phone": "(604) 555-4729",
         "Fax": "(604) 555-3745"
       },
       {
         "CustomerID": "BSBEV",
         "CompanyName": "B's Beverages",
         "ContactName": "Victoria Ashworth",
         "ContactTitle": "Sales Representative",
         "Address": "Fauntleroy Circus",
         "City": "London",
         "Region": null,
         "PostalCode": "EC2 5NT",
         "Country": "UK",
         "Phone": "(171) 555-1212",
         "Fax": null
       },
       {
         "CustomerID": "CACTU",
         "CompanyName": "Cactus Comidas para llevar",
         "ContactName": "Patricio Simpson",
         "ContactTitle": "Sales Agent",
         "Address": "Cerrito 333",
         "City": "Buenos Aires",
         "Region": null,
         "PostalCode": "1010",
         "Country": "Argentina",
         "Phone": "(1) 135-5555",
         "Fax": "(1) 135-4892"
       },
       {
         "CustomerID": "CENTC",
         "CompanyName": "Centro comercial Moctezuma",
         "ContactName": "Francisco Chang",
         "ContactTitle": "Marketing Manager",
         "Address": "Sierras de Granada 9993",
         "City": "México D.F.",
         "Region": null,
         "PostalCode": "05022",
         "Country": "Mexico",
         "Phone": "(5) 555-3392",
         "Fax": "(5) 555-7293"
       },
       {
         "CustomerID": "CHOPS",
         "CompanyName": "Chop-suey Chinese",
         "ContactName": "Yang Wang",
         "ContactTitle": "Owner",
         "Address": "Hauptstr. 29",
         "City": "Bern",
         "Region": null,
         "PostalCode": "3012",
         "Country": "Switzerland",
         "Phone": "0452-076545",
         "Fax": null
       },
       {
         "CustomerID": "COMMI",
         "CompanyName": "Comércio Mineiro",
         "ContactName": "Pedro Afonso",
         "ContactTitle": "Sales Associate",
         "Address": "Av. dos Lusíadas, 23",
         "City": "Sao Paulo",
         "Region": "SP",
         "PostalCode": "05432-043",
         "Country": "Brazil",
         "Phone": "(11) 555-7647",
         "Fax": null
       },
       {
         "CustomerID": "CONSH",
         "CompanyName": "Consolidated Holdings",
         "ContactName": "Elizabeth Brown",
         "ContactTitle": "Sales Representative",
         "Address": "Berkeley Gardens 12 Brewery",
         "City": "London",
         "Region": null,
         "PostalCode": "WX1 6LT",
         "Country": "UK",
         "Phone": "(171) 555-2282",
         "Fax": "(171) 555-9199"
       },
       {
         "CustomerID": "DRACD",
         "CompanyName": "Drachenblut Delikatessen",
         "ContactName": "Sven Ottlieb",
         "ContactTitle": "Order Administrator",
         "Address": "Walserweg 21",
         "City": "Aachen",
         "Region": null,
         "PostalCode": "52066",
         "Country": "Germany",
         "Phone": "0241-039123",
         "Fax": "0241-059428"
       },
       {
         "CustomerID": "DUMON",
         "CompanyName": "Du monde entier",
         "ContactName": "Janine Labrune",
         "ContactTitle": "Owner",
         "Address": "67, rue des Cinquante Otages",
         "City": "Nantes",
         "Region": null,
         "PostalCode": "44000",
         "Country": "France",
         "Phone": "40.67.88.88",
         "Fax": "40.67.89.89"
       },
       {
         "CustomerID": "EASTC",
         "CompanyName": "Eastern Connection",
         "ContactName": "Ann Devon",
         "ContactTitle": "Sales Agent",
         "Address": "35 King George",
         "City": "London",
         "Region": null,
         "PostalCode": "WX3 6FW",
         "Country": "UK",
         "Phone": "(171) 555-0297",
         "Fax": "(171) 555-3373"
       },
       {
         "CustomerID": "ERNSH",
         "CompanyName": "Ernst Handel",
         "ContactName": "Roland Mendel",
         "ContactTitle": "Sales Manager",
         "Address": "Kirchgasse 6",
         "City": "Graz",
         "Region": null,
         "PostalCode": "8010",
         "Country": "Austria",
         "Phone": "7675-3425",
         "Fax": "7675-3426"
       }
     ]
   }

3. Finally, in the ZappySys driver, in Data Source (URL or File Path), write the path of your json file. In this example, c:\sql\sample.json. Also expand Data format and select Default.
   
   

   properties to query a local json file

   NOTE: When the Data format is set to OData and it is a local json file, the error message is “Data processing error: URI formats are not supported”. If the Data format is set to Original, the error message is “Query execution error: Requested value ‘Original’ was not found.” To solve this issue, make sure that in Other Settings, the Data format is set to Default.

How to query JSON or REST API

Our ZappySys driver is a very intuitive tool and you can write queries to data like a simple database table.

Let’s take a look to some examples:

In the ODBC Data Source Administrator, press Configure.

Configure ZappySys settings

On Preview page, write this query:

select 
"CustomerID",
"CompanyName",
"ContactName",
"ContactTitle",
"Address",
"City",
"Region",
"PostalCode",
"Country",
"Phone",
"Fax"
from [value] 
where City='Berlin'

The query will show the customers where the city is Berlin:

Query json information using filters

You can use in the where clause with double quotes:

WHERE City="Berlin"

or simple quotes:

WHERE City='Berlin'

It is also valid to comment code. The following example shows how to comment a single line (the where clause):

select "CustomerID", "CompanyName
from [value]
--where City='Berlin'

The following example shows how to comment multiple lines

/*
This example shows how 
to comment
multiple lines of code
*/

select "CustomerID", "CompanyName from [value]

Schedule / Automate Excel file creation using command line

There will be a time when you need to export REST API to Excel in an automated way (Scheduled Task). Possibly via Batch file, Python code, PowerShell script or Scheduler. Check this article to learn more.

You can use ZappyShell for DB (Command Line Tool) to export JSON / SOAP / REST API data to Excel Sheet. It supports any ODBC Connection string as a source and you can export to CSV, Excel, JSON and XML.

Here is a sample command line. You can automate it via Scheduled Job (e.g. Windows Scheduler or SQL Agent Job)

First create below file (name it script.txt)

export "SELECT * FROM $ LIMIT 20 WITH(Src='https://httpbin.org/json',Filter='$.slideshow.slides[*]' , RequestMethod='GET')"  -o "c:\data.xlsx" --connstr "DSN=MyAPIConnection;" -y

Then call this (assuming you have zappyshell in c:\zappyshell folder

c:\zappyshell>db.exe exec c:\script.txt

You can also use DSN less connection string such as below

export "SELECT * FROM $ LIMIT 20 WITH(Src='https://httpbin.org/json',Filter='$.slideshow.slides[*]' , RequestMethod='GET')"  -o "c:\temp\data.xlsx" --connstr "DRIVER={ZappySys JSON Driver};" -y

Export JSON / SOAP XML / REST API to Excel File – Create Excel File using Automated Command Line (Schedule)

Calling XML SOAP Web Service in Excel

So far we have looked at examples to consume data using JSON driver. Now let’s look at an example, to call XML SOAP Web Service in Excel.

To call SOAP API you need to know Request XML Body Structure. If you are not sure how to create SOAP Request body then no worries. Check this article to learn how to generate SOAP Request body using the Free tool SoapUI. Basically, you have to use SoapUI to generate Request XML and after that, you can replace parameters as needed in the generated body.

What is SOAP Web Service?

If you are new to SOAP Web Service sometimes referred as XML Web Service then please read some concept about SOAP Web service standard from this link There are two important aspects in SOAP Web service.

1. Getting WSDL file or URL
2. Knowing exact Web Service URL

What is WSDL

In very simple term WSDL (often pronounced as whiz-dull) is nothing but a document which describes Service metadata (e.g. Functions you can call, Request parameters, response structure etc). Some service simply give you WSDL as xml file you can download on local machine and then analyze or sometimes you may get direct URL (e.g. http://api.mycompany.com/hr-soap-service/?wsdl )

Example SQL Query for SOAP API call using ZappySys XML Driver

Here is an example SQL query you can write to call SOAP API. If you not sure about many details then check next few sections on how to use XML Driver User Interface to build desired SQL query to POST data to XML SOAP Web Service without any coding.

SELECT * FROM $
WITH(
	 Src='http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx'
	,DataConnectionType='HTTP'
	,CredentialType='Basic' --OR SoapWss
	,SoapWssPasswordType='PasswordText'
	,UserName='myuser'
	,Password='pass$$w123'
	,Filter='$.soap:Envelope.soap:Body.GetHolidaysAvailableResponse.GetHolidaysAvailableResult.HolidayCode[*]'
	,ElementsToTreatAsArray='HolidayCode'	
	,RequestMethod='POST'	
	,Header='Content-Type: text/xml;charset=UTF-8 || SOAPAction: "http://www.holidaywebservice.com/HolidayService_v2/GetHolidaysAvailable"'
	,RequestData='
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:hol="http://www.holidaywebservice.com/HolidayService_v2/">
   <soapenv:Header/>
   <soapenv:Body>
      <hol:GetHolidaysAvailable>
         <!--type: Country - enumeration: [Canada,GreatBritain,IrelandNorthern,IrelandRepublicOf,Scotland,UnitedStates]-->
         <hol:countryCode>UnitedStates</hol:countryCode>
      </hol:GetHolidaysAvailable>
   </soapenv:Body>
</soapenv:Envelope>'
)

Now let's look at steps to create SQL query to call SOAP API. Later we will see how to generate code for your desired programming language (e.g. C# or SQL Server)

Video Tutorial - Introduction to SOAP Web Service and SoapUI tool

Before we dive into details about calling SOAP API using ZappySys XML Driver, lets first understand what is SOAP API and how to create SOAP requests using SoapUI tool. You will learn more about this process in the later section. The video contains some fragment about using SOAP API in SSIS but just ignore that part because we will be calling Soap API using ZappySys ODBC Driver rather than SSIS Components.  

Using SoapUI to test SOAP API call / Create Request Body XML

Assuming you have downloaded and installed SoapUI from here, now we are ready to use WSDL for your SOAP Web Service Calls. If you do not have WSDL file or URL handy then contact your API provider (sometimes you just have to add ?wsdl at the end of your Service URL to get WSDL so try that. Example: http://mycompany/myservice?wsdl ). If you don't know what is WSDL then in short, WSDL is Web service Description Language (i.e. XML file which describes your SOAP Service). WSDL helps to craft SOAP API request Body for ZappySys XML Driver. So Let's get started.

1. Open SoapUI and click SOAP button to create new SOAP Project
2. Enter WSDL URL or File Path of WSDLFor example WSDL for our sample service can be accessed via this URL

   http://www.dneonline.com/calculator.asmx?wsdl

   
   Create new SOAP API Project in SoapUI tool for SOAP API Testing
3. Once WSDL is loaded you will see possible operations you can call for your SOAP Web Service.
4. If your web service requires credentials then you have to configure it. There are two common credential types for public services (SOAP WSS or BASIC )
   1. To use SOAP WSS Credentials select request node and enter UserId, Password, and WSS-PasswordType (PasswordText or PasswordHash)
      
      Configure SOAP WSS Credentials for SoapUI (SOAP API Testing Tool)
   2. To use BASIC Auth Credentials select request node and double-click it. At the bottom click on Auth (Basic) and From Authorization dropdown click Add New and Select Basic.
      Configure Basic Authorization for SoapUI (SOAP API Testing Tool)
5. Now you can test your request first Double-click on the request node to open request editor.
6. Change necessary parameters, remove optional or unwanted parameters. If you want to regenerate request you can click on Recreate default request toolbar icon.
   Create SOAP Request XML (With Optional Parameters)
7. Once your SOAP Request XML is ready, Click the Play button in the toolbar to execute SOAP API Request and Response will appear in Right side panel. Test SOAP API using SoapUI Tool (Change Default XML Body / Parameters, Execute and See Response)

Create DSN using ZappySys XML Driver to call SOAP API

Once you have tested your SOAP API in SoapUI tool, we are ready to use ZappySys XML driver to call SOAP API in your preferred BI tool or Programming language.

1. First open ODBC Data Sources (search ODBC in your start menu or go under ZappySys > ODBC PowerPack > ODBC 64 bit)
2. Goto System DSN Tab (or User DSN which is not used by Service account)
3. Click Add and Select ZappySys XML Driver ZappySys ODBC Driver for XML / SOAP API
4. Configure API URL, Request Method and Request Body as below ZappySys XML Driver - Calling SOAP API - Configure URL, Method, Body
5. (This step is Optional) If your SOAP API requires credentials then Select Connection Type to HTTP and configure as below.
   ZappySys XML Driver - Configure SOAP WSS Credentials or Basic Authorization (Userid, Password)
6. Configure-Request Headers as below (You can get it from Request > Raw tab from SoapUI after you test the request by clicking the Play button) Configure SOAP API Request Headers - ZappySys XML Driver
7. Once credentials entered you can select Filter to extract data from the desired node. Make sure to select array node (see special icon) or select the node which contains all necessary columns if you don't have array node. Select Filter - Extract data from nested XML / SOAP API Response (Denormalize Hierarchy)
8. If prompted select yes to treat selected node as Array (This is helpful when you expect one or more record for selected node) Treat selected node as XML Array Option for SOAP API Response XML

Preview SOAP API Response / Generate SQL Code for SOAP API Call

Once you configure settings for XML Driver now you can preview data or generate example code for desired language (e.g. C#, Python, Java, SQL Server). Go to Preview tab and you will see default query generated based on settings you entered in previous sections. Attributes listed in WITH clause are optional. If you omit attribute in WITH clause it will use it from Properties tab.

Preview Data

Preview SOAP API Response in ZappySys XML Driver

Generate Code Option

Generate Example Code for ZappySys Driver

SOAP / REST API pagination in Excel

SELECT * FROM $
WITH(
SRC=@'https://zappysys.com/downloads/files/test/pagination_nextlink_inarray_1.json'
,NextUrlAttributeOrExpr = '$.nextlink'  --keep reading until this attribute is missing. If attribute name contains dot then use brackets like this $.['my.attr.name']
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/page-xml.aspx?page=1&mode=DetectBasedOnResponseStatusCode'
,PagingMode='ByUrlParameter'
,PagingByUrlAttributeName='page'
,PagingByUrlEndStrategy='DetectBasedOnResponseStatusCode'
,PagingByUrlCheckResponseStatusCode=401
,IncrementBy=1
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/cust-<%page%>.xml'
,PagingMode='ByUrlPath'
,PagingByUrlAttributeName='<%page%>'
,PagingByUrlEndStrategy='DetectBasedOnRecordCount'
,IncrementBy=1
)

SELECT * FROM $
LIMIT 25
WITH(
	 Src='https://wordpress.org/news/wp-json/wp/v2/categories?per_page=10'
	,PagingMode='ByResponseHeaderRfc5988'
	,WaitTimeMs='200' --//wait 200 ms after each request
)

SOAP / REST API Error Handling in Excel

Sometimes errors occur... they just do and there is nothing you can do! Or can you? Actually, in ODBC PowerPack you can handle them in two ways.

METHOD 1 - Using Error Handling Options

When to use?

You may want to use them when your source is a resource located on the Internet; e.g. a file on a website, a file on an FTP server or just a plain API HTTP response. By default, when a remote server returns an error, data retrieval is stopped, an error is raised and no data is given back to you. This might not be always desirable.

Scenario 1

Imagine a scenario, that there is a web server which each day at 12 AM releases a new JSON file with that day's date as filename, e.g. http://www.some-server.com/data/2018-06-20.json. And, of course, you want to download it and use it daily in your Power BI report. But you have a problem: Power BI report data sources are refreshed each hour and you may get HTTP 404 status code (no file was found) when a file is not released yet. Which consequentially means other data sources won't be updated as well and you will see old and cached data on the report. That's where you could use Continue on any error or Continue when Url is invalid or missing (404 Errors) to avoid an error being raised and let other data sources to be updated.

Scenario 2

Another scenario is when you expect a web server to raise some kind of HTTP error when accessing a URL. You don't want ODBC Data Source to raise an error but instead, you want to get response data. That's where you can use Continue on any error or alike together with  Get response data on error to continue on an error and get the data:

METHOD 2 - Using Connection [Retry Settings]

Another scenario you may run into is a buggy web server. You ask it to give you some file or data and it, like a snotty kid, just doesn't give it to you! You have to ask twice or thrice before it does its job. If that's the case, you have to retry HTTP requests using Connection:

Security / Connection Types (Options for HTTP, OAuth, SOAP) in Excel

If you need to authenticate or authorize your user to access a web resource, you will need to use one of the Connections:

• HTTP
• OAuth

HTTP Connection

Use HTTP Connection for simple Windows, Basic, NTLM or Kerberos authentication. Just fill in a username and a password and you are good to go!

You can also use HTTP Connection for more sophisticated authentication like:

• SOAP WSS (when accessing a SOAP WebService)
• Static Token / API Key (when need to pass an API key in HTTP header)
• Dynamic Token (same as Static Token method except that each time you need to log in and retrieve a fresh API key)
• JWT Token (As per RFC 7519)

OAuth

If you are trying to access REST API resource, it is a huge chance, you will need to use OAuth Connection. Read this article to understand how OAuth authentication and authorization works and how to use it (article originally was written for SSIS PowerPack, but the concepts and UI stay the same): 
https://zappysys.com/blog/rest-api-authentication-with-oauth-2-0-using-ssis/

Conclusion

In this article, we learned how to use the ZappySys ODBC PowerPack to import JSON to Excel. We used the OData protocol and then we learned how to import from a JSON file to Excel.  With ZappySys ODBC Power Pack, you can query REST API information or JSON files using SQL and filter the information or write custom queries according to your needs.

References

• JSON – Introduction
• Learn REST: A RESTful Tutorial

Keywords: How to import JSON to Excel (Load File or REST API), how to convert JSON to Excel, Import JSON to Excel 2016, Load JSON to Excel 2013, Import REST API to Excel 2010

The post How to import JSON to Excel (Load File, REST API, SOAP XML) appeared first on ZappySys Blog.