Export REST API to CSV is in some cases necessary to process the data because many tools can handle CSV files. In this new article, we will show different ways to export the data. The first example will do it using C#. The second example with use Python. Let’s take a look at these examples.

Requirements to export REST API to CSV

1. First of all, you will need ZappySys ODBC PowerPack installed.
2. Secondly, you will require Visual Studio installed for the C# example.
3. Finally, it is necessary Python installed for the Python example.

Export REST API to CSV using C#

C# is a pretty popular programing language. In the first example, we will show how to display the REST API information to a CSV file named. Let’s take a look at the code:

1. First of all, we will connect to REST API using a connection to the following Data Path:
   

   https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json

   using (OdbcConnection conn = 
         new OdbcConnection("Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))

2. Secondly, we create a connection using a file stream. We will save the results to a file in the c:\sql\sample.csv:
   

   FileStream fs = new FileStream("C:\\sql\\sample.txt", FileMode.Create);
   StreamWriter writer = new StreamWriter(fs);
   StringBuilder output = new StringBuilder();

3. Also, we will add a query to REST API. Note that with the ZappySys ODBC PowerPack, you can do a simple SQL query to get REST API data. This is pretty simple and intuitive:
   

   OdbcCommand cmd = new OdbcCommand(
        @"SELECT  CustomerID,CompanyName FROM value", conn);

4. In addition, we will read the data and close the connections:
   

   conn.Close();  
   writer.Close(); 
   fs.Close();

5. The complete code will be the following:
   

   using System;
   using System.IO;
   using System.Data.Odbc;
   
   public class Program
   {
       public static void Main()
       {
           var outpath = @"C:\temp\sample.txt";
   
           using (var conn = new OdbcConnection("Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))
           {
               conn.Open();
               var cmd = new OdbcCommand(@"SELECT CustomerID,CompanyName FROM $ WITH(Filter='$.value[*]')", conn);
               
               //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
               cmd.CommandTimeout=600; // 600-seconds
               
               var rdr = cmd.ExecuteReader();
   
               using (var fs = new FileStream(outpath, FileMode.Create))
               {
                   using (var writer = new StreamWriter(fs))
                   {
                       //write file header
                       writer.WriteLine("CustomerID,CompanyName");
   
                       while (rdr.Read())
                       {
                           //write file row
                           writer.WriteLine("{0},{1}", rdr["CustomerID"], rdr["CompanyName"]);
                       }
                       conn.Close(); //close connection
                       writer.Close();
                       fs.Close();
                   }
               }
           }
   
           //Read from file and display the content
           Console.Write(File.ReadAllText(outpath));
   
           Console.WriteLine("\r\n===== Press any key to end the program =====\r\n");
           Console.Read();
       }
   }

    
6. Finally, you will be able to see the file created:
   

   REST API into CSV

Export REST API to CSV using Python

Python is another really popular programming language. The popularity is growing a lot. In this example, we will learn how to Export REST API to CSV using Python.

1. First of all, you will need to install Pip if not included in Python. Pip is Package Installer.
   For instructions about the installation, refer to this link.
2. Secondly, you will also need the pyodbc. The pyodbc allows connecting to ODBC using Python. To install it go to the scripts folder of Python where Python is installed and run this command:pip install pyodbc.
3. Once that pyodbc is installed, we will run the following code:Full Code
   

   import csv
   import pyodbc
   
   conn = pyodbc.connect(
       r'DRIVER={ZappySys JSON Driver};'
       )
   cursor = conn.cursor()	
   rows = cursor.execute("SELECT CustomerID,CompanyName FROM value WHERE COUNTRY='Germany' WITH (SRC='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json')") 
   with open(r'C:\sql\cus2.csv', 'w', newline='') as csvfile:
       writer = csv.writer(csvfile)
       writer.writerow([x[0] for x in cursor.description])  
       for row in rows:
           writer.writerow(row)

4. Now lets’s understand parts of above code. We have used the csv and pyodbc modules in the code:
   

   import csv
   import pyodbc

5. Also, we connect to the ZappySys ODBC Driver:
   

   conn = pyodbc.connect(
   r'DRIVER={ZappySys JSON Driver};'
   )

6. In addition, we have used a cursor to get the rows and send a SQL query to get data from the REST API:
   

   cursor = conn.cursor() 
   rows = cursor.execute("SELECT CustomerID,CompanyName FROM value 
   WHERE Country='Germany' WITH 
   (SRC='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json')")

7. Following code is to open the CSV file stream:
   

   with open(r'C:\sql\customer.csv', 'w', newline='') as csvfile:

8. Finally, we will write the data from REST API into the CSV file:
   

   writer = csv.writer(csvfile)
   writer.writerow([x[0] for x in cursor.description]) 
   for row in rows:
   writer.writerow(row)

9. To conclude, if everything is OK, you will be able to see the created CSV file:
   

   Python code REST API

Using ODBC DSN in Connection String

So far we have seen DSN less connection string approach for ODBC Driver but now lets look at another way to use ODBC Driver in your C# or Python code. You can define many settings on DSN Datasource rather than setting in the ConnectionString.

Configure DSN for REST API Connection

1. First of all, we will access the following URL:
   

   https://services.odata.org/V3/Northwind/Northwind.svc/Orders?$format=json

2. Secondly, in the windows start menu, Search for “ODBC” open the ODBC Data Sources.
3. Also, in the ODBC Administrator, press Add and select the ZappySys JSON
   Driver:
   

   Add ODBC JSON

4. Finally, specify the URL of step 1 and save the configuration:
   

   URL ODBC

Using ODBC DSN in C# Code

Now to use ODBC DSN you created simply change our previous C# Code as below (Just one line)

using (var conn = new OdbcConnection("DSN=Your-DSN-Name-Goes-Here"))
{
    conn.Open();
    ...........
    ...........
    ...........

Using ODBC DSN in C# code to call REST API

Using ODBC DSN in Python Code

Now to use ODBC DSN you created simply change our previous C# Code as below (Just one line)

conn = pyodbc.connect(
    r'DSN=Your-DSN-name-Goes-Here;'
    )
	
	...........
	...........
	...........

ZappySys JSON /REST API Driver Query Examples

Reading from XML files or API can be done using the same way as previous sections except you have to use ZappySys XML Driver. Read help file here to see json query examples.

ZappySys XML / SOAP Driver Query Examples

Reading from XML files or API can be done using the same way as previous sections except you have to use ZappySys XML Driver. Read help file here to see xml query examples.

Conclusion

To conclude, in this article, we show how to access REST API using C# and Python. We used the ZappySys ODBC PowerPack that allows accessing to REST API data and JSON files using SQL queries. It is possible to create simple SQL queries and access the data. It is also possible to access to XML files and Web API with this tool. If you liked this tool you can test the ZappySys ODBC PowerPack here.

References

Finally, if you want to read more about this topic, refer to these links:

• Download ZappySys ODBC PowerPack Installer
• How To Use the ODBC .NET Managed Provider in Visual C# .NET and Connection Strings
• Python ODBC Wiki

The post How to export REST API to CSV using c# or Python appeared first on ZappySys Blog.

In this post, We will use ZappySys ODBC Powerpack for calling REST API in C# (i.e. CSharp). We will create an ODBC connection to REST API and consume it different ways (e..g Bind API data to Data Grid, Combo Box or Write to File). ZappySys ODBC Driver allows to connect to many other REST API such as Facebook, Google, OneDrive, SharePoint and thousands of other sources that support REST API.

Requirements

1. First of all, you will need ZappySys ODBC PowerPack installed. This driver is a very powerful tool to connect with ODBC to REST API, JSON files, XML files, WEB API, OData and more.
2. Secondly, you will need Visual Studio Installed.

Getting Started

Let’s look at few examples to consume REST API or JSON data in C# applications (WPF, Winform, Console App or even Web Application such as ASP.net MVC or Webforms).

Calling REST API in C# to show REST API results in the console

In order to start, we will get the values using REST API. We will use C# to get the values. In this example, we will use the following URL to get data:

https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json

1. First of all, we will first create a Visual Studio Windows form using C#.
2. Secondly, we will add the following code to get the values from the API:

   using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};" 
        + "DataPath = 'https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"
        ))
   {
       conn.Open();
       OdbcCommand cmd = new OdbcCommand("SELECT CustomerID,CompanyName FROM value", conn);
       
       //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
       cmd.CommandTimeout=600; // 600-seconds
       
       var rdr = cmd.ExecuteReader();
       while (rdr.Read())
       {
           Console.WriteLine("---- Fetching Row -------");
           for (int i = 0; i < rdr.FieldCount; i++)
           {
               Console.Write("Field {0}={1} ", i, rdr[i]);
           }
           Console.WriteLine("");
       }
   }

3. This code connects to ODBC using the ZappySys JSON Driver and connection the customers URL: Also, we open the connection and send the SQL query.  With this query, you can use a simple SQL Query to REST API. Here you have the lists of sentences in SQL supported by the driver:

   SELECT
   [* | [ expression [[AS] column_name_alias] [, ...] ]
   [FROM table_name]
   [WHERE condition [, ...] ]
   [GROUP BY expression [, ...] ]
   [HAVING condition [, ...] ]
   [ORDER BY expression [ASC | DESC] [, ...] ]
   [LIMIT row_count]
   [WITH (option_name=option_value] [, ...]) ]

4. Finally, we show the information in the console:
   

   C# show rest API information

    

Here is full C# code to read from REST API in C#. Create a new Console Application Project in Visual Studio (File > New > Project > Visual C# > Console Application )

using System;
using System.IO;
using System.Data.Odbc;

public class Program
{
    public static void Main()
    {
        var outpath = @"C:\temp\sample.txt";
        
        using (var conn = new OdbcConnection("Driver={ZappySys JSON Driver};"
            +"DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))
        {
            conn.Open();
            var cmd = new OdbcCommand(@"SELECT CustomerID,CompanyName FROM $ WITH(Filter='$.value[*]')", conn);

            //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
            cmd.CommandTimeout=600; // 600-seconds
            var rdr = cmd.ExecuteReader();

            using (var fs = new FileStream(outpath, FileMode.Create))
            {
                using (var writer = new StreamWriter(fs))
                {
                    //write file header
                    writer.WriteLine("CustomerID,CompanyName");

                    while (rdr.Read())
                    {
                        //write file row
                        writer.WriteLine("{0},{1}", rdr["CustomerID"], rdr["CompanyName"]);
                    }
                    conn.Close(); //close connection
                    writer.Close();
                    fs.Close();
                }
            }
        }

        //Read from file and display the content
        Console.Write(File.ReadAllText(outpath));

        Console.WriteLine("\r\n===== Press any key to end the program =====\r\n");
        Console.Read();
    }
}

Calling REST API in C# to show REST API results in a combo box

In the next example, we will show how to call REST API in C# and load the data in a combo box. To execute this code you need to create a WinForm Project in Visual Studio  (File > New > Project > Visual C# > Windows Form Application)

1. First of all, in a C# project, add the following code:
   

   try
   {
       using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))
       {
           conn.Open();
           OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);
   
           //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
           cmd.CommandTimeout=600; // 600-seconds
   
           var rdr = cmd.ExecuteReader();
           while (rdr.Read())
           {
               Console.WriteLine("---- Fetching Row -------");
               for (int i = 0; i < rdr.FieldCount; i++)
               {
                   comboBox1.Items.Add(rdr[i].ToString());
               }
               Console.WriteLine("");
               conn.Close();
           }
       }
   }
   catch (Exception ex)
   {
       MessageBox.Show(ex.ToString());
   }

2. Secondly, you will need to add a combo box in the project.
3. The code is similar to the previous example, but we are filling a
   combo box with the following line of code:
   

   comboBox1.Items.Add(rdr[i].ToString());

4. In addition, the query is different because we are doing a select to just
   one column:
   

   OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);

5. Finally, execute the code to see the results:
   

   Load combo box c# REST API

Calling REST API in C# to show REST API in a Data Grid

The DataGridView is very popular to retrieve data. In this example, we will show how to load REST API in the DataGridView.

1. First of all, add the following code:
   

   try
   {
       string Conn = "Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';";
       string Sql = "select CustomerID,CompanyName from value";
   
       using (OdbcConnection con = new OdbcConnection(Conn))
       using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con))
       {
           DataTable table = new DataTable();
           dadapter.Fill(table);
   
           this.dataGridView1.DataSource = table;
       }
   }
   catch (Exception ex)
   {
       MessageBox.Show(ex.ToString());
   }

2. We first create an ODBC connection with the SQL query:
   

   string Conn = "Driver={ZappySys JSON Driver};
   DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';";
   string Sql = "select CustomerID,CompanyName from value";
   using (OdbcConnection con = new OdbcConnection(Conn))

3. Also, we create a Data Table and fill with data:
   

   using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con))
   {
        DataTable table = new DataTable();
        dadapter.Fill(table);

4. In addition, we will add the table to the dataGridView:
   

   this.dataGridView1.DataSource = table;

5. Finally, execute the code to see the results:
   

   Datagridview c#

Read from JSON file in C# (Single or Multiple files)

The ZappySys ODBC PowerPack allows connecting not only to REST API, WEB API but also to local files. In this example, we will connect to a file named test.json. Let’s take a look at this example to see how to do it:

1. First of all, we will have a JSON file named test.json with this content:
   

   {
     "value": [
       {
         "CustomerID": "ALFKI",
         "CompanyName": "Alfreds Futterkiste",
         "ContactName": "Maria Anders",
         "ContactTitle": "Sales Representative",
         "Address": "Obere Str. 57",
         "City": "Berlin",
         "Region": null,
         "PostalCode": "12209",
         "Country": "Germany",
         "Phone": "030-0074321",
         "Fax": "030-0076545"
       },
       {
         "CustomerID": "ANATR",
         "CompanyName": "Ana Trujillo Emparedados y helados",
         "ContactName": "Ana Trujillo",
         "ContactTitle": "Owner",
         "Address": "Avda. de la Constitución 2222",
         "City": "México D.F.",
         "Region": null,
         "PostalCode": "05021",
         "Country": "Mexico",
         "Phone": "(5) 555-4729",
         "Fax": "(5) 555-3745"
       },
       {
         "CustomerID": "ANTON",
         "CompanyName": "Antonio Moreno Taquería",
         "ContactName": "Antonio Moreno",
         "ContactTitle": "Owner",
         "Address": "Mataderos 2312",
         "City": "México D.F.",
         "Region": null,
         "PostalCode": "05023",
         "Country": "Mexico",
         "Phone": "(5) 555-3932",
         "Fax": null
       },
       {
         "CustomerID": "AROUT",
         "CompanyName": "Around the Horn",
         "ContactName": "Thomas Hardy",
         "ContactTitle": "Sales Representative",
         "Address": "120 Hanover Sq.",
         "City": "London",
         "Region": null,
         "PostalCode": "WA1 1DP",
         "Country": "UK",
         "Phone": "(171) 555-7788",
         "Fax": "(171) 555-6750"
       },
       {
         "CustomerID": "BOLID",
         "CompanyName": "Bólido Comidas preparadas",
         "ContactName": "Martín Sommer",
         "ContactTitle": "Owner",
         "Address": "C/ Araquil, 67",
         "City": "Madrid",
         "Region": null,
         "PostalCode": "28023",
         "Country": "Spain",
         "Phone": "(91) 555 22 82",
         "Fax": "(91) 555 91 99"
       },
   
       {
         "CustomerID": "ERNSH",
         "CompanyName": "Ernst Handel",
         "ContactName": "Roland Mendel",
         "ContactTitle": "Sales Manager",
         "Address": "Kirchgasse 6",
         "City": "Graz",
         "Region": null,
         "PostalCode": "8010",
         "Country": "Austria",
         "Phone": "7675-3425",
         "Fax": "7675-3426"
       }
     ]
   }

2. Secondly, we will create the following code to load the data in a combo box:
   NOTE: To read from multiple files use wild card pattern (e.g. c:\data\test*.json)
   

   using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};" +
       @"DataPath = 'c:\data\test.json'; "))
   {
       conn.Open();
       OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);
   
       //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
       cmd.CommandTimeout=600; // 600-seconds
   
       var rdr = cmd.ExecuteReader();
       while (rdr.Read())
       {
           for (int i = 0; i < rdr.FieldCount; i++)
           {
   
               comboBox1.Items.Add(rdr[i].ToString());
           }
   
       }
   }

3. Finally, execute the code to see the results:
   

   Read JSON file in C#

Calling REST API in C# to get Gmail information

The next example will show how to get Gmail information in C# using REST API.

1. First of all, go to the ODBC Administrator in Windows:
   

   Windows ODBC Data Sources

2. Secondly, in the ODBC Administrator press the add button:
3.  Also, select the ZappySys JSON driver.
4. In addition, enter the DSN and the URL. In this example, we will use the following URL:
   

   https://www.googleapis.com/gmail/v1/users/me/messages/1642915a12765abe

5. Where 1642915a12765abe is the ID of the email that you can get from the email:
   

   REST API gmail

6. Additionally, you need to select the authentication type to Oauth, select the GET HTTP method and press the Click to configure link.
7. In the same way, we can configure the OAuth parameters. Select the Google OAuth provider, in scopes select the following URL:
   

   https://mail.google.com

   

   OAuth credential REST API Gmail

8. The code used to get data is the following:
   

   try
               {
                   string Conn = "dsn=ZappySys JSON Gmail;";
                   string Sql = "select * from [_root_]";
   
                   using (OdbcConnection con = new OdbcConnection(Conn))
                   using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con))
                   {
                       DataTable table = new DataTable();
                       dadapter.Fill(table);
   
                       this.dataGridView1.DataSource = table;
                   }
               }
               catch (Exception ex)
               {
   
   
                   MessageBox.Show(ex.ToString());

9. Note that we are invoking a DSN in the code:
   

   string Conn = "dsn=ZappySys JSON Gmail;";

10. Also, note that the query used to get data is the following:
    

    string Sql = "select * from [_root_]";

11. If everything is OK, you will be able to see the mail information in the dataGridView:
    

    Gmail DataGrid View

ZappySys JSON /REST API Driver Query Examples

Reading from XML files or API can be done using the same way as previous sections except you have to use ZappySys XML Driver. Read help file here to see json query examples.

ZappySys XML / SOAP Driver Query Examples

Reading from XML files or API can be done using the same way as previous sections except you have to use ZappySys XML Driver. Read help file here to see xml query examples.

Calling XML SOAP Web Service in C#

So far we have looked at examples to consume data using JSON driver. Now lets look at an example, to call XML SOAP Web Service in Qlik.

To call SOAP API you need to know Request XML Body Structure. If you are not sure how to create SOAP Request body then no worries. Check this article to learn how to generate SOAP Request body using the Free tool SoapUI. Basically, you have to use SoapUI to generate Request XML and after that, you can replace parameters as needed in the generated body.

What is SOAP Web Service?

If you are new to SOAP Web Service sometimes referred as XML Web Service then please read some concept about SOAP Web service standard from this link There are two important aspects in SOAP Web service.

1. Getting WSDL file or URL
2. Knowing exact Web Service URL

What is WSDL

In very simple term WSDL (often pronounced as whiz-dull) is nothing but a document which describes Service metadata (e.g. Functions you can call, Request parameters, response structure etc). Some service simply give you WSDL as xml file you can download on local machine and then analyze or sometimes you may get direct URL (e.g. http://api.mycompany.com/hr-soap-service/?wsdl )

Example SQL Query for SOAP API call using ZappySys XML Driver

Here is an example SQL query you can write to call SOAP API. If you not sure about many details then check next few sections on how to use XML Driver User Interface to build desired SQL query to POST data to XML SOAP Web Service without any coding.

SELECT * FROM $
WITH(
	 Src='http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx'
	,DataConnectionType='HTTP'
	,CredentialType='Basic' --OR SoapWss
	,SoapWssPasswordType='PasswordText'
	,UserName='myuser'
	,Password='pass$$w123'
	,Filter='$.soap:Envelope.soap:Body.GetHolidaysAvailableResponse.GetHolidaysAvailableResult.HolidayCode[*]'
	,ElementsToTreatAsArray='HolidayCode'	
	,RequestMethod='POST'	
	,Header='Content-Type: text/xml;charset=UTF-8 || SOAPAction: "http://www.holidaywebservice.com/HolidayService_v2/GetHolidaysAvailable"'
	,RequestData='
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:hol="http://www.holidaywebservice.com/HolidayService_v2/">
   <soapenv:Header/>
   <soapenv:Body>
      <hol:GetHolidaysAvailable>
         <!--type: Country - enumeration: [Canada,GreatBritain,IrelandNorthern,IrelandRepublicOf,Scotland,UnitedStates]-->
         <hol:countryCode>UnitedStates</hol:countryCode>
      </hol:GetHolidaysAvailable>
   </soapenv:Body>
</soapenv:Envelope>'
)

Now let's look at steps to create SQL query to call SOAP API. Later we will see how to generate code for your desired programming language (e.g. C# or SQL Server)

Video Tutorial - Introduction to SOAP Web Service and SoapUI tool

Before we dive into details about calling SOAP API using ZappySys XML Driver, lets first understand what is SOAP API and how to create SOAP requests using SoapUI tool. You will learn more about this process in the later section. The video contains some fragment about using SOAP API in SSIS but just ignore that part because we will be calling Soap API using ZappySys ODBC Driver rather than SSIS Components.  

Using SoapUI to test SOAP API call / Create Request Body XML

Assuming you have downloaded and installed SoapUI from here, now we are ready to use WSDL for your SOAP Web Service Calls. If you do not have WSDL file or URL handy then contact your API provider (sometimes you just have to add ?wsdl at the end of your Service URL to get WSDL so try that. Example: http://mycompany/myservice?wsdl ). If you don't know what is WSDL then in short, WSDL is Web service Description Language (i.e. XML file which describes your SOAP Service). WSDL helps to craft SOAP API request Body for ZappySys XML Driver. So Let's get started.

1. Open SoapUI and click SOAP button to create new SOAP Project
2. Enter WSDL URL or File Path of WSDLFor example WSDL for our sample service can be accessed via this URL

   http://www.dneonline.com/calculator.asmx?wsdl

   
   Create new SOAP API Project in SoapUI tool for SOAP API Testing
3. Once WSDL is loaded you will see possible operations you can call for your SOAP Web Service.
4. If your web service requires credentials then you have to configure it. There are two common credential types for public services (SOAP WSS or BASIC )
   1. To use SOAP WSS Credentials select request node and enter UserId, Password, and WSS-PasswordType (PasswordText or PasswordHash)
      
      Configure SOAP WSS Credentials for SoapUI (SOAP API Testing Tool)
   2. To use BASIC Auth Credentials select request node and double-click it. At the bottom click on Auth (Basic) and From Authorization dropdown click Add New and Select Basic.
      Configure Basic Authorization for SoapUI (SOAP API Testing Tool)
5. Now you can test your request first Double-click on the request node to open request editor.
6. Change necessary parameters, remove optional or unwanted parameters. If you want to regenerate request you can click on Recreate default request toolbar icon.
   Create SOAP Request XML (With Optional Parameters)
7. Once your SOAP Request XML is ready, Click the Play button in the toolbar to execute SOAP API Request and Response will appear in Right side panel. Test SOAP API using SoapUI Tool (Change Default XML Body / Parameters, Execute and See Response)

Create DSN using ZappySys XML Driver to call SOAP API

Once you have tested your SOAP API in SoapUI tool, we are ready to use ZappySys XML driver to call SOAP API in your preferred BI tool or Programming language.

1. First open ODBC Data Sources (search ODBC in your start menu or go under ZappySys > ODBC PowerPack > ODBC 64 bit)
2. Goto System DSN Tab (or User DSN which is not used by Service account)
3. Click Add and Select ZappySys XML Driver ZappySys ODBC Driver for XML / SOAP API
4. Configure API URL, Request Method and Request Body as below ZappySys XML Driver - Calling SOAP API - Configure URL, Method, Body
5. (This step is Optional) If your SOAP API requires credentials then Select Connection Type to HTTP and configure as below.
   ZappySys XML Driver - Configure SOAP WSS Credentials or Basic Authorization (Userid, Password)
6. Configure-Request Headers as below (You can get it from Request > Raw tab from SoapUI after you test the request by clicking the Play button) Configure SOAP API Request Headers - ZappySys XML Driver
7. Once credentials entered you can select Filter to extract data from the desired node. Make sure to select array node (see special icon) or select the node which contains all necessary columns if you don't have array node. Select Filter - Extract data from nested XML / SOAP API Response (Denormalize Hierarchy)
8. If prompted select yes to treat selected node as Array (This is helpful when you expect one or more record for selected node) Treat selected node as XML Array Option for SOAP API Response XML

Preview SOAP API Response / Generate SQL Code for SOAP API Call

Once you configure settings for XML Driver now you can preview data or generate example code for desired language (e.g. C#, Python, Java, SQL Server). Go to Preview tab and you will see default query generated based on settings you entered in previous sections. Attributes listed in WITH clause are optional. If you omit attribute in WITH clause it will use it from Properties tab.

Preview Data

Preview SOAP API Response in ZappySys XML Driver

Generate Code Option

Generate Example Code for ZappySys Driver

POST data to REST API URL from file in C#

Above example was POST data to API URL but what if your Request Body is large and you have saved that to file? Well here is the way to get your request body from a file (Use @ symbol before path).

SELECT * FROM $
WITH 
(METHOD='POST' 
,HEADER='Content-Type:text/plain || x-hdr1:AAA'
,SRC='http://httpbin.org/post'
,BODY='@c:\files\dump.xml'
,IsMultiPart='True'
)

REST API Pagination in C#

SELECT * FROM $
WITH(
SRC=@'https://zappysys.com/downloads/files/test/pagination_nextlink_inarray_1.json'
,NextUrlAttributeOrExpr = '$.nextlink'  --keep reading until this attribute is missing. If attribute name contains dot then use brackets like this $.['my.attr.name']
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/page-xml.aspx?page=1&mode=DetectBasedOnResponseStatusCode'
,PagingMode='ByUrlParameter'
,PagingByUrlAttributeName='page'
,PagingByUrlEndStrategy='DetectBasedOnResponseStatusCode'
,PagingByUrlCheckResponseStatusCode=401
,IncrementBy=1
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/cust-<%page%>.xml'
,PagingMode='ByUrlPath'
,PagingByUrlAttributeName='<%page%>'
,PagingByUrlEndStrategy='DetectBasedOnRecordCount'
,IncrementBy=1
)

SELECT * FROM $
LIMIT 25
WITH(
	 Src='https://wordpress.org/news/wp-json/wp/v2/categories?per_page=10'
	,PagingMode='ByResponseHeaderRfc5988'
	,WaitTimeMs='200' --//wait 200 ms after each request
)

Error Handling in REST API / SOAP

Sometimes errors occur... they just do and there is nothing you can do! Or can you? Actually, in ODBC PowerPack you can handle them in two ways.

METHOD 1 - Using Error Handling Options

When to use?

You may want to use them when your source is a resource located on the Internet; e.g. a file on a website, a file on an FTP server or just a plain API HTTP response. By default, when a remote server returns an error, data retrieval is stopped, an error is raised and no data is given back to you. This might not be always desirable.

Scenario 1

Imagine a scenario, that there is a web server which each day at 12 AM releases a new JSON file with that day's date as filename, e.g. http://www.some-server.com/data/2018-06-20.json. And, of course, you want to download it and use it daily in your Power BI report. But you have a problem: Power BI report data sources are refreshed each hour and you may get HTTP 404 status code (no file was found) when a file is not released yet. Which consequentially means other data sources won't be updated as well and you will see old and cached data on the report. That's where you could use Continue on any error or Continue when Url is invalid or missing (404 Errors) to avoid an error being raised and let other data sources to be updated.

Scenario 2

Another scenario is when you expect a web server to raise some kind of HTTP error when accessing a URL. You don't want ODBC Data Source to raise an error but instead, you want to get response data. That's where you can use Continue on any error or alike together with  Get response data on error to continue on an error and get the data:

METHOD 2 - Using Connection [Retry Settings]

Another scenario you may run into is a buggy web server. You ask it to give you some file or data and it, like a snotty kid, just doesn't give it to you! You have to ask twice or thrice before it does its job. If that's the case, you have to retry HTTP requests using Connection:

REST API / SOAP Connection Types in C# (OAuth / HTTP)

If you need to authenticate or authorize your user to access a web resource, you will need to use one of the Connections:

• HTTP
• OAuth

HTTP Connection

Use HTTP Connection for simple Windows, Basic, NTLM or Kerberos authentication. Just fill in a username and a password and you are good to go!

You can also use HTTP Connection for more sophisticated authentication like:

• SOAP WSS (when accessing a SOAP WebService)
• Static Token / API Key (when need to pass an API key in HTTP header)
• Dynamic Token (same as Static Token method except that each time you need to log in and retrieve a fresh API key)
• JWT Token (As per RFC 7519)

OAuth

If you are trying to access REST API resource, it is a huge chance, you will need to use OAuth Connection. Read this article to understand how OAuth authentication and authorization works and how to use it (article originally was written for SSIS PowerPack, but the concepts and UI stay the same): 
https://zappysys.com/blog/rest-api-authentication-with-oauth-2-0-using-ssis/

Performance consideration for Web API Calls

SELECT * FROM value
WITH(
	 Src='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json&$filter=Country eq ''USA'''
	,DataFormat='Odata'
)

-- Slow query - Client-side filtering
SELECT * FROM value
WHERE Country ='USA'
WITH(
	 Src='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json'
	,DataFormat='Odata'
)

SELECT * FROM $
WITH 
(  SRC='https://myhost.com/some-api'
  ,CachingMode='All'  --cache metadata and data rows both
  ,CacheStorage='File' --or Memory
  ,CacheFileLocation='c:\temp\myquery.cache'
  ,CacheEntryTtl=300 --cache for 300 seconds
)

SELECT * FROM $ 
LIMIT 10 --//add this just to test how fast you can get 10 rows
WITH(
  Filter='$.LargeArray[*]--FAST' --//Adding --FAST option turn on STREAM mode (large files)
 ,SRC='https://zappysys.com/downloads/files/test/large_file_100k_largearray_prop.json.gz'
 --,SRC='c:\data\large_file.json.gz'
 ,IncludeParentColumns='False'  --//This Must be OFF for STREAM mode (read very large files)
 ,FileCompressionType='GZip' --Zip or None (Zip format only available for Local files)
)

Other considerations for Web API calls in C#

Conclusion

In this article, we saw how to use C# to call REST API. We show how to do REST API calls in C# using the ZappySys ODBC driver. Also, we show how to get data from local files and get Gmail information with REST API. If you liked this article and you want to try, you can download the ZappySys ODBC installer here.

References

• ZappySys ODBC installer.
• How To Use the ODBC .NET Managed Provider in Visual C# .NET and Connection Strings

The post Calling REST API in C# (Read JSON Data) appeared first on ZappySys Blog.