Calling REST API in C# (Read JSON Data)

In this post, We will use ZappySys ODBC Powerpack for calling REST API in C# (i.e. CSharp). We will create an ODBC connection to REST API and consume it different ways (e..g Bind API data to Data Grid, Combo Box or Write to File). ZappySys ODBC Driver allows to connect to many other REST API such as Facebook, Google, OneDrive, SharePoint and thousands of other sources that support REST API.

Requirements

First of all, you will need ZappySys ODBC PowerPack installed. This driver is a very powerful tool to connect with ODBC to REST API, JSON files, XML files, WEB API, OData and more.
Secondly, you will need Visual Studio Installed.

Getting Started

Let’s look at few examples to consume REST API or JSON data in C# applications (WPF, Winform, Console App or even Web Application such as ASP.net MVC or Webforms).

Calling REST API in C# to show REST API results in the console

In order to start, we will get the values using REST API. We will use C# to get the values. In this example, we will use the following URL to get data:

https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json

1	https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json

First of all, we will first create a Visual Studio Windows form using C#.

Secondly, we will add the following code to get the values from the API:

using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};" 
     + "DataPath = 'https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"
     ))
{
    conn.Open();
    OdbcCommand cmd = new OdbcCommand("SELECT CustomerID,CompanyName FROM value", conn);
    
    //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
    cmd.CommandTimeout=600; // 600-seconds
    
    var rdr = cmd.ExecuteReader();
    while (rdr.Read())
    {
        Console.WriteLine("---- Fetching Row -------");
        for (int i = 0; i < rdr.FieldCount; i++)
        {
            Console.Write("Field {0}={1} ", i, rdr[i]);
        }
        Console.WriteLine("");
    }
}

using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};"

+ "DataPath = 'https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"

))

{

conn.Open();

OdbcCommand cmd = new OdbcCommand("SELECT CustomerID,CompanyName FROM value", conn);

//Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.

cmd.CommandTimeout=600; // 600-seconds

var rdr = cmd.ExecuteReader();

while (rdr.Read())

{

Console.WriteLine("---- Fetching Row -------");

for (int i = 0; i < rdr.FieldCount; i++)

{

Console.Write("Field {0}={1} ", i, rdr[i]);

}

Console.WriteLine("");

}

This code connects to ODBC using the ZappySys JSON Driver and connection the customers URL: Also, we open the connection and send the SQL query. With this query, you can use a simple SQL Query to REST API. Here you have the lists of sentences in SQL supported by the driver:

SELECT
[* | [ expression [[AS] column_name_alias] [, ...] ]
[FROM table_name]
[WHERE condition [, ...] ]
[GROUP BY expression [, ...] ]
[HAVING condition [, ...] ]
[ORDER BY expression [ASC | DESC] [, ...] ]
[LIMIT row_count]
[WITH (option_name=option_value] [, ...]) ]

SELECT

[* | [ expression [[AS] column_name_alias] [, ...] ]

[FROM table_name]

[WHERE condition [, ...] ]

[GROUP BY expression [, ...] ]

[HAVING condition [, ...] ]

[ORDER BY expression [ASC | DESC] [, ...] ]

[LIMIT row_count]

[WITH (option_name=option_value] [, ...]) ]

Finally, we show the information in the console:
C# show rest API information

Here is full C# code to read from REST API in C#. Create a new Console Application Project in Visual Studio (File > New > Project > Visual C# > Console Application )

using System;
using System.IO;
using System.Data.Odbc;

public class Program
{
    public static void Main()
    {
        var outpath = @"C:\temp\sample.txt";
        
        using (var conn = new OdbcConnection("Driver={ZappySys JSON Driver};"
            +"DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))
        {
            conn.Open();
            var cmd = new OdbcCommand(@"SELECT CustomerID,CompanyName FROM $ WITH(Filter='$.value[*]')", conn);

            //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
            cmd.CommandTimeout=600; // 600-seconds
            var rdr = cmd.ExecuteReader();

            using (var fs = new FileStream(outpath, FileMode.Create))
            {
                using (var writer = new StreamWriter(fs))
                {
                    //write file header
                    writer.WriteLine("CustomerID,CompanyName");

                    while (rdr.Read())
                    {
                        //write file row
                        writer.WriteLine("{0},{1}", rdr["CustomerID"], rdr["CompanyName"]);
                    }
                    conn.Close(); //close connection
                    writer.Close();
                    fs.Close();
                }
            }
        }

        //Read from file and display the content
        Console.Write(File.ReadAllText(outpath));

        Console.WriteLine("\r\n===== Press any key to end the program =====\r\n");
        Console.Read();
    }
}

using System;

using System.IO;

using System.Data.Odbc;

public class Program

{

public static void Main()

{

var outpath = @"C:\temp\sample.txt";

using (var conn = new OdbcConnection("Driver={ZappySys JSON Driver};"

+"DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))

{

conn.Open();

var cmd = new OdbcCommand(@"SELECT CustomerID,CompanyName FROM $ WITH(Filter='$.value[*]')", conn);

//Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.

cmd.CommandTimeout=600; // 600-seconds

var rdr = cmd.ExecuteReader();

using (var fs = new FileStream(outpath, FileMode.Create))

{

using (var writer = new StreamWriter(fs))

{

//write file header

writer.WriteLine("CustomerID,CompanyName");

while (rdr.Read())

{

//write file row

writer.WriteLine("{0},{1}", rdr["CustomerID"], rdr["CompanyName"]);

}

conn.Close(); //close connection

writer.Close();

fs.Close();

}

//Read from file and display the content

Console.Write(File.ReadAllText(outpath));

Console.WriteLine("\r\n===== Press any key to end the program =====\r\n");

Console.Read();

}

Calling REST API in C# to show REST API results in a combo box

In the next example, we will show how to call REST API in C# and load the data in a combo box. To execute this code you need to create a WinForm Project in Visual Studio (File > New > Project > Visual C# > Windows Form Application)

First of all, in a C# project, add the following code:

try
{
    using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))
    {
        conn.Open();
        OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);

        //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
        cmd.CommandTimeout=600; // 600-seconds

        var rdr = cmd.ExecuteReader();
        while (rdr.Read())
        {
            Console.WriteLine("---- Fetching Row -------");
            for (int i = 0; i < rdr.FieldCount; i++)
            {
                comboBox1.Items.Add(rdr[i].ToString());
            }
            Console.WriteLine("");
            conn.Close();
        }
    }
}
catch (Exception ex)
{
    MessageBox.Show(ex.ToString());
}

try

{

using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';"))

{

conn.Open();

OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);

//Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.

cmd.CommandTimeout=600; // 600-seconds

var rdr = cmd.ExecuteReader();

while (rdr.Read())

{

Console.WriteLine("---- Fetching Row -------");

for (int i = 0; i < rdr.FieldCount; i++)

{

comboBox1.Items.Add(rdr[i].ToString());

}

Console.WriteLine("");

conn.Close();

}

catch (Exception ex)

{

MessageBox.Show(ex.ToString());

}

Secondly, you will need to add a combo box in the project.
The code is similar to the previous example, but we are filling a
combo box with the following line of code:

C#

comboBox1.Items.Add(rdr[i].ToString());

1

comboBox1.Items.Add(rdr[i].ToString());
In addition, the query is different because we are doing a select to just
one column:

C#

OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);

1

OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);
Finally, execute the code to see the results:
Load combo box c# REST API

Calling REST API in C# to show REST API in a Data Grid

The DataGridView is very popular to retrieve data. In this example, we will show how to load REST API in the DataGridView.

First of all, add the following code:

try
{
    string Conn = "Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';";
    string Sql = "select CustomerID,CompanyName from value";

    using (OdbcConnection con = new OdbcConnection(Conn))
    using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con))
    {
        DataTable table = new DataTable();
        dadapter.Fill(table);

        this.dataGridView1.DataSource = table;
    }
}
catch (Exception ex)
{
    MessageBox.Show(ex.ToString());
}

try

{

string Conn = "Driver={ZappySys JSON Driver};DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';";

string Sql = "select CustomerID,CompanyName from value";

using (OdbcConnection con = new OdbcConnection(Conn))

using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con))

{

DataTable table = new DataTable();

dadapter.Fill(table);

this.dataGridView1.DataSource = table;

}

catch (Exception ex)

{

MessageBox.Show(ex.ToString());

}

We first create an ODBC connection with the SQL query:

string Conn = "Driver={ZappySys JSON Driver};
DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';";
string Sql = "select CustomerID,CompanyName from value";
using (OdbcConnection con = new OdbcConnection(Conn))

string Conn = "Driver={ZappySys JSON Driver};

DataPath='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json';";

string Sql = "select CustomerID,CompanyName from value";

using (OdbcConnection con = new OdbcConnection(Conn))

Also, we create a Data Table and fill with data:

C#

using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con)) { DataTable table = new DataTable(); dadapter.Fill(table);

1
2
3
4

using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con))
{
DataTable table = new DataTable();
dadapter.Fill(table);
In addition, we will add the table to the dataGridView:

C#

this.dataGridView1.DataSource = table;

1

this.dataGridView1.DataSource = table;
Finally, execute the code to see the results:
Datagridview c#

Read from JSON file in C# (Single or Multiple files)

The ZappySys ODBC PowerPack allows connecting not only to REST API, WEB API but also to local files. In this example, we will connect to a file named test.json. Let’s take a look at this example to see how to do it:

First of all, we will have a JSON file named test.json with this content:

{
  "value": [
    {
      "CustomerID": "ALFKI",
      "CompanyName": "Alfreds Futterkiste",
      "ContactName": "Maria Anders",
      "ContactTitle": "Sales Representative",
      "Address": "Obere Str. 57",
      "City": "Berlin",
      "Region": null,
      "PostalCode": "12209",
      "Country": "Germany",
      "Phone": "030-0074321",
      "Fax": "030-0076545"
    },
    {
      "CustomerID": "ANATR",
      "CompanyName": "Ana Trujillo Emparedados y helados",
      "ContactName": "Ana Trujillo",
      "ContactTitle": "Owner",
      "Address": "Avda. de la Constitución 2222",
      "City": "México D.F.",
      "Region": null,
      "PostalCode": "05021",
      "Country": "Mexico",
      "Phone": "(5) 555-4729",
      "Fax": "(5) 555-3745"
    },
    {
      "CustomerID": "ANTON",
      "CompanyName": "Antonio Moreno Taquería",
      "ContactName": "Antonio Moreno",
      "ContactTitle": "Owner",
      "Address": "Mataderos 2312",
      "City": "México D.F.",
      "Region": null,
      "PostalCode": "05023",
      "Country": "Mexico",
      "Phone": "(5) 555-3932",
      "Fax": null
    },
    {
      "CustomerID": "AROUT",
      "CompanyName": "Around the Horn",
      "ContactName": "Thomas Hardy",
      "ContactTitle": "Sales Representative",
      "Address": "120 Hanover Sq.",
      "City": "London",
      "Region": null,
      "PostalCode": "WA1 1DP",
      "Country": "UK",
      "Phone": "(171) 555-7788",
      "Fax": "(171) 555-6750"
    },
    {
      "CustomerID": "BOLID",
      "CompanyName": "Bólido Comidas preparadas",
      "ContactName": "Martín Sommer",
      "ContactTitle": "Owner",
      "Address": "C/ Araquil, 67",
      "City": "Madrid",
      "Region": null,
      "PostalCode": "28023",
      "Country": "Spain",
      "Phone": "(91) 555 22 82",
      "Fax": "(91) 555 91 99"
    },

    {
      "CustomerID": "ERNSH",
      "CompanyName": "Ernst Handel",
      "ContactName": "Roland Mendel",
      "ContactTitle": "Sales Manager",
      "Address": "Kirchgasse 6",
      "City": "Graz",
      "Region": null,
      "PostalCode": "8010",
      "Country": "Austria",
      "Phone": "7675-3425",
      "Fax": "7675-3426"
    }
  ]
}

{

"value": [

{

"CustomerID": "ALFKI",

"CompanyName": "Alfreds Futterkiste",

"ContactName": "Maria Anders",

"ContactTitle": "Sales Representative",

"Address": "Obere Str. 57",

"City": "Berlin",

"Region": null,

"PostalCode": "12209",

"Country": "Germany",

"Phone": "030-0074321",

"Fax": "030-0076545"

{

"CustomerID": "ANATR",

"CompanyName": "Ana Trujillo Emparedados y helados",

"ContactName": "Ana Trujillo",

"ContactTitle": "Owner",

"Address": "Avda. de la Constitución 2222",

"City": "México D.F.",

"Region": null,

"PostalCode": "05021",

"Country": "Mexico",

"Phone": "(5) 555-4729",

"Fax": "(5) 555-3745"

{

"CustomerID": "ANTON",

"CompanyName": "Antonio Moreno Taquería",

"ContactName": "Antonio Moreno",

"ContactTitle": "Owner",

"Address": "Mataderos 2312",

"City": "México D.F.",

"Region": null,

"PostalCode": "05023",

"Country": "Mexico",

"Phone": "(5) 555-3932",

"Fax": null

{

"CustomerID": "AROUT",

"CompanyName": "Around the Horn",

"ContactName": "Thomas Hardy",

"ContactTitle": "Sales Representative",

"Address": "120 Hanover Sq.",

"City": "London",

"Region": null,

"PostalCode": "WA1 1DP",

"Country": "UK",

"Phone": "(171) 555-7788",

"Fax": "(171) 555-6750"

{

"CustomerID": "BOLID",

"CompanyName": "Bólido Comidas preparadas",

"ContactName": "Martín Sommer",

"ContactTitle": "Owner",

"Address": "C/ Araquil, 67",

"City": "Madrid",

"Region": null,

"PostalCode": "28023",

"Country": "Spain",

"Phone": "(91) 555 22 82",

"Fax": "(91) 555 91 99"

{

"CustomerID": "ERNSH",

"CompanyName": "Ernst Handel",

"ContactName": "Roland Mendel",

"ContactTitle": "Sales Manager",

"Address": "Kirchgasse 6",

"City": "Graz",

"Region": null,

"PostalCode": "8010",

"Country": "Austria",

"Phone": "7675-3425",

"Fax": "7675-3426"

}

]

}

Secondly, we will create the following code to load the data in a combo box:
NOTE: To read from multiple files use wild card pattern (e.g. c:\data\test*.json)

using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};" +
    @"DataPath = 'c:\data\test.json'; "))
{
    conn.Open();
    OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);

    //Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.
    cmd.CommandTimeout=600; // 600-seconds

    var rdr = cmd.ExecuteReader();
    while (rdr.Read())
    {
        for (int i = 0; i < rdr.FieldCount; i++)
        {

            comboBox1.Items.Add(rdr[i].ToString());
        }

    }
}

using (OdbcConnection conn = new OdbcConnection("Driver={ZappySys JSON Driver};" +

@"DataPath = 'c:\data\test.json'; "))

{

conn.Open();

OdbcCommand cmd = new OdbcCommand("SELECT CustomerID FROM value", conn);

//Increases the timeout duration from the default 30 seconds, which may be insufficient in certain scenarios.

cmd.CommandTimeout=600; // 600-seconds

var rdr = cmd.ExecuteReader();

while (rdr.Read())

{

for (int i = 0; i < rdr.FieldCount; i++)

{

comboBox1.Items.Add(rdr[i].ToString());

}

Finally, execute the code to see the results:
Read JSON file in C#

Calling REST API in C# to get Gmail information

The next example will show how to get Gmail information in C# using REST API.

First of all, go to the ODBC Administrator in Windows:
Windows ODBC Data Sources
Secondly, in the ODBC Administrator press the add button:
Also, select the ZappySys JSON driver.
In addition, enter the DSN and the URL. In this example, we will use the following URL:

https://www.googleapis.com/gmail/v1/users/me/messages/1642915a12765abe

1

https://www.googleapis.com/gmail/v1/users/me/messages/1642915a12765abe
Where 1642915a12765abe is the ID of the email that you can get from the email:
REST API gmail
Additionally, you need to select the authentication type to Oauth, select the GET HTTP method and press the Click to configure link.
In the same way, we can configure the OAuth parameters. Select the Google OAuth provider, in scopes select the following URL:

https://mail.google.com

1

https://mail.google.com

OAuth credential REST API Gmail

The code used to get data is the following:

           try
            {
                string Conn = "dsn=ZappySys JSON Gmail;";
                string Sql = "select * from [_root_]";

                using (OdbcConnection con = new OdbcConnection(Conn))
                using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con))
                {
                    DataTable table = new DataTable();
                    dadapter.Fill(table);

                    this.dataGridView1.DataSource = table;
                }
            }
            catch (Exception ex)
            {


                MessageBox.Show(ex.ToString());

try

{

string Conn = "dsn=ZappySys JSON Gmail;";

string Sql = "select * from [_root_]";

using (OdbcConnection con = new OdbcConnection(Conn))

using (OdbcDataAdapter dadapter = new OdbcDataAdapter(Sql, con))

{

DataTable table = new DataTable();

dadapter.Fill(table);

this.dataGridView1.DataSource = table;

}

catch (Exception ex)

{

MessageBox.Show(ex.ToString());

Note that we are invoking a DSN in the code:

C#

string Conn = "dsn=ZappySys JSON Gmail;";

1

string Conn = "dsn=ZappySys JSON Gmail;";
Also, note that the query used to get data is the following:

C#

string Sql = "select * from [_root_]";

1

string Sql = "select * from [_root_]";
If everything is OK, you will be able to see the mail information in the dataGridView:
Gmail DataGrid View

ZappySys JSON /REST API Driver Query Examples

Reading from XML files or API can be done using the same way as previous sections except you have to use ZappySys XML Driver. Read help file here to see json query examples.

ZappySys XML / SOAP Driver Query Examples

Reading from XML files or API can be done using the same way as previous sections except you have to use ZappySys XML Driver. Read help file here to see xml query examples.

Calling XML SOAP Web Service in C#

So far we have looked at examples to consume data using JSON driver. Now lets look at an example, to call XML SOAP Web Service in Qlik.

To call SOAP API you need to know Request XML Body Structure. If you are not sure how to create SOAP Request body then no worries. Check this article to learn how to generate SOAP Request body using the Free tool SoapUI. Basically, you have to use SoapUI to generate Request XML and after that, you can replace parameters as needed in the generated body.

What is SOAP Web Service?

If you are new to SOAP Web Service sometimes referred as XML Web Service then please read some concept about SOAP Web service standard from this link There are two important aspects in SOAP Web service.

Getting WSDL file or URL
Knowing exact Web Service URL

What is WSDL

In very simple term WSDL (often pronounced as whiz-dull) is nothing but a document which describes Service metadata (e.g. Functions you can call, Request parameters, response structure etc). Some service simply give you WSDL as xml file you can download on local machine and then analyze or sometimes you may get direct URL (e.g. http://api.mycompany.com/hr-soap-service/?wsdl )

Example SQL Query for SOAP API call using ZappySys XML Driver

Here is an example SQL query you can write to call SOAP API. If you not sure about many details then check next few sections on how to use XML Driver User Interface to build desired SQL query to POST data to XML SOAP Web Service without any coding.

SELECT * FROM $
WITH(
	 Src='http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx'
	,DataConnectionType='HTTP'
	,CredentialType='Basic' --OR SoapWss
	,SoapWssPasswordType='PasswordText'
	,UserName='myuser'
	,Password='pass$$w123'
	,Filter='$.soap:Envelope.soap:Body.GetHolidaysAvailableResponse.GetHolidaysAvailableResult.HolidayCode[*]'
	,ElementsToTreatAsArray='HolidayCode'	
	,RequestMethod='POST'	
	,Header='Content-Type: text/xml;charset=UTF-8 || SOAPAction: "http://www.holidaywebservice.com/HolidayService_v2/GetHolidaysAvailable"'
	,RequestData='
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:hol="http://www.holidaywebservice.com/HolidayService_v2/">
   <soapenv:Header/>
   <soapenv:Body>
      <hol:GetHolidaysAvailable>
         <!--type: Country - enumeration: [Canada,GreatBritain,IrelandNorthern,IrelandRepublicOf,Scotland,UnitedStates]-->
         <hol:countryCode>UnitedStates</hol:countryCode>
      </hol:GetHolidaysAvailable>
   </soapenv:Body>
</soapenv:Envelope>'
)

Now let's look at steps to create SQL query to call SOAP API. Later we will see how to generate code for your desired programming language (e.g. C# or SQL Server)

Video Tutorial - Introduction to SOAP Web Service and SoapUI tool

Before we dive into details about calling SOAP API using ZappySys XML Driver, lets first understand what is SOAP API and how to create SOAP requests using SoapUI tool. You will learn more about this process in the later section. The video contains some fragment about using SOAP API in SSIS but just ignore that part because we will be calling Soap API using ZappySys ODBC Driver rather than SSIS Components.

Using SoapUI to test SOAP API call / Create Request Body XML

Assuming you have downloaded and installed SoapUI from here, now we are ready to use WSDL for your SOAP Web Service Calls. If you do not have WSDL file or URL handy then contact your API provider (sometimes you just have to add ?wsdl at the end of your Service URL to get WSDL so try that. Example: http://mycompany/myservice?wsdl ). If you don't know what is WSDL then in short, WSDL is Web service Description Language (i.e. XML file which describes your SOAP Service). WSDL helps to craft SOAP API request Body for ZappySys XML Driver. So Let's get started.

Open SoapUI and click SOAP button to create new SOAP Project
Enter WSDL URL or File Path of WSDLFor example WSDL for our sample service can be accessed via this URL
```
http://www.dneonline.com/calculator.asmx?wsdl
```
Create new SOAP API Project in SoapUI tool for SOAP API Testing
Once WSDL is loaded you will see possible operations you can call for your SOAP Web Service.
If your web service requires credentials then you have to configure it. There are two common credential types for public services (SOAP WSS or BASIC )
1. To use SOAP WSS Credentials select request node and enter UserId, Password, and WSS-PasswordType (PasswordText or PasswordHash)
  
  Configure SOAP WSS Credentials for SoapUI (SOAP API Testing Tool)
2. To use BASIC Auth Credentials select request node and double-click it. At the bottom click on Auth (Basic) and From Authorization dropdown click Add New and Select Basic.
  Configure Basic Authorization for SoapUI (SOAP API Testing Tool)
Now you can test your request first Double-click on the request node to open request editor.
Change necessary parameters, remove optional or unwanted parameters. If you want to regenerate request you can click on Recreate default request toolbar icon.
Create SOAP Request XML (With Optional Parameters)
Once your SOAP Request XML is ready, Click the Play button in the toolbar to execute SOAP API Request and Response will appear in Right side panel. Test SOAP API using SoapUI Tool (Change Default XML Body / Parameters, Execute and See Response)

Create DSN using ZappySys XML Driver to call SOAP API

Once you have tested your SOAP API in SoapUI tool, we are ready to use ZappySys XML driver to call SOAP API in your preferred BI tool or Programming language.

First open ODBC Data Sources (search ODBC in your start menu or go under ZappySys > ODBC PowerPack > ODBC 64 bit)
Goto System DSN Tab (or User DSN which is not used by Service account)
Click Add and Select ZappySys XML Driver ZappySys ODBC Driver for XML / SOAP API
Configure API URL, Request Method and Request Body as below ZappySys XML Driver - Calling SOAP API - Configure URL, Method, Body
(This step is Optional) If your SOAP API requires credentials then Select Connection Type to HTTP and configure as below.
ZappySys XML Driver - Configure SOAP WSS Credentials or Basic Authorization (Userid, Password)
Configure-Request Headers as below (You can get it from Request > Raw tab from SoapUI after you test the request by clicking the Play button) Configure SOAP API Request Headers - ZappySys XML Driver
Once credentials entered you can select Filter to extract data from the desired node. Make sure to select array node (see special icon) or select the node which contains all necessary columns if you don't have array node. Select Filter - Extract data from nested XML / SOAP API Response (Denormalize Hierarchy)
If prompted select yes to treat selected node as Array (This is helpful when you expect one or more record for selected node) Treat selected node as XML Array Option for SOAP API Response XML

Preview SOAP API Response / Generate SQL Code for SOAP API Call

Once you configure settings for XML Driver now you can preview data or generate example code for desired language (e.g. C#, Python, Java, SQL Server). Go to Preview tab and you will see default query generated based on settings you entered in previous sections. Attributes listed in WITH clause are optional. If you omit attribute in WITH clause it will use it from Properties tab.

Preview Data

Preview SOAP API Response in ZappySys XML Driver

Generate Code Option

Generate Example Code for ZappySys Driver

POST data to REST API URL from file in C#

Above example was POST data to API URL but what if your Request Body is large and you have saved that to file? Well here is the way to get your request body from a file (Use @ symbol before path).

SELECT * FROM $
WITH 
(METHOD='POST' 
,HEADER='Content-Type:text/plain || x-hdr1:AAA'
,SRC='http://httpbin.org/post'
,BODY='@c:\files\dump.xml'
,IsMultiPart='True'
)

SELECT * FROM $

WITH

(METHOD='POST'

,HEADER='Content-Type:text/plain || x-hdr1:AAA'

,SRC='http://httpbin.org/post'

,BODY='@c:\files\dump.xml'

,IsMultiPart='True'

)

REST API Pagination in C#

Even we set up ODBC Data Source to get the data, it may not be enough. Usually, if you are getting a huge data set from API provider, it won't give it to you in one HTTP response. Instead, it gives back only a subset of data and provides a mechanism for data pagination. The good news is that ZappySys ODBC Driver includes many options to cover virtually any pagination method.

Below you will find a few examples of API pagination. If you need something more sophisticated check the below link (the article was written for SSIS PowerPack but UI options and concepts apply to ODBC Driver too):

https://zappysys.com/blog/ssis-rest-api-looping-until-no-more-pages-found/

Paginate by Response Attribute

This example shows how to paginate API calls where you need to paginate until the last page detected. In this example, next page is indicated by some attribute called nextlink (found in response). If this attribute is missing or null then it stops fetching the next page.

SELECT * FROM $
WITH(
SRC=@'https://zappysys.com/downloads/files/test/pagination_nextlink_inarray_1.json'
,NextUrlAttributeOrExpr = '$.nextlink'  --keep reading until this attribute is missing. If attribute name contains dot then use brackets like this $.['my.attr.name']
)

Paginate by URL Parameter (Loop until certain StatusCode)

This example shows how to paginate API calls where you need to pass page number via URL. The driver keeps incrementing page number and calls next URL until the last page detected (401 error). There are few ways to indicate the last page (e.g. By status code, By row count, By response size). If you don't specify end detection then it will use the default (i.e. No records found).

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/page-xml.aspx?page=1&mode=DetectBasedOnResponseStatusCode'
,PagingMode='ByUrlParameter'
,PagingByUrlAttributeName='page'
,PagingByUrlEndStrategy='DetectBasedOnResponseStatusCode'
,PagingByUrlCheckResponseStatusCode=401
,IncrementBy=1
)

Paginate by URL Path (Loop until no record)

This example shows how to paginate API calls where you need to pass page number via URL Path. The driver keeps incrementing page number and calls next URL until the last page is detected. There are few ways to indicate the last page (e.g. By status code, By row count, By response size). If you don't specify end detection then it will use the default (i.e. No records found).

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/cust-<%page%>.xml'
,PagingMode='ByUrlPath'
,PagingByUrlAttributeName='<%page%>'
,PagingByUrlEndStrategy='DetectBasedOnRecordCount'
,IncrementBy=1
)

Paginate by Header Link (RFC 5988)

API like GitHub / Wordpress use Next link in Headers (RFC 5988)

SELECT * FROM $
LIMIT 25
WITH(
	 Src='https://wordpress.org/news/wp-json/wp/v2/categories?per_page=10'
	,PagingMode='ByResponseHeaderRfc5988'
	,WaitTimeMs='200' --//wait 200 ms after each request
)

Error Handling in REST API / SOAP

Sometimes errors occur... they just do and there is nothing you can do! Or can you? Actually, in ODBC PowerPack you can handle them in two ways.

METHOD 1 - Using Error Handling Options

When to use?

You may want to use them when your source is a resource located on the Internet; e.g. a file on a website, a file on an FTP server or just a plain API HTTP response. By default, when a remote server returns an error, data retrieval is stopped, an error is raised and no data is given back to you. This might not be always desirable.

Scenario 1

Imagine a scenario, that there is a web server which each day at 12 AM releases a new JSON file with that day's date as filename, e.g. http://www.some-server.com/data/2018-06-20.json. And, of course, you want to download it and use it daily in your Power BI report. But you have a problem: Power BI report data sources are refreshed each hour and you may get HTTP 404 status code (no file was found) when a file is not released yet. Which consequentially means other data sources won't be updated as well and you will see old and cached data on the report. That's where you could use Continue on any error or Continue when Url is invalid or missing (404 Errors) to avoid an error being raised and let other data sources to be updated.

Scenario 2

Another scenario is when you expect a web server to raise some kind of HTTP error when accessing a URL. You don't want ODBC Data Source to raise an error but instead, you want to get response data. That's where you can use Continue on any error or alike together with Get response data on error to continue on an error and get the data:

METHOD 2 - Using Connection [Retry Settings]

Another scenario you may run into is a buggy web server. You ask it to give you some file or data and it, like a snotty kid, just doesn't give it to you! You have to ask twice or thrice before it does its job. If that's the case, you have to retry HTTP requests using Connection:

REST API / SOAP Connection Types in C# (OAuth / HTTP)

If you need to authenticate or authorize your user to access a web resource, you will need to use one of the Connections:

HTTP
OAuth

ZappySys XML Driver - HTTP and OAuth Connection Types

HTTP Connection

Use HTTP Connection for simple Windows, Basic, NTLM or Kerberos authentication. Just fill in a username and a password and you are good to go!

You can also use HTTP Connection for more sophisticated authentication like:

SOAP WSS (when accessing a SOAP WebService)
Static Token / API Key (when need to pass an API key in HTTP header)
Dynamic Token (same as Static Token method except that each time you need to log in and retrieve a fresh API key)
JWT Token (As per RFC 7519)

OAuth

If you are trying to access REST API resource, it is a huge chance, you will need to use OAuth Connection. Read this article to understand how OAuth authentication and authorization works and how to use it (article originally was written for SSIS PowerPack, but the concepts and UI stay the same): https://zappysys.com/blog/rest-api-authentication-with-oauth-2-0-using-ssis

Performance consideration for Web API Calls

While calling APIs you may face some performance issues. There are a few tips you can consider to speed up things.

Use Server-side filtering if possible in URL or Body Parameters

Many API supports filtering your data by URL parameters or via Body. Whenever possible try to use such features. Here is an example of odata API, In the below query the first query is faster than the second query because in the first query we filter at the server.

SELECT * FROM value
WITH(
	 Src='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json&$filter=Country eq ''USA'''
	,DataFormat='Odata'
)

-- Slow query - Client-side filtering
SELECT * FROM value
WHERE Country ='USA'
WITH(
	 Src='https://services.odata.org/V3/Northwind/Northwind.svc/Customers?$format=json'
	,DataFormat='Odata'
)

Avoid Special features in SQL Query (e.g. WHERE, Group By, Order By)

ZappySys API engine triggers client-side processing if special features are used in Query. Following SQL Features will trigger Client-Side processing which is several times slower than server-side processing. So always try to use simple query (Select col1, col2 .... from mytable )

WHERE Clause
GROUP BY Clause
HAVING Clause
ORDER BY
FUNCTIONS (e.g. Math, String, DateTime, Regex... )

LIMIT clause does not trigger client-side processing.

Consider using pre-generated Metadata / Cache File

Use META option in WITH Clause to use static metadata (Pre-Generated)There are two more options to speedup query processing time. Check this article for details.

select * from value WITH( meta='c:\temp\meta.txt' )
--OR--
select * from value WITH( meta='my-meta-name' )
--OR--
select * from value WITH( meta='[ {"Name": "col1",&nbsp;"Type": "String", Length: 100},&nbsp;{"Name": "col2",&nbsp;"Type": "Int32"} ...... ]' )

Enable Data Caching Options (Found on Property Grid > Advanced Mode Only )

Consider using Metadata / Data Caching Option

ZappySys API drivers support Caching Metadata and Data rows to speed up query processing. If your data doesn't change often then you can enable this option to speed up processing significantly. Check this article for details how to enable Data cache / metadata cache feature for datasource level or query level. To define cache option at query level you can use like below.

SELECT * FROM $
WITH 
(  SRC='https://myhost.com/some-api'
  ,CachingMode='All'  --cache metadata and data rows both
  ,CacheStorage='File' --or Memory
  ,CacheFileLocation='c:\temp\myquery.cache'
  ,CacheEntryTtl=300 --cache for 300 seconds
)

Use --FAST Option to enable Stream Mode

ZappySys JSON / XML drivers support --FAST suffix for Filter. By using this suffix after Filter driver enables Stream Mode, Read this article to understand how this works.

SELECT * FROM $ 
LIMIT 10 --//add this just to test how fast you can get 10 rows
WITH(
  Filter='$.LargeArray[*]--FAST' --//Adding --FAST option turn on STREAM mode (large files)
 ,SRC='https://zappysys.com/downloads/files/test/large_file_100k_largearray_prop.json.gz'
 --,SRC='c:\data\large_file.json.gz'
 ,IncludeParentColumns='False'  --//This Must be OFF for STREAM mode (read very large files)
 ,FileCompressionType='GZip' --Zip or None (Zip format only available for Local files)
)

Other considerations for Web API calls in C#

There are few settings you can coder while calling Web API

API Limit / Throttling

While calling public API or other external web services one important aspect you have to check, how many requests are allowed by your API. Especially when you use API pagination options to pull many records you have to slow down based on API limits. For example, your API may allow you only 5 requests per second. Use Throttling Tab on Driver UI to set delay after each request.

2D Array Transformation

If you are using JSON or XML API Driver then possible you may have to transform your data using 2D array transformation feature. Check this link for more information.

Conclusion

In this article, we saw how to use C# to call REST API. We show how to do REST API calls in C# using the ZappySys ODBC driver. Also, we show how to get data from local files and get Gmail information with REST API. If you liked this article and you want to try, you can download the ZappySys ODBC installer here.