In our previous blog we saw how to read JIRA data in SQL Server. Now let’s learn how to Import SAP S/4HANA OData Service Data Into MS Access. SAP S/4HANA provides OData REST API interface to access data in your application using HTTP Protocol. We will use ODBC XML Driver to read SAP data and load into MS Access.

So, let’s get started.

About SAP HANA / OData REST API Service

You can expose your SAP Data using ODATA REST API Service. Here is a good article which shows how to expose data as OData Service.   For more information on SAP OData Service feature check this SAP help page. If you are new to OData Standard then read here to know more how OData can facilitate data extraction using HTTP REST API. If you need Sample XML based OData Service then use below test URLs.

https://services.odata.org/Northwind/Northwind.svc/
https://services.odata.org/Northwind/Northwind.svc/Customers
https://services.odata.org/Northwind/Northwind.svc/Orders
https://services.odata.org/Northwind/Northwind.svc/Invoices
https://services.odata.org/Northwind/Northwind.svc/Products

Now let’s look at how to read SAP Data using ODBC Driver. At this point we assume you have exposed your data as OData Endpoint.

Read SAP S/4HANA data using XML Driver

First let’s look at steps to configure XML Driver. We will read data from SAP S/4HANA OData Service and then in next section we will look at how to load data into MS Access or other target.

1. First, Windows search, write ODBC and select the ODBC Data sources (32 bits)
   

   Open ODBC Data Source

2. As a second step, in ODBC Data source Administrator press the Add button.
   

   Add ZappySys

3. In this step, create the new data source, select ZappySys XML Driver.
   

   Add new zappysys xml driver

4. Now edit that XML data source to configure it. Enter your OData Service URL its typically like below. Replace 3 parts with your own value (i.e. replace {MY-INSTANCE},  {MY-PROJECT}, {MY-TABLE})
   

   https://{MY-INSTANCE}/sap/opu/odata/sap/{MY-PROJECT}/{MY-TABLE}

   For example if you are hosting SAP HANA in Cloud Instance then your URL may look like below
   

   https://myXXXXXX-api.s4hana.ondemand.com/sap/opu/odata/sap/MyTestProject/PurchaseOrders

   And create New ZS-HTTP connection in it select Basic Authentication and enter your SAP HANA UserID / Password to call OData Service and select the desire filter.
   

   XML Driver : SAP S/4HANA OData Service Configuration

5. Now go to Data Format / Compression (Zip/GZip) tab and select Data Format as OData to get all the records.
   

   Data Format : Odata

6. Finally, now using Query Builder and Code Generator we will generate the query and click on Preview the data.
   

   Generate Query

7. That’s it we are ready to load SAP S/4HANA OData Service data to MS Access.

Load SAP S/4HANA OData Service in MS Access

1. In MS Access, go to External Data Ribbon and select New Data Source and select From Other Sources and ODBC Database.
   

   A configuration in MS Access: Import Data From REST API

2. Select the Source and destination of the data, select Import the source data into a new table in the current database and press OK.
   

   Get External Data – ODBC Database

3. Select your newly created JSON Driver Data Source and click OK.
   
   

   Get External Data – Select your newly created ODBC Data Source (JSON Driver)

    

4. Select tables from the list of tables in the Import Objects screen and click OK.
   
   

   Get External Data – Select tables from Import Objects Screen

5. Select Save Import Steps from the screen and click on Save Import button.
   

   Get External Data – Select Save Import Steps

6. That’s it If everything is OK, you will be able to see the data.
   

   Data Imported Successfully in MS Access DB Table

How to link REST API data to Access

The linking process is very similar to importing data into the Access table. Follow steps of the previous paragraph – How to import REST API data to Access – but on step 2 select the second option to create a linked table instead. Then at last it’s asked to select a unique record identifier, don’t select anything:

Linked table approach – Don’t select a unique Record identifier

SAP S/4HANA Integration with Other BI Tools

ZappySys ODBC Drivers built using ODBC standard which is widely adopted by industry for a long time. Which mean the majority of BI Tools / Database Engines / ETL Tools already there will support native / 3rd party ODBC Drivers. Below is the small list of most popular tools / programming languages our Drivers support. If your tool / programming language doesn't appear in the below list, which means we have not documented use case but as long as your tool supports ODBC Standard, our drivers should work fine.  

BI / Reporting Tools Integration	ETL Tools Integration	Programming Languages Integration
Microsoft Power BI Tableau SSRS (SQL Reporting Services) QlikView /Qlik Sense MicroStrategy Google Sheet Microsoft Excel Microsoft Access Oracle OBIEE Many more (not in this list).....	Informatica PowerCenter (Windows) Informatica Cloud SSIS (SQL Integration Services) SQL Server Talend Data Studio Pentaho Kettle Oracle OBIEE Many more (not in this list).....	Visual Studio C# C++ JAVA Python PHP PowerShell T-SQL (Using Linked Server)

 

Conclusion

So in this blog, we learned how to Import SAP S/4HANA OData Service Data in Access using ODBC XML / SOAP API Driver in a very simple way. You can achieve many more functionalities with this tool. Check our blogs/articles on XML File / REST API Driver to find out what this tool is capable of more.

References

Finally, you can use the following links for more information about the use of SAP S/4HANA OData Service with our tools:

• SAP API Business Hub
• Landing Page for ODBC XML / SOAP API Driver, you can also find Tutorial Video here.
• Help File: Documentation of XML Driver.
• Blog/articles link: https://zappysys.com/blog/category/odbc-powerpack/odbc-drivers/xml-soap-api-driver/

The post Import SAP S/4HANA OData Service Data Into Ms Access via ODBC Driver appeared first on ZappySys Blog.

In our previous article, we talked about how to read from Dynamics CRM using easy to use SSIS Dynamics CRM Source connector. In this post, you will learn how to use SSIS JSON Source (for REST API) instead of CRM Source. Using SSIS JSON Source you can read virtually any REST API or JSON file data including Dynamics CRM / Dataverse. For this article, our goal is to learn How to read Microsoft Dynamics 365 data using Web API (OData API) in SSIS. We will use drag and drop approach without any coding.

Why use Web API (OAuth) for Dynamics 365 CRM?

Microsoft has announced depreciation of older CRM API (SOAP API with WS-Trust Authentication) which means they are no longer going to support the older style of authentication so no option but to switch to newer authentication / API.

In the older approach (SOAP API with WS-Trust Authentication) you must pass your User Name and Password to authenticate, also if MFA / 2FA (Multi-Factored Auth) is turned on by organization policy you cannot use your user id / password in API call. In new authentication (i.e. OAuth) you have to use Client Id and Client Secret only.

Now let’s look at the step by step approach to call Dynamics 365 REST API.

Step-1 : Register Dynamics CRM / Dataverse App obtain App Id / Client Secret

The very first step to access Microsoft Dynamics CRM 365 API is to obtain Application Id (also referred as Client Id)  and Client Secret. We will use this information later in this article (on OAuth connection UI).

1. Register Dynamics CRM / Dataverse App With Azure and get the credential (App Id, Client Secret).

Step-2 : Read Microsoft Dynamics 365 API data in SSIS using JSON Source

Let’s start with an example. We use SSIS JSON Source Task to make calls to Microsoft Dynamics 365 API, we will load Dynamics 365 Leads data to SQL Server.

Please refer to this link for more information: Microsoft Dynamics 365 Web API Reference

Find out API Url for your environment (organization url)

Once we are done with the creation of the Azure AD app to obtain App Id /Secret out next step is to find out the base API URL for your organization / environment.

If you have multiple environments (i.e. Prod, Sandbox) then each environment might have its own unique URL)

1. To find your API Service URL, Click on Gear Icon
2. Choose Advanced Settings
3. Click down arrow on Settings
4. Click Customization
5. Click Developer Resource Link
6. On this page now you can see various URLs. For examples used in this article, we need to get is Service Root URL for Web API

How to Find Organization Id / Web API URL / SOAP API Service URL (Unique Environment Instance URL)

Create SSIS Data Flow to read data from Dynamics CRM / Dataverse

1. First of All, Drag and drop Data Flow Task from SSIS Toolbox and double click it to edit.
   

   Drag and Drop SSIS Data Flow Task from SSIS Toolbox

2. Furthermore, Drag and Drop JSON Source(REST API and File) File Task from the SSIS Toolbox.
   

   Drag and Drop JSON Source Component

Create OAuth Connection for Dynamics CRM / Dataverse

Now lets look at how to create and configure OAuth connection to call Web API for Dynamics CRM / Dataverse

1. Check Use Credentials Option and select a new ZS-OAuth Connection from the drop-down.
   

   Check Use Credentials

2. Once OAuth Connection UI opens up. Configure like below on General Tab.
   Change OAuth Version to OAuth2
   Change Grant Type to Client Credentials
   Enter Client Id (Azure AD App Id) (Obtained in the Previous Section)
   Enter Client Secret (Obtained in the Previous Section)
   Enter Token URL (see below)
   

   Configure OAuth Connection for Azure AD App – Client Credentials Grant (Tenant Specific Token EndPoints)

   For Client Credentials Grant
   Enter Token URL as https://login.microsoftonline.com/<your-tenant-id>/oauth2/token
   (You can find Tenant ID on the Overview page of your Azure AD App)

   For Default Grant (3-legged – requires User Login Prompt)
   Enter Auth URL as https://login.microsoftonline.com/common/oauth2/authorize
   Enter Token  URL as https://login.microsoftonline.com/common/oauth2/token

    

   

   Where to find Token Endpoint with Tenant ID in the URL (Azure AD App)

    

3. Now go to Advanced Tab and configure like belowEnter Extra Attribute for /auth as  

   resource=https://<your-instance-id>.api.<your-crm-region>.dynamics.com

   

   Client Credentials Grant for Dynamics CRM / Dataverse – API Access (Azure AD App / OAuth)

4. Click OK to Save connection.

Get all Accounts, Contacts, and Leads using Dynamics 365 API

Now let’s configure API URL from where you like to read data.

For example, if you just use the root URL then it will list all possible entries

Root URL:

https://{{organization-uri}}/api/data/v9.1/

1. Double click on the Component to Edit the Configurations.
2. Change Data Format to OData.
3. In the Web URL Field, you can enter any valid URL for Web API. Please refer to this link of Dynamics 365 Rest API for more information.Example Data URLs

   https://{{organization-uri}}/api/data/v9.1/
   https://{{organization-uri}}/api/data/v9.1/accounts
   https://{{organization-uri}}/api/data/v9.1/contacts
   https://{{organization-uri}}/api/data/v9.1/leads

   Here, replace {{organization-uri}} in URL to your Microsoft Dynamics 365 site domain name (Example zappy6.api.crm.dynamics.com Dynamics 365 URL).

   

   Set URL in JSON Source

Preview / Query Dynamics 365 Data

1. Now select Array filter as

   $.value[*]

     (Click Select Filter Button – Choose value node)
2. Click on the Preview button to get the data.
   

   Preview Data

3. Finally, click OK to save JSON Source settings.
4. That’s it we are ready to load Dynamics 365 API data to SQL Server.

Load Microsoft Dynamics 365 API data into SQL Server

ZappySys SSIS PowerPack makes it easy to load data from various sources such as REST, SOAP, JSON, XML, CSV or from other source into SQL Server, or PostgreSQL, or Amazon Redshift, or other targets. The Upsert Destination component allows you to automatically insert new records and update existing ones based on key columns. Below are the detailed steps to configure it.

Step 1: Add Upsert Destination to Data Flow

1. Drag and drop the Upsert Destination component from the SSIS Toolbox.
2. Connect your source component (e.g., JSON / REST / Other Source) to the Upsert Destination.

SSIS - Data Flow - Drang and Drop Upsert Destination Component

Step 2: Configure Target Connection

1. Double-click the Upsert Destination component to open the configuration window.
2. Under Connection, select an existing target connection or click NEW to create a new connection.
   • Example: SQL Server, or PostgreSQL, or Amazon Redshift.

Step 3: Select or Create Target Table

1. In the Target Table dropdown, select the table where you want to load data.
2. Optionally, click NEW to create a new table based on the source columns.

Configure SSIS Upsert Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS

Step 4: Map Columns

1. Go to the Mappings tab.
2. Click Auto Map to map source columns to target columns by name.
3. Ensure you check the Primary key column(s) that will determine whether a record is inserted or updated.
4. You can manually adjust the mappings if necessary.

SSIS Upsert Destination - Columns Mappings

Step 5: Save Settings

• Click OK to save the Upsert Destination configuration.

Step 6: Optional: Add Logging or Analysis

• You may add extra destination components to log the number of inserted vs. updated records for monitoring or auditing purposes.

Step 7: Execute the Package

• Run your SSIS package and verify that the data is correctly inserted and updated in the target table.

SSIS Upsert Destination Execution

Conclusion

In this post you have seen how to Read Microsoft Dynamics 365 API data in SSIS. We used JSON Source Component to call Dynamics 365 REST API and load data into SQL server. You can download SSIS PowerPack here to try many other scenarios not discussed in this blog along with 70+ other components.

References

Finally, you can use the following links for more information about the use of Dynamics 365 REST API with our tools:

• JSON Source, you can also find Tutorial Video here.
• Help File: Documentation of SSIS Json Source
• Microsoft Dynamics CRM Web API Reference
• Dynamics 365 Web API EntityType Reference

The post How to read Microsoft Dynamics 365 OData API data in SSIS (CRM Web API) appeared first on ZappySys Blog.

In our previous post, we saw how to call REST API in SSIS. Now let’s learn how to read data from SAP S4 / HANA OData Service (i.e. S/4HANA). SAP HANA provides an OData REST API interface to access data in your application using HTTP Protocol. We will use the SSIS XML Source component to read SAP data and load it into the SQL Server Table in a few clicks.

About SAP HANA / OData REST API Service

You can expose your SAP Data using ODATA REST API Service. Here is a good article that shows how to expose data as an OData Service. For more information on the SAP OData Service feature check the SAP help page. If you are new to OData standard then read an article to learn more about it. If you need an XML-based OData Service then use the test URLs below:

https://services.odata.org/Northwind/Northwind.svc/
https://services.odata.org/Northwind/Northwind.svc/Customers
https://services.odata.org/Northwind/Northwind.svc/Orders
https://services.odata.org/Northwind/Northwind.svc/Invoices
https://services.odata.org/Northwind/Northwind.svc/Products

Now let’s look at how to read SAP Data in SSIS. At this point, we assume you have exposed your data as OData Endpoint.

Step-by-Step Instructions

Let’s look at the step-by-step approach to how to read SAP data using SSIS components. Then we will load SAP HANA data into SQL Server.

Read SAP HANA data using XML Source

First, let’s look at the steps on how to configure XML Source. We will read data from SAP S4 / HANA OData Service and then we will look at how to load data into SQL Server in the next section.

1. Open an SSIS package.
2. Drag Data Flow Task from SSIS Toolbox and double-click it to open it.
3. From the SSIS Toolbox drag ZS XML Source onto the background of Data Flow (in the designer). Double-click it to open the XML Source configuration window.
4. Change Data Format to OData. Selecting this option will take care of REST API pagination too.
5. Enter your OData Service URL. Its format looks similar to the one shown below. Replace 3 parts with your own value (i.e. replace {MY-INSTANCE}, {MY-PROJECT}, {MY-TABLE}):
   

   https://{MY-INSTANCE}/sap/opu/odata/sap/{MY-PROJECT}/{MY-TABLE}

   For example, if you are hosting SAP HANA in a cloud instance then your URL may look like this:
   

   https://myXXXXXX-api.s4hana.ondemand.com/sap/opu/odata/sap/MyTestProject/PurchaseOrders

6. Check the Use Credentials checkbox and select the New ZS-HTTP Connection option.
7. In HTTP Connection Manager select the Basic Authentication option and enter your SAP HANA UserID / Password.
   

   Configure SSIS HTTP Connection (SAP S4 / HANA OData Service – Using Basic Authentication)

8. Let’s get back to the XML Source connector we created in the previous stage. Press the Select Filter button or directly type

   $.feed.entry[*]

     into the field as shown below:
   

   Select Filter for OData Service (XML REST API)

9. Click Preview to make sure you can see the records.
10. Click OK to save the configuration and close the configuration window.

In the next section, we will configure the target to load SAP data into SQL Server using the ZappySys Upsert Destination Data Flow component.

Load SAP HANA data into SQL Server using Upsert Destination

ZappySys SSIS PowerPack makes it easy to load data from various sources such as REST, SOAP, JSON, XML, CSV or from other source into SQL Server, or PostgreSQL, or Amazon Redshift, or other targets. The Upsert Destination component allows you to automatically insert new records and update existing ones based on key columns. Below are the detailed steps to configure it.

Step 1: Add Upsert Destination to Data Flow

1. Drag and drop the Upsert Destination component from the SSIS Toolbox.
2. Connect your source component (e.g., JSON / REST / Other Source) to the Upsert Destination.

SSIS - Data Flow - Drang and Drop Upsert Destination Component

Step 2: Configure Target Connection

1. Double-click the Upsert Destination component to open the configuration window.
2. Under Connection, select an existing target connection or click NEW to create a new connection.
   • Example: SQL Server, or PostgreSQL, or Amazon Redshift.

Step 3: Select or Create Target Table

1. In the Target Table dropdown, select the table where you want to load data.
2. Optionally, click NEW to create a new table based on the source columns.

Configure SSIS Upsert Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS

Step 4: Map Columns

1. Go to the Mappings tab.
2. Click Auto Map to map source columns to target columns by name.
3. Ensure you check the Primary key column(s) that will determine whether a record is inserted or updated.
4. You can manually adjust the mappings if necessary.

SSIS Upsert Destination - Columns Mappings

Step 5: Save Settings

• Click OK to save the Upsert Destination configuration.

Step 6: Optional: Add Logging or Analysis

• You may add extra destination components to log the number of inserted vs. updated records for monitoring or auditing purposes.

Step 7: Execute the Package

• Run your SSIS package and verify that the data is correctly inserted and updated in the target table.

SSIS Upsert Destination Execution

Conclusion

In this post, we saw how easy it is to read SAP S4 HANA data using XML Source SSIS connector that is part of ZappySys SSIS PowerPack. Download it, try it out, and see it for yourself! It has many more connectors that were not mentioned in this post but could be also very helpful in your daily work.

The post Read SAP S4 / HANA data in SSIS (OData REST API) appeared first on ZappySys Blog.

In this blog, we will learn How to read data from Azure DevOps in SSIS and load into SQL Server Table. Azure DevOps was formally known as TFS Online / VSTS.

In this article we will learn, how to read Issues (Work Items) using Azure DevOps API (i.e. VSTS API). In last few years Microsoft has changed marketing names few times for TFS Service. For example everything started with  Visual Source Safe (VSS) > TFS > TFS Online > Visual Studio Online > VSTS (Visual Studio Team Services)) and finally now its called Azure DevOps. In this article we might use terms like VSTS or TFS  Online or Azure DevOps but not to get confused its all about Calling Azure Dev Ops API.

In this article we will try to cover few topics such as.

• Register an OAuth App to call REST API for Azure DevOps API (i.e. VSTS / TFS Online)
• Read all Issues (Work Items) data from VSTS with SSIS and finally
• Create new Issue/Bug in Visual Studio Team Service Project WorkItems.
• Authenticate and Call Azure DevOps using PAT (Personal Access Tokens) rather than OAuth 2.0

We will go through the steps to read Issues data from Visual Studio Team Service and Load into SQL Server using  SSIS REST API Task and SSIS JSON Source.

In nutshell, this post will focus on how to call VSTS API using SSIS.

So let’s get started.

Prerequisite

1. First, you will need to have SSIS installed
2. Secondly, make sure to have SSDT
3. Finally, do not forget to install ZappySys SSIS PowerPack

Call Azure DevOps API Using PAT (Personal Access Token)

There are two ways to call Azure DevOps API. You can use OAuth 2.0 or use PAT (i.e. Personal Access Token) In the next section we will see how to use OAuth 2.0 if you like more secure way to authenticate (OAuth 2.0 requires more steps).

Microsoft offers Static API Token (referred as PAT – Personal Access Token) which you can use like a password (it remains valid for 1 year before you have regenerate).

So let’s get started.

Create PAT (Personal Access Token)

1. Visit to

   https://dev.azure.com/YOUR_COMPANY_NAME/_usersSettings/tokens

    . Create Personal Access Token (i.e. PAT ) by clicking User Settings icon > Personal Access Token (see below)  (detailed steps listed here )
   

   Call Azure DevOps API (formally known as VSTS / TFS Online) – Create Personal Access Token (i.e. PAT)

2. On Token Configuration Screen change Expiration to maximum (i.e. one year from today)
3. Now select permission – We will call Analytics API below (but you can also check Work Items section if you need to call some other API for read / write purpose)
   

   Change Permission / Expiration Date for Azure DevOps PAT (Personal Access Token) – Read / Write Work Items, Call Analytics API Example

Call Azure DevOps OData API in SSIS – Read WorkItems  Example (Analytics API Service)

Now lets configure JSON Source to read WorkItems by calling  Azure DevOps Analytics API (in Previous few sections we call different set of APIs). Check this link for more information on AzureDevOps Analytics API (its OData format).

For demo purpose we are calling WorkItems entity but you can read from many other entities like (Read Areas, Iterations, Projects, Teams, Tags etc). Check this link to see which Entity (i.e. Table) you can query for specific OData API version.

So let’s get started

1. Go to SSIS designer and drag Data Flow Task on the Designer Surface
   
2. Then double click Data Flow and from toolbox drag ZS JSON Source
   
3. Double click JSON Source to configure like below
   1. First change Data Format to OData (Analytics API we will call in this section is OData format)
   2. Enter URL to Query data like below. Change YOUR_COMPANY_NAME with your own name. Below API URL will call Analytics will
      

      https://analytics.dev.azure.com/YOUR_COMPANY_NAME/Odbc/_odata/v3.0/WorkItems?$select=WorkItemId,Title,WorkItemType,State,CreatedDate&$orderby=CreatedDate desc&$top=11

   3. Check Use Credentials and Select new ZS-HTTP connection (instead of OAUTH)
4. On HTTP Connection UI Configure below
   1. Enter Some URL – This will be ignored anyways (e.g. https://dev.azure.com )
   2. Change Credentials Type to Basic (User ID / Password)
   3. Enter some random word in User ID -i.e. fakeusername (this is ignored anyways but needs some non-empty value for older version of SSIS PowerPack else validation fails)
   4. Enter PAT (i.e. Personal Access Token) obtained in the previous section
5. Click OK to Save Connection UI and go back to JSON Source
6. On JSON Source, Enter or Select Array Filter as 

   $.value[*]

7. Click Preview data see its working using HTTP connection method

Configure JSON Source – Call Azure DevOps Api using PAT (Personal Access Token)

Calling Azure DevOps API in SSIS using OAuth 2.0

Now in this section we will see how to use more secure but little harder approach to authenticate. We will use OAuth 2.0 this time rather than static token (PAT) which we used in previous section.

In order to start, we will show several examples. ZappySys includes an SSIS JSON Source that will help you to call VSTS API, read various data from VSTS with SSIS, create a new issue on VSTS with SSIS and do many more actions using REST API. To learn more about VSTS API check this help file.

Create VSTS App (for OAuth 2.0 API Call)

To access VSTS data using REST API call you need to create a VSTS app. Follow the mentioned steps below to register VSTS OAuth App:

1. you can directly login to the Visual Studio Team Service – TFS Online site with your credentials, and redirect to the Visual Studio Team Service App Registration for creating an app.
2. you can open MyApps Page.
   

   Visual Studio Team Service: My Apps

   Visual Studio Team Service: My Apps[/caption]

3. Click on Create new application link, and you will be redirected to the Visual Studio Team Service App Registration Page.
4. Make sure you enter Authorization call base URL. We suggest to below call back URL if you are unsure. We will use exact same URL on OAuth connection UI (Advanced Tab)

   https://zappysys.com/oauth

5. 

   Visual Studio Team Service: New OAuth App Registration

6. Select the Authorization scopes as per your requirements, and click on Create Application button.
   For example to read Work Items or Create new Issues programmatically you can check following scope (i.e. OAuth permission).Check – Work items (full)  (i.e. vso.work_full scope)
   Note: The scopes selected on your registered application (below screenshot) must match the scopes entered in the ZappySys OAuth connection scopes field (see below). If they do not match exactly, an InvalidScope error can be produced.
   
   
   

   Visual Studio Team Service: Select Authorization Scopes

7. Once you are done with the Creation of an App certainly, you can edit the Application If you want to change Redirect URL or something.
8. furthermore, Go to MyApps Page and you have your App’s Lists there with App’s App ID as Client ID and App Secret as Client Secret.
   

   Visual Studio Team Service: Your Apps with App’s Credentials

finally, we are ready with an App. Let´s start with an example. In this example, we will show how to retrieve all Issues data from Visual Studio Team Service – TFS Online and create a new Issue in Visual Studio Team Service – TFS Online Work Items List under your Project with SSIS.

Load VSTS data to SQL Server using SSIS JSON Source

Now once you registered OAuth app in the Visual Studio Team Service – TFS Online website we can move to SSIS Part.

Configure OAuth Connection Manager

1. Right click on Connection Managers Panel to Create New OAuth Connection, and Context Menu will appear, Select New Connection from the Context Menu.
   

   Connection Manager Panel: Select Connection

2. Select ZS-OAUTH Connection Manager from the Connection Managers list.
   

   SSIS Connection Mangers List: Select O-AUTH Connection Manager

3. Configure the new OAuth Connection for VSTS(Visual Studio Team Service – TFS Online) as below.
4. First lets enter Call back URL (Same URL you entered when you created App earlier)SSIS OAuth Connection Manager: Advanced Tab Configurations
5. For VSTS API you need to set the Refresh Token File Path as below because VSTS API Token’s Refresh Token keeps changing.
6. Then configure the App on the General tab like below (i.e. OAuth Provider, Client ID, Client Secret, Scopes, etc.)
   Scope must be valid one from this list (only those scope allowed which are checked during app creation). For example during app creation you check Work items (full) then you can enter vso.work_full in scope.
   Note: The scopes selected on your registered application must match the scopes entered in the scopes field (below screenshot). If they do not match exactly, an InvalidScope error can be produced.
   
7. finally, you are done with the OAuth Configurations. Click on Generate Token button to Generate token and then click on Test Connection button and click on OK to save the connection manager configurations settings.

How to get ProjectID?

If you don’t know the ProjectID, then you can also enter ProjectName in place of ProjectID in Request URL. you can also call GET Request for getting Lists of Projects. the Request URL for getting the list of Projects is:

https://{accountName}.visualstudio.com/_apis/projects?api-version=4.1

For this, we will use SSIS REST API Task

1. First of All, Goto Control Flow Section, Drag and Drop SSIS REST API Task from the SSIS Toolbox.
   

   Drag and Drop REST API Task

2. furthermore, Double click on that Component to Edit the Configurations and Configure the task like the below-attached screesnshot.
   Some basic configurations need to be done in SSIS REST API Task.
   # Request Tab: 

   * Select Request URL Access Mode to Url from Connection.
   * Select Url Connection to your configured ZS-OAuth Connection Manager.
   * Enter Request URL: https://zappysys.visualstudio.com/_apis/projects?api-version=4.1
   * Select HTTP Request Method: GET

   

   SSIS REST API Tsk: Request Tab Configurations

   Create New Variable from Variable Menu

   

   SSIS User Variables: Create a new Variable and set Value

   # Response Tab: 

   * Select Response Content Type: Json
   * Set Response Content Filter Expression: $.value[1].id
   * check the checkbox of Save Response Content
   * Set Save Mode to Save to Variable
   * Select your Variable: {{User::ProjectID}}

   

   SSIS REST API Task: Response Tab Configurations

3.  finally, you are done with the configurations and do Test Request/Response and your ProjectID will be saved in {{User:: ProjectID}}

Configure JSON Source Component

1. First of all, Create a new SSIS Project or Open Existing Project in Visual Studio  (File > New > Project > Business Intelligence > Integration Services)
2. Once SSIS Package is open, Goto Data Flow Section, Drag and Drop ZS JSON Source Component from the SSIS Toolbox.
   

   Drag and Drop JSON Source Component

3. furthermore, Double click on that Component to Edit the Configurations. Select File path or web URL from the Access Mode drop-down.
   Enter Web URL like this:
   

   https://{accountName}.almsearch.visualstudio.com/{project}/_apis/search/workitemsearchresults?api-version=4.1-preview.1

   here, We have used 2 Parameters.

   * {accountName} is the name of the Visual Studio Team Services – TFS Online account.
   * {project} is Project ID or project name (For more See this Visual Studio Team Service API Documentation for getting filtered Issues data Request API)
   We have used Search Work Items for Fetching Paginated Work Items(Issues, Bugs)

   You can use Variable too in your Request URL, to make URL Dynamic, See the below-attached Screenshot.
   To make URL dynamic first create an SSIS variable like below screenshot and then change URL to use Variable Placeholder like this

   https://zappysys.almsearch.visualstudio.com/{{User::ProjectID}}/_apis/search/workitemsearchresults?api-version=4.1-preview.1

    

SSIS JSON Source Component: Enter the Request URL

JSON Source Configuration for Pagination

1. now, we are going to set Pagination related configurations of Visual Studio Team Service – TFS Online for Getting all WorkItems with Search POST REST Request API. Here in our example,
   Request Method: POST
   Body Content-Type: JSON (application/json)
   Request Body:
   
   

   {
     "searchText": "a",
     "$skip": <%page%>,
     "$top": 100,
     "filters": {
       "System.TeamProject": [
         "ProductTesting"
       ]
     }
   }

   here, SearchText: The SearchText for filtering WorkItems based on the keywords match.
   $skip: Number of results to be skipped
   $top: Number of results to be returned.
   
   you can take any number of the page size. for example, you can set 100 in both $skip and $top properties (I have set 10 for the Demo Purpose only)
   filters: Filters to be applied. Set it to null if there are no filters to be applied.
   System.TeamProject: ProjectName in which you want to do filtering of WorkItems.
   

   SSIS JSON Source Component: Configurations of Settings Tab

2.  You can Configure the Pagination Tab something like this:
   

   SSIS JSON Source Component: Pagination Tab Settings

3. furthermore, Select the Filter to set the Filter.
   

   SSIS JSON Source Component Configurations: Select Filter

4. It seems like you are all OK with the configurations, Click on the Preview button to see the response data.
   

   SSIS JSON Source Component: See the Preview

5. finally, Click on OK button to save JSON Source settings.

Configure OLE DB Destination Component for dumping Contacts data into SQL

1. first of all, Drag any destination (i.e. OLEDB Destination) from your SSIS toolbox. For demo purpose, we will load Contacts data into SQL Server Table.
   

   Drag and Drop OLE DB Destination Component

2. Configure OLEDB Destination (Select Connection and Target Table). You can Create a new Table for the destination by clicking a NEW button next to Table drop-down.
   

   OLE DB Destination Component – Configure for dumping WorkItems of Project from VSTS data in SQL Table

3. almost you are done with the configurations, just need to configure mappings, So for that Click on the Columns Tab to configure Mappings. also, Click on OK button to Save the configurations of OLE DB Destination Component.
   

   OLE DB Destination – Column Configurations

4. furthermore, You can see the requests in Fiddler to see the Paginated requests flows.
   

   Fiddler – See All The Requests

5. finally, Execute your package to Load Contacts data from Visual Studio Team Service – TFS Online to SQL Server.

Create a new WorkItem on Visual Studio Team Service – TFS Online (Use SSIS REST API Task)

Now let’s look at one more example to call Visual Studio Team Service(VSTS) – TFS Online REST API. This time we will create a new WorkItem using REST API call in SSIS. For this, we will use SSIS REST API Task

1. First of All, Goto Control Flow Section, Drag and Drop SSIS REST API Task from the SSIS Toolbox.
   

   Drag and Drop REST API Task

2. furthermore, Double click on that Component to Edit the Configurations and Configure the task like the below-attached screesnshot.
   

   ::::: Request Settings :::::
   
   * Select Request URL Access Mode to select Url from Connection.
   * Select Url Connection: Select your created OAUTH Connection Manager.
   * Enter Reqeust URL
         Request URL: POST https://{accountName}.visualstudio.com/{project}/_apis/wit/workitems/${type}?api-version=4.1
         here, 
         {accountName} - The name of the Visual Studio Team Services account.
         {project} - Project ID or project name
         {type} - WorkItems Type (Task, Bug, Issue, Requirement etc)
   * Select HTTP Request Method as PATCH method
   * Enter Request Body.
         Request Body:
         [
            {
                "op": "add",
                "path": "/fields/System.Title",
                "value": "Sample task"
            }
         ]
   * Select Body Content Type: Text/Plain
   *Enter Request Header:
           Content-Type: application/json-patch+json
   (For more Information See this Visual Studio Team Service - TFS Online API Documentation for Creating a new WorkItem
   https://docs.microsoft.com/en-us/rest/api/vsts/wit/work%20items/create?view=vsts-rest-4.1)

   

   REST API Task: Configurations

3. So all good with the Configurations let’s do Test Request/Response by clicking on Test Request/Response button. See the below-attached request/response screen.
   

   Create New WorkItem in VSTS Account: Do Test Request/Response

4. finally, you have created a WorkItem in VSTS successfully using VSTS REST API and REST API Task, See in VSTS Site.
   

   VSTS Site: See the Newly Created WorkItem

5. And that’s it. similarly, you can Update/Delete particular WorkItem in WorkItems List in VSTS.

Conclusion

In this blog, we learned how to read Visual Studio Team Services – TFS(Team Foundation Server) Online data in SSIS, How to register an OAuth App for Visual Studio Team Service REST API, How to get all WorkItems data from Visual Studio Team Service  – TFS(Team Foundation Server) Online and create a new WorkItem on VSTS with SSIS using ZS JSON Source Component and SSIS REST API Task in a very simple way. You can achieve many more functionalities with the use of both of these tools. Check our blogs/articles on REST API Task https://zappysys.com/blog/tag/ssis-rest-api-task/ and JSON Source Component https://zappysys.com/blog/tag/ssis-json-source/ to find out what these tools are capable of more.

References

Finally, you can use the following links for more information about the use of VSTS API with our tools:

• Visual Studio Team Service Documentation for API
• Landing Page for REST API Task, you can also find Tutorial Video here.
• Landing Page for JSON Source Component, you can also find Tutorial Video here.
• Help File: Documentation of SSIS REST API Task and JSON Source Component.

The post Read Azure DevOps data in SSIS (VSTS / TFS Online) – Call REST API / Load to SQL Server appeared first on ZappySys Blog.

In this article, we will learn how to utilize the SSIS JSON Source Component to retrieve JSON data from an OData service and perform OData pagination on significant results. According to the OData specification, data can be presented in either JSON or XML format. For the XML format, refer to this article. This article primarily covers the OData JSON API example.

REST API Pagination concept

Most of the REST API Services limit the size of the dataset returned in a single request. When more data is found, it simply includes a pointer to the following result set. If you wish to fetch all data (e.g., loop through), you must check the following result set indicator. If it is not null, you can continue fetching until all data is retrieved. This article covers several pagination methods.

Reading OData API in SSIS using JSON Source

Let’s discuss how to read the OData API and Paginate Automatically. Below is a typical OData service Response (JSON Format). Notice how it includes the nextLink attribute, which points to the following URL to fetch more data. When you set Data Format to OData in the JSON Source, it will handle pagination for you.

Request URL:

https://services.odata.org/V3/Northwind/Northwind.svc/Order_Details?$format=json

Response:

{
  "odata.metadata": "https://services.odata.org/V3/Northwind/Northwind.svc/$metadata#Order_Details",
  "value": [
    {
      "OrderID": 10248,
      "ProductID": 11,
      "Discount": 0
    },
 ........
 ........
  ],
  "odata.nextLink": "Order_Details?$skiptoken=10436,75"
}

Step-By-Step

In this section, you will learn how to use the JSON Source Adapter to extract data from the OData API.

1. Firstly, you need to download and install SSIS ZappySys PowerPack.
2. Once you have finished the first step, Open Visual Studio and create a New SSIS Package Project.
3. Now, Drag and Drop the SSIS Data Flow Task from the SSIS Toolbox.
   

   SSIS Data Flow Task – Drag and Drop

4. Double-click on the Data Flow task to see the Data Flow designer surface.
5. From the SSIS toolbox, drag and drop the JSON Source onto the Data Flow Designer surface.
   

   SSIS JSON Source – Drag and Drop

6. Double-click JSON Source and enter URL for OData API (You may use the below example URL for demo)
   

   https://services.odata.org/V3/Northwind/Northwind.svc/Order_Details?$format=json

    
7. Now configure Array Filter as below (you may see $.value[*] or some other node if URL is different)
   

   OData Paging using SSIS JSON Source

8. Click Preview data and OK to save
   

   Read JSON File data from Web URL Example in SSIS

9. Now click OK to save.
10. Connect Your Source with a target like ZS Trash destination and execute the package. You will notice in the log that it paginates to return all rows
    

    Fetch OData in SSIS (REST API Example)

Loading OData API into SQL Server / Other Target using SSIS

ZappySys SSIS PowerPack makes it easy to load data from various sources such as REST, SOAP, JSON, XML, CSV or from other source into SQL Server, or PostgreSQL, or Amazon Redshift, or other targets. The Upsert Destination component allows you to automatically insert new records and update existing ones based on key columns. Below are the detailed steps to configure it.

Step 1: Add Upsert Destination to Data Flow

1. Drag and drop the Upsert Destination component from the SSIS Toolbox.
2. Connect your source component (e.g., JSON / REST / Other Source) to the Upsert Destination.

SSIS - Data Flow - Drang and Drop Upsert Destination Component

Step 2: Configure Target Connection

1. Double-click the Upsert Destination component to open the configuration window.
2. Under Connection, select an existing target connection or click NEW to create a new connection.
   • Example: SQL Server, or PostgreSQL, or Amazon Redshift.

Step 3: Select or Create Target Table

1. In the Target Table dropdown, select the table where you want to load data.
2. Optionally, click NEW to create a new table based on the source columns.

Configure SSIS Upsert Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS

Step 4: Map Columns

1. Go to the Mappings tab.
2. Click Auto Map to map source columns to target columns by name.
3. Ensure you check the Primary key column(s) that will determine whether a record is inserted or updated.
4. You can manually adjust the mappings if necessary.

SSIS Upsert Destination - Columns Mappings

Step 5: Save Settings

• Click OK to save the Upsert Destination configuration.

Step 6: Optional: Add Logging or Analysis

• You may add extra destination components to log the number of inserted vs. updated records for monitoring or auditing purposes.

Step 7: Execute the Package

• Run your SSIS package and verify that the data is correctly inserted and updated in the target table.

SSIS Upsert Destination Execution

Twitter(Now X) REST API Paging Example using SSIS JSON Source

The Twitter (Now X) REST API is not an OData format, but you can see a similar concept there as well. On Twitter, you don’t have a partial URL like OData for the following link, but it provides a unique identifier for some sort of record. Read here for more information on the Twitter REST API Paging technique.

In our case, SSIS JSON Source Supports Paging very well, so we are covered. To loop through multiple result sets of Twitter data, simply configure the following four properties. See the screenshot below.

Twitter(X) REST API – Paging Example -Loop through resultset using cursor

Conclusion

Based on the above examples, you can see that handling Raging in REST API calls is not straightforward, but using Components like SSIS JSON Source can take some work off your plate.

The post How to read OData in SSIS – REST API Example appeared first on ZappySys Blog.

The REST API is becoming increasingly popular in the Data Integration landscape. If you are an SSIS Developer (not a coder) and stumbled upon a question What is REST API and how to consume API inside an SSIS package?” then you are at the right place :). We encourage you to read this article carefully and follow the various links we provided.

In this article, you will learn about many REST API Tips / Tools on the topic of how to call a REST API using the SSIS REST API Task, JSON Source, or XML Source. You will also learn how to perform HTTP GET and HTTP POST Requests without needing to know any programming languages (e.g., C#, Java, Python) – simply do a drag-and-drop operation in SSIS.

What is REST API / RESTful Web Service

Let’s first understand what a REST API is, also sometimes referred to as a RESTful Web Service. This is a new buzzword that you will likely hear frequently. More and more services are available in the Cloud, which makes it obvious to come up with some way for you to access data in a more firewall-friendly manner. What can be a better option than accessing it using a several-decade-old HTTP protocol, so no additional configuration is required for anybody accessing Cloud Services?

So, in short highlights for REST Web Service

• REST Web Service is a stateless client-server service model
• By passing an HTTP verb, you can perform server-side action over the standard HTTP protocol (e.g., GET, POST, LIST, DELETE, HEAD)
• You can pass parameters via URL query string and via HTTP Headers
• If you are doing HTTP POST, then you can pass additional data in the Request Body along with the other two methods described above
• REST API works by sending an HTTP Request and getting an HTTP Response
• The HTTP Response of a web service can contain Headers and Response Data, which can be in binary, text, JSON, XML, or other formats.
• You can secure your data by simply sending it over the HTTPS (Secure HTTP) protocol or encrypting values passed along with your request.

Tasks/Components in SSIS for Consuming RESTful API / WebService

	JSON Source (Read from REST API, JSON file or OData Service): Use this dataflow component when you have to fetch data from a REST API web service like a table. This component enables you to extract JSON data from web services and de-normalize nested structures, allowing you to save the data to a Relational database, such as SQL Server, or any other target (Oracle, FlatFile, Excel, MySQL). This component also supports reading local JSON files or direct JavaScript strings (Wildcard pattern support, e.g., c:\data\file*.json).
	Web API Destination (POST data to API URL): Use this data flow component when you need to load data into a target system via API calls (POST to URL). E.g., Loading Contacts into a Marketing System like MailChimp for email campaigns. Loading documents into CRM or document storage systems, e.g,. Couchbase or ElasticSearch.
	REST API Tak: Use this task when you don’t want to pull REST API data in tabular format but want to call the REST API for POST data to the server, DELETE data from the server, or things like download an HTML page, extract Authentication tokens, etc, where you are not necessarily dealing with data in tabular format. This task also allows you many other options, such as saving the RAW response into a variable or a file.
	XML Source (SOAP, File, REST): Use this dataflow component when you have to fetch data from an XML or SOAP web service and consume data like a table. This component allows you to extract data from a web service and save it to SQL Server or any other target (Oracle, FlatFile, Excel, MySQL). This component also supports reading local XML files or a direct XML string.
	CSV Source (File, REST): Use this dataflow component when you have to fetch data from File or REST API which returns data in CSV format . This component allows you to extract data from a web service and save it to SQL Server or any other target (Oracle, FlatFile, Excel, MySQL). This component also supports reading local CSV files or direct CSV string.

Using SSIS JSON Source to read from REST API and load into SQL Server

If you need to consume a REST API Service and store the result into SQL Server or any other RDBMS/FLAT File, then you can check this SSIS JSON Source. The JSON Source can deserialize your nested JSON (it also supports JSONPath filter expressions).

REST API Task – Ad-hoc web requests – Call REST API (POST, DELETE)

HTTP GET Request using SSIS Web Service Task or JSON/XML Source

In this example, we will retrieve JSON data from an OData web service.

• Download and Install SSIS PowerPack
• Create a new sample SSIS package in BIDS/SSDT
• From the SSIS Toolbox, look for items starting with “ZS”. Drag and Drop [ZS Rest API Web Service Task] to Designer Surface
• Configure HTTP Request properties as below
• Configure HTTP Response properties as below
• Run SSIS Package

Pass values in HTTP Headers

Calling REST API in SSIS using REST API Task, Pass headers, Body, Url Parameters

SSIS REST Api Task – HTTP GET, Test SSIS Web Service Call, Pass Custom Header

REST API response validation ( By status code, content, header)

Pass values in the Query String dynamically

HTTP POST Request using SSIS Web Service Task or JSON/XML Source

Passing Credentials to your Web Service

Basic Authentication – Set Authorization Header (Base64 Encoding)

Token-based approach – such as OAuth

Windows Authentication (NTLM)

By default, ZappySys REST API Tasks/Components use your default credentials. You can also use the HTTP Connection Manager with the “Use Credentials”> “Use Windows Authentication” option. See this article

Pass Client Certificate

Sometimes you may have to pass a client certificate along with your web request. The server checks the certificate passed along with your request and, if it matches, proceeds with your request. Here is the real-world use case How Azure Management Api uses Client Certificate.

ZappySys HTTP Connection Manager and OAuth Connection Manager both support passing a Client certificate.

Saving HTTP Web Service Response to a File or Variable

Let’s break down how to save the response from a ZappySys REST API Task into either an SSIS variable or a file within your SSIS package.

Scenario: Imagine you’re using the ZappySys REST API Task to retrieve JSON data from a URL. You want to store this data for further processing within your SSIS package.

1. Saving the REST API Response to an SSIS Variable

This is ideal when you need to manipulate the data within your SSIS package.

1. Add a ZappySys REST API Task to your Control Flow:
   Drag and drop the task onto your SSIS package’s Control Flow.
2. Configure the REST API Task:
   • Set the Connection Property to your REST API Connection Manager. (Optional)
   • Specify the URL of the API endpoint.
   • Configure any necessary headers, query parameters, or request body.
   • Go to the “Response Settings” tab.
3. Configure the Output Variable:
   • Check-mark Save Response content
   • Select Save Mode as “Save to Variable”
   • Select or create an SSIS variable of type String. This variable, named User::WeatherResponse will store the REST API response.
   

   Saving the REST API Response to an SSIS Variable

2. Saving the REST API Response to a File

This is useful for archiving the response or when you need to process the data outside of SSIS.

1. Add a ZappySys REST API Task to your Control Flow:
   Drag and drop the task onto your SSIS package’s Control Flow.
2. Configure the REST API Task:
   • Set the Connection Property to your REST API Connection Manager. (Optional)
   • Specify the URL of the API endpoint.
   • Configure any necessary headers, query parameters, or request body.
   • Go to the “Response Settings” tab.
3. Configure the Output File:
   • Check-mark Save Response content
   • Select Save Mode as “Save to File”
   • In the “Enter File Path” property, specify the path and filename where you want to save the REST API response. For example, if you are fetching JSON data from a product API, you can save it to the following file: C:\Data\Products.json.

Saving the REST API Response to a File

REST APIs enable array filtering to extract specific data from JSON or XML responses. This is achieved through techniques such as query parameters, JSONPath (for JSON), and XPath (for XML), allowing for the precise selection of elements. Additionally, regular expressions can be employed for string-based filtering and data validation.

REST APIs enable array filtering to extract specific data from JSON or XML responses.

Key Setting: “Treat response as binary”

As highlighted in the steps, the “Treat response as binary” option in the “Response” tab is the most important setting for correctly saving binary files. If you don’t check this, the task might try to interpret the binary data as text.

Example Scenario:

Let’s say you want to download a ZIP file from a REST API or a Public URL:

• API URL: https://example.com/api/download/archive.zip
• File Path: C:\Downloads\archive.zip
• In the ZS Rest API Task’s “Response” tab, you would:
  • Check-mark Save Response content
  • Select Save Mode as “Save to File”
  • Check “Treat response as binary”.
  • In the “Enter File Path” property, specify the path and filename where you want to save the REST API response. C:\Downloads\archive.zip.
  

  REST API Task – Download File

By following these steps, you can effectively use the ZappySys Rest API Task to download and save binary content from REST APIs to files within your SSIS workflows.

Save HTTP Web Response Headers / StatusCode

HTTP Web Response Validation

REST API Task supports validating your response for a specific header, status code, or content value. See the Validation tab for more information. Using this feature, you can throw an error if a certain header is missing from the response or reject the response if the StatusCode is other than 200.

Extract a single value from JSON/XML Web Response (e.g., Token)

The most common use case is if you are getting a token from your JSON REST API service. Assume that Service sends you a response in the following format, but you only care about the access_token property from the response below. In that case you can use JSON Path expression like this $.data.access_token

{
    data: {
        access_token: 111223344444,
        expires_in: 3600
    }
}

REST API Testing / Debugging Tools (3rd party)

Now, let’s look at a few popular tools that you will need to test/debug REST API calls during your development cycle.

Testing Rest API

There are a  few ways to test REST APIs, and each tool has its pros and cons, but the most popular tool to test REST API is Postman, so we will start from there

1. Postman – GUI
   Postman offers an excellent User interface to test the API. You can edit URLs, methods, and parameters, and add authentication with ease. See our article on how to migrate Postman settings to zappysys UI. Many API Providers offer easy-to-use Postman settings files for Testing API in Postman. If your API offers such a file, simply import it into Postman and start testing before using the ZappySys UI.
2. SoapUI – GUI
   SoapUI is another tool to test SOAP or REST APIs. They are primarily known for SOAP API Testing, but in recent versions, they have enhanced REST API Testing capability.
3. cURL – Command line
   If you want to call/test the API via the command line, then this is the most popular tool. Many API vendors use cURL syntax in their API documentation. Here is how to migrate cURL syntax to ZappySys UI.
4. Fiddler- GUI
   Fiddler has limited capability to test API call – See Replay in Composer option
5. ZappySys REST API Task
   You can use the ZappySys REST API Task to test the API inside Visual Studio. Edit URL / Headers / Body, etc, and click the Test Request Response option

Debugging REST API – Proxy

So far, we have seen client tools to test Request/Response for the URL you entered in the UI, but what if you want to Sniff Requests sent by some applications for which you have no way to know what Request Data / URL, or Body is sent behind the scenes? What response is it getting back and so on? Well, not to worry… Telerik Fiddler is the most popular tool at your fingertips and it’s FREE.

1. Fiddler – HTTP /  HTTPS Proxy / Debugger
2. Wireshark – Low-level Network Packet Sniffer

Conclusion

The REST API is becoming increasingly popular every day. With that, Data integration from RESTful services is going to be a challenge. The SSIS REST API Task and SSIS JSON Source can help resolve some of the issues you may encounter with SSIS. Use JSON Source if you need to store data into a SQL Table. Use REST API Task if you need to make an ad-hoc HTTP/REST request (such as get token, Delete Record, etc). ZappySys REST API Components gives you total control over your REST API Integration challenges without learning a programming language (e.g., Python, Ruby, C#, Java).

The post How to Call REST API in SSIS – Read JSON / XML / CSV appeared first on ZappySys Blog.