Read Freshdesk data in SSIS – REST API Call

Contents

Introduction

In this post we will learn how to read Freshdesk data in SSIS using ZappySys JSON / REST API Source. We will see how to authenticate, paginate / parse and load Freshdesk data into SQL Server Table in just few clicks. If you don’t use Freshdesk and considering using Zendesk instead then check our previous post on how to Read Zendesk data in SSIS.

Freshdesk is a cloud based / managed service like Zendesk to for Support / Customer service.

Prerequisites

Before we perform the steps listed in this article, you will need to make sure the following prerequisites are met:
  1. SSIS designer installed. Sometimes it is referred to as BIDS or SSDT (download it from the Microsoft site).
  2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
  3. Make sure ZappySys SSIS PowerPack is installed (download it, if you haven't already).
  4. (Optional step). Read this article, if you are planning to deploy packages to a server and schedule their execution later.

Getting Started

So let’s get started. We will see step by step instructions to configure Freshdesk API connection, setup JSON Source to read data from Freshdesk and finally load data into SQL Server table. We will use List Tickets API in this article for demo purpose but read other useful information on that page if you are new to REST API.

Setup Freshdesk REST API Connection

Very first step to call any Freshdesk API in SSIS is to configure HTTP connection (Basic Authentication – Using UserID/ Password).

  1. Open Visual Studio (i.e. SSDT) to create new SSIS Package
  2. From Control Flow SSIS Toolbox drag REST API task
  3. Select URL From HTTP Connection Option from Drop down
  4. Click New HTTP Connection
  5. Configure HTTP connection like below.
  6. Enter any Valid URL (This will be ignored if you Override URL on REST API Task)
  7. Select Basic Authentication (Use User ID and Password)
    Configure Freshdesk REST API Connection in SSIS - Basic Authentication (User ID / Password)

    Configure Freshdesk REST API Connection in SSIS – Basic Authentication (User ID / Password)

  8. Click OK to close HTTP Connection
  9. On Main UI of REST API Task, check Use Direct URL and enter below URL (Change YourDomain Part)
    https://{your-domain}.freshdesk.com/api/v2/tickets
  10. Click Test Request / Response. If you see OK status then you have successfully called Freshdesk API. That’s all.
    Calling Freshdesk REST API in SSIS ( Read Tickets example)

    Calling Freshdesk REST API in SSIS ( Read Tickets example)

In the section we will see how to Parse REST API response and load data into SQL Table.

Read Freshdesk data in SSIS

Now let’s look at steps to configure SSIS JSON Source to read from Freshdesk REST API.

  1. Drag Data Flow task from toolbox. Double click to go to Data Flow designer
  2. Drag ZS JSON Source (For File / REST API)
  3. Configure JSON Source like below.
  4. Check Use Connection option and select HTTP connection created in the previous section.
  5. Enter API URL from which you like to read data. In out example we want to read tickets created after 2019-01-01 so we will call API like below. You can skip ?updated_since=2019-01-01 if you like to read all tickets.
  6. You can also Use Variable Placeholder in URL to make things dynamic like below
  7. Click Preview to see sample data.
    Read Freshdesk data in SSIS - Fetch Tickets example (With REST API pagination)

    Read Freshdesk data in SSIS – Fetch Tickets example (With REST API pagination)

  8. Configure Pagination like below. Basically you have to select Pagination Mode = URL Parameter Mode. Enter Page num indicator as  page
    Configure REST API pagination (URL Parameter Mode)

    Configure REST API pagination (URL Parameter Mode)

  9. Thats it. Now click OK to save UI.

Load Freshdesk data into SQL Server Table

Now let's look at how to load data into target like SQL Server, Oracle or Flat File. In below example we will see loading data into SQL Server database but steps may remain same for other targets which can be accessed using OLEDB Drivers (e.g. Oracle).
  1. Inside Data Flow, Drag and drop Upsert Destination Component from SSIS Toolbox
  2. Connect our Source component to Upsert Destination
  3. Double click Upsert Destination to configure it
  4. Select Target Connection or click NEW to create new connectionConfigure SSIS Upsert Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS Configure SSIS Upsert Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS
  5. Select Target Table or click NEW to create new table based on source columns
  6. Click on Mappings Tab to Auto map columns by name. You can change mappings as you need SSIS Upsert Destination - Columns Mappings SSIS Upsert Destination - Columns Mappings
  7. Click OK to Save Upsert Destination Settings
  8. That's it, You are now ready to run data flow. NOTE: If you wish to debug data flow and see records when you run, add data viewer by right click on blue arrow > Click Enable Data Viewer
  9. To execute data flow, Right click anywhere inside Data Flow Surface and click Execute Task
 

Conclusion

In this article we saw how easy it is to read data from any REST API service like Freshdesk REST API and load into Relational DB like SQL Server. Try SSIS PowerPack for FREE and explore many other 70+ Tasks /Components not discussed in this article.

 

Posted in HTTP Connection, REST API, REST API Integration, REST Connector, SSIS JSON Source (File/REST), SSIS REST API Task and tagged , , , .