Read WordPress data using SSIS REST API Call

Introduction

WordPressIn this post you will learn how to Read WordPress data using SSIS REST API Call (Drag and drop approach without any coding).  We will use SSIS JSON/ REST API Source to extract data from WordPress API. In this blog we will see how to get all WordPress Posts, Pages and Categories using JSON Source and load it into the SQL Server.

Now let’s look at step by step approach to call WordPress REST API.

Prerequisite

  1. First, you will need to have SSIS installed
  2. Secondly, make sure to have SSDT
  3. Thirdly, You have obtained WordPress account credential with read and write permission.
  4. Finally, do not forget to install ZappySys SSIS PowerPack

What is WordPress

WordPress is a free and open-source content management system based on PHP & MySQL. Features include a plugin architecture and a template system. It is most associated with blogging but supports other types of web content including more traditional mailing lists and forums, media galleries, and online stores.

Read data from WordPress using SSIS JSON Source

Let’s start with an example. We use SSIS JSON Source Task to make call to WordPress API, we will load WordPress Pages data to SQL Server.

Get all Posts, Pages and Categories using WordPress API

  1. First of All, Drag and drop Data Flow Task from SSIS Toolbox and double click it to edit.Drag and Drop SSIS Data Flow Task from SSIS Toolbox
  2. Furthermore, Drag and Drop JSON Source(REST API and File) File Task from the SSIS Toolbox.
    Drag and Drop JSON Source Component

    Drag and Drop JSON Source Component

  3. Double click on that Component to Edit the Configurations. Select File path or web URL from the Access Mode drop-down.
    Enter Web URL like this to Get all Pages. Please refer to this link of WordPress API for more information.
    To Get all Posts
    To Get all Categories
    Here, your-wordpress-domain is your WordPress site domain name (here we are using our zappysys.com wordpress site).

    * page is the index of the first item returned in the page of results.
    * per_page the maximum number of items that can be returned per page. Each API endpoint may have a different limit for the number of items returned, and these limits may change without notice. (For more See this WordPress API Documentation for Pagination)

    Set URL in JSON Source

    Set URL in JSON Source

  4. Check Use Credentials Option and select a new ZS-HTTP Connection from the drop-down.
  5. Configure new HTTP Connection for WordPress as below.
    Basically, you need to enter 3 pieces on the General tab (i.e. WordPress Site URL, UserName and Password)

    JIRA Connection: HTTP Connection Manager

    WordPress Connection: HTTP Connection Manager

  6. Likewise, we are going to set Pagination related configurations for WordPress API request. For example, if you want to read 100 records in each response and keep doing it until last page found then set Page=1, Per_page=100 and Increment by 1.
    JSON Source – WordPress request Pagination Configurations Settings

    JSON Source – WordPress request Pagination Configurations Settings

  7. Select the desire Filter and click on Preview button to see the response data.
    JSON Source: Preview WordPress Data

    JSON Source: Preview WordPress Data

  8. Finally, click OK to save JSON Source settings.
  9. That’s it we are ready to load WordPress data to SQL Server.

Load WordPress data into SQL Server

Now let's look at how to load data into target like SQL Server, Oracle or Flat File. In below example we will see loading data into SQL Server database but steps may remain same for other targets which can be accessed using OLEDB Drivers (e.g. Oracle).
  1. Inside Data Flow, Drag OLEDB Destination from SSIS Toolbox
  2. Connect our Source component to OLEDB Destination
  3. Double click OLEDB Destination to configure it
  4. Select Target Connection or click NEW to create new connection
    Configure SSIS OLEDB Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS

    Configure SSIS OLEDB Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS

  5. Select Target Table or click NEW to create new table based on source columns
  6. Click on Mappings Tab to Auto map columns by name. You can change mappings as you need
    Create NEW Table SSIS OLEDB Destination

    Create NEW Table SSIS OLEDB Destination

  7. Click OK to Save OLEDB Destination Settings
  8. That's it, You are now ready to run data flow. NOTE: If you wish to debug data flow and see records when you run, add data viewer by right click on blue arrow > Click Enable Data Viewer
  9. To execute data flow, Right click anywhere inside Data Flow Surface and click Execute Task
 

Conclusion

In this post you have seen how easy it is to Read WordPress data using SSIS REST API Call in few clicks. We used JSON Source Component to call WordPress REST API and load data into SQL server. You can download SSIS PowerPack here to try many other scenarios not discussed in this blog along with 70+ other components.

References

Finally, you can use the following links for more information about the use of WordPress REST API with our tools:

Posted in JSON File / REST API Driver, SSIS JSON Source (File/REST) and tagged , , , .