In our previous blog post we saw how to import SOAP API data in Power BI. Now let’s continue to explore more use cases. In this article we will check how to Connect WorkDay in Power BI. We will show you step by step approach on how to call Workday SOAP API in Power BI and create Power BI reports / dashboards using imported datasets.

We will use ZappySys XML / SOAP Driver for this demonstration.

Requirements

1. First, you will need to have Power BI Desktop installed
2. Make sure you have ZappySys ODBC Power Pack installed
3. Download and have install SoapUI

Get Started

Once above requirements are fulfilled, let’s learn how to use Workday SOAP API and import workday data in Power BI.

Here are high level steps to import Workday data in Power BI using ZappySys XML / SOAP Driver.

1. Craft Workday SOAP request XML using tool like SoapUI for desired operation (e.g Get_Employee )
2. Configure ODBC DSN for Workday SOAP API Connection using ZappySys XML / SOAP Driver. (Select HTTP Connection with SOAP WSS mode using workday Userid / password).
3. Test SQL Query for desired SOAP API call
4. Finally, Import Data in Power BI using ODBC Connection, Supply Custom SQL Query during Import Process (Crafted in previous step).

Now let’s look at each step in detail in the following sections

Craft Workday SOAP request XML using SoapUI

Now let’s look at how to use tool like SoapUI to craft SOAP API requests. It would be helpful if you can refer this article to learn more about using SoapUI tool (Check SoapUI section in that article).

Obtain Workday SOAP WSDL URL (API Metadata URL)

First step to consume workday api using SSIS is download SOAP WSDL file. WSDL is an XML file which describes available API operations in Workday. It also describes the structure of XML request / response. Here is the list of available WSDL for various Workday API Services. You can right click on WSDL icon and save to local disk or use Direct WSDL URL in next section for importing in SoapUI tool.

You can find more information about Other Workday API here.

Obtain Workday API URL

Once you have WSDL file, next step is craft correct URL for API service you like to call. Service name can be obtained from here (Check service column). Notice my-instance and my-tenantid placeholders (Its specific to your company).

Syntax:  

https://<workday host name>.workday.com/ccx/service/<tenant name>/<service-name>

https://MY-INSTANCE.workday.com/ccx/service/MY-TenantID/Human_Resources

Load WSDL in SoapUI

Now it’s time to load WSDL and craft some SOAP Request and test. For detailed steps check this (watch Video to learn SoapUI tool) .

1. Open SoapUI tool
2. Click new File > New Soap Project
3. Enter WSDL Path and Name of the project
4. You will see many different operations in the imported project. For example below screenshot shows Human_Resources service and Get Employee API call. You can fill out necessary parameters in your request / remove optional parameters from XML as needed.
   

   Create SOAP Request Body from WSDL (Using SoapUI tool)

Configure Soap Request Body, URL and Credentials in SoapUI

Now it’s time to configure SOAP Request service URL and Credentials. To generate correct Credentials Header in SOAP Request use tool like SoapUI. Check this article to learn more how to generate WSS Security header.

1. Go to the API method you like to call and double click on the default Request node. Check this article to learn more.
2. Change default request parameters (Add /Remove as per your need). You can click on Re-generate icon in the toolbar to reset the request and create optional parameters.
3. Enter the Service URL
   Example:  https://myinstance0001.workday.com/ccx/service/AB11111111/Human_Resources
4. In the properties window select WSS-Password Type as PasswordDigest, Enter workday username and password
5. Once you done click Play button to execute the request. If things go well then you should see response on your right panel as below.
   

   SoapUI – Call Workday SOAP API (Configure WSS Security, Service URL, Body, Test Request / Response)

Creating ODBC DSN for Workday Connection

Once you test correct parameters in your XML Request we can create ODBC DSN for Workday.

1. Open start menu and search for “ODBC 64”. Launch ODBC Data sources for 64 bit
   

   Open ODBC Data Sources from Startup menu

2. Click Add and select ZappySys XML Driver
   

   ZappySys ODBC Driver for XML / SOAP API

3. Once driver UI is visible change default DSN name (e.g. change something to  WorkdayDSN)
4. Enter API EndPoint URL (Get it from SoapUI) and Select connection type as HTTP.
   Example URL:  https://YOUR-INSTANCE.workday.com/ccx/service/YOUR-TENANT/Human_Resources
5. Configure HTTP connection as below.
   

   Configure Workday Connection in ODBC DSN – XML / SOAP Driver

6. Click OK to save connection.
7. Now Configure other settings as below.
   1. Select Method as POST
   2. Select Content Type as text/xml
   3. Enter one required Header for SOAP API as below
      

      SOAPAction :  ""

   4. Enter Body (Get it from SoapUI ). Here is example body for Get_Employee request.
      

      <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:bsvc="urn:com.workday/bsvc">
         <soapenv:Header/>
         <soapenv:Body>
            <bsvc:Employee_Get>
               <bsvc:Employee_Reference>
                  <bsvc:Integration_ID_Reference>
                     <bsvc:ID>XXXX-1234-CHANGE-THIS-XXXXXXX</bsvc:ID>
                  </bsvc:Integration_ID_Reference>
               </bsvc:Employee_Reference>
            </bsvc:Employee_Get>
         </soapenv:Body>
      </soapenv:Envelope>

      NOTE: Change ID to your own record id.
   5. ( Optional ) If your API returns more than one record (e.g. if you call API like Get_Workers) then select Filter as per below screenshot (Click Yes if asked to treat selection as array). For example in below screenshot we called Get_Workers request which returns multiple records under wd:Worker node. Look for Array Icon in the Filter selection. Skip this step if no array extraction needed.
8. After Configuration your screen may look like below.
   

   Configure Workday SOAP Request – URL, Body, Filter

9. Now go to Preview Tab and you will see default SQL query generated for you. Here is example query (Replace Body part as per your need)
   

   SELECT * FROM $
   /*Below WITH clause is optional, uncomment parameter(s) values you like to set 
    which are different than DSN properties.
   */
   WITH(
   	Src='https://xxxxxxxxxx.workday.com/ccx/service/yyyyyyyyy/Human_Resources'
   	--,ElementsToTreatAsArray='wd:Worker'
   	--,DataConnectionType='HTTP'
   	--,SoapWssPasswordType='PasswordDigest'
   	--,UserName='testuser001'
   	--,CredentialType='SoapWss'
   	--,Filter='$.env:Envelope.env:Body.wd:Get_Workers_Response.wd:Response_Data.wd:Worker[*]'
   	--,RequestContentTypeCode='TextXmlUtf8'
   	--,RequestMethod='POST'
   	--,Header='SOAPAction: ""'	
   	,RequestData='
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:bsvc="urn:com.workday/bsvc">
      <soapenv:Header/>
      <soapenv:Body>
           .......... .......... .......... 
           .......... FROM SOAPUI .........  
           .......... .......... ..........        
      </soapenv:Body>
   </soapenv:Envelope>'
   )

10. Click Preview button to see data. Copy your SQL Query for later step.
    

    Preview Workday Data (Test SQL query for ODBC XML / SOAP Driver)

11. Click OK to save DSN

Import Workday data in Power BI

Now let’s move to the final step. We will use SQL query and DSN created in previous step for data load in Power BI.

1. Open Power BI Desktop
2. Click on Get Data > More Data… > Other > ODBC
   

   Import SOAP API in Power BI (Using ZappySys XML ODBC Driver)

3. Select your ODBC DSN Source from the dropdown (Assuming you have created a ODBC Data Source using ZappySys XML Driver – See the previous Section)
4. Expand Advanced Options to enter custom Query.
5. Enter your SOAP API call SQL query here. For example purpose, you can use below query to get all JOB categories.
   

   SELECT * FROM $
   WITH(
   	 Src='https://xxxxxxxxxx.workday.com/ccx/service/yyyyyyyyyy/Human_Resources'	
   	--,DataConnectionType='HTTP'
   	--,SoapWssPasswordType='PasswordText'
   	--,UserName='testuser001'
   	--,Password='###########'
   	--,CredentialType='SoapWss'
   	,RequestMethod='POST'
   	,RequestContentTypeCode='TextXmlUtf8'
   	,Filter='$.soapenv:Envelope.soapenv:Body.bsvc:Get_Job_Categories_Response.bsvc:Response_Data.bsvc:Job_Category[*]'
   	,ElementsToTreatAsArray='bsvc:Job_Category'
   	,RequestData='
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:bsvc="urn:com.workday/bsvc">
      <soapenv:Body>
         <bsvc:Get_Job_Categories_Request/>
      </soapenv:Body>
   </soapenv:Envelope>'
   )

    
   

   Import Workday SOAP Web Service Data in Power BI (ZappySys XML Driver)

6. Once you see data preview click OK to import.
7. After data import is done you can edit your dataset (e.g. remove unwanted columns)
8. You can also edit Source query after data is imported (See below)
   

   Edit Power BI Data Source SQL after REST / SOAP Web Service Import

9. That’s it. You can now create your dashboards using data we just imported. See below example.
   

   Import Workday data in Power BI dashboard

Workday API Pagination – Retrieve more than 100 rows

By default Workday API will return only 100 records unless you specify Page / Count in Response_Filter . Regardless it will only send you 999 maximum rows. To fetch more records after first page you have to setup pagination in XML Driver. Below screenshot

1. Select Paging Mode= POST Data Mode
2. Enter Page Number Indicator (e.g. <%page%> )
3. Click browse button for Max Pages Expression   (select node which indicates how many pages in response. e.g. Total_Pages)
   

   Workday SOAP API Pagination Settings in ZappySys XML / SOAP Driver

4. Once this is done go to Settings Tab. Edit your Body and make sure you supply Response_Filter. Pass Response_Filter parameter as below. See how we used <%page%> placeholder. You can use tool like fiddler to see multiple response at runtime. Run package and check fiddler. In the below example we are fetching maximum 300 rows per response. It will keep reading until all pages are done. For example if you have 1000 rows to read then it will take 4 response behind the scene.
   

   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:bsvc="urn:com.workday/bsvc">
      <soapenv:Body>
         <bsvc:Get_Job_Categories_Request bsvc:version="">
           <bsvc:Response_Filter>
               <bsvc:Page><%page%></bsvc:Page>
               <bsvc:Count>300</bsvc:Count>
            </bsvc:Response_Filter>
         </bsvc:Get_Job_Categories_Request>
      </soapenv:Body>
   </soapenv:Envelope>

   

   Workday API Pagination – Supply Response_Filter (Page Size and Current Page Number)

That’s it. Now if you run your query it will fetch all pages and you can now read more than 1000 rows. Make sure you configure Count at reasonable size (Rows per Page) in above screenshot we use 5 just to demo small sample. We suggest you to to set around 300.

How to pass parameters to SQL query in Power BI

let
    vKey=paraAPIKey,
    Source = Odbc.Query(
"dsn=ZS - OData Customers", 
"SELECT * FROM $ WITH (SRC='http://httpbin.org/post'," 
& "METHOD='POST'," 
& "HEADER='Content-Type:application/json'," 
& "BODY=@'{""CallerId"":1111, ""ApiKey"":""" & vKey & """}')")
in
    Source

How to debug Web Requests using Fiddler

There will be a time you like to see RAW HTTP Requests generated by driver for NetSuite SOAP API.  If you like to do that check this article.

Conclusion

Consuming data from Workday API or any XML / SOAP based API in your Reporting / ETL apps can be tricky. But we have seen how to solve this challenging task using ZappySys XML Driver in few clicks. Download ODBC PowerPack and explore many other API integration scenarios not discussed in this article.

The post Connect Workday in Power BI – Import SOAP Data appeared first on ZappySys Blog.

In our previous post, we saw in general how to call REST API in Power BI. Now let’s look at how to call SOAP API in Power BI (e.g. WorkDay, Google AdWords services). SOAP API is in XML format so we will be using ZappySys XML Driver for a demonstration in this article.

To learn how to automate dataset refresh on a scheduled basis you can again refer to the previous post because that won’t be covered in this article.

So let’s get started !!!

Requirements

1. First, you will need to have Power BI Desktop (FREE) installed
2. Make sure you have ZappySys ODBC Power Pack installed

Creating SQL Queries to call SOAP API in Power BI (Using XML Driver)

Very first step to call SOAP API in Power BI is to understand how to craft SOAP Requests using ZappySys XML Driver. Once you play with sample SOAP queries you can copy/paste that into Power BI.

To call SOAP API you need to know Request XML Body Structure. If you are not sure how to create SOAP Request body then no worries. Check this article to learn how to generate SOAP Request body using the Free tool SoapUI. Basically, you have to use SoapUI to generate Request XML and after that, you can replace parameters as needed in the generated body.

What is SOAP Web Service?

If you are new to SOAP Web Service sometimes referred as XML Web Service then please read some concept about SOAP Web service standard from this link There are two important aspects in SOAP Web service.

1. Getting WSDL file or URL
2. Knowing exact Web Service URL

What is WSDL

In very simple term WSDL (often pronounced as whiz-dull) is nothing but a document which describes Service metadata (e.g. Functions you can call, Request parameters, response structure etc). Some service simply give you WSDL as xml file you can download on local machine and then analyze or sometimes you may get direct URL (e.g. http://api.mycompany.com/hr-soap-service/?wsdl )

Example SQL Query for SOAP API call using ZappySys XML Driver

Here is an example SQL query you can write to call SOAP API. If you not sure about many details then check next few sections on how to use XML Driver User Interface to build desired SQL query to POST data to XML SOAP Web Service without any coding.

SELECT * FROM $
WITH(
	 Src='http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx'
	,DataConnectionType='HTTP'
	,CredentialType='Basic' --OR SoapWss
	,SoapWssPasswordType='PasswordText'
	,UserName='myuser'
	,Password='pass$$w123'
	,Filter='$.soap:Envelope.soap:Body.GetHolidaysAvailableResponse.GetHolidaysAvailableResult.HolidayCode[*]'
	,ElementsToTreatAsArray='HolidayCode'	
	,RequestMethod='POST'	
	,Header='Content-Type: text/xml;charset=UTF-8 || SOAPAction: "http://www.holidaywebservice.com/HolidayService_v2/GetHolidaysAvailable"'
	,RequestData='
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:hol="http://www.holidaywebservice.com/HolidayService_v2/">
   <soapenv:Header/>
   <soapenv:Body>
      <hol:GetHolidaysAvailable>
         <!--type: Country - enumeration: [Canada,GreatBritain,IrelandNorthern,IrelandRepublicOf,Scotland,UnitedStates]-->
         <hol:countryCode>UnitedStates</hol:countryCode>
      </hol:GetHolidaysAvailable>
   </soapenv:Body>
</soapenv:Envelope>'
)

Now let's look at steps to create SQL query to call SOAP API. Later we will see how to generate code for your desired programming language (e.g. C# or SQL Server)

Video Tutorial - Introduction to SOAP Web Service and SoapUI tool

Before we dive into details about calling SOAP API using ZappySys XML Driver, lets first understand what is SOAP API and how to create SOAP requests using SoapUI tool. You will learn more about this process in the later section. The video contains some fragment about using SOAP API in SSIS but just ignore that part because we will be calling Soap API using ZappySys ODBC Driver rather than SSIS Components.  

Using SoapUI to test SOAP API call / Create Request Body XML

Assuming you have downloaded and installed SoapUI from here, now we are ready to use WSDL for your SOAP Web Service Calls. If you do not have WSDL file or URL handy then contact your API provider (sometimes you just have to add ?wsdl at the end of your Service URL to get WSDL so try that. Example: http://mycompany/myservice?wsdl ). If you don't know what is WSDL then in short, WSDL is Web service Description Language (i.e. XML file which describes your SOAP Service). WSDL helps to craft SOAP API request Body for ZappySys XML Driver. So Let's get started.

1. Open SoapUI and click SOAP button to create new SOAP Project
2. Enter WSDL URL or File Path of WSDLFor example WSDL for our sample service can be accessed via this URL

   http://www.dneonline.com/calculator.asmx?wsdl

   
   Create new SOAP API Project in SoapUI tool for SOAP API Testing
3. Once WSDL is loaded you will see possible operations you can call for your SOAP Web Service.
4. If your web service requires credentials then you have to configure it. There are two common credential types for public services (SOAP WSS or BASIC )
   1. To use SOAP WSS Credentials select request node and enter UserId, Password, and WSS-PasswordType (PasswordText or PasswordHash)
      
      Configure SOAP WSS Credentials for SoapUI (SOAP API Testing Tool)
   2. To use BASIC Auth Credentials select request node and double-click it. At the bottom click on Auth (Basic) and From Authorization dropdown click Add New and Select Basic.
      Configure Basic Authorization for SoapUI (SOAP API Testing Tool)
5. Now you can test your request first Double-click on the request node to open request editor.
6. Change necessary parameters, remove optional or unwanted parameters. If you want to regenerate request you can click on Recreate default request toolbar icon.
   Create SOAP Request XML (With Optional Parameters)
7. Once your SOAP Request XML is ready, Click the Play button in the toolbar to execute SOAP API Request and Response will appear in Right side panel. Test SOAP API using SoapUI Tool (Change Default XML Body / Parameters, Execute and See Response)

Create DSN using ZappySys XML Driver to call SOAP API

Once you have tested your SOAP API in SoapUI tool, we are ready to use ZappySys XML driver to call SOAP API in your preferred BI tool or Programming language.

1. First open ODBC Data Sources (search ODBC in your start menu or go under ZappySys > ODBC PowerPack > ODBC 64 bit)
2. Goto System DSN Tab (or User DSN which is not used by Service account)
3. Click Add and Select ZappySys XML Driver ZappySys ODBC Driver for XML / SOAP API
4. Configure API URL, Request Method and Request Body as below ZappySys XML Driver - Calling SOAP API - Configure URL, Method, Body
5. (This step is Optional) If your SOAP API requires credentials then Select Connection Type to HTTP and configure as below.
   ZappySys XML Driver - Configure SOAP WSS Credentials or Basic Authorization (Userid, Password)
6. Configure-Request Headers as below (You can get it from Request > Raw tab from SoapUI after you test the request by clicking the Play button) Configure SOAP API Request Headers - ZappySys XML Driver
7. Once credentials entered you can select Filter to extract data from the desired node. Make sure to select array node (see special icon) or select the node which contains all necessary columns if you don't have array node. Select Filter - Extract data from nested XML / SOAP API Response (Denormalize Hierarchy)
8. If prompted select yes to treat selected node as Array (This is helpful when you expect one or more record for selected node) Treat selected node as XML Array Option for SOAP API Response XML

Preview SOAP API Response / Generate SQL Code for SOAP API Call

Once you configure settings for XML Driver now you can preview data or generate example code for desired language (e.g. C#, Python, Java, SQL Server). Go to Preview tab and you will see default query generated based on settings you entered in previous sections. Attributes listed in WITH clause are optional. If you omit attribute in WITH clause it will use it from Properties tab.

Preview Data

Preview SOAP API Response in ZappySys XML Driver

Generate Code Option

Generate Example Code for ZappySys Driver

Import XML SOAP Web Service in Power BI (Using XML Driver)

Once you know how to craft SQL Queries for SOAP API calls, and you save created DSN as per the previous section we are ready to load data into Power BI from XML Web Service.

So let’s get started.

1. First launch Power BI and Click on Get Data >> More Data >> Other >> ODBC
   

   Import SOAP API in Power BI (Using ZappySys XML ODBC Driver)

2. Select your ODBC DSN Source from the dropdown (Assuming you have created a ODBC Data Source using ZappySys XML Driver – See the previous Section)
3. Expand Advanced Options to enter custom Query.
4. Enter your SOAP API call SQL query here. For example purpose, we will use below query to try demo SOAP web service.
   

   SELECT * FROM $
   WITH(
   	 Src='http://www.holidaywebservice.com/HolidayService_v2/HolidayService2.asmx'
   	,DataConnectionType='HTTP'
   	,CredentialType='Basic' --OR SoapWss
   	,SoapWssPasswordType='PasswordText'
   	,UserName='myuser'
   	,Password='pass$$w123'
   	,Filter='$.soap:Envelope.soap:Body.GetHolidaysAvailableResponse.GetHolidaysAvailableResult.HolidayCode[*]'
   	,ElementsToTreatAsArray='HolidayCode'	
   	,RequestMethod='POST'	
   	,Header='Content-Type: text/xml;charset=UTF-8 || SOAPAction: "http://www.holidaywebservice.com/HolidayService_v2/GetHolidaysAvailable"'
   	,RequestData='
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:hol="http://www.holidaywebservice.com/HolidayService_v2/">
      <soapenv:Header/>
      <soapenv:Body>
         <hol:GetHolidaysAvailable>
            <!--type: Country - enumeration: [Canada,GreatBritain,IrelandNorthern,IrelandRepublicOf,Scotland,UnitedStates]-->
            <hol:countryCode>UnitedStates</hol:countryCode>
         </hol:GetHolidaysAvailable>
      </soapenv:Body>
   </soapenv:Envelope>'
   )

   

   Enter custom SQL to load SOAP Web Service Data in Power BI (ZappySys XML Driver)

5. Once you see data preview click OK to import.
6. After data import is done you can edit your dataset (e.g. remove unwanted columns)
7. You can also edit Source query after data is imported (See below)
   

   Edit Power BI Data Source SQL after SOAP Web Service Import

8. That’s it. Use your dataset to build Power BI dashboard and when you ready to publish just hit Publish Button.

Using Parameters in Power BI (Dynamic Query)

let
    vKey=paraAPIKey,
    Source = Odbc.Query(
"dsn=ZS - OData Customers", 
"SELECT * FROM $ WITH (SRC='http://httpbin.org/post'," 
& "METHOD='POST'," 
& "HEADER='Content-Type:application/json'," 
& "BODY=@'{""CallerId"":1111, ""ApiKey"":""" & vKey & """}')")
in
    Source

SOAP XML / REST API pagination in Power BI

SELECT * FROM $
WITH(
SRC=@'https://zappysys.com/downloads/files/test/pagination_nextlink_inarray_1.json'
,NextUrlAttributeOrExpr = '$.nextlink'  --keep reading until this attribute is missing. If attribute name contains dot then use brackets like this $.['my.attr.name']
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/page-xml.aspx?page=1&mode=DetectBasedOnResponseStatusCode'
,PagingMode='ByUrlParameter'
,PagingByUrlAttributeName='page'
,PagingByUrlEndStrategy='DetectBasedOnResponseStatusCode'
,PagingByUrlCheckResponseStatusCode=401
,IncrementBy=1
)

SELECT * FROM $
WITH (
SRC=@'https://zappysys.com/downloads/files/test/cust-<%page%>.xml'
,PagingMode='ByUrlPath'
,PagingByUrlAttributeName='<%page%>'
,PagingByUrlEndStrategy='DetectBasedOnRecordCount'
,IncrementBy=1
)

SELECT * FROM $
LIMIT 25
WITH(
	 Src='https://wordpress.org/news/wp-json/wp/v2/categories?per_page=10'
	,PagingMode='ByResponseHeaderRfc5988'
	,WaitTimeMs='200' --//wait 200 ms after each request
)

SOAP XML / REST API Error Handling in Power BI

Sometimes errors occur... they just do and there is nothing you can do! Or can you? Actually, in ODBC PowerPack you can handle them in two ways.

METHOD 1 - Using Error Handling Options

When to use?

You may want to use them when your source is a resource located on the Internet; e.g. a file on a website, a file on an FTP server or just a plain API HTTP response. By default, when a remote server returns an error, data retrieval is stopped, an error is raised and no data is given back to you. This might not be always desirable.

Scenario 1

Imagine a scenario, that there is a web server which each day at 12 AM releases a new JSON file with that day's date as filename, e.g. http://www.some-server.com/data/2018-06-20.json. And, of course, you want to download it and use it daily in your Power BI report. But you have a problem: Power BI report data sources are refreshed each hour and you may get HTTP 404 status code (no file was found) when a file is not released yet. Which consequentially means other data sources won't be updated as well and you will see old and cached data on the report. That's where you could use Continue on any error or Continue when Url is invalid or missing (404 Errors) to avoid an error being raised and let other data sources to be updated.

Scenario 2

Another scenario is when you expect a web server to raise some kind of HTTP error when accessing a URL. You don't want ODBC Data Source to raise an error but instead, you want to get response data. That's where you can use Continue on any error or alike together with  Get response data on error to continue on an error and get the data:

METHOD 2 - Using Connection [Retry Settings]

Another scenario you may run into is a buggy web server. You ask it to give you some file or data and it, like a snotty kid, just doesn't give it to you! You have to ask twice or thrice before it does its job. If that's the case, you have to retry HTTP requests using Connection:

Security / Connection Types (Options for HTTP, OAuth, SOAP) in Power BI

If you need to authenticate or authorize your user to access a web resource, you will need to use one of the Connections:

• HTTP
• OAuth

HTTP Connection

Use HTTP Connection for simple Windows, Basic, NTLM or Kerberos authentication. Just fill in a username and a password and you are good to go!

You can also use HTTP Connection for more sophisticated authentication like:

• SOAP WSS (when accessing a SOAP WebService)
• Static Token / API Key (when need to pass an API key in HTTP header)
• Dynamic Token (same as Static Token method except that each time you need to log in and retrieve a fresh API key)
• JWT Token (As per RFC 7519)

OAuth

If you are trying to access REST API resource, it is a huge chance, you will need to use OAuth Connection. Read this article to understand how OAuth authentication and authorization works and how to use it (article originally was written for SSIS PowerPack, but the concepts and UI stay the same): 
https://zappysys.com/blog/rest-api-authentication-with-oauth-2-0-using-ssis/

Other Considerations for Calling Web API in Power BI

Conclusion

In this article, we explored possibilities to load SOAP API data in Power BI without any coding or ETL approach. You can build powerful dashboards and reports by consuming data from any REST / XML SOAP API services using ZappySys Drivers. You can download ZappySys ODBC Drivers and try yourself to learn many other features not covered in this article.

The post Calling SOAP API in Power BI (Read XML Web Service data) appeared first on ZappySys Blog.

In our previous article, we saw step-by-step approach to read data from workday using SSIS. In this article, we will focus on how to load SQL Server data to Workday (e.g. POST, Create, Update). We will use SSIS Web API Destination and the combination of other Transforms such as SSIS Template Transform and SSIS XML Generator Transform .

The main thing in this article how to create SOAP Request correctly inside your request. This article explains how to use Free tools like SoapUI to create Workday Request. Once you do that you can call virtually any Workday API using the same technique. If you need more help contact our Support and we will be happy to help you step by step.

Step-By-Step : Import SQL Server data to Workday using SSIS

Lets build our SSIS Package to load data from SQL Server or any Source (e.g. Oracle, DB2) to Workday using SOAP API calls. Using below approach you can Create new records or Update existing records in Workday. You can also delete records with same concept.

Basic steps outlined below.

1. Fetch records from SQL Source and build XML Request for each Row (e.g. Create Account)
2. Build XML Request (SOAP Body) using Template Transform or  SSIS XML Generator Transform. If you have Array nodes (e.g. One to Many) then you have to use SSIS XML Generator Transform else use Template Transform for ease of use.
3. Pass input record (e.g. SOAP Body) to Web API destination to call Workday API call (CREATE, UPDATE, DELETE requests)
4. Parse XML Response (i.e. output) using SSIS XML Parser Transform or save raw XML to SQL Server database.
5. Redirect Bad rows or failed requests to log file for review

Now let’s look at each step in detail

Obtain Workday API URL

Once you have WSDL file, next step is craft correct URL for API service you like to call. The service name can be obtained from here (Check service column)

Syntax: https://<workday host name>.workday.com/ccx/service/<tenant name>/<service-name>
Example: https://MY-INSTANCE.workday.com/ccx/service/MY-TenantID/Human_Resources

Craft SOAP Body (XML API Request) using SoapUI

Now its time to craft some SOAP Request. Check steps outlined here (Use SoapUI tool) . Once you have Request Body XML you can change parameters as per your need.

Here is sample  SOAP XML Body for Get Employee call from Human_Resopurces service.

Creating SSIS Connection for Workday SOAP API call using WSS Security

Loading SQL Server data to Workday using SSIS

Let’s look at the real-world scenario. You have Accounts table stored in SQL Server and you like to create same accounts in Workday by calling appropriate API calls.

1. Drag Data flow task from SSIS Toolbox. Double click to edit.
2. Drag OLEDB Source configure to read SQL Table (e.g. Accounts)
3. Drag ZS Template Transform from the toolbox. Connect OLEDB Source to Template Transform. If you need flexible XML Generation then use XML Generator Transform but it may require some learning curve so for simplicity we are skipping that from this article.
4. Enter your SOAP request in the template text (like below) you like to call (This is obtained from the previous section – using tool like SoapUI)
   For Example: To create a new account you can use enter like below. Replace xxxxxxxxxx with Columns placeholder.
   To insert Column name as Placeholder click <<Insert Placeholder>> and then Select [Columns] node. Template Transform outputs column name TemplateOutput. You can use this as Body to feed next step (i.e. Call Workday API using Web API Destination )When you insert placeholder to make sure you use XML Encoded Columns if you expecting a Long text or special characters part of your data.
   Syntax for encoded value is   

   <%CustomerName,FUN_XMLENC%>

     . You don’t need FUN_XMLENC for numeric fields. For normal placeholder without encoding use just name with column placeholder indicators e.g.  

   <%Amount%>

   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:bsvc="urn:com.workday/bsvc">
      <soapenv:Header/>
      <soapenv:Body>
         <bsvc:Workday_Account_for_Worker_Add>
            <bsvc:Worker_Reference>
               <!--You have a CHOICE of the next 2 items at this level-->
               <bsvc:Employee_Reference>
                  <bsvc:Integration_ID_Reference>
                     <bsvc:ID>xxxxxxxxxx</bsvc:ID>
                  </bsvc:Integration_ID_Reference>
               </bsvc:Employee_Reference>
               <bsvc:Contingent_Worker_Reference>
                  <bsvc:Integration_ID_Reference>
                     <bsvc:ID>xxxxxxxxxxxx</bsvc:ID>
                  </bsvc:Integration_ID_Reference>
               </bsvc:Contingent_Worker_Reference>
            </bsvc:Worker_Reference>
            <bsvc:Workday_Account_for_Worker_Data>
               <!--type: string-->
               <bsvc:User_Name>xxxxxxxxxxxxxx</bsvc:User_Name>
            </bsvc:Workday_Account_for_Worker_Data>
         </bsvc:Workday_Account_for_Worker_Add>
      </soapenv:Body>
   </soapenv:Envelope>

   

   Template Transform –
   Create Workday SOAP Request – Create new records using SSIS

5. Drag ZS Web API Destination from SSIS Toolbox. Connect Template Transform to Web API Destination.
6. Configure Web API Destination as below (See we used same HTTP connection created in the previous section)
7. Select HTTP Connection
8. Select Input Column for Body (select TemplateOutput)
9. Select Body Content Type as XML (text/xml;charset=UTF-8)
10. Click Raw Edit above Headers grid. Enter following text (We just need one header)
    

    SOAPAction: ""

    

    SSIS Web API Destination –
    Loading SQL Server data to Workday (SOAP Call – Create, Insert, Update, Delete Records)

11. Now you can connect Web API Destination Output (Blue Arrow to Either XML Parser or some other Destination). Web API destination gives you a response in Raw XML Format. This article explains how to use JSON or XML Parser to parse API response coming from Web API destination.

Video Tutorial (See Part#4) – Create XML and POST data to SOAP Web Service or REST API

Now lets look at examples of creating XML from multiple data sources and POST XML request to SOAP Web Service URL or any other XML Based REST API URL. This video has 5 parts to cover full length tutorial but if you want to fast forward to see XML Generator Transform and Web API Destination then see part#4.

Sending complex SOAP Request (Nested Parent-Child Array)

If you have to call more complex SOAP request which requires Parent-child structure (Nested Array) then you cant use Template Transform. In that case, check this article for real-world example.

Create nested XML for SOAP Request – Multiple inputs (POST data to API using SSIS Web API Destination)

Extract single XML node from SOAP Response

There will be a time when you need to extract just one value out of your response and save into SSIS variable / use it later on. If that’s the case then refer to this article.

Conclusion

In this article we have learned how to load data from SQL Server to Workday using SSIS ( drag and drop approach without coding). Combination of few ZappySys Components (e.g. Web API Destination, XML Generator ) makes it super simple to call any Web API to load data from one system to API endpoint. Download SSIS PowerPack to try many other automation scenarios not discussed in this article.

The post Load SQL Server data to Workday using SSIS / SOAP API appeared first on ZappySys Blog.

In this article you will learn how to get data from WorkDay SOAP web service in few clicks using SSIS SOAP XML Source. All you need to know how to create SOAP Request correctly and provide Authorization (e.g. Userid, Password) inside your request (You can generate that easily using Free tools like SoapUI). Once you do that you can call virtually any WorkDay API using our SSIS XML Source. If you need more help contact our Support and we will be happy to help you step by step.

How to call Workday API in SSIS (Read or Write Data)

Here are high level steps to read or write Workday data in SSIS.

1. Obtain Workday WSDL URL (Service Metadata) and API URL for your tenant
2. Craft POST request XML using tool like SoapUI for desired operation (e.g Get_Employee )
3. Configure SSIS HTTP Connection (for SOAP WSS) using API url and workday Userid / password.
4. Call Workday API using any of these tasks or components SSIS XML Source or SSIS REST API TASK or SSIS Web API Destination to read / write data.

Now lets look at each step in detail in the following sections

Obtain Workday SOAP WSDL URL (API Metadata URL)

First step to consume workday api using SSIS is download SOAP WSDL file. WSDL is XML file which describes available API operations and structure of request and response. Here is the list of available WSDL for various Workday API Services. Right click on WSDL icon and save to local disk. We will use this WSDL in next section to craft SOAP Request using SoapUI tool

You can find more information about Other Workday API here.

Obtain Workday API URL

Once you have WSDL file, next step is craft correct URL for API service you like to call. Service name can be obtained from here (Check service column)

Syntax:

https://<workday host name>.workday.com/ccx/service/<tenant name>/<service-name>

https://MY-INSTANCE.workday.com/ccx/service/MY-TenantID/Human_Resources

Craft SOAP Body (XML API Request) using SoapUI

Now its time to craft some SOAP Request. Check steps outlined here (Use SoapUI tool) . Once you have Request Body XML you can change parameters as per your need.

Here is sample  SOAP XML Body for Get Employee call from Human_Resopurces service.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:bsvc="urn:com.workday/bsvc">
   <soapenv:Header/>
   <soapenv:Body>
      <bsvc:Employee_Get>
         <bsvc:Employee_Reference>
            <bsvc:Integration_ID_Reference>
               <bsvc:ID>gero et</bsvc:ID>
            </bsvc:Integration_ID_Reference>
         </bsvc:Employee_Reference>
      </bsvc:Employee_Get>
   </soapenv:Body>
</soapenv:Envelope>

Create SOAP Request Body from WSDL (Using SoapUI tool)

Creating SSIS Connection for Workday SOAP API call using WSS Security

To create new connection for workday perform the following steps.

1. Two ways you can create HTTP connection for workday
   First approach: Right click in the connection managers panel and click “New Connection…” and Select ZS-HTTP connection from the connection type list and click OK.
   — OR —
   Second approach: If you are already on SSIS XML Source or SSIS REST API TASK or SSIS Web API Destination UI then click [New] next to the Connection Dropdown.
2. Once HTTP Connection UI is visible configure following way.
   1. Enter API URL for Workday (Make sure you don’t enter WSDL URL found here ). Your API URL will be something like below.
      Syntax: https://<workday host name>.workday.com/ccx/service/<tenant name>/<service-name>
      Example: https://wd1-impl-services1.workday.com/ccx/service/MyTenantID/Human_Resources
   2. Select credential type as SOAP WSS (This setting is only found in v2.6.4 or Higher)
   3. Enter your workday userid and password
   4. For WSS password type setting you leave it default (Not set) or change to PasswordHash for more secure communication.
   5. Click OK to save.

SSIS Workday Integration – Create New SOAP Service Connection for Workday API Service (SOAP WSS)

 

Calling Workday sample API using SSIS REST API Task

Here is how you can call any Workday API action from control flow using SSIS REST API TASK. Below is simple way to execute API in SSIS but if you need to parse API response into rows and column then check next section on how to use SSIS XML Source

Configure following settings

1. Select URL Access Mode to URL from Connection
2. Select HTTP Connection created in the previous section
3. Change Request Method to POST
4. Change Content Type to XML (text/xml;charset=UTF-8)
5. Enter SOAP Body (Obtained in the previous section using SoapUI )
6. Click Raw Edit above Headers grid. Enter below string
   

   SOAPAction: ""

7. Click Test Request / Response. You should see Response data.

Calling workday API using SSIS REST API Task

Write workday data using SSIS XML Task (Import from SQL Server to Workday)

Check this article to learn how to load sql server data data to workday

Reading Workday data using SSIS XML Task (Export from workday to SQL Server)

In this section we will not use Here are high level steps.

1. Drag Data Flow Task from SSIS Toolbox
2. Drag SSIS XML Source  from the toolbox.
3. Enter the API URL (Do not enter WSDL URL)
   Example URL => https://wd5-impl-services1.workday.com/ccx/service/{{tenant}}/Human_Resources
4. Select HTTP Connection created in the previous section
5. Check Use Credentials Option and Select HTTP connection created in the previous section
6. Change Request Method to POST
7. Change Content Type to XML (text/xml;charset=UTF-8)
8. Enter SOAP Body (Obtained in the previous section using SoapUI )
9. Click Raw Edit above Headers grid. Enter below string
   

   SOAPAction: ""

10. Click Preview to see the data
11. Click OK to save the UI
12. Now you can connect your XML Source to any target such as OLEDB Destination (SQL Server Table)

Here is the example screenshot …

Get data from WorkDay Web Service (Call XML SOAP API in SSIS)

Workday API Pagination – Retrieve more than 100 rows

By default Workday API will return only 100 records unless you specify Page / Count in Response_Filter . Regardless it will only send you 999 maximum rows. To fetch more records after first page you have to setup pagination in XML Source as below. You will need latest version to use this feature (version 2.7.8 or higher).

Workday SOAP API Pagination Settings in SSIS XML Source

And one more thing you have change in Body as below. Pass Response_Filter parameter as below. See how we used <%page%> placeholder. You can use tool like fiddler to see multiple response at runtime. Run package and check fiddler.

Workday API Pagination – Supply Response_Filter (Page Size and Current Page Number)

Sample SOAP Request for Workday API Call

Here is sample request. You have to fill out placeholders (e.g. xxxxxxxx ) when you submit it

There are two ways to pass userid/password inside SOAP Header. WSS PlainText Password or Password Digest. If you using second method to hash password then use Tool like SoapUI.

<env:Envelope
    xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
    <env:Header>
        <wd:Workday_Common_Header xmlns:wd="urn:com.workday/bsvc">
            <wd:Include_Reference_Descriptors_In_Response>true</wd:Include_Reference_Descriptors_In_Response>
        </wd:Workday_Common_Header>
    </env:Header>
    <env:Body>
        <wd:Put_Job_Profile_Request xmlns:wd="urn:com.workday/bsvc" wd:Add_Only="false" wd:version="v26.0">
            <wd:Job_Profile_Reference>
                <wd:ID wd:type="Job_Profile_ID">{2}</wd:ID>
            </wd:Job_Profile_Reference>
            <wd:Job_Profile_Data>
                <wd:Job_Code>xxxxxxxx</wd:Job_Code>
                <wd:Effective_Date>xxxxxxxx</wd:Effective_Date>
                <wd:Job_Classification_Data wd:Delete="false">
                    <wd:Job_Classification_Reference>
                        <wd:ID wd:type="Job_Classification_Reference_ID">xxxxxxxx</wd:ID>
                    </wd:Job_Classification_Reference>
                </wd:Job_Classification_Data>
            </wd:Job_Profile_Data>
        </wd:Put_Job_Profile_Request>
    </env:Body>
</env:Envelope>

Video Tutorial (See Part#4) – Create XML and POST data to SOAP Web Service or REST API

Now lets look at examples of creating XML from multiple data sources and POST XML request to SOAP Web Service URL or any other XML Based REST API URL. This video has 5 parts to cover full length tutorial but if you want to fast forward to see XML Generator Transform and Web API Destination then see part#4.

Manually Passing WSS Credentials (UserID / Password)

When you use HTTP connection using SOAP Setting you dont have to worry about generating WSS Security Header. But for some reason you have to do that manually not to worry. See below section to add UserID / Password Header manually. Again Skip this section if you are using SSIS HTTP Connection along with SOAP WSS Option.

To generate correct Credentials Header in SOAP Request use tool like SoapUI. Check this article to learn more how to generate WSS Security header.

1. Assuming that you already gone through this article to learn how to use SoapUI.
2. Open SoapUI. Click on the request you want to generate SOAP Request Code.
3. Enter username and password in request properties grid
4. Now right click in the XML Request Pane and click “Add WSS UserName Token”
   

   SoapUI Example – Adding credentials to Workday API call.

5. Select Password Method (e.g. PasswordText or PasswordDigest)
   

   SoapUI- Using PasswordDigest Method to insert userid / password inside Workday SOAP Request Body

6. Once you done. You may see new tags inserted under <soapenv:header> tag (For example see below section). You can now use the SOAP Request as your Body in your SSIS XML Source, REST API Task or Web API Destination.

Passing WSS Credentials in SOAP Header – PasswordText Method

If you select PasswordText Method then your SOAP code may look like below.

<soapenv:Envelope xmlns:bsvc="urn:com.workday/bsvc" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:Header>
      <wsse:Security soapenv:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
         <wsse:UsernameToken wsu:Id="UsernameToken-BF23D830F28697AA1614674076904673">
            <wsse:Username>myuser1</wsse:Username>
            <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">mypass123</wsse:Password>
         </wsse:UsernameToken>
      </wsse:Security>
   </soapenv:Header>
   <soapenv:Body>
      <bsvc:Get_Account_Sets_without_Dependencies_Request>
         <!--You have a CHOICE of the next 2 items at this level-->
      </bsvc:Get_Account_Sets_without_Dependencies_Request>
   </soapenv:Body>
</soapenv:Envelope>

Passing WSS Credentials in SOAP Header – PasswordDigest Method

If you select PasswordDigest Method then your SOAP code may look like below. This is for example purpose to give you idea. Technically hardcoding Hash which is signed by Creation Timestamp is useless. Always use SSIS HTTP connection Manager if you like to use this method.

<soapenv:Envelope xmlns:bsvc="urn:com.workday/bsvc" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:Header>
      <wsse:Security soapenv:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
         <wsse:UsernameToken wsu:Id="UsernameToken-BF23D830F28697AA1614674078408854">
            <wsse:Username>myuser1</wsse:Username>
            <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">ycSUpSwmaNimO+QojCVwjlUDL+E=</wsse:Password>
            <wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">tPkEttb09CF39xIW2X/P3Q==</wsse:Nonce>
            <wsu:Created>2016-07-01T21:17:20.885Z</wsu:Created>
         </wsse:UsernameToken>
      </wsse:Security>
   </soapenv:Header>
   <soapenv:Body>
      <bsvc:Get_Account_Sets_without_Dependencies_Request>
         <!--You have a CHOICE of the next 2 items at this level-->
      </bsvc:Get_Account_Sets_without_Dependencies_Request>
   </soapenv:Body>
</soapenv:Envelope>

Other Useful connectors for SOAP/XML and JSON REST API

SSIS XML Source (FILE, SOAP, REST API Connector)
SSIS JSON Source (File, REST Connector)

Here are few more useful links on how to Call SOAP API in general:

Calling SOAP Web Service in SSIS (XML Source)

How to get data from Google AdWords using SSIS

Also check this link to learn how to hand craft SOAP request to call Workday API.

Workday XML SOAP API Reference:

Workday REST API reference link

Conclusion

In this article we have learned how to load data from SQL Server to Workday using SSIS ( drag and drop approach without coding). Combination of few ZappySys Components (e.g. Web API Destination, XML Generator ) makes it super simple to call any Web API to load data from one system to API endpoint. Download SSIS PowerPack to try many other automation scenarios not discussed in this article.

The post How to get data from Workday in SSIS using SOAP/REST API appeared first on ZappySys Blog.