ssis http connection Archives | ZappySys Blog

How to create BIML custom task or component

ZappySys — Thu, 21 Sep 2017 16:42:01 +0000

Introduction

BIML, or Business Intelligence Markup Language, is a nice technology that lets you generate SSIS packages using a bit of scripting and parameters. Check for more info if you are new to it:

In this article, we will cover how to create a BIML custom task or component. For this purpose, we will use the ZappySys PowerPack Task or Data Flow component as an example. At the end of this article you will find sample BIML files you can download and fiddle around.

Use BIML to create SSIS packages that use PowerPack tasks or components

BIML Elements

BIML uses

CustomTask

CustomComponent

, and

CustomSsisConnection

elements to create custom SSIS connectors. In each of them, you will need to specify

CreationName

or a similar attribute/element, which defines what kind of component you want to create. E.g., for LoggingTask it would look like this:

ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack120, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502

You will find the connector name in DTSX. For Tasks, you have to specify the CreationName attribute; for components, it is UserComponentTypeName. For the Rest API Task, it looks similar to this:

NOTE: If your SSIS package is of a higher version than SQL Server 2012, convert it to SQL Server 2012 first.
For a different SQL Server SSIS package version, you will need to use a different DLL name, e.g.:

SQL Server 2008: ZappySys.PowerPack100
SQL Server 2012: ZappySys.PowerPack110
SQL Server 2014: ZappySys.PowerPack120
SQL Server 2016: ZappySys.PowerPack130
SQL Server 2017: ZappySys.PowerPack140

So, to generate a SQL Server 2017 SSIS package, the custom connector creation name becomes like this:
ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack140, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502

Create a BIML custom task with CustomTask
You will need to use the
CustomTask
element to create PowerPack Tasks in your Control Flow. So open your DTSX where you use a PowerPack Task, find out the CreationName
and then copy child element in
element. (e.g., LoggingTask). You will need to escape it and use it in your BIML as well.
DTSX snippet
...

...
BIML snippet
A Logging Task under the Tasks element will look like this:
...

...

NOTE: Notice the escaped
XML element under
element.
Create BIML custom components using CustomComponent
Use the
CustomComponent
BIML element to generate these components in DataFlow:

Sources
Transformations
Destinations

If you need a connection, then use the CustomSsisConnection element.
Again, you will have to copy the
UserComponentTypeName
property value and use it in BIML to initialize a component. For a custom connection, it is a bit different: just use the CreationName attribute value from DTSX in the BIML element. Also, don’t forget to copy property values from DTSX and mirror them in BIML.
Below are DTSX and BIML snippets showing a Data Flow that contains a JSON Source using an HTTP Connection and is connected to a Trash Destination.
DTSX snippet
...

...

2
true
ZappySys.PowerPack.Adapter.JsonSource.JsonSource, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502
...

...

ZappySys.PowerPack.Adapter.TrashDestination.TrashDestination, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502
...

...
BIML snippet
...

2
true

...
Resources

BIML documentation
CustomTask documentation
CustomComponent documentation
CustomSsisConnection documentation

Download sample BIMLs

Sample BIML with PowerPack DataFlow Components
Sample BIML with PowerPack DataFlow Components with Connection
Sample BIML with PowerPack Task

The post How to create BIML custom task or component appeared first on ZappySys Blog.

Calling REST API in SSIS with Basic Authentication / Userid-Password (BASE64 Authorization header)

ZappySys — Thu, 27 Aug 2015 20:58:17 +0000

Introduction

In this blog post you will explain how to pass basic credentials (i.e. UserID/Password) along with your web request. You will also learn about setting up Authorization Header for HTTP Web Request in Base64 manually. To learn more about how to consume / call REST API in SSIS check this article.

HTTP Authorization Header basics

As per HTTP Standard you can pass credentials very simple way using basic Authorization header. Below is the sample of Basic Authorization header.

As you can see it consist of HeaderName=Authorization and Value=some base64 encoded string
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
As per standard Base64 encoded string is made up with two elements. your_userid:your_password

NOTE: Base64 is encoding and not encryption method. So never hand over your encoded Authorization header string to anyone. Anyone can decode Base64 encoded string easily by using tools like this

Passing Basic Credentials along with HTTP Web Request

You can pass credentials (i.e. UserID/Password) in two different ways.

Automatic approach – Using ZS-HTTP Connection Manager (BASE64 Encoded Header automatically generated)
Manual approach – Supplying Authorization where UserID/Password encoded as Base64

Automatic approach – Using ZS-HTTP Connection Manager

Now lets look at how to use automatic approach to pass Basic Authorization in following 3 different tasks/components

Passing basic credentials in SSIS REST API Task
Passing basic credentials in SSIS JSON Source
Passing basic credentials in SSIS XML Source

Using SSIS REST API Task (Pass Basic credentials using Connection Manager)

The simplest way to pass credentials is use HTTP Connection Manager. If you are using REST API Task then you can use Url from Connection Access Mode. And on connection manager specify UserId and password to pass Basic Authorization Header

Advantages of this method are..
1) Its very simple
2) Credentials are encrypted in SSIS Package

NOTE: We recommend using https:// Urls whenever possible otherwise your web request can be decoded in plain text by any servers sitting between you and target and possibly they can get your password by decoding base64 header.

Analyze Base64 Encoded HTTP Authorization Header generated by HTTP Connection Manager

You can analyze Webrequest in some 3rd party Free Debugging tools such as Fiddler (Check this article). As you see how its sending Authorization header in Base64 encoded string.

Base64 Encoded HTTP Basic Authorization Header – View in Fiddler

Using SSIS JSON/XML Source (Pass Basic credentials using Connection Manager)

You can use HTTP Connection manager in JSON Source and XML Source too. Only difference is you have to check Use Credentials checkbox and select HTTP connection manager from available connection managers dropdown.

Select AccessMode as “File Path or Web URL”
Enter API URL
Check “Use Credentials” option
Click “New” next to Connection dropdown or select New “ZS-HTTP” connection from dropdown
Configure HTTP Connection as below (i.e. Enter URL, Select Basic Auth, Enter UserID, Password)
Clock OK on HTTP connection manager UI to save and now Test JSON or XML Source UI by clicking Preview (Assuming no other configuration needed)

SSIS JSON Source – Passing Basic Credentials (Supply UserID / Password) using HTTP Connection

Manual approach – Supplying Authorization where UserID/Password encoded as Base64

Our JSON Source, XML Source and REST API Task they all support passing BASE64 Encoded String for Authorization header (see below screenshot). Setting up userid and password in HTTP Connection Manger would do same trick as setting up it manually.

To encode any Header value in BASE64 format you simply have to add BASE64ENC format specifier after variable name.

Example: {{User::varUserAndPassword,BASE64ENC}}

SSIS JSON Source Task – Pass Base64 Encoded HTTP Authorization Header

Reference

Wikipedia Article – HTTP Basic Authorization Header

Download FREE SSIS Components

The post Calling REST API in SSIS with Basic Authentication / Userid-Password (BASE64 Authorization header) appeared first on ZappySys Blog.