powerpack Archives | ZappySys Blog https://zappysys.com/blog/tag/powerpack/ SSIS / ODBC Drivers / API Connectors for JSON, XML, Azure, Amazon AWS, Salesforce, MongoDB and more Fri, 21 Nov 2025 03:37:45 +0000 en-US hourly 1 https://wordpress.org/?v=6.4.4 https://zappysys.com/blog/wp-content/uploads/2023/01/cropped-zappysys-symbol-large-32x32.png powerpack Archives | ZappySys Blog https://zappysys.com/blog/tag/powerpack/ 32 32 How to create BIML custom task or component https://zappysys.com/blog/how-to-use-biml-script-to-create-biml-custom-task-or-component/ Thu, 21 Sep 2017 16:42:01 +0000 http://zappysys.com/blog/?p=2025 Introduction BIML, or Business Intelligence Markup Language, is a nice technology that lets you generate SSIS packages using a bit of scripting and parameters.  Check for more info if you are new to it: https://en.wikipedia.org/wiki/Business_Intelligence_Markup_Language http://bimlscript.com/ In this article, we will cover how to create a BIML custom task or component. For this purpose, we will use […]

The post How to create BIML custom task or component appeared first on ZappySys Blog.

]]> Introduction

BIML, or Business Intelligence Markup Language, is a nice technology that lets you generate SSIS packages using a bit of scripting and parameters.  Check for more info if you are new to it:

In this article, we will cover how to create a BIML custom task or component. For this purpose, we will use the ZappySys PowerPack Task or Data Flow component as an example. At the end of this article you will find sample BIML files you can download and fiddle around.

Use BIML to create SSIS packages that use PowerPack tasks or components

BIML Elements

BIML uses 

CustomTask
, 
CustomComponent
, and 
CustomSsisConnection
elements to create custom SSIS connectors. In each of them, you will need to specify 
CreationName
or a similar attribute/element, which defines what kind of component you want to create. E.g., for LoggingTask it would look like this:

ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack120, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502

You will find the connector name in DTSX. For Tasks, you have to specify the CreationName attribute; for components, it is UserComponentTypeName. For the Rest API Task, it looks similar to this:

<DTS:Executable
 DTS:refId="Package\Sequence Container\ZS REST API Task 1.EventHandlers[OnError]\ZS REST API Task"
 DTS:CreationName="ZappySys.PowerPack.Task.RestApiTask.RestApiTask, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502"

NOTE: If your SSIS package is of a higher version than SQL Server 2012, convert it to SQL Server 2012 first.

For a different SQL Server SSIS package version, you will need to use a different DLL name, e.g.:

  • SQL Server 2008: ZappySys.PowerPack100
  • SQL Server 2012: ZappySys.PowerPack110
  • SQL Server 2014: ZappySys.PowerPack120
  • SQL Server 2016: ZappySys.PowerPack130
  • SQL Server 2017: ZappySys.PowerPack140

So, to generate a SQL Server 2017 SSIS package, the custom connector creation name becomes like this:

ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack140, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502

Create a BIML custom task with CustomTask

You will need to use the 

CustomTask
 element to create PowerPack Tasks in your Control Flow. So open your DTSX where you use a PowerPack Task, find out the 
CreationName
 and then copy child element in 
<DTS:ObjectData>
element. (e.g., LoggingTask). You will need to escape it and use it in your BIML as well.

DTSX snippet

... 
<DTS:Executable
    DTS:refId="Package\ZS Logging Task"
    DTS:CreationName="ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502"
    ...
    <DTS:Variables />
    <DTS:ObjectData>
    <LoggingTask
        Message="Hello from {{System::PackageName}}. I started on {{System::StartTime,yyy-MM-dd hh:mm:ss tt}}"
        LogVariableName=""
        MessageType="Information"
        LogMode="ExecutionLog"
        LogFilePath=""
        LogFileAppend="True"
        LogDateTime="True"
        DateTimeFormat="HH:mm:ss"
        LogLevel="Normal"
        PrefixTimestamp="False" />
    </DTS:ObjectData>
</DTS:Executable>
...

BIML snippet

A Logging Task under the Tasks element will look like this:

...
<CustomTask Name="Logging Task" CreationName="ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack120, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502" TaskContact="ZappySys">
    <ObjectData>
        &lt;LoggingTask
        Message="Hello from {{System::PackageName}}. I started on {{System::StartTime,yyy-MM-dd hh:mm:ss tt}}"
        LogVariableName=""
        MessageType="Information"
        LogMode="ExecutionLog"
        LogFilePath=""
        LogFileAppend="True"
        LogDateTime="True"
        DateTimeFormat="HH:mm:ss"
        LogLevel="Normal"
        PrefixTimestamp="False" /&gt;
 </ObjectData>
</CustomTask>
...

NOTE: Notice the escaped 
<LoggingTask>
 XML element under 
<ObjectData>
 element.

Create BIML custom components using CustomComponent

Use the 

CustomComponent
 BIML element to generate these components in DataFlow:

  • Sources
  • Transformations
  • Destinations

If you need a connection, then use the CustomSsisConnection element.

Again, you will have to copy the 

UserComponentTypeName
property value and use it in BIML to initialize a component. For a custom connection, it is a bit different: just use the CreationName attribute value from DTSX in the BIML element. Also, don’t forget to copy property values from DTSX and mirror them in BIML.

Below are DTSX and BIML snippets showing a Data Flow that contains a JSON Source using an HTTP Connection and is connected to a Trash Destination.

DTSX snippet

...
<DTS:ConnectionManagers>
    <DTS:ConnectionManager DTS:refId="Package.ConnectionManagers[ZS-HTTP Connection]" DTS:CreationName="ZS-HTTP" DTS:DTSID="{E57D23E9-E50A-4C4A-BB19-B91E776887CF}" DTS:ObjectName="ZS-HTTP Connection">
        <DTS:ObjectData>
            <ConnectionManager ConnectionString="Url=https://httpbin.org/anything;CredentialType=Notset;UserName=;Timeout=100000;UseProxy=False;ProxyUrl=;UseProxyCreds=False;ProxyUserName=;DoNotUseDefaultProxy=False;UseClientCertificate=False;CertificatePath=;CertificateStorageType=CurrentUser;CertificateStoreName=My;CertificateThumbprint=;RetryMode=None;RetryCountMax=3;RetryWaitTimeMs=2000;RetryMultiplyWaitTime=False;RetryMessageList=;RetryStatusCodeList=" />
...
<DTS:Executable DTS:refId="Package\Dataflow" DTS:CreationName="SSIS.Pipeline.3" DTS:DTSID="{C59866C4-9732-4B3A-8F80-7784E52C63F5}" DTS:ExecutableType="SSIS.Pipeline.3" DTS:LocaleID="-1" DTS:ObjectName="Dataflow" DTS:TaskContact="Performs high-performance data extraction, transformation and loading;Microsoft Corporation; Microsoft SQL Server v10; (C) 2007 Microsoft Corporation; All Rights Reserved;http://www.microsoft.com/sql/support/default.asp;1">
    <DTS:ObjectData>
    <pipeline BLOBTempStoragePath="" bufferTempStoragePath="" version="1">
    <components>
    <component refId="Package\Dataflow\JsonSource" componentClassID="DTS.ManagedComponentWrapper.3" name="JsonSource" usesDispositions="true" version="25">
        <properties>
            <property dataType="System.Int32" name="AccessMode" typeConverter="ZappySys.Common.Ssis.FileValueAccessMode, ZappySys.Common, Version=1.0.2017.10830, Culture=neutral, PublicKeyToken=1903265a93eae502">2</property>
            <property dataType="System.Boolean" name="UseConnection">true</property>
            <property dataType="System.String" name="UserComponentTypeName">ZappySys.PowerPack.Adapter.JsonSource.JsonSource, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502</property>
            ...
        </properties>
        <connections>
            <connection refId="Package\Dataflow\JsonSource.Connections[HTTP 1]" connectionManagerID="Package.ConnectionManagers[ZS-HTTP Connection]" connectionManagerRefId="Package.ConnectionManagers[ZS-HTTP Connection]" name="HTTP 1" />
        </connections>
        <outputs>
            <output refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput]" errorRowDisposition="FailComponent" name="JsonSourceOutput" truncationRowDisposition="FailComponent">
                <outputColumns>
                    <outputColumn refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].Columns[Url]" dataType="wstr" externalMetadataColumnId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].ExternalColumns[url]" length="50" lineageId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].Columns[Url]" name="Url" />
                </outputColumns>
                <externalMetadataColumns isUsed="True">
                    <externalMetadataColumn refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].ExternalColumns[url]" dataType="wstr" length="50" name="url" />
                </externalMetadataColumns>
            </output>
            ...
        </outputs>
    </component>
    <component refId="Package\Dataflow\TrashDestination" componentClassID="DTS.ManagedComponentWrapper.3" name="TrashDestination" version="2">
        <properties>
            <property dataType="System.String" name="UserComponentTypeName">ZappySys.PowerPack.Adapter.TrashDestination.TrashDestination, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502</property>
            ...
        </properties>
        <inputs>
            <input refId="Package\Dataflow\TrashDestination.Inputs[TrashInput]" name="TrashInput">
                <externalMetadataColumns />
            </input>
        </inputs>
    </component>
    </components>
    <paths>
        <path refId="Package\Dataflow.Paths[TrashInput]" endId="Package\Dataflow\TrashDestination.Inputs[TrashInput]" name="TrashInput" startId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput]" />
    </paths>
...

BIML snippet

...
<Connections>
    <CustomSsisConnection Name="ZS-HTTP Connection" CreationName="ZS-HTTP" ObjectData="&lt;ConnectionManager ConnectionString=&quot;Url=https:&#x2F;&#x2F;httpbin.org&#x2F;anything;CredentialType=Notset;UserName=;Timeout=100000;UseProxy=False;ProxyUrl=;UseProxyCreds=False;ProxyUserName=;DoNotUseDefaultProxy=False;UseClientCertificate=False;CertificatePath=;CertificateStorageType=CurrentUser;CertificateStoreName=My;CertificateThumbprint=;RetryMode=None;RetryCountMax=3;RetryWaitTimeMs=2000;RetryMultiplyWaitTime=False;RetryMessageList=;RetryStatusCodeList=&quot; &#x2F;&gt;"/>
</Connections>
<Packages>
<Package Name="DataFlow with Connection Sample" ConstraintMode="Linear">
<Tasks>
    <Dataflow Name="Dataflow">
    <Transformations>
    <CustomComponent Name="JsonSource" UsesDispositions="true" ComponentTypeName="ZappySys.PowerPack.Adapter.JsonSource.JsonSource, ZappySys.PowerPack130, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502">
        <CustomProperties>
            <CustomProperty Name="AccessMode" DataType="Int32">2</CustomProperty>                              
            <CustomProperty Name="UseConnection" DataType="Boolean">true</CustomProperty>
        </CustomProperties>
        <OutputPaths>
            <OutputPath Name="JsonSourceOutput" ErrorRowDisposition="FailComponent" TruncationRowDisposition="FailComponent">
                <ExternalColumns>
                    <ExternalColumn Name="url" DataType="String" Length="50" />
                </ExternalColumns>                                  
                <OutputColumns>
                    <OutputColumn Name="Url" ExternalMetadataColumnName="url" DataType="String" Length="50" />
                </OutputColumns>
            </OutputPath>
            <OutputPath Name="Json Error Output" IsErrorOutput="true">
            <OutputColumns>
                <OutputColumn Name="Url" />
            </OutputColumns>
            </OutputPath>                                  
        </OutputPaths>
        <Connections>
        <Connection Name="HTTP 1" ConnectionName="ZS-HTTP Connection" />
        </Connections>                             
    </CustomComponent>
    <CustomComponent Name="TrashDestination" ComponentTypeName="ZappySys.PowerPack.Adapter.TrashDestination.TrashDestination, ZappySys.PowerPack130, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502">
        <InputPaths>
            <InputPath OutputPathName="JsonSource.JsonSourceOutput" Identifier="TrashInput" SsisName="TrashInput"/>
        </InputPaths>
    </CustomComponent>
    </Transformations>
...

Resources

Download sample BIMLs

The post How to create BIML custom task or component appeared first on ZappySys Blog.

]]>