Introduction
BIML, or Business Intelligence Markup Language, is a nice technology that lets you generate SSIS packages using a bit of scripting and parameters. Check for more info if you are new to it:
In this article, we will cover how to create a BIML custom task or component. For this purpose, we will use the ZappySys PowerPack Task or Data Flow component as an example. At the end of this article you will find sample BIML files you can download and fiddle around.
Use BIML to create SSIS packages that use PowerPack tasks or components
BIML Elements
BIML uses CustomTask, CustomComponent, and CustomSsisConnection elements to create custom SSIS connectors. In each of them, you will need to specify CreationName or a similar attribute/element, which defines what kind of component you want to create. E.g., for LoggingTask it would look like this:
|
1 |
ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack120, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502 |
You will find the connector name in DTSX. For Tasks, you have to specify the CreationName attribute; for components, it is UserComponentTypeName. For the Rest API Task, it looks similar to this:
|
1 2 3 |
<DTS:Executable DTS:refId="Package\Sequence Container\ZS REST API Task 1.EventHandlers[OnError]\ZS REST API Task" DTS:CreationName="ZappySys.PowerPack.Task.RestApiTask.RestApiTask, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502" |
For a different SQL Server SSIS package version, you will need to use a different DLL name, e.g.:
- SQL Server 2008: ZappySys.PowerPack100
- SQL Server 2012: ZappySys.PowerPack110
- SQL Server 2014: ZappySys.PowerPack120
- SQL Server 2016: ZappySys.PowerPack130
- SQL Server 2017: ZappySys.PowerPack140
So, to generate a SQL Server 2017 SSIS package, the custom connector creation name becomes like this:
|
1 |
ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack140, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502 |
Create a BIML custom task with CustomTask
You will need to use the CustomTask element to create PowerPack Tasks in your Control Flow. So open your DTSX where you use a PowerPack Task, find out the CreationName and then copy child element in <DTS:ObjectData> element. (e.g., LoggingTask). You will need to escape it and use it in your BIML as well.
DTSX snippet
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
... <DTS:Executable DTS:refId="Package\ZS Logging Task" DTS:CreationName="ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502" ... <DTS:Variables /> <DTS:ObjectData> <LoggingTask Message="Hello from {{System::PackageName}}. I started on {{System::StartTime,yyy-MM-dd hh:mm:ss tt}}" LogVariableName="" MessageType="Information" LogMode="ExecutionLog" LogFilePath="" LogFileAppend="True" LogDateTime="True" DateTimeFormat="HH:mm:ss" LogLevel="Normal" PrefixTimestamp="False" /> </DTS:ObjectData> </DTS:Executable> ... |
BIML snippet
A Logging Task under the Tasks element will look like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
... <CustomTask Name="Logging Task" CreationName="ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack120, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502" TaskContact="ZappySys"> <ObjectData> <LoggingTask Message="Hello from {{System::PackageName}}. I started on {{System::StartTime,yyy-MM-dd hh:mm:ss tt}}" LogVariableName="" MessageType="Information" LogMode="ExecutionLog" LogFilePath="" LogFileAppend="True" LogDateTime="True" DateTimeFormat="HH:mm:ss" LogLevel="Normal" PrefixTimestamp="False" /> </ObjectData> </CustomTask> ... |
Create BIML custom components using CustomComponent
Use the CustomComponent BIML element to generate these components in DataFlow:
- Sources
- Transformations
- Destinations
If you need a connection, then use the CustomSsisConnection element.
Again, you will have to copy the UserComponentTypeName property value and use it in BIML to initialize a component. For a custom connection, it is a bit different: just use the CreationName attribute value from DTSX in the BIML element. Also, don’t forget to copy property values from DTSX and mirror them in BIML.
Below are DTSX and BIML snippets showing a Data Flow that contains a JSON Source using an HTTP Connection and is connected to a Trash Destination.
DTSX snippet
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
... <DTS:ConnectionManagers> <DTS:ConnectionManager DTS:refId="Package.ConnectionManagers[ZS-HTTP Connection]" DTS:CreationName="ZS-HTTP" DTS:DTSID="{E57D23E9-E50A-4C4A-BB19-B91E776887CF}" DTS:ObjectName="ZS-HTTP Connection"> <DTS:ObjectData> <ConnectionManager ConnectionString="Url=https://httpbin.org/anything;CredentialType=Notset;UserName=;Timeout=100000;UseProxy=False;ProxyUrl=;UseProxyCreds=False;ProxyUserName=;DoNotUseDefaultProxy=False;UseClientCertificate=False;CertificatePath=;CertificateStorageType=CurrentUser;CertificateStoreName=My;CertificateThumbprint=;RetryMode=None;RetryCountMax=3;RetryWaitTimeMs=2000;RetryMultiplyWaitTime=False;RetryMessageList=;RetryStatusCodeList=" /> ... <DTS:Executable DTS:refId="Package\Dataflow" DTS:CreationName="SSIS.Pipeline.3" DTS:DTSID="{C59866C4-9732-4B3A-8F80-7784E52C63F5}" DTS:ExecutableType="SSIS.Pipeline.3" DTS:LocaleID="-1" DTS:ObjectName="Dataflow" DTS:TaskContact="Performs high-performance data extraction, transformation and loading;Microsoft Corporation; Microsoft SQL Server v10; (C) 2007 Microsoft Corporation; All Rights Reserved;http://www.microsoft.com/sql/support/default.asp;1"> <DTS:ObjectData> <pipeline BLOBTempStoragePath="" bufferTempStoragePath="" version="1"> <components> <component refId="Package\Dataflow\JsonSource" componentClassID="DTS.ManagedComponentWrapper.3" name="JsonSource" usesDispositions="true" version="25"> <properties> <property dataType="System.Int32" name="AccessMode" typeConverter="ZappySys.Common.Ssis.FileValueAccessMode, ZappySys.Common, Version=1.0.2017.10830, Culture=neutral, PublicKeyToken=1903265a93eae502">2</property> <property dataType="System.Boolean" name="UseConnection">true</property> <property dataType="System.String" name="UserComponentTypeName">ZappySys.PowerPack.Adapter.JsonSource.JsonSource, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502</property> ... </properties> <connections> <connection refId="Package\Dataflow\JsonSource.Connections[HTTP 1]" connectionManagerID="Package.ConnectionManagers[ZS-HTTP Connection]" connectionManagerRefId="Package.ConnectionManagers[ZS-HTTP Connection]" name="HTTP 1" /> </connections> <outputs> <output refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput]" errorRowDisposition="FailComponent" name="JsonSourceOutput" truncationRowDisposition="FailComponent"> <outputColumns> <outputColumn refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].Columns[Url]" dataType="wstr" externalMetadataColumnId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].ExternalColumns[url]" length="50" lineageId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].Columns[Url]" name="Url" /> </outputColumns> <externalMetadataColumns isUsed="True"> <externalMetadataColumn refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].ExternalColumns[url]" dataType="wstr" length="50" name="url" /> </externalMetadataColumns> </output> ... </outputs> </component> <component refId="Package\Dataflow\TrashDestination" componentClassID="DTS.ManagedComponentWrapper.3" name="TrashDestination" version="2"> <properties> <property dataType="System.String" name="UserComponentTypeName">ZappySys.PowerPack.Adapter.TrashDestination.TrashDestination, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502</property> ... </properties> <inputs> <input refId="Package\Dataflow\TrashDestination.Inputs[TrashInput]" name="TrashInput"> <externalMetadataColumns /> </input> </inputs> </component> </components> <paths> <path refId="Package\Dataflow.Paths[TrashInput]" endId="Package\Dataflow\TrashDestination.Inputs[TrashInput]" name="TrashInput" startId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput]" /> </paths> ... |
BIML snippet
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
... <Connections> <CustomSsisConnection Name="ZS-HTTP Connection" CreationName="ZS-HTTP" ObjectData="<ConnectionManager ConnectionString="Url=https://httpbin.org/anything;CredentialType=Notset;UserName=;Timeout=100000;UseProxy=False;ProxyUrl=;UseProxyCreds=False;ProxyUserName=;DoNotUseDefaultProxy=False;UseClientCertificate=False;CertificatePath=;CertificateStorageType=CurrentUser;CertificateStoreName=My;CertificateThumbprint=;RetryMode=None;RetryCountMax=3;RetryWaitTimeMs=2000;RetryMultiplyWaitTime=False;RetryMessageList=;RetryStatusCodeList=" />"/> </Connections> <Packages> <Package Name="DataFlow with Connection Sample" ConstraintMode="Linear"> <Tasks> <Dataflow Name="Dataflow"> <Transformations> <CustomComponent Name="JsonSource" UsesDispositions="true" ComponentTypeName="ZappySys.PowerPack.Adapter.JsonSource.JsonSource, ZappySys.PowerPack130, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502"> <CustomProperties> <CustomProperty Name="AccessMode" DataType="Int32">2</CustomProperty> <CustomProperty Name="UseConnection" DataType="Boolean">true</CustomProperty> </CustomProperties> <OutputPaths> <OutputPath Name="JsonSourceOutput" ErrorRowDisposition="FailComponent" TruncationRowDisposition="FailComponent"> <ExternalColumns> <ExternalColumn Name="url" DataType="String" Length="50" /> </ExternalColumns> <OutputColumns> <OutputColumn Name="Url" ExternalMetadataColumnName="url" DataType="String" Length="50" /> </OutputColumns> </OutputPath> <OutputPath Name="Json Error Output" IsErrorOutput="true"> <OutputColumns> <OutputColumn Name="Url" /> </OutputColumns> </OutputPath> </OutputPaths> <Connections> <Connection Name="HTTP 1" ConnectionName="ZS-HTTP Connection" /> </Connections> </CustomComponent> <CustomComponent Name="TrashDestination" ComponentTypeName="ZappySys.PowerPack.Adapter.TrashDestination.TrashDestination, ZappySys.PowerPack130, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502"> <InputPaths> <InputPath OutputPathName="JsonSource.JsonSourceOutput" Identifier="TrashInput" SsisName="TrashInput"/> </InputPaths> </CustomComponent> </Transformations> ... |
Resources
- BIML documentation
- CustomTask documentation
- CustomComponent documentation
- CustomSsisConnection documentation
Download sample BIMLs
- Sample BIML with PowerPack DataFlow Components
- Sample BIML with PowerPack DataFlow Components with Connection
- Sample BIML with PowerPack Task



