Sending an email with HTML can be difficult if you want to be stylish. In this article, we will show you some examples with CSS that will help you send better emails. We will use the HTML Email Task. You can check the article here

First, drag and drop the HTML Email Task. Next, let’s work on formatting and combining rows. This will help create more flexible tables within the email. After making these changes, you can use the preview button to check the result.

Note: Make sure the <style> is the first thing you use in the body of the email.

<style>
    table {
border-collapse: collapse;
width: 100%;
border: 1px solid green;
}

    th,
    td {
border: 1px solid blue;
padding: 5px;
}

    th {
background-color: orange;
}

    td.resultado {
font-weight: bold;
}
</style>

<table border="1">
    <tr>
        <th>Header 1</th>
        <th>Header 2</th>
    </tr>
    <tr>
        <td rowspan="2">Combined Cell</td>
        <td>Row 1, Cell 2</td>
    </tr>
    <tr>
        <td>Row 2, Cell 2</td>
    </tr>
</table>

Example for the tables

In the following example, we’ll demonstrate how to modify the email’s body and text, including changing colors, size, and font styles for specific text elements.

<style>
body {
font-family: Arial, sans-serif; /* Change font */
background-color: #bdb5f0; /* Change background color */
}

h1 {
color: #5bb258; /* Change color for 'h1' elements*/
}

p {
color: #da0e0e; /* Change color and line height for 'p' elements*/
line-height: 1.5;
}

.highlight {
background-color: yellow; /* highlight a section */
}

#important {
font-weight: bold; /* bold only id = important*/
}
a.visited-link {
color: blue;
/* initial color */
text-decoration: none;
/* removes decoration */
}

/* clicked url */
a.visited-link:visited {
color: red;
/* it will change the color from the URL */
}
</style>

<h1>HTML with CSS Examples</h1>
<p>This is a simple HTML document with some CSS styling examples.</p>
<p>This paragraph has a <span class="highlight">highlighted background</span>.</p>
<p id="important">This paragraph is important.</p>
<a href="https://zappysys.com/products/ssis-powerpack/" class="visited-link">Zappysys SSIS</a>
<a href="https://zappysys.com/products/odbc-powerpack/" class="visited-link">Zappysys ODBC</a>

Example with font, color and background

After implementing these changes, this is how your email will appear. Please note that this is just an example, and you have various options to further customize your email.

Conclusion

In conclusion, CSS can transform your emails into more visually appealing and effective communication tools. For more information on using CSS in HTML files, you can explore this resource here. With CSS, you can create even more impressive email examples

The post Some tips for the SSIS Send HTML Email Task appeared first on ZappySys Blog.

SSIS PowerPack offers versatile ways to make your data integration dynamic through the use of expressions and variables. Often, you’ll encounter scenarios where you need to consume data from dynamic sources, such as URLs, headers, or request bodies. In this article, we’ll explore different techniques for achieving dynamic data manipulation using expressions and variables.

Making some common expressions for SSIS

In SSIS PowerPack, you have access to a wide range of expressions that can be applied in various contexts. Here’s a list of some common SSIS expressions that you can leverage:

Concatenation: Combining multiple strings or variables into one. For example, to create a file path: "C:\\Folder\\" + @[User::FileName]

Date and Time Functions: Functions like GETDATE()

Conditional Expressions: The ternary operator or the IF() function can be used to implement conditional logic. For example, (@[User::number] > 70) ? "Approve" : "Fail"

String Functions: Functions like LEN(), SUBSTRING(), UPPER(), LOWER(), and TRIM() are often used to manipulate string data.

Mathematical Expressions: Like Adding two or more numeric variables@[User::number1]+ @[User::number2]

NULL Handling: Replace NULL with a default value ISNULL(@[User::NullableField]) ? "N/A" : @[User::NullableField]

Body, URL, and Header Expressions

SSIS PowerPack supports the use of expressions with variables in the body, URL, and headers. This flexibility allows for easy editing and dynamic value assignment. Variable placeholders, which are directly visible, can be used to inject dynamic values. Take a look at the screenshot below to understand how to use variable placeholders, which are replaced by the actual values of SSIS variables at runtime through expressions:

Some examples of how expressions with variables can be used in components.

JSON Destination

In scenarios where you need to send data to a JSON destination, expressions with variables can be highly beneficial.

Example for REST API Task

The REST API task is another area where dynamic data manipulation is crucial. Expressions and variables can be applied effectively in this context.

Example for Source Components

JSON source example: you can employ expressions and variables to ensure your data retrieval is dynamic and flexible.

Data Flow Expressions and Control Flow

To further enhance your dynamic data handling capabilities, you can use expressions in both the data flow and control flow of SSIS. Here are some articles that delve into the specifics of using expressions in these contexts:

How to parameterize SSIS Task Property using Expression to make it dynamic

How to set SSIS Data Flow component property using expression and variable

Conclusion

By harnessing expressions and variables in SSIS PowerPack, you can create dynamic and adaptable data integration solutions tailored to your specific needs.

The post Common SSIS expressions example in our tool appeared first on ZappySys Blog.

Sometimes Data Flow source does not contain an ID column, or a number indicating a row number, which you need in further steps. To solve this, you may use a free ZappySys PowerPack component Template Transform. So let’s take a look at how to accomplish that.

Tip

Add Template Transform

Go to the Data Flow, drag and drop Template Transform from SSIS toolbox, and connect it right after the source, which does not provide ID/row number:

Configure Template Transform

Template Transform will provide the row number we want. So configure it like this: in the text area enter <<FUN_SEQUENCE>> and optionally you can give this column a name, e.g. RowNumber.

Configure Template Transform to get row numbers for each row in SSIS

By default Template Transform’s output type is DT_NTEXT. So if you wish to change the type to, for example, integer, use the Derived Column connector and CAST function to change it, e.g.:

(DT_I4)(DT_WSTR,100)RowNumber

Derived column added to convert DT_NTEXT column to an integer type.

Result

In the data viewer you would see that you have a column with a different row number in each row:

Results: How to add row numbers to rows in SSIS data flow

The post How to add row numbers in SSIS data flow appeared first on ZappySys Blog.

In this post, we will see how to create SSIS Package Programmatically to use ZappySys SSIS PowerPack. This blog post assumes you have basic knowledge of Programming language like C# and you have installed Visual Studio Express or a similar product (VS Code not supported).

Add SSIS Dll References

The very first thing is to create a C# Project (i.e. Console or WinForm). Add SSIS dll reference.

You will need the following references in your project. The below example is for SSIS 2017 (Version 14.x.x)

• For SSIS 2012 change to Version=11.0.0.0
• For SSIS 2014 change to Version=12.0.0.0
• For SSIS 2016 change to Version=13.0.0.0
• For SSIS 2017 change to Version=14.0.0.0
• For SSIS 2019 change to Version=15.0.0.0
• And may be future versions same way ….

<Reference Include="Microsoft.SqlServer.Dts.Design"/>
<Reference Include="Microsoft.SqlServer.DTSPipelineWrap" />
<Reference Include="Microsoft.SQLServer.DTSRuntimeWrap" />
<Reference Include="Microsoft.SQLServer.ManagedDTS" />
<Reference Include="Microsoft.SqlServer.PipelineHost" />

Step-By-Step Example (Create SSIS Package Programmatically – C#)

Now let’s see each piece to create SSIS Package in code and later we will show the full code.

Import Namespaces

First step is at the top you need to import some namespaces.

using Microsoft.SqlServer.Dts.Runtime;
using Microsoft.SqlServer.Dts.Pipeline;
//using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
//using Microsoft.SqlServer.Dts.Runtime.Wrapper;

Create SSIS Package

Here how you can create SSIS Package instance in code

var p = new Package();

Add SSIS Connection

Now lets add connection manager to our package instance. To add connection you will need two pieces. Creation Name (i.e. ZS-SFTP) and ConnectionString. You can find creation name from exsing Dtsx file by opening in text editor.

var conn1=p.Connections.Add("ZS-SFTP");
conn1.Name = "MySftpConnection";
conn1.ConnectionString= "Protocol=Sftp;ServerName=127.0.0.1;Port=0;LogonType=Normal;UserName=zstest;Password=zstest123";

How to find CreationName of SSIS Connection manager

To find the creation name of your Connection Type open an existing DTSX file where you had similar connection type in the package. Locate node like below and check CrewationName. It can be a friendly name or can be fully qualified assembly name. Both ways is fine but a friendly name is easy to remember.

<DTS:ConnectionManagers>
  <DTS:ConnectionManager
    DTS:refId="Package.ConnectionManagers[RedshiftConn1]"
    DTS:CreationName="ZS-POSTGRESQL"
    .........
    .........

How to find ConnectionString

Now another thing you need to know is the exact connectionstring. You can go to SSIS Designer and right-click on Connection Icon and check Properties Grid. It will show you ConnectionString or you can use the above technique > Open DTSX file in notepad and find connectinstring there.

ConnectionString doesnt include any passwords / secrets. So manually append those properties in your connectionstring when you set in code.

Add SSIS Task

Now let’s add task in our SSIS Package instance. To create SSIS Task programmatically in code, you need to know the exact type name (i.e. creation name). As we discussed earlier check your dtsx file to know the creation name for the task you like to create. In the below code we used a friendly name which is not supported in SSIS 2012 so for that you must use full name. For SSIS 2014 or higher you can use friendly names.

You can also find it in file like this (example of SSIS 2017 and 2019 folders)
C:\Program Files (x86)\Microsoft SQL Server\140\DTS\UpgradeMappings\ZappySys.PowerPack140.Extensions.xml
C:\Program Files (x86)\Microsoft SQL Server\150\DTS\UpgradeMappings\ZappySys.PowerPack150.Extensions.xml

//var SSIS_VERSION = "110"; //110=SQL2012, 120=SQL2014, 130=SQL2016, 140=SQL2017, 150=SQL2019
//For SSIS 2012
var SFTP_TASK_2012 = "ZappySys.PowerPack.Task.SecureFtpTask.SecureFtpTask, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502";
//For SSIS 2014 or higher
//Goto folder like this and find zappysys extension file 
//(e.g. 140) => C:\Program Files (x86)\Microsoft SQL Server\140\DTS\UpgradeMappings\ZappySys.PowerPack140.Extensions.xml
//Find Task creation name there
var SFTP_TASK = "ZappySys.SecureFtpTask";

var tsk1 = p.Executables.Add(SFTP_TASK) as TaskHost;
tsk1.Properties["SourceStorageConnection"].SetValue(tsk1, "MySftpConnection");
tsk1.Properties["StorageAction"].SetValue(tsk1, 10); //DownloadFileFromFtp [10] find numeric value for this enum from help file> properties https://zappysys.com/onlinehelp/ssis-powerpack/scr/ssis-sftp-ftp-ftps-task.htm
tsk1.Properties["SourcePathValue"].SetValue(tsk1, "subfolder/localsubfile1.csv");
tsk1.Properties["TargetPathValue"].SetValue(tsk1, @"c:\temp\");
tsk1.Properties["FileOverwriteAction"].SetValue(tsk1, 2); //OverwriteAll [2]

Execute SSIS Package

Now let’s execute the package using the below line

var status =p.Execute();

Check Execution Errors

At the end you need to check execution errors if any.

if (status != DTSExecResult.Success)
{
	string errs = p.Errors.Cast<DtsError>().Aggregate("",
           (current, err) => current + string.Format("Source={0}, Err={1}", err.Source, err.Description));
	throw new Exception("Package status was >> " + status + " >> " + errs);
}

Full Code

Here is the full code

using System;
using System.Linq;
using System.Runtime.InteropServices;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime;
/*
For SSIS 2012 - Compile with .net 4.0
For SSIS 2014-2017 - Compile with .net 4.5 
For SSIS 2019 - Compile with .net 4.6.2
 */

//current one is SSIS 2017 (140 dlls)
/*
 * References needed for SSIS 2017 , for 2019 change 14.0.0.0 to 15.0.0.0
 * 
    <Reference Include="Microsoft.SqlServer.DtsMsg, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL" />
    <Reference Include="Microsoft.SqlServer.Dts.Design, Version=14.0.0.0, PublicKeyToken=89845dcd8080cc91" />
    <Reference Include="Microsoft.SqlServer.DTSPipelineWrap, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL" />
    <Reference Include="Microsoft.SQLServer.DTSRuntimeWrap, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91" />
    <Reference Include="Microsoft.SQLServer.ManagedDTS, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL" />
    <Reference Include="Microsoft.SqlServer.PipelineHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL" /> 
 */
namespace SSISCodeDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            
            //Create_SSIS_Controlflow_Task_Demo();
            Create_SSIS_Dataflow_Source_Component_Demo();

            //Create_SSIS_Connection_Demo();
            //Create_SSIS_Variable_Demo();
            //Create_SSIS_Connect_Tasks_Demo();

        }

        private static void Create_SSIS_Controlflow_Task_Demo()
        {
            var p = new Microsoft.SqlServer.Dts.Runtime.Package();

            var conn1 = p.Connections.Add("ZS-SFTP");
            conn1.Name = "MySftpConnection";
            conn1.ConnectionString = "Protocol=Sftp;ServerName=127.0.0.1;Port=0;LogonType=Normal;UserName=zstest;Password=zstest123";

            //var SSIS_VERSION = "110"; //110=SQL2012, 120=SQL2014, 130=SQL2016, 140=SQL2017, 150=SQL2019
            //For SSIS 2012
            var SFTP_TASK_2012 = "ZappySys.PowerPack.Task.SecureFtpTask.SecureFtpTask, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502";
            //For SSIS 2014 or higher
            //Goto folder like this and find zappysys extension file (e.g. 140) => C:\Program Files (x86)\Microsoft SQL Server\140\DTS\UpgradeMappings\ZappySys.PowerPack140.Extensions.xml
            //Find Task creation name there
            var SFTP_TASK = "ZappySys.SecureFtpTask";

            var tsk1 = p.Executables.Add(SFTP_TASK) as TaskHost;
            tsk1.Name = "SFTP_Task1";
            tsk1.Properties["SourceStorageConnection"].SetValue(tsk1, "MySftpConnection");
            tsk1.Properties["StorageAction"].SetValue(tsk1, 10); //DownloadFileFromFtp [10] find numeric value for this enum from help file> properties https://zappysys.com/onlinehelp/ssis-powerpack/scr/ssis-sftp-ftp-ftps-task.htm
            tsk1.Properties["SourcePathValue"].SetValue(tsk1, "subfolder/localsubfile1.csv");
            tsk1.Properties["TargetPathValue"].SetValue(tsk1, @"c:\temp\");
            tsk1.Properties["FileOverwriteAction"].SetValue(tsk1, 2); //OverwriteAll [2]


            /*
             Go to package file (*.dtsx) > Open in Notepad > and findout which properties set. Do same way (ignore some default proprties)
             Some enum properties needs numeric values > Check Task / Component help file to findout number for that

      <DTS:ObjectData>
        <SecureFtpTask
          StorageAction="DeleteFtpFile"
          FileOverwriteAction="OverwriteAll"
          TreatRegXForFullPath="False"
          ContinueOnError="False"
...................
...................
...................            
            PrefixTimestamp="False" />
      </DTS:ObjectData>
    </DTS:Executable>             
             */

            //Application.

            /* use below code to save to disk file 
            var app = new Microsoft.SqlServer.Dts.Runtime.Application();
            var filepath = @"c:\temp\demo.dtsx";
            app.SaveToXml(filepath, p, null);

            --load from disk to memory
            p = app.LoadPackage(filepath, null);

            */
            var status = p.Execute();

            if (status != DTSExecResult.Success)
            {
                var errs = "";
                foreach (var e in p.Errors)
                {
                    errs = errs + string.Format("Source={0}, Err={1}", e.Source, e.Description) + "\r\n";
                }

                throw new Exception("Package status was >> " + status + " >> " + errs);
            }

            Console.WriteLine("Execution Completed... Press any key to exit!!!");
            Console.ReadKey();
        }
        private static void Create_SSIS_Dataflow_Source_Component_Demo()
        {
            var p = new Microsoft.SqlServer.Dts.Runtime.Package();


            /*============ STEP-1: Create Dataflow Task =========== */

            var tsk1 = p.Executables.Add("STOCK:PipelineTask") as TaskHost;
            var dataFlowTask = tsk1.InnerObject as MainPipe;

            /*============ STEP-2: Create Connection =========== */
            var conn1 = p.Connections.Add("ZS-HTTP");
            conn1.Name = "MyConnection";
            conn1.ConnectionString = "Url=https://httpbin.org/get;CredentialType=Basic;UserName=test;Password=test123";


            /*============ STEP-3: Create Source =========== */
            var sourceComponent = dataFlowTask.ComponentMetaDataCollection.New();

            try
            {
                sourceComponent.ComponentClassID = "ZappySys.JsonSource";
                var sourceInstance = sourceComponent.Instantiate();
                sourceInstance.ProvideComponentProperties();

                sourceComponent.Name = "MySource";
                /**** Special Case for JSON Source - if user wants connection to use then we create Runtime connection. For all other components this is always created when ProvideComponentProperties() is called*/
                if (sourceComponent.RuntimeConnectionCollection.Count <= 0)
                {
                    var dtsRuntimeConnection = sourceComponent.RuntimeConnectionCollection.New();
                    dtsRuntimeConnection.Name = "Connection"; 
                    //dtsRuntimeConnection.Description = "This can be ZS-HTTP, ZS-OAUTH or ZS-SALESFORCE";
                }
                sourceComponent.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.GetExtendedInterface(conn1);
                sourceComponent.RuntimeConnectionCollection[0].ConnectionManagerID = conn1.ID;

                sourceInstance.SetComponentProperty("UseConnection", true);//JsonSource specific -- for all other components connection always used
                sourceInstance.SetComponentProperty("DirectPath", "https://httpbin.org/basic-auth/test/test123");
                //sourceInstance.SetComponentProperty("Filter", "$.value[*]");

                //make connection active before next step
                sourceInstance.AcquireConnections(null);

                //after connection + props set ... call ReInitialize meta... it creates columns / output / input etc
                sourceInstance.ReinitializeMetaData(); 
            }
            catch (COMException ex)
            {
                throw new Exception("Error Happened ==> " + sourceComponent.GetErrorDescription(ex.ErrorCode));
            }




            /*============ STEP-4: Create Target (without Reinitialize) =========== */

            var targetComponent = dataFlowTask.ComponentMetaDataCollection.New();
            CManagedComponentWrapper targetInstance=null;
            try
            {
                targetComponent.ComponentClassID = "ZappySys.TrashDestination";
                targetInstance = targetComponent.Instantiate();
                targetInstance.ProvideComponentProperties();

                targetComponent.Name = "MyDestination";

                targetInstance.SetComponentProperty("SaveDataToFile", true);
                targetInstance.SetComponentProperty("FilePath", @"c:\temp\dump.txt");
                targetInstance.SetComponentProperty("OverwriteFileIfExists", true);

                //--- Trash destination doesnt have connection but incase destination uses connection... set below way
                //if you have connection then set this way
                //var conn2 = p.Connections.Add("....."); //Creation name i.e. ZS-HTTP
                //conn2.Name = "MyConnection2";
                //conn3.ConnectionString = "Url=https://httpbin.org/get;CredentialType=Basic;UserName=test;Password=test123";
                //if (targetComponent.RuntimeConnectionCollection.Count <= 0)
                //{
                //    var dtsRuntimeConnection = targetComponent.RuntimeConnectionCollection.New();
                //    dtsRuntimeConnection.Name = "Connection"; 
                //}
                //sourceComponent.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.GetExtendedInterface(conn2);
                //sourceComponent.RuntimeConnectionCollection[0].ConnectionManagerID = conn2.ID;
                //make connection active before next step
                //targetInstance.AcquireConnections(null);
            }
            catch (COMException ex)
            {
                throw new Exception("Error Happened ==> " + targetComponent.GetErrorDescription(ex.ErrorCode));
            }

            /*============ STEP-5: Connect Source to Target =========== */

            //before we call reintializemetadata for dest .. lets attach src and dest so .. dest can use virtual inputs if needed
            var output = sourceComponent.OutputCollection[0]; //to make it simple lets pick first output but possible component can have more outputs
            var input = targetComponent.InputCollection[0];

            dataFlowTask.PathCollection.New().AttachPathAndPropagateNotifications(output, input);

            /*============ STEP-6: Configure Target =========== */
            try
            {
                //after connection + props set and source to target attached... call ReInitialize meta... it creates columns / output / input etc
                targetInstance.ReinitializeMetaData(); 
            }
            catch (COMException ex)
            {
                throw new Exception("Error Happened ==> " + targetComponent.GetErrorDescription(ex.ErrorCode));
            }


            /*============ STEP-6: Save Package =========== */

            //* use below code to save to disk file 
            var app = new Microsoft.SqlServer.Dts.Runtime.Application();
            var filepath = @"c:\temp\demo.dtsx";
            app.SaveToXml(filepath, p, null);

            /*============ STEP-7: Load Package =========== */
            //--load from disk to memory
            p = app.LoadPackage(filepath, null);

            //*/

            /*============ STEP-8: Execute Package =========== */
            var status = p.Execute();

            /*============ STEP-9: Print execution status / log =========== */
            if (status != DTSExecResult.Success)
            {
                var errs = "";
                foreach (var e in p.Errors)
                {
                    errs = errs + string.Format("Source={0}, Err={1}", e.Source, e.Description) + "\r\n";
                }

                throw new Exception("Package status was >> " + status + " >> " + errs);
            }

            Console.WriteLine("Execution Completed... Press any key to exit!!!");
            Console.ReadKey();
        }

        private static void Create_SSIS_Variable_Demo()
        {
            var p = new Microsoft.SqlServer.Dts.Runtime.Package();
            
            p.Variables.Add("MyVariableName1", false, "User", "Some value");
        }

        private static void Create_SSIS_Connection_Demo()
        {
            var p = new Microsoft.SqlServer.Dts.Runtime.Package();

            var conn1 = p.Connections.Add("ZS-SFTP");
            conn1.Name = "MySftpConnection";
            conn1.ConnectionString = "Protocol=Sftp;ServerName=127.0.0.1;Port=0;LogonType=Normal;UserName=zstest;Password=zstest123";

        }

        private static void Create_SSIS_Connect_Tasks_Demo()
        {
            var p = new Microsoft.SqlServer.Dts.Runtime.Package();

            var SFTP_TASK = "ZappySys.SecureFtpTask";

            var tsk1 = p.Executables.Add(SFTP_TASK) as TaskHost;
            tsk1.Name = "SFTP_Task1";
            /*set some props - see Create_SSIS_Controlflow_Task_Demo */

            var tsk2 = p.Executables.Add(SFTP_TASK) as TaskHost;
            tsk2.Name = "SFTP_Task2";
            /*set some props - see Create_SSIS_Controlflow_Task_Demo */
            
            //add link between task1-->task2
            var pc = p.PrecedenceConstraints.Add(tsk1, tsk2);
            pc.EvalOp = DTSPrecedenceEvalOp.Constraint;
            pc.Value = DTSExecResult.Success;
            pc.LogicalAnd = true; //only useful if multiple inputs coming and you want all previous steps to be Green vs Any One Green .. if its any use LogicalAnd=False

            //If you need to connect with expression use below
            //pc.EvalOp = DTSPrecedenceEvalOp.ExpressionAndConstraint;
            //pc.Expression = "@SomeVariable == true";
        }

       

     
    }
}

The post How to create SSIS Package Programmatically (Add / Execute Tasks) appeared first on ZappySys Blog.

In the previous article, we learned how to copy settings from curl to ZappySys products, and in this article, we will continue on how to migrate Postman API call to SSIS / ODBC PowerPack. Postman is a very popular client tool to test API calls. Many API providers document how to call their API using Postman. In this article we will focus on how to use the same settings in ZappySys Products to implement API integration in SSIS or other apps like Power BI, Informatica, Excel, Tableau, SSRS… many ODBC apps.

When you are connecting to an API you may have examples of how to do that in Postman. If you know how to do it in Postman, you are one step away from making it work in SSIS PowerPack and ODBC PowerPack (JSON, XML, CSV Drivers, and other). Most of the examples will show how to do that specifically using ODBC PowerPack and JSON Driver, but since ODBC PowerPack UI is very similar to the connectors of SSIS PowerPack, it will be quite clear where to use it in SSIS PowerPack as well. In the examples where UI differs significantly, both – ODBC and SSIS PowerPack – examples will be presented.

Prerequisites

In this article we assume you meet these requirements:

When migrating Postman API call to SSIS PowerPack

1. Postman installed.
2. SSIS designer installed. Sometimes it is referred to as BIDS or SSDT (download it from Microsoft site).
3. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
4. ZappySys SSIS PowerPack is installed (download it).

When migrating Postman API call to to ODBC PowerPack

1. Postman installed.
2. ZappySys ODBC PowerPack installed (download it).

The Basics

This section shows how to migrate simple options such as URL, HTTP Request Method, and HTTP Headers.

How to configure the URL and HTTP Request Method

Migrate Postman API call to SSIS / ODBC (API URL setting to ODBC Driver (GET / POST) )

How to configure HTTP Headers

Migrate Postman HTTP Headers setting to ODBC Driver

Posting data

In this section, you will find how to post data in ZappySys products. You will learn how to post different formats of data, upload files and send multi-part data.

How to configure POST raw body

Migrate Postman Body / Content-Type setting to ODBC Driver

How to configure POST JSON body

Migrate Postman Body / Content-Type setting to ODBC Driver

Use this table to determine what kind of Body Content Type to configure:

How to configure POST form/x-www-form-urlencoded data

Migrate Postman Body (x-www-form-urlencoded) setting to ODBC Driver

How to configure multipart form POST data

Migrate Postman Multi-Part File Upload setting to ODBC Driver (Upload File)

Authentication

This section will cover all common authentication methods, starting from using API Key to OAuth 2.0.

How to authenticate using API Key in HTTP headers

Authentication – Migrate Postman API Key (via header) setting to ODBC Driver

How to authenticate using API Key as a parameter in URL

Authentication – Migrate Postman API Key (via URL query parameter) setting to ODBC Driver

How to authenticate using Bearer Token

Authentication – Migrate Postman Bearer Token setting to SSIS / ODBC Driver

How to authenticate using Basic Authentication

In ODBC PowerPack

Authentication – Migrate Postman Basic Auth setting to SSIS / ODBC Driver (Login using Userid / password)

In SSIS PowerPack

In SSIS PowerPack use the same configuration as above, just create ZS-HTTP Connection Manager using instructions below:

Authentication – Migrate Postman Basic Auth setting to SSIS JSON / XML Source (Login using Userid / password)

How to authenticate using OAuth

In ODBC PowerPack

1. Configure main options:
   

   Authentication – Migrate Postman OAuth setting to SSIS / ODBC Drivers ( JSON / XML / CSV API) – Client ID / Secret / Scope / Auth and Token Url (Client Credential Grant Type, Password Grant Type, Code Authorization Grant Type)

2. Configure Callback/Return Url:
   

   Authentication – Migrate Postman OAuth setting to SSIS / ODBC Drivers ( Callback / Redirect URL )

If you are using a known, public API service, you may try locating it in the OAuth Providers list, which will save you from configuring Authorization and Access Token URLs:

For simplicity and if your company policy allows it or at least for testing purposes, you can use a Default OAuth Application created by ZappySys:

In SSIS PowerPack

In SSIS PowerPack use the same configuration as above, just create ZS-OAUTH Connection Manager using instructions below:

Authentication – Migrate Postman OAuth setting to SSIS JSON / XML Source

How to authenticate using AWS Signature

In ODBC PowerPack

Authentication – Migrate Postman AWS API call setting to SSIS / ODBC Drivers ( JSON / XML / CSV API call)

In SSIS PowerPack

In SSIS PowerPack use the same configuration as above, just create ZS-OAUTH Connection Manager using instructions below:

How to authenticate using NTLM Authentication

In ODBC PowerPack

In SSIS PowerPack

In SSIS PowerPack use the same configuration as above, just create ZS-HTTP Connection Manager using instructions below:

Conclusion

This article was quite self-explanatory, but if you have any questions or you have an authentication method that is not listed here and you don’t know how to proceed, please, contact us and we will try to help you: https://zappysys.com/support/.

The post How to migrate Postman API call to SSIS / ODBC PowerPack appeared first on ZappySys Blog.

Sometimes, we need to validate the input columns to check for extra or missing columns in SSIS. ZappySys includes some pretty nice tools, but some of them cannot check the required values. To fix that problem, you can complement these tools with our Regular Expression Parser Task.

We will use Regular Expressions to validate the columns. For example, we want to check that the file has 3 columns with specific names, if it contains extra columns or there are missing some columns, we want to send an error. We will show you how to do this.

We will be using this ZappySys SSIS PowerPack component to make things work:

SSIS Regex Task

Creating a CSV file

1. To start, we will create a plain text or CSV file with some columns. Create a file named file.txt or any name of your preference.
2. Add data similar to the following:

   name,lastname,email
   Clint,Eastwood,ceastwood@gmail.com
   Cameron,Diaz,cdiaz@gmail.com

ZS Regular Expression Parser to detect missing columns in SSIS

1. First, SSDT opens an SSIS project.
2. Now, it is time to use the ZS Regular Expression Parser.
3. Secondly, drag and drop the ZS Regular Expression in the design pane:
   

   SSIS Regular Expression parser

4. In AccessMode, use Connection and create a new connection.
5. Select the file you created before. Click OK to save the configuration.
   

   Regular Expressions Parser Task – Create a connection to a local file

6. Add the following expression to the Task and test it:
   

   ^name,lastname\r\n

7. In the previous scenario, the expression will fail because it is missing the email.
   

   Regular Expressions Parser Task – detect missing columns example fails

8. Finally, the following Expression

   ^name,lastname,email\r\n

     will pass, because the email is included.

   

Conclusion

In this article, we learned how to validate and check the columns in a file in ZappySys using the ZS Regular Expression Parser. In the example, we created an expression to detect an exact match of columns. It will raise an error if a failure occurs.

If you want to enjoy this and other ZappySys tools, you can download our product here.

The post How to detect missing columns in SSIS appeared first on ZappySys Blog.

In this new article, we will show you how to perform PGP encryption using SSIS (encrypt / decrypt files using public / private key). Our previous article was about SFTP using our SFTP task for SSIS. Now we will show how to encrypt the information.

Requirements

1. First of all, you will need SSDT for Business Intelligence for SSIS projects.
2. Download and install FREE PGP Command line gnupg for Windows.
3. Optionally you can install SSIS PowerPack.

What is PGP / GPG (GnuPG) / OpenPGP ?

PGP stands for Pretty Good Privacy. It was created by Phil Zimmermann. PGP is the most popular email encryption in the world. It is an Encryption that requires a public key, a private key. You can encrypt, decrypt and sign documents, emails. PGP Tools now acquired by Symantec. In sort PGP Tools developed by Phil are not Open Source any more so use other on if you really need FREE one.

GPG (i.e. GnuPG)) stands for GNU Privacy Guard. This software is for multiple platforms like Linux and Windows. In this article, we will focus on Windows because we are using SSIS that is mainly installed in Windows (at least by the moment). This tool is free, so we will focus on this one instead of trying other licensed versions.

OpenPGP is standard around Public / Private Key Cryptography (RFC4880) . Check this page. Both GPG and PGP follow OpenPGP Standard however there will be a case when they are not 100% compatible with each other.

For more information read this link and this link.

Getting started

In this article we will use GPG only which is essentially same features as PGP but GPG is Open Source around OpenPGP Standard and PGP is not open source.

So let’s get started. First of all, we will start with a simple the command line.

GPG command line example

After you install gpg command line we can open it to get started. There are many more examples explained here and here.

1. First of all, check the requirements and open the command line.
2. Secondly, in the command line write gpg –help this will list all the commands available:
   

   gpg help command

3. Also, in order to generate your personal key pair, write the following command:
   

   gpg --gen-key

4. Finally, the command previously provided will ask for a name and email, write an O to press OK and a Passphrase. You will be able to create your key pair with that information.

PGP encryption using SSIS introduction

It is possible to perform PGP encryption using SSIS with the GPG command line which is free. We can use  native execute process task that comes with SSIS. In this article, we will show some commands of gpg.

1. Open a new SSIS Package.
2. Drag Execute Process Task from SSIS Toolbox and double click to configure (see next step).
3. Using SSIS Execute Process Task, we will invoke the gpg commands. There are different ways to invoke the gpg commands. One way is to execute command stored in a .bat file or .cmd file
4. To invoke command from a file create a file called gpg.cmd and type

   gpg --version

      , save file to path like c:\sql\gpg.cmd for this demo.
5. Configure Execute Process Task something like this if you invoke command from batch file:
   

   SSIS invoke gpg

6. Also, note that the windows style is hidden. That will hide the command line when running the package.
7. In addition, the StandardOutputVariable will create an object variable in the output.
8. To verify the output, we will use the ZS Logging task
   

   SSIS output results

9. Click OK to save Execute Process Task UI.
10. Now drag ZS Logging task from toolbox and configure like below. This will Log output captured by gpg command. This step is optional but will help you a lot if you want to know what happened in previous step (e.g. capture error message)
    

    SSIS Logging task

Execute GPG / PGP commands directly in SSIS Execute Process Task (Pass Arguments)

1. In the previous example, we used a cmd file with the commands. Sometimes we need to use SSIS variables and pass information dynamically. In this new example, we will invoke the gpg command line directly and send the arguments in the arguments property like this:

SSIS execute gpg encryption

Generate public key in PGP encryption using SSIS

Now lets look at common task with PGP cryptography which is generate Public / Private Key pair.

1. In order to generate a public key, you need to run the following arguments in SSIS:
   

   SSIS generation of public key

   --armor --output "c:\sql\ssiskey.txt" --export "Name"

2. In addition, the key name is the name of the pair key created and the
   output contains the path of the stored key.

How to encrypt a file in PGP encryption using SSIS

1. First of all, we will encrypt a file named ssiskey.txt:
2. The argument used is the following:
   

   --recipient "Name" --output "c:\sql\ssiskey.txt.gpg" --encrypt "c:\sql\ssiskey.txt"

   

   Encrypt file gpg in SSIS

3. We are creating an encrypted file named ssiskey.txt.pgp

How to decrypt a file in PGP encryption using SSIS

1. In order to start, we will decrypt the file created previously:
   

   Decrypt file using PGP

2. Also, we will use the following argument:
   

   --decrypt-files "c:\sql\ssiskey.txt.gpg"

    

How to import a public key in PGP encryption using SSIS

1. In order to send a message and decrypt it, you need a public key. If you publish a public key, the recipient needs to import the public key. To import a key use the following argument:
   

   gpg --import "c:\sql\publickey.txt".

   You can download a public key here:
   • Public key
2. Secondly, in the Execute Process Task, will look like this:
   

   How to import a public key in SSIS

How to encrypt multiple files in PGP encryption using SSIS

1. If you want to encrypt multiple files, you can use the ZS Advanced File System combined with the Foreach loop container to filter and select the files to encrypt.
2. In addition, the package will look like this:
   

   Send a list of files to the foreach loop

3. Also, in the Advanced File System, we will list the files as ADO.net DataTable:
   

   Store list of files in SSIS

4. In the next step, we will use the Foreach ADO Enumerator in the Foreach Loop container:
   

   Foreach loop from the Advanced Files system

5. Next, we will select the first column (column 0) to get the file path from the list.
   

   Send column 0 that contains the full path

6. Finally, we will use the following expression for the argument property to encrypt all the txt files of the ADO list:
   

   "--recipient \"Daniel Calbimonte\" --output \""+ @[User::filepath]+".gpg\""+" 
   --encrypt \""+@[User::filepath]+"\""

   

   SSIS expression to encrypt multiple files

Troubleshooting PGP encryption using SSIS

1. First of all, if your package works in SSDT or VS and fails in the agent, verify that you are using a proxy account with permissions to run the gpg command line. For more information about SQL Agent proxies, refer to this link.

References

For more information, refer to these links:

• Practical Introduction to GNU Privacy Guard in Windows
• The GNU Privacy Handbook

The post SSIS PGP Encryption / Decryption (Using FREE GPG Tool) appeared first on ZappySys Blog.

In this short article, you will learn how to write Regular expressions in SSIS (i.e. Regex) and what tool to use to test them. You will also find helpful resources on how to write more sophisticated expressions and learn more about them. For demo purposes, we will use FREE SSIS Regex Parser Task to parse and extract the text using Regex.

You can use Regular expressions in several SSIS PowerPack connectors:

• SSIS Regex Parser Task (FREE),
• Azure Blob Storage Task,
• Advanced File System Task,
• Amazon S3 Storage Task and others.

Writing Regular Expressions in SSIS

Based on the SSIS component you use it will include the filename in filtering options or match the text: See below Syntax to write Regex in ZappySys tools. We support additional construct {{X, Y}} at the end of Regex to control two parameters. This additional construct is useful for data extracted from matching regex. If you want to Test patterns and not worry about data extraction then no need to use last {{

Occurance_Index

Group_Index_Or_Name

Syntax:

<your Regular Expression>[{{Occurance_Index|*[,Group_Index_Or_Name]}}]

Where:

Occurance_Index

Group_Index_Or_Name

How to name a group?

E.g. (\w+)@(?<domain>\w+.com)

How to use the group name in the match extract?

E.g. (\w+)@(?<domain>\w+.com){{0,domain}}

Example Input:

Let’s assume we have the following input text. We will test various Expressions.

Customer => AAA
Email => aaa@google.com
Phone => 101-222-3333
========
Customer => BBB
Email => bbb@yahoo.com
Phone => 102-222-3333
========
Customer => CCC
Email => ccc@hotmail.com
Phone => 103-222-3333
========
Customer => DDD
Email => ddd@outlook.com
Phone => 104-222-3333

Sample Regex Expressions

(.*)

More Regular Expression Examples

Regex Examples (Using SSIS Regular Expression Parser Task)

Here is an example how Regex

(.+)_(.+)_(.+){{1,2}}

Using Regular Expressions in SSIS Regex Parser Task (Extract Groups)

Using Groups / Occurrence Index

Some tasks like SSIS Regex Parser Task (FREE) supports extracting value from specific occurrence and specific part of matched pattern using special syntax at the end of your pattern (see below).

Your Regex Pattern Here{{OccuranceIndex,GroupIndex}}

Where :
OccuranceIndex is 0 based (0=extract first occurrence)
GroupIndex is 0 based (0=extract first matching group from pattern. First group is always entire text. )

See above screenshot in previous section for example.

Tools

The best tool we’ve found to write and test Regex is Regex Hero (will require IE with Silverlight if you want to use it online, in the browser):

Another great site for Regex testing is https://regex101.com (Works in any browser unlike previous one)

and few more sites as below

http://www.regexr.com/
http://www.regexlib.com/
http://www.regular-expressions.info/

Resources

Regular Expressions cheat-sheet to hang on the wall

Regular Expressions quick reference

The post Using Regular Expressions in SSIS appeared first on ZappySys Blog.

In this article, we will cover the steps of how to run an SSIS package with sensitive data on SQL Server.

In an SSIS package sensitive data such as tokens and passwords are by default encrypted by SSIS with your Windows account which you use to create a package. So if ProtectionLevel of a package is set to EncryptSensitiveWithUserKey (or DontSaveSensitive), SSIS will fail to decrypt tokens/passwords, when you run it from another machine using another Windows account. To circumvent this, when you are creating an SSIS package that uses authentication components (e.g. an OAuth Connection Manager or an HTTP Connection Manager with credentials, or any other component that uses sensitive data), consider using parameters/variables to pass tokens/passwords to the package. In this way, you won’t face authentication-related errors when the package is deployed to a production server. Below you will find out the ways how to parameterize and configure parameters when running an SSIS package in a SQL Job.

Is it necessary to parameterize all the authentication details/credentials?

No. You just need to parameterize those properties that are needed in the process of authentication or making a request. For example, if you are using OAuth Connection Manager, you will need to parameterize RefreshToken, and ClientSecret in the Authorization Code Grant (3-Legged Auth) authentication scenario. If you use Password Grant, you will need to parameterize the Password property as well. You do not need to parameterize AccessToken, because it is generated using a RefreshToken.

But you may consider parameterizing environment-specific properties such as URL, ClientID, UserName, and similar even if they are not encrypted by SSIS. You would need to do that when you work in one environment when developing and testing packages while in another in production, UAT, etc. For example, you would use one OAuth Application while developing but another in production — you would also need to parameterize ClientID and perhaps Authorize and Token URLs too.

How to parameterize an SSIS Connection Manager

Most often sensitive data is used by SSIS Connection Manager, in password, OAuth token, etc. properties. To pass a token or a password dynamically you will have to use a variable/parameter together with expression. 2012 and above versions of Visual Studio or BIDS have a nice way to do it, while in Visual Studio 2008 you will have to set it up manually. Another way is to set sensitive data using SSIS Package Configuration file, which is applicable when you have enabled package deployment model (obsolete method).

Using parameters or variables

Below you will learn how to set a property using a variable/parameter together with an expression. If you are not sure, what property to set, check the documentation of connection managers to find out.

Using a parameter

1. Right click on a Connection Manager and press Parameterize:
   

   Connection Manager parameterization

2. Then select an appropriate property you want to parametrize, e.g.

   Password

   , and give a name to the parameter. For OAuth Connection Manager we will use 

   RefreshToken

   .
   

   Parameterizing OAuth Connection Manager property “RefreshToken”

Using a variable

Below is an example how to set any property of a Data Flow using an expression and a variable. You can do the same thing for a Connection Manager – just create a variable, find a property named

Password

RefreshToken

Using a configuration file (obsolete method)

So if you wanted to parameterize

RefreshToken

<?xml version="1.0"?>
<DTSConfiguration>
    <DTSConfigurationHeading>
        <DTSConfigurationFileInfo GeneratedBy="ZappySys\ZappySys" GeneratedFromPackageName="-Sandbox-" GeneratedFromPackageID="{152417C7-3764-4910-91E3-1D35C8FE89AE}" GeneratedDate="2017-11-03 19:22:13"/>
    </DTSConfigurationHeading>
    <Configuration ConfiguredType="Property" Path="\Package.Connections[OAuth 1].Properties[RefreshToken]" ValueType="String">
        <ConfiguredValue>real-refresh-token-value-goes-here</ConfiguredValue>
    </Configuration>
</DTSConfiguration>

How to pass a variable/parameter to an SSIS Connection Manager

Executing a package from SSIS Catalog

When you are executing a package from SSIS Catalog to set a property you will have to use a parameter.

1. Select SQL Server Agent Job and access its properties. Then select

   SSIS Catalog

    as Package source and select your package:
   

   Running SSIS package from SSIS Catalog

2. In Configuration tab parameter will be already present, so just click on

   [...]

    button and input your parameter value:
   
   

   Configuring parameter for SSIS package which is run from SSIS Catalog

Using a parameter from SSIS Environment

Another way of passing a parameter when using SSIS Catalog is to use Environment parameter.

1. Create an environment if you don’t have one already and access its properties:
   

   Creating and configuring SSIS Environment

2. Click Variables and create a parameter with appropriate type and don’t forget to check

   Sensitive checkbox

   :
   

   Creating a sensitive parameter in SSIS environment

3. Then go to your SSIS Catalog project and click

   Configure

   :
   
4. Click

   References

    and

   Add

    button. Select the environment you had created in first step:
   

   Adding SSIS environment to SSIS Catalog project

5. Then click

   Parameters

    and 

   [...]

     button on the parameter you want to set:
   

   Setting an environment variable for SSIS Catalog project

6. The result should look similar to this:
   

   Result after you configured SSIS Catalog project to use an environment variable

7. Then in SQL Server Agent Job properties,

   Configuration

    tab select the environment you created. Configured parameter will be set automatically:
   

   Using environment variable in SQL Server Agent Job

SSIS Sensitive Data Issue with OAuth Authentication

When working with OAuth-based connectors in SSIS (such as Dynamics CRM, Dataverse, or other Azure AD–secured APIs), it’s common to store credentials like Client ID and Client Secret inside SSIS variables or parameters.

However, incorrect handling of sensitive data in SSIS can cause SQL Server Agent job failures, even when the package runs successfully in Visual Studio (SSDT).

SSIS Package Works in Visual Studio but Fails in SQL Server Agent

A common scenario occurs when an SSIS package executes without issues in Visual Studio, but fails when scheduled or executed via SQL Server Agent.

In one real-world case, a Dynamics CRM connector was configured using OAuth authentication, and the following values were stored in SSISDB parameters:

• Application (Client) ID
• Client Secret

When both parameters were marked as Sensitive, the package failed during SQL Agent execution, despite working correctly at design time.

Why Marking SSIS Variables as Sensitive Causes Job Failures

This behavior is not specific to ZappySys; it is a well-known SSIS limitation:

• SSIS encrypts values marked as Sensitive
• Decryption depends on the execution context (user vs SQL Agent service account)
• Over-encrypting non-secret values can prevent successful runtime decryption

This issue frequently appears when non-secret OAuth parameters are incorrectly marked as sensitive.

How to Store OAuth Client ID and Client Secret in SSIS Securely

For OAuth-based authentication in SSIS (Dynamics CRM, Dataverse, Azure AD APIs, etc.), use the following recommended configuration:

Only true secrets should be marked as Sensitive. The Client ID is not confidential and does not require encryption.

Best Practice: Use SSISDB Sensitive Parameters Instead of SSIS Variables

To avoid SQL Agent execution issues, always use SSIS Catalog (SSISDB) parameters for sensitive values:

1. Deploy the package to SSISDB
2. Convert connection properties into Project or Package Parameters
3. Mark only sensitive values (such as Client Secret) as Sensitive
4. Assign values through SSISDB Environments

SSISDB encrypts sensitive parameters securely and allows SQL Server Agent to decrypt them correctly at runtime.

For detailed steps, refer to the official documentation:

SSIS Parameterization Guide for ZappySys Connectors

Common Mistakes When Handling Sensitive Data in SSIS

• Marking Client ID or Tenant ID as Sensitive
• Using SSIS variables marked as Sensitive instead of SSISDB parameters
• Assuming all OAuth fields require encryption

SSIS Sensitive Data and SQL Agent Execution: Key Takeaways

• SQL Server Agent failures often occur due to incorrect Sensitive parameter configuration
• OAuth connectors typically require only the Client Secret to be protected
• Proper SSISDB parameterization ensures reliable scheduled execution

Pro Tip: If your SSIS package runs in Visual Studio but fails in SQL Server Agent,
review which parameters are marked as Sensitive before troubleshooting the connector itself.

Executing a package from a File system / SSIS Package Store / SQL Server

When you are executing a package from a File system / SSIS Package Store or SQL Server you will have to use a variable (parameters won’t work).

Using variables in SQL Server Agent Job configuration

You can pass a variable in SQL Server Agent Job properties in

Set values

Using a variable in SQL Server Agent Job

Using a configuration file in SQL Server Agent Job configuration

You can pass configuration file in SQL Server Agent Job properties in

Configurations

Using a package configuration file in SQL Server Agent Job

Resources

• Microsoft article about Package Protection Level.
• Using sensitive parameters (SSIS series).

The post How to parameterize properties in SSIS package and configure them in a SQL job appeared first on ZappySys Blog.

BIML, or Business Intelligence Markup Language, is a nice technology that lets you generate SSIS packages using a bit of scripting and parameters.  Check for more info if you are new to it:

• https://en.wikipedia.org/wiki/Business_Intelligence_Markup_Language
• http://bimlscript.com/

In this article, we will cover how to create a BIML custom task or component. For this purpose, we will use the ZappySys PowerPack Task or Data Flow component as an example. At the end of this article you will find sample BIML files you can download and fiddle around.

Use BIML to create SSIS packages that use PowerPack tasks or components

BIML Elements

BIML uses

CustomTask

CustomComponent

CustomSsisConnection

CreationName

ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack120, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502

You will find the connector name in DTSX. For Tasks, you have to specify the CreationName attribute; for components, it is UserComponentTypeName. For the Rest API Task, it looks similar to this:

<DTS:Executable
 DTS:refId="Package\Sequence Container\ZS REST API Task 1.EventHandlers[OnError]\ZS REST API Task"
 DTS:CreationName="ZappySys.PowerPack.Task.RestApiTask.RestApiTask, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502"

For a different SQL Server SSIS package version, you will need to use a different DLL name, e.g.:

• SQL Server 2008: ZappySys.PowerPack100
• SQL Server 2012: ZappySys.PowerPack110
• SQL Server 2014: ZappySys.PowerPack120
• SQL Server 2016: ZappySys.PowerPack130
• SQL Server 2017: ZappySys.PowerPack140

So, to generate a SQL Server 2017 SSIS package, the custom connector creation name becomes like this:

ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack140, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502

Create a BIML custom task with CustomTask

You will need to use the

CustomTask

CreationName

<DTS:ObjectData>

DTSX snippet

... 
<DTS:Executable
    DTS:refId="Package\ZS Logging Task"
    DTS:CreationName="ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502"
    ...
    <DTS:Variables />
    <DTS:ObjectData>
    <LoggingTask
        Message="Hello from {{System::PackageName}}. I started on {{System::StartTime,yyy-MM-dd hh:mm:ss tt}}"
        LogVariableName=""
        MessageType="Information"
        LogMode="ExecutionLog"
        LogFilePath=""
        LogFileAppend="True"
        LogDateTime="True"
        DateTimeFormat="HH:mm:ss"
        LogLevel="Normal"
        PrefixTimestamp="False" />
    </DTS:ObjectData>
</DTS:Executable>
...

BIML snippet

A Logging Task under the Tasks element will look like this:

...
<CustomTask Name="Logging Task" CreationName="ZappySys.PowerPack.Task.LoggingTask.LoggingTask, ZappySys.PowerPack120, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502" TaskContact="ZappySys">
    <ObjectData>
        &lt;LoggingTask
        Message="Hello from {{System::PackageName}}. I started on {{System::StartTime,yyy-MM-dd hh:mm:ss tt}}"
        LogVariableName=""
        MessageType="Information"
        LogMode="ExecutionLog"
        LogFilePath=""
        LogFileAppend="True"
        LogDateTime="True"
        DateTimeFormat="HH:mm:ss"
        LogLevel="Normal"
        PrefixTimestamp="False" /&gt;
 </ObjectData>
</CustomTask>
...

<LoggingTask>

<ObjectData>

Create BIML custom components using CustomComponent

Use the

CustomComponent

• Sources
• Transformations
• Destinations

If you need a connection, then use the CustomSsisConnection element.

Again, you will have to copy the

UserComponentTypeName

Below are DTSX and BIML snippets showing a Data Flow that contains a JSON Source using an HTTP Connection and is connected to a Trash Destination.

DTSX snippet

...
<DTS:ConnectionManagers>
    <DTS:ConnectionManager DTS:refId="Package.ConnectionManagers[ZS-HTTP Connection]" DTS:CreationName="ZS-HTTP" DTS:DTSID="{E57D23E9-E50A-4C4A-BB19-B91E776887CF}" DTS:ObjectName="ZS-HTTP Connection">
        <DTS:ObjectData>
            <ConnectionManager ConnectionString="Url=https://httpbin.org/anything;CredentialType=Notset;UserName=;Timeout=100000;UseProxy=False;ProxyUrl=;UseProxyCreds=False;ProxyUserName=;DoNotUseDefaultProxy=False;UseClientCertificate=False;CertificatePath=;CertificateStorageType=CurrentUser;CertificateStoreName=My;CertificateThumbprint=;RetryMode=None;RetryCountMax=3;RetryWaitTimeMs=2000;RetryMultiplyWaitTime=False;RetryMessageList=;RetryStatusCodeList=" />
...
<DTS:Executable DTS:refId="Package\Dataflow" DTS:CreationName="SSIS.Pipeline.3" DTS:DTSID="{C59866C4-9732-4B3A-8F80-7784E52C63F5}" DTS:ExecutableType="SSIS.Pipeline.3" DTS:LocaleID="-1" DTS:ObjectName="Dataflow" DTS:TaskContact="Performs high-performance data extraction, transformation and loading;Microsoft Corporation; Microsoft SQL Server v10; (C) 2007 Microsoft Corporation; All Rights Reserved;http://www.microsoft.com/sql/support/default.asp;1">
    <DTS:ObjectData>
    <pipeline BLOBTempStoragePath="" bufferTempStoragePath="" version="1">
    <components>
    <component refId="Package\Dataflow\JsonSource" componentClassID="DTS.ManagedComponentWrapper.3" name="JsonSource" usesDispositions="true" version="25">
        <properties>
            <property dataType="System.Int32" name="AccessMode" typeConverter="ZappySys.Common.Ssis.FileValueAccessMode, ZappySys.Common, Version=1.0.2017.10830, Culture=neutral, PublicKeyToken=1903265a93eae502">2</property>
            <property dataType="System.Boolean" name="UseConnection">true</property>
            <property dataType="System.String" name="UserComponentTypeName">ZappySys.PowerPack.Adapter.JsonSource.JsonSource, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502</property>
            ...
        </properties>
        <connections>
            <connection refId="Package\Dataflow\JsonSource.Connections[HTTP 1]" connectionManagerID="Package.ConnectionManagers[ZS-HTTP Connection]" connectionManagerRefId="Package.ConnectionManagers[ZS-HTTP Connection]" name="HTTP 1" />
        </connections>
        <outputs>
            <output refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput]" errorRowDisposition="FailComponent" name="JsonSourceOutput" truncationRowDisposition="FailComponent">
                <outputColumns>
                    <outputColumn refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].Columns[Url]" dataType="wstr" externalMetadataColumnId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].ExternalColumns[url]" length="50" lineageId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].Columns[Url]" name="Url" />
                </outputColumns>
                <externalMetadataColumns isUsed="True">
                    <externalMetadataColumn refId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput].ExternalColumns[url]" dataType="wstr" length="50" name="url" />
                </externalMetadataColumns>
            </output>
            ...
        </outputs>
    </component>
    <component refId="Package\Dataflow\TrashDestination" componentClassID="DTS.ManagedComponentWrapper.3" name="TrashDestination" version="2">
        <properties>
            <property dataType="System.String" name="UserComponentTypeName">ZappySys.PowerPack.Adapter.TrashDestination.TrashDestination, ZappySys.PowerPack110, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502</property>
            ...
        </properties>
        <inputs>
            <input refId="Package\Dataflow\TrashDestination.Inputs[TrashInput]" name="TrashInput">
                <externalMetadataColumns />
            </input>
        </inputs>
    </component>
    </components>
    <paths>
        <path refId="Package\Dataflow.Paths[TrashInput]" endId="Package\Dataflow\TrashDestination.Inputs[TrashInput]" name="TrashInput" startId="Package\Dataflow\JsonSource.Outputs[JsonSourceOutput]" />
    </paths>
...

BIML snippet

...
<Connections>
    <CustomSsisConnection Name="ZS-HTTP Connection" CreationName="ZS-HTTP" ObjectData="&lt;ConnectionManager ConnectionString=&quot;Url=https:&#x2F;&#x2F;httpbin.org&#x2F;anything;CredentialType=Notset;UserName=;Timeout=100000;UseProxy=False;ProxyUrl=;UseProxyCreds=False;ProxyUserName=;DoNotUseDefaultProxy=False;UseClientCertificate=False;CertificatePath=;CertificateStorageType=CurrentUser;CertificateStoreName=My;CertificateThumbprint=;RetryMode=None;RetryCountMax=3;RetryWaitTimeMs=2000;RetryMultiplyWaitTime=False;RetryMessageList=;RetryStatusCodeList=&quot; &#x2F;&gt;"/>
</Connections>
<Packages>
<Package Name="DataFlow with Connection Sample" ConstraintMode="Linear">
<Tasks>
    <Dataflow Name="Dataflow">
    <Transformations>
    <CustomComponent Name="JsonSource" UsesDispositions="true" ComponentTypeName="ZappySys.PowerPack.Adapter.JsonSource.JsonSource, ZappySys.PowerPack130, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502">
        <CustomProperties>
            <CustomProperty Name="AccessMode" DataType="Int32">2</CustomProperty>                              
            <CustomProperty Name="UseConnection" DataType="Boolean">true</CustomProperty>
        </CustomProperties>
        <OutputPaths>
            <OutputPath Name="JsonSourceOutput" ErrorRowDisposition="FailComponent" TruncationRowDisposition="FailComponent">
                <ExternalColumns>
                    <ExternalColumn Name="url" DataType="String" Length="50" />
                </ExternalColumns>                                  
                <OutputColumns>
                    <OutputColumn Name="Url" ExternalMetadataColumnName="url" DataType="String" Length="50" />
                </OutputColumns>
            </OutputPath>
            <OutputPath Name="Json Error Output" IsErrorOutput="true">
            <OutputColumns>
                <OutputColumn Name="Url" />
            </OutputColumns>
            </OutputPath>                                  
        </OutputPaths>
        <Connections>
        <Connection Name="HTTP 1" ConnectionName="ZS-HTTP Connection" />
        </Connections>                             
    </CustomComponent>
    <CustomComponent Name="TrashDestination" ComponentTypeName="ZappySys.PowerPack.Adapter.TrashDestination.TrashDestination, ZappySys.PowerPack130, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1903265a93eae502">
        <InputPaths>
            <InputPath OutputPathName="JsonSource.JsonSourceOutput" Identifier="TrashInput" SsisName="TrashInput"/>
        </InputPaths>
    </CustomComponent>
    </Transformations>
...

Resources

• BIML documentation
• CustomTask documentation
• CustomComponent documentation
• CustomSsisConnection documentation

Download sample BIMLs

• Sample BIML with PowerPack DataFlow Components
• Sample BIML with PowerPack DataFlow Components with Connection
• Sample BIML with PowerPack Task

The post How to create BIML custom task or component appeared first on ZappySys Blog.