In this post, we will show how to save a list of files into a table a table using SSIS. The article will show how to get a list of files and then store the list of files into an Excel file.

Prerequisites

Before we perform the steps listed in this article, you will need to make sure the following prerequisites are met:

1. SSIS designer installed. Sometimes it is referred to as BIDS or SSDT (download it from the Microsoft site).
2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
3. Make sure ZappySys SSIS PowerPack is installed (download it).
4. Optional (If you want to Deploy and Schedule ) – Deploy and Schedule SSIS Packages

Step-by-step process to save the list of files and folders into an article

Save the information in a variable with the Advanced file system task

1. Drag and drop the Advance File System Task from the SSIS toolbox, chose to Get file list as ADO.net Data table and select an object variable type

Getting the file list in a variable

Save the result in a file with Logging task in order to save a list of files into a table

2. The next step is to save the result in a HTML file: select the file destination, uncheck append and prefix datetime. Select the file in log mode and none in message type, and insert the variable in the message as follows:

Save the value from the variable in a file without a prefix

Read the table and save it on a new file in order to save a list of files into a table

3. Now, Drag and Drop SSIS Data Flow Task from SSIS Toolbox

4. Double click on the DataFlow task to see DataFlow designer surface.

5. From the SSIS toolbox drag and drop HTML source and insert the path for the new file, you can preview the file if it was previously created.

Preview the HTML file

7. Then you can use a destination like the Excel destination, JSON destination or CSV destination

Save the path list into an excel file

If everything is OK, the information will be stored in Excel. However, you can select any destination.

Conclusion

In this article, we show how to save a list of files in a table. Basically, we use the Advanced file system to save the list in a file. We use the Logging Task

The post How to save a list of files into a table using SSIS. appeared first on ZappySys Blog.

In this new article, we will show you how to perform PGP encryption using SSIS (encrypt / decrypt files using public / private key). Our previous article was about SFTP using our SFTP task for SSIS. Now we will show how to encrypt the information.

Requirements

1. First of all, you will need SSDT for Business Intelligence for SSIS projects.
2. Download and install FREE PGP Command line gnupg for Windows.
3. Optionally you can install SSIS PowerPack.

What is PGP / GPG (GnuPG) / OpenPGP ?

PGP stands for Pretty Good Privacy. It was created by Phil Zimmermann. PGP is the most popular email encryption in the world. It is an Encryption that requires a public key, a private key. You can encrypt, decrypt and sign documents, emails. PGP Tools now acquired by Symantec. In sort PGP Tools developed by Phil are not Open Source any more so use other on if you really need FREE one.

GPG (i.e. GnuPG)) stands for GNU Privacy Guard. This software is for multiple platforms like Linux and Windows. In this article, we will focus on Windows because we are using SSIS that is mainly installed in Windows (at least by the moment). This tool is free, so we will focus on this one instead of trying other licensed versions.

OpenPGP is standard around Public / Private Key Cryptography (RFC4880) . Check this page. Both GPG and PGP follow OpenPGP Standard however there will be a case when they are not 100% compatible with each other.

For more information read this link and this link.

Getting started

In this article we will use GPG only which is essentially same features as PGP but GPG is Open Source around OpenPGP Standard and PGP is not open source.

So let’s get started. First of all, we will start with a simple the command line.

GPG command line example

After you install gpg command line we can open it to get started. There are many more examples explained here and here.

1. First of all, check the requirements and open the command line.
2. Secondly, in the command line write gpg –help this will list all the commands available:
   

   gpg help command

3. Also, in order to generate your personal key pair, write the following command:
   

   gpg --gen-key

4. Finally, the command previously provided will ask for a name and email, write an O to press OK and a Passphrase. You will be able to create your key pair with that information.

PGP encryption using SSIS introduction

It is possible to perform PGP encryption using SSIS with the GPG command line which is free. We can use  native execute process task that comes with SSIS. In this article, we will show some commands of gpg.

1. Open a new SSIS Package.
2. Drag Execute Process Task from SSIS Toolbox and double click to configure (see next step).
3. Using SSIS Execute Process Task, we will invoke the gpg commands. There are different ways to invoke the gpg commands. One way is to execute command stored in a .bat file or .cmd file
4. To invoke command from a file create a file called gpg.cmd and type

   gpg --version

      , save file to path like c:\sql\gpg.cmd for this demo.
5. Configure Execute Process Task something like this if you invoke command from batch file:
   

   SSIS invoke gpg

6. Also, note that the windows style is hidden. That will hide the command line when running the package.
7. In addition, the StandardOutputVariable will create an object variable in the output.
8. To verify the output, we will use the ZS Logging task
   

   SSIS output results

9. Click OK to save Execute Process Task UI.
10. Now drag ZS Logging task from toolbox and configure like below. This will Log output captured by gpg command. This step is optional but will help you a lot if you want to know what happened in previous step (e.g. capture error message)
    

    SSIS Logging task

Execute GPG / PGP commands directly in SSIS Execute Process Task (Pass Arguments)

1. In the previous example, we used a cmd file with the commands. Sometimes we need to use SSIS variables and pass information dynamically. In this new example, we will invoke the gpg command line directly and send the arguments in the arguments property like this:

SSIS execute gpg encryption

Generate public key in PGP encryption using SSIS

Now lets look at common task with PGP cryptography which is generate Public / Private Key pair.

1. In order to generate a public key, you need to run the following arguments in SSIS:
   

   SSIS generation of public key

   --armor --output "c:\sql\ssiskey.txt" --export "Name"

2. In addition, the key name is the name of the pair key created and the
   output contains the path of the stored key.

How to encrypt a file in PGP encryption using SSIS

1. First of all, we will encrypt a file named ssiskey.txt:
2. The argument used is the following:
   

   --recipient "Name" --output "c:\sql\ssiskey.txt.gpg" --encrypt "c:\sql\ssiskey.txt"

   

   Encrypt file gpg in SSIS

3. We are creating an encrypted file named ssiskey.txt.pgp

How to decrypt a file in PGP encryption using SSIS

1. In order to start, we will decrypt the file created previously:
   

   Decrypt file using PGP

2. Also, we will use the following argument:
   

   --decrypt-files "c:\sql\ssiskey.txt.gpg"

    

How to import a public key in PGP encryption using SSIS

1. In order to send a message and decrypt it, you need a public key. If you publish a public key, the recipient needs to import the public key. To import a key use the following argument:
   

   gpg --import "c:\sql\publickey.txt".

   You can download a public key here:
   • Public key
2. Secondly, in the Execute Process Task, will look like this:
   

   How to import a public key in SSIS

How to encrypt multiple files in PGP encryption using SSIS

1. If you want to encrypt multiple files, you can use the ZS Advanced File System combined with the Foreach loop container to filter and select the files to encrypt.
2. In addition, the package will look like this:
   

   Send a list of files to the foreach loop

3. Also, in the Advanced File System, we will list the files as ADO.net DataTable:
   

   Store list of files in SSIS

4. In the next step, we will use the Foreach ADO Enumerator in the Foreach Loop container:
   

   Foreach loop from the Advanced Files system

5. Next, we will select the first column (column 0) to get the file path from the list.
   

   Send column 0 that contains the full path

6. Finally, we will use the following expression for the argument property to encrypt all the txt files of the ADO list:
   

   "--recipient \"Daniel Calbimonte\" --output \""+ @[User::filepath]+".gpg\""+" 
   --encrypt \""+@[User::filepath]+"\""

   

   SSIS expression to encrypt multiple files

Troubleshooting PGP encryption using SSIS

1. First of all, if your package works in SSDT or VS and fails in the agent, verify that you are using a proxy account with permissions to run the gpg command line. For more information about SQL Agent proxies, refer to this link.

References

For more information, refer to these links:

• Practical Introduction to GNU Privacy Guard in Windows
• The GNU Privacy Handbook

The post SSIS PGP Encryption / Decryption (Using FREE GPG Tool) appeared first on ZappySys Blog.

This tutorial explains how to call SSRS reports from SSIS, export them to files (PDF, Excel, Word, HTML, and more), email them as attachments or as HTML in the message body, pass dynamic parameters, and automate delivery without relying only on fixed SSRS subscriptions. You will use the ZappySys Reporting Services Task (SSRS Report Generator) from SSIS PowerPack.

Also, make sure to have an SSRS report deployed. For more information about creating reports and deploying, refer to these links:

• Create a Basic Table Report (SSRS Tutorial)
• Publishing Reports to a Report Server

Why Automate SSRS Reports from SSIS?

SSRS subscriptions are useful for fixed schedules, but they can be limiting when you need event-driven report delivery. For example, you may want to send a batch of reports after an ETL load, data validation, billing process, or other workflow is finished. Standard SSRS subscriptions usually rely on hardcoded schedules and predefined parameters, so they are not always easy to invoke dynamically from a process or adjust at runtime. ZappySys SSRS Reporting Services Task helps solve this by letting SSIS control when reports run, which parameters are used, where files are saved, and how reports are emailed.

What You Will Learn

• How to call and export SSRS reports from an SSIS package.
• How to pass dynamic SSRS report parameters from SSIS variables.
• How to send SSRS reports by email as attachments.
• How to send a server SSRS report as an HTML email body (Email Settings tab checkbox or legacy two-step flow).
• How to automate report file names, paths, and post-processing steps.

SSRS Subscriptions vs. SSIS Automation

Tasks Used in This Tutorial

The examples below use these ZappySys SSIS PowerPack components:

	SSIS Report Generator Task (Export SSRS Reports)
	Advanced SSIS File System Task
	Send HTML Email Task

Video: Reporting Services Task in SSIS

This walkthrough demonstrates the SSRS Report Generator / Reporting Services Task in SSIS:

Export SSRS Reports from SSIS

First you will export a server-hosted SSRS report to a file. The task supports Word (.docx, .doc), Excel (.xlsx, .xls), PDF, TIFF, CSV, XML, and HTML (server reports).

Export SSRS Report to File (Word, Excel, PDF, CSV, HTML, XML)

1. First of all drag and drop the ZS Reporting Task that is included with our ZappySys SSIS PowerPack:
   

   SSRS task in SSIS

2. Secondly, in general tab, make sure that the Export server report is enabled.
3. Additionally, specify the report service URL. If you are not sure what your report service URL is, see Microsoft guidance for publishing reports to a report server (linked in the prerequisites section above).
4. In addition, in report path, select the report that you want to export to a file.
5. Also, check the output as a file option:
   

   Export report to file

6. Additionally, in the export settings tab, go to Save as and specify the path and extension of the file to export. You can export to Excel, Word, PDF, Tiff, CSV and XML:
   

   Export SSRS report using SSIS to Excel, Word, PDF

7. Finally, run the package and the file will be created:
   

   SSRS report exported to docx

Date and Time in the Export File Name

When you export SSRS reports from SSIS, you may want to add the current date or time to the report file name. For example, you can save the report as Filename20180323.xlsx, where 20180323 means March 23, 2018.

There are two ways to save a report file with date/time in the path:

1. New method: Use placeholder functions directly in the SSRS task Save As path under the Export Settings tab.
2. Legacy method: Export the report as a file first, and then rename the file using ZappySys Advanced File System Task.

The following legacy example shows how to add a date suffix after the report is exported.

1. First of all, we will use the ZappySys Advanced File System Task:
2. Secondly, we will use the option Rename files and rename using the fun_getdate function like this:
   Filename<<yyyy-MM-dd,FUN_GETDATE>>.xlsx
3. Also, you can watch all the available examples of functions by pressing the Expressions (x) button:
   

   Special SSIS functions

4. Finally, press insert variable and function examples and you will be able to see all the examples of special functions to simplify your life:
   

   Samples of functions

Send SSRS Report as an Email Attachment

The Reporting Services Task can render the report and send it by email. You need SMTP settings (server, credentials, port). Follow these steps:

1. First of all, in the general tab, check the email report file and uncheck the Output as file option:
   

   Output SSRS report in SSIS as attachment

2. Also, on the Email Settings tab, choose New under SMTP connections to create a connection. Provide the SMTP server name, credentials, and port. You can find sample SMTP hosts and ports in this third-party list.
3. In addition, you have an option to specify the sender, receiver, subject, and body. It is possible to write the body in HTML format. You can also specify SSIS variables. The following HTML code shows how to write in bold,
   italic, with bullets and more:
   

   <h1>This is an example using headers</h1>
   <p><strong>This is an example to write using bold</strong> <strong><br />
   </strong><em>This is an example to write using italic <br /></em>
   This is an example using underline<br /><br />
   </p>
   <ol>
     <li>This is an example with lists</li>
     <li>Second item in the list</li>
     <li>Third of the list</li>
   </ol>
   <ul style="list-style-type:circle">
     <li>This is an example with bullets</li>
     <li>Bullet sample </li>
     <li>Working with bullets</li>
   </ul>

    
   

   Dynamic HTML in SSIS

4. Additionally, in Visual Studio go to the menu and SSIS, Variables and create two variables that will be used in the report named Title and ReportName of type string and specify values for them:
   

   Using SSIS variables in emails

5. Finally, execute the Package and check your email received:
   

   Email received SSIS Html format

Send SSRS Report as HTML in the Email Body (Not as an Attachment)

By default, Output as email usually sends the rendered report as an attachment. To put the report inside the email as HTML, use one of the methods below.

Important: HTML export applies to server reports only. Local RDL mode does not support exporting as HTML; use a report deployed to SSRS.

Method 1: Email Settings Checkbox (Recommended, Newer Builds)

If your SSIS PowerPack build is from after May 6, 2026, use this single-step option.

In the Reporting Services Task, open the Email Settings tab and enable Use report output as Email body (*.htm / *.html export formats only). Configure SMTP and recipients as usual, and set the export format to HTML (.htm or .html). The task embeds the rendered report HTML in the email body—no separate Send HTML Email Task is required.

Email Settings: use report output as HTML email body (single-step)

Sample HTML Email (Embedded body with images)

Sending SSRS Report as Email embedded body (With image support)

Method 2: SSRS Task Plus Send HTML Email Task (Legacy)

Use this on builds on or before May 6, 2026, or when you want full control over the email HTML in Send HTML Email Task.

Summary of the two-step flow:

1. Reporting Services Task exports a server report to an HTML file (for example C:\temp\sales-order.html).
2. Send HTML Email Task reads that file into the message body using FUN_FILE_READ.

Legacy two-step flow: SSRS Task plus Send HTML Email Task

1. In the ZappySys Reporting Services Task, use Export server report, not local report mode.
2. Choose Output as file and export the SSRS report as an HTML file to the local system, for example C:\temp\sales-order.html.
3. Add ZappySys Send HTML Email Task in the next step of your SSIS package.
4. In the email body, use the FUN_FILE_READ placeholder function to read the HTML file exported in the previous step and paste the entire HTML content into the email body:

<<C:\temp\sales-order.html,FUN_FILE_READ>>

Pass SSRS Report Parameters from SSIS

Many SSRS reports define parameters (for example date range or customer ID). The Reporting Services Task can map SSIS variables to those SSRS parameters so values are chosen when the package runs. You need a report that includes at least one parameter to try this. To learn how to author parameters in SSRS, see Microsoft’s report parameters documentation.

1. First of all, you will need an SSRS Report with a parameter:
   

   Reporting Service Parameter

2. Secondly, in SSDT or Visual Studio, in the menu, go to SSIS and variables. Create an SSIS variable:
   

   Variables in SSIS

3.  Finally, in Reporting Services Task in Visual Studio, go to Report Parameters and map the SSRS parameter with the SSIS variable:
   

   Map SSIS variable with SSRS parameters

Data-Driven Export for Multiple SSRS Reports

Data-driven mode lets you drive many report exports from a SQL Server configuration table: different reports, paths, email options, and parameters per row. The task can generate the sample table schema for you.

1. First of all, select the data-driving report export using config table.
2.  Also, you will need to specify the connection to the SQL Server
   database.
3. In addition, go to config table, press new to create a new table:
   

   SSRS Data-driven in SSIS

4. Additionally, ZappySys allows creating a sample table that can help you to understand how to configure local reports, Calling SSRS Reports in SSIS and save to files, specify paths, ports, email options and more:
   

   CREATE TABLE [{{SchemaName}}].[{{TableName}}](
   	Id int identity(1,1) NOT NULL  PRIMARY KEY, 
   	GroupName varchar(100) NULL,  /* Used to filter reports */
   	ReportType varchar(10) NOT NULL, /*Local or Server*/
   	ReportServiceUrl  varchar(500) null, 
   	ReportPath varchar(500) NOT NULL, 
   	IsEnabled bit NOT NULL default(1),
   	ExportAsFile bit NOT NULL default(1),
   	ExportAsEmail bit NOT NULL default(0),
   	Parameters varchar(max) NULL, /* ParameterName1;IsMultiValue;Value1|....ParameterNameN;IsMultiValue;ValueN */
   	ExportPath varchar(500) NULL,
   	AddFileSuffix bit NULL,
   	ExportFileSuffix varchar(500) NULL, /* e.g. _ddMMyyyy, _ddMMyyyyhhmmss, _ddMMyyyyhhmmss.fff (see this URL http://msdn.microsoft.com/en-us/library/az4se3k1(v=vs.110).aspx) */
   	Description varchar(500) NULL,
   	EmailFrom varchar(255) NULL,
   	EmailTo varchar(1000) NULL,
   	EmailCc varchar(1000) NULL,
   	EmailBcc varchar(1000) NULL,
   	EmailSubject varchar(500) NULL,	
   	EmailIsHtml bit NULL default(1),	
   	EmailBodyText varchar(max) NULL,
   	EmailAttachmentFile varchar(500) NULL, /*Unused: For future use only*/
   	LastExportStartDate datetime NULL,
   	LastExportEndDate datetime NULL,
   	LastStatus varchar(20) NULL,
   	LastMessage varchar(max) NULL,
   	Datasets varchar(max) NULL, /*if ReportType=local and report file is using dataset(s) then supply data in this column (multiple values must be pipe delimited). e.g. Dataset1|Dataset2)*/
   	DatasetConnectionStrings varchar(max) NULL, /*if ReportType=local and report file is using dataset(s) then supply data in this column (connection string must be same order as datasets (multiple values must be pipe delimited) enter just one connectionstring if all datasets using same connection. ConnectionString must be ADO.NET for SQLServer, ODBC or OLEDB connection string and correct driver must be installed on the system)*/
   	DatasetQueries varchar(max) NULL /*if ReportType=local and report file is using dataset(s) then supply data in this column (queries must be in the same order as dataset(s) listed in Datasets column. enter multiple queries in this format <xml><query>{query for 1st dataset}</query><query>{query for 2nd dataset}</query></xml>)*/
   );

    
   

   Sample Database in SSIS for SSRS

5. Finally, you can check the table with the configuration:
   

   Data-driven table created in SSIS

Export a Local SSRS RDL Report from SSIS

With local reports, you can work with rdl files. The rdl (Report Definition Language) files are part of the SSRS. They are physical files that contain the report format and other components. Using the export local report you can customize queries and see the results as reports. Here we will explain how to do it.

1. First of all, in the Reporting Services task in Visual Studio, make sure to select the option Export local report.
2. Secondly, in report path, you need to specify your rdl file path.
3. In addition, check the dataset. The dataset is usually detected by default. It is included in the SSRS.
4. Also, in Dataset Connection String, you will need to specify the connection information. You need to specify the SQL Server name, Database name, Authentication type. For more information about connection string, click here.
5. Finally, in dataset queries, you need to specify your query using the XML and query tags:
   

   SSIS local SSRS report

Move Exported Reports to a UNC Path

After export, you can copy or move files—for example to a shared folder via a UNC path—using Advanced File System Task.

1.  First of all, specify your file name to move. You can use *.xlsx for example to move all the files.
2. Secondly, specify the UNC path in destination:
   

   SSIS move files to UNC path

3. Also, It is possible to sort files by name, size, modified date or other properties:
   

   Sort file in SSIS by name, size, modified date

4. Finally, press the Examples link, you can see some custom queries that you can create:
   

   Sample SSIS queries to files

Related SSIS and SSRS Automation Scenarios

You can use this SSIS-based SSRS automation approach in many real-world workflows where a fixed SSRS subscription is not flexible enough:

• Send daily ETL validation reports only after the data load succeeds.
• Generate customer-specific reports by passing customer IDs or date ranges as parameters.
• Email invoice, billing, or sales reports after a business process is completed.
• Export audit reports to a shared folder with date/time in the file name.
• Run multiple SSRS reports from a configuration table and deliver each report to different recipients.

FAQ: SSRS Reports from SSIS

Can I call an SSRS report from SSIS?

Yes. You can use ZappySys SSRS Reporting Services Task inside an SSIS package to call a server SSRS report or a local RDL report and export it to formats such as PDF, Excel, Word, CSV, XML, TIFF, or HTML.

Can SSIS pass parameters to an SSRS report?

Yes. You can create SSIS variables and map them to SSRS report parameters in the Reporting Services Task. This is useful when report values such as customer, date range, region, or batch ID must be decided at runtime.

How do I send an SSRS report by email from SSIS?

You can use the Reporting Services Task to render the report and send it as an email attachment. For more advanced HTML email scenarios, export the report first and then use ZappySys Send HTML Email Task.

Can I send an SSRS report as the HTML body of an email?

Yes, but use server report mode. In newer builds (after May 6, 2026), open the Email Settings tab and check Use report output as Email body (*.htm / *.html export formats only), with an HTML export. On older versions, export the report to HTML and use Send HTML Email Task with a placeholder such as <<C:\temp\sales-order.html,FUN_FILE_READ>>.

Why use SSIS instead of SSRS subscriptions?

Use SSIS when the report must run after a workflow finishes, when parameters must be supplied dynamically, or when output file names, destinations, and email rules depend on package logic. SSRS subscriptions are better for simpler fixed schedules and predefined parameters.

Next Steps: Download and Try SSIS PowerPack

Automate SSRS from SSIS with the ZappySys Reporting Services Task—event-driven runs, dynamic parameters, flexible export paths, and richer email options than typical subscriptions alone.

• Download SSIS PowerPack — trial and licensed builds.
• SSRS Report Generator Task (Reporting Services Task) — product overview.
• SSIS PowerPack home — all tasks and components.

Conclusion

In summary, calling SSRS reports from SSIS is straightforward with the ZappySys Reporting Services Task. It fits scenarios where SSRS subscriptions are too rigid: jobs triggered after ETL or other processes, parameters chosen at runtime, and customized file naming or delivery. This guide covered export to file, email attachments, HTML-in-body email (new checkbox or legacy two-step flow), date/time file names, UNC moves, parameter mapping, data-driven batches, and local RDL export.

References

For more reading:

• Reporting Services Tutorial
  
• SSRS Report Generator Task in SSIS
• SSIS Send HTML Email Task
• Advanced SSIS File System Task
• Format Specifiers / Placeholder Functions

The post Calling SSRS Reports in SSIS (Export / Email) appeared first on ZappySys Blog.

In this article, we will show how to send values from an SSIS Variable to an HTML file. We will use the SSIS Advanced File System Task to store the list of system files of a folder into a variable and then we will use the SSIS Logging Task to store the variable in an HTML file.

In the second part, we will read values from an HTML table and export the data into SQL Server. We will use the HTML file created in the first example. We will also use the ZS HTML Table Source to read the data from the HTML table and export to SQL Server using the OLEDB Destination.

We will be using this ZappySys SSIS PowerPack component to make things work:

	SSIS Logging Task (Free)
	SSIS HTML Table Source
	Advanced SSIS File System Task

Finally, we will show how to read data from a SQL Server query and store into an HTML file. We will use the Execute SQL Task combined with the ZS SSIS Logging Task.

Requirements

1. First, you will need SSDT installed.
2. Secondly, ZappySys PowerPack for SSIS.
3. Finally, SQL Server installed.

Getting started

How to save the list of system files into SQL an HTML table

In the first example, we will use our ZappySys Advanced File System to copy a list of the system files in a folder. This list of files will be stored in an object variable and then we will save to a file.

1. In the first place, we will use the Advanced File System Task and select the Get file list ADO.net DataTable action. We will list the files of the c:\sql folder and save the results into a variable:
   

   SSIS list files

2. In addition, we will create a variable of type object named listfiles that will store the files information:
   

   SSIS object variable

3. Also, we will use the ZS Logging Tasks and join with the ZS Advanced File. The ZS Logging Task will be used to store the variable into a file:
   

   SSIS save variable to HTML

4. In addition, it is possible to send the variable values into the Execution Log (SSIS output), to a file, to a MessageBox:
   

   SSIS output variable to log, message box and file variable to read and write data to HTML in SSIS

5. In this example, we will show the values into an HTML file named listfiles.html
   

   SSIS save variable iHTMLhtml

6. After executing the package, you will be able to see the HTML with a list of files, names, size, modification and creation information:
   

   ssis list files

How to copy data from an HTML table to SQL Server

The next example will save data from a file named listfiles.html created in the previous example with an HTML table. The example will export the table to SQL Server. However, with our HTML Table Source, you can export to any destination.

1. In order to start, we will Drag and drop a data flow task and double click it:
   

   SSIS data flow

2. Next, in the Data Flow task, use the HTML Table Source and join to the OLEDB Destination:
   

   SSIS HTML table SQL server

3. Also, in URLs enter the path of the HTML file created in the first example:
   

   SSIS HTML table read

4. In addition, go to the OLEDB Source, specify the SQL Server name, database, and log in to it.
   In the name of table or view press new to create a new table:
   

   CREATE TABLE [dbo.listfiles] (
   [_rowNumber] int,
   [_groupName] nvarchar(255),
   [FullPath] nvarchar(255),
   [Name] nvarchar(255),
   [Size] int,
   [AgeInDays] int,
   [LastEditInDays] int,
   [LastModifiedDate] datetime,
   [CreationDate] datetime,
   [LastModifiedDateUtc] datetime,
   [CreationDateUtc] datetime,
   [Extension] nvarchar(255),
   [IsReadOnly] nvarchar(255),
   [FolderPath] nvarchar(255)
   )

   

   SSIS OLEDB create table

5. Also, go to Mappings page to map all the columns.
   

   SSIS map columns

6. Finally, run the package and you will have all the data exported.

How to export SQL Server results to an HTML in SSIS

The following example will show how to export the results of an SQL query into an HTML file.

1. In order to start, we will use the Execute SQL Task. Specify the option Full result set and create an OLEDB connection specifying the Server Name, database name and the select statement to a table:
   

   SSIS SQL query results

2. Secondly, go to the Result Set page and in the Result Name, set 0 and in Variable name, create a new variable:
   

   ssis sql result to variable

3. Also, create a variable of type object:
   

   SSIS object data type

4. Finally, use the ZS Logging Task to save the results in a File. Specify the Log file path and insert the variable:
   

   SSIS save to HTML file

Conclusion about read and write data to HTML in SSIS

To conclude, in this article, we learned how to export a list of files system properties and names to SQL Server. How to read an HTML table to SQL Server and finally how to export SQL Server results to an HTML in SSIS. There are nice tools to read and write data to HTML in SSIS

If you liked the tools, you can download them HERE.

References

• SSIS PowerPack ZappySys
• Integration Services (SSIS) Variables
• Execute SQL Task

The post How to read and write data to HTML in SSIS appeared first on ZappySys Blog.

In this short article, you will learn how to write Regular expressions in SSIS (i.e. Regex) and what tool to use to test them. You will also find helpful resources on how to write more sophisticated expressions and learn more about them. For demo purposes, we will use FREE SSIS Regex Parser Task to parse and extract the text using Regex.

You can use Regular expressions in several SSIS PowerPack connectors:

• SSIS Regex Parser Task (FREE),
• Azure Blob Storage Task,
• Advanced File System Task,
• Amazon S3 Storage Task and others.

Writing Regular Expressions in SSIS

Based on the SSIS component you use it will include the filename in filtering options or match the text: See below Syntax to write Regex in ZappySys tools. We support additional construct {{X, Y}} at the end of Regex to control two parameters. This additional construct is useful for data extracted from matching regex. If you want to Test patterns and not worry about data extraction then no need to use last {{

Occurance_Index

Group_Index_Or_Name

Syntax:

<your Regular Expression>[{{Occurance_Index|*[,Group_Index_Or_Name]}}]

Where:

Occurance_Index

Group_Index_Or_Name

How to name a group?

E.g. (\w+)@(?<domain>\w+.com)

How to use the group name in the match extract?

E.g. (\w+)@(?<domain>\w+.com){{0,domain}}

Example Input:

Let’s assume we have the following input text. We will test various Expressions.

Customer => AAA
Email => aaa@google.com
Phone => 101-222-3333
========
Customer => BBB
Email => bbb@yahoo.com
Phone => 102-222-3333
========
Customer => CCC
Email => ccc@hotmail.com
Phone => 103-222-3333
========
Customer => DDD
Email => ddd@outlook.com
Phone => 104-222-3333

Sample Regex Expressions

(.*)

More Regular Expression Examples

Regex Examples (Using SSIS Regular Expression Parser Task)

Here is an example how Regex

(.+)_(.+)_(.+){{1,2}}

Using Regular Expressions in SSIS Regex Parser Task (Extract Groups)

Using Groups / Occurrence Index

Some tasks like SSIS Regex Parser Task (FREE) supports extracting value from specific occurrence and specific part of matched pattern using special syntax at the end of your pattern (see below).

Your Regex Pattern Here{{OccuranceIndex,GroupIndex}}

Where :
OccuranceIndex is 0 based (0=extract first occurrence)
GroupIndex is 0 based (0=extract first matching group from pattern. First group is always entire text. )

See above screenshot in previous section for example.

Tools

The best tool we’ve found to write and test Regex is Regex Hero (will require IE with Silverlight if you want to use it online, in the browser):

Another great site for Regex testing is https://regex101.com (Works in any browser unlike previous one)

and few more sites as below

http://www.regexr.com/
http://www.regexlib.com/
http://www.regular-expressions.info/

Resources

Regular Expressions cheat-sheet to hang on the wall

Regular Expressions quick reference

The post Using Regular Expressions in SSIS appeared first on ZappySys Blog.

In this blog post, you will see how to remove invalid characters from XML using SSIS. We will use the search and replace feature of the Advanced File System Task.

Remove Invalid characters from XML.

The XML file specification specifies which characters can be part of XML data and which should be avoided. If you use an invalid character, the XML Parser can throw an error indicating that it found an invalid character. Here is a very useful link that explains which characters are allowed in XML and which are not.

Using SSIS and Regex to clean invalid XML Characters

To clean invalid characters, we can use the following regular expression. This basically removes any characters outside the allowed character range.

[^\x09\x0A\x0D\x20-\xD7FF\xE000-\xFFFD\x10000-x10FFFF]

The above expression is sufficient for most cases, but it is not the best solution. If you really want the most accurate expression per the XML Specs, use the expression below. It can be SLOW for large XML files.

[^\u0009\u000a\u000d\u0020-\ud7ff\ue000-\ufffd]|([\ud800-\udbff](?![\udc00-\udfff]))|((?<![\ud800-\udbff])[\udc00-\udfff])

Remove any characters

If you don’t have a range of valid characters but you know a list of invalid characters, use the expression below (without the ^). For example, if you want to remove A, C, and E; then use the below

[ACE]

--OR-- Use hex codes

[\x41\x43\x45]

You can also use a range

Step-By-Step Example

1. Download and install SSIS PowerPack from here.
2. Create a new SSIS Project.
3. Drag the ZS Advanced File System Task from the SSIS Toolbox to the control flow designer.
4. Double-click the task to edit its properties, as shown below. Enter the following expression for a Regular Expression.
   

   [^\x09\x0A\x0D\x20-\xD7FF\xE000-\xFFFD\x10000-x10FFFF]

5. Save the package and execute. After you run the task, it will remove (replace with blank) any invalid character in your XML file.

Here is the screenshot of the Task configuration.

SSIS Advanced File System Task – Search and Replace Option -Remove invalid XML characters using Regex (Regular Expression)

The post How to remove invalid characters from XML using SSIS and Regex appeared first on ZappySys Blog.

In this small blog post you will learn How to move files using SSIS Advanced File System Task and How to wait until file is unlocked using C# Script Task.

How to check if file is locked (SSIS C# Script Task)

If you want to check if file is locked in C# then below code will do the trick. But no worry if you dont know C#. ZappySys SSIS PowerPack comes with two tasks which can solve this issue without coding. You can use SSIS Advanced File System Task with Get file lock status action or Use SSIS Validation Task which has option to throw error on lock condition or you can continue by saving lock status into variable and continue without throwing error.

// Attempt to open the file exclusively. -- If you get erro means file is locked
using (FileStream fs = new FileStream(fullPath,
	FileMode.Open, FileAccess.ReadWrite,
	FileShare.None, 100))
{
	fs.ReadByte();

	// If we got this far the file is ready
	break;
}

How to wait until file is unlocked (SSIS C# Script Task)

Now lets check real example which will first check for locked file using SSIS Validation Task and then if file is locked then wait until file is unlocked (with some timeout hardcoded in script).

If specified wait time is reached then script will throw error. Once file is unlocked SSIS Advanced File System Task will copy file to target.

SSIS – How to check if file is locked. Wait until file is unlocked (C# Code – SSIS Script Task).

SSIS Validation Task -Store file lock status into variable

SSIS Validation Task -Check if file is locked. Save status to variable or throw error.

SSIS Advanced File System Task – Copy, Move, Rename, Delete multiple files

SSIS Advanced File System Task. Copy, Move, Rename, Delete multiple files

SSIS C# Script Task – Check file is locked, wait until file is unlocked

#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO;
#endregion

namespace ST_334a75922c6a47a5b4ac21ee8e4e5fff
{
	[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
	public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
	{
		public void Main()
		{
			// TODO: Add your code here
            try
            {
                var file = Dts.Variables["User::filePath"].Value.ToString();

                bool continueIfFileMissing = false;
                if (!System.IO.File.Exists(file))
                {
                    if (continueIfFileMissing)
                    {
                        Dts.TaskResult = (int)ScriptResults.Success;
                        return;
                    }
                    LogError("File not found: " + file);
                    Dts.TaskResult = (int)ScriptResults.Failure;
                }
                    
                WaitForFile(file, maxWaitInSec:30);
                Dts.TaskResult = (int)ScriptResults.Success;
            }
            catch (Exception ex)
            {
                LogError(ex.Message);
                Dts.TaskResult = (int)ScriptResults.Failure;
            }           
			
		}
        /// <summary>
        /// Blocks until the file is not locked any more.
        /// </summary>
        /// <param name="fullPath"></param>
        bool WaitForFile(string fullPath,int maxWaitInSec)
        {
            int numTries = 0;
            while (true)
            {
                ++numTries;
                try
                {
                    // Attempt to open the file exclusively.
                    using (FileStream fs = new FileStream(fullPath,
                        FileMode.Open, FileAccess.ReadWrite,
                        FileShare.None, 100))
                    {
                        fs.ReadByte();

                        // If we got this far the file is ready
                        break;
                    }
                }
                catch (Exception ex)
                {
                   if(numTries==1 ||  numTries % 20 ==0 )
                      LogWarning(string.Format("WaitForFile {0} failed to get an exclusive lock: {1}",fullPath, ex.Message));

                    //if (numTries > 10)
                    //{
                    //    LogWarning(string.Format(
                    //        "WaitForFile {0} giving up after 10 tries",
                    //        fullPath));
                    //    return false;
                    //}

                   if (numTries >= maxWaitInSec * 2)
                   {
                       throw new Exception("Max wait time reached for file : " + fullPath + ". Waited for " + maxWaitInSec + " seconds but lock not released");
                   }

                    // Wait for the lock to be released
                    System.Threading.Thread.Sleep(500);
                }

                
            }

            LogInformation( string.Format( "WaitForFile {0} returning true after {1} tries",fullPath, numTries));
            return true;
        }

        private void LogInformation(string msg)
        {
            bool again = false;
            Dts.Events.FireInformation(0, "ScriptTask", msg, "", 0,ref again);
        }
        
        private void LogError(string msg)
        {
            Dts.Events.FireError(0, "ScriptTask", msg, "", 0);
        }
        private void LogWarning(string msg)
        {
            Dts.Events.FireWarning(0, "ScriptTask", msg, "", 0);
        }
        #region ScriptResults declaration
        /// <summary>
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        /// 
        /// This code was generated automatically.
        /// </summary>
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

	}
}

Download Sample Package

Below sample will work only if you have SSIS PowerPack Installed. Download it from here it will take only 1 minute to install
Download SSIS 2012 – Sample Package (Process Locked file)

Conclusion

Processing and detecting locked files in SSIS can be tricky but using small C# script it can save you some headache. Download Advanced File System Task to try many options not available in native File System Task.

Other Keywords:
Check locked files in SSIS
How to check whether file is locked or not in SSIS
Detect locked file in SSIS
Wait until file is unlocked using C# script
How to handle file locking issue in SSIS using C# script

The post SSIS check file is locked and wait until file is unlocked (C# Script) appeared first on ZappySys Blog.

In this post you will learn how to use Advanced File System Task to get list of files and folders into variable. Advanced File System Task is significantly better than native SSIS File System Task

How to get list of files in SSIS

Most simplest way to get list of files in SSIS is use wildcard pattern as below screenshot. You can also use Scan Recursive option to include files inside child folders. For advanced filtering options use Filter and Sorting Tab

1. Download and install SSIS PowerPack from here
2. From toolbox of SSIS designer drag ZS Advanced File System Task
3. Double click Advanced File System task to configure it
4. Select Action as [Get file list as ADO.net DataTable]
5. Now select files you want to get using wildcard pattern (e.g. c:\ssis\file*.txt)
6. In the result variable select variable or create new variable with Object datatype. This variable will host file list in the form of ADO.net DataTable

SSIS Advanced File System Task – get file list as ADO.net Recordset into variable

Loop through files in SSIS using ForEach Loop Task (use ADO.net Recordset variable)

Variable Mappings for ForEach Loop Task – Loop through files in a folder using SSIS

Get files modified in last N days

To get files modified in last N days you can use Advanced File System Task.

1. Download and install SSIS PowerPack from here
2. From toolbox of SSIS designer drag ZS Advanced File System Task
3. Double click Advanced File System task to configure it
4. Select Action as [Get file list as ADO.net DataTable]
5. Now select files you want to get using wildcard pattern (e.g. c:\ssis\file*.txt)
6. Go to Filter and Sorting tab. In the Where Field enter expression LastEditInDays < 5 This will give you list of files modified in last 5 days. Here are some more examples.
   

   //////////////////////////////
   Examples:
   //////////////////////////////
   AgeInDays < 5  //Get files created in last 5 days
   LastEditInDays > 3 //Get files modified before 3 days
   AgeInDays > 10 AND LastEditInDays < 3  //Get files created before 10 days and modified in last 3 days
   Size < 1024  //Get files where File size is less than 1024 bytes
   CreationDate BETWEEN #2015-01-01# AND #2015-01-01#   
   Name IN ('File1.txt', 'Download.msi')
   FullPath LIKE 'c:\temp\da*')
   Extension IN ('.txt','.csv')
   LastModifiedDate > #2013-01-01T16:00:00#
   IsReadOnly = True
   /////////////////////////////////////////
   //Supported Columns for expression:
   /////////////////////////////////////////
   Name, FullPath, Extension, FolderPath, AgeInDays, LastEditInDays, Size, CreationDate, CreationDateUtc, LastModifiedDate, LastModifiedDateUtc, IsReadOnly

   

   SSIS Advanced File System Task – Get files modified in last N days

Get files with specific size

To get files with specific size you can use Where Expression on Size attribute (Size is in Bytes)

1. Download and install SSIS PowerPack from here
2. From toolbox of SSIS designer drag ZS Advanced File System Task
3. Double click Advanced File System task to configure it
4. Select Action as [Get file list as ADO.net DataTable]
5. Now select files you want to get using wildcard pattern (e.g. c:\ssis\file*.txt)
6. Go to Filter and Sorting tab. In the Where Field enter expression Size < 1024 This will give you list of files which are less than 1024 bytes (e.g 1KB).
   

   Size < 1024

Get files with Include/Exclude pattern (Use Regular Expression)

Sometimes you have need to list files with specific name patterns which is hard to get using simple wildcard then use Advanced Regular Expression option. For this use wildcard in Path (e.g. c:\ssis\*.*) and then in the Filter and Sorting tab use Include / Exclude Regular Expressions (e.g. RegX) for file name pattern.

For Example if you want to include all files containing *MMDD.* pattern but not *.zip file or *.gz then use below expressions

Include ==>  ^w+_\d\d\d\d.\w+$   
Exclude ==>  \.zip|\.gz

How to get list of folders in SSIS (list directories)

Coming soon

Loop through files or folders in SSIS using ForEachLoop Task

Coming soon

Read Recordset and load into database (e.g. SQL Server Table)

Coming soon

Conclusion

Microsoft native file system task has very limited features and many scenarios require you to write C# / VB.net Script. But using ZS Advanced File System Task you can eliminate need for scripting and use simple drag and drop approach. To use this task Download SSIS PowerPack

The post Get list of files in SSIS for Looping appeared first on ZappySys Blog.

Many times you have requirement to search and replace in files (single or multiple file) content using SSIS. If you are not C# or VB.net programmer then you may find yourself at dead end struggling how to achieve this in SSIS. If you have this need then you can use Advanced File System Task

Search and replace in files using Simple text search

In this option you can specify text you want to search and text you want to replace with. You can check ignore text if you dont want case-sensitive search/replace.

Extended text search (special characters allowed)

For extended search mode you may use any of the following in your search/replace string

• \n – for line feed
• \r – for return character
• \0 – for null character
• \t – for tab character
• \v – for vertical tab character

Example:
Search: “Hello\r\nWorld”
Replace: “Hello World”

Regular expression search/replace

Search and Replace File Content in SSIS (Find sub string, Search for special character)

Advanced File System Task

If you want to perform some other operations then here is the list of few features supported by Advanced File System Task.

• File operations for single or multiple files using wildcard(e.g. copy, move, rename, delete, write to file, get properties)
• Folder operations such as delete folder and create folder
• Support for bulk rename (e.g. by specifying {%name%}.{%ext%}.old as new name will rename all selected files to *.old)
• Search and Replace in files (Regular expression search supported)
• Change file encoding
• Get file properties as data table which you can loop easily (Using ForEach Loop – ADO Enumerator)
• Option to continue on error (e.g. Copy file only when source file found)
• Auto create missing target directory
• Use placeholders anywhere in the path, makes it easy to timestamp folders or files (e.g. c:\Archive\{{System::StartTime,yyyMMdd}})
• Regular Expression based Include Pattern (e.g. ^(file\d\d).(\w+)$ will include file01.exe,file22.csv …)
• Regular Expression based Exclude Pattern (e.g. (.exe|.msi) will exclude files with extension exe or msi)
• Sort based on specific property (e.g. Order Size DESC – extract top 10 files by size)
• Advanced SQL like filter support (e.g. Size> 1024 AND AgeInDays > 10 AND Extension IN(‘.txt’,’.csv’) will include file greater than 1024 bytes and extension is txt or csv)
• Support for Regular Expression based Include Pattern (e.g. ^(Customer\d\d).(\w+)$ )
• Support for SQL Server 2005, 2008, 2012, 2014, 2016 (32 bit and 64 bit)

Download FREE SSIS Components

Conclusion

Search and replace in many files can be tricky but using Advanced File System Task it can be few click. Click here  to download.

The post Search and replace in files using SSIS – No Coding appeared first on ZappySys Blog.

Many times during your ETL process you receive files which are in different encoding than you expect. Example all files are in UTF-16 format but your application expect them to be in UTF-8. Or Sometimes files are in ASCII format and you want to convert to UTF-8.

In this post you will see how to use Advanced File System Task to convert multiple files into desired encoding (e.g. ASCII, UTF-8, UTF-16).

Steps to Convert file encoding for Multiple Files in SSIS

1. Download and Install SSIS PowerPack
2. From you SSIS Toolbox drag Advanced File System Task
3. Double click task to configure it
4. Select “Change Encoding of files” action from dropdown as below screenshot
5. Enter path or pattern for source file you want to convert to different encoding
6. Select desired encoding from Encoding dropdown (e.g. if you files are in ASCII and you want to convert them to UTF-8 .. just select UTF-8)
7. Execute package and observe SSIS execution log to find out which files are converted to new encoding.

Change file encoding using SSIS Advanced File System Task

Change file encoding to UTF-8 from ASCII using SSIS Advanced File System Task

SSIS Execution log – Convert files to UTF-8 using SSIS Advanced File System Task

The post How to change file encoding in SSIS (UTF8, ASCII or UTF16) appeared first on ZappySys Blog.