How to detect missing columns in SSIS

ZappySys Team — Tue, 08 Oct 2019 23:12:08 +0000

How to detect extra columns or missing columns

Sometimes, we need to validate the input columns to check for extra or missing columns in SSIS. ZappySys includes some pretty nice tools, but some of them cannot check the required values. To fix that problem, you can complement these tools with our Regular Expression Parser Task.

We will use Regular Expressions to validate the columns. For example, we want to check that the file has 3 columns with specific names, if it contains extra columns or there are missing some columns, we want to send an error. We will show you how to do this.

We will be using this ZappySys SSIS PowerPack component to make things work:

SSIS Regex Task

Prerequisites

Before we perform the steps listed in this article, you will need to make sure the following prerequisites are met:

SSIS designer installed. Sometimes it is referred to as BIDS or SSDT (download it from the Microsoft site).
Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
Make sure ZappySys SSIS PowerPack is installed (download it, if you haven't already).
(Optional step). Read this article, if you are planning to deploy packages to a server and schedule their execution later.

Creating a CSV file

To start, we will create a plain text or CSV file with some columns. Create a file named file.txt or any name of your preference.

Add data similar to the following:

name,lastname,email
Clint,Eastwood,ceastwood@gmail.com
Cameron,Diaz,cdiaz@gmail.com

ZS Regular Expression Parser to detect missing columns in SSIS

First, SSDT opens an SSIS project.
Now, it is time to use the ZS Regular Expression Parser.
Secondly, drag and drop the ZS Regular Expression in the design pane:
SSIS Regular Expression parser
In AccessMode, use Connection and create a new connection.
Select the file you created before. Click OK to save the configuration.
Regular Expressions Parser Task – Create a connection to a local file
Add the following expression to the Task and test it:
```
^name,lastname\r\n
```
In the previous scenario, the expression will fail because it is missing the email.
Regular Expressions Parser Task – detect missing columns example fails
Finally, the following Expression
```
^name,lastname,email\r\n
```
will pass, because the email is included.

Conclusion

In this article, we learned how to validate and check the columns in a file in ZappySys using the ZS Regular Expression Parser. In the example, we created an expression to detect an exact match of columns. It will raise an error if a failure occurs.

If you want to enjoy this and other ZappySys tools, you can download our product here.