SSIS Azure Queue Storage Source

Help > Source Adapters > Azure Queue Storage Source >

SSIS-Azure-Queue-Storage-Source-Adpater
Azure Queue Storage Source can be used to read messages from Microsoft Azure Queue Storage using SSIS.

Content

Video Tutorial
Step-By-Step
Component Properties
Error Handling
Settings UI
See also

Video Tutorial

Step-By-Step

In this section you will learn how to read messages from Azure Queue Storage. For this demo we are assuming that you already have Queue created.

In order to connect to Azure Storage Service from SSIS you will need Storage Account Name and Access Key. Ask your SysAdmin or responsible person to provide that information to you. Here is sample Account Name and Access Key(this is just example key which may differ in your case)
```
Account Name: mystorageaccount
Access Key: Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
```
Microsoft also provides offline simulator to test Azure Storage functionality (Blob, Table, Queue). This is great way to try out Azure Storage functionality without worrying about Cost but we strongly recommend you to test everything with Azure Cloud Account before going to Production.
Click here to read more how to get your Storage Account Name and Access Key.

If you don't have Azure Storage account then its easy to get Trial Azure Storage account or use your MSDN to get credit each month which will be more than enough for real testing. If you prefer to avoid all this hassle then Microsoft provides another great way to test Azure Storage functionality in totally offline mode on your local machine. You can just download Azure Storage Emulator and start testing. Click here to learn more how to configure Azure Storage Emulator for Testing and Development purpose.

To create or view Azure queue you can use FREE tool like Microsoft Azure Storage Explorer.
Once you have Account Name (its like UserID) and Access Key (its like password) you may proceed to create new SSIS package. In BIDS/SSDT create new SSIS package.
Now, Drag and Drop SSIS Data Flow Task from SSIS Toolbox.
Double click on the Data Flow task to see Data Flow designer surface.
From the SSIS toolbox drag and drop ZS Azure Queue Storage Source on the dataflow designer surface.
Double click Azure Queue Storage Source to configure it.
Click on [New] button for Select Connection Manager. Enter Account Name and Access Key and Click Test connection to verify credentials.
Click OK to save connection information.
Select queue from the dropdown and click preview.
Click OK to save settings.
From the SSIS toolbox drag and drop Trash Destination on the data flow designer surface.
Now single click on the Azure Queue Storage Source once you see blue arrow from source ... connect it to Trash Destination.
Double click on ZS Trash Destination to Configure it.
Click on OK button to save Trash Destination configure setting UI.
Right click on the path and Add Data Viewer.
Execute the package and verify source data in data viewer.
After messages read from Azure Queue, same message cannot be read again until VisibilityTimeout. By default Queue visibility timeout is used (-1 means use default) but you can override it on the source UI

Component Properties

Property Name

Description

LoggingMode

LoggingMode determines how much information is logged during Package Execution. Set Logging mode to Debugging for maximum log.

Available Options (Use numeric value listed in bracket if you have to define expression on this property (for dynamic behavior).

Option	Description
Normal [0]	Normal
Medium [1]	Medium
Detailed [2]	Detailed
Debugging [3]	Debugging

PrefixTimestamp

When you enable this property it will prefix timestamp before Log messages.

QueueName

Queue name for this action

MaxRows

Maximum number of rows to get from queue. Keep fetching data from queue until total message count >=MaxRows or response is empty (no message).

VisibilityTimeout

This is message visibility timeout in seconds. Set -1 to use default timeout (default queue level setting). During visibility timeout period message becomes unavailable after you read it so adjust this timeout as per your need. Set timeout to 0 if you want to allow read operation for same message without any wait time. If you set this timout too small then same message might appear back in read operation and it will throw duplicate message error.

MaxRetryOnNullMessageReceive

Maximum retry count for NULL message receive before it stops calling read message from queue.

OutputDateAsLocalTime

Output any UTC date as Local time.

Error handling

This component supports error output.
Any bad value will cause Error or Truncation (e.g. If your sample value is 5 character long but output column length is set to 3 chars). If you have error output attached with RedirectRow option on Error, Truncation then you can capture bad rows along with exact reason of error.
Use error handling tab to set option for error handling. By default component fails on any data error or truncation issue

Setting UI

Articles / Tutorials

Click here to see all articles for [SSIS Azure Queue Storage Source] category

How to Read/Write Azure Queue Storage data in SSIS

Introduction In our previous blog, we saw you how to read/write Azure Table Storage data. Now in this blog, we will see How to Read/Write Azure Queue Storage data in SSIS. To illustrate, we will use ZappySys SSIS PowerPack, which includes several tasks to import/export data from multiples sources to multiple destinations like flat files, Azure, AWS, databases, […]

Read and Write in Azure Queue Storage in SSIS

Introduction In our previous blog we saw How to load data into Azure Blob Storage. Now in this blog, we will How to Read and Write in Azure Queue Storage in SSIS. To illustrate, we will use ZappySys SSIS PowerPack, Which includes several tasks to import/export data from multiples sources to multiple destinations like flat files, Azure, […]