SSIS Amazon S3 Task (SSIS AWS S3 Task)

SSIS Amazon S3 Task (SSIS AWS S3 Task) can be used to perform various operations with Amazon S3 Storage objects (buckets and files) (e.g. Download, Upload, Copy, Move, Rename, Delete etc). You can also get single property of any S3 file or get list of files as ADO.net Table and which can be easily loop through using ForEachLoop Task. Most of operations in this task support recursive search using wild card pattern (e.g. Myfile*.txt). Amazon S3 is one of the most popular Cloud Storage Services available right now.

Features

30+ Amazon S3 operations for S3 buckets and files (e.g. Download, Upload, Copy, Delete…)
Download small or very large S3 Files (i.e. AWS S3 file) to local machine
Upload small or very large local file(s) to AWS S3 service
Delete, Rename, List, Get Property, Copy, Move, Create, Set Permission … and many more operations
Support IAM Role for EC2 instances. Check this article.
Encrypt data using Client Side and Server Side Encryption Options (AES 256, RSA 1024)
Generate PreSigned URL to transfer to other cloud systems (such as Google Cloud or Azure Cloud)
Support for Get List operation so you can use SSIS ForEachLoop to loop through files
Download, Upload, Copy, Delete are multi threaded (i.e. fast parallel operations)
Support for SQL Server 2022, 2019, 2017, 2016, 2014, 2012 (32/64 bit) and now Azure Data Factory

Watch this video on YouTube

From SSIS Toolbox drag Amazon S3 Storage Task on the designer surface

Drag and Drop Amazon S3 Storage Task on the designer surface

Configure SSIS Amazon Storage Connection for S3 /Redshift /DynamoDB

Create and Configure Amazon S3 Storage Connection

SSIS Amazon S3 Task Operations – Upload, Download, Delete, Create, List, Rename Files, Folders, Buckets

How to Download Files from SSIS Amazon S3 Task

SSIS Amazon S3 Task – Download Files to Local Disk

SSIS Amazon S3 Task Encryption Option (Server Side and Client Side)

ui-amazon-s3-task-storage-security

SSIS Amazon S3 Task Metadata (Key / Value Pairs)

Advanced Filter for Delete / Upload /Download / List – Regx Pattern / SQL like expression

Advanced Search in SSIS Amazon S3 Task – Use Regular Expression Patterns, SQL like expression for Sort / Filter

SSIS Amazon S3 Task Execution Log

Run or Execute Package.

SSIS Amazon S3 Task Execution Log

Articles / Tutorials

Click here to see all articles for [SSIS Amazon Storage Task] category

Download the Latest File from Amazon S3 Storage using SSIS

Introduction In our previous article, we sawÃÂ SSIS Amazon S3 Storage Task examples. Now let’s look at how to Download the Latest File from Amazon S3 Storage using SSIS.ÃÂ Microsoft SSIS includes the Amazon S3 Storage Task that allows us to upload files to Amazon S3 Storage, download files, creating local and remote directories an more. In […]

SSIS Amazon S3 Storage Task examples (Download, Upload, Delete Files / Folders)

Introduction In this article, we will showÃÂ SSIS Amazon S3 Storage task examples. Uploading files using Amazon S3 Storage Task or downloading files are common requirements in SSIS. In order to work with Amazon S3 Storage, Microsoft SSIS includes the Amazon S3 Storage Task that allows to upload files to Amazon S3 Storage, download files, creating […]

Setting Metadata key / value in SSIS Amazon S3 / Azure Storage Task

Introduction In this post we will see how to set Metadata key/value in Azure Blob Storage Task and Amazon Storage Task (S3 Task). We will also see how to set them dynamically at runtime (Use SSIS Expression). Set Amazon S3 Task Metadata (Key / Value Pairs) Here is how you can set Metadata in Amazon […]

Copy Amazon S3 Files in SSIS to different AWS Account

Introduction In our previous blog we saw how to upload data to Amazon S3ÃÂ now let’s look at how to Copy Amazon Files from one AWS account to another AWS account (Server Side Copy) using SSIS Amazon Storage Task. (Must be version v2.7.9 or higher) If you are doing file copy within same account then there […]

How to delete Amazon S3 File / Folder / Bucket in SSIS

Introduction In our previous posts we looked at how to use SSIS Amazon Storage Task (Amazon S3 Task)ÃÂ to automate some Amazon S3 related tasks in SSIS.ÃÂ In this blog post will continue discovering more use cases. Let’s learn how to delete Amazon S3 File, delete Amazon S3 Folder and delete Amazon S3 Bucket. Deleting S3 Files […]

Using Regular Expressions in SSIS

Introduction In this short article you will learn how to write Regular expressions in SSIS (i.e. Regex) and what tool to use to test them. You will also find helpful resources on how to write more sophisticated expressions and learn more about them. For demo purpose we will use FREEÃÂ SSIS Regex Parser TaskÃÂ to parse and […]

Import / Export data from Amazon Athena using SSIS

Introduction In our previous post we explored unlimited possibilities to call Amazon AWS API using SSIS. Now in this post we will learn how to import / export data from Amazon AthenaÃÂ using SSIS. Amazon Athena is very similar service like Google BigQuery which we documented already.ÃÂ To read data from Amazon Athena we will use […]

Filter by file size, name, date for Amazon / Azure Storage SSIS Task

Introduction Sometimes you may need to make an advanced filtering on file path, file date, size etc in SSIS Amazon S3 Task orÃÂ SSIS Azure Blob Storage Task . For example, you have a directory “Customers“, which contains client directories. Each of them contains a directory named “Invoices“, which is partitioned by year. E.g. “Customers/Microsoft/Invoices/2015”. So […]

SSIS – Copy Amazon S3 files from AWS to Azure

Introduction Azure and AWS both are most popular Cloud Platforms. In this blog post we will learn how to copy or move Amazon S3 files to Azure Blob StorageÃÂ without any coding or scripting ÃÂ (AWS to Azure File Copy / Migration Scenario). To achieve this objective we will use following Drag and Drop SSIS ÃÂ Tasks (i.e. […]

Loading data from SQL Server to Amazon S3 in SSIS (Split Files, GZip)

Introduction In this blog post you will see how easy it is to load large amount of data from SQL Server to Amazon S3 Storage. For demo purpose we will use SQL Server as relational source but you can use same steps for any database engine such as Oracle, MySQL, DB2. In this post we […]