In our previous article, we saw SSIS Amazon S3 Storage Task examples. Now let’s look at how to Download the Latest File from Amazon S3 Storage using SSIS. Microsoft SSIS includes the Amazon S3 Storage Task that allows us to upload files to Amazon S3 Storage, download files, creating local and remote directories an more. In this article, we will show how to use the SSIS Amazon S3 Storage task included in ZappySys SSIS PowerPack.
In nutshell, this post will focus on how to Download the Latest File from Amazon S3 Storage.
PrerequisitesBefore we perform steps listed in this article, you will need to make sure following prerequisites are met:
- SSIS designer installed. Sometimes it is referred as BIDS or SSDT (download it from Microsoft site).
- Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
- Make sure ZappySys SSIS PowerPack is installed (download it).
- Optional (If you want to Deploy and Schedule ) - Deploy and Schedule SSIS Packages
What is Amazon S3 Storage
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides easy-to-use management features so you can organize your data and configure finely-tuned access controls to meet your specific business, organizational, and compliance requirements. Amazon S3 is designed for 99.999999999% (11 9’s) of durability, and stores data for millions of applications for companies all around the world.
In order to start, we will show several examples. ZappySys includes an SSIS Amazon S3 Storage Task that will allow you to access files/folders from Amazon S3 Storage to the Local machine, Upload files(s) to Amazon S3 Storage. It will also support Delete, Rename, List, Get Property, Copy, Move, Create, Set Permission … and many more operations. Here we are showing you is, How to Download the Latest File from Amazon S3 Storage.
Step-By-Step to Download the Latest File from Amazon S3 Storage
Let´s start with an example. In this SSIS Amazon S3 Storage Task File task example, we will download the latest file using Amazon S3 Storage Task.
- First of All, Drag and Drop Amazon S3 Storage Task from SSIS Toolbox and double click it to edit
- The following options are available in Amazon S3 Storage Task:
- You can also overwrite files and folder, skip all, overwrite if the source is older, overwrite if the file size is different, rename files and more:
- Let’s select the Download Amazon file(s) to local directory. Now select/add The Source path, Check Recursive Scan(*Optional) to get all the sub-files and subfolders and files, after that add Target Path if you want to, This option will download files. In Path AccessMode we can use Direct to write the path directly or to use an SSIS variable.
- Now go to Advanced Filter & Sorting and configure it like below screen to download the latest file only.
- That’s it now execute the package and it will download the latest file from the Amazon S3 Storage Task to Local File Machine.
- Sometimes you may need to make advanced filtering on file path, file date, size, etc. for that refer to this article for more details.
To conclude, we can say that working with Amazon S3 Storage is now very simple. In this article, we looked at how to Download the Latest File from Amazon S3 Storage using SSIS. We used Adnavced Filtering & sorting option. If you liked the tasks you can start using them by downloading SSIS PowerPack from our web site here.
Thanks for your time.