In our previous article, we saw SSIS Amazon S3 Storage Task examples. Now let’s look at how to Download the Latest File from Amazon S3 Storage using SSIS. Microsoft SSIS includes the Amazon S3 Storage Task that allows us to upload files to Amazon S3 Storage, download files, creating local and remote directories an more. In this article, we will show how to use the SSIS Amazon S3 Storage task included in ZappySys SSIS PowerPack.

In nutshell, this post will focus on how to Download the Latest File from Amazon S3 Storage.

What is Amazon S3 Storage

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides easy-to-use management features so you can organize your data and configure finely-tuned access controls to meet your specific business, organizational, and compliance requirements. Amazon S3 is designed for 99.999999999% (11 9’s) of durability, and stores data for millions of applications for companies all around the world.

Getting Started

In order to start, we will show several examples. ZappySys includes an SSIS Amazon S3 Storage Task that will allow you to access files/folders from Amazon S3 Storage to the Local machine, Upload files(s) to Amazon S3 Storage. It will also support Delete, Rename, List, Get Property, Copy, Move, Create, Set Permission … and many more operations. Here we are showing you is, How to Download the Latest File from Amazon S3 Storage.

Step-By-Step to Download the Latest File from Amazon S3 Storage

Let´s start with an example. In this SSIS Amazon S3 Storage Task File task example, we will download the latest file using Amazon S3 Storage Task.

1. First of All, Drag and Drop Amazon S3 Storage Task from SSIS Toolbox and double click it to edit
   

   Drag and Drop Amazon S3 Storage Task

2. The following options are available in Amazon S3 Storage Task:
   

   Options to upload, download files in SSIS

3. You can also overwrite files and folder, skip all, overwrite if the source is older, overwrite if the file size is different, rename files and more:
   

   Overwrite files in SSIS

4. Let’s select the Download Amazon file(s) to local directory. Now select/add The Source path, Check Recursive Scan(*Optional) to get all the sub-files and subfolders and files, after that add Target Path if you want to, This option will download files. In Path AccessMode we can use Direct to write the path directly or to use an SSIS variable.
   

   SSIS Amazon S3 Storage Task – Download files from Amazon S3 Storage

5. Now go to Advanced Filter & Sorting and configure it like below screen to download the latest file only.
   

   Storage Task configuration to download the latest file only using sorting

6. That’s it now execute the package and it will download the latest file from the Amazon S3 Storage Task to Local File Machine.
7. Sometimes you may need to make advanced filtering on file path, file date, size, etc. for that refer to this article for more details.

Conclusion

To conclude, we can say that working with Amazon S3 Storage is now very simple. In this article, we looked at how to Download the Latest File from Amazon S3 Storage using SSIS. We used Adnavced Filtering & sorting option. If you liked the tasks you can start using them by downloading SSIS PowerPack from our web site here.

Thanks for your time.

References

• About SSIS PowerPack
• Integration Services (SSIS) Expressions
• Help File: Amazon Storage Task

The post Download the Latest File from Amazon S3 Storage using SSIS appeared first on ZappySys Blog.

In this article, we will cover the points on how to load 10 million rows from SQL Server to Snowflake in just 3 minutes.

Snowflake is a data warehousing platform that resides in a cloud. Basically, it is a data warehouse software exposed as a service. It allows integrating many data sources via internal Snowflake Partner apps and loading them into the Snowflake storage engine. Another part of Snowflake is a computing engine that is responsible for serving your SQL queries. Both engines can work independently thus users that are querying a data warehouse are not affected by a data load that is happening at the same time. Snowflake is an elastic service, which means you pay only for the resources used. Specifically, you pay only for each second of the processing time you use.

To show how data loading works in Snowflake, we will take 10 million rows from SQL Server and load them into Snowflake using SSIS and ZappySys SSIS PowerPack. These SSIS PowerPack connectors will be used to achieve the task:

Export CSV File Task

Step-by-step – How to load 10 million rows from SQL Server to Snowflake in 3 minutes

Getting started

To achieve the goal, we will use a slightly modified Northwind database (example database from Microsoft) and SnowSQL – a command-line tool provided by Snowflake. So in the first steps, you will need to install the Northwind database and SnowSQL. After that, we will proceed to create a table in Snowflake – a table we will load the data into – and a file format which will be used to load data from the Snowflake staging area into the destination table. Once that’s complete, we will proceed in creating an SSIS package, adding and configuring the connectors, and finally running it to get the results. Let’s proceed!

Install a Northwind database

Download and run the creation script of a modified Northwind database. The only thing that was modified is a “CustomersForSnowflake” view added which returns 10M rows.

Install and configure SnowSQL command-line tool

Download and install SnowSQL; you will find the instructions on how to do that in https://docs.snowflake.net/manuals/user-guide/snowsql-install-config.html.

Once installed, you will need to configure the default Snowflake account name, user name, and password:

1. Go to Windows File Explorer and enter this path:
   %USERPROFILE%\.snowsql\
   Usually, when entered the path looks similar to this one:
   C:\Users\myUserName\.snowsql
2. Find a file named config and open it.
3. Then configure your credentials:

Create a table in Snowflake

Login to your Snowflake account, open a Worksheet and execute this query:

create or replace table Customers(
ID number,
CustomerID nchar(5),
CompanyName nvarchar(40),
ContactName nvarchar(30),
ContactTitle nvarchar(30),
Address nvarchar(60),
City nvarchar(15),
Region nvarchar(15),
PostalCode nvarchar(10),
Country nvarchar(15),
Phone nvarchar(24),
Fax nvarchar(24)
)

We will load 10M customers from the Northwind database and load them into this table.

Create a file format in Snowflake

Then in the same Worksheet create a file format for zipped CSV files by executing this query:

create or replace file format GzipCsvFormat
type = csv
field_delimiter = ','
null_if = ('NULL', 'null')
empty_field_as_null = true
compression = gzip
field_optionally_enclosed_by = '"'

We will use this file format when loading data from a Snowflake stage to the Snowflake destination table. This file format defines CSV format which is used by ZappySys Export CSV Task in SSIS.

Create an SSIS package

We are ready to create a new SSIS package and load some data into Snowflake. We are going to use two methods of how we load data into Snowflake. In one method we will load data into Snowflake’s local storage, while in the second one we will stage data in Amazon S3 bucket. Decide how you want to stage files, and then choose the appropriate workflow when creating a new package:

Loading 10 million rows from SQL Server to Snowflake

Add & configure ZappySys Export CSV Task

Firstly, drag and drop ZappySys Export CSV Task from SSIS toolbox and follow the instructions on how to configure it:

Configure source

1. Proceed with configuring the data source, from which you will be exporting data. We will use the Northwind database as an example:
   

   Export CSV Task: configuring OLE DB Connection to export data from SQL Server to Snowflake

2. Then move on and select it and enter the query or table name you want to export data from:
   

   Export CSV Task: configuring source SQL query for data loading from SQL Server to Snowflake

   We will be using this query:
   select top 10000000 * from CustomersForSnowflake

3. Then in the Split Options tab split the exported CSV into many files, e.g. into 50 MB chunks:
   

   Using data split options in the Export CSV Task

   NOTE: Snowflake recommends having files that are 10-100 MB in size when compressed. So it is perfectly safe to configure the value to 100 MB and above.

Configure target

Depending on which staging approach you chose to use, set the export target to a local path or S3 bucket:

Using local Snowflake storage

Just set the appropriate Save Mode and a file path:

Saving files locally to upload them to Snowflake local stage later

Using S3 storage

For storing staging files in S3, follow these steps:

1. Set Save Mode to Save to Connection and select <New ZS-AWS-STORAGE Connection>:
   

   Export CSV Task: configuring Amazon S3 endpoint as the target for data export

2. Then select S3 as Storage Service and fill in Access and Secret Key, select your region (optional):
   

   Configuring Amazon Connection Manager to store staging data in S3

3. Then in the Export CSV Task window, in Target tab configure it similarly:
   

   Export CSV Task: configuring target to store staging data in S3

   Another option you may consider is to use Amazon Storage Task or Azure Storage Task to upload files to S3 in a separate step after the Export CSV Step. For this perform the following steps.

   1. On the Target tab, set Save Mode to Save to Path (Local Disk) instead of Connection to S3 or Azure.
   2. Drag & Drop Amazon Storage Task or Azure Storage Task  and connect with the previous Export CSV Step
   3. Configure Storage Task to upload Local files to S3 or Azure Blob Storage.
   4. Continue to the next section for more instructions.

Add Execute Process Task to create a staging area

We are ready to add Execute Process Task to create a staging area in Snowflake. Again, depending on where you will store staging files, Snowflake’s local storage or Amazon S3, use one of the approaches below:

Create Snowflake local staging

Creating a local stage in Snowflake using SnowSQL command-line tool and SSIS

File path:

C:\Program Files\Snowflake SnowSQL\snowsql.exe

Arguments:

-q "CREATE OR REPLACE STAGE CustomersStaging" -d DEMO_DB -s Public

Create Amazon S3 staging

Creating an Amazon S3 stage in Snowflake using SnowSQL command-line tool and SSISFile path:

C:\Program Files\Snowflake SnowSQL\snowsql.exe

Arguments:

CREATE OR REPLACE STAGE CustomersStaging url='s3://your-bucket-name/destinationFolder/' credentials=(aws_key_id='AKIAXXXXXXXXXXXXXXXX' aws_secret_key='6p1ayaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaLXz88')

Add Execute Process Task to upload files to the staging area (local staging approach only)

We still need to upload CSV files that were saved locally for local Snowflake staging. So add another Execute Process Task and configure it similarly:

Uploading local CSV files to Snowflake stage using SnowSQL command-line tool and SSISFile path:

C:\Program Files\Snowflake SnowSQL\snowsql.exe

Arguments:

-q "PUT file://e:/temp/*.csv @DEMO_DB.PUBLIC.CustomersStaging"

Add Execute Process Task to copy files from the Snowflake/Amazon S3 stage to Snowflake table

Finally, add the final Execute Process Task to issue a command to load files from the stage into a real Snowflake table:

Copying data from a staging area to a Snowflake table using SSIS

File path:

C:\Program Files\Snowflake SnowSQL\snowsql.exe

Arguments:

-q "COPY INTO CUSTOMERS FROM @CustomersStaging file_format = (format_name = 'GZIPCSVFORMAT')" -d DEMO_DB -s Public

Execute the package

We are ready to execute the package, so just run it. In this example you see the workflow when using Amazon S3 as stage:

Executing an SSIS package to load data from Snowflake to SQL Server

The Results

Once the package executes, we can check the results:

Successful execution of SSIS package when loading data from Snowflake to SQL Server

If we go to Snowflake and execute the SELECT query, we see all 10M customers loaded:

Successful loading of 10 million rows from SQL Server to Snowflake using SSIS and ZappySys SSIS PowerPack

All done in less than 3 mins:

Conclusion

We achieved just what we aimed for – we loaded 10 million rows from SQL Server to Snowflake; all done in less than 3 minutes! Two approaches were taken into consideration: in one we staged CSV files in the local Snowflake stage and in the other we staged files in the Amazon S3 bucket. To accomplish this we used ZappySys Export CSV Task, which allowed us to export data from the SQL Server view to CSV format, split files into smaller ones, zip them and store them locally or upload to the Amazon S3 bucket. Finally, we used a standard SSIS Task “Execute Process Task” to issue Snowflake commands such as creating the stage area in Snowflake, uploading CSVs into the local stage and loading data into the Snowflake table.

Download a sample package

Load 10M rows from SQL Server into Snowflake.dtsx.zip

References

https://docs.snowflake.net/manuals/user-guide/data-load-bulk.html

https://docs.snowflake.net/manuals/user-guide/snowsql.html

https://docs.snowflake.net/manuals/user-guide/data-load-s3-create-stage.html

https://docs.snowflake.net/manuals/sql-reference/sql/create-file-format.html

https://docs.snowflake.net/manuals/sql-reference/sql/put.html

https://docs.snowflake.net/manuals/sql-reference/sql/copy-into-table.html

The post Load 10M rows from SQL Server to Snowflake in 3 minutes appeared first on ZappySys Blog.

There might be a case when you have many CSV, JSON or XML files in Amazon S3 bucket and you want them to be imported straight into a SQL Server table. Here come ZappySys ODBC PowerPack and ZappySys Data Gateway (part of ODBC PowerPack) which will enable you to accomplish that. ZappySys ODBC PowerPack includes powerful Amazon S3 CSV, Amazon S3 JSON and Amazon S3 XML drivers that let you connect to an S3 bucket and read the contents from the files in the bucket. Bringing in ZappySys Data Gateway allows doing that right from a SQL Server. Let’s begin and see how to import Amazon S3 files into SQL Server.

These drivers are related to this article:

Amazon S3 Driver (for XML Files)

Amazon S3 Driver (for JSON Files)

Amazon S3 Driver (for CSV Files)

Prerequisites

1. Have an AWS account.
2. Existing Amazon S3 bucket.
3. SQL Server instance installed (can be a SQL Server Express LocalDB instance).
4. SQL Server Management Studio (SSMS) installed.
5. ZappySys ODBC PowerPack installed (must be installed on a Windows machine; can be a different machine than SQL Server is installed on).

Getting started

We will start from a simple example of how to load contacts from a JSON that are located in Amazon S3 bucket, then we will proceed by loading contacts from many XML files that are compressed in ZIP, and finally, we will traverse many folders and subfolders to gather CSVs and load contacts from them.

Step-by-Step – Import JSON file located at Amazon S3 into SQL Server

Open and configure ZappySys Data Gateway

Create an Amazon S3 JSON data source in ZappySys Data Gateway

1. The first thing you will have to do is to create a data source in ZappySys Data Gateway. Just click Add button, give the data source a name, e.g. “MyContactsJSON“, and then select Native – ZappySys Amazon S3 JSON Driver:
   

   Adding a JSON data source in ZappySys Data Gateway to load it from Amazon S3 bucket into SQL Server

2. Then click Edit and add the Data Gateway user you created in Users tab. We will use this user later when adding a Linked Server to the Data Gateway to authenticate:
   

   Adding a user to the newly created Amazon S3 JSON data source

3. Once you do that, then click Edit to configure the data source:
   

   Preparing to configure the Amazon S3 JSON data source to load a JSON from Amazon S3 bucket to SQL Server

4. When a window opens, click Click here to Configure the Connection and enter your Access Key and Secret Key:
   

   Configuring the authentication to Amazon S3 bucket in ZappySys Data Gateway

5. After that, select a JSON file you want to load, and then click Select Filter button to choose data you want to be displayed in SQL Server:
   

   Configuring the JSON data source to load a JSON from Amazon S3 bucket into SQL Server

6. Go to the Preview tab and click the Preview Data button to make sure everything is configured correctly and preview the results:
   

   Previewing JSON data in the data source based on Amazon S3 JSON Driver

Now you are ready to create a Linked Server and connect to the data source you just created. If you used “MyContactsJSON” as the data source name, make sure to use the same name when creating a Linked Server.

Set up a SQL Server Linked Server

Execute the SQL query

Once you created the Linked Server to ZappySys Data Gateway, you are ready to execute the SQL query and load data into SQL Server. Supposedly, you created the Linked Server with the name “GATEWAY”, then open SSMS and execute this query:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see similar results after you execute the query:

The results of getting JSON data straight from Amazon S3 bucket using ZappySys Amazon S3 JSON driver

Now we are ready to proceed to the next section and import many XML files at once.

Step-by-Step – Import many XML files located at Amazon S3 into SQL Server

Overview

Supposedly, you have many XMLs where each one is zipped and you want to load them all into a SQL Server table:

Zipped XML files located in Amazon S3 bucket to be loaded into SQL Server

The first thing you will have to do is to create a data source, based on ZappySys Amazon S3 XML Driver. Let’s proceed and just do that.

Create an Amazon S3 XML data source in ZappySys Data Gateway

Follow the same steps as in Create an Amazon S3 JSON data source in ZappySys Data Gateway section when adding a new data source, except for these two steps:

1. When adding a data source select Native – ZappySys Amazon S3 XML Driver as Connector Type:
   

   Choosing “ZappySys Amazon S3 XML Driver” to load XMLs to SQL Server

2. Then click Edit and edit the data source similarly like in this window:
   

   Configuring data source based on ZappySys Amazon S3 XML Driver to load XMLs into SQL Server

Now we are ready to set up a Linked Server to this newly created data source.

Set up a SQL Server Linked Server

Follow the very same steps from the section above where we added the Linked Server to a JSON data source, except that change Datasource property of the Linked Server to match the name of the Amazon S3 XML data source you created. Once you do that, you are ready to load XMLs into your SQL Server.

Execute the SQL query

Again, if you created the Linked Server with name “GATEWAY”, execute this SQL query in SSMS:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see a similar view once you execute the query:

The data of many XMLs loaded from Amazon S3 bucket into SQL Server

Now we are ready to move to the next section and recursively scan CSVs and load them into SQL Server.

Step-by-Step – Import CSV file located at Amazon S3 into SQL Server

Overview

Let’s say you have many CSVs that are located in a folder tree, including subfolders and you want them all in your SQL Server:

Many CSV files located in Amazon S3 bucket folders and subfolders to be loaded into SQL Server

We will follow the same steps to accomplish that as we did for loading JSON and XMLs.

Create an Amazon S3 CSV data source in ZappySys Data Gateway

Follow the steps as in Create an Amazon S3 JSON data source in ZappySys Data Gateway section when adding a new data source, except for these two steps:

1. When adding a CSV data source select Native – ZappySys Amazon S3 CSV Driver as Connector Type:
   

   Choosing “ZappySys Amazon S3 CSV Driver” to load CSVs to SQL Server

2. Then click Edit and edit the data source similarly:
   

   Configuring data source based on ZappySys Amazon S3 CSV Driver to load CSVs into SQL Server

Now we are ready to proceed and add a Linked Server to this data source.

Set up a SQL Server Linked Server

Again, follow the very same steps from the section above where we added the Linked Server to a JSON data source, except that change Datasource property of the Linked Server to match the name of the Amazon S3 CSV data source we created. Once you do that, you are ready to load CSVs into your SQL Server.

Execute the SQL query

Once again, if you created the Linked Server with name “GATEWAY”, execute this SQL query in SSMS:

SELECT * INTO MyContacts FROM openquery([GATEWAY], 'SELECT * FROM $')
SELECT * FROM MyContacts

The results

You should see a similar view, once you execute the query:

The results of loading many CSVs from Amazon S3 bucket into SQL Server

Conclusion

We had a goal to import Amazon S3 files into SQL Server from SQL Server itself. We learned how to load JSON, XML and CSV files into SQL Server, which some of them were zipped and were located at different folders and subfolders. It was possible to accomplish that using ODBC PowerPack, ZappySys Data Gateway and ODBC PowerPack drivers – Amazon S3 JSON Driver, Amazon S3 XML Driver, and Amazon S3 CSV Driver. At first, we created data sources in the Data Gateway, then we created Linked Servers in SQL Server and connected to ZappySys Data Gateway, which finally made it possible to load data from Amazon S3 bucket straight from SQL queries in SSMS.

Check Amazon S3 Integration with Other BI Tools (Power BI, Excel, SSRS, MS Access, etc.)

ZappySys ODBC Drivers built using ODBC standard which is widely adopted by industry for a long time. Which mean the majority of BI Tools / Database Engines / ETL Tools already there will support native / 3rd party ODBC Drivers. Below is the small list of most popular tools / programming languages our Drivers support. If your tool / programming language doesn't appear in the below list, which means we have not documented use case but as long as your tool supports ODBC Standard, our drivers should work fine.  

BI / Reporting Tools Integration	ETL Tools Integration	Programming Languages Integration
Microsoft Power BI Tableau SSRS (SQL Reporting Services) QlikView /Qlik Sense MicroStrategy Google Sheet Microsoft Excel Microsoft Access Oracle OBIEE Many more (not in this list).....	Informatica PowerCenter (Windows) Informatica Cloud SSIS (SQL Integration Services) SQL Server Talend Data Studio Pentaho Kettle Oracle OBIEE Many more (not in this list).....	Visual Studio C# C++ JAVA Python PHP PowerShell T-SQL (Using Linked Server)

 

The post Import Amazon S3 files into SQL Server (CSV/JSON/XML Driver) appeared first on ZappySys Blog.

In our previous blog we saw how to load data into Amazon S3. Now in this blog, we will see How to read Amazon S3 Storage Files in SSIS (CSV, JSON, XML Format files). To illustrate, we will use ZappySys SSIS PowerPack, which includes several tasks to import/export data from multiples sources to multiple destinations like flat files, Azure, AWS, databases, Office files and more. They are Coding free, drag and drop high-performance suite of Custom SSIS Components and SSIS Tasks. If you like perform File operations on Amazon S3 Files (e.g. Download, Upload, Create, Delete) then check these articles.

In nutshell, this post will focus on how to Read Amazon S3 Storage CSV, JSON and XML Files using respective SSIS Source tasks.

Components Mentioned in this article

	Amazon S3 CSV File Source
	Amazon S3 CSV File Destination
	Amazon S3 XML File Source
	Amazon S3 JSON File Source

Prerequisite

1. First, you will need to have SSIS installed
2. Secondly, make sure to have SSDT
3. You have obtained Amazon S3 account access key / secret key.
4. Finally, do not forget to install ZappySys SSIS PowerPack

What is Amazon S3 Storage

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides easy-to-use management features so you can organize your data and configure finely-tuned access controls to meet your specific business, organizational, and compliance requirements. Amazon S3 is designed for 99.999999999% (11 9’s) of durability, and stores data for millions of applications for companies all around the world.

Getting Started

In order to start, we will show several examples. ZappySys includes an SSIS Amazon S3 Source for CSV/JSON/XML File that will help you in reading CSV, JSON and XML Files from Amazon S3 to the Local machine, Upload files(s) to Amazon S3 Storage. It will also support Delete, Rename, List, Get Property, Copy, Move, Create, Set Permission … and many more operations. Here we are showing you is, How to download files from Amazon S3 Storage.

You can connect to your Amazon S3 Account by entering your storage account credentials.

Read Amazon S3 Storage Files in SSIS (CSV, JSON, XML)

Let´s start with an example. In this SSIS Amazon S3 Source for CSV/JSON/XML File task example, we will read CSV/JSON/XML files from Amazon S3 Storage to SQL Server database.

1. First of All, Drag and drop Data Flow Task from SSIS Toolbox and double click it to edit.
   

   Drag and Drop SSIS Data Flow Task from SSIS Toolbox

2. Drag and Drop relevant Amazon S3 Source for CSV/JSON/XML File Task from the SSIS Toolbox.
   

   Add Amazon S3 Source Tasks

3. Create a connection for Amazon S3 Storage Account.
   

   Create Amazon S3 Storage Connection

4. Select the relevant single file to read from Amazon S3 Storage in their relevant source of CSV/JSON/XML File Task.
   

   Select File From Amazon S3 Storage

5. We can also read the multiple files stored in Amazon S3 Storage using wildcard pattern supported e.g. dbo.tblNames*.csv / dbo.tblNames*.json / dbo.tblNames*.xml in relevant source task
   

   Use wildcard pattern .* to read multiple files data

6. We can also read the zip and gzip compressed files also without extracting it in the specific Amazon S3 Source for CSV/JSON/XML File Task.
   

   Reading zip and gzip compressed files (stream mode)

7. Finally, we are ready to load this file(s) data into the SQL Server.

Load Amazon S3 files data into SQL Server

ZappySys SSIS PowerPack makes it easy to load data from various sources such as REST, SOAP, JSON, XML, CSV or from other source into SQL Server, or PostgreSQL, or Amazon Redshift, or other targets. The Upsert Destination component allows you to automatically insert new records and update existing ones based on key columns. Below are the detailed steps to configure it.

Step 1: Add Upsert Destination to Data Flow

1. Drag and drop the Upsert Destination component from the SSIS Toolbox.
2. Connect your source component (e.g., JSON / REST / Other Source) to the Upsert Destination.

SSIS - Data Flow - Drang and Drop Upsert Destination Component

Step 2: Configure Target Connection

1. Double-click the Upsert Destination component to open the configuration window.
2. Under Connection, select an existing target connection or click NEW to create a new connection.
   • Example: SQL Server, or PostgreSQL, or Amazon Redshift.

Step 3: Select or Create Target Table

1. In the Target Table dropdown, select the table where you want to load data.
2. Optionally, click NEW to create a new table based on the source columns.

Configure SSIS Upsert Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS

Step 4: Map Columns

1. Go to the Mappings tab.
2. Click Auto Map to map source columns to target columns by name.
3. Ensure you check the Primary key column(s) that will determine whether a record is inserted or updated.
4. You can manually adjust the mappings if necessary.

SSIS Upsert Destination - Columns Mappings

Step 5: Save Settings

• Click OK to save the Upsert Destination configuration.

Step 6: Optional: Add Logging or Analysis

• You may add extra destination components to log the number of inserted vs. updated records for monitoring or auditing purposes.

Step 7: Execute the Package

• Run your SSIS package and verify that the data is correctly inserted and updated in the target table.

SSIS Upsert Destination Execution

Conclusion

Above all, in this blog, we learned how to Read Amazon S3 Storage Files in SSIS. We used Amazon S3 Source for CSV file, Amazon S3 Source for JSON file and Amazon S3 Source for XML file to read the file(s) from Amazon S3 Storage and load data into SQL server. You can download SSIS PowerPack here to try many other scenarios not discussed in this blog along with 70+ other components.

References

Finally, you can use the following links for more information:

• SSIS Amazon S3 Source for CSV/JSON/XML File
• Introduction to Amazon S3 Storage Service
• How to Get Your Amazon AWS Access Key & Secret Key

The post Read Amazon S3 Storage Files in SSIS (CSV, JSON, XML) appeared first on ZappySys Blog.

In our previous post we explored unlimited possibilities to call Amazon AWS API using SSIS. Now in this post we will learn how to import / export data from Amazon Athena using SSIS. Amazon Athena is very similar service like Google BigQuery which we documented already.  To read data from Amazon Athena we will use ZappySys JSON / REST API Source which supports excellent JSON parsing capability like this.

Prerequisites

Before we do hello world demo for calling Amazon AWS API, you will need to make sure following prerequisites are met.

1. SSIS designer installed. Sometimes it is referred as BIDS or SSDT (download it from Microsoft site).
2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
3. Access to valid AWS credentials (Access Key, Secret Key for your IAM User). Click here to learn more about IAM users and Access Key/Secret Key
4. Make sure SSIS PowerPack is installed. Click here to download.

Steps needed to read data from Amazon Athena

To read data from Amazon Athena you need to do atleast 3 steps at very high level. Later on this article we will see each step in depth.

1. Call StartQueryExecution API to send SQL Query (Its Job Style API so this call starts Job)
2. Keep checking Job Status by calling  GetQueryExecution API (In the response check State field for Status – We need to look for word SUCCESS)
3. Once Job is finished it produces CSV file in S3 Bucket (Output Location is supplied in Step-1)

Read Amazon Athena Query output in SSIS (S3 CSV files)

AWS Console – Athena Query Editor / Configure Output Settings

Few things you might have to setup before you can call Athena API in SSIS

1. Make sure your IAM user has correct permission (We used existing policies for ease of use but you can use custom policy for precise permission set)
2. Configure default output location for your Athena queries

Configure IAM user permission to call Amazon Athena API

Set Athena Query Output Location

Testing Athena Query inside Amazon AWS Console (Query Editor)

Creating Connection to call Athena API in SSIS

Any API calls made to Amazon Athena service using ZappySys JSON / REST API Source  or  ZS REST API Task needs OAuth connection.

For REST API Task configure as below (More details about using REST API Task is in the next section).

1. Drag ZS REST API Task from Control Flow Toolbox
2. Double click to edit.
3. Change Request URL Access Mode to [Url from Connection]
4. Now select <<New OAuth Connection>> from connection drop down
5. On OAuth connection select Amazon AWS API v4 as Provider
6. Enter your Access Key and Secret Key

Create SSIS Connection for Amazon AWS API Call (Athena REST API Example)

Creating Table in Amazon Athena using API call

For this demo we assume you have already created sample table in Amazon Athena. If you wish to automate creating amazon athena table using SSIS then you need to call CREATE TABLE DDL command using ZS REST API Task.

1. In the previous ZS REST API Task select OAuth connection (See previous section)
2. Enter Request Url as below
   
   

   https://athena.us-east-1.amazonaws.com/

3. In the HTTP Headers enter below 2 headers
   

   X-Amz-Target: AmazonAthena.StartQueryExecution
   Content-Type: application/x-amz-json-1.1

4. Change Request method to POST
5. In the Body enter below
   

   {
    "ResultConfiguration": {
    "OutputLocation": "s3://my-bucket/output-files/"
    },
    "QueryString": "create external table tbl01 (CustomerID STRING, CompanyName STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION 's3://my-bucket/input-files/';",
    "ClientRequestToken": "{{System::ExecutionInstanceGUID}}"
   }

   Notice: Rather than using hard coded  DDL SQL Statement we can use variable placeholder (e.g.

   "QueryString": "{{User::Query,FUN_JSONENCODE}}"

      ). Use 

   FUN_JSONENCODE

    to replace new line with \r\n else JSON becomes invalid.
6. Now click Test Request / Response. This will create a new Athena table which we will query in later section.

Notice that in above Body ClientRequestToken is a unique number each time you have to call StartQueryExecution API to make it simple we used SSIS system variable System::ExecutionInstanceGUID. Its unique for each execution. You can also use newid() function from SQL Server and save result to variable if you like.

Here is the screenshot of above Request

RESt API Task – Create table in Amazon Athena using SSIS (StartQueryExecution API Call)

Uploading data for Amazon Athena (Source files)

Once you have athena table created, we will need some data to query. Loading data to amazon athena table is nothing but upload files to S3. If you noticed we added below line in CREATE TABLE statement in the previous section. This means all source files are located at this S3 location.

LOCATION 's3://my-bucket/input-files/'

We have written blog post to explain this process (Click Here – How to load data from SQL server to S3 files).

Extract data from Amazon Athena using SSIS (Query Athena)

Now its time to read some data by writing SQL query for Athena data (i.e. S3 files). As we mentioned earlier, reading data from Athena can be done using following steps.

1. Call  StartQueryExecution API  using REST API Task to execute your Query (It returns you QueryExecutionID). Use same technique as Create Table example. Replace CREATE Table command in body with your SQL Query (New line must be replaced by \r\n in JSON).
2. Use REST API Task to call GetQueryExecution API and configure periodic status check loop (Task keeps waiting until Success or Failure Value not found in Response)
3. Once data is available read data using S3 CSV File Source
4. At last you can clean up query output files or keep it if some other process wants to read it (You can also clean up S3 files using bucket retention policy automatically at scheduled interval).

Now lets see each step in depth

Step1-Start Amazon Athena Query Execution

Now first thing is to execute Athena Query by calling StartQueryExecution API . Below steps are almost same steps as we saw in section Creating Table in Amazon Athena using API call

Only part is different here is SQL query is SELECT query rather than CREATE TABLE.

1. Drag and drop ZS REST API Task from SSIS toolbox
   
2. Enter Request Url as below
   
   

   https://athena.us-east-1.amazonaws.com/

3. In the HTTP Headers enter below 2 headers
   

   X-Amz-Target: AmazonAthena.StartQueryExecution
   Content-Type: application/x-amz-json-1.1

4. Change Request method to POST
5. In the Body enter below
   

   {
     "ResultConfiguration": {
       "OutputLocation": "s3://my-bucket/output-files/"
     },
     "QueryString": "select * from sampledb.elb_logs\r\nlimit 100000",
     "ClientRequestToken": "{{System::ExecutionInstanceGUID}}"
   }

   Notice: Rather than using hard coded  SQL Query we can use variable placeholder (e.g.

   "QueryString": "{{User::Query,FUN_JSONENCODE}}"

      ). Use 

   FUN_JSONENCODE

    to replace new line with \r\n else JSON becomes invalid.

   

6. Now go to Response Settings Tab and configure below way to extract QueryExecutionId (we will need it later).

   Set Extract Type as Json.
   Enter Expression as

   $.QueryExecutionId

   Check Save output to Variable and create new Variable (we can name as QueryExecutionId )
   

   Call Athena SQL, Save Amazon Athena QueryExecutionId in SSIS Variable

7. That’s it now we can move to next step

Step2 – Wait until Athena Query Execution is done

New version of ZS REST API Task includes Status check feature. This feature avoids complex looping logic. Here is how to configure to check Job Status (Wait until Query execution is finished). Steps are almost same as previous section except three things ( Headers, Body and Status Check Tab)

For Status check we will call GetQueryExecution API .

1. Drag and drop ZS REST API Task from SSIS toolbox
   
2. Enter Request Url as below
   
   

   https://athena.us-east-1.amazonaws.com/

3. In the HTTP Headers enter below 2 headers
   

   X-Amz-Target: AmazonAthena.GetQueryExecution
   Content-Type: application/x-amz-json-1.1

4. Change Request method to POST
5. In the Body enter below
   

   {"QueryExecutionId": "{{User::QueryExecutionId}}"}

   Notice: Here we have used Variable in the body. This variable we extracted in the previous step. It contains QueryExecution ID for the Job we submitted.
   

   SSIS Amazon Athena – Call GetQueryExecution API (Status Check)

6. Now go to Status Check Tab and configure below way to implement Status check loop. We want to stop check once status is SUCCESS or FAILED or CANCELLED. If we find FAILED or CANCELLED in State field then we must fail the step with error.
   1. Check Enable Status Check
   2. Enter Success Value as

      SUCCEEDED

   3. Check Fail Task option and enter this regex in the text box 

      FAILED|CANCELLED

   4. In the Interval enter 5 seconds delay after each iteration.
7. That’s it. Status check Tab will look like below. Now we can move to next step
   

   REST API Status check Loop – Keep checking until JOB is done

Step3-Read data from Athena Query output files (CSV / JSON stored in S3 bucket)

When you create Athena table you have to specify query output folder and data input location and file format (e.g. CSV, JSON, Avro, ORC, Parquet …) they can be GZip, Snappy Compressed. Once you execute query it generates CSV file. Zappysys can read CSV, TSV or JSON files using S3 CSV File Source or S3 JSON File Source connectors. In this section we will use CSV connector to read Athena output files.

Here is how to Athena output data

1. Drag and drop Data flow task from SSIS Toolbox
   
2. Double click data flow and drag and drop  ZS Amazon S3 CSV File Source
   
3. Double click to configure it.
4.  Click New Connection and configure S3 Connection
5. Once connection is created, browse to S3 bucket / folder location where Athena outputs files.
6. Replace file name with variable like below
   

   my-bucket/output-files/{{User::QueryExecutionId}}.csv

    

Reading data from Amazon Athena CSV files (Stored in S3)

Putting all together – Athena Data Extract Example

Here is final package with all pieces we talked.

Conclusion

You have seen that using ZappySys SSIS PowerPack how quickly you can integrate with Amazon Athena and other AWS Cloud Services.  Download SSIS PowerPack and try it out by yourself.

Keywords: Extract data from Amazon Athena | Read from Amazon Athena Query file | Import / Export data from Amazon Athena | Fetch data from Amazon Athena

The post Import / Export data from Amazon Athena using SSIS appeared first on ZappySys Blog.

In this blog post you will learn how to call Amazon AWS API using SSIS (virtually any API) without a single line of code (No more JAVA, C#, Ruby, Python). Yes you heard it right 🙂 . If you are a SSIS / ETL Developer or even coder everyone loves drag & drop interface. SSIS has many advantages over other approaches such as Programming SDKs, Command Lines. Main advantage is ease of use, security and long term maintenance without learning expensive coding approach.

In this post we will use following components to show various possibilities to implement Amazon AWS API integration scenarios inside your ETL workflows.

	JSON / REST API Source (REST API, JSON File or OData Service): Use this dataflow component when you have to fetch data from REST API webservice like a table. This component allows you to extract JSON data from webservice and de-normalize nested structure so you can save to Relational database such as SQL Server or any other target (Oracle, FlatFile, Excel, MySQL). This component also supports reading local JSON files or direct JSON string (Wildcard pattern supported too e.g. c:\data\file*.json).
	Web API Destination (POST data to API URL) : Use this dataflow component when you have to call API inside Dataflow (see POST to URL). Possible Use case: Call Lambda function with various parameters for each input record found in database.
	REST API Task : Use this task when you don’t want to pull REST API data in tabular format but want to call rest API for POST data to server, DELETE data from server or things like download HTML page, extract Authentication tokens etc where you not necessarily dealing data in tabular format. This task also allows you many other options such as saving RAW response into variable or file.
	XML Source (SOAP, File, REST) : Use this dataflow component when you have to fetch data from XML or SOAP webservice and consume data like a table. This component allows you to extract data from webservice and save to SQL Server or any other target (Oracle, FlatFile, Excel, MySQL). This component also supports reading local XML files or direct XML string.

Prerequisites

Before we do hello world demo for calling Amazon AWS API, you will need to make sure following prerequisites are met.

1. SSIS designer installed. Sometimes it is referred as BIDS or SSDT (download it from Microsoft site).
2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
3. Access to valid AWS credentials (Access Key, Secret Key for your IAM User). Click here to learn more about IAM users and Access Key/Secret Key
4. Make sure SSIS PowerPack is installed. Click here to download.

Step-By-Step Example-1 (Call AWS API)

Now lets call some simple GET API call using SSIS REST API Task.

1. Install  SSIS PowerPack (Skip this step if you already installed SSIS PowerPack.
2. Open Visual Studio and create new Integration Services Project
3. Open SSIS Package and check your SSIS Toolbox you will see many tasks/components starting with ZS
4. From Control flow SSIS Toolbox Drag & drop ZS REST API Task.
5. Double click the REST API task to configure it.
6. On REST API Task change URL Access mode drop down to URL from Connection
7. Now in the connection dropdown click New ZS-OAUTH connection and configure connection as below
   1. Once you see OAuth connection dialog box change Provider Type from Custom to Amazon AWS API (v4)
   2. In the ClientId enter your AWS Access Key
   3. In the ClientSecret enter your AWS Secret Key
   4. Click OK to save connection
      

      Configure SSIS OAuth Connection – Use Amazon AWS API Provider, Enter Access Key, Secret Key

8. On REST API Task change few more settings as below
   1. Enter API URL you like to call (In our case we will use S3 API (Simple Storage Service). We assume you have ListBucket permission to make this call. If you dont have such permission try to get Full path of File (choose small file) . You have to tweak API url to adjust Service Type, Region,Bucket, Path
      

      https://s3.us-east-1.amazonaws.com
      -- OR -- Use below (list files) if you have single bucket permission---
      https://s3.us-east-1.amazonaws.com/YOUR-BUCKET

   2. Click Test Request. If you have valid Permission and setup looks ok then you will see Response window like below. Using this technique you can call any API to execute AWS operations (E.g. start EC2 VM, Create SQS Queue, Call Lambda Function, Drop or Update resource)
      

      Calling Amazon AWS API Using SSIS REST API Task Example. (Call any AWS API without SDK. such as EC2 API, SQS API, Lambda API, Redshift API, DynamoDB API)

Step-By-Step Example-2 (Loading data from AWS API to SQL Server)

Now lets do more interesting scenario. We will call AWS S3 API to get S3 File list from Bucket. After extract we will save that list to SQL Server Table. Since Amazon S3 API is XML based API we will use ZappySys SSIS XML Source. For JSON based API use JSON Source instead. XML Source / JSON Source both can parse API response into Rows and Columns so you can easily store it into SQL Server. Now lets see how to do this.

1. From Control flow SSIS Toolbox Drag & drop Data Flow Task
2. Double click Data Flow Task. From SSIS Toolbox Drag & drop ZS SSIS XML Source.
3. Double click the XML Source to configure it.
4. In the URL text box enter API URL like below to list S3 Files for specified bucket (Change YOUR-BUCKET to your own name)
   

   https://s3.us-east-1.amazonaws.com/YOUR-BUCKET

5. Now check Use Credentials and Select same Amazon API connection we created in previous example.
6. Click on Select Filter (This step allows us to flatten the XML hierarchy. Select the node which is Array icon like below. If prompted to treat selected node as array click Yes.
   

   Configure SSIS XML Source – Get Amazon S3 File List from specific bucket (AWS API)

7. Click Preview to see your data. Click OK to save.
   

   Preview Data in XML Source – Amazon S3 File List (Calling AWS API)

8. Now attach your XML source to target like OLEDB Destination to load data to SQL Server or other Target (e.g. Oracle, MySQL)
9. Execute SSIS Package to load data from Amazon AWS API to SQL Server.
   

   SSIS Package – Read Amazon S3 File List and load into SQL Server Table (Call AWS API Example)

File Upload Example – Low level API – Call PUT request

There will be a time when you want to take total control of your AWS API calls. One example is if you wish to Upload / Write data to S3 then components like ZappySys Amazon S3 CSV Destination or Amazon Storage Task might need additional permission such as HeadObject . If you have only write permission on bucket then this will fail to execute Task. In such case you can use REST API Task like below way.

Basically we exported data from Relation Database to CSV File using Export CSV File Task and then we uploaded file content using REST API Task.

Upload Text File to S3 (i.e. JSON, CSV, XML …)

Your URL Format can be like this. Region codes can be found here

https://YOUR-BUCKET.s3.YOUR-REGION.amazonaws.com/YOUR-FILE
https://YOUR-BUCKET.s3.YOUR-REGION.amazonaws.com/SOME-FOLDER/SOME-SUB-FOLDER/YOUR-FILE

Upload file to Amazon S3 – Call low level API (PUT request)

Upload Binary File to S3 (i.e. Zip, mp3, gzip, png, jpeg…)

Above method only works for Text Files. If you have Binary files (e.g. Zip file, mp3, png, jpeg) then you can use below workaround.

1. Check Is Multi Part / File Upload Option next to the Body editor (Read this Post for more info)
2. In the Body enter file path like below
   

   @c:\folder\some-file.xyz

3. That’s it, now your binary file can be uploaded to S3 same way as Text file we uploaded in earlier section.

NOTE: Binary file option only works in the latest Build Uploaded after 5/21/2020

Debugging AWS API Command Line Requests using Fiddler (Web Proxy)

Before we see more examples of calling AWS API lets first learn how to capture Request data using aws command line (CLI). We will use Fiddler to capture AWS API Requests.

1. Download and Install Fiddler (Free Tool)
2. Install  aws command line
3. Open command prompt and type aws configure command to set credentials.  For more info see configure aws credentials .. see below example.
   

   c:> aws configure
   AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
   AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
   Default region name [None]: us-west-2
   Default output format [None]: json

4. Once you set credentials launch Fiddler
5. On the Fiddler Tools menu > Click Fiddler Option > HTTPS > Check Decrypt HTTPS Traffic
6. Once you do that you may be asked to Trust Fiddler Certificate click OK
7. Close and Open Fiddler to apply setting
8. Now any command you type in aws command line will show up in fiddler. This command we can use in SSIS REST API Task or XML Source to call virtually Any API AWS supports.
9. For example to call some lambda function (supply input json data from file) use below command and watch Fiddler Trace. Notice we added –no-verify-ssl option so we can see requests in custom web proxy like fiddler (This option will trust Fiddler certificate).
   

   c:\>aws lambda invoke --function-name HelloWorld c:\temp\outputfile.txt --no-verify-ssl --payload file://c://test/customer.json

    
   

   Capture AWS Commandline (CLI) request using Fiddler

10. Once you have this information you can use it inside ZappySys Components which supports API calls (e.g. REST API Task, JSON Source, XML Source). For this example we will use REST API Task to call same Lambda Function. Things to change to call any API is URL, Method, ContentType, Body. If its GET call then you wont have Body.
    

    Calling Amazon AWS Lambda Function using SSIS REST API Task

     

11. That’s it.. You can now take this same concept and call virtually API AWS API right insight SSIS without any SDK or command line tools.

Call API Gateway Endpoint (Default URL)

If you wish to call API hosted on Amazon API Gateway Service then enter direct URL. You have to use OAuth Connection (AWS v4 Provider) as previous section.

Typical URL may look like as below if you calling API gateway Endpoint.

https://c5hhigf5mh.execute-api.us-east-1.amazonaws.com/prod/pets

Call AWS API with Region and Custom Service Name

There will be a time when you will have to call AWS API URL and it doesnt indicate Region / Service but you have to supply part of signature for Authentication.  In this case just enter AWS Service and Region along with ClientID, Secret Fields (Custom Service and  Region attributes were introduced in v3.0 or higher so if you are not seeing on OAuth Connection UI then you probably running older version).

Call Amazon Athena API

Check this article for detailed instructions

Call AWS EC2 API

Here is an example of calling EC2 API. This is example of listing EC2 instances.

Request:

POST https://ec2.us-east-1.amazonaws.com/
Content-Type: application/x-www-form-urlencoded; charset=utf-8

Action=DescribeInstances&Version=2016-11-15

Call AWS Lambda API

Here is some example of calling Lambda function

Request:

POST https://lambda.YOUR-REGION.amazonaws.com/2015-03-31/functions/YOUR-FUNCTION/invocations

{you-json-input-goes-in-body}

Call AWS ElasticSearch

If you are using AWS Hosted Managed ElasticSearch then also you can use OAuth AWS v4 Provider as below. Assuming you have configured correct Policy to allow your IAM User Account / IP address.

Call AWS Hosted ElasticSearch REST API in SSIS (V4 Request Signing)

Call AWS API using Native Task/Components

ZappySys provides many High quality Tasks / Components for AWS integration. See below list.

Amazon AWS Cloud Integration
Amazon S3 Task	Amazon Redshift Data Transfer Task	Amazon Redshift ExecuteSql Task
Amazon Redshift Cluster Management Task	Amazon DynamoDB Source	Amazon DynamoDB Destination
Amazon Redshift Source	Amazon SQS Queue Source	Amazon SQS Queue Destination

Conclusion

Amazon AWS Cloud integration from SSIS packages becoming more and more common scenario. ZappySys provides easy to use no coding connectors to achieve many time consuming scenarios. Having clean drag and drop approach is not only faster but more secure using inbuilt SSIS framework. Try SSIS PowerPack to explore many other scenarios not discussed in this article.

The post How to call Amazon AWS API using SSIS (EC2, Lambda, API Gateway, SQS) appeared first on ZappySys Blog.

Sometimes if you loading data to Redshift via COPY command or using SSIS Redshift Data Transfer Task to move data to redshift then you will have to make sure your S3 Bucket is in the same region as Redshift Cluster. Many people don’t know how to verify that because its hidden in the console. This post will help you to find it easily.

Find Amazon S3 bucket location (AWS Region endpoint)

Below is step-by-step instructions how to find S3 Bucket region (i.e. location)

1. Open your AWS Console by visiting https://console.aws.amazon.com/
2. From dashboard click on S3 option (or visit https://console.aws.amazon.com/s3/home )
3. You will see all buckets in the left side list
4. Click on desired S3 bucket name
5. Click on Properties Tab at the top
6. Now you will see Region for the selected bucket along with many other properties.

Sometimes You may see US Standard as region name that means your bucket is located in US East (N. Virginia)

If you want to know Region Code rather than name check this link : Region Endpoints for Amazon S3

How to find amazon s3 bucket location (AWS region endpoint)

Download/Upload data to S3 bucket via Command line

If you want to automate S3 file download/upload then check this command line tool

ZappyShell Command line tools for Amazon S3

Upload local files to Amazon S3 from command line

List S3 buckets using command line

List S3 file information using command line

List S3 files using command line

Download/Upload data to S3 bucket from SSIS

Load data to Amazon Redshift using S3 Bucket and COPY command

Below video shows how to use S3 bucket to perform bulk data load into Amazon Redshift using SSIS Redshift Data Transfer Task

Conclusion

Amazon S3 provides most common way to store/load data for various AWS related services (e.g. EMR, Redshift). Zappysys is one of the top leading providers who make tools for AWS, NoSQL and other Cloud connectivity providers. Check SSIS PowerPack (For SSIS) or ZappyShell Command line for AWS if you want command line solution. ZappyShell is just single exe file does all heavy lifting for you 🙂

Cheers!

The post How to check Amazon S3 Bucket location (Region) appeared first on ZappySys Blog.