There are several tricks related to converting SQL to CSV. For example, dates, working with variables, converting text, working with stored procedures. In this article, we will work with several tips to convert SQL Server data to the CSV format.

For this article, we will use the following ZappySys component:

SSIS Export CSV File Task

How to convert SQL to CSV using specific dates

1. First, to convert the date to a specific format, you can use the Export CSV task.
2. This task allows you to convert SQL Server tables into CSV files. We will show how to set the date to a specific format:
   

   Export from SSIS SQL Server to CSV in a specific date format
3. The query used for this purpose is the following:
   

   SELECT [OrderID] ,FORMAT([OrderDate], 'dd/MM/yyyy') as mydateformat FROM [dbo].[Orders]

4. As you can see, you can send custom queries in the Export CSV task.
   In this example, we utilize the T-SQL Format function, which enables us to convert the date to a custom format. In this example, we are converting the OrderDate column into the format dd/MM/yyyy.
5. Also, specify the destination path:
   

   Export path csv
6. Finally, run the package. In this example, we converted the date to the format dd/MM/yyyy.

How to convert SQL Server to CSV using dynamic column names

1. A common question is to have column names based on SSIS variables. This time, we will show how to do this.
2. First of all, we will need an SSIS variable that will contain the column name:
   

   SSIS variable with a column name
3. Secondly, we need to use the value of the variable as the column name. We will use the alias (AS in SQL) to create a custom name.
   

   SSIS column name

4. The SQL query will look like this:
   

   SELECT [ShipperID] as {{User::column}} FROM [Northwind].[dbo].[Shippers]

5. The previous example used the SSIS variable named column to store the name of the column. If the Variable value is ‘column1’, the name of the column will be ‘column1’.

How to convert SQL Server to CSV to change the decimal point to a decimal comma

1. Most of the countries in the world use the decimal point or the decimal comma. For decimals, some countries use the comma and others the period. In this new example, we will show how to convert decimal data into a comma.
2. In order to convert, we will convert and work with a text data type. We will use the following script to create a table with data:
   

   create table decimalpoint
   (
   decimal nvarchar(50)
   )
   
   insert into decimalpoint values 
   ('3,45'),
   ('3,44'),
   ('6,56')

3. Secondly, we will use the following T-SQL Sentence: select replace(decimal,’,’,’.’) as mydecimal from decimalpoint.
   

   select replace(decimal,',','.') from decimalpoint

4. In addition, we will create a path to export.
5. If everything is fine, the CSV file created will convert the period decimal to a coma decimal.

How to export data from a parameterized SQL Server stored procedure into a
CSV file

1. In this new example, we will use a stored procedure with parameters and store the results into a CSV file.
2.  First of all, we will use the following parameterized stored procedure:
   

   create procedure [dbo].[Sales by Year]
   @Beginning_Date DateTime, @Ending_Date DateTime AS
   SELECT Orders.ShippedDate, Orders.OrderID, "Order Subtotals".Subtotal, 
   DATENAME(yy,ShippedDate) AS Year
   FROM Orders INNER JOIN "Order Subtotals" 
   ON Orders.OrderID = "Order Subtotals".OrderID
   WHERE Orders.ShippedDate Between @Beginning_Date And @Ending_Date
   GO

3. Secondly, we will invoke the stored procedure like this:
   

   Invoice stored procedure and export to csv

4. Finally, if everything is fine, the data will be exported to the CSV file after running the package.

Conclusion

In this article, we demonstrate how to export data to CSV from SQL Server using various date formats. We also learned how to change the types of decimals used. For more fun, please download our components.

The post How to convert SQL to CSV in SSIS appeared first on ZappySys Blog.

ZappySys provide high-performance drag and drop connectors for Amazon Redshift Integration. In our previous post we saw how to bulk load SQL data into Redshift using S3 staging technique (COPY command). Now in this post, you will see How to Read / Write Amazon Redshift Data in SSIS Data flow without need for S3 Staging. We will use  ZS Amazon Redshift components (Source / Destination). If you need to Upsert / delete data from Redshift Table then use Upsert Destination (Not discussed in this article).

ZappySys developed many AWS related components but in this article, we will look at 2 Components for Amazon Redshift Integration Scenarios (Read and Write). We will discuss how to use SSIS Amazon Redshift Source and SSIS Amazon Redshift Destination. Amazon Redshift Source connector supports familiar SQL Like query language so any DBA or SQL user can learn it in no time. ZappySys developed a highly sophisticated engine to turn your SQL Query into native Amazon Redshift query. Load query at runtime from the variables or Direct value.

Prerequisite

Before we perform the steps listed in this article, you will need to make sure the following prerequisites are met:

1. First, you will need to have SSIS installed
2. Secondly, make sure to have SSDT
3. Thirdly, You have obtained Amazon Redshift Credential.
4. Finally, do not forget to install ZappySys SSIS PowerPack.

Component Mention in this article

	Amazon Redshift Source
	Amazon Redshift Destination

What is Amazon Redshift

Before we talk How to Read / Write Amazon Redshift Data in SSIS lets talk what is Amazon Redshift (or sometimes referred to as AWS Redshift). Amazon Redshift is a Cloud-based Data warehouse service. This type of system also referred to as MPP (Massively Parallel Processing). Amazon Redshift uses a highly modified version of PostgreSQL Engine behind the scene. Amazon Redshift provides the advantage of Scale as you go, at a very low cost compared to onsite dedicated hardware/software approach. In this article, we will try to learn How to Read / Write Amazon Redshift Data in SSIS.

Getting Started

In order to start, we will show several examples. ZappySys includes an Amazon Redshift Source and Destination, that will help you How to Read / Write Amazon Redshift Data in SSIS. You can connect to your Amazon Redshift Account by entering credentials.

SSIS Amazon Redshift Source Connector can be used to read data from Amazon Redshift. You can use a simple Table mode or write custom SQL Query to extract desired data. You can use multiple modes (i.e. Table or Query) to read data from the Amazon Redshift Database. Support for data preview and max rows and Dynamic query (using SSIS Variable placeholder e.g. select * from mytable where id='{{User:: MyVar}}’).

SSIS Amazon Redshift Destination can be used to bulk insert a large number of records to Amazon Redshift storage from any data source using SSIS. Amazon Redshift is a cloud-based Data Warehouse service from Amazon AWS. Here, you can bulk Insert, Update and Delete Amazon Redshift data and supporting the batch size.

Click here to see Amazon Redshift related articles.

Write Data using Amazon Redshift Destination in SSIS

Overview

In this tutorial, you will learn how to write data in Amazon Redshift Database. You can use ZS DummyData Source for practice it or select suitable ZS Source Adapter (In this case it’s from SQL Server Source). So, Let’s start with an example. In this SSIS we will write data into Amazon Redshift Database using ZS Amazon Redshift Destination.

How to write data into Amazon Redshift Table

1. In order to connect to the Amazon Redshift Database from SSIS, you will need credentials. Ask your SysAdmin or responsible person to provide that you.
2. For this sample exercise, we will create a table in Redshift.
   

   CREATE TABLE public."tbl_pg_Customers"
   (
       "RecordID" character varying(50) NOT NULL,
       "CustomerID" character varying(50),
       "CustomerName" character varying(50),
       "Address1" character varying(50),
       "City" character varying(50),
       "State" character varying(50),
       "Zip" character varying(50),
       "Country" character varying(50),
       PRIMARY KEY ("RecordID")
   );

3. Once you have created a Table, you may proceed to create a new SSIS package. In BIDS/SSDT create a new SSIS package.
4. In your SSIS toolbox, you should see many new Icons starting with “ZS” (i.e. ZappySys). If you don’t see them then check this.
5. Now, Drag and Drop SSIS Data Flow Task from SSIS Toolbox.
   

   Drag and Drop SSIS Data Flow Task from SSIS Toolbox

6. Double click on the Data Flow Task to see Data flow designer surface.
7. Here, In Visual Studio, Drag and Drop the OLE DB Source and Amazon Redshift Destination in the design surface and join the components with the blue arrow.
   

   SSIS OLE DB Source and Amazon Redshift Destination – Drag and Drop

8. Now, we need OLE DB Connection and Amazon Redshift Connection. Click here for Create Amazon Redshift Connection.

How to Create an OLE DB Connection.

1. Let’s, Right-click on Connection Managers Panel to Create OLEDB Connection, so you can use Source and Context Menu will appear, Select New OLEDB Connection from the Context Menu.
   

   SSIS OLE DB Connection – Create Connection

2. Now click on the New button to create a connection.
   

   SSIS OLE DB Connection – Create New Connection

3. Let’s configure connection Manager, just follow the steps one as we have created in following the image.
   

   SSIS OLE DB Connection – Configure Connection

4. After all, Click on OK button to save configure setting UI.

Configure SSIS ZS Amazon Redshift Destination

1. Double click on OLE DB Source for configure it.
2. Let’s Configure in Connection Manager, just follow below image steps.
   

   SSIS OLE DB Source – Configure Source Editors

3. Click OK to save OLE DB Source Editor UI settings.
4. Now double click on Amazon Redshift Destination to configure it. In the [Connection Manager] tab Select Amazon Redshift Connection.
   

   SSIS Amazon Redshift Destination – Connection Manager

5. Click on [Component Properties] tab and Pick the Destination Table from Tables dropdown list.
   

   SSIS Amazon Redshift Destination – Component Properties

6. Click on [Column mappings] tab and verify mapping. If the target table is new Table and it’s empty then all input columns will be automatically mapped as shown below. If the table is an existing table with some data then you can manually pick a mapping column by dragging it from source list to target list.
   

   SSIS Amazon Redshift Destination – Column Mappings

7. Execute the package and verify that the data is saved.
   

   SSIS Amazon Redshift Destination – Run or Execute Package

Read Data using Amazon Redshift Source in SSIS

Overview

In this tutorial, you will learn how to read data from Amazon Redshift in SSIS using ZS Amazon Redshift Source adapter. In this SSIS we will read data with SQL Query and Table Mode. So Let’s begin it.

How to Read data from Amazon Redshift Table

1. You may proceed to create a new SSIS package. In BIDS/SSDT create a new SSIS package.
2. From the SSIS toolbox drag and drop Data Flow Task on the control flow designer surface.
   

   SSIS Data Flow Task – Drag and Drop

3. Double click on the Data Flow Task to see the Data Flow design surface.
4. From the SSIS Toolbox, Drag and Drop Amazon Redshift Source in the data flow designer surface.
   

   SSIS Amazon Redshift Source – Drag and Drop

5. Now, We need Amazon Redshift Connection. Click here for creating Amazon Redshift Connection.

Configure SSIS ZS Amazon Redshift Source – Query Mode

1. Here, we are getting data using Query Mode with dynamic expression value.
2. Let’s Create Variables, In Visual Studio Right Click on Design Panel, Select Variables.
   

   SSIS Create Variables

3. Make sure correct DataType and Value, use the following image.
   

   SSIS Created Variables with Value

4. Double click on Amazon Redshift Source to configure it.
5. Select the connection we have created before, set AccessMode to Query and use the following SQL Query(You can replace table or column name) and insert Variables we have created before.
   

   SELECT  * FROM "public"."tbl_pg_Orders"
   WHERE "CustomerID" like '%R%' AND "OrderAmount" > '{{User::OrderAmount}}' AND
   "{{User::ColumnName}}" = '{{User::Value}}'

   

   SSIS Amazon Redshift Source – Configure with SQL Query Mode

6. Click on Preview button to see Data Preview and OK button to save configure setting UI.
7. Here, you can use any of our suitable ZS Destination Adapter.
8. From the SSIS toolbox drag and drop Trash Destination on the Data Flow designer surface.
   

   SSIS Trash Destination – Drag and Drop

9. Now single click on the Amazon Redshift Source, once you see the blue arrow from the source…connect it to Trash Destination.
10. Double click on Trash Destination to Configure it.
    

    SSIS Trash Destination – Configure

11. Click on OK button to save Trash Destination configure setting UI.
12. That’s all, just Run or Execute the package and verify data.
    

    SSIS Amazon Redshift Source – Run or Execute the Package

Configure SSIS ZS Amazon Redshift Source – Table Mode

1. Double click on Amazon Redshift Source to configure it.
2. Select the connection we have created before. Set Access Mode to Table and Select Table from table Drop down list.
   

   SSIS Amazon Redshift Source – Configure with Table Mode

3. Click on Preview button to see Data Preview and OK button to save Amazon Redshift Source configure setting UI.
4. From the SSIS toolbox, Drag and Drop Trash Destination on the Data flow designer surface.
   

   SSIS Trash Destination – Drag and Drop

5. Now single click on the Amazon Redshift Source, once you see the blue arrow from a source… connect it to Trash Destination.
6. Double click on ZS Trash Destination to configure it.
   

   SSIS Trash Destination – Configure

7. Click on OK button to save Trash Destination configure setting UI.
8. That’s all, just run or execute your package and see data.
   

   SSIS Amazon Redshift Source – Run or Execute Package

Conclusion

In this blog, we learn How to Read / Write Amazon Redshift Data in SSIS. We used Amazon Redshift Destination for write data and Amazon Redshift Source to read data. You can download SSIS PowerPack here to try many other scenarios not discussed in this blog along with 70+ other components.

References

Finally, you can use the following links for more information:

• Amazon Redshift Source
• Amazon Redshift Destination
• Document of Amazon Redshift
• Help File:
  • Amazon Redshift Source
  • Amazon Redshift Destination

The post How to Read / Write Amazon Redshift Data in SSIS appeared first on ZappySys Blog.

In this article, we will see how to read How to write data into CSV file in SSIS from SQL Server. We will also explain how to compress (gzip) and split  CSV files into multiple files. To illustrate, we will use ZappySys SSIS PowerPack, which includes several tasks to import/export data from multiples sources to multiple destinations like flat files, Azure, AWS, databases, Office files and more. They are Coding free, drag and drop high-performance suite of Custom SSIS Components and SSIS Tasks. We will use the ZS CSV File Destination component for this article.

What is CSV File Destination?

CSV File Destination can be used to write data in CSV / TSV file format. You can also split large files by row count or size at runtime. It also supports writing files directly in a compressed format such as GZip (*.gz).

Prerequisites

Before we perform the steps listed in this article, you will need to make sure the following prerequisites are met:

1. SSIS designer installed. Sometimes it is referred to as BIDS or SSDT (download it from Microsoft site).
2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
3. Make sure ZappySys SSIS PowerPack is installed (download it).

Component Mentioned in this article

CSV File Destination

How to Extract Data from SQL Server Table to CSV File Destination.

In this tutorial, we are going to load data from SQL Server Table using OLE DB Source and ZS CSV File Destination. You can use ZS DummyData Source too for practice it.

1. Firstly, you need to Download and Install SSIS ZappySys PowerPack.
2. Once you finished the first step, Open Visual Studio and Create New SSIS Package Project.
3. Now, Drag and Drop SSIS Data Flow Task from SSIS Toolbox.
   

   Drag and Drop SSIS Data Flow Task from SSIS Toolbox

4. Double click on the Data Flow task to see the Data Flow designer surface.
5. Here, In Visual Studio, drag and drop the OLE DB Source and CSV File Destination in the design surface and join the tasks with the blue arrow.
   

   SSIS OLE DB Source and CSV File Destination – Drag and Drop

6. Here, you can use our ZS Data Source too.
7. But, we are going through OLE DB Source so need OLE DB Connection and File Connection for CSV File Destination.

How to create OLE DB Connection.

1. Lets, create the first OLE DB Connection.
2. Let’s, Right-click on Connection Managers Panel to Create OLE DB Connection, so you can use Source and Context Menu will appear, Select New OLE DB Connection from the Context Menu.
   

   Create OLE DB Connection

3. Now, click on New Button to create Connection.
   

   Create New OLE DB Connection

4. Let’s Configure Connection Manager, just Follow steps one by one as we have created.
   

   Configure OLE DB Connection Manager

5. Click on Test Connection to see correct configure it.
6. Click on OK button to save connection configure setting UI.

How to Create a File Connection.

1. Let’s, Right-click on Connection Managers Panel to Create File Connection, so you can use Destination Path, and Context Menu will appear, Select New Connection from the Context Menu. In the File Connection Manager, select the Usage type and specify the path of the CSV file.
   CSV File: You can download good one CSV file from here for practice.
   

   https://zappysys.com/downloads/files/test/invoices.csv
   https://zappysys.com/downloads/files/test/invoices.csv.gz
   https://zappysys.com/downloads/files/test/invoices.csv.zip

   

   Create a New File Connection

2. Click on OK button to save connection configure setting UI.

Let’s see how to write data from SQL Server Table to CSV File.

1. Double click on OLE DB Source for configure it.
2. Let’s Configure in Connection Manager, just follow below image steps.
   

   Configure OLE DB Source Editor

3. Click OK to Save OLE DB Source Editor UI Settings.
4. Let’s configure CSV File Destination, Double click on it, In the File Connection Manager, select the file connection created before.
   

   SSIS ZS CSV File Destination – Configure Connection Manager Tab

5. In the Component Properties, set the Append property to False, enter the File Path and set the Overwrite file if exists to True.
   

   SSIS ZS CSV File Destination – Configure Component Properties Tab

6. In the Input Columns Tab, check the columns that you want as load data.
   

   SSIS ZS CSV File Destination – Configure Input column Tab

7. Click on OK button to save configure setting UI.
8. Finally, run the package and the data will be exported from source to destination immediately.
   

   SSIS ZS CSV FIle Destination – Execute the Package

How to do file split using CSV File Destination.

1. Double click on CSV File Destination to configure it.
2. In the Component Properties set Enable File Split to True and File Split Mode BySize.
   

   SSIS ZS CSV File Destination – File Split

3. Click OK button to save configure setting UI.
4. That’s all, Now you can run or execute the package and you will see numbers of CSV file in Destination Folder.

How to do file compression using CSV File Destination.

1. Double click on CSV File Destination to configure it.
2. In the Component Properties set Compression Type to GZip or Zip and File Split Mode BySize.
   

   SSIS ZS CSV File Destination – Compression Type

3. Click OK button to save configure setting UI. and You can Run or Execute.

How to do the maximum number of rows per file using CSV File Destination.

1. Double click on CSV File Destination to configure it.
2. In the Component Properties, you can set Max Rows Per File and Max Size Per File as per your need.
   

   SSIS ZS CSV File Destination – Max Rows Per File

3. Click OK button to save configure setting UI. and You can Run or Execute.

Conclusion

After all, we saw you how to write data into CSV File from SQL Server Table using OLE DB Connection, File Connection, and load into CSV File. We also learned, how to do various things like split CSV File, Compression File and the maximum number of rows per file of data. To explore many other scenarios not discussed in this article download SSIS PowerPack from here (includes 70+ Components).

References

Finally, you can use the following URL for more information.

• Help File: CSV File Destination

The post How to write data into CSV file in SSIS (GZip / Split) appeared first on ZappySys Blog.