Introduction
In today’s data-driven world, the ability to seamlessly integrate and transfer files between different platforms is crucial for efficient business operations. If you’re a user of SQL Server Integration Services (SSIS) and looking to integrate SharePoint Online into your data workflow, you’re in luck. ZappySys SSIS PowerPack offers a powerful SharePoint Online Connector that makes uploading files to SharePoint Online a breeze. In this guide, we’ll walk you through the process of uploading files to SharePoint Online using SSIS and the ZappySys SSIS PowerPack SharePoint Online Connector.
Prerequisites
Before we perform the steps listed in this article, you will need to make sure the following prerequisites are met:- SSIS designer installed. Sometimes it is referred to as BIDS or SSDT (download it from the Microsoft site).
- Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.
- Make sure ZappySys SSIS PowerPack is installed (download it, if you haven't already).
- (Optional step). Read this article, if you are planning to deploy packages to a server and schedule their execution later.
In this section, we will learn how to configure and use SharePoint Online Connector in the API Source to upload files on SharePoint Online.
- Begin with opening Visual Studio and Creating a New Project.
- Select Integration Service Project and in the new project window set the appropriate name and location for the project. Click OK.
- In the new SSIS project screen you will find the following:
- SSIS ToolBox on the left sidebar
- Solution Explorer and Property Window on the right bar
- Control flow, data flow, event Handlers, Package Explorer in tab windows
- Connection Manager Window at the bottom
SSIS Toolbox – ZappySys Tasks
NOTE: If you don’t see ZappySys SSIS PowerPack Task or Components in the SSIS Toolbox, please refer to this help link. - Now, Drag and Drop the SSIS Data Flow Task from the SSIS Toolbox. Double-click on the Data Flow Task to see the Data Flow Designer.
- From the SSIS toolbox drag and drop API Source (Predefined Templates) on the data flow designer surface, and double-click on it to edit it:
- Select New Connection to create a new connection:
- Use a preinstalled SharePoint Online Connector from the Popular Connector List or press the Search Online radio button to download SharePoint Online Connector. Once downloaded simply use it in the configuration:
SSIS ZS API Connection Manager – Select SharePoint Online Connector
- Proceed with selecting the desired Authentication Type. Then select API Base URL (in most cases default one is the right one). Finally, fill in all the required parameters and set optional parameters if needed.
- After configuring all the required properties in the API Connection Manager, please click on the “Test Connection” button to confirm that the connection is established successfully.
You may press a link Steps to Configure which will help set certain parameters.
SSIS ZS API Connection Manager – Generate Token – SharePoint Online with User Credentials
You may press a link Steps to Configure which will help set certain parameters.
SSIS API Source – Generate Token – SharePoint Online with Application Credentials
In the API Source Component, we’ve chosen the API Connection Manager. Next, we need to configure the remaining properties of the component to upload a local file to SharePoint Online.
- Choose the Upload file endpoint from the “Select Table/Endpoint” dropdown menu.
- Select the desired Drive ID from the DriveId drop-down.
- Specify the desired source file path for the file being uploaded by setting the DiskFilePath parameter.
- Specify the desired Target Path by setting the SharePointFilePath parameter.
Please refer to the sample screenshot below.
SSIS API Source – SharePoint Online – Upload Single File
Once you have configured all the mentioned properties above, please click on the “Preview Data” button to initiate the API call. With just a few clicks, we have successfully configured the call to SharePoint Online using the ZappySys SharePoint Online Connector for uploading a local file to SharePoint Online.
In the API Source Component, we’ve selected the API Connection Manager. Next, we need to configure the remaining properties of the component to upload multiple files to SharePoint Online. Additionally, we should utilize a Foreach Loop Container to iterate through each file one by one in a loop.
- Create 3 SSIS variables as described below.
Variable Name Datatype Value varFileLists Object System.Object varFileName String fake.extension varFullFilePath String C:/fake/fake.extension - Go to Control Flow, From the SSIS toolbox drag and drop ZS Advanced File System Task on the control flow designer surface, and double click on it to edit it:
- Select the “Get file list as ADO.NET Data Table” action from the Action drop-down menu. Set the file path in the Path textbox. Choose the variable (Object type) where you want to store the file list, and then click the OK button to save the settings.
SSIS Advanced File System Task – Get file list as ADO.net Data Table
- Now, drag and drop the “Foreach Loop Container” from the SSIS Toolbox and connect it with the “ZS Advanced File System Task.” Double-click on the “Foreach Loop Container” to edit its properties.
SSIS Foreach Loop Container – Drag & Drop
- In the Foreach Loop Editor’s “Collection” tab, choose the Enumerator as “Foreach ADO Enumerator.” In the “ADO object source variable” field, select your SSIS object-typed variable. Select the “Rows in the first table” radio option.
SSIS Foreach Loop Editor – Collection – Select Variable & Enumeration mode
- In the Foreach Loop Editor’s “Variable Mappings” tab, select the variables that you want to map to the values from the collection, and then click the “OK” button to save your selections.
SSIS Foreach Loop Editor – Variable Mappings
- Inside the Foreach Loop Container, drag and drop the Data Flow Task, and then rename it according to your preference.
SSIS Data Flow Task – Drag & Drop inside Foreach Loop Container
- Double-click on the Data Flow Task to see the Data Flow Designer, From the SSIS toolbox drag and drop API Source (Predefined Templates) on the data flow designer surface, and double-click on it to edit it:
- Double-click on the API Source component to configure it for uploading multiple files to SharePoint Online.
- Choose the Upload file endpoint from the “Select Table/Endpoint” dropdown menu.
- Please specify the desired SSIS Variable [varFileName] containing the uploaded file name by setting the SharePointFilePath parameter.
- To specify the desired local SSIS variable [varFullFilePath], which holds the file path, set the DiskFilePath parameter.
When configuring the SSIS variable directly in the Raw Edit, JSON may become invalid due to the requirement of slashes within the local file path. To address this, we use a JSONENC Placeholder Function for proper escaping.
<<{{User::varUserFullFilePath}},FUN_JSONENC>> - If you wish to upload the local file to a specific folder in SharePoint Online, please set the folderName in the SharePointFilePath parameter.
Please refer to the sample screenshot below.
SSIS API Source – SharePoint Online – Upload Multiple files configuration
To configure dynamic filenames and file paths, you can set variables. Click on “Raw Edit,” and in the “Edit Parameters” popup, you can use SSIS variables, as shown in the above image.
Once you have configured all the mentioned properties above, please click on the “Preview Data” button to initiate the API call.
After that, you can execute the SSIS Package and check the execution to upload multiple files on SharePoint Online. Below is the sample screenshot attached.
Multiple files uploaded on SharePoint Online using SSIS
With just a few clicks, we have successfully configured the call to SharePoint Online using the ZappySys SharePoint Online Connector for uploading multiple files to SharePoint Online.
Conclusion
In this blog, we learned how to connect with SharePoint Online using SSIS API Connection Manager and upload single or multiple files to SharePoint Online using a combination of SSIS components, including the SSIS API Source component, SSIS ZS Advanced File System Task, Foreach Loop Container, and SSIS variables.
