Read / write YouTube data inside your app, perform many Youtube operations without coding using easy to use high performance API Connector for YouTube
In this article you will learn how to
quickly and efficiently integrate YouTube data in SSIS without coding.
We will use high-performance YouTube Connector
to easily connect to YouTube and then access the data inside SSIS.
Let's follow the steps below to see how we can accomplish that!
YouTube Connector for SSIS is based on ZappySys API Connector Framework
which is a part of SSIS PowerPack.
It is a collection of high-performance SSIS connectors that enable you
to integrate data with virtually any data provider supported by SSIS, including SQL Server.
SSIS PowerPack supports various file formats, sources and destinations,
including REST/SOAP API, SFTP/FTP, storage services, and plain files, to mention a few
(if you are new to SSIS and SSIS PowerPack, find out more on how to use them).
In this section we will learn how to configure and use YouTube Connector in API Source to extract data from YouTube.
Begin with opening Visual Studio and Create a New Project.
Select Integration Service Project and in new project window set the appropriate name and location for project. And click OK.
In the new SSIS project screen you will find the following:
SSIS ToolBox on left side bar
Solution Explorer and Property Window on right bar
Control flow, data flow, event Handlers, Package Explorer in tab windows
Connection Manager Window in the bottom
Note: If you don't see ZappySys SSIS PowerPack Task or Components in SSIS Toolbox, please refer to this help link.
Now, Drag and Drop SSIS Data Flow Task from SSIS Toolbox. Double click on the Data Flow Task to see Data Flow designer.
From the SSIS toolbox drag and API Source (Predefined Templates) on the data flow designer surface, and double click on it to edit it:
Select New Connection to create a new connection:
Use a preinstalled YouTube Connector from Popular Connector List or press Search Online radio button to download YouTube Connector. Once downloaded simply use it in the configuration:
YouTube
Proceed with selecting the desired Authentication Type. Then select API Base URL (in most cases default one is the right one). Finally, fill in all the required parameters and set optional parameters if needed. You may press a link Steps to Configure which will help set certain parameters. More info is available in Authentication section.
Steps how to get and use YouTube credentials
This walkthrough will help you create an API Key which you will be able to use in all requests and authenticate accordingly.
Click OAuth Consent Screen Tab. Enter necessary details and Save.
Click Credentials Tab
Click CREATE CREDENTIALS (some where in topbar) and select OAuth Client ID option.
When prompted Select Application Type as Desktop App and click Create to receive your ClientID and Secret. Later on you can use this information now to configure Connection.
Go to OAuth Consent Screen tab. Under Publishing Status click PUBLISH APP to ensure your refresh token doesnt expire often. If you planning to use App for Private use then do not have to worry about Verification Status after Publish.
Steps how to get and use YouTube credentials
Use these steps to authenticate as service account rather than Google / GSuite User. Learn more about service account here
Basically to call Google API as Service account we need to perform following steps listed in 3 sections (Detailed steps found in the help link at the end)
Create Project
First thing is create a Project so we can call Google API. Skip this section if you already have Project (Go to next section)
From the Project Dropdown (usually found at the top bar) click Select Project
On Project Popup click CREATE PROJECT
Once project is created you can click Select Project to switch the context (You can click on Notification link or Choose from Top Dropdown)
Click ENABLE APIs AND SERVICES
Now we need to Enable three APIs one by one (YouTube Data API, YouTube Analytics API, YouTube Reporting API)
Search YouTube Data API. Select and click ENABLE
Search YouTube Analytics API. Select and click ENABLE
Search YouTube Reporting API. Select and click ENABLE
Create Service Account
Once Project is created and APIs are enabled we can now create a service account under that project. Service account has its ID which looks like some email ID (not to confuse with Google /Gmail email ID)
From the Project Dropdown (usually found at the top bar) click Select Project
Enter Service account name and Service account description
For Role, do not select anything for now and Click Continue and then click Done. Next we will create Key.
Create Key
Once service account is created we need to create key file (i.e. credentials).
In the Cloud Console, click the email address for the service account that you created.
Click Keys.
Click Add key, then click Create new key.
Click Create and select P12 format. A P12 key file is downloaded to your computer. We will use this file in our API connection.
Click Close.
Now you may use downloaded *.p12 key file as secret file and Service Account Email as Client ID (e.g. some-service-account-name@your-project-id.iam.gserviceaccount.com ).
Add Permission
Now last thing is give read/write permission to Service Account. Basically you can create or open Google Sheet and add the Service Account as an editor to it as below.
Copy the email address of your service account we created in previous step (its usually like this some-service-account-name@your-project-id.iam.gserviceaccount.com).
Create or select an existing Google Sheet.
Navigate to Sheet for which you like to give read/write access to Service Account.
Click on the Share button in the top right, and add the email address of the service account as an editor. Here is how to share file(s) with specific people. Juse share with Service Account (use Service Account Email found on previous section)
In upper section we learned how to read data, now in this section we will learn how to configure YouTube in the API Source to POST data to the YouTube.
Read the data from the source, being any desired source component. In example we will use ZappySys Dummy Data Source component.
From the SSIS Toolbox drag and drop API Destination (Predefined Templates) on the Data Flow Designer surface and connect source component with it, and double click to edit it.
Select New Connection to create a new connection:
API Destination - YouTube
Read / write YouTube data inside your app, perform many Youtube operations without coding using easy to use high performance API Connector for YouTube
Use a preinstalled YouTube Connector from Popular Connector List or press Search Online radio button to download YouTube Connector. Once downloaded simply use it in the configuration:
YouTube
Proceed with selecting the desired Authentication Type. Then select API Base URL (in most cases default one is the right one). Finally, fill in all the required parameters and set optional parameters if needed. You may press a link Steps to Configure which will help set certain parameters. More info is available in Authentication section.
Steps how to get and use YouTube credentials
This walkthrough will help you create an API Key which you will be able to use in all requests and authenticate accordingly.
Click OAuth Consent Screen Tab. Enter necessary details and Save.
Click Credentials Tab
Click CREATE CREDENTIALS (some where in topbar) and select OAuth Client ID option.
When prompted Select Application Type as Desktop App and click Create to receive your ClientID and Secret. Later on you can use this information now to configure Connection.
Go to OAuth Consent Screen tab. Under Publishing Status click PUBLISH APP to ensure your refresh token doesnt expire often. If you planning to use App for Private use then do not have to worry about Verification Status after Publish.
Steps how to get and use YouTube credentials
Use these steps to authenticate as service account rather than Google / GSuite User. Learn more about service account here
Basically to call Google API as Service account we need to perform following steps listed in 3 sections (Detailed steps found in the help link at the end)
Create Project
First thing is create a Project so we can call Google API. Skip this section if you already have Project (Go to next section)
From the Project Dropdown (usually found at the top bar) click Select Project
On Project Popup click CREATE PROJECT
Once project is created you can click Select Project to switch the context (You can click on Notification link or Choose from Top Dropdown)
Click ENABLE APIs AND SERVICES
Now we need to Enable three APIs one by one (YouTube Data API, YouTube Analytics API, YouTube Reporting API)
Search YouTube Data API. Select and click ENABLE
Search YouTube Analytics API. Select and click ENABLE
Search YouTube Reporting API. Select and click ENABLE
Create Service Account
Once Project is created and APIs are enabled we can now create a service account under that project. Service account has its ID which looks like some email ID (not to confuse with Google /Gmail email ID)
From the Project Dropdown (usually found at the top bar) click Select Project
Enter Service account name and Service account description
For Role, do not select anything for now and Click Continue and then click Done. Next we will create Key.
Create Key
Once service account is created we need to create key file (i.e. credentials).
In the Cloud Console, click the email address for the service account that you created.
Click Keys.
Click Add key, then click Create new key.
Click Create and select P12 format. A P12 key file is downloaded to your computer. We will use this file in our API connection.
Click Close.
Now you may use downloaded *.p12 key file as secret file and Service Account Email as Client ID (e.g. some-service-account-name@your-project-id.iam.gserviceaccount.com ).
Add Permission
Now last thing is give read/write permission to Service Account. Basically you can create or open Google Sheet and add the Service Account as an editor to it as below.
Copy the email address of your service account we created in previous step (its usually like this some-service-account-name@your-project-id.iam.gserviceaccount.com).
Create or select an existing Google Sheet.
Navigate to Sheet for which you like to give read/write access to Service Account.
Click on the Share button in the top right, and add the email address of the service account as an editor. Here is how to share file(s) with specific people. Juse share with Service Account (use Service Account Email found on previous section)
Select the desired endpoint, change/pass the properties values, and go to the Mappings tab to map the columns.
API Destination - YouTube
Read / write YouTube data inside your app, perform many Youtube operations without coding using easy to use high performance API Connector for YouTube
Finally, map the desired columns:
API Destination - YouTube
Read / write YouTube data inside your app, perform many Youtube operations without coding using easy to use high performance API Connector for YouTube
That's it; we successfully configured the POST API Call. In a few clicks we configured the YouTube API call using ZappySys YouTube Connector
Load YouTube data into SQL Server using Upsert Destination (Insert or Update)
Once you configured the data source, you can load YouTube data into SQL Server using Upsert Destination.
Upsert Destination can merge or synchronize source data with the target table.
It supports Microsoft SQL Server, PostgreSQL, and Redshift databases as targets.
Upsert Destination also supports very fast bulk upsert operation along with bulk delete.
Upsert operation
- a database operation which performs INSERT or UPDATE SQL commands
based on record's existence condition in the target table.
It inserts records that don't have matching records in the target table
or updates them, if they do,
by matching them by key columns.
Upsert Destination supports INSERT, UPDATE, and DELETE operations,
so it is similar to SQL Server's MERGE command, except it can be used directly in SSIS package.
From the SSIS Toolbox drag-and-drop Upsert Destination component onto the Data Flow designer background.
Connect your SSIS source component to Upsert Destination.
Double-click on Upsert Destination component to open configuration window.
Start by selecting the Action from the list.
Next, select the desired target connection
or create one by clicking <New [provider] Connection> menu item from the Target Connection dropdown.
Then select a table from the Target Table list
or click New button to create a new table based on the source columns.
Continue by checking Insert and Update options according to your scenario
(e.g. if Update option is unchecked, no updates will be made).
Finally, click Map All button to map all columns
and then select the Key columns to match the columns on:
Click OK to save the configuration.
Run the package and YouTube data will be merged with the target table in SQL Server, PostgreSQL, or Redshift:
Done!
Deploy and schedule SSIS package
After you are done creating SSIS package, most likely, you want to deploy it to SQL Server Catalog and run it periodically.
Just follow the instructions in this article:
YouTube Connector support following actions for REST API integration.
If some actions are not listed below then you can easily edit Connector file and enhance out of the box functionality.
Get video details
Read videos details by IDs or chart or myRating (pass one of them) [
Read more...]
Parameter
Description
Ids (up to 25 comma-separated)
Comma-separated IDs of YouTube videos (max 25 IDs)
chart
The chart parameter identifies the chart that you want to retrieve.
Option
Value
mostPopular
mostPopular
myRating
Option
Value
like
like
dislike
dislike
part
Option
Value
contentDetails
contentDetails
fileDetails
fileDetails
id
id
liveStreamingDetails
liveStreamingDetails
localizations
localizations
player
player
processingDetails
processingDetails
recordingDetails
recordingDetails
snippet
snippet
statistics
statistics
status
status
suggestions
suggestions
topicDetails
topicDetails
hl
maxHeight
Acceptable values are 72 to 8192, inclusive.
maxResults
Acceptable values are 1 to 50, inclusive. Default value is 5.
The date for which to retrieve the report in YYYYMMDD format. Supported functions now|today|yesterday|weekstart|weekend|monthstart|monthend|yearstart|yearend and supported intervals for add/subtract are ms|s|sec|min|h|hour|y|year|d|day|m|month|y|year. Example monthstart-3d (subtract 3days from month start)
Option
Value
today
today
yesterday
yesterday
monthstart
monthstart
monthend
monthend
yearstart
yearstart
yearend
yearend
weekstart
weekstart
weekend
weekend
yearstart-1y
yearstart-1y
yearend-1y
yearend-1y
monthstart-1d
monthstart-1d
monthend+1d
monthend+1d
End Date (yyyy-MM-dd)
The date for which to retrieve the report in YYYYMMDD format. Supported functions now|today|yesterday|weekstart|weekend|monthstart|monthend|yearstart|yearend and supported intervals for add/subtract are ms|s|sec|min|h|hour|y|year|d|day|m|month|y|year. Example monthstart-3d (subtract 3days from month start)
Option
Value
today
today
yesterday
yesterday
monthstart
monthstart
monthend
monthend
yearstart
yearstart
yearend
yearend
weekstart
weekstart
weekend
weekend
yearstart-1y
yearstart-1y
yearend-1y
yearend-1y
monthstart-1d
monthstart-1d
monthend+1d
monthend+1d
currency
dimensions
Option
Value
adType
adType
ageGroup
ageGroup
asset
asset
audienceType
audienceType
channel
channel
claimedStatus
claimedStatus
contentOwner
contentOwner
country
country
day
day
deviceType
deviceType
elapsedVideoTimeRatio
elapsedVideoTimeRatio
gender
gender
insightPlaybackLocationDetail
insightPlaybackLocationDetail
insightPlaybackLocationType
insightPlaybackLocationType
insightTrafficSourceDetail
insightTrafficSourceDetail
insightTrafficSourceType
insightTrafficSourceType
liveOrOnDemand
liveOrOnDemand
operatingSystem
operatingSystem
playlist
playlist
province
province
sharingService
sharingService
subscribedStatus
subscribedStatus
subtitleLanguage
subtitleLanguage
uploaderType
uploaderType
video
video
filters
A list of filters that should be applied when retrieving YouTube Analytics data. The documentation for channel reports and content owner reports identifies the dimensions that can be used to filter each report, and the Dimensions document defines those dimensions.
Option
Value
Example1
country==IT
Example2
video==dMH0bHeiRNg
Example3
gender==FEMALE
Example4
dimension==some_value
includeHistoricalChannelData
Option
Value
true
true
false
false
maxResults
sort
A comma-separated list of dimensions or metrics that determine the sort order for YouTube Analytics data. By default the sort order is ascending. The - prefix causes descending sort order.
Option
Value
adType
adType
ageGroup
ageGroup
asset
asset
audienceType
audienceType
channel
channel
claimedStatus
claimedStatus
contentOwner
contentOwner
country
country
day
day
deviceType
deviceType
elapsedVideoTimeRatio
elapsedVideoTimeRatio
gender
gender
insightPlaybackLocationDetail
insightPlaybackLocationDetail
insightPlaybackLocationType
insightPlaybackLocationType
insightTrafficSourceDetail
insightTrafficSourceDetail
insightTrafficSourceType
insightTrafficSourceType
liveOrOnDemand
liveOrOnDemand
operatingSystem
operatingSystem
playlist
playlist
province
province
sharingService
sharingService
subscribedStatus
subscribedStatus
subtitleLanguage
subtitleLanguage
uploaderType
uploaderType
video
video
Generic Request
This is generic endpoint. Use this endpoint when some actions are not implemented by connector. Just enter partial URL (Required), Body, Method, Header etc. Most parameters are optional except URL. [
Read more...]
Parameter
Description
Url
API URL goes here. You can enter full URL or Partial URL relative to Base URL. If it is full URL then domain name must be part of ServiceURL or part of TrustedDomains
Body
Request Body content goes here
IsMultiPart
Set this option if you want to upload file(s) using either raw file data (i.e., POST raw file data) or send data using the multi-part encoding method (i.e. Content-Type: multipart/form-data).
A multi-part request allows you to mix key/value pairs and upload files in the same request. On the other hand, raw upload allows only a single file to be uploaded (without any key/value data).
==== Raw Upload (Content-Type: application/octet-stream) =====
To upload a single file in raw mode, check this option and specify the full file path starting with the @ sign in the Body (e.g. @c:\data\myfile.zip)
==== Form-Data / Multipart Upload (Content-Type: multipart/form-data) =====
To treat your request data as multi-part fields, you must specify key/value pairs separated by new lines in the RequestData field (i.e., Body). Each key/value pair should be entered on a new line, and key/value are separated using an equal sign (=). Leading and trailing spaces are ignored, and blank lines are also ignored.
If a field value contains any special character(s), use escape sequences (e.g., for NewLine: \r\n, for Tab: \t, for at (@): @). When the value of any field starts with the at sign (@), it is automatically treated as a file you want to upload. By default, the file content type is determined based on the file extension; however, you can supply a content type manually for any field using this format: [YourFileFieldName.Content-Type=some-content-type].
By default, file upload fields always include Content-Type in the request (non-file fields do not have Content-Type by default unless you supply it manually). If, for some reason, you don't want to use the Content-Type header in your request, then supply a blank Content-Type to exclude this header altogether (e.g., SomeFieldName.Content-Type=).
In the example below, we have supplied Content-Type for file2 and SomeField1. All other fields are using the default content type.
See the example below of uploading multiple files along with additional fields. If some API requires you to pass Content-Type: multipart/form-data rather than multipart/form-data, then manually set Request Header => Content-Type: multipart/mixed (it must start with multipart/ or it will be ignored).
file1=@c:\data\Myfile1.txt
file2=@c:\data\Myfile2.json
file2.Content-Type=application/json
SomeField1=aaaaaaa
SomeField1.Content-Type=text/plain
SomeField2=12345
SomeFieldWithNewLineAndTab=This is line1\r\nThis is line2\r\nThis is \ttab \ttab \ttab
SomeFieldStartingWithAtSign=\@MyTwitterHandle
Filter
Enter filter to extract array from response. Example: $.rows[*] --OR-- $.customers[*].orders[*]. Check your response document and find out hierarchy you like to extract
Headers for Request. To enter multiple headers use double pipe or new line after each {header-name}:{value} pair
Generic Request (Bulk Write)
This is a generic endpoint for bulk write purpose. Use this endpoint when some actions are not implemented by connector. Just enter partial URL (Required), Body, Method, Header etc. Most parameters are optional except URL. [
Read more...]
Parameter
Description
Url
API URL goes here. You can enter full URL or Partial URL relative to Base URL. If it is full URL then domain name must be part of ServiceURL or part of TrustedDomains
IsMultiPart
Set this option if you want to upload file(s) using either raw file data (i.e., POST raw file data) or send data using the multi-part encoding method (i.e. Content-Type: multipart/form-data).
A multi-part request allows you to mix key/value pairs and upload files in the same request. On the other hand, raw upload allows only a single file to be uploaded (without any key/value data).
==== Raw Upload (Content-Type: application/octet-stream) =====
To upload a single file in raw mode, check this option and specify the full file path starting with the @ sign in the Body (e.g. @c:\data\myfile.zip)
==== Form-Data / Multipart Upload (Content-Type: multipart/form-data) =====
To treat your request data as multi-part fields, you must specify key/value pairs separated by new lines in the RequestData field (i.e., Body). Each key/value pair should be entered on a new line, and key/value are separated using an equal sign (=). Leading and trailing spaces are ignored, and blank lines are also ignored.
If a field value contains any special character(s), use escape sequences (e.g., for NewLine: \r\n, for Tab: \t, for at (@): @). When the value of any field starts with the at sign (@), it is automatically treated as a file you want to upload. By default, the file content type is determined based on the file extension; however, you can supply a content type manually for any field using this format: [YourFileFieldName.Content-Type=some-content-type].
By default, file upload fields always include Content-Type in the request (non-file fields do not have Content-Type by default unless you supply it manually). If, for some reason, you don't want to use the Content-Type header in your request, then supply a blank Content-Type to exclude this header altogether (e.g., SomeFieldName.Content-Type=).
In the example below, we have supplied Content-Type for file2 and SomeField1. All other fields are using the default content type.
See the example below of uploading multiple files along with additional fields. If some API requires you to pass Content-Type: multipart/form-data rather than multipart/form-data, then manually set Request Header => Content-Type: multipart/mixed (it must start with multipart/ or it will be ignored).
file1=@c:\data\Myfile1.txt
file2=@c:\data\Myfile2.json
file2.Content-Type=application/json
SomeField1=aaaaaaa
SomeField1.Content-Type=text/plain
SomeField2=12345
SomeFieldWithNewLineAndTab=This is line1\r\nThis is line2\r\nThis is \ttab \ttab \ttab
SomeFieldStartingWithAtSign=\@MyTwitterHandle
Filter
Enter filter to extract array from response. Example: $.rows[*] --OR-- $.customers[*].orders[*]. Check your response document and find out hierarchy you like to extract
Headers
Headers for Request. To enter multiple headers use double pipe (||) or new line after each {header-name}:{value} pair
Conclusion
In this article we showed you how to connect to YouTube in SSIS and integrate data without any coding, saving you time and effort.
It's worth noting that ZappySys API Driver allows you to connect not only to YouTube,
but to any Java application that supports JDBC
(just use a different JDBC driver and configure it appropriately).
We encourage you to download YouTube Connector for SSIS and see how easy it is to use it for yourself or your team.
If you have any questions, feel free to contact ZappySys support team.
You can also open a live chat immediately by clicking on the chat icon below.