SSIS tutorial: How to get data from Github

Introduction

GitHub is one of the most widely used platforms for source control, DevOps collaboration, and software delivery. Teams often need GitHub data, such as repositories, issues, pull requests, and commits, in SQL Server for analytics, reporting, and governance.

In this tutorial, you will learn how to connect GitHub REST APIs to SQL Server Integration Services (SSIS) by using ZappySys SSIS PowerPack. This approach helps you build automated ETL pipelines without writing custom API client code.

Prerequisites

1. SQL Server Integration Services (SSIS) is installed.
2. Visual Studio with SSIS extension or SQL Server Data Tools (SSDT) is available.
3. A GitHub account with access to the repositories you want to extract.
4. ZappySys SSIS PowerPack is installed on your machine.

Steps

Create a GitHub Personal Access Token

1. Sign in to your GitHub account.
2. Go to Settings, then find the Developer settings at the bottom of the page.
   

   Github – Go to Developer settings

3. Now go to Personal access tokens and create a new token (fine-grained or classic, depending on your security policy).
4. Select permissions required for your extraction scenario (for example, repository metadata, issues, or pull requests), owner, expiration date and more.
5. Generate the token and copy it to a secure location.
   

   Github – Generate a new token

Create an HTTP Connection in SSIS

1. Create or open your SSIS project and package.
2. In Connection Managers, right-click and select New Connection….
3. Select ZS-HTTP from the SSIS connectors list.
   

   Create a new HTTP connection

4. Set Credentials Type to Static token/API key.
5. Paste your GitHub token in the token value field.
6. Click OK to save the connection.
   

   Github- Connection configuration

Create and Configure JSON Source

1. Drag a Data Flow Task into Control Flow and open it.
2. In Data Flow, drag and drop the ZS JSON Source.
3. Open the JSON Source editor and enable Use credentials.
4. Select the GitHub HTTP connection you created earlier.
5. Set the API URL. Example to extract repositories for the authenticated user:
   
   https://api.github.com/user

   1	https://api.github.com/user

6. Set HTTP Request Method to GET.
7. Set the following values in the HTTP headers:
   1. Accept: application/vnd.github+json
   2. X-GitHub-Api-Version: 2022-11-28
8. Set JSON Path Filter if you need it.
9. Click Preview to confirm rows are returned, then click OK.
   

   Github – JSON source configuration

Use Other GitHub Endpoints (Optional)

You can reuse the same SSIS package pattern with different GitHub endpoints:

• Repositories (org): https://api.github.com/orgs/{org}/repos
• Issues: https://api.github.com/repos/{owner}/{repo}/issues
• Pull Requests: https://api.github.com/repos/{owner}/{repo}/pulls
• Commits: https://api.github.com/repos/{owner}/{repo}/commits

Configure Destination and Load to SQL Server

1. Add an OLE DB Destination (or another destination of your choice) in the Data Flow.
2. Connect the output arrow from JSON Source to the destination component.
3. Configure your SQL Server connection and target table.
4. Map JSON source columns to destination columns in the Mappings section.
5. Click OK and execute the package.
6. Validate that the GitHub rows have been loaded into your SQL Server table.
   

   Github – Final Result

Conclusion

You now have a working SSIS integration to extract GitHub data by using ZappySys SSIS PowerPack and the GitHub REST API. With this pattern, you can automate the extraction of repository, issue, pull request, and commit data into SQL Server for reporting and analytics.

Explore the full capabilities of the ZappySys SSIS PowerPack to optimise your data integration tasks. By applying this pattern, you can extend your solution to include additional transformations, data validation, error handling, and scheduling through SQL Server Agent.

Need Help?

If you encounter any issues, our support team is here to help:

1. Live Chat — Use the chat widget (bottom-right corner of this page)
2. Email — support@zappysys.com
3. Support Center — Visit the ZappySys Support Portal