Google BigQuery Connector
Documentation
Version: 12
Documentation

Authentication User Account


Description

User accounts represent a developer, administrator, or any other person who interacts with Google APIs and services. User accounts are managed as Google Accounts, either with Google Workspace or Cloud Identity. They can also be user accounts that are managed by a third-party identity provider and federated with Workforce Identity Federation. [API reference]

Instructions

Follow these steps on how to create Client Credentials (User Account principle) to authenticate and access BigQuery API in SSIS package or ODBC data source:

WARNING: If you are planning to automate processes, we recommend that you use a Service Account authentication method. In case, you still need to use User Account, then make sure you use a system/generic account (e.g. automation@my-company.com). When you use a personal account which is tied to a specific employee profile and that employee leaves the company, the token may become invalid and any automated processes using that token will start to fail.

Step-1: Create project

This step is optional, if you already have a project in Google Cloud and can use it. However, if you don't, proceed with these simple steps to create one:

  1. First of all, go to Google API Console.

  2. Then click Select a project button and then click NEW PROJECT button:

    Start creating a new project in Google Cloud
  3. Name your project and click CREATE button:

    Create a new project in Google Cloud
  4. Wait until the project is created:

    Wait until project is created in Google Cloud
  5. Done! Let's proceed to the next step.

Step-2: Enable Google Cloud APIs

In this step we will enable BigQuery API and Cloud Resource Manager API:

  1. Select your project on the top bar:

    Select project in Google Cloud
  2. Then click the "hamburger" icon on the top left and access APIs & Services:

    Access APIs and services in Google Cloud
  3. Now let's enable several APIs by clicking ENABLE APIS AND SERVICES button:

    Enable API for project in Google Cloud
  4. In the search bar search for bigquery api and then locate and select BigQuery API:

    Search for API in Google Cloud
  5. If BigQuery API is not enabled, enable it:

    Enable Google BigQuery API
  6. Then repeat the step and enable Cloud Resource Manager API as well:

    Enable Cloud Resource Manager API
  7. Done! Let's proceed to the next step.

Step-3: Create OAuth application

  1. First of all, click the "hamburger" icon on the top left and then hit VIEW ALL PRODUCTS:

    View all products in Google Cloud
  2. Then access Google Auth Platform to start creating an OAuth application:

    Open Google Auth Platform in Google Cloud
  3. Start by pressing GET STARTED button:

    Start creating an app in Google Cloud
  4. Next, continue by filling in App name and User support email fields:

    Fill app info in Google Cloud
  5. Choose Internal option, if it's enabled, otherwise select External:

    Choose app audience in Google Cloud
  6. Optional step if you used Internal option in the previous step. Nevertheless, if you had to use External option, then click ADD USERS to add a user:

    Add test user in Google Cloud app
  7. Then add your contact Email address:

    Enter app contact info in Google Cloud
  8. Finally, check the checkbox and click CREATE button:

    Create app in Google Cloud
  9. Done! Let's create Client Credentials in the next step.

Step-4: Create Client Credentials

  1. In Google Auth Platform, select Clients menu item and click CREATE CLIENT button:

    Start creating app client in Google Cloud
  2. Choose Desktop app as Application type and name your credentials:

    Create OAuth app client in Google Cloud
  3. Continue by opening the created credentials:

    View app client credentials in Google Cloud
  4. Finally, copy Client ID and Client secret for the later step:

    Use client ID and secret to read Google REST API data
  5. Done! We have all the data needed for authentication, let's proceed to the last step!

Step-5: Configure connection

  1. Now go to SSIS package or ODBC data source and use previously copied values in User Account authentication configuration:

    • In the ClientId field paste the Client ID value.
    • In the ClientSecret field paste the Client secret value.
  2. Press Generate Token button to generate Access and Refresh Tokens.

  3. Then choose ProjectId from the drop down menu.

  4. Continue by choosing DatasetId from the drop down menu.

  5. Finally, click Test Connection to confirm the connection is working.

  6. Done! Now you are ready to use Google BigQuery Connector!

Parameters

Parameter Label Required Default value Options Description
UseCustomApp UseCustomApp YES True Use your own app credentials or inbuilt app provided by ZappySys for ease of use. If you choose UseCustomApp=true then make sure to obtain your own ClientId and Secret using steps provided (Click [Steps to Configure] link found next to Authentication Type dropdown)
ProjectId ProjectId (Choose after [Generate Token] clicked) YES Login to https://console.cloud.google.com/bigquery and choose Project dropdown at the top to see list of Projects. Over there you will find ProjectID next to ProjectName. You need to get ProjectID which has BigQuery API support enabled.
DatasetId DatasetId (Choose after [Generate Token] clicked and ProjectId selected) YES Default Dataset Name you like to use when listing tables (e.g. MyDataset).
ClientId ClientId YES, if UseCustomApp, otherwise NO
ClientSecret ClientSecret YES, if UseCustomApp, otherwise NO
Scope Scope https://www.googleapis.com/auth/bigquery https://www.googleapis.com/auth/bigquery.insertdata https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/cloud-platform.read-only https://www.googleapis.com/auth/devstorage.full_control https://www.googleapis.com/auth/devstorage.read_only https://www.googleapis.com/auth/devstorage.read_write
RetryMode RetryMode RetryWhenStatusCodeMatch
Name Value
None None
RetryAny RetryAny
RetryWhenStatusCodeMatch RetryWhenStatusCodeMatch
RetryStatusCodeList RetryStatusCodeList 429|503
RetryCountMax RetryCountMax 5
RetryMultiplyWaitTime RetryMultiplyWaitTime True
Location Job Location
Name Value
System Default
Data centers in the United States US
Data centers in the European Union EU
Columbus, Ohio us-east5
Iowa us-central1
Las Vegas us-west4
Los Angeles us-west2
Montréal northamerica-northeast1
Northern Virginia us-east4
Oregon us-west1
Salt Lake City us-west3
São Paulo southamerica-east1
Santiago southamerica-west1
South Carolina us-east1
Toronto northamerica-northeast2
Delhi asia-south2
Hong Kong asia-east2
Jakarta asia-southeast2
Melbourne australia-southeast2
Mumbai asia-south1
Osaka asia-northeast2
Seoul asia-northeast3
Singapore asia-southeast1
Sydney australia-southeast1
Taiwan asia-east1
Tokyo asia-northeast1
Belgium europe-west1
Finland europe-north1
Frankfurt europe-west3
London europe-west2
Madrid europe-southwest1
Milan europe-west8
Netherlands europe-west4
Paris europe-west9
Warsaw europe-central2
Zürich europe-west6
AWS - US East (N. Virginia) aws-us-east-1
Azure - East US 2 azure-eastus2
Custom Name (Type your own) type-region-id-here
The geographic location where the job should run. For Non-EU and Non-US datacenters we suggest you to supply this parameter to avoid any error.
ReturnUrl Redirect URL (Only for Web App)
Name Value
https://zappysys.com/oauth/ https://zappysys.com/oauth/
Only specify this if you have created Credential as Web Application rather than Desktop. In Desktop App you dont have to supply Return URL (its always localhost). When you keep this blank it uses default value http://localhost:[some_random_port_each_time] for redirect_url)