Google BigQuery Connector
Documentation
Version: 11
Documentation

Authentication :: Service Account (Using Private Key File) [OAuth]


Description

No description available [API Help..]

Instructions

Use these steps to authenticate as service account rather than Google / GSuite User. Learn more about service account here

Basically to call Google API as Service account we need to perform following steps listed in 3 sections (Detailed steps found in the help link at the end)

Create Project

First thing is create a Project so we can call Google API. Skip this section if you already have Project (Go to next section)
  1. Go to Google API Console
  2. From the Project Dropdown (usually found at the top bar) click Select Project
  3. On Project Propup click CREATE PROJECT
  4. Once project is created you can click Select Project to switch the context (You can click on Notification link or Choose from Top Dropdown)
  5. Click ENABLE APIS AND SERVICES
  6. Now we need to Enable two APIs one by one (BigQuery API and Cloud Resource Manager API).
  7. Search BigQuery API. Select and click ENABLE
  8. Search Cloud Resource Manager API. Select and click ENABLE

Create Service Account

Once Project is created and APIs are enabled we can now create a service account under that project. Service account has its ID which looks like some email ID (not to confuse with Google /Gmail email ID)
  1. Go to Create Service Account
  2. From the Project Dropdown (usually found at the top bar) click Select Project
  3. Enter Service account name and Service account description
  4. Click on Create. Now you should see an option to assign Service Account permissions (See Next Section).

Give Permission to Service Account

By default service account cant access BigQuery data or List BigQuery Projects so we need to give that permission using below steps.
  1. After you Create Service Account look for Permission drop down in the Wizard.
  2. Choose BigQuery -> BigQuery Admin role so we can read/write data. (NOTE: If you just need read only access then you can choose BigQuery Data Viewer)
  3. Now choose one more Project -> Viewer and add that role so we can query Project Ids.
  4. Click on Continue. Now you should see an option to Create Key (See Next Section).

Create Key (P12)

Once service account is created and Permission is assigned we need to create key file.
  1. In the Cloud Console, click the email address for the service account that you created.
  2. Click Keys.
  3. Click Add key, then click Create new key.
  4. Click Create and select P12 format. A P12 key file is downloaded to your computer. We will use this file in our API connection.
  5. Click Close.
  6. Now you may use downloaded *.p12 key file as secret file and Service Account Email as Client ID (e.g. some_name@some_name.iam.gserviceaccount.com).

Manage Permissions / Give Access to Other Projects

We saw how to add permissions for Service Account during Account Creation Wizard but if you ever wish to edit after its created or you wish to give permission for other projects then perform forllowing steps.
  1. From the top Select Project for which you like to edit Permission.
  2. Go to IAM Menu option (here)
    Link to IAM: https://console.cloud.google.com/iam-admin/iam
  3. Goto Permissions tab. Over there you will find ADD button.
  4. Enter Service account email for which you like to grant permission. Select role you wish to assign.

Parameters

Parameter Label Required Options Description Help
ClientId Service Account Email YES This is service account email ID (e.g. some_name@my_project.iam.gserviceaccount.com)
PrivateKeyPath P12 Service Account Private Key Path (i.e. *.p12) YES File path for p12 file (i.e. Private Key file for service account). Keep this key file secure
Scope Scope NO Permission(s) you like to request
ProjectId ProjectId YES Login to https://console.cloud.google.com/bigquery and choose Project dropdown at the top to see list of Projects. Over there you will find ProjectID next to ProjectName. You need to get ProjectID which has BigQuery API support enabled.
DatasetId DatasetId (Choose after ProjectId) YES Default Dataset Name you like to use when listing tables (e.g. MyDataset).
RetryMode RetryMode NO
Option Value
None None
RetryAny RetryAny
RetryWhenStatusCodeMatch RetryWhenStatusCodeMatch
RetryStatusCodeList RetryStatusCodeList NO
RetryCountMax RetryCountMax NO
RetryMultiplyWaitTime RetryMultiplyWaitTime NO
Location Job Location NO
Option Value
System Default
Data centers in the United States US
Data centers in the European Union EU
Columbus, Ohio us-east5
Iowa us-central1
Las Vegas us-west4
Los Angeles us-west2
Montréal northamerica-northeast1
Northern Virginia us-east4
Oregon us-west1
Salt Lake City us-west3
São Paulo southamerica-east1
Santiago southamerica-west1
South Carolina us-east1
Toronto northamerica-northeast2
Delhi asia-south2
Hong Kong asia-east2
Jakarta asia-southeast2
Melbourne australia-southeast2
Mumbai asia-south1
Osaka asia-northeast2
Seoul asia-northeast3
Singapore asia-southeast1
Sydney australia-southeast1
Taiwan asia-east1
Tokyo asia-northeast1
Belgium europe-west1
Finland europe-north1
Frankfurt europe-west3
London europe-west2
Madrid europe-southwest1
Milan europe-west8
Netherlands europe-west4
Paris europe-west9
Warsaw europe-central2
Zürich europe-west6
AWS - US East (N. Virginia) aws-us-east-1
Azure - East US 2 azure-eastus2
Custom Name (Type your own) type-region-id-here
The geographic location where the job should run. For Non-EU and Non-US datacenters we suggest you to supply this parameter to avoid any error. API Help