In this article, we will show you step by step approach on how to register an Azure AD (Active Directory) app to access Dynamics CRM 365 API (also same for CDS / Dataverse API). You might have heard that Microsoft is deprecating an older way of authenticating (i.e. User ID / Password-based WS-Trust Authentication) on April 2022. You must update your jobs / code to use OAuth 2.0 by April 2022 for any CRM related API calls.

In other words rather than User Id / password now you can use more secure way by using OAuth rather than WS-Trust.

Going forward Microsoft wants you to use OAuth Token rather than user name/password to call Dynamics CRM 365 API programmatically.

Here is the official link which describes the dates for WS-Trust sunset

Register App for CRM / CDS / Dataverse in Azure / Active Directory

The first step is to register your app in Azure Portal under Active Directory using the below steps.

1. Ask your Active Directory Admin to login to Azure Portal and navigate to > Active Directory Blade.
   OR
   You can access it by searching “Azure Active directory” like below.
   

   Open Azure Active Directory

2. Now click on the App Registrations icon and click New App Registration
   
   

   New App Registrations under Azure Active Directory

3. Now next step is to give your App name to identify easily. Select App is used for your Org only option (Default) and last enter Redirect URL as below (This URL is used for 3-legged Authorization Code Grant only). We prefer you enter a URL which is OAuth friendly (If you don’t know one Use

   https://zappysys.com/oauth

     this is recommended URL if you working with ZappySys Products)
   

   Azure AD App Registration – Single / Multi-Tenant with Redirect URL (i.e. Call back)

    

4. Now once App is created you can find the Application (Client) Id from overview page
   

   Azure AD App – Find Application / Client Id on overview page

Add API Permissions for the App

Once App is created we need to Add Permission. Basically, we will set which style of APIs can be called by this App and what Actions can be performed.

1. To add API Permissions for this AD App click on API Permissions and then click on Add Permission
   

   Add API Permissions for Azure AD App

2. Find the Dynamics CRM icon and click on it
   

   Add API Permission for Dynamics CRM API access (Azure AD App)

3. Now you will see just one permission for Dynamics CRM (user_impersonation). As we said earlier for Dynamics CRM API security is controlled by User Roles in CRM Portal itself which will see in the next section. For now, just choose Delegated Permissions,  check user_impersonation and click Add Permissions
   

   Select API Permissions for Dynamics CRM App (Azure AD App)

4. Now the last step in Adding API permissions is Grant Admin Consent. If you are logged in as an Azure AD User with the correct admin role then no worries else you have to ask your Azure AD admin to consent for you so this App can be used to read/write data from CRM.

Create Client Secret

Once App is created next step is to generate a Client Secret. Perform the following steps to create a secret that you can use when you call API

1. Click on Certificates and Secrets and then New Secret
   

   Azure AD App Create Client Secret / Set Name, Expiration Date

2. Enter Name for Secret (You can give some friendly name if you create multiple secrets to control multiple processes). For example SecretForTesting, SecretForProduction etc.Select a longer expiration date (i.e. 24 months) if you don’t like to change secret often in production jobs else choose shorter expiration (i.e. 6 Months or Custom).
3. Once you Create a Secret make sure you copy it to a secure location. Once the page is closed it won’t appear again.
   

   Copy Client Secret for Azure AD App

Associate App User / Set Roles in CRM 365 / Dataverse Environment

Once we have App Id and Client Secret we can go to CRM Portal to add our App as a valid CRM User. And the last step will be to attach some role(s) to App User (See Next section).

Method-1 (New way)

Here are the steps to configure User / Security Roles in New account. Below steps will give you access to Dataverse Entities via API Call. To see all Dataverse Entity you can log in to https://make.powerapps.com/ and click Tables

1. Click on the below link to login to new portal to manage CRM /Dataverse environment
   https://admin.powerplatform.microsoft.com/environments
2. Create an Application User using these steps (When prompted choose the same App we created in the previous section)
3. Now Go to App Users Screen and Edit Security Roles using the below steps
   

   See available CRM 365 Online / Datavese Environments

   

   See available CRM 365 Online / Datavese Application Users

   

   Edit Security Roles for CRM 365 Online / Datavese Application Users

Method-2 (Legacy)

If you prefer to use an older interface (and it’s still available to you) then you can use the below steps to perform the same thing we described in the previous section.

NOTE: Microsoft may soon retire the older interface and force to use the previous way only.

1. To Add our Azure AD App as CRM Application user goto your CRM Portal. Click on Gear icon for Settings > Advanced Settings
   

   Dynamics CRM Advanced Settings

2. Click on Settings dropdown > Security (Under System Column)
   

   Dynamics CRM Security Settings (Under Advanced Settings – System)

3. On the security, screen click on the Users icon
   
   

   Manage Dynamics CRM Users (Add, Edit, Delete)

4. Once User Screen shows up click on the User Type Dropdown and choose Application Users and click New
   

   Add new Application User in Dynamics CRM

5. Once User screen shows up. Enter your Application Id (obtained from the Azure AD App > Overview screen – app created earlier). Click Save and you will notice all other fields auto populates.
   

   Adding Azure AD App as Dynamics CRM 365 – Application User

6. And the last step is to Attach some CRM Role to the new Application User.
   For demo purposes we will choose “System Admin” but for production use, you should grant correct permission carefully. There are many inbuilt roles you can choose from.
   

   Associate Security Role(s) for Dynamics CRM Application User

Find your Organization Id for CRM / Dataverse API calls

Now find your Org Id before we can call CRM / Dataverse APIs in the next section.

To find your Org Id you can perform the following steps.

Method-1 (New way – Dataverse Portal)

1. Go to https://admin.powerplatform.microsoft.com/
2. Click on the Environment you like to access
3. On the Details, page copy your Org Id from the Environment URL For example URL might look like below… You just need to copy the first part (Use this on CRM Connection > Organization Field)
   org8640f7e7.crm.dynamics.com
   

   How to find Organization Id for Microsoft Dataverse / CDS / CRM 365 API calls

Method-2 (Old way)

Another way to find your Organization Id is from CRM Portal

If you have multiple environments (i.e. Prod, Sandbox) then each environment might have its own unique URL)

1. To find your API Service URL, Click on Gear Icon
2. Choose Advanced Settings
3. Click down arrow on Settings
4. Click Customization
5. Click Developer Resource Link
6. On this page now you can see various URLs. For examples used in this article, we need to get is Service Root URL for Web API
   

   How to Find Organization Id / Web API URL / SOAP API Service URL (Unique Environment Instance URL)

Using Client ID (i.e. App Id) and Client Secret to Call Dynamics CRM 365 API / CDS / Dataverse

That’s it now we are ready to use this App Id and Client Secret to make CRM API calls.

To confirm if it’s working you can create a Dynamics CRM Connection (using SSIS PowerPack ). Choose OAuth option on Dynamics CRM Connection

Connect to Dynamics CRM / Dataverse / PowerApps API using OAuth (Client ID /Secret)

Conclusion

In this article, we saw how to Register Azure AD App to call various Microsoft APIs (for this article we focused mainly on Dynamics CRM 365 API only)

The post How to register App for Dynamics CRM 365 / CDS / Dataverse API (Azure AD / OAuth) appeared first on ZappySys Blog.

In our previous article, we talked about how to read from Dynamics CRM using easy to use SSIS Dynamics CRM Source connector. In this post, you will learn how to use SSIS JSON Source (for REST API) instead of CRM Source. Using SSIS JSON Source you can read virtually any REST API or JSON file data including Dynamics CRM / Dataverse. For this article, our goal is to learn How to read Microsoft Dynamics 365 data using Web API (OData API) in SSIS. We will use drag and drop approach without any coding.

Why use Web API (OAuth) for Dynamics 365 CRM?

Microsoft has announced depreciation of older CRM API (SOAP API with WS-Trust Authentication) which means they are no longer going to support the older style of authentication so no option but to switch to newer authentication / API.

In the older approach (SOAP API with WS-Trust Authentication) you must pass your User Name and Password to authenticate, also if MFA / 2FA (Multi-Factored Auth) is turned on by organization policy you cannot use your user id / password in API call. In new authentication (i.e. OAuth) you have to use Client Id and Client Secret only.

Now let’s look at the step by step approach to call Dynamics 365 REST API.

Step-1 : Register Dynamics CRM / Dataverse App obtain App Id / Client Secret

The very first step to access Microsoft Dynamics CRM 365 API is to obtain Application Id (also referred as Client Id)  and Client Secret. We will use this information later in this article (on OAuth connection UI).

1. Register Dynamics CRM / Dataverse App With Azure and get the credential (App Id, Client Secret).

Step-2 : Read Microsoft Dynamics 365 API data in SSIS using JSON Source

Let’s start with an example. We use SSIS JSON Source Task to make calls to Microsoft Dynamics 365 API, we will load Dynamics 365 Leads data to SQL Server.

Please refer to this link for more information: Microsoft Dynamics 365 Web API Reference

Find out API Url for your environment (organization url)

Once we are done with the creation of the Azure AD app to obtain App Id /Secret out next step is to find out the base API URL for your organization / environment.

If you have multiple environments (i.e. Prod, Sandbox) then each environment might have its own unique URL)

1. To find your API Service URL, Click on Gear Icon
2. Choose Advanced Settings
3. Click down arrow on Settings
4. Click Customization
5. Click Developer Resource Link
6. On this page now you can see various URLs. For examples used in this article, we need to get is Service Root URL for Web API

How to Find Organization Id / Web API URL / SOAP API Service URL (Unique Environment Instance URL)

Create SSIS Data Flow to read data from Dynamics CRM / Dataverse

1. First of All, Drag and drop Data Flow Task from SSIS Toolbox and double click it to edit.
   

   Drag and Drop SSIS Data Flow Task from SSIS Toolbox

2. Furthermore, Drag and Drop JSON Source(REST API and File) File Task from the SSIS Toolbox.
   

   Drag and Drop JSON Source Component

Create OAuth Connection for Dynamics CRM / Dataverse

Now lets look at how to create and configure OAuth connection to call Web API for Dynamics CRM / Dataverse

1. Check Use Credentials Option and select a new ZS-OAuth Connection from the drop-down.
   

   Check Use Credentials

2. Once OAuth Connection UI opens up. Configure like below on General Tab.
   Change OAuth Version to OAuth2
   Change Grant Type to Client Credentials
   Enter Client Id (Azure AD App Id) (Obtained in the Previous Section)
   Enter Client Secret (Obtained in the Previous Section)
   Enter Token URL (see below)
   

   Configure OAuth Connection for Azure AD App – Client Credentials Grant (Tenant Specific Token EndPoints)

   For Client Credentials Grant
   Enter Token URL as https://login.microsoftonline.com/<your-tenant-id>/oauth2/token
   (You can find Tenant ID on the Overview page of your Azure AD App)

   For Default Grant (3-legged – requires User Login Prompt)
   Enter Auth URL as https://login.microsoftonline.com/common/oauth2/authorize
   Enter Token  URL as https://login.microsoftonline.com/common/oauth2/token

    

   

   Where to find Token Endpoint with Tenant ID in the URL (Azure AD App)

    

3. Now go to Advanced Tab and configure like belowEnter Extra Attribute for /auth as  

   resource=https://<your-instance-id>.api.<your-crm-region>.dynamics.com

   

   Client Credentials Grant for Dynamics CRM / Dataverse – API Access (Azure AD App / OAuth)

4. Click OK to Save connection.

Get all Accounts, Contacts, and Leads using Dynamics 365 API

Now let’s configure API URL from where you like to read data.

For example, if you just use the root URL then it will list all possible entries

Root URL:

https://{{organization-uri}}/api/data/v9.1/

1. Double click on the Component to Edit the Configurations.
2. Change Data Format to OData.
3. In the Web URL Field, you can enter any valid URL for Web API. Please refer to this link of Dynamics 365 Rest API for more information.Example Data URLs

   https://{{organization-uri}}/api/data/v9.1/
   https://{{organization-uri}}/api/data/v9.1/accounts
   https://{{organization-uri}}/api/data/v9.1/contacts
   https://{{organization-uri}}/api/data/v9.1/leads

   Here, replace {{organization-uri}} in URL to your Microsoft Dynamics 365 site domain name (Example zappy6.api.crm.dynamics.com Dynamics 365 URL).

   

   Set URL in JSON Source

Preview / Query Dynamics 365 Data

1. Now select Array filter as

   $.value[*]

     (Click Select Filter Button – Choose value node)
2. Click on the Preview button to get the data.
   

   Preview Data

3. Finally, click OK to save JSON Source settings.
4. That’s it we are ready to load Dynamics 365 API data to SQL Server.

Load Microsoft Dynamics 365 API data into SQL Server

ZappySys SSIS PowerPack makes it easy to load data from various sources such as REST, SOAP, JSON, XML, CSV or from other source into SQL Server, or PostgreSQL, or Amazon Redshift, or other targets. The Upsert Destination component allows you to automatically insert new records and update existing ones based on key columns. Below are the detailed steps to configure it.

Step 1: Add Upsert Destination to Data Flow

1. Drag and drop the Upsert Destination component from the SSIS Toolbox.
2. Connect your source component (e.g., JSON / REST / Other Source) to the Upsert Destination.

SSIS - Data Flow - Drang and Drop Upsert Destination Component

Step 2: Configure Target Connection

1. Double-click the Upsert Destination component to open the configuration window.
2. Under Connection, select an existing target connection or click NEW to create a new connection.
   • Example: SQL Server, or PostgreSQL, or Amazon Redshift.

Step 3: Select or Create Target Table

1. In the Target Table dropdown, select the table where you want to load data.
2. Optionally, click NEW to create a new table based on the source columns.

Configure SSIS Upsert Destination Connection - Loading data (REST / SOAP / JSON / XML /CSV) into SQL Server or other target using SSIS

Step 4: Map Columns

1. Go to the Mappings tab.
2. Click Auto Map to map source columns to target columns by name.
3. Ensure you check the Primary key column(s) that will determine whether a record is inserted or updated.
4. You can manually adjust the mappings if necessary.

SSIS Upsert Destination - Columns Mappings

Step 5: Save Settings

• Click OK to save the Upsert Destination configuration.

Step 6: Optional: Add Logging or Analysis

• You may add extra destination components to log the number of inserted vs. updated records for monitoring or auditing purposes.

Step 7: Execute the Package

• Run your SSIS package and verify that the data is correctly inserted and updated in the target table.

SSIS Upsert Destination Execution

Conclusion

In this post you have seen how to Read Microsoft Dynamics 365 API data in SSIS. We used JSON Source Component to call Dynamics 365 REST API and load data into SQL server. You can download SSIS PowerPack here to try many other scenarios not discussed in this blog along with 70+ other components.

References

Finally, you can use the following links for more information about the use of Dynamics 365 REST API with our tools:

• JSON Source, you can also find Tutorial Video here.
• Help File: Documentation of SSIS Json Source
• Microsoft Dynamics CRM Web API Reference
• Dynamics 365 Web API EntityType Reference

The post How to read Microsoft Dynamics 365 OData API data in SSIS (CRM Web API) appeared first on ZappySys Blog.