In this article, we will show you step by step approach on how to register an Azure AD (Active Directory) app to access Dynamics CRM 365 API (also same for CDS / Dataverse API). You might have heard that Microsoft is deprecating an older way of authenticating (i.e. User ID / Password-based WS-Trust Authentication) on April 2022. You must update your jobs / code to use OAuth 2.0 by April 2022 for any CRM related API calls.

In other words rather than User Id / password now you can use more secure way by using OAuth rather than WS-Trust.

Going forward Microsoft wants you to use OAuth Token rather than user name/password to call Dynamics CRM 365 API programmatically.

Here is the official link which describes the dates for WS-Trust sunset

Register App for CRM / CDS / Dataverse in Azure / Active Directory

The first step is to register your app in Azure Portal under Active Directory using the below steps.

1. Ask your Active Directory Admin to login to Azure Portal and navigate to > Active Directory Blade.
   OR
   You can access it by searching “Azure Active directory” like below.
   

   Open Azure Active Directory

2. Now click on the App Registrations icon and click New App Registration
   
   

   New App Registrations under Azure Active Directory

3. Now next step is to give your App name to identify easily. Select App is used for your Org only option (Default) and last enter Redirect URL as below (This URL is used for 3-legged Authorization Code Grant only). We prefer you enter a URL which is OAuth friendly (If you don’t know one Use

   https://zappysys.com/oauth

     this is recommended URL if you working with ZappySys Products)
   

   Azure AD App Registration – Single / Multi-Tenant with Redirect URL (i.e. Call back)

    

4. Now once App is created you can find the Application (Client) Id from overview page
   

   Azure AD App – Find Application / Client Id on overview page

Add API Permissions for the App

Once App is created we need to Add Permission. Basically, we will set which style of APIs can be called by this App and what Actions can be performed.

1. To add API Permissions for this AD App click on API Permissions and then click on Add Permission
   

   Add API Permissions for Azure AD App

2. Find the Dynamics CRM icon and click on it
   

   Add API Permission for Dynamics CRM API access (Azure AD App)

3. Now you will see just one permission for Dynamics CRM (user_impersonation). As we said earlier for Dynamics CRM API security is controlled by User Roles in CRM Portal itself which will see in the next section. For now, just choose Delegated Permissions,  check user_impersonation and click Add Permissions
   

   Select API Permissions for Dynamics CRM App (Azure AD App)

4. Now the last step in Adding API permissions is Grant Admin Consent. If you are logged in as an Azure AD User with the correct admin role then no worries else you have to ask your Azure AD admin to consent for you so this App can be used to read/write data from CRM.

Create Client Secret

Once App is created next step is to generate a Client Secret. Perform the following steps to create a secret that you can use when you call API

1. Click on Certificates and Secrets and then New Secret
   

   Azure AD App Create Client Secret / Set Name, Expiration Date

2. Enter Name for Secret (You can give some friendly name if you create multiple secrets to control multiple processes). For example SecretForTesting, SecretForProduction etc.Select a longer expiration date (i.e. 24 months) if you don’t like to change secret often in production jobs else choose shorter expiration (i.e. 6 Months or Custom).
3. Once you Create a Secret make sure you copy it to a secure location. Once the page is closed it won’t appear again.
   

   Copy Client Secret for Azure AD App

Associate App User / Set Roles in CRM 365 / Dataverse Environment

Once we have App Id and Client Secret we can go to CRM Portal to add our App as a valid CRM User. And the last step will be to attach some role(s) to App User (See Next section).

Method-1 (New way)

Here are the steps to configure User / Security Roles in New account. Below steps will give you access to Dataverse Entities via API Call. To see all Dataverse Entity you can log in to https://make.powerapps.com/ and click Tables

1. Click on the below link to login to new portal to manage CRM /Dataverse environment
   https://admin.powerplatform.microsoft.com/environments
2. Create an Application User using these steps (When prompted choose the same App we created in the previous section)
3. Now Go to App Users Screen and Edit Security Roles using the below steps
   

   See available CRM 365 Online / Datavese Environments

   

   See available CRM 365 Online / Datavese Application Users

   

   Edit Security Roles for CRM 365 Online / Datavese Application Users

Method-2 (Legacy)

If you prefer to use an older interface (and it’s still available to you) then you can use the below steps to perform the same thing we described in the previous section.

NOTE: Microsoft may soon retire the older interface and force to use the previous way only.

1. To Add our Azure AD App as CRM Application user goto your CRM Portal. Click on Gear icon for Settings > Advanced Settings
   

   Dynamics CRM Advanced Settings

2. Click on Settings dropdown > Security (Under System Column)
   

   Dynamics CRM Security Settings (Under Advanced Settings – System)

3. On the security, screen click on the Users icon
   
   

   Manage Dynamics CRM Users (Add, Edit, Delete)

4. Once User Screen shows up click on the User Type Dropdown and choose Application Users and click New
   

   Add new Application User in Dynamics CRM

5. Once User screen shows up. Enter your Application Id (obtained from the Azure AD App > Overview screen – app created earlier). Click Save and you will notice all other fields auto populates.
   

   Adding Azure AD App as Dynamics CRM 365 – Application User

6. And the last step is to Attach some CRM Role to the new Application User.
   For demo purposes we will choose “System Admin” but for production use, you should grant correct permission carefully. There are many inbuilt roles you can choose from.
   

   Associate Security Role(s) for Dynamics CRM Application User

Find your Organization Id for CRM / Dataverse API calls

Now find your Org Id before we can call CRM / Dataverse APIs in the next section.

To find your Org Id you can perform the following steps.

Method-1 (New way – Dataverse Portal)

1. Go to https://admin.powerplatform.microsoft.com/
2. Click on the Environment you like to access
3. On the Details, page copy your Org Id from the Environment URL For example URL might look like below… You just need to copy the first part (Use this on CRM Connection > Organization Field)
   org8640f7e7.crm.dynamics.com
   

   How to find Organization Id for Microsoft Dataverse / CDS / CRM 365 API calls

Method-2 (Old way)

Another way to find your Organization Id is from CRM Portal

If you have multiple environments (i.e. Prod, Sandbox) then each environment might have its own unique URL)

1. To find your API Service URL, Click on Gear Icon
2. Choose Advanced Settings
3. Click down arrow on Settings
4. Click Customization
5. Click Developer Resource Link
6. On this page now you can see various URLs. For examples used in this article, we need to get is Service Root URL for Web API
   

   How to Find Organization Id / Web API URL / SOAP API Service URL (Unique Environment Instance URL)

Using Client ID (i.e. App Id) and Client Secret to Call Dynamics CRM 365 API / CDS / Dataverse

That’s it now we are ready to use this App Id and Client Secret to make CRM API calls.

To confirm if it’s working you can create a Dynamics CRM Connection (using SSIS PowerPack ). Choose OAuth option on Dynamics CRM Connection

Connect to Dynamics CRM / Dataverse / PowerApps API using OAuth (Client ID /Secret)

Conclusion

In this article, we saw how to Register Azure AD App to call various Microsoft APIs (for this article we focused mainly on Dynamics CRM 365 API only)

The post How to register App for Dynamics CRM 365 / CDS / Dataverse API (Azure AD / OAuth) appeared first on ZappySys Blog.

In this release we focused on fixing some bugs with Dynamics CRM Source , Dynamics CRM Destination and Reporting Services Task.

Version 2.9.6.10621 [Jun 24, 2019]

New Features/Improvements

• NEW: Excel Source – Add option to read from any sheet by number rather than hard coded table name (e.g. $4$ , $5$ … )

Bug fixes

• FIX: Dynamics CRM Destination – Timeout property is ignored – You may get error – The request channel timed out while waiting for a reply after 00:00:59. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding
• FIX: Dynamics CRM Source – Virtual Fields not included in output or they are NULL
• FIX: General – Sometimes you may see blank OAuth Screen (If error occurred behind the scene)
• FIX: Reporting Services Task – LastStatus field not updated
• FIX: Reporting Services Task – Local report file throws dll load error at run time
• FIX: Reporting Services Task – You may get error on Save UI – Concurrency violation: the UpdateCommand affected 0 of the expected 1 records
• FIX: Salesforce Source – You may get NULL for related Object fields
• FIX: Upsert Destination – When you select all columns as Key – You may get error – Incorrect syntax near the keyword ‘WHEN’

The post SSIS PowerPack v2.9.5 released (Maintenance Release) appeared first on ZappySys Blog.

In our previous article, we talked about how to read from Dynamics CRM / Dataverse (Dynamics 365 / On-Premises CRM / PowerApp). Now let’s look at how to Load data into Dynamics CRM / Dataverse (Dynamics 365 / On-Premises CRM / PowerApp) using SSIS. We will see a few important aspects of loading data into Dynamics CRM / Dataverse Online (i.e. Dynamics 365) and On-premises / IFD (Internet facing hosted instance).

We will show you how to use SSIS Dynamics CRM Destination connector for Bulk Insert, Update, Delete and Upsert Operations. This article uses Dynamics 365 Online for CRM Integration Examples but concepts are same for other types of installation (e.g. IFD or On-Premises)

Requirements

1. Make sure you have SSIS Designer installed (i.e. SSDT-BI)
2. Download and Installed ZappySys SSIS PowerPack which includes an SSIS Dynamics CRM Connectors.
3. Finally, make sure you have Microsoft Dynamics CRM / Dataverse credentials handy.

That’s it. So let’s get started.

Video Tutorial – CRM Insert, Upsert, Delete, Update

Getting Started

In this section, we will see how to perform bulk insert, update delete and upsert operations using  SSIS Dynamics CRM Destination

Configure SSIS Dynamics CRM / Dataverse Connection

The first step to read or write Dynamics CRM / Dataverse data is to make sure we have working Dynamics CRM connection in SSIS.

1. Create new or open existing SSIS Project in Visual Studio.
2. Open SSIS Package and Go to SSIS designer.
3. Right click in the connection manager panel and click New connection:
   

   Create a new SSIS Dynamics CRM / Dataverse Connection

4. In the connection type selection, Select the ZS-DYNAMICS-CRM Connection.
5. We will use a CRM Online (Office 365) connection, region, a username and a password and select organization from the dropdown as below and then click Test connection:
   

   Configure Dynamics CRM / Dataverse Connection in SSIS

Bulk Insert – SQL Server data Import into Dynamics CRM / Dataverse using SSIS

Now, let’s see steps to do SQL Server data Import into Dynamics CRM / Dataverse using SSIS. Below steps assume you already have sample data in some SQL Server Table which will be used to create new CRM accounts. You can enter few account names you like to create in the CRM.

1. Open SSIS Package and drag Data Flow task from Control Flow SSIS Toolbox
2. Go to data flow designer. Drag and drop OLEDB Source
3. Configure OLEDB Source to read from some sample data table (e.g. dbo.Accounts). If you want to be lazy and just test quickly you can use ZS CSV Source (Direct Value Mode) instead of OLEDB Source.Sample Data May be like below
   

   name,address1_city,address1_country
   Flipkart,Bengaluru,India
   Microsoft,seattle,USA
   Nokia,Espoo,Finland

   Once you configure Source (Connection, Table Name etc.) click OK to save it.
   

   CSV Source – Sample Accounts for CRM Insert Demo

4. Drag and drop  ZS Dynamics CRM Destination from the toolbox.
5. Connect Source to your Destination component.
6. Double click CRM destination to configure it.
7. On the Connection Managers tab, Select Dynamics CRM / Dataverse connection we created in the previous section.
   

   Select Connection for Dynamics CRM / Dataverse Destination

8. On the Component Properties Tab Select Table Name.
   

   Select Target Table for Dynamics CRM / Dataverse Insert

9. On the Mapping Tab map few columns (e.g.  name, address1_city, and address1_country)
   

   Configure Target Column Mapping for CRM Data Load

10. Click OK to save CRM destination.
11. Right click in the data flow designer, click Execute Task
    

    Execute SSIS Package – Loading Data into Dynamics CRM / Dataverse from CSV / SQL Server

Bulk Upsert – Update or Insert Dynamics CRM / Dataverse Records in SSIS

In the previous section we saw how to Insert New records. However sometimes you want to Upsert (Insert or Update). So if record already exists Update else Insert. In order to achieve this you have to set two additional attributes. For CRM Upsert you have to perform few changes to previous steps

Create Alternate Key for CRM Upsert

To perform Upsert operation you can map Table’s Primary Key or Alternate Key. Primary Key usually starts with table name and ends with “id” (e.g. for account table its accountid ).

Many times Source system don’t have CRM Table Primary Key. In such case you may use Alternate Key in the mapping (JOIN Key). To use Alternate key for Upsert you have first define Alternate Key in CRM Portal. Check this link for exact steps.

Create Alternate Key for Dynamics CRM / Dataverse Online Table (Useful for Update or Upsert Operation)

Configure SSIS CRM Destination for Upsert

1. Double click CRM destination. On the Component Properties Tab, Change Operation to UpdateOrInsert
2. Specify UpsertKeyField to either Table’s Primary Key Column Name or Alternate Key Name.
   

   Dynamics CRM / Dataverse Upsert Records using SSIS CRM Destination (Specify Alternate Key / Primary Key)

3. Now go to Mappings tab and confirm Key field is mapped.
   

   Configure Dynamics CRM / Dataverse Mapping for Upsert in SSIS

4. Click OK and Run Package to test CRM Upsert Operation.

If you want to use primary key for Upsert or Update Operation and not sure correct column name then you may use ZappySys Dynamics CRM / Dataverse Source FetchXml Query Mode and check Object Browser (Table Info Tab >> PrimaryIdAttribute in the grid).

Primary Key name usually starts with table name and ends with “id” (e.g. for account table its accountid ).

Find Primary Key for Dynamics CRM / Dataverse Table

Handling Inserted / Updated Records Output

Now lets look at how to detect which records got inserted and which records got Update? In Upsert Mode each record can be directed to one of the following three Outputs.

• Inserted Records [Blue Arrow]
• Updated Records [Blue Arrow]
• Records with Error [Red Arrow]

You can select Red or Blue arrow coming out from CRM Destination component. Drag and drop output arrow on any down stream component.

Bulk Update – Modify Dynamics CRM / Dataverse Records in SSIS

If you want to do bulk Update its very same steps as Upsert Operation except you have to set Operation = Update in the properties grid. For Key field you have to set primary key or Alternate Key.

Dynamics CRM / Dataverse Update in SSIS

Bulk Delete – Delete Dynamics CRM / Dataverse Records in SSIS

Bulk delete doesn’t require many settings. All you have to change is Operation = Delete on Component Properties Screen. On the Mapping Screen Map only one column (Primary Key). If you are not sure which one is primary key please refer to the previous section (Upsert Operation).

Delete records from Dynamics CRM / Dataverse Online using SSIS (Primary Key mapping for CRM Bulk Delete)

Some times you may get below errors. If you get this error make sure you delete all child records first before deleting parent. For example if you are deleting account then make sure you have to delete leads / contacts for that account or update leads / contacts so they don’t refer to account being deleted.

Error: 0xC0047062 at Dynamics CRM / Dataverse Bulk Delete, Delete Accounts [2]: System.Exception: Exception: The object you tried to delete is associated with another object and cannot be deleted.
Error Details: 
ApiExceptionSourceKey=Plugin/Microsoft.Crm.Common.ObjectModel.AccountService

Error Handling

Many times you want to ignore bad records and continue processing good records. In such case you can use Dynamics CRM / Dataverse Error Output. Just grad Red arrow from CRM Destination and Drop on target component. Select Redirect Option for Error , Truncation both and Click OK.

Once you set Error Handling your component will continue to run without failing whole data flow. Later you can review errors (if you saving to destination).

Configure Error Handling in SSIS Dynamics CRM / Dataverse Destination

Loading records into Dynamics CRM / Dataverse Table using SSIS . Redirect Bad rows to Error Output.

Handling Special Data Types (PartyList, Lookup, PickList)

Certain datatypes in CRM requires special handling. Check this article to learn more about how to insert data into columns with PartyList, Lookup or PickList field.

Conclusion

In this article, We have seen how easy it is to integrate SQL Server data with Dynamics CRM / Dataverse. Using drag and drop connector you can perform Bulk insert, update, delete and Upsert operations. Download ZappySys SSIS PowerPack to learn more about other possible scenarios which are not discussed in this article.

The post Load data into Dynamics CRM using SSIS – Insert, Upsert, Delete, Update appeared first on ZappySys Blog.