I am pleased to release Power Merge a free community solution to bulk merge duplicate records in Dynamics 365 and Dataverse Power Apps. The purpose of this solution is to allow you to easily identify and merge multiple duplicate contact records in one go, providing a custom guided interface you can embed into your contact forms and easily surface the fields you would like to compare to identify the Master records. This tool has been extremely valuable for organisations dealing with data cleansing operations by simplifying duplicate data management for users and administrators in preserving accurate contact data and improving customer experience.
Current Duplicate Merge Limitations in Dynamics 365
Dynamics 365 Customer Engagement provides the ability to detect duplicate records and facilitate the merging of duplicates by allowing you to create duplicate detection rules. Once the duplicate detection rules are activated it will notify users every time a duplicate is created and prompt to merge. However there is currently a 5000 Duplicates Limit in Dynamics 365, what this means is that the out of the box Bulk Duplicate detection feature will not work if there are more than 5000 duplicates in your system which makes it more of a preventative feature than a bulk cleansing operation.
Another significant limitation is the Dynamics 365 Merge dialogue only supports merging 2 duplicates at a time, in most cases there will be more than 2 duplicates and we need to make the process as simple as possible for admins to maintain data quality.
Finally we are not able to customise the fields we want to compare during the merge, and is limited to a handful of attributes which is not enough to help users make the correct decision on which records values should persist on the master record.
Introducing Power Merge to Bulk Merge Duplicates in Dynamics 365
I developed the Power Merge solution to overcomes the above limitations by allowing you to customise the merge fields and well as related records displayed on the dialogue allowing you to easily view all the duplicate records and fields as well as number of related records to easily identify the record which should be the Master record and easily merge all the duplicates within a single dialogue.
Step 1) Download the Power Merge Solution below, and install the managed solution in your Dynamics 365 environment
Step 2) You will need need to create your Duplicate Detection Rules so duplicates can be detected accordingly. You can create multiple duplicate detection rules for the same entity to support OR conditions, I have also provide a sample below on the correct way to setup you duplicate detection rules ensuring you ignore blank values;
Step 3) Add the Custom Fields you would like to be displayed during your duplicate detection
The Solution contains a configuration entity call Merge Config, you will need to create a Merge Config record and populate the attributes you wish the Bulk Merge Duplicates screen to display as shown below;
Step 4) Optional – In order to identify the Master record more easily, I have added a custom field to store a duplicate score. You can use this field to populate an integer value based on how complete the contact record is, by either scoring certain attributes or activities or statuses using Calculated fields, or externally with an ETL job.
Step 5) The Power Merge Form below displays the duplicate contacts for any the given record, allowing you to pick the records and respective fields you wish to preserve on the master record. You can update the Merge Config record to display any fields you would like to be displayed on this screen to help your staff clearly see the full dataset to make the most accurate decisions on preserving the customer data, such as latest contact details and gdpr option preferences are persisted on the remaining master record.
Step 6) Once you have selected the duplicate records and attributes you wish to preserve on the Master record you will then be allowed to Click the Merge button. The Merge operation will be completed for all the records, Power Merge supports merging over 10 duplicates in one go!
The Bulk Duplicate Merge process will firstly overwrite the values from the duplicate records that you selected onto the master record, and the re associate all related records to the Master record before deactivating the duplicates with a status of inactive. This means the merge process preserves the duplicate records and links them to the Master record so you can access them later if required via the Power Merge Form.
Please feel free to contact me if you would like to request add any new features, expect some new updates very soon enjoy! #MayThePowerBeWithYou