Power Community

Power Community

Bulk Merge Duplicate Contacts in Dynamics 365 and Dataverse Power Apps

I am pleased to release Power Merge a free community solution to bulk merge duplicate records in Dynamics 365 and Dataverse Power Apps. The purpose of this solution is to allow you to easily identify and merge multiple duplicate contact records in one go, providing a custom guided interface you can embed into your contact forms and easily surface the fields you would like to compare to identify the Master records. This tool has been extremely valuable for organisations dealing with data cleansing operations by simplifying duplicate data management for users and administrators in preserving accurate contact data and improving customer experience.

Current Duplicate Merge Limitations in Dynamics 365

Dynamics 365 Customer Engagement provides the ability to detect duplicate records and facilitate the merging of duplicates by allowing you to create duplicate detection rules. Once the duplicate detection rules are activated it will notify users every time a duplicate is created and prompt to merge. However there is currently a 5000 Duplicates Limit in Dynamics 365, what this means is that the out of the box Bulk Duplicate detection feature will not work if there are more than 5000 duplicates in your system which makes it more of a preventative feature than a bulk cleansing operation.

Another significant limitation is the Dynamics 365 Merge dialogue only supports merging 2 duplicates at a time, in most cases there will be more than 2 duplicates and we need to make the process as simple as possible for admins to maintain data quality.

Finally we are not able to customise the fields we want to compare during the merge, and is limited to a handful of attributes which is not enough to help users make the correct decision on which records values should persist on the master record.

Introducing Power Merge to Bulk Merge Duplicates in Dynamics 365

I developed the Power Merge solution to overcomes the above limitations by allowing you to customise the merge fields and well as related records displayed on the dialogue allowing you to easily view all the duplicate records and fields as well as number of related records to easily identify the record which should be the Master record and easily merge all the duplicates within a single dialogue.

Step 1) Download the Power Merge Solution below, and install the managed solution in your Dynamics 365 environment


Step 2) You will need need to create your Duplicate Detection Rules so duplicates can be detected accordingly. You can create multiple duplicate detection rules for the same entity to support OR conditions, I have also provide a sample below on the correct way to setup you duplicate detection rules ensuring you ignore blank values;

Step 3) Add the Custom Fields you would like to be displayed during your duplicate detection

The Solution contains a configuration entity call Merge Config, you will need to create a Merge Config record and populate the attributes you wish the Bulk Merge Duplicates screen to display as shown below;

Step 4) Optional – In order to identify the Master record more easily, I have added a custom field to store a duplicate score. You can use this field to populate an integer value based on how complete the contact record is, by either scoring certain attributes or activities or statuses using Calculated fields, or externally with an ETL job.

Step 5) The Power Merge Form below displays the duplicate contacts for any the given record, allowing you to pick the records and respective fields you wish to preserve on the master record. You can update the Merge Config record to display any fields you would like to be displayed on this screen to help your staff clearly see the full dataset to make the most accurate decisions on preserving the customer data, such as latest contact details and gdpr option preferences are persisted on the remaining master record.

Step 6) Once you have selected the duplicate records and attributes you wish to preserve on the Master record you will then be allowed to Click the Merge button. The Merge operation will be completed for all the records, Power Merge supports merging over 10 duplicates in one go!

The Bulk Duplicate Merge process will firstly overwrite the values from the duplicate records that you selected onto the master record, and the re associate all related records to the Master record before deactivating the duplicates with a status of inactive. This means the merge process preserves the duplicate records and links them to the Master record so you can access them later if required via the Power Merge Form.

Please feel free to contact me if you would like to request add any new features, expect some new updates very soon enjoy! #MayThePowerBeWithYou

- Advertisement -spot_img


Please enter your comment!
Please enter your name here

- Advertisement -spot_img
- Advertisement - Advertisement

Latest News

Power Fx: Error handling graduates to preview

We are thrilled to announce that the long-time experimental feature Formula-level error handling has moved forward to preview. As a result, you and your end users will enjoy higher reliability and more transparency about what is happening in your apps. It’s a huge step. Adding error handling to an existing language turned out to be a very tall order, touching almost everything, from data types, to functions, to the runtime. Thank you for all of your support, feedback, and patience as we got this done. What does it mean for you? Your apps will more reliably detect and report errors.You can write blank/null values to a database.You can detect and replace errors with the IsError, IsErrorOrBlank, and IfError functions.You can control error reporting and logging at a central location with App.OnError.You can create and throw your own custom errors with the Error function. Error handling is a big change in behavior. By entering preview, we are signaling that we believe we are done, that we anticipate no further significant changes from here. Many of you already use error handling in production and this move to preview should only embolden more of you to do so. If significant changes are needed from here, we will treat them as a separate feature. We are rolling this out slowly as it is such a big change. All of you will soon see that the Formula-level error handling switch has moved from experimental to preview in the settings (as of version 3.22082). It will still be default to off for most tenants. Over the coming weeks we will slowly change the default for new apps only to on across the tenants. Makers can still disable this feature and will be able to do so for a long time. I say again: we are changing the default for new apps only. Existing apps will continue running as they always have. We have no plans at this time to turn this on for existing apps, and as this is such a big change, we may never do this and make this a permanently available switch. Your feedback will guide us. The documentation for Error, IfError, IsError, IsErrorOrBlank functions and the App.OnError property covers these changes. IfError and IsError are very similar to their Excel counterparts. We are also working on overview docs that will be released shortly. But before that, let’s take a brief tour. Let’s start with what Excel does, the inspiration for Power Fx. For an error like division by zero, Excel is very clear that something has gone wrong with a # error message that shows right in the cell. This error will propagate to other cell formulas if A1 is used in a formula: Today, without error handling, Power Apps won’t report anything in this scenario, instead treating the division by zero error as a blank value. That’s not good, as the maker and the end user of the app have no idea something may have gone wrong: Errors happen. Unexpected data flows in, networks go down, storage fills up, to name just a few situations that an app may encounter in the real world. Makers don’t often think through all the ways that things can go sideways which makes default error handling even more important. Returning a blank for an error is also a problem because blank is a legitimate value in our type system and in many databases. Without error handling, Power Apps won’t allow you to write a blank to a database instead thinking it is an error. So, instead of returning an easy to ignore or misinterpret blank value, with error handling turned on we now report an error to the end user (the error banner) and show the formula as having an error to the maker (the red filled in circle on the control): Further, if you look at the value of the formula, it is not a blank but an error value. Just as any formula can result in a blank, now any formula can also result in an error: Now, we still aren’t showing an error in the label control itself as Excel does. We couldn’t do this generically because, unlike Excel, the error could be on a property of a control for which there is no way to display the error. For example, where should an error on a slider control? Where should an error be shown for an imperative operation in the middle of a button’s OnSelect formula? We settled on showing the end user banner and flagging the control in the design experience. That’s not to say you can’t detect and display an error in that label control. Error handling provides a wealth of mechanisms to control how errors are handled and reported. For example in this case, we can wrap the division by zero with an IfError function to return a custom message in the label: The Text function call is required for type compatibility. Or we can use IfError to throw a different, more specific error with the Error function: Or we can have a catchall for all errors in the app with App.OnError. For example, we can log the error and present a different message to the end user: If we look at the log, we see the details of the captured error from FirstError (and there is also an AllErrors), including where it happened and when it was detected: The possibilities are endless! You now have all the tools you need to detect, replace, report, and log errors, including a good default behavior if you never take advantage of these tools. And, bonus, you can also now write blank (or null) values to databases. Please let us know what you think in the Power Apps community forum. There is a dedicated and active space for error handling discussions at Error Handling – Power Platform Community (microsoft.com).

More Articles Like This

- Advertisement -spot_img