QMM AD – Incorrect Directory Synchronization Agent Matching, Caching and Repairing

QMM AD stores matching data in ADLDS and in Cache DB

From our experience, the Directory Service Agent component (DSA) from Quest Migration Manager for Active Directory is a reliant and powerful way to synchronize Active Directory objects and attribute data from one domain to another (and vice versa). It also has the ability to synchronize user passwords by installing a single agent on one domain controller. More than this, DSA is also responsible for mailbox creation in the Quest setup and synchronizes mailbox and Active Directory permissons.

The speed of delta synchronization (synchronizing changes of object attributes) is a combination of matching and caching. Quest DSA uses an AD LDS database to create matching objects that describe the synchronization relationship between an object in source domain and its peer in the target domain.

However, the ADLDS matching objects are most important when starting the synchronization and performing a Full Resync. In the ongoing synchronization, DSA takes the matching information from its cache which is a JET database located in “…\DSA\CONFIG\Cache” directory. The cache database file can grow up to a size which exceeds the size of the AD LDS by far. If disk space matters on your DSA machine, have an eye on the cache file size first.

Solving incorrect matching

By default, Quest Active Directory sync knows 3 criteria for object matching (the way, how DSA identifies, whether it has to merge an existing account in target or create a new one) – mail address, sid-sidHistory, samAccountName. Both decisions (merge or create) have consequences since DSA will create or modify the matching object and bind objects together, that should form a unity (or not).

However, we do not live in a perfect world and situations occur where the matching went wrong.

Real word scenario:

1. Group A is created in source domain, mail-enabled and filled with 10 members. It is part of DSA migration scope.
2. DSA picks up the group and looks up the matching criteria. All 3 criteria are activated and mail has highest precedence. DSA does not find a peer and creates a new group A in target domain with e-mail and the link resolver fills the group membership with the target user objects. DSA also creates a matching object and updates the cache file. So good so far.
3. Now somebody decides to create a new group B in source domain and shifts the mail address from group A to group B while the mail address on group A is renamed in the same step.
4. DSA will recognize that group B is existing and looks up for matching criteria. It will find a match for the mail address in group A of the target domain and will set up a matching of group source B to target A. It also will replicate the membership from source B to target A.
5. We have now a lot of uncomfortable issues. Membership in the DLs looks different for users in source and users in target domain. Group A in target has 2 entries in sidHistory, one for source group A and one for source group B. The matching attribute in group A from target domain is now filled with the object GUID from group B in source domain and the proxyAddresses including X.500 are mixed as well. Other attributes depend on your skip list settings
6.  And we still have group A in source. Since the matching criteria of sid-sidHistory is still valid, you can end up with a flip/flop, so that DSA runs over the two accounts and whenever there is a new attribute change on one of the source groups, it can either be group A or group B which is merged to group A in target.

OK, we should try to clean up the confusion.

1. We better remove mail address matching in our setup since it has problems with the domestic way of changing groups in this customer environment. We clean up all wrong values of group A in target. We run a full resync (which is restricted to once per quarter)
2. Same thing again, because the matching attribute was filled with the wrong value and the matching sid-sidHistory was still in place.
3. We clean up again and delete the matching attribute. We modify group B in source to trigger DSA and expect that a new group B in target is created.
4. Do we succeed? No. Of course not. There is a wrong matching object for group B (and group A) in ADLDS. OK. We clean up again and we delete the matching objects in ADLDS.
5. No way. The same thing happens again. No group B in target, but a matching of group A and group B to group A in target.
6. This time we stop DSA, clean up group A in target domain including wrong entries in proxyAddresses, sidHistory and delete the matching attributes. We delete the cache file and start with Full Resync – and we succeed

It’s all about cache. All the cleanup and repair actions can fail as long as the cache file still contains the wrong linking. Since a selective cleaning of the wrong object matching of the cache is not possible (anyone to try?), we always will need a full resync (of thousand objects) to repair a single object pair with wrong matching.

An alternative would have been to delete all 3 groups and create fresh objects. I would call it the “brute force method”. Not acceptable in many cases though.