QMM 8.10 error: Agent is not ready to start – SCP not found

We used Quest Migration Manager 8.10 recently in a project at a customer for a combined Active Directory and Exchange migration. Overall target was to integrate a Windows 2003 domain cross forest and cross org into the central AD Forest with several child domains. Since from mail perspective our migration source was Exchange 2007 and our migration target Exchange 2013, we decided to use the Native Move Job option along with the Migration Manager for Exchange Agent (MAgE) services.

Situation:

The customer environment look like the following:
Source Domain in Single Domain Forest with Domain Controllers on Windows 2003 and Exchange 2007 as mail system.
Target Domain was one of several child domains in the central Forest. All domain controllers running Windows 2012 R2 and mail system was Exchange 2013 SP1.
All Exchange 2013 servers had been deployed to root domain which also kept all important system and admin accounts.
To limit complexity in the setup of Quest Migration Manager 8.10, we decided to use a single administrative account from target Forest’s root domain and granted all necessary permissions in the domains to run both, Active Directory and Exchange migration. Only for access to source Exchange 2007 when running the move request, we used an account from source domain with Org Admin permissions.

Native Move Job
Setup for Native Move Job

Installation of Migration Manager 8.10. on a member server in target domain (best practice recommendation) including all cumulative hotfixes went smoothly. After successful Directory Synchronization, we connected to the Exchange source and target Organization and finally deployed 2 Instances of the MAgE agent for native mailbox move jobs on our agent host and console server. Note: For agent hosts Windows 2012 R2 is currently (May 2014) not supported. You have to stay with Windows 2008 R2 here.

Problem:

However, after starting the agent services running with our administrative account , we recognized, that we could not open the log file of the agent in the Log Panel inside the Migration Manager for Exchange GUI. We searched for the log file and found it in “c:\progamdata\quest software\Migration Agent for Exchange\NativeMove directory”.

scp not found
Log snippet from MAgE agent

The log file showed that the agent was not starting to process the migration collection due to missing settings and then went to sleep. The lines of error:

 

Waiting for agent settings: Not found: (&(objectClass=serviceConnectionPoint) …..

Agent is not ready to start. Agent going to sleep at 1 minute.

repeated over and over.

Obviously the agent tried to execute an LDAP query to find a connection point in Active Directory.
Note: Currently QMM 8.10 uses 3 different systems to store configuration data: An ADLDS server, a SQL Server Instance and the Active Directory (ADDS).

Service Connection Point (SCP):

We ran the query which was shown in the log file against the target domain and we could find the Service Connection Point (SCP) immediately in the System container of the domain naming context.

QMM_8.10_SCP

The Service Connection Point consists primarily of the keywords array attribute and the serviceBindingInformation attribute. The QMM MAgE looks for the serviceBindingInformation attribute to get its SQL connection properties. In SQL it will finally find all information to process the collection.
QMM_8.10_SCP_3
We do not know why Developers at Dell Software made this process so complex. However, in our setup the agent could not find the Service Connection Point, because the agent was looking in the domain, where its service account was located and this was the root domain of the forest while the agent host had installed the SCP during installation in the child domain where the computer account was member of.

Solution:

Switching the agent host and agent service account to an account from child domain would have been a solution, but was not in compliance with customer policy to host all system accounts in root domain.
Moving agent host and console to root domain would not have meet best practices and would have interfered running directory synchronization.

So we ended up in giving the agent just what it requested:
We manually created a Service Connection Point in the root domain and copied all serviceBindingInformation values over.

The agent started immediately and worked without errors.

For future design we can only recommend to store Service Connection Point in the Configuration Partition as Exchange and lots of other software. Using the domain naming context will always lead to problems in a big Enterprise environment with Active Directory consisting of multiple domains in a  forest.

 

Powershell 5 in Windows Management Framework V5 Preview

Microsoft released a Preview of the Windows Management Framework V5. As in the past, this package ships with the according version of Powershell. Powershell V5 will bring interesting new Features.

Among those are:

  1. OneGet module with a set of comandlets to manage Software packages
  2. Commandlets to manage L2 Layer Network Switches

Find the introduction article for Windows Managagement Framework V5 by Jeffrey Snover here on TechNet.

You can dowload the Preview here.

However, the mixture of Powershell Versions we find at customer Environments will get wider and wider. Same is valid for modules like ActiveDirectory or SnapIns for Exchange. One will need to start with a lot of checks in the beginning of the code when a script is planned to be used universally.

Recovery Manager for Active Directory Forest Edition – 8.5.1 release is out

Dell Software released Version 8.5.1 of Recovery Manager for Active Directory Forest Edition.

The new release ships with multiple new features which were requested by many customers for a Long time:

Building virtual test environments
In the past the clone wizard provided only limited capabilities to build a test lab on base of our production Active Directory. Now the built-in logic of the Forest Recovery is used to create virtual lab to mirror an existing Active Directory.
The program component Active Directory Virtualization Lab works with MS System Center Virtual Machine Manager, VMWare ESX and VMWare vCenter.

Active Directory management capabilities
The tool now helps to manage Global catalog functionality and FSMO roles throughout your Forest’s domain controller infrastructure

Web Interface

The new Web Interface for Recovery Manager for Active Directory allows you to distribute the tool’s frontend much more easily

Support for BitLocker Drive Encryption when recovering Domain Controllers
If BitLocker is enabled on the domain controllers to be recovered, the tool will take care to disable BitLocker before starting recovery and to enable afterwards.

Enhanced Management Shell
Release 8.5.1 also ships with new Powershell commandlets to manage RMAD activities by script:
Expand-RMADBackup extracts the content of a backup file to a location you can select (same functionality as Extract Wizard from GUI)
Remove-RMADUnpackedComponent cleans up unpacked data from former restore operations
Remove-RMADCollectionItem removes items from a specific computer collection.

With Recovery Manager for Active Directory Forest Edition (RMAD FE) 8.5.1 you can restore Domain Controllers running the following OS Versions:

  • Microsoft Windows Server 2012 (including Server Core installation)
  • Microsoft Windows Server 2008 R2 without Service Pack or with Service Pack 1 (including Server Core installation)
  • Microsoft Windows Server 2008 with Service Pack 1 or Service Pack 2 (including Server Core installation)
  • Microsoft Windows Server 2003 R2 without Service Pack or with Service Pack 2
  • Microsoft Windows Server 2003 without Service Pack or with Service Pack 1 or Service Pack 2

You can install Recovery Manager for Active Directory Forest Edition 8.5.1 on the following platforms:

  • Microsoft Windows Server 2012
  • Microsoft Windows 8
  • Microsoft Windows Server 2008 R2 without Service Pack or with Service Pack 1
  • Microsoft Windows Server 2008 with Service Pack 1 or Service Pack 2
  • Microsoft Windows 7 without Service Pack or with Service Pack 1
  • possible but not recommended:
    Microsoft Windows Vista with Service Pack 2
    Microsoft Windows Server 2003 R2 with Service Pack 2
    Microsoft Windows Server 2003 with Service Pack 2
    Microsoft Windows XP with Service Pack 3

You can view all additional Information on the DELL Website here.

If you need assistance in deploying the Software and set up an Active Directory Forest Recovery plan with this tool, please get in contact here.

How to write or migrate sidHistory with Powershell (2)

SIDCloner.dll
When you look in WWW to find a way to write sidHistory attribute, you probably will stop at the marvelous SID Cloner website created by Jiri Formacek, MSFT (http://code.msdn.microsoft.com/windowsdesktop/SIDCloner-add-sIDHistory-831ae24b). Jiri provides a managed class library that implements the SIDCloner class, which we can use in Powershell and any .NET programming code.
The SID Cloner class is built upon native API to migrate sidHistory and therefore uses the DsAddSidHistory function under the hood (http://msdn.microsoft.com/en-us/library/windows/desktop/ms675918(v=vs.85).aspx).
Although we do not need to install ADMT on any machine to run the SID Cloner code, we still have to consider to meet the same requirements for the migration setup as ADMT does. SID Cloner and ADMT come from the same “mothership” DsAddSidHistory.

SidHistory requirements
In brief we need the following prerequisites to be in place before we can start writing sidHistory (http://msdn.microsoft.com/en-us/library/windows/desktop/ms677982(v=vs.85).aspx):

+ a trust relationship must exist between source and target domain
+ source and target domain must not be in the same Active Directory forest
+ for source domain all actions will have the PDC Emulator as target; you cannot bind to another DC than the one with the PDC Emulator role
+ Auditing must be enabled in source domain
+ a domain-local group “domain$$$” must be created in source domain
+ a special registry key must be created on PDC Emulator DC in source domain: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\TcpipClientSupport
+ Audit Mode must be turned on in each domain and also Account Management auditing of Success/Failure events must turned to on.
+ the Security Identifier we want to transfer must not already exist in any sidHstory attributes of objects in target domain

Permissions:
For running Powershell code based on SID Cloner you do not necessarily need domain admin credentials in target domain. While read permissions on objects in source domain are sufficient (you are reading the “standard” attribute objectSID there), the permissions to modify the object in target domain by writing the sidHistory value requires more:
+ full access permissions to the object (better OU)
+ special permission “migrateSIDHistory” on the Active Directory domain object in target domain
migrateSidHistoryPermission

With all the requirements settled, you are able to migrate sidHistory by using the sample script, that Jiri published on the SID Cloner Website.

However, the most easy way to use the 4 fold overload with SIDCloner did never work in our tests.
Overload with 4 arguments means, you simply define source and target domain, source account from which you want to take the objectSID and target account where you want to write sidHistory.
This approach will probably never succeed because
a) the strict PDC Emulator access is not guaranteed when using a domain call
b) the credentials of the interactively logged user are obviously not passed to the DsAddSidHistory function inside the SID Cloner dll

Let’s have more findings around this Topics in Part 3 of this blog post.