New Release of DELL Migration Manager for Exchange 8.11

While we were all waiting for the arrival of Migration Manager for Exchange 9.0, DELL Software released a new version 8.11 of its Exchange migration flagship.

New in the release 8.11 is the support for installing the Migration Manager components on Windows Server 2012 and 2012 R2 OS platforms. On the other hand, installations on Windows Server 2008 and Windows 7 are not supported anymore:

  • Migration Manager supports migration from the source Exchange 2013 organization to the target Exchange 2010 or Exchange 2013 organization using Native Move Job.
  • All Dell Migration Manager components can now be installed on Microsoft Windows Server 2012 and Microsoft Windows Server 2012 R2
  • Operating systems not supported from Migration Manager console:
    – Microsoft Windows Server 2008 Service Pack 1 or higher (x86 edition)
    – Microsoft Windows 7 Service Pack 1 or higher (x64 edition)
    – Microsoft Windows Server 2008 Service Pack 1 or higher (x64 edition)
    – Microsoft Windows 7 Service Pack 1 or higher (x86 edition)
  • Operating systems not supported from Migration Agent for Exchange:
    – Microsoft Windows Server 2008 Service Pack 1 or higher (x86 edition)
    – Microsoft Windows Server 2008 Service Pack 1 or higher (x64 edition)
    – Microsoft Windows 7 Service Pack 1 or higher (x86 edition)

You can find a complete list of changes and release related information of the product here:

https://support.software.dell.com/download-install-detail/5775891?prodjid=1-7GCB8-249&utm_campaign=42330-32498-SU-GL-MigrationManagerforExchange&utm_medium=email&utm_source=Eloqua

Recommended Blog Post: The good, the bad and sIDHistory

Based on various experiences with Exchange 2010 behavior after an Exchange and Active Directory Inter Forest Migration, Exchange PRO Ingo Gegenwarth published this interesting Post:

The clueless guy

This post is about my personal journey with a cross-forest migration.

When it comes to account migration there is no way to do so without sIDHistory. It would be really hard to have a smooth migration without.

By using this attribute a end-user most likely won’t experience any impact…..unless you start doing a cleanup of this attribute.

In terms of Exchange users might see something like this

Calendar_Error

or this

Inbox_Error

But what’s behind those issues and how could you mitigate this? I was part of a migration, where those issues popped up and I’m going to describe how you could determine possible impact for end-users before it happens.

View original post 3,675 more words

Exchange Migration first & Mailbox Folder Permissions

In many inter forest migration projects, the mailbox migration to the new Exchange Organization into the new forest is performed first and the user migration is performed in a separate step at a later time. After the mailbox migration and before migrating the user objects, you have a classic Resource Forest scenario. Users are created as disabled user accounts in the target forest, receiving a linked mailbox, connected to the source Active Directory user. Important AD attribute in this scenario is msExchMasterAccountSID. This attribute of the disabled target user object holds the objectSID of the source user account. This allows to connect to the own mailbox and shared mailbox resources (Delegate permissions etc.) with the active source user object.

ExMigirst01

Did you ever thought about mailbox folder permissions in this scenario? 

For every migrated folder permission (e.g. with QMM for Exchange) and also every time a user manually adds mailbox folder permissions for another (not Active Directory migrated) user in the target mailbox, the SID of the source user object is added to the mailbox folder permissions. In this example, we’ve selected the not migrated user UserA from the Global Address list and added him as delegate for the Inbox and the calendar for the Info MBX:  ExMigirst02

At some point, the Active Directory migration will start. During this process, the user account in the target domain will be activated and the Linked Mailbox is converted to a User Mailbox. This action will clear the attribute msExchMasterAccountSID. This is necessary because the target account will now be used to access the own mailbox and resources of other mailboxes. If a migrated user is now added to the mailbox folder permissions, the target SID will be added and no longer the source SID. Let’s use the mailbox example above and add the migrated user UserB as additional delegate for the Calendar of the Info MBX:ExMigirst03

In this example, UserB will of course not have any problems accessing the Info MBX. But what happens if UserA will be migrated and the user starts accessing the Info MBX with TARGETDOMAIN\UserA? The SID of the target account has no permissions on the Inbox and the Calendar folder. Will UserA loose access to these folders now? Generally, the answer is YES, UserA will lose access! But…

In Active Directory Migration projects, it is best practice to migrate the SIDHistory to the target user account. In this case, the objectSID of the source user is copied to the attribute SIDHistory of the target account. For our example, it means that UserA will not lose access to the Info MBX because his Access Token contains the Source SID which has permissions on the Inbox and Calendar folder in the Info MBX.

SIDHistory CleanUp & mailbox folder permissions?

Clearing SIDHistory is part of most of the migration projects. Before clearing the SIDHistory attribute of the target accounts, it is required to replace the source SIDs with the corresponding target SIDs inside the mailbox folder permissions. This process is called ReACLn. Without this action, many users will lose access to shared mailbox resources when the SIDHistory attribute is cleared.


Exchange Processing Wizard (Part of Dell Migration Manager for Active Directory)

Dell migration Manager for Active Directory contains the Exchange Processing Wizard. This wizard is able to replace existing source SIDs with the matching target SIDs for permissions inside the exchange environment. The wizard is using the matching information in the QMM AD LDS database, created during the directory synchronization.

To ReACL permissions inside the mailboxes, we have to select the option “Update client permissions”:

ExMigirst04

Now we can choose to process all Public Folders and Mailboxes or select individual Mailboxes or Public Folder or even skip Public Folders or Mailboxes completely:

ExMigirst05

The wizard provides the possibility to only process one server or process multiple servers in parallel.

Known limitation of the Exchange Processing Wizard:

The wizard is unable to set the Free/Busy permissions Free/Busy time, subject, location. After processing, the permission is changed to Free/Busy time only:

ExMigirst06


Good to know – Check real SID behind folder permissions

Get-MailboxfolderPermission: Unfortunately, as long as the SIDHistory is set for a user, Exchange will always resolve the permissions to the target account. So Exchange will always show the TARGETDOMAIN\User although in fact the source SID has permissions on the mailbox folder. You will also see the same result if you query folder permissions via EWS (e.g. with EWSEditor).

MFCMAPI:

To check which SID is really behind the permission, you can use MFCMAPI to access the mailbox.

  1. Create a new profile for mailbox and disable Exchange Cached Mode.
  2. Start MFCMAPI
  3. Click Session->Logon and choose the profile you’ve created in step 1.
  4. Double Click the Mailbox entry and now navigate to the folder for which you want to display the permissions.
  5. On the right side, now double click PR_NT_SECURITY_DESCRIPTOR

In the Smart View, you can see which SID is really behind the Access Control Entry.

QMM 8.10 error: Agent is not ready to start – SCP not found

We used Quest Migration Manager 8.10 recently in a project at a customer for a combined Active Directory and Exchange migration. Overall target was to integrate a Windows 2003 domain cross forest and cross org into the central AD Forest with several child domains. Since from mail perspective our migration source was Exchange 2007 and our migration target Exchange 2013, we decided to use the Native Move Job option along with the Migration Manager for Exchange Agent (MAgE) services.

Situation:

The customer environment look like the following:
Source Domain in Single Domain Forest with Domain Controllers on Windows 2003 and Exchange 2007 as mail system.
Target Domain was one of several child domains in the central Forest. All domain controllers running Windows 2012 R2 and mail system was Exchange 2013 SP1.
All Exchange 2013 servers had been deployed to root domain which also kept all important system and admin accounts.
To limit complexity in the setup of Quest Migration Manager 8.10, we decided to use a single administrative account from target Forest’s root domain and granted all necessary permissions in the domains to run both, Active Directory and Exchange migration. Only for access to source Exchange 2007 when running the move request, we used an account from source domain with Org Admin permissions.

Native Move Job
Setup for Native Move Job

Installation of Migration Manager 8.10. on a member server in target domain (best practice recommendation) including all cumulative hotfixes went smoothly. After successful Directory Synchronization, we connected to the Exchange source and target Organization and finally deployed 2 Instances of the MAgE agent for native mailbox move jobs on our agent host and console server. Note: For agent hosts Windows 2012 R2 is currently (May 2014) not supported. You have to stay with Windows 2008 R2 here.

Problem:

However, after starting the agent services running with our administrative account , we recognized, that we could not open the log file of the agent in the Log Panel inside the Migration Manager for Exchange GUI. We searched for the log file and found it in “c:\progamdata\quest software\Migration Agent for Exchange\NativeMove directory”.

scp not found
Log snippet from MAgE agent

The log file showed that the agent was not starting to process the migration collection due to missing settings and then went to sleep. The lines of error:

 

Waiting for agent settings: Not found: (&(objectClass=serviceConnectionPoint) …..

Agent is not ready to start. Agent going to sleep at 1 minute.

repeated over and over.

Obviously the agent tried to execute an LDAP query to find a connection point in Active Directory.
Note: Currently QMM 8.10 uses 3 different systems to store configuration data: An ADLDS server, a SQL Server Instance and the Active Directory (ADDS).

Service Connection Point (SCP):

We ran the query which was shown in the log file against the target domain and we could find the Service Connection Point (SCP) immediately in the System container of the domain naming context.

QMM_8.10_SCP

The Service Connection Point consists primarily of the keywords array attribute and the serviceBindingInformation attribute. The QMM MAgE looks for the serviceBindingInformation attribute to get its SQL connection properties. In SQL it will finally find all information to process the collection.
QMM_8.10_SCP_3
We do not know why Developers at Dell Software made this process so complex. However, in our setup the agent could not find the Service Connection Point, because the agent was looking in the domain, where its service account was located and this was the root domain of the forest while the agent host had installed the SCP during installation in the child domain where the computer account was member of.

Solution:

Switching the agent host and agent service account to an account from child domain would have been a solution, but was not in compliance with customer policy to host all system accounts in root domain.
Moving agent host and console to root domain would not have meet best practices and would have interfered running directory synchronization.

So we ended up in giving the agent just what it requested:
We manually created a Service Connection Point in the root domain and copied all serviceBindingInformation values over.

The agent started immediately and worked without errors.

For future design we can only recommend to store Service Connection Point in the Configuration Partition as Exchange and lots of other software. Using the domain naming context will always lead to problems in a big Enterprise environment with Active Directory consisting of multiple domains in a  forest.

 

Migration Manager for Exchange (QMM/EX) – version 8.10 is available with full Exchange 2013 support

Migration Manager for Exchange 8.10 is available for download via the Dell/Quest support website along with CPUU 5.2. Remarkable is the support for single-hop Exchange 2003 to Exchange 2013 migration scenarios and public folder migration coexistence to Exchange 2013 public folder mailbox technology.
Together with Exchange 2013 as target mailbox system, wave 15 of Office 365 is also officially supported.

According to the release notes, QMM Exchange 8.10 provides the following features:

  • Migrate Microsoft Exchange 2000–2010 mailboxes to Microsoft Exchange 2013 without locking users out of their mailboxes.
  • Benefit from direct one-step migration from legacy Microsoft Exchange 2000–2010 servers to Microsoft Exchange 2013.
  • Migrate your public folders from Microsoft Exchange 2000–2010 to Microsoft Exchange 2013 with co-existence enabled.
  • Benefit from full two-way calendar (free/busy) synchronization between Microsoft Exchange 2000-2010 and Microsoft Exchange 2013 organizations during the migration.
  • Migrate Microsoft Exchange 2000–2010 mailboxes to Microsoft Office 365 Wave 15 without locking users out of their mailboxes and without extra pre-migration steps. Set up full two-way calendar (free/busy) synchronization during the migration process.
  • Consolidate your multiple source forests into the single Office 365 cloud.
  • Migrate your Microsoft Exchange organization to an existing hybrid environment with single sign-on enabled.
  • Use Migration Manager for Active Directory to create mail-enabled user accounts, and benefit from using the native move job for your Microsoft Exchange 2003–2010 to Microsoft Exchange 2003–2010 migrations (an Exchange 2010 CAS server in the source or target organization is required).

Client Profile Updating Utility 5.2 (aka EMWProf):

  • Client Profile Updating Utility (CPUU): Support switching Microsoft Outlook 2013 user profiles from source to target Exchange server
  • Client Profile Updating Utility (CPUU): Support for Windows 8 as an end-user platform
  • Client Profile Updating Utility (CPUU): Support for Microsoft Exchange Server 2013 as a target.

(Source: “Release Notes for Quest Migration Manager for Exchange, April 30th”)

Exchange Migration to Office 365 can change behind the scenes – wave 15 rolls in

In these days Microsoft is moving from wave 14 to wave 15 for Office 365 cloud installations. This means a service transition from Exchange 2010 Server backend to Exchange 2013 Server backend, Lync 2010 to Lync 2013 and other cool updates.

However, if you have travelled a long way to get a smooth migration setup (with all the back and forth of finding the right strategy and technical conditions) Microsoft can make you a big surprise by changing your Exchange target infrastructure. You started your migration project with Exchange 2010 in the cloud as target system and you end up with Exchange 2013.

This is “by design” when moving services to the cloud – as MVP Sean McNeill stated in his post [http://office365evangelist.com/?p=938]:
This is an important questions because with a move to the cloud, the company give up some control on when, and even if, you will go through an upgrade of the service. The company now relies on the Service Provider, Microsoft in this case, to handle the upgrade and the cadence of the upgrades. This needs to be fully understood and accepted by a company moving to the Cloud.”

To mitigate the risk of forcing the customers to update in times where it is just neither “comfortable” nor “amusable” – as it might be in the middle of an Exchange migration project – Microsoft offers to postpone the update one time. The Office 365 admins receive a notification e-mail which announces the update schedule. From that information the customer has 3 weeks to decide that he better postpone or let Microsoft execute. When he decides to postpone, Microsoft will not start the update for the next 2 months. The timespan to complete the wave 15 upgrade is end of 2013 latest, which means your upgrade cannot be later than this deadline.
http://community.office365.com/en-us/wikis/upgrade/what-to-expect-during-the-service-upgrade.aspx

For more information check the Office 365 Upgrade Center: http://community.office365.com/en-us/wikis/upgrade/office-365-service-upgrade-center-for-enterprise.aspx

Dell/Quest Software seems to recognize first problems in running migration projects and recommends to postpone the Office 365 tenant upgrade by contacting Microsoft.

https://support.quest.com/SolutionDetail.aspx?id=SOL105116&pr=Notes+Migrator+for+Exchange

Quest Migration Manager for Exchange®: 10 commendments for EMWProf usage

Updating Outlook profiles after completing the back end mailbox switch is required to connect the user to the target mailbox.
Quest Migration Manager for Exchange ships with the CPUU (Client Profile Updating Utility), aka EMWProf, to streamline and automate this task. Although the configuration of a starter script is easy, we usually end up with an educated script that does a lot around the EMWProf procedure to avoid issues and to prepare for other services like Lync and Exchange UM.
Based on the experience of several long time projects we recommend to have a look into the following aspects.

10 things to know about Outlook profile updating with EMWProf (CPUU)

 

  1. Check if Outlook 2010 is configured with multiple accounts. Since there is a restriction with CPUU, users with multiple Exchange accounts in a single Outlook 2010 profile have to reconfigure Outlook or need to be supported after migration.
  2. Running EMWProf with administrative credentials can save a lot of access problems when switching profiles. Passwords are encrypted as a feature of the CPUU and this works fine.
  3. Running into the EMWProf script each time when a user logs on, creates too much impact. You can create a group, where membership is given after backend mailbox switch. You can create a check in the EMWProf script that checks whether EMWProf did run before on the client and prevents additional starting of the EMWProf procedure.
  4. Disable Windows Search. In many cases, Windows Search locked the ost file because of indexing and EMWProf could not rename it, what EMWProf caused to fail. You cannot stop the Windows Search service because it restarts by itself. Disable and enable afterwards. With Lync client installed and integrated with Exchange/Outlook, several Lync processes are active and block EMWProf from working properly.
  5. If the user has multiple profiles configured in Outlook 2010, EMWProf tries to process all of them and will create a return code that is not unique. You should first scan the registry for Outlook profiles in CPUU script and then run dedicated EMWProf for each profile, to get the return code. EMWProf will also send a message for each profile separately, which makes it easy to differ between good and bad, important and unimportant.
  6. Deployment of EMWProf via logon script is not enough in nowadays. Many laptops stay in Hibernate only and are unlocked w/o domain logon. As a fallback, a link in the Goodbye message should point to the EMWProf script and can be executed by the user. More advanced solutions distribute the EMWProf binaries via software distribution and the EMWProf script checks first, whether the binaries exist locally and if not, it pulls them from a remote share before executing. This helps a lot in small bandwidth scenarios. Even more educated solutions use a migration database where the EMWProf script can upload the results of the client side part of the mailbox migration.
  7. For terminal server use you should configure a specific EMWProf script. Processing of offline profiles is not necessary there as an example. Make the script as slim as possible and it will work fast and with less issues.
  8. Sometimes localized Goodbye/Welcome messages sent by QMM switch procedure are important. You can change the message text per Mail Source Agent. If you have regional based setup, you can send localized language mails easily by feeding the MSAs with a specific message text.
  9. Do you need a Welcome message sent by QMM? It has advantages and disadvantages. If mailbox switch and Outlook switch was successful, why confusing the user with more technical explanations and notifications.
  10. Heaven and hell of transferring read/unread status of items. QMM mail agents sync read/unread status and CPUU does as well to fill the last gap. This is very helpful to make the target mailbox experience for the user close to the “zero impact” (Quest language!) idea. However, this feature can turn into black, when the user starts to work for weeks with his Laptop and his new mailbox, then goes back to his Desktop Computer, is still connected to his old mailbox and executes EMWProf again. It does what it should and will make the items in target look like in the source mailbox. Be careful, we have seen assistants being very set up when realizing that they have 120 unread messages in the inbox from past weeks [again]. For the very same reason we do not use the SwitchRESMBX utility in situations where people work with passive (not yet switched) mailboxes in target.

Dell Quest Migration Tools: Readiness for Windows Server 2012 and Exchange 2013

Actually we can see a good market response to the new Microsoft Server flagships, Windows Server 2012 and Exchange 2013.
Migration projects are still ahead and probably will not die in 2013.

The following table shows the readiness of the Quest migration tool suite  and answers the question whether the tool can be installed on a Windows Server 2012, whether it can migrate Active Directory to Windows 2012 and mail systems to Exchange 2013.

Microsoft did not release a new version of ADMT yet (the most actual version is still 3.2), that is fully compliant with Windows 2012 functional mode domains, nor can you install ADMT 3.2 on a Windows Server 2012 member server. Actually, a migration from a Windows 2008 R2 domain to a Windows 2012 domain with 2012 functional level can neither be achieved by using the native tool (ADMT) nor Quest Migration Manager for Active Directory.

Active Directory Product Version Tool installation on Windows Server 2012? Can backup/restore AD data on Windows Server 2012 DCs? OR Can migrate data to Windows 2012 DCs?
Recovery Manager for Active Directory Forest Edition 8.2.1 yes yes
Recovery Manager for Active Directory 8.2.1 yes yes
Migration Manager for Active Directory 8.9 No no
       
Migration Product Version Tool installation on Windows Server 2012? Can migrate data to Exchange 2013? Office 365?
Migration Manager for Exchange 8.9 no no/yes
Migration Manager for Exchange IntraOrg Edition 1.0.1 no yes
Notes Migrator for Exchange 4.6.1 no (no Windows 8 admin workstation) yes/yes
Coexistence Manager for Notes 3.4 no yes/yes
Groupwise Migrator for Exchange 4.2 no (no Windows 8 admin workstation) yes/yes

 

Quest Migration Manager for Exchange®: Issues with emwmta.dat file growing to the limit of 2 GB – “cannot open database” error

In some of the large scale migrations (or even in smaller ones when running few mail target agents) we recognize the following issue: Mail Target Agent slows down dramatically and does not seem to process *.pst files completely any further.
The characteristic message in the MTA log file is a frequent logging of the error 3049. You see a normal logon to target mailbox, then the synchronization of a message deletion and then when MTA tries to write to the tracking database it fails.

19.02.2013 21:23:55       CaeMAPISession::OpenPST            TraceMsg             0             Trying to open store.
19.02.2013 21:23:55       CStore::LogInfo   Informational      4905      InfoStore is ‘Mailbox – <DISPLAYNAME>’, locale ‘1033’, codepage ‘1252’, EntryId ‘0000000
[…]
19.02.2013 21:23:55       SyncMachine::Init             TraceMsg             2203      Locking tracking database ‘d:\emwmta.dat’.
[…]
19.02.2013 21:24:00       MailImport::ImportDeletedMessages         TraceMsg             1406               Synchronizing deleted messages: 1 message(s) will be deleted.
19.02.2013 21:24:00       SyncMachine::CreateMessage       Error      3049     
Cannot open database ”.  It may not be a database that your application recognizes, or the file may be corrupt.. ExchangeId:

We investigated the so called tracking database, an MS Access style agent database of the Quest agent, which can be found in the root drive of the agent installation location. There is a *.dat file for every agent. In our scenario, only the *.dat files of the Mail Target Agents gave us headaches.
emwmta

We could identify that the error is shown in the log files whenever the tracking database has reached the size of 2.097.152 KB, which exactly gives 2 GB. And 2 GB is the limitation of Access databases, see discussions here.Now the important question was how to fix this. Support came up with the idea of re-installing agents, but did not want us to delete the *.dat file, which is not helpful when re-installing the agents. Apart from this, repairing the agent deletes your files in the queue.In the end we ran through a row of lab tests and decided to take the risk of deleting the file and starting with a new one for the sake of moving on with the sync. At least for our mail target agents we can confirm that this did the trick without impacting the sync quality.Please contact Dell Support before moving any steps in this direction.

Quest Migration Manager for Exchange® – Pimp your QMM calendar sync power

In one of our previous posts [http://wp.me/p300CJ-15] we showed how important the calendar synchonization is in your Inter-Org Exchange migration project when using Quest Migration Manager for Exchange. The Quest approach runs 2 mailboxes for every user (1 in source, 1 in target organization) which enables a long coexistence but also requires a fast running calendar synchronization between those mailboxes for actual Free/Busy information. As less mailboxes per calendar sync agent instance as faster your calendar sync will run. That is a simple truth. But it is not the only one.

When we started to design our Quest Mail and Calendar Sync Agent setup in our large scale project, the customer was very generous and spent us 25 virtual machines with 2 processor cores each. We installed 9 calendar sync agent instances (QMM 8.8) on each of the servers. The customer still had some resources left and spent us another 2 cores for each virtual machines. Running this “QMM server farm” we expected to meet and even exceed the customer’s expectation of 20 minutes calendar sync latencies when bringing over a new calendar entry. However, the performance was not like expected.
Investigation on the servers gave us the following insight on processor usage:

cal_sync_instances

You can see that the first 2 cores are heavily used while the other 2 cores are in “idle mode”. Some experts started to create a powershell script to set processor affinity to the QMM calendar sync agent (CSA) processes which you can find here. set_proc_affinity

But sometimes things are simpler than they look at first sight. In every Quest mail or calendar sync agent folder you can find an RCmd.ini file. The RCmd.ini file contains the string “ProcessorIndex” which defines the processor instance to be used by the agent. When we parsed the files we recognized the distribution as shown in red column. Only the cores 0 and 1 were defined in the RCmd.ini, but not 2 and 3.

instances

We modified the RCmd.ini files of all calendar sync agent instances like shown in the green column. A new investigation on the server presented a different picture of a well-balanced processor usage:

cal_sync_instances_d

With this insight it was a straight-forward task to create a powershell script that parsed through all agent instances on all QMM agent hosts to adjust the ProcessorIndex settings in the RCmd.ini files.

Good to know how things work.