Web Application Proxy Event 13007

Written by Robert Kettel

When you start to use Web Application Proxy Server (WAP) as a replacement for ISA, TMG or UAG and publish Active Sync through it, you might face a lot of Event 13007 warnings in the Microsoft-Windows-Web Application Proxy/Admin event log (I mean, really “a lot”). These are paired with various Event IDs 13006 (“Connection to the backend server failed. Error: (0x80072efe)”).

On the other hand, you don’t get any complaints from the user community. There does not seem to be an impact at all.

Where do these warnings come from? Do they impact our service and can we prevent them from showing up over and over again?

The main cause for the warning 13007 and 13006 is how devices with Exchange Active Sync (EAS) and Direct Push technology connect to Exchange.

Following this Microsoft TechNet article, “a mobile device that’s configured to synchronize with an Exchange 2013 server issues an HTTPS request to the server. This request is known as a PING. The request tells the server to notify the device if any items change in the next 15 minutes in any folder that’s configured to synchronize. Otherwise, the server should return an HTTP 200 OK message. The mobile device then stands by. The 15-minute time span is known as a heartbeat interval.”

With other words, there is a steady HTTPS session for 15 minutes between the EAS device and the Exchange backend which must be supported by all components taking part in the HTTPS session build, usually firewalls, load balancers and (in our case) the WAP servers which proxy the HTTPS session coming from the LBs to the Exchange backend. The long lasting session request is finally terminated by the Exchange server by posting an HTTP 200 message.

Now, looking at the default settings of our WAP servers we find a parameter which can have an influence on that behavior.
The default value for the InactiveTransactionTimeoutSec parameter is 300 (= 5 minutes). That means that if the accessing party does retrieve new responses from the backend service defined in the application settings for more than 5 minutes, the connection is identified as “timed out” and dropped by the WAP server.

From that perspective a connection to the Exchange Backend Service is timed out for the Web Application Proxy (causing a warning event 13007), when
a) the heartbeat interval of Direct Push is longer than the InactiveTransactionTimeoutSec
AND
b) the session was not renewed by the device
AND
c) there was nothing to synchronize in the first 300 seconds of the connection

However, if the device’s HTTPS session is dropped by the WAP server, it will automatically re-initiate a new session (the same what the device would do when getting an HTTP 200 OK message from the backend Exchange server). Therefore this is not a critical behavior at all.

The difference can be found how the device reacts on the dropped connection in comparison with how it handles the HTTP 200 OK post.
In the latter case, the device starts a new HTTPS session with the same heartbeat interval again.
In the first case, the device “assumes” that 15-minute HTTPS requests are blocked and re-initiates a new session with only 8 minutes (480 seconds) – which is still beyond the 300 second default setting of the WAP servers.

To avoid the Event 13007, the InactiveTransactionTimeOutSec parameter needs to be set to a value greater than the defined Active Sync max period. By default, the Active Sync device starts with a 15-minute interval, which would fit into a value of 910 for the InactiveTransactionTimeoutSec.

Since the parameter can be set for each published application individually, you luckily do not need to touch other published applications than Exchange Active Sync.

Example command:

Get-WebApplicationProxyApplication ExchangeActiveSync | Set-WebApplicationProxyApplication –InactiveTransactionsTimeoutSec

Note:

A support engineer with Microsoft Premier stated that there is currently no way to suppress the Event 13007 from appearing in the log files completely.

New Release of DELL Migration Manager for Exchange 8.11

While we were all waiting for the arrival of Migration Manager for Exchange 9.0, DELL Software released a new version 8.11 of its Exchange migration flagship.

New in the release 8.11 is the support for installing the Migration Manager components on Windows Server 2012 and 2012 R2 OS platforms. On the other hand, installations on Windows Server 2008 and Windows 7 are not supported anymore:

  • Migration Manager supports migration from the source Exchange 2013 organization to the target Exchange 2010 or Exchange 2013 organization using Native Move Job.
  • All Dell Migration Manager components can now be installed on Microsoft Windows Server 2012 and Microsoft Windows Server 2012 R2
  • Operating systems not supported from Migration Manager console:
    – Microsoft Windows Server 2008 Service Pack 1 or higher (x86 edition)
    – Microsoft Windows 7 Service Pack 1 or higher (x64 edition)
    – Microsoft Windows Server 2008 Service Pack 1 or higher (x64 edition)
    – Microsoft Windows 7 Service Pack 1 or higher (x86 edition)
  • Operating systems not supported from Migration Agent for Exchange:
    – Microsoft Windows Server 2008 Service Pack 1 or higher (x86 edition)
    – Microsoft Windows Server 2008 Service Pack 1 or higher (x64 edition)
    – Microsoft Windows 7 Service Pack 1 or higher (x86 edition)

You can find a complete list of changes and release related information of the product here:

https://support.software.dell.com/download-install-detail/5775891?prodjid=1-7GCB8-249&utm_campaign=42330-32498-SU-GL-MigrationManagerforExchange&utm_medium=email&utm_source=Eloqua

Exchange Migration first & Mailbox Folder Permissions

In many inter forest migration projects, the mailbox migration to the new Exchange Organization into the new forest is performed first and the user migration is performed in a separate step at a later time. After the mailbox migration and before migrating the user objects, you have a classic Resource Forest scenario. Users are created as disabled user accounts in the target forest, receiving a linked mailbox, connected to the source Active Directory user. Important AD attribute in this scenario is msExchMasterAccountSID. This attribute of the disabled target user object holds the objectSID of the source user account. This allows to connect to the own mailbox and shared mailbox resources (Delegate permissions etc.) with the active source user object.

ExMigirst01

Did you ever thought about mailbox folder permissions in this scenario? 

For every migrated folder permission (e.g. with QMM for Exchange) and also every time a user manually adds mailbox folder permissions for another (not Active Directory migrated) user in the target mailbox, the SID of the source user object is added to the mailbox folder permissions. In this example, we’ve selected the not migrated user UserA from the Global Address list and added him as delegate for the Inbox and the calendar for the Info MBX:  ExMigirst02

At some point, the Active Directory migration will start. During this process, the user account in the target domain will be activated and the Linked Mailbox is converted to a User Mailbox. This action will clear the attribute msExchMasterAccountSID. This is necessary because the target account will now be used to access the own mailbox and resources of other mailboxes. If a migrated user is now added to the mailbox folder permissions, the target SID will be added and no longer the source SID. Let’s use the mailbox example above and add the migrated user UserB as additional delegate for the Calendar of the Info MBX:ExMigirst03

In this example, UserB will of course not have any problems accessing the Info MBX. But what happens if UserA will be migrated and the user starts accessing the Info MBX with TARGETDOMAIN\UserA? The SID of the target account has no permissions on the Inbox and the Calendar folder. Will UserA loose access to these folders now? Generally, the answer is YES, UserA will lose access! But…

In Active Directory Migration projects, it is best practice to migrate the SIDHistory to the target user account. In this case, the objectSID of the source user is copied to the attribute SIDHistory of the target account. For our example, it means that UserA will not lose access to the Info MBX because his Access Token contains the Source SID which has permissions on the Inbox and Calendar folder in the Info MBX.

SIDHistory CleanUp & mailbox folder permissions?

Clearing SIDHistory is part of most of the migration projects. Before clearing the SIDHistory attribute of the target accounts, it is required to replace the source SIDs with the corresponding target SIDs inside the mailbox folder permissions. This process is called ReACLn. Without this action, many users will lose access to shared mailbox resources when the SIDHistory attribute is cleared.


Exchange Processing Wizard (Part of Dell Migration Manager for Active Directory)

Dell migration Manager for Active Directory contains the Exchange Processing Wizard. This wizard is able to replace existing source SIDs with the matching target SIDs for permissions inside the exchange environment. The wizard is using the matching information in the QMM AD LDS database, created during the directory synchronization.

To ReACL permissions inside the mailboxes, we have to select the option “Update client permissions”:

ExMigirst04

Now we can choose to process all Public Folders and Mailboxes or select individual Mailboxes or Public Folder or even skip Public Folders or Mailboxes completely:

ExMigirst05

The wizard provides the possibility to only process one server or process multiple servers in parallel.

Known limitation of the Exchange Processing Wizard:

The wizard is unable to set the Free/Busy permissions Free/Busy time, subject, location. After processing, the permission is changed to Free/Busy time only:

ExMigirst06


Good to know – Check real SID behind folder permissions

Get-MailboxfolderPermission: Unfortunately, as long as the SIDHistory is set for a user, Exchange will always resolve the permissions to the target account. So Exchange will always show the TARGETDOMAIN\User although in fact the source SID has permissions on the mailbox folder. You will also see the same result if you query folder permissions via EWS (e.g. with EWSEditor).

MFCMAPI:

To check which SID is really behind the permission, you can use MFCMAPI to access the mailbox.

  1. Create a new profile for mailbox and disable Exchange Cached Mode.
  2. Start MFCMAPI
  3. Click Session->Logon and choose the profile you’ve created in step 1.
  4. Double Click the Mailbox entry and now navigate to the folder for which you want to display the permissions.
  5. On the right side, now double click PR_NT_SECURITY_DESCRIPTOR

In the Smart View, you can see which SID is really behind the Access Control Entry.

QMM 8.10 error: Agent is not ready to start – SCP not found

We used Quest Migration Manager 8.10 recently in a project at a customer for a combined Active Directory and Exchange migration. Overall target was to integrate a Windows 2003 domain cross forest and cross org into the central AD Forest with several child domains. Since from mail perspective our migration source was Exchange 2007 and our migration target Exchange 2013, we decided to use the Native Move Job option along with the Migration Manager for Exchange Agent (MAgE) services.

Situation:

The customer environment look like the following:
Source Domain in Single Domain Forest with Domain Controllers on Windows 2003 and Exchange 2007 as mail system.
Target Domain was one of several child domains in the central Forest. All domain controllers running Windows 2012 R2 and mail system was Exchange 2013 SP1.
All Exchange 2013 servers had been deployed to root domain which also kept all important system and admin accounts.
To limit complexity in the setup of Quest Migration Manager 8.10, we decided to use a single administrative account from target Forest’s root domain and granted all necessary permissions in the domains to run both, Active Directory and Exchange migration. Only for access to source Exchange 2007 when running the move request, we used an account from source domain with Org Admin permissions.

Native Move Job
Setup for Native Move Job

Installation of Migration Manager 8.10. on a member server in target domain (best practice recommendation) including all cumulative hotfixes went smoothly. After successful Directory Synchronization, we connected to the Exchange source and target Organization and finally deployed 2 Instances of the MAgE agent for native mailbox move jobs on our agent host and console server. Note: For agent hosts Windows 2012 R2 is currently (May 2014) not supported. You have to stay with Windows 2008 R2 here.

Problem:

However, after starting the agent services running with our administrative account , we recognized, that we could not open the log file of the agent in the Log Panel inside the Migration Manager for Exchange GUI. We searched for the log file and found it in “c:\progamdata\quest software\Migration Agent for Exchange\NativeMove directory”.

scp not found
Log snippet from MAgE agent

The log file showed that the agent was not starting to process the migration collection due to missing settings and then went to sleep. The lines of error:

 

Waiting for agent settings: Not found: (&(objectClass=serviceConnectionPoint) …..

Agent is not ready to start. Agent going to sleep at 1 minute.

repeated over and over.

Obviously the agent tried to execute an LDAP query to find a connection point in Active Directory.
Note: Currently QMM 8.10 uses 3 different systems to store configuration data: An ADLDS server, a SQL Server Instance and the Active Directory (ADDS).

Service Connection Point (SCP):

We ran the query which was shown in the log file against the target domain and we could find the Service Connection Point (SCP) immediately in the System container of the domain naming context.

QMM_8.10_SCP

The Service Connection Point consists primarily of the keywords array attribute and the serviceBindingInformation attribute. The QMM MAgE looks for the serviceBindingInformation attribute to get its SQL connection properties. In SQL it will finally find all information to process the collection.
QMM_8.10_SCP_3
We do not know why Developers at Dell Software made this process so complex. However, in our setup the agent could not find the Service Connection Point, because the agent was looking in the domain, where its service account was located and this was the root domain of the forest while the agent host had installed the SCP during installation in the child domain where the computer account was member of.

Solution:

Switching the agent host and agent service account to an account from child domain would have been a solution, but was not in compliance with customer policy to host all system accounts in root domain.
Moving agent host and console to root domain would not have meet best practices and would have interfered running directory synchronization.

So we ended up in giving the agent just what it requested:
We manually created a Service Connection Point in the root domain and copied all serviceBindingInformation values over.

The agent started immediately and worked without errors.

For future design we can only recommend to store Service Connection Point in the Configuration Partition as Exchange and lots of other software. Using the domain naming context will always lead to problems in a big Enterprise environment with Active Directory consisting of multiple domains in a  forest.

 

Migration Manager for Exchange (QMM/EX) – version 8.10 is available with full Exchange 2013 support

Migration Manager for Exchange 8.10 is available for download via the Dell/Quest support website along with CPUU 5.2. Remarkable is the support for single-hop Exchange 2003 to Exchange 2013 migration scenarios and public folder migration coexistence to Exchange 2013 public folder mailbox technology.
Together with Exchange 2013 as target mailbox system, wave 15 of Office 365 is also officially supported.

According to the release notes, QMM Exchange 8.10 provides the following features:

  • Migrate Microsoft Exchange 2000–2010 mailboxes to Microsoft Exchange 2013 without locking users out of their mailboxes.
  • Benefit from direct one-step migration from legacy Microsoft Exchange 2000–2010 servers to Microsoft Exchange 2013.
  • Migrate your public folders from Microsoft Exchange 2000–2010 to Microsoft Exchange 2013 with co-existence enabled.
  • Benefit from full two-way calendar (free/busy) synchronization between Microsoft Exchange 2000-2010 and Microsoft Exchange 2013 organizations during the migration.
  • Migrate Microsoft Exchange 2000–2010 mailboxes to Microsoft Office 365 Wave 15 without locking users out of their mailboxes and without extra pre-migration steps. Set up full two-way calendar (free/busy) synchronization during the migration process.
  • Consolidate your multiple source forests into the single Office 365 cloud.
  • Migrate your Microsoft Exchange organization to an existing hybrid environment with single sign-on enabled.
  • Use Migration Manager for Active Directory to create mail-enabled user accounts, and benefit from using the native move job for your Microsoft Exchange 2003–2010 to Microsoft Exchange 2003–2010 migrations (an Exchange 2010 CAS server in the source or target organization is required).

Client Profile Updating Utility 5.2 (aka EMWProf):

  • Client Profile Updating Utility (CPUU): Support switching Microsoft Outlook 2013 user profiles from source to target Exchange server
  • Client Profile Updating Utility (CPUU): Support for Windows 8 as an end-user platform
  • Client Profile Updating Utility (CPUU): Support for Microsoft Exchange Server 2013 as a target.

(Source: “Release Notes for Quest Migration Manager for Exchange, April 30th”)

Dell Quest Migration Tools: Readiness for Windows Server 2012 and Exchange 2013

Actually we can see a good market response to the new Microsoft Server flagships, Windows Server 2012 and Exchange 2013.
Migration projects are still ahead and probably will not die in 2013.

The following table shows the readiness of the Quest migration tool suite  and answers the question whether the tool can be installed on a Windows Server 2012, whether it can migrate Active Directory to Windows 2012 and mail systems to Exchange 2013.

Microsoft did not release a new version of ADMT yet (the most actual version is still 3.2), that is fully compliant with Windows 2012 functional mode domains, nor can you install ADMT 3.2 on a Windows Server 2012 member server. Actually, a migration from a Windows 2008 R2 domain to a Windows 2012 domain with 2012 functional level can neither be achieved by using the native tool (ADMT) nor Quest Migration Manager for Active Directory.

Active Directory Product Version Tool installation on Windows Server 2012? Can backup/restore AD data on Windows Server 2012 DCs? OR Can migrate data to Windows 2012 DCs?
Recovery Manager for Active Directory Forest Edition 8.2.1 yes yes
Recovery Manager for Active Directory 8.2.1 yes yes
Migration Manager for Active Directory 8.9 No no
       
Migration Product Version Tool installation on Windows Server 2012? Can migrate data to Exchange 2013? Office 365?
Migration Manager for Exchange 8.9 no no/yes
Migration Manager for Exchange IntraOrg Edition 1.0.1 no yes
Notes Migrator for Exchange 4.6.1 no (no Windows 8 admin workstation) yes/yes
Coexistence Manager for Notes 3.4 no yes/yes
Groupwise Migrator for Exchange 4.2 no (no Windows 8 admin workstation) yes/yes

 

Coexistence Manager for Notes – new release 3.3

Dell/Quest Software released a new version of Coexistence Manager for Notes 3.3. Coexistence Manager for Notes provides coexistence between Exchange and Notes E-Mail Systems and eases migration activities in large scale infrastructures.

The release 3.3 is compatible with Exchange 2013, Exchange 2010 SP3 and Outlook 2013 and provides enhanced iOS Support.