Saturday, May 3, 2014

Dealing with Health Proxy Probe Messages in Exchange 2013 Managed Availability

Exchange 2013 introduces a new feature called Managed Availability which performs system monitoring of various components of Exchange infrastructure and provides the ability to detect and recover from problems as soon as they occur.  One of the tests Exchange Managed Availability performs is to send an "Inbound Proxy Probe" message once ever 5 minutes from an Exchange 2013 front end server to the backend server.  If you have a multi-role server deployment, this proxy probe occurs locally on the Exchange 2013 server between the frontend mail service "Microsoft Exchange Frontend Transport" and the backend transport service "Microsoft Exchange Transport". The purpose of this test is to ensure transport functionality is working as expected between these components.

This test however can be problematic and two issues are often seen by Exchange Administrators including:
  • In the event Content Filtering (Intelligent Message Filter) is configured on the Exchange 2013 backend with the install-AntispamAgents.ps1 script, Inbound Proxy Probe messages will be quarantined, rejected or deleted by the spam filtering engine.
  • The Inbound Proxy Probe messages get delivered to the Exchange 2013 backend health mailbox and stored - these can build up over time causing clutter.
Both of these issues have been addressed below.

Exchange Content Filtering Blocking Probe Messages

Exchange Content Filtering can block messages relayed between the Exchange 2013 front end and Exchange 2013 back end.  In my lab environment I have a Spam Confidence Level (SCL) set to 5 then quarantine, this is resulting in a large volume of probe messages being quarantined in my spam mailbox as shown below.


In my lab environment these "Subject: Inbound proxy probe" messages are being sent from postmaster@at.local to inboundproxy@contoso.com (the address for the inbound proxy as from Exchange 2013 SP1).


To stop these messages from being caught by Intelligent Message Filter simply put in an exclusion by using the Set-ContentFilterConfig command.  In my environment the command I used was as follows:

Set-ContentFilterConfig -BypassedSenders postmaster@at.local -BypassedRecipients inboundproxy@contoso.com

Another option stopping the Monitoring mailboxes from being filtered by IMF as follows:

Get-Mailbox -Monitoring | Set-Mailbox -Monitoring -SCLDeleteThreshold 9 -SCLDeleteEnabled:$false -SCLRejectThreshold 9 -SCLRejectEnabled:$false -SCLQuarantineThreshold:9 -SCLQuarantineEnabled:$false -SCLJunkThreshold 9 -SCLJunkEnabled:$false

The messages were coming from the sender postmaster@at.local and the recipient was inboundproxy@contoso.com, as a result this exclusion stops IMF from detecting these messages as potential spam.

Build Up of Health Probe Messages

As Inbound proxy probe messages are sent ever 5 minutes, a build up of these messages can accumulate on the Exchange 2013 backend health mailboxes.  To view the health mailboxes in an Exchange Management Shell, use the Get-Mailbox command with the -Monitoring switch.  To view item counts of the health mailboxes, run the following command

Get-Mailbox -Monitoring | Get-MailboxStatistics | ft DisplayName,ItemCount,LastLogonTime


Note: Every Mailbox Database contains two health mailboxes in Exchange 2013 by default.

As you see in my environment, my health mailboxes have a large build up of probe messages shown by the item count.  To control the build-up of messages in the health mailboxes, you can simply leverage Exchange Retention - something which has been around for a while in Exchange!  To do this you need to create both a retention policy and a retention tag which can be done with Exchange Management Shell (EMS) or by using the new Exchange Administration Centre (EAC).

First Create a Retention Tag, I called mine "Delete items older then 2 days" and configured the tag as follows:


Then create a Retention Policy and link the Tag.  I called my Retention Policy "Health Mailbox Retention Policy".


Apply the retention policy only to your Health Mailboxes which can be done with the following command:

Get-Mailbox -Monitoring | Set-Mailbox -RetentionPolicy "Health Mailbox Retention Policy"

Check that it applied with the following command:

Get-Mailbox -Monitoring | fl *RetentionPolicy*

Now the Managed Folder Assistant will automatically delete emails older then two days.  The Managed Folder Assistant is always running and begins cleaning emails at times when the server is at low utilisation as to not disrupt business, however you can force the Managed Folder Assistant to do the first initial clean-up of your health mailboxes with the following command:

Get-Mailbox -Monitoring | Start-ManagedFolderAssistant

After it finishes its initial cleanup, you will notice the item count within these health mailboxes has significantly reduced.


Note: In the event you create additional mailbox databases, new health mailboxes will be created.  Ensure you link the retention policy to any new health mailboxes.

5 comments:

  1. Hello,

    I entered the "Set-ContentFilterConfig -BypassedSenders postmaster@ourdomain.com -BypassedRecipients inboundproxy@contoso.com" and I'm still receiving emails in my spam mailbox.
    I also did a manual force sync of AD and restarted Transport service on all my 2013 MBX servers. Does it take a while to take effect?

    Thanks,

    Rocky

    ReplyDelete
    Replies
    1. Hey Rocky,
      You should use set-contentfilterconfig -bypassedsenders inboundproxy@contoso.com
      Since the mails come from inboundproxy@contoso.com it have to be in the bypassedsender not bypassedrecipients.
      Greets Gamba

      Delete
  2. Thanks for this! Does MS just expect you want these to accumulate forever? Their lack of documentation is frustrating.

    ReplyDelete
  3. I received a fist full of errors when I ran the retention policy command:

    Unable to generate the e-mail address. Unable to load address module 'C:\Program Files\Microsoft\Exchange Server\V15\Mailbox\address\SMTP\AMD64\inproxy.dll' for address type 'SMTP'. Additional message: 'The specified module could not be found'.

    Investigating the path, I noticed I do not have a 'Mailbox' folder under V15. This is an OOTB install of CU7 and everything seems to be working ok. I came upon your rather excellent post as I implemented the anti-spam agents and started experiencing the inboundproxy@contoso.com messages.

    Any ideas?

    ReplyDelete
    Replies
    1. Obviously this was not an exchange issue, at least not functionally. After installing CU8 the appropriate directories were present. I cannot explain why they were ever missing, but am happy to report that I could successfully complete the steps you outlined in this article - removing thousands of messages in the monitoring mailboxes. THANK YOU!

      Delete