I was making another OCD sweep of my Application Logs and found the following error on several Exchange 2013 servers:
After some deep troubleshooting and 2 hours on the phone with Microsoft, they confirmed it's a known issue since Exchange 2013 CU9. The servers that I am on are running Exchange 2013 CU12, and they still haven't found a way to suppress this error.
CU9 introducted a new Transport Agent called System Probe Drop SMTP Agent. Microsoft introducted this new SMTP Agent due to the overwheming number of complaints with logging and Exchange Managed Availability. If you're not familiar with Managed Availability, I strongly recommend reviewing this article HERE for more information. In a nutshell, it's Exchanges way of performing self-monitoring and remediation. Pretty neat!!
So what does this new agent do, and why does it cause these 1040 Events in the Application logs?
As you know with each mailbox database, there are health mailboxes which are used to send messages for the purpose of monitoring the availability of services and various customer touch points (CTPs) within Exchange. These health mailboxes used to stir up quite a bit of "noise" in the transport logs and journaling services. The System Probe Drop SMTP Agent debuted in CU9 as a way of delivering these monitoring messages bypassing the traditional transport queue. And whatever bypasses transport, bypasses logging and journaling which is great news in the case of these health mailboxes. Unfortuantely, this new SMTP agent does not drop these monitoring emails in a traditional manner, but instead, removes all email recipients. This method of dropping the email to avoid traditional transport queue has had an adverse effect. The emails still get passed down the pipeline (it's just a different Agent handling it), but now the relevant sender/recipient pairs are invalid. This results in the email delivery failing for all of these health mailbox "monitoring emails". The following screenshot shows an example of this:
As a result, Event ID 1040 for the MSExchangeTransport service is logged in the Application Logs saying that a small percentage of emails are failing. As confirmed by the screenshot above, they are the probe messages generated by the health mailboxes.
Long story short - as long as you don't have valid messages with an EventID of FAIL, then this error is safe to ignore. I recommend running the following cmdlet to verify this:
Get-MessageTrackingLog -ResultSize unlimited -Server %ServerName% -EventID FAIL