Changing Perimeter Gateways (Based on EOP to Proofpoint Experience)

About a year ago my organization decided to move away from EOP (Exchange Online Protection) and implement Proofpoint’s cloud-based Email Protection service. As I go through my notes here and recall how smoothly that migration went I’d like to share a few lessons learned from that experience:

  1. Clean up your current gateway solution. I had taken ownership of EOP from our security team about six months before we started planning the Proofpoint cutover, and during that period I was able to streamline the EOP configuration. IIRC I started with about 43 rules, and got it down into the teens by consolidating rules (I had at least three for blacklisting!). Beyond consolidation some things are just no longer needed and can be deleted.
  2. Decide which rules\policies should be actioned on the internet perimeter and which rules\policies should stay within the Organization. When using Exchange Online you may need to keep some rules there so they apply to intra-organizational emails.
  3. For every rule\policy you plan to move, have a strategy for testing it once the new gateway is configured.
  4. Test the new gateway from the outside world. You can fully configure and validate your new gateway before the cutover using various SMTP tools.
  5. Make sure your SPF record is updated with the new gateway IPs. If you are using O365 and placing a gateway in front of it, your SPF needs to have the new gateway IPs, all of the O365 IPs, and if you have an on-prem Exchange envrionment the external IPs that your Edges or Transport Servers use to communicate with Exchange Online. This can be done prior to the cutover.
  6. During the cutover window all you should have to do is update your MX records and configure your messaging environment to only accept traffic from your new gateway. And test – always test.


Recovering from Exchange Database Dismount due to Transaction Log Accumulation

As we all know, Microsoft Exchange uses transaction logs to commit data to the mailbox databases. One of the first critical issues we usually run into during our messaging career is a database dismount due to running out of capacity for the transaction logs. This is usually caused by at least one failed backup, and the emergency fix is to clear some of the oldest transaction logs from the drive so you can remount the database. Once the DB is back up (in a DAG you’ll have to manually delete the same transaction logs from each copy to restore HA), you may want to quickly truncate all of the remaining possible logs to maximize the time you have before the problem happens again. To do that, Microsoft has created a tool for testing VSS capabilities on an Exchange server (versions 2010, 2013, and 2016), but it has the additional benefit of kicking off the log truncation process while it runs. You can find the tool here:

A few things to note before using the VSSTester.ps1 script:

  • You would run this tool on the server with the active copy of the database
  • If this is the server the backup failed on, you may need to reboot (you could restart the Information Store service instead but IMHO if you’re going to move all the DBs off that server to bounce the service might as well reboot to clear out any other gremlins). For the tool to complete successfully all of the VSS Writers and such need to be in the correct state
  • Once this has successfully cleared your logs on the active copy the passive copies should truncate their logs as expected.
  • If you’re having problems running the tool on one server you can always make another copy of the DB active on another server and run it there