Thursday, November 17, 2011

Load Balance SMTP with F5 BIG-IP

The F5 BIG-IP has a template for Exchange 2010 which assists administrators with configuring load balancing for Outlook Anywhere, Active Sync and Outlook Web App. This template does not configure SMTP load balancing. There are many circumstances where you may want an SMTP endpoint IP address to be highly available and load balanced between multiple hub transport servers.

In this post I will go through and show you how to configure the BIG-IP LTM for load balancing the SMTP protocol and the challenges associated with this. This article was written using the F5 BIG-IP LTM VE version 10.2.3.

Create a Health Monitor

Create a health monitor which monitors the Exchange 2010 SMTP service on our Exchange 2010 servers. The heath monitor will send SMTP HELO requests on a regular basis to ensure the SMTP servers are healthy.

Expand Local Traffic and click Add next to Monitors.



I called the monitor SMTP_Monitor. Set the Type to SMTP. Provided an interval of 120 seconds meaning the monitor will send an SMTP HELO every 2 minutes to the Ex2010 servers to see if they are still online. Configured the Alias service port to 25.



Create a Pool for the SMTP Servers

A load balancing pool is a logical set of devices, in our case SMTP servers, that you group together to receive and process traffic. To create a new pool under Pool List click Add.



I called the pool smtp_pool. Select the SMTP_Monitor we created earlier. Select the load balancing method as Round Robin. Add the SMTP servers to our pool in which we wish to distribute inbound SMTP connections to.



Create an SMTP Virtual Server

Create an SMTP Virtual Server on the F5 BIG-IP which will allow the BIG-IP system to listen on TCP25 to load balance incoming SMTP sessions. To do this under Virtual Servers --> Virtual Server List click add.



I called the SMTP Virtual Server SMTP_VS. Under Destination I specified 172.16.51.174. This is the Virtual IP the BIG-IP will listen on for incoming SMTP traffic. Select a service port of 25 and place the device in an enabled state.

Under configuration set SNAT Pool to Auto Map (I have explained what Auto Map is below).



Scroll down further and under resources set the Default Pool to smtp_pool along with the default persistence profile to source_addr.



Test our BIG-IP SMTP Virtual Server

The F5 BIG-IP device should now be configured to load balance SMTP requests between the two Exchange 2010 servers. In your Virtual Server List the SMTP_VS should come up green.



From a command prompt verify you can telnet our SMTP virtual server on 172.16.51.174 on port 25.



We can see that it successfully connected to one of the SMTP servers in our load balancing pool "smtp_pool"



At this point your F5 BIG-IP is successfully load balancing SMTP.

The Problem...

When building an email solution it is absolutely critical to avoid becoming an open SMTP relay. Most organisations implement relay restrictions by locking anonymous relay down to certain source IP addresses on their internal network such as applications and printers. Source IP is generally the preferred method as Administrators do not have to deal with SMTP authentication methods. The list of IP addresses who are allowed relay anonymously are usually configured on the Exchange SMTP receive connectors. However when dealing with load balancers such as a F5 BIG-IP Local Traffic Manager this becomes a difficult task.

Why?

Whilst load balancing connections the F5 BIGIP uses SNAT to re-write the source IP address on the SMTP packets to one of its "Self IP" addresses or "Virtual IP" addresses. This means the Exchange servers will see all requests coming from the same IP address making it impossible to determine which request belongs to what client. This is illustrated in the following diagram:



However I have a workaround for you. If we setup two SNAT addresses on the F5 BIG-IP for example 172.16.51.174 and 172.16.51.175 we can configure our BIG-IP to say any source IP addresses that need to be an anonymous open relay hit our Exchange 2010 servers from 172.16.51.174 ELSE hit our Exchange 2010 servers from 172.16.51.175. This solution means we need to configure our list of allowed IP addresses for SMTP relay on our F5 BIG-IP instead of our Exchange SMTP Receive Connectors.

Create a Data Group List

First we must create a list of IP addresses we want to allow anonymous relay for on our F5 BIG-IP. These are the IP addresses we would normally configure on our Exchange receive connectors. To do this we need to create a new data group list.

Add a new data group list by expanding iRules --> Data Group List and clicking the add button.



I called my Data Group List smtp_relay_allowed and specified the IP address 172.16.51.21. You can add as many IP addresses as you want for anonymous relay.



Create a new iRule

An iRule is a powerful and flexible feature of BIG-IP devices which provide you with unprecedented control to directly manipulate and manage any IP application traffic. By creating an iRule we can instruct the BIG-IP to return a different SNAT address based on on the condition. We want to instruct our BIG-IP to perform the following:

IF a clients source IP is on our Data Group List THEN use an SNAT address of 172.16.51.174 ELSE use the SNAT address of 172.16.51.175.

To create the iRule under Local Traffic Select iRule --> iRule List and click the add button.



I called my iRule smtp_irule and created the following code to perform my required conditions as mentioned above.



A copy of the code:

when CLIENT_ACCEPTED {
set accepted_snat "172.16.51.174"

if { [ class exists smtp_relay_allowed ] }
{
if { [class match [IP::client_addr] equals $::smtp_relay_allowed] }
{
snat $accepted_snat
} else {
snat automap
}
} else {
snat automap
}
}


Automap is a feature of the BIGIP where it automatically selects a Self IP at random to use for the SNAT translation. A Self IP is an IP you have assigned to the BIGIP manually under your network configuration. This is different to a Virtual IP address which is created when you setup a virtual server. I only have one Self IP on my BIG IP set to 172.16.51.175 and one virtual IP set to 172.16.51.174 used by all my F5 virtual servers on different TCP ports. As a result automap will ONLY select 172.16.51.175.

Why would you want multiple source SNAT IP addresses?

For each connection made from the BIG-IP to your load balanced servers a TCP source port needs to be opened for the communication. TCP only has 65535 ports for source and destination traffic so if the number of connections exceeded the number of available ports, the BIG-IP would not be able to take new connections. In this event you could add an additional Self IP and rely on the Automap feature or create an SNAT pool which is a predefined list of IP addresses the BIG-IP is allowed to use for SNAT.

I recommend reading the chapter on SNAT configuration from the F5 website:

http://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/ltm_configuration_guide_10_0_0/ltm_snat.html

How do I configure Exchange 2010:

On your Exchange 2010 servers create your two receive connectors as normal. Create one receive connector configured for your anonymous relay and setup yoru default receive connector for all other SMTP traffic. Both receive connectors must listen on port 25. Do this on all Exchange servers in your pool.

For information on how to configure your application receive connector for anonymous relay follow this blog post:

http://clintboessen.blogspot.com/2009/07/allow-network-applications-to-relay.html

Apply my new iRule to the SMTP Virtual Server

Next we need to attach the iRule to the SMTP virtual server in the F5 configuration screen. To do this go to your Virtual Servers --> Click Virtual Server List then select our SMTP_VS created earlier.



Select Resources then under iRules click Manage.



Select our smtp_irule out of the list available then click finish.



Testing our Configuration

So time to test the configuration to ensure it works as configured. Lets telnet my BIG-IP SMTP Virtual Server from the host we allowed 172.16.51.23 by running the following command:

telnet 172.16.51.174 25

I then wrote some random comments in the telnet session so we can identify our server 172.16.51.23 in our SMTP logs.



I then repeated the procedure from another server which is not in our Data Group List created above.



In our SMTP logs on our Exchange 2010 server as expected the 172.16.51.23 server came from 172.16.51.174 and the non trusted IP came from 172.16.51.175.

32 comments:

  1. Great article, thanks a lot!

    ReplyDelete
  2. So comprehensive, thanks for the detail!

    ReplyDelete
  3. If you're trying to run this on v11 with a CMP enabled device you need to call the data group differently.

    This line should like this:

    if { [class match [IP::client_addr] equals "smtp_relay_allowed"] }

    Changes to "class" are available here (http://devcentral.f5.com/wiki/iRules.class.ashx)

    ReplyDelete
    Replies
    1. Thanks for the tip!! I've been beating my head against the wall for hours trying to figure it out.

      Delete
  4. Bad kind of implementation if you need the real source address of the client in for antispamming stuff (RBL, greylisting, blacklisting, etc).

    No surprise that this is for Exchange.. if there is traffic its ok !!! .. not always !!

    ReplyDelete
  5. The comment by Anonymous is correct, this configuration should only be used when the F5 BIG-IP is put behind your spam filter otherwise things such as RBL's will not work.

    ReplyDelete
  6. I've got a lot of experience implementing F5 in an ISP environment, and using SNAT is not the correct way to do it.

    F5 works best when it is the default gateway for a network and anything which needs to talk to load balanced service should be routed. A typical setup is similar to the following:

    Client:
    IP Address: 192.168.1.1/24
    Gateway: 192.168.1.254

    Server:
    IP Address: 192.168.2.1/24
    Gateway: 192.168.2.254

    F5
    Self IP 1: 192.168.1.254/24
    Self IP 2: 192.168.2.254/24
    VIP: 10.10.10.10

    Using the above setup, the following occurs:

    Client sends packet source ip 192.168.1.1 to destination ip 10.10.10.10
    F5 rewrites packet to destination 192.168.2.1, leaving the source IP alone
    Server receives packet source ip 192.168.1.1 destination ip 192.168.2.1
    Server replies as normal
    F5 performs address translation, rewriting source ip to 10.10.10.10 and leaving the destination IP alone


    This works brilliantly, and allows you to take advantage of the full set of features supported by the F5 platform, such as SSL Offload and full featured iRules.

    If you cannot modify your network to accomodate this, you can utilize the nPath routing (Direct Server Return) to preserve IP addresses, but this has it's own challenges (you can't use SSL offload or most kinds of iRule statements).

    Hope that helps someone else who comes across this article.

    ReplyDelete
  7. Hi Nom,

    This article was intended for customers who do not have an F5 BIG-IP as their default gateway. I do not work for an ISP, and I go out to clients and implement F5 as a load balancer. Most clients already have underlining network infrastructure in place and do not want to setup an F5 BIG-IP to act as their default gateway.

    Kind Regards,
    Clint

    ReplyDelete
  8. I have a question. We're running a 3 server DAG and each of the servers is in a pool on the F5 with a VM that uses this pool. The problem is the F5 seems to be stripping the original source IP and replacing it with its own IP. This makes it impossible for our RBL filtering to work properly and we're getting a lot of SPAM.

    Do you know a work around for this?

    ReplyDelete
  9. Hello,
    We want to configure F5 to load balance SMTP traffic. We want internet emails to arrive on F5 and then it should load balance the traffic to the Hub Transport servers on the backend.
    We have more than 200 SMTP domains.
    (Note:
    1: This is to load balance SMTP and provide SMTP redundancy. No Edge transport servers are being used
    2: F5 version is 11.0.0 & Exchange 2010)

    Can anybody suggest, how will we operate this many SMTP domains?

    Many Thanks..!
    -Mayur

    ReplyDelete
  10. I think that this article shows how to do RBLs on the BigIP. Why allow a connection to be processed by the BigIP and then hit your mail server before you block it. If nothing else, you allow a hacker to use up your BigIP resources and your server resources. The BigIP is your security shield against many different attacks. You could even parse the SMTP protocol commands and filter on that. See this: https://devcentral.f5.com/wiki/iRules.SMTPProxy.ashx

    ReplyDelete
  11. What is the point in setting persistence for SMTP? Doesn't this force an external SMTP server to always hit the same internal Exchange Server?
    For instance, we have several partners the send us large quantities of email and they are set to use our Transport Servers as SmartHosts. With a Source Presist profile setup for SMTP, each of the Partner SMTP servers only hits a single internal Tansport. When I remove the persist profile the SMTP rule I can go into my Transport's SMTP logs and see that the Partner SMTP servers are hitting all of internal Transports.

    ReplyDelete
  12. Hi Clint

    I've got a client who has discovered that some of his applications and devices are failing in sending email using SMTP via the F5 to the new consolidated Exchange (using F5) solution. I know its vague, I'm trying to get more information but I'll respond as I get the info....

    ReplyDelete
    Replies
    1. This this person ever give a reason as I have the same issue

      Delete
  13. is there anyone working on the upgrade of version 10 to 11 bigipf5

    ReplyDelete
  14. Hi. I'm new to configuring Exchange with the F5 so I wanted to ask for clarification for my own sanity. I've created the application relay recive connector in Exchange 2010 using the same method as in the link you provided. Using your example above, would I enter 172.16.51.174 into the "Receive mail from remote servers.." section on the Network tab to prevent unauthorized connections to this receive connector - or will this only work if I check the Anonymous Users box on the Permission Group tab and allow any IP to use this connector? Thanks! Rob

    ReplyDelete
  15. I have followed the instructions but when I connect to the VIP with telnet it times out. If I telnet directly to the SMTP server it works just fine. Any idea how to troubleshoot why?

    ReplyDelete
  16. Thanks for sharing the valuable info regarding SMTP and putting lots of efforts to make this blog with helpful images and sharing with us.

    SMTP Services

    ReplyDelete
  17. Thanks for this information however, how would I keep my originating IP's? The client IP's to keep track of malicious emails being sent, if they are being SNAT or automaped? Is there a way to keep the original IP's I have tried nPath but that is not working any ideas?

    ReplyDelete
  18. THis is a great article thanks for posting. I was wondering if you would do the same thing if you were using just an IIS SMTP server, for outgoing emails only?

    ReplyDelete
  19. Dean132 there is no reason why you cant use IIS SMTP with the above solution.

    ReplyDelete
  20. Hi All,
    in a similar configuration I had the problem that SMTP Virtual Server was dropping a lot of connection and the iRule was not parsed as expected. In my environment I modified the Health Monitor to use the dafault TCP instead of SMTP. Than I modified the SMTP Virtual Server to perform autosnat (instead of doing it into the iRule) and the iRule using the pool command instead of snat (or forward).
    Regards.
    Red,

    ReplyDelete
  21. To check the IP you're coming from there's no need to type random comments and check the logs, you can simply issue a "helo" command and the Exchange server will reply back with the IP they see as source of the connection (tested on Exchange 2013)

    ReplyDelete
  22. Dear Clint, does this instruction used so SMTP which ask authentication for relaying email can be relayed from f5? And i had configured ste by step like your configuration, but when i do telnet to the server, the result is :

    220 **********************************************************************************************

    is it okay or is that any faulty about this result?

    ReplyDelete
  23. Really good site dude. I found the best VPN for a mobile device here http://thebestproxyserver.com/overplay-review/

    ReplyDelete
  24. Clint,
    I've set this up on a v11.1 LTM, and am getting an error:

    TCL error: /Common/o365-smtp-relay_irule - bad IP address format (line 7) invoked from within "snat $accepted_snat"

    Here's the format of my iRule as it stands now:

    when CLIENT_ACCEPTED {
    set accepted_snat "o365-smtp-relay_snat"

    if { [ class exists smtp_relay_allowed ] }
    {
    if { [class match [IP::client_addr] equals "smtp_relay_allowed"] }
    {
    snat $accepted_snat
    } else {
    snat automap
    }
    } else {
    snat automap
    }
    }

    Has there been a syntax change to the 'snat' command since your initial post was written? I'm struggling to figure it out, and would appreciate any help.

    Thanks!

    Jeff B

    ReplyDelete
  25. Hi Jeff, this code will work on v11.1 LTM:

    when CLIENT_ACCEPTED {
    set accepted_snat "10.1.8.12"

    if { [ class exists smtp_relay_allowed ] }
    {
    if { [class match [IP::client_addr] equals smtp_relay_allowed] }
    {
    snat $accepted_snat
    } else {
    snat automap
    }
    } else {
    snat automap
    }
    }

    ReplyDelete
  26. In v10 you wrote the equals bit like "equals $::smtp_relay_allowed", now you need to write it as "equals smtp_relay_allowed" without the $::

    Regards,
    Clint

    ReplyDelete
  27. ahmad, the telnet output your receiving is when you have a Cisco device with IP Inspect on SMTP. It inspects for SMTP traffic but cannot definite the difference between SMTP and ESMTP and as a result communication fails. Remove the inspect rule.

    ReplyDelete
  28. Although I found my query PTCL SMTP Settings for Outlook on other blog. But I would like to say your blog is quite informative..
    Thanks Abid Bhatti

    ReplyDelete
  29. I have the F5 in L2 so, I'm not using SNAT, in this scenary is not necessary use this iRule right?

    But the problem is that when i do telnet to the Virtual Server, the result is :

    220 *******

    ReplyDelete