Search
StarWind is a hyperconverged (HCI) vendor with focus on Enterprise ROBO, SMB & Edge

A highly available SMTP relay solution – Part I

  • April 7, 2020
  • 20 min read
Cloud and Virtualization Architect. Didier is an IT veteran with over 20 years of expertise in Microsoft technologies, storage, virtualization, and networking. Didier primarily works as an expert advisor and infrastructure architect.
Cloud and Virtualization Architect. Didier is an IT veteran with over 20 years of expertise in Microsoft technologies, storage, virtualization, and networking. Didier primarily works as an expert advisor and infrastructure architect.

Introduction

In the year 2020, I still deploy SMTP relay solutions. It is perfectly fine to do so. Many on-premises environments have software and equipment that relies on SMTP to send notifications. These notifications can be purely informational but can also be warnings and alerts. Mail is so ubiquitous that it is and remains a widely used approach to inform people about anything you can imagine.

e-mail

If no other logging or monitoring options are available, emails can be parsed and dumped into monitoring systems, send over SMS, etc. There are plenty of opportunities to automate, extend, and enhance the capabilities of email. It is a well know, familiar format, and a standard.

What about security and the need to run a mail server? If you do not write the code yourself, a Web API based approach (REST) might not be an option. But as long as you use TLS with your on-premises SMTP relay deployment and leverage the SMTP API of a provider, you get the best of both worlds. This way, we use an API abstraction to perform SMTP message handling remotely over an HTTP connection.

Setting this up is also a fun exercise to do with a junior as it touches on the principles of load balancing, SMTP, certificates, and consuming cloud services on-premises.

Requirements & Components

Below we will present an overview of the components we will use to build the solution. Some of these you might already have in place. In that case, this can be a small and inexpensive project.

High availability and load balancing

When relying on email notifications, it can be wise to deploy a highly available SMTP relay solution. Some times even georedundant. You can find an example of such a georedundant solution in a Quick Demo Video Of Site Failover With KEMP LoadMaster Global Balancing. Here, in this article, I will be discussing the high availability aspect of an SMTP relay service within a site or campus. For both high availability and load balancing, I leverage a KEMP Technologies LoadMaster. They provide a free version for your lab or fully-featured trial licenses. You can get physical, virtual, and cloud appliances from them.

KEMP LoadMaster

A robust SMTP provider

One common factor is that many devices and applications have no access to external networks or the internet. That is why we look at an on-premise deployment for the SMTP relay service. The SMTP relay service takes care of that challenge. The actual mail delivery service is something we gladly offload to the smart host of a cloud provider. I use SendGrid as they offer excellent services, and they provide a free account for use in the lab. I also have an account for SendGrid already via my Azure lab, which allows for up to 25K emails per month for free.

SendGrid

Reliable Internet Connectivity

As said, it is only the internal SMTP relay service that needs internet connectivity to send emails out. In case of an internet outage, the on-premises SMTP relay service queues the messages. As longs as someone still cares about getting the mail, the default settings are long enough for you to fix the connectivity issue.

Depending on your needs, you need a redundant internet connection and different email providers to protect against failures of those services. But that is not the focus of this article.

A highly available virtualization solution

The virtual machines (SMTP, KEMP) should be highly available, and for that, you can use Hyper-V and Starwind Virtual SAN, for example.

Hyper-V and Starwind Virtual SAN

If you do not have a high available virtualization solution, you should at least host the different SMTP virtual machines on separate Hyper-V hosts with local storage. The load balancer will make sure the mail flow goes through an available virtual machine.

A good on-premises SMTP Server

Windows Server has a built-in SMTP feature that I leverage. By having two virtual machines, we have all we need to make it highly available via load balancing.

We have two significant options for SMTP relay, and that is to do it anonymously or authenticated. We need anonymously because we still have plenty of devices and software that do not offer authentication. On the other hand, I always provide a secured SMTP relay service as well. It is our default choice when the client supports authentication.

Domain

We can provide both on the same virtual machines. That is what I do, but I do create two separate SMTP services with their distinct IP addresses for this. For one, this means both types of SMTP relay can have their different queues and logs, which helps during troubleshooting. Secondly, you have the option of being a bit more liberal in whitelisting IP addresses that are allowed to connect and relay if you have authentication.

Preparation work before configuring the SMTP services

We have seen the components we will use to build the solution. Now, we will go over the actual implementation. We will do so in more or less detail, depending on the need to do so. As you will understand, this is not a complete manual of every component used.

The below image is a high-level overview of what we will build.

Preparation work before configuring the SMTP services

Setting up the LoadMaster virtual services for load balancing

Log on to the LoadMaster and create two new virtual services. We will name them “Anonymous SMTP Relay” (192.168.2.201:25) and “Authenticated SMTP Relay” (192.168.2.202:587).”

The settings for both are pretty straight forward and identical for both virtual services, bar the IP address and ports. In the figure below, you see an overview for you to follow. It is the configuration for the Authenticated SMTP relay virtual service.

LoadMaster virtual services for load balancing

Note that the “Service Type” is Generic, and the “Server Initiating Protocols” is SMTP. For the health check, make sure you have the correct port and that the “Real Server Check Method” is Mail (SMTP) Protocol.

There are some things you must keep in mind. When you select “Transparency” (highlighted in yellow), you will see the client IP address in the SMTP Logs instead of the VIP. Note that for this to work, you must set the gateway on the SMTP virtual machines to the LoadMaster VIP IP address. You must also know that this means that in the SMTP connection and relay whitelists, you must add the IP addresses of the clients, or they won’t be able to connect.

Then there is the “Subnet Originating Requests” setting, be careful with that one. We do not need it here as we are not offloading SSL across multiple subnets. Leave it disabled, and the real servers see the traffic coming from the VS address. If you enable it, the real servers see the traffic coming from the local LoadMaster node interface address. As you can imagine, this also has its impact on when you set the allowed IP addresses for connecting and relaying over the SMTP relay service.

We will remind you of this when we set up the SMTP relay services.

Another thing to note is that under “Advanced Properties” you can select the IP address of the clients that you allow to connect to the VIP (whitelisting) or the ones you want to block (blacklisting). It helps limit what devices and applications can use the SMTP relay.

That’s pretty much all there is to it. But until we configure the SMTP servers, the health checks will not turn green. Select the IP address of the clients that you allow to connect to the VIP

Now is also an excellent time to create the DNS records for both services. Add an A-record for smtp.datawisetech.corp (192.168.2.201) and smtpauth.datawisetech.corp (192.168.2.202) to the correct DNS zone.

Create the DNS records for both services

Creating the SendGrid SMTP API keys

Login on https://sendgrid.com and navigate to “Email API” and select Integration Guide. That is where you choose the SMTP Relay option.

Creating the SendGrid SMTP API keys

Give your API key a meaningful name and click on Create Key.

Give your API key a meaningful name

You will now see your API key. Save it in a secure place as this is the only opportunity you have to do so. The keys are no longer obtainable after this step. If you lose it, you need to create a new one. That is why I create multiple keys for specific services. That makes changing one less impactful.

Integrate using our web API or SMTP Relay

Check “I’ve updated my settings” and click on “Next: Verify Integration” as we must now verify the integration.

But to do so, we first need to set up our SMTP servers and service. So let’s do that. To avoid having to wait for that, we just use a PowerShell script here. We send an authenticated email with “apikey” as the user, and for the password, we paste the API key in here. The key is fake, by the way, don’t worry.

Note that we disable certificate validation for this to work. We enable it again afterward. Make sure no endpoint protection solution is blocking PowerShell from sending the email.

After you have sent and received a test email, you can finalize the verification process for the API key in SendGrid.

 Finalize the verification process for the API key in SendGrid

SendGrid will let you know when they see the mail flow over their systems. If not, check your settings and try again.

Check your settings and try again

Preparing the SMTP Server virtual machines

Install the SMTP Server feature on your virtual machines. Doing so does not require a restart.

Preparing the SMTP Server virtual machines

When you install the SMTP Server feature, this adds the firewall rule for port 25. We manually add the rules for port 587, which we will use for authenticated SMTP. We use PowerShell for this. Be sure to change the local address to the IP address you use for the authenticated SMTP relay service on your virtual machine.

In regards to networking, you can choose to work with one NIC or multiple NICs for the two SMTP relay services. That depends on your situation. I use one NIC with two IP addresses as these are on the same subnet and VLAN in my example.

IP settings

Remember that the gateway depends on whether you use transparent mode in Layer 7 on the LoadMaster or not. If you use transparency, you must use the LoadMaster VIP here.

I requested a certificate for server authentication from my certificate authority with smtpauth.workinghardinit.corp as FQDN. I install it on each server node in the personal store of the Local Machine certificate store. You can also use commercial, self-signed, or free certificates.

Local Machine certificate store

On both servers, I create the logging folder structure and the SMTP relays folders we will use.

SMTPlogs MailRelays

Change the SMTP service “Startup type” to Automatic. You want it to be running after a reboot.

Change the SMTP service "Startup type" to Automatic

We have now done all the preparation work, and we can configure the actual SMTP Services. For that, I invite you to read part II of this series.

Hey! Found Didier’s article helpful? Looking to deploy a new, easy-to-manage, and cost-effective hyperconverged infrastructure?
Alex Bykovskyi
Alex Bykovskyi StarWind Virtual HCI Appliance Product Manager
Well, we can help you with this one! Building a new hyperconverged environment is a breeze with StarWind Virtual HCI Appliance (VHCA). It’s a complete hyperconverged infrastructure solution that combines hypervisor (vSphere, Hyper-V, Proxmox, or our custom version of KVM), software-defined storage (StarWind VSAN), and streamlined management tools. Interested in diving deeper into VHCA’s capabilities and features? Book your StarWind Virtual HCI Appliance demo today!