When we speak of “Email Authentication”, we’re referring to a technique that is meant to provide to the recipient of a message some level of certainty that the message actually originated with the claimed source of the message. The idea is to achieve some level of defense against fraudulent email, such as phishing and spoofing, that could erode a recipient’s trust in receiving email. That said, the act of sending authenticated email does not, in and of itself, assert that the email is good or wanted email; it only means that the mail is such that a reputation for the authenticated party can be reliably established and used in email acceptance and placement decisions.

There are two primary forms of email authentication in use today—Sender Policy Framework (SPF) and DomainKeys Identifed Mail (DKIM). In this blog post, I’ll explain what SPF is, and how it works.

SPF Overview

Sender Policy Framework (SPF), to paraphrase RFC 7208, is a protocol that not only allows an organization to authorize hosts and networks to use its domain names when sending email, but also provides a way that a receiving host can check that authorization.

When you’re done reading this post, I hope you’ll have learned the following:

• SPF is a “path-based” authentication system.

• SPF policies are declared in DNS TXT records.

• Validation is keyed to the Return-Path domain (what we here at SparkPost call the “bounce domain”) or the HELO domain.

• Validation can be done early in the SMTP transaction, so it’s a good check to use to quickly reject unauthenticated mail.

• It is prone to a relatively high percentage of false negatives.

• 
  

“Path-Based” Authentication

SPF is deemed to be a path-based authentication system because it’s tied solely to the path the message took to get from its origin to its destination. When a sending server initiates an SMTP transaction with a receiving server, the receiving server will determine whether or not the sending server is authorized to send that message based on the domain’s SPF policy. It’s solely a decision based on where the message is coming from, with nothing whatsoever to do with the content of the message.

Declaring an SPF Policy

A domain’s SPF policy record is just a specifically formatted DNS TXT record, commonly containing one or more of the following “mechanisms”:

• v=spf1 – Required first token to indicate that TXT record is SPF record (a domain can have multiple TXT records)

• ipv4, ipv6 – Used to specify IP addresses and networks that are permitted to send mail for the domain

• a – If the sending domain has a DNS “A” record that resolves to the sending IP, the IP is permitted

• mx – If the sending IP is also one of the MX records for the sending domain, the IP is permitted

• all – Required last token, always modified:

  • -all – Only what’s listed here can pass; reject failures.

  • ~all – Stuff that’s not here should softfail; accept it but note it.

  • ?all – Not sure if stuff that’s not here should pass.

  • +all – We think SPF is useless; pass everything.

Other less common mechanisms that are still in widespread use are:

• include – Used when a domain not only has its own servers, but also uses someone else’s servers. Must be used judiciously. Each include is another DNS query, and SPF records can’t require more than ten DNS queries to resolve.

• redirect – When domain A and domain B are owned by the same entity and use the same infrastructure. The owners typically want to maintain just one copy of the SPF record for both, and configure B to redirect queries to A.

• exists – If a domain has lots of small, non-contiguous network blocks, it can use this mechanism to specify its SPF record, instead of listing them all. Useful to stay within size limit (512 bytes) for DNS response.

A note about the “ptr” Mechanism

A final mechanism, “ptr” existed in the original specification for SPF, but has been declared “do not use” in the current specification. The ptr mechanism was used to declare that if the sending IP address had a DNS PTR record that resolved to the domain name in question, then that IP address was authorized to send mail for the domain.

The current status of this mechanism is that it should not be used. However, sites doing SPF validation must accept it as valid.

Some Example SPF Records

Here are some example records, and their meanings. This example shows RFC 1918 addresses, but I recognize that such addresses will never show up in real SPF records.

• MX record, one IP address, one IP network, softfail everything else:

  • “v=spf1 mx ipv4:10.0.0.1 ipv4:192.168.24.0/24 ~all”

• A record, two IP networks, reject everything else:

  • “v=spf1 a ipv4:10.0.3.0/23 ipv4:192.168.55.0/26 -all”

• We send no mail:

  • “v=spf1 -all”

• We think SPF is stupid:

  • “v=spf1 +all”

• The SPF record for the domain sparkpostmail.com (redirect mechanism used)

  • “v=spf1 redirect=_spf.sparkpostmail.com”

• The SPF record for _spf.sparkpostmail.com (exists and ptr mechanisms used):

  • “v=spf1 exists:%{i}._spf.sparkpostmail.com ptr:sparkpostmail.com ptr:spmta.com ptr:flyingenvelope.com ~all”

At SparkPost, we’re currently using the ptr mechanism in our SPF record. We have found it necessary due to a lack of universal support for validating the exists mechanism. And to date, we’ve seen no SPF failures result due to our using the ptr mechanism.

How Does SPF Authentication Work?

Here is what a typical SMTP transaction looks like when SPF is involved:

• The sending server (client) connects to receiving server

• The receiving server notes the client’s connecting IP address

• They exchange greetings, including client’s HELO hostname

• Client issues SMTP MAIL FROM command

The receiving server can now look up SPF record for either MAIL FROM domain or HELO hostname to determine whether or not the connecting IP is authorized to send mail for the domain, and decide whether or not to accept the message.

MAIL FROM or HELO – Which to Check?

The SPF specification stipulates that receiving sites MUST check SPF for the MAIL FROM domain, and RECOMMEND checking it for the HELO hostname. In practice, checking only happens on the MAIL FROM domain, except perhaps for those times when the MAIL FROM address is the NULL sender (i.e., “<>”), in which case the HELO hostname is the only identity available.

SPF Is Not Foolproof

While it seems a relatively straightforward way to authenticate email, SPF is vulnerable to failure in the form of false negatives. These failures can occur more frequently than many are comfortable with.

Here are two common scenarios where perfectly legitimate mail can fail SPF validation and so appear to be fraudulent:

• Automated forwarding, where a message sent to jsmith@alumni.university.edu, a mailbox set to automatically forward all mail to jsmith@isp.com

• Mailing lists, where mail sent to talk-about-stuff@listserv.foo.com gets “exploded” and sent to each subscriber

In both of these cases, if the Return-Path is unchanged from its original, the mail may fail SPF checks (depending on the modifier used with the ‘all’ mechanism). This is because it arrives at its final destination from an intermediate server, not the original one, and that intermediate server is not likely to be in the domain’s SPF record. A technique called “Sender Rewriting Scheme” or “SRS” can mitigate this risk, and some larger sites have adopted it, but it’s not universal.

Wrap Up

So that’s SPF authentication, how it works, how to declare an SPF policy, and what the pitfalls are in using SPF. SPF was the first email authentication scheme to achieve widespread adoption, but it’s not the only one out there. SPF authentication is most effective when deployed in combination with other anti-fraud techniques.