Jump to content

Sender Policy Framework

From Wikipedia, the free encyclopedia
This is an old revision of this page, as edited by Coderjo (talk | contribs) at 20:01, 4 May 2010 (The example domain is example.com, not example.org). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Sender Policy Framework (SPF), as defined in RFC 4408, is an e-mail validation system designed to prevent e-mail spam by addressing a common vulnerability, source address spoofing. SPF allows administrators to specify which hosts are allowed to send e-mail from a given domain by creating a specific DNS SPF record in the public DNS. Mail exchangers then use the DNS to check that mail from a given domain is being sent by a host sanctioned by that domain's administrators.[1]

Principles of operation

Further information: [[:Sender Rewriting Scheme (SRS)]]

Normal SMTP allows any computer to send an e-mail claiming to be from anyone. Thus, it is easy for spammers to send e-mail from forged addresses. This makes it difficult to trace back to where the spam truly comes from, and easy for spammers to hide their true identity in order to avoid responsibility. Many believe that the ability for anyone to forge sender addresses (also known as Return-Paths) is a security flaw in modern SMTP, caused by an undesirable side-effect of the deprecation of source routes.

SPF allows the owner of an Internet domain to use special format of DNS records ("SPF", type 99) to specify which machines are authorized to transmit e-mail for that domain. For example, the owner of the example.org domain can designate which machines are authorized to send e-mail whose sender e-mail address ends with "@example.org". Receivers checking SPF can reject messages from unauthorized machines before receiving the body of the message. Thus, principles of operations are quite similar to those of DNSBL, except that SPF leverages the authority delegation scheme of the real Domain Name System. Early implementations used TXT records for quick deployment before the new type was available in many server programs. Use of TXT records for SPF as a transitional mechanism, although still common in 2010, is deprecated.

The sender address is transmitted at the beginning of the SMTP dialog. If the server rejects the sender, the unauthorized client should receive a rejection message, and if that client was a relaying MTA, a bounce message to the original sending address may be generated. If the server accepts the sender, and subsequently also accepts the recipient(s) and the body of the message, it should insert a Return-Path header in the message's body in order to save the sender address. While the address in the Return-Path often matches other originator addresses in the mail header like "From:" or "Sender:" this is not necessarily the case, and SPF does not prevent forgeries of these other addresses.

Spammers can send e-mail with an SPF PASS result if they have an account in a domain with a sender policy, or abuse a compromised system in this domain. However, doing so makes the spammer easier to trace and prosecute.

The main benefit of SPF is to people whose e-mail addresses are forged in the Return-Paths. They receive a large mass of unsolicited error messages and other auto-replies, making it difficult to use e-mail normally. If such people use SPF to specify their legitimate sending IPs with a FAIL result for all other IPs, receivers checking SPF can reject forgeries, thus reducing or eliminating the amount of backscatter.

SPF has potential advantages beyond helping identify unwanted e-mail. In particular, if a sender provides SPF information, then receivers can use SPF PASS results in combination with a white list to identify known reliable senders. Scenarios like compromised systems and shared sending mailers limit this use.

Reasons to implement

If a domain publishes an SPF record, spammers and phishers are less likely to forge e-mails pretending to be from that domain, since the forged e-mails are more likely to be caught in spam filters which check the SPF record. Therefore, an SPF protected domain is less attractive to spammers and phishers. Since an SPF protected domain is less attractive as a spoofed address, it is less likely to be blacklisted by spam filters and so ultimately the legitimate e-mail from the domain is more likely to get through.[2]

FAIL and forwarding

SPF does not allow plain message forwarding. When a domain publishes an SPF FAIL policy, then legitimate messages sent to receivers forwarding their mail to third parties can be rejected and bounced if any of the following occur:

  1. The forwarder does not rewrite the Return-Path, unlike mailing lists.
  2. The next hop does not white list the forwarder.
  3. This hop checks SPF.

This is a necessary and obvious feature of SPF – checks behind the "border" MTA (MX) of the receiver cannot work directly.

Publishers of SPF FAIL policies must accept this potential problem. They should test (e.g., with a SOFTFAIL policy) until they are satisfied with the results. See below for a list of alternatives to plain message forwarding.

HELO tests

For an empty Return-Path as used in error messages and other auto-replies, an SPF check of the HELO-identity is mandatory.

With a bogus HELO identity the result NONE would not help, but for valid host names SPF also protects the HELO identity. This SPF feature was always supported as an option for receivers, and later SPF drafts including the final specification recommend to check the HELO always.

This allows to white list sending mailers based on a HELO PASS, or to reject all mails after a HELO FAIL. It can also be used in reputation systems (any white or black list is a simple case of a reputation system).

Implementation

Compliance with SPF consists of three loosely related tasks:

Publish a policy
Domains identify the machines authorized to send e-mail on their behalf. Domains do this by adding additional records to their existing DNS information: every domain name that has an A record or MX record deserves an SPF record specifying the policy if it is used either in an email address or as HELO/EHLO argument.
Check and use SPF information
Receivers use ordinary DNS queries, which are typically cached to enhance performance. Receivers then interpret the SPF information as specified and act upon the result.
Revise mail forwarding
Plain mail forwarding is not allowed by SPF. The alternatives are
  • remailing, i.e. replacing the original sender with one belonging to the local domain,
  • refusing, i.e. answering 551 User not local; please try <user@example.com>,
  • whitelisting on the target server, so that it will not refuse a forwarded message, and
  • Sender Rewriting Scheme, a more complicated mechanism that handles routing non-delivery notifications to the original sender.

Thus, the key issue in SPF is the specification for the new DNS information that domains set and receivers use. The records are laid out like this (in typical DNS-syntax): 

example.com. IN SPF "v=spf1 a mx -all"

"v=" defines the version of SPF used. The following words provide mechanisms to use to determine if a domain is eligible to send mail. The "a" and "mx" specify the systems permitted to send messages for the given domain. The "-all" at the end specifies that, if the previous mechanisms did not match, the message should be rejected.

Mechanisms

Eight mechanisms are defined:

ALL Matches always; used for a default result like -all for all IPs not matched by prior mechanisms.
A If the domain name has an address record (A or AAAA) that can be resolved to the sender's address, it will match.
IP4 If the sender is in a given IPv4 address range, match.
IP6 If the sender is in a given IPv6 address range, match.
MX If the domain name has an MX record resolving to the sender's address, it will match (i.e. the mail comes from one of the domain's mail servers).
PTR If the domain name (PTR record) for the client's address is in the given domain and that domain name resolves to the client's address (forward-confirmed reverse DNS), match.
EXISTS If the given domain name resolves to any address, match (no matter the address it resolves to). This is rarely used. Along with the SPF macro language it offers more complex matches like DNSBL-queries.
INCLUDE If the included (a misnomer) policy passes the test this mechanism matches. This is typically used to include policies of more than one ISP.

Qualifiers

Each mechanism can be combined with one of four qualifiers:

  • + for a PASS result. This can be omitted; e.g., +mx is the same as mx.
  • ? for a NEUTRAL result interpreted like NONE (no policy).
  • ~ for SOFTFAIL, a debugging aid between NEUTRAL and FAIL. Typically, messages that return a SOFTFAIL are accepted but tagged (see SPF FAIL Rejection below).
  • - for FAIL, the mail should be rejected (see below).

Modifiers

The modifiers allow for future extensions to the framework. To date only the two modifiers defined in the RFC 4408 have been widely deployed:

  • exp=some.example.com gives the name of a domain with a DNS TXT record (interpreted using SPF's macro language) to get an explanation for FAIL results—typically a URL which is added to the SMTP error code. This feature is rarely used.
  • redirect=some.example.com can be used instead of the ALL-mechanism to link to the policy record of another domain. This modifier is easier to understand than the somewhat similar INCLUDE-mechanism.

Error handling

As soon as SPF implementations detect syntax errors in a sender policy they must abort the evaluation with result PERMERROR. Skipping erroneous mechanisms cannot work as expected, therefore include:bad.example and redirect=bad.example also cause a PERMERROR.

Another safety guard is the maximum of ten mechanisms querying DNS, i.e. any mechanism except from IP4, IP6, and ALL. Implementations can abort the evaluation with result SOFTERROR when it takes too long or a DNS query times out, but they must return PERMERROR if the policy directly or indirectly needs more than ten queries for mechanisms. Any redirect= also counts towards this processing limit.

A typical SPF HELO policy v=spf1 a -all may execute up to three DNS queries: (1) SPF, (2) TXT (deprecated, but for backwards compatibility during the transition), and (3) A or AAAA. This last query counts as the first mechanism towards the limit (10). In this example it is also the last, because ALL needs no DNS lookup.

Caveats

SPF avoids abuses of domain names; it does not validate that a given e-mail actually comes from the claimed user. Users within a given domain can forge each others' address ("cross-user forgery" or even "cross-domain" forgery where outbound mail servers are shared), if permitted by the signature process.

SPF FAIL rejection

SPF FAIL policies can be an effective but dangerous tool. Some publishers of SPF policies try to avoid the dangers by using SOFTFAIL (designed for limited testing periods) instead of FAIL. But SOFTFAIL can be even more dangerous than FAIL with receivers rejecting FAIL and accepting SOFTFAIL tagged as potential junk.

A reject in a forwarding scenario is a clean decision. The forwarder will send an error message to the address in the Return-Path. Typically the error message (bounce) contains an explanation of the SMTP error and the failing (forwarded to) address. The original sender can then send his mail again directly to this address bypassing the forwarder, a crude emulation of the normal SMTP error code 551 user not local.

However, an accepted SOFTFAIL tagged as potential junk could be deleted by the final recipient. This is a user who has arranged his forwarding in a way that cannot work with SPF; this user could also be careless with checking his potential junk and simply delete it.

The same line of argument also suggests that receivers should take the SPF recommendation to reject SPF FAIL seriously where possible. Accepting SPF FAIL results as potential junk can be more dangerous than simply rejecting FAILed mails. While senders with an SPF FAIL can be expected to know what it means, the same is obviously not the case for a receiver arranging his forwarding in a way that cannot work with SPF.

Checkpoints

Checking SPF behind the "border" MTA (MX) is not impossible; the relevant info is usually noted in a Received timestamp line added by one of the MXs of the receiver. But at this time it is too late to reject SPF FAIL; the checking entity can essentially only delete FAILing mail.

Experts should be able to get this right, but it is no "plug and play" situation; therefore the SPF specification recommends to check SPF only at the "border" (MX) in the SMTP session, not later. Later SPF checks can also have unexpected results if the publishers of sender policies do not plan modifications of their policy carefully with regard to DNS cache expiration.

DoS attack

An Internet draft[3] discussed concerns related to the scale of an SPF answer leading to network exploits as a means to corrupt the DNS. This issue is also covered in the security considerations of the SPF RFC. The SPF project did a detailed analysis [4] of this draft and claimed that SPF does not pose any unique threat of DNS DoS.

Relationship with DKIM

SPF validates the message envelope (the SMTP bounce address), not the message contents (header and body) – this is the distinction between SMTP (as specified in STD 10 or RFC 5321) and Internet Message Format (as specified in STD 11 or RFC 5322). It is orthogonal and complementary to DomainKeys Identified Mail (DKIM), which signs the contents (including headers).

In brief, SPF validates MAIL FROM vs. its source server; DKIM validates "From:" by cryptographic means.

Sender ID

Sender ID [RFC 4406], is a more recent solution to the problem of message validation, and defines a pair of closely-related tests. One validates a message's Purported Responsible Address (PRA) as defined in [RFC 4407]. The other validates a message's Reverse-Path (also known as MAIL-FROM address) as defined in [RFC 4408].

Sender ID uses DNS records that are also used by SPF. Depending on the content of the record, this may mean that Sender ID heuristics would be applied incorrectly to a message. Depending on the actions associated by the recipient with those heuristics, the message may not be delivered or may be discarded on receipt. Those publishing SPF DNS records should consider the advice given in section 3.4 of [RFC 4406] and may wish to publish both v=spf1 and spf2.0 records to avoid the conflict.

History

The idea to limit by IP address who could send mail using a given sender domain may date back as far as 1997. The first public mention of the concept was in 2000 but went mostly unnoticed. No mention was made of the concept again until a first attempt at an SPF-like specification was published in 2002 on the IETF "namedroppers" mailing list by David Green, who was unaware of the 2000 mention of the idea. The very next day, Paul Vixie posted his own SPF-like specification on the same list. These posts ignited a lot of interest, and eventually led to the forming of the IETF Anti-Spam Research Group (ASRG) and their mailing list, where the SPF idea was debated among a subscriber base that seemed to grow exponentially day by day. Among the proposals submitted to the ASRG were "Reverse MX" by Hadmut Danisch, and "Designated Mailer Protocol" by Gordon Fecyk.[5]

In June 2003, Meng Weng Wong merged the RMX and DMP specifications[6] and solicited suggestions from other programmers. Over the next six months, a large number of changes were made and a large community had started working on SPF.[7]

Originally SPF stood for Sender Permitted From and was sometimes also called SMTP+SPF, but it was changed to Sender Policy Framework in February 2004.

In early 2004, the IETF created the MARID working group and tried to use SPF and Microsoft's CallerID proposal as the basis for what is now known as Sender ID.

After the collapse of MARID the SPF community returned to the original "classic" version of SPF. In July 2005 this version of the specification was approved by the IESG as an IETF experiment, inviting the community to observe SPF during the two years following publication. On April 28, 2006, the SPF RFC was published as experimental RFC 4408.

Controversy

In 2004, Steven M. Bellovin wrote an e-mail that discusses his concerns with SPF.[8] Some of these include:

  • SPF originally used TXT records in DNS, which are supposed to be free-form text with no semantics attached. SPF proponents readily acknowledge that it would be better to have records specifically designated for SPF, but this choice was made to enable rapid implementation of SPF. In July 2005, IANA assigned the Resource Record type 99 to SPF. During the transition, SPF publishers may publish both record types and SPF checkers may check for either type. It may likely take many years before all DNS software fully supports this new record.
  • As of the time he wrote his message, there was no consensus that this is the right way to go. Some major e-mail service providers have not bought into this scheme. Unless and until they do, it does not help much, either for their customers (who make up a substantial proportion of the user population) or for everyone else (since their addresses could be forged). It is worth noting that since this concern was raised, Google Mail and AOL, among others, have embraced SPF.[9]
  • Bellovin's strongest concerns involve the underlying assumptions of SPF (SPF's "semantic model"). When using SPF, the SPF DNS records determine how a sender is allowed to send. That means that the owner of the domain will control how senders are allowed to send. People who use "portable" e-mail addresses (such as e-mail addresses created by professional organizations) will be required to use the domain owner's SMTP sender, which may not currently even exist. Organizations providing these "portable" addresses could, however, create their own 'mail submission agents (MSAs) (RFC 4409) or offer VPNs. Besides, SPF only ties the SMTP Return-Path to permitted MSAs; users are still free to use their RFC 2822 addresses elsewhere.

There are other concerns about the impact of widespread use of SPF, notably the impact on various legitimate forms of email spoofing[10], such as forwarding services, SMTP use by people with multiple identities, etc. (For example, a person who uses their home ISP's SMTP servers to send mail with their work email as the address.)

Deployment

Anti-spam software such as SpamAssassin version 3.0.0 and ASSP implement SPF. Many mail transfer agents (MTAs) support SPF directly such as Courier, CommuniGate Pro, Wildcat, and Microsoft Exchange, or have patches/plug-ins available that support SPF, including Postfix, Sendmail, Exim, and Qmail. More than one million domains publish SPF FAIL -all policies.[11]

In a survey published in 2007, 5% of the .com and .net domains had some kind of SPF policy. In 2009, a continuous survey run at Nokia Research reports that 51% of the tested domains specify an SPF policy.[12] These results can include trivial policies like v=spf1 ?all.[13] In April 2007, BITS, a division of the Financial Services Roundtable, published e-mail security recommendations for its members including SPF deployment.[14]

In 2008, the Messaging Anti-Abuse Working Group (MAAWG) published a paper about email-authentication covering SPF, Sender ID, and DKIM.[15]. In their "Sender Best Communication Practices" the MAAWG stated: "At the very least, senders should incorporate SPF records for their mailing domains".[16]

See also

References

  1. ^ "Sender Policy Framework: Introduction".
  2. ^ "Why should I implement a SPF record on my domain?". Email Manual. 2009. Retrieved 2010-01-01. {{cite web}}: Unknown parameter |month= ignored (help)
  3. ^ SPF DoS Exploitation (expired Internet draft)
  4. ^ OpenSPF analysis of the DoS attack draft
  5. ^ "SPF: History/Pre-SPF". Retrieved 16 May 2009.
  6. ^ For a comparison among RMX, DMP and SPF, see RMX and DMP compared on the historical openspf site.
  7. ^ "SPF: History/SPF-2003". Retrieved 16 May 2009.
  8. ^ Steve Bellovin expresses doubts (Jan 2004)
  9. ^ "SPF Information". AOL Postmaster. Retrieved 2007-10-04.
  10. ^ "Problems with Designated Sender". Taughannock Networks. Retrieved 2009-12-16.
  11. ^ "SPF -all Domain Survey". 2008. Retrieved 2008-07-23.
  12. ^ "Nokia Research Report on SFP Adoption".
  13. ^ Liu, Cricket (2007). "Handicapping New DNS Extensions and Applications". ONLamp. Retrieved 2007-10-04. {{cite web}}: Unknown parameter |month= ignored (help)
  14. ^ "BITS Email Security Toolkit" (PDF). BITS. 2007. Retrieved 2008-06-13. {{cite web}}: Unknown parameter |month= ignored (help)
  15. ^ Crocker, Dave (2008). "Trust in Email Begins with Authentication" (PDF). MAAWG. Retrieved 2008-06-13. {{cite web}}: Unknown parameter |month= ignored (help)
  16. ^ "MAAWG Sender Best Communications Practices - Version 2.0" (PDF). MAAWG. 2008-06-02. Retrieved 2008-06-13.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Sender_Policy_Framework&oldid=360136661"