How emails are sent from Xero
How emails are sent from Xero
Technical discussion
In June 2013 we made a change to the way emails are sent from Xero. Some of our users have asked us why the change was necessary and whether we are planning to provide more options in the future. Be warned this is a fairly technical topic so take a deep breath.
Xero allows organisations to raise invoices and statements and send them to their customers.
Sending email on behalf of hundreds of thousands of Xero organisations, to the millions of their customers is a significant undertaking, and one that we take seriously.
Recently we changed the mechanism by which we send customer invoice emails. This post is an attempt to describe the new setup, the reasoning behind the decision, and a roadmap for future changes that we hope to make.
Setting the scene:
The email landscape has changed dramatically over the past 5 years, and mail servers and email ISPs have all developed unique practices to try and shield their users from spam and other unsolicited email that currently floods the internet.
Over the past 10 years, several standards and defacto practices have evolved to help mail servers make decisions on which emails to trust:
Sender Policy Framework (SPF) a 10 year old standard that lets the owner of a mail domain (e.g. ) specify which mail servers are allowed to send email
DomainKeys Identified Mail (DKIM) a slightly newer standard that allows the sender of an email to digitally sign their email to prove that it came from them
Grey listing a heuristic that some receiving mail servers use to slow down the receipt of email, on the theory that spammers won't attempt to queue and retry messages
Spam filtering a set of algorithms that receiving mails servers can run on email to try and determine if the content of the email is "spammy"
Blacklisting various databases of IP addresses or email addresses that are known to send spam
Domainbased Message Authentication, Reporting & Conformance (DMARC) a recent specification that combines SPF and DKIM to allow email senders to prove that the mail was sent by them
We realise that our users absolutely, positively want email that they send to be received by their customers their cashflow and business depend on timely payments. Even though email is by design an unreliable communication mechanism, we are trying to make it as robust and reliable
as possible for Xero customers to use email to send invoices to their customers.
Our previous approach:
Our previous approach to sending invoice emails from within Xero was to "impersonate" or send email "on behalf" of the user currently logged in to Xero.
e.g. If joe.bloggs@ had signed up to Xero, and proved that they controlled the email address, then we would happily send out email that looked like this:
From: Joe Bloggs To: A Customer Subject: Here is your invoice, pay me now!
In 2006, this was a fairly reliable solution, and most mail servers would happily accept these emails and deliver them to the recipient.
In the subsequent years, it was possible for Joe Bloggs to add an SPF record to their to indicate that Xero was able to send email on their behalf. This worked reasonably well for a few more years.
The Problem:
It is now no longer possible for SaaS vendors to reliably send email "on behalf" of their customers.
If a recipient mail server receives an email like the above example, the mail server will perform some checks to try and decide whether this email was legitimately sent by Joe Bloggs.
One of the indications the server will use is the IP address of the server that delivered the message to it. If the server does not look like a legitimate server for Joe Bloggs' ISP (), then that's a negative mark against the email. Traditionally a combination of reverse DNS lookups and SPF would be used to perform this check.
These days, setting SPF records is not sufficient to prove the authenticity of an email, and this example email illustrates the reason why:
Envelope Sender: joe.bloggs@ From: Joe Bloggs To: A Customer Subject: Here is your invoice, pay me now!
The "Envelope Sender" address is never displayed to regular email users, but it is actually the
address that SPF uses to test the authenticity of an email. In this example, if this email message arrived from a server that was legitimately run by "", then technically that's all that is needed to be legitimate from an SPF perspective even though the email displayed in the recipient's mail client doesn't indicate it comes from !
For this reason, SPF is no longer the solution to sending email "on behalf" of customers like it was when we launched Xero a few years ago.
We have had this confirmed by email experts from Facebook, Yahoo! and JPMorgan Chase when we spoke to them in February this year. In fact, they confirmed what we were seeing, which is that there's no way to reliably send email while pretending to be someone else.
The previous method of sending Xero emails out by impersonating the Xero user was untenable, and we would see continuing increases in the numbers of emails that would fail to deliver.
Problems with SPF:
We've already discussed how SPF on it's own is not sufficient, and that it relies on the verification of a part of the email message that's not actually displayed to an end user.
Our invoice emails actually looked roughly like this:
Envelope Sender: xyzzy123@notifications. From: Joe Bloggs To: A Customer Subject: Here is your invoice, pay me now!
For this email to pass an SPF test, the IP address of the mail server is compared to the SPF records of the "Envelope Sender" domain of notifications.. Xero has control of these records, and so we were able to make sure that the SPF record for notifications. was always valid.
Some mail servers would also run SPFlike tests against the visible "From" address that displays in the email, even though that's not strictly part of the SPF standard, effectively asking the question: "Is Xero's email server allowed to send email from ?"
In order to pass this test, every Xero customer would need to create an SPF record in their DNS to allow Xero's mail servers to send email on their behalf.
For some Xero customers that are ITsavvy, and in control of their own DNS, this was indeed possible, and over the past few years approximately one hundred Xero customers have managed to do this successfully.
However, for most Xero customers this is infeasible:
Many Xero customers use an "ISP" email address, rather than their own domain name (e.g. Joe.Bloggs@ rather than Joe.Bloggs@). This means that they have no control over the configuration of their ISP.
Customers that had their own domain name (e.g. ) but weren't able to change DNS, had a large company with strict IT policies, or were part of a multinational. These customers aren't easily able to create SPF records.
Customers that use SPF, but didn't include Xero's servers in their SPF records may be hardfailing all impersonated mail from other sources. Our analysis showed this was potentially the case for about 7% of customers.
Corporate customer's mail servers would often reject Xero invoice emails that they sent to themselves, even if SPF was correctly set up. E.g. a corporate mail server for would say "I've recieved mail from 's mail server's that is pretending to be from . I know that I'm the only mail server for , so this is obviously spam" and throw it away.
Creating valid SPF records is actually hard. One only needs to look around the internet and on our own Community to see that people don't understand the nuances of SPF, and a simple mistake in configuration can stop delivery of all your company's email. This goes against our mantra of being the "World's easiest accounting system".
The (2013) solution:
We absolutely want to deliver email for our customers, to their customers. Asking our users to download invoices and send them from their desktop is cumbersome and inefficient although we realise for some customers that it will always be their preferred approach so that they have complete control.
Candidate solutions:
Continue impersonating email Unreliable, not recommended by experts. Requires every Xero user to configure SPF correctly, even though that won't guarantee delivery.
Send email via our customer's email servers Technically infeasible (we'd have to collect login credentials, connect to hundreds of thousands of authenticated mail servers to send email, and deal with the myriad of different ways that could fail). We'd be outsourcing the "email delivery" issues to our customers, rather than being responsible for delivering mail correctly on their behalf.
Deliver email with a . "From" address We can guarantee that messages are correctly authenticated. We control the full infrastructure for the email sending.
We can track deliverability and bounces.
Ultimately, we made the decision to choose the third option, and change the way that we deliver invoice email. We did consider a hybrid approach, or allowing users to optin to one or other approach, but we decided that there were benefits in having a single, reliable delivery mechanism that required no configuration by our customers.
The change we made in June 2013 changed the format of our invoice emails to look like this:
Envelope Sender: xyzzy123@notifications. From: Joe Bloggs ReplyTo: Joe.Bloggs@ To: A Customer Subject: Here is your invoice, pay me now!
To break down the individual changes, and the rationale behind them:
The "Envelope Sender" address, remains as a unique @notifications. address. This is the domain that is used in SPF to validate that our mail server is allowed to send this email. This address is not displayed in user's email clients, and the unique part at the beginning of the email address allows us to track bounces and failures back to the individual sender.
The "From" address has been broken into two parts: The Display Name "Joe Bloggs", which is now configurable per organisation in Xero The email address messageservice@post., which is no longer configurable
The ReplyTo address is now configurable per organisation (e.g. Joe.Bloggs@). This is the address that any customer replies to your invoice emails will be directed to and displayed to them, if they press the Reply button in their mail client.
Our testing showed that in many mail clients, the recipient of this email would see nothing untoward, and simple actions like reading and replying to emails would not display the "" mail addresses.
In some mail clients (e.g. Outlook 2013), the new email format means that recipients can now see "" in the invoice emails that you send. This is an unavoidable sideeffect of this technical decision, and it was not made for marketing reasons!
The reason that some mail clients display "", is that those clients want to show the user the information they have used to make the "trust decision" that has lead to the email not being marked as spam. Our new email format leverages 's trust and email delivery mechanism, so Outlook wants to make it clear to the user who receives the email the reason why it has appeared in their inbox.
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- secure email
- how to manually configure your enterprise email profile on
- how to setup your email address in microsoft outlook 2007
- how emails are sent from xero
- student and adjunct email troubleshooting tips
- get comfortable with email ela area public library
- setting up an existing e mail in microsoft outlook
Related searches
- how much are my ee bonds worth
- how to rid thc from urine
- how much are my treasury bonds worth
- how much are newspaper ads
- how safe are annuities in retirement
- how much are my bonds worth
- how much are my bonds worth today
- how much are savings bonds worth today
- how to make money from stocks
- are proceeds from life insurance taxable
- how long are you immune from covid
- are proceeds from life insurance policy taxed