iOS 7 Mail App uses multi-folder body searches by default

This is a technical post. Regular FastMail users subscribed to receive email updates from the FastMail blog can just ignore this post.

We’ve recently been testing out the Mail application in iOS 7 and found that there’s been a couple of significant changes, especially when you search your email. If you run an IMAP server with users that connect using the iOS Mail application, you might be interested in what we found. The two main ones are:

  1. Rather than searching the current folder, by default, it searches all folders. To do that, it opens many IMAP connections at once for searching each folder concurrently. We’re not sure on the upper bound on the number of connections it will make, but saw at least 10 in one case.
  2. Rather than searching the Subject/To/Cc/From fields, by default, it searches all those fields as well as the message body.

Both of these changes are actually great for the user experience, but they create potentially large headaches for IMAP server maintainers.

Clients making multiple IMAP connections at once isn’t new, but the number of potential IMAP connections an iOS client might now make is large. Some administrators limit the total number of IMAP connections a user can make at once. They might have to raise this or iOS might start returning unexpected results.

The bigger concern is the body searching. The search string iOS now sends is:

tag UID SEARCH RETURN (ALL) (OR FROM "Foo" (OR SUBJECT "Foo" (OR TO "Foo" (OR CC "Foo" BODY "Foo")))) NOT DELETED

IMAP search semantics suggest that a body search is supposed to be a pure sub-string search (BODY <string>: Messages that contain the specified string in the body of the message). Depending on your IMAP server, your message bodies may or may not be indexed in a way that allows sub-string searching. If they’re not, then a BODY search is potentially very expensive, requiring every message to be opened and searched individually. Doing that simultaneously across many folders might generate a sudden and significant IO load on a server.

Our plan at FastMail is to detect iOS clients, and convert all searches into FUZZY searches. This causes matches to be done on “terms” rather than pure sub-strings, but allows us to use our xapian powered index which should make matching and fetching results much, much quicker.

Posted in Technical. Comments Off

Fastmail uses perfect forward secrecy with https/TLS connections

There’s been a number of articles recently about perfect forward secrecy (PFS). The main aim of PFS is to ensure that even if the private SSL/TLS key for www.fastmail.fm was ever compromised, it would still be impossible to decrypt any existing captured traffic between users and our server. If you’re looking for more information, the linked articles above are worth reading to get a better overview. For PFS to work, both the server (us) and the client (your web browser) must support it.

Fastmail has supported PFS via ECDHE for some time now (since July 2012). Unfortunately a few browsers don’t support ECDHE.

Today we’ve updated our ciphers to the best practice recommended by SSL Labs. Using the SSL Labs site tester on www.fastmail.fm shows that we now support PFS on all major browsers except for IE 8 on Windows XP, which has no support for PFS and so can never support it.

We’re pretty sure that this change won’t have any compatibility issues with old clients (which should fall back to older ciphers), but we’ll keep an eye out in case there’s any reported problems.

Posted in News, Technical. Comments Off

Custom login screens for family/business accounts

We’ve released a new feature that allows family and business accounts to easily customise the login screen for their users. Just login to your family/business and go to Manage –> Customise Login.

To make customisation quick and easy, you only need to specify 3 things:

  1. An overall theme for the page (4 to choose from)
  2. Some text to appear on the login screen (defaults to “Your Business/Family Name webmail login”)
  3. A logo you want to appear on the page. You can upload any JPG/PNG you have, it will automatically be resized to fit the login screen appropriately

By default, the custom login screen will then appear at http://mail.yourdomain.com.

If you host DNS for your domain with us, this will just work automatically. If you use an external DNS provider, you’ll need to create a CNAME record for mail.yourdomain.com that points to www.fastmail.fm. (Correction: This was previously mail.messagingengine.com, which was incorrect)

An example page is viewable at http://mail.digitalintegrity.com

For resellers, any sub-business will automatically get your business customised login screen by default, though you can also explicitly customise the login screen for each sub-business if you want.

Posted in News. Comments Off

Push events, NAT TCP connection timeouts, and device sleep

This is a technical post. Regular FastMail users subscribed to receive email updates from the FastMail blog can just ignore this post.

When we released the new user interface last year, one of the improvements included was push updates when new emails arrived.

In theory, push events are conceptually quite easy to do. We open a connection from your web browser to the server (see this blog post for details), then when a new email arrives, we send a message down the open connection to let the browser know. It then fetches the details about the new email(s) and refreshes the display.

Unfortunately, in the real world, it’s not quite that easy. The biggest problem is that when a mailbox is mostly idle (no new mail arriving), the connection from the browser to the server will be idle. While this shouldn’t be a problem, it turns out it often is.

As we have noted before, some of our users are behind NAT gateways/stateful firewalls that have short state timeouts. If you leave a TCP connection idle for too long (variable from 2 to 30 minutes depending on the device), these start dropping any new packets on the connection.

In the case of a push connection from the server to the client, this is particularly bad. When a new email arrives, the server will try and send data to the client, and then be told the connection is dead at that point. That’s fine for the server, it can then clean up the connection. However, the client will never see any data from the server, and neither will the client ever know that the gateway/firewall has broken the connection. The client will think it is still connected to the server and has no way of knowing that the connection has actually been broken. This is purely a consequence of the way the TCP protocol works. The only way for the client to be able to tell the connection is broken is to send some data down the connection, and there are only 2 ways that can happen.

  1. If the client has enabled TCP keepalive on the socket. Currently only Chrome on Windows does this.
  2. If the client sent some data down the connection to the server. Unfortunately the eventsource specification doesn’t provide any way to do this; it basically assumes the underlying TCP connection is always reliable and only the server can send to the client.

One way to try and work around this issue is for the server to send regular “ping” events to the client, sufficiently often that the gateway/firewall knows the connection is still alive. This is relatively straightforward to do, but causes other problems.

If the ping events come too fast, it can cause some clients to never go into sleep mode. For instance, we used to send ping events every 60 seconds. It was noted that on an iPad if you left the FastMail webpage open in Safari and put the iPad down, the iPad itself would never actually go to sleep. The screen would stay on, draining the battery very quickly.

Because of that, we decided to go the other direction and disabled the ping events, but that ends up back at the other end of the scale where sometimes push just seems to randomly stop working.

As there’s no perfect solution to this problem, we’re now changing again to a new trade off.

  1. The server will send regular “ping” events to the client at 5 minute intervals. This should be enough for most gateways/firewalls to keep the connection open, but long enough apart to allow devices to go to sleep.
  2. If the client doesn’t see a ping event after 6 minutes, it assumes the connection has died, drops the existing connection and creates a new one. This should at least allow push events to work to some extent on connections with gateways/firewalls with low timeouts.

This change has now been rolled out everywhere. Based on initial testing, we think that this time we’ve got the balance between theory and reality right.

Posted in Technical. Comments Off

Google Authenticator now supported for two-factor authentication

FastMail has long supported various methods of two-factor authentication for additional account security, from generated one-time-passwords, to SMS, to Yubikey. Today we’ve added another method to our stable – the Google Authenticator method, otherwise known as Time-based One Time Passwords (RFC 6238). With this you can use your iOS, Android or almost any other mobile device as your second factor when authenticating, increasing the security on your account without requiring you to carry an additional object around with you.

A Google Authenticator alternative login can be configured in the Alternate Logins section of your account settings screen. If you’re using the official Google clients, then you can use its support for QR codes to make setup super-easy. You can however choose to use any number of other clients that support this authentication mechanism; all will work with our implementation.

Please refer to our Google Authenticator help page for more details.

Posted in Uncategorized. Comments Off

Reading pane available

Today we rolled out support for a longstanding feature request we’ve had here at FastMail: a reading pane in our web interface. Displaying the mailbox listing next to the selected conversation means you can go through your email without switching between two different screens, and you can see at a glance what other messages are in your mailbox whilst reading an email. This works particularly well in today’s age of widescreen computers and tablets, making good use of all that horizontal screen space.

You’ll find the option to choose a layout that shows the reading pane in the Settings, as part of the “Theme” group of settings. You’ll also find here an option to hide the sidebar, which is useful on smaller devices where you want to use the space for the reading pane instead. Note, when logging in on an iPad we automatically enable the reading pane and hide the sidebar to make optimal use of the space available.

The reading pane is not available in the classic interface.

Posted in News. Comments Off

Dropbox integration now available

“Cloud” storage is a big deal these days, and there’s all sorts of great ways to make your files available everywhere. FastMail has had an online file storage facility for years, but there’s other popular services out there too. So recently we started thinking about how we could let you use other file services from inside FastMail.

Dropbox is one of the most popular of these services, and we think they do a great job of making it easy to access your files everywhere, no matter what kind of device you use. That philosophy fits well with what we’re trying to do with FastMail, so it seemed like a natural fit to make the two services work together.

So today we’ve released our Dropbox integration. Now when you attempt to attach a file, you’ll be offered a new option: “Attach From Dropbox”. The first time you use this you’ll be asked to sign in to Dropbox and authorise FastMail, but after that you’ll be able to browse and attach files from Dropbox just as easily as you can from your FastMail file storage. Similarly, you can save attachments from your messages directly to Dropbox.

We think this is pretty great. Now you can do things like save a document to the Dropbox folder on your computer at home, then attach it to an email you write from a friend’s computer. you could save some photos you received in a message to your Dropbox, and then use the Dropbox app on your phone to access and share them, and have them ready for you when you get home without having to do anything else.

Note that at no time does this give Dropbox access to your mail or any data stored by FastMail. All access to your Dropbox is done using a random authentication token that can not be linked back to your FastMail account.

We’re pretty excited about this feature, and hope you like it!

Posted in News. Comments Off
Follow

Get every new post delivered to your Inbox.

Join 5,143 other followers