A Google Authenticator alternative login can be configured in the Alternate Logins section of your account settings screen. If you’re using the official Google clients, then you can use its support for QR codes to make setup super-easy. You can however choose to use any number of other clients that support this authentication mechanism; all will work with our implementation.

Please refer to our Google Authenticator help page for more details.

You’ll find the option to choose a layout that shows the reading pane in the Settings, as part of the “Theme” group of settings. You’ll also find here an option to hide the sidebar, which is useful on smaller devices where you want to use the space for the reading pane instead. Note, when logging in on an iPad we automatically enable the reading pane and hide the sidebar to make optimal use of the space available.

The reading pane is not available in the classic interface.

Dropbox is one of the most popular of these services, and we think they do a great job of making it easy to access your files everywhere, no matter what kind of device you use. That philosophy fits well with what we’re trying to do with FastMail, so it seemed like a natural fit to make the two services work together.

So today we’ve released our Dropbox integration. Now when you attempt to attach a file, you’ll be offered a new option: “Attach From Dropbox”. The first time you use this you’ll be asked to sign in to Dropbox and authorise FastMail, but after that you’ll be able to browse and attach files from Dropbox just as easily as you can from your FastMail file storage. Similarly, you can save attachments from your messages directly to Dropbox.

We think this is pretty great. Now you can do things like save a document to the Dropbox folder on your computer at home, then attach it to an email you write from a friend’s computer. you could save some photos you received in a message to your Dropbox, and then use the Dropbox app on your phone to access and share them, and have them ready for you when you get home without having to do anything else.

Note that at no time does this give Dropbox access to your mail or any data stored by FastMail. All access to your Dropbox is done using a random authentication token that can not be linked back to your FastMail account.

We’re pretty excited about this feature, and hope you like it!

Until now, searching for email in FastMail has been slow and idiosyncratic. Searching for words entered into the search box would only search the To/Cc/From/Subject headers of messages in the current folder. It wouldn’t search the message body content or across multiple folders. While there were options that made both of these possible, they were incredibly slow, making finding the messages you wanted frustrating.

Now though, when you enter words in the search box, we search the To/Cc/From/Subject headers as well as the entire message body content, and we do it across all messages in all folders. Additionally, when searching for a word, we’ll show a snippet of the message content with the search term(s) highlighted in the preview area. We’ll also search for multiple different forms of the word e.g. searching for “condition” will find condition, conditions, conditional, conditionally, etc.

Most importantly, by making sure all this data is indexed, the searches are very fast, even with a million messages across dozens of folders. In most cases, it should only take a few seconds to search all your email and we have plans in progress that should speed this up even more over the coming months.

We’ve also built an easy to access advanced search builder. When you click in the search box, an auto-complete popup will appear as you type. The final item in the pop-up is an “Advanced search” option. Clicking on this will show an overlay that lets you construct an advanced search without having to leave the screen.

If you find yourself repeating the same search frequently, you can save it by clicking the “Save” button next to the search in the sidebar. The search will appear with your list of folders. Keyboard power users can quickly select the search using the “g” shortcut folder finder, just like a normal folder.

Note: The new search currently only works with the new (AJAX) interface. If you’re using the “classic” interface or the mobile interface, you’ll get the old search system for now. We hope to fix this in the future. Also if you want the old search behaviour (including sub-string matching), use the substr:(sometext) operator as described on the help page.

• fastmail.im
• fastmail.mx
• fastmail.se
• fastmail.tw

Over the past few years, we’ve had sporadic reports of users bayes databases being corrupted and reset back to empty. When this happened, it would cause email delivery for that user to fall back to using the global bayes database, which decreased the overall accuracy of their spam detection until they retrained the database with more spam and non-spam messages.

I had tried multiple times to track down what was causing this issue, but each time with no luck. Each time the problem occurred, there was an error message in the logs of this form.

bayes: bayes db version 0 is not able to be used, aborting!

Often searching the internet for an error message will find other people that have had the same problem and tracked down the solution, but in this case it didn’t. Each time I tried to work through the code to see what was going wrong, I reached a dead end and couldn’t see any obvious problem.

Since the corruptions were very intermittent and losing a bayes database isn’t critical, doesn’t cause email to be lost or inaccessible, and can be rebuilt just by reporting email as spam/non-spam again, tracking this down was always a bit of a lower priority issue.

Recently though, after one more corruption report too many, I decided once and for all to track down what was causing it. Bit by bit over the course of several weeks, I added more and more logging information to the server code to track down where in the code the problem was occurring.

The logging results proved to be very odd. In the vast majority of cases it showed that writing to a particular database worked fine, but every now and then, it caused data to be lost. Eventually I managed to create a reproducible test case. It turned out to be very odd issue because performing a particular programming action with a database library worked fine the first 5 or 6 times, but on the 6th or 7th, it would cause data to become lost. Clearly something odd is happening in the lower level library code.

Fortunately there was a straight forward workaround to the problem, so I’ve now patched our code with the workaround, and over the last few weeks I’ve monitored the logs which show the original error message above has completely disappeared and no databases are being corrupted any more.

I’ve reported bugs to the underlying modules causing the problems, so hopefully long term they’ll fixed as well.

https://rt.cpan.org/Public/Bug/Display.html?id=83060

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6901

This change should initially be invisible to users and things should continue to work as they were. In the long term, it will allow us to support more features and faster updates to DNS in the future.

• Undo. Accidentally moved a message, deleted a
  contact, marked something unread etc? No problem. Your last action can
  now be undone; just click the “Undo” link in the confirmation message.
  Or, if you’re a keyboard user, hit ‘z’. Note, you can’t undo sending a
  message.
• The pinned status of a message is now shown at the top on the
  conversation read screen, so you can see it even if the message is
  collapsed.
• Security options and logs are now grouped together in their own
  section under “Account”, to make it easier to manage the security of
  your account. This includes changing your password, seeing (and remotely
  logging out) any existing sessions, and creating alternative
  logins.
• The mailbox screen now shows an icon next to messages that have been
  replied to. With conversations enabled, this shows if the most recent
  message in the folder for that conversation has been replied to.
• The “More” menu at the top right of each expanded message now has a
  “Reply to Sender” option if the message was sent to a mailing list, and
  an “Edit as New” option for all messages.
• The unread count is now shown first in the title of a page, so you
  can still see it even if the tab cuts the title short.
• Better support for non-conversations mode. Now faster and fully
  non-conversational: replies to messages are no longer threaded with the
  message being replied to.

And, of course, several more minor refinements and bug fixes.

A quick bit of research and testing yielded the problem: Chrome limits itself to a maximum of 6 concurrent connections to a single origin across the whole browser. Each tab was loading a full instance of the mail application, which meant it was creating an EventSource object and connection to our push server, to be notified of new deliveries (see this previous post for how that works). Since these connections are permanent (that’s the whole idea!), opening lots of tabs quickly used up all the available connections, with none left to fetch any actual data. To the user, this appeared as “Could not connect to server” error messages.

The solution to this problem was not immediately obvious. Ideally, we would like to maintain a single push connection and share it between the tabs, but there’s no API for getting a reference to other tabs or windows in the browser, even if they’re pointed to the same domain. Then I remembered that setting a property on local storage triggers a “storage” event on the window object of every open tab with the same origin. This, I realised, could be used to synchronise behaviour across tabs.

The concept is fairly simple. Only one tab keeps a push connection; we call this the master tab. When it receives a push event, it broadcasts it by setting the event as a property on local storage called “broadcast”. When a tab receives the storage event for this key, it reads the JSON-encoded event object from local storage and processes it as though it had been received via an EventSource object.

The tricky part comes in coordinating between the tabs who should be master. The master tab also sets a value called “ping” on local storage roughly every 30 seconds to the current time stamp. When a tab first loads it checks for this value; if it is greater than 45 seconds ago it presumes there is no current master, so it becomes master. Otherwise, it becomes a slave. However, whilst it is a slave, it continuously monitors for storage events with a key of “ping”, and if it hasn’t heard a ping within a 45 second period, it takes over as master. This switches control to another tab when the master tab closes. On browsers supporting the “unload” event we can make the changeover happen pretty much instantly, by setting the “ping” value to 0 in local storage when the tab is closed.

This all works very well, but there’s one problem remaining: race conditions. There is no API for taking out an explicit lock on local storage, so the spec advocates the use of a per-origin mutex which would be acquired by scripts once they try to access the storage, and then released when the script finishes. Not all browsers have adopted this. The Chrome developers, for example, have decided the performance penalty is too great. Therefore, in some browsers, it is possible for scripts in different tabs to interleave such that, for example, each tries to take master at the same time, then each notices another has taken it so none end up as master! The solution we have adopted is to add a random component to the delay between pings and waiting for pings. This makes it unlikely that two tabs will both attempt to take master at the same time. Of course this can still happen, but should it do so, the random variation in each new master sending out a ping should ensure that one is quickly turned back to a slave. It will be eventually consistent, which is good enough for our purposes.

In case this is of use to anyone else, here’s the code we use (rewritten slightly to use pure JS rather than be based on our library code). It’s also available as a gist on github. You can try it out on this test page; just open the page in several windows or tabs, then close the master and see the control pass to another. You can also broadcast a message from any tab to the other tabs.

function WindowController () {
    var now = Date.now(),
        ping = 0;
    try {
        ping = +localStorage.getItem( 'ping' ) || 0;
    } catch ( error ) {}
    if ( now - ping > 45000 ) {
        this.becomeMaster();
    } else {
        this.loseMaster();
    }
    window.addEventListener( 'storage', this, false );
    window.addEventListener( 'unload', this, false );
}

WindowController.prototype.isMaster = false;
WindowController.prototype.destroy = function () {
    if ( this.isMaster ) {
        try {
            localStorage.setItem( 'ping', 0 );
        } catch ( error ) {}
    }
    window.removeEventListener( 'storage', this, false );
    window.removeEventListener( 'unload', this, false );
};

WindowController.prototype.handleEvent = function ( event ) {
    if ( event.type === 'unload' ) {
        this.destroy();
    } else {
        var type = event.key,
            ping = 0,
            data;
        if ( type === 'ping' ) {
            try {
                ping = +localStorage.getItem( 'ping' ) || 0;
            } catch ( error ) {}
            if ( ping ) {
                this.loseMaster();
            } else {
                // We add a random delay to try avoid the race condition in 
                // Chrome, which doesn't take out a mutex on local storage. It's
                // imperfect, but will eventually work out.
                clearTimeout( this._ping );
                this._ping = setTimeout(
                    this.becomeMaster.bind( this ),
                    ~~( Math.random() * 1000 )
                );
            }
        } else if ( type === 'broadcast' ) {
            try {
                data = JSON.parse(
                    localStorage.getItem( 'broadcast' )
                );
                this[ data.type ]( data.event );
            } catch ( error ) {}
        }
    }
};

WindowController.prototype.becomeMaster = function () {
    try {
        localStorage.setItem( 'ping', Date.now() );
    } catch ( error ) {}

    clearTimeout( this._ping );
    this._ping = setTimeout( this.becomeMaster.bind( this ),
        20000 + ~~( Math.random() * 10000 ) );

    var wasMaster = this.isMaster;
    this.isMaster = true;
    if ( !wasMaster ) {
        this.masterDidChange();
    }
};

WindowController.prototype.loseMaster = function () {
    clearTimeout( this._ping );
    this._ping = setTimeout( this.becomeMaster.bind( this ),
        35000 + ~~( Math.random() * 20000 ) );

    var wasMaster = this.isMaster;
    this.isMaster = false;
    if ( wasMaster ) {
        this.masterDidChange();
    }
};

WindowController.prototype.masterDidChange = function () {};

WindowController.prototype.broadcast = function ( type, event ) {
    try {
        localStorage.setItem( 'broadcast',
            JSON.stringify({
                type: type,
                event: event
            })
        );
    } catch ( error ) {}
};

In short, classic is here to stay.

For all that the interface has looked similar over the past few years, it’s had many changes under the hood.

Much of the interface is fully internationalised, both in classic and new. The code is all shared with the My Opera Mail product, where multiple language support is a core requirement.

It works much more nicely on small devices, in particular with Opera Mini.

Where possible, changes for the new interface have been rewritten as shared “library code” and integrated into both interfaces simultaneously. Some things (search, for example) work differently. But most core logic, and of course all low level mail routing and storage, are fully shared across our infrastructure.

This all adds up to a pile of “invisible” work we have done to make maintenance easier in future. Even the new search uses the same query builder library, so back-porting fully cross-folder search capability to classic will be achievable if there is demand.

Unlike the old interface, which was a completely separate copy of the code and grew stale over the years, there was never a “fork” (as it’s called in software development) for the new interface.

Indeed, you may have noticed that many screens on the new interface are really just “rebranded” classic. It’s the same HTML code as the classic desktop and mobile screens, with a different title bar. When you go back to the Mail or Address tabs, it reloads the javascript and hands control back. This was a deliberate decision to speed up the areas of our site where people spend 99% of their time (statistic taken from logs, not made up) without duplicating the rarely used screens. The client-side mailbox screen uses less bandwidth and is more responsive than the classic mechanism of downloading an entire html page on every click.

When we say “supported indefinitely” it really does mean that we have no plans to remove classic. There’s no internal timeline in our heads. The core technology is used by both interfaces, and we’re updating them together.

Finally to address concerns about continued IMAP access.

We have invested heavily on improvements to the Cyrus IMAP server, both myself and Greg Banks in the Australian office (who was hired to work full time on Cyrus, and is doing an awesome job).

Our new conversation features are built directly into Cyrus, and fully integrated into its replication system. Other features like storing previews and undelete information along with messages have been created by adding support to Cyrus for the standard message annotations described in RFC5257, contributing that work back to the community.

You can read more about the Cyrus project at http://cyrusimap.org/. This reliable and standards compliant server is the core of our technology stack. We’re not moving away from IMAP, even as we extend the server to support our specific use-cases.

You can read (or even download and play with) the exact code that runs on our servers from
http://github.com/brong/cyrus-imapd/ – our production systems run on the “fastmail” branch.

Bron.