Domain management at FastMail

Helping out a user yesterday with a few of his own domains helped me realise that a number of people don’t know how FastMail can be a valuable tool for managing domains, and the Web sites, email accounts and DNS settings associated with them. The one proviso is that you need an Enhanced individual account, or a family/business account.

What we can do

Email: The obvious one, but there’s quite a few options. We make it easy to set up email and channel it into your main or other FastMail Inbox. So if you have domains that you haven’t bothered setting up with email because you thought it was too hard, try it with FastMail. See the “Setting up your domain” section below to get started.

You can accept email for specific addresses (eg john@example.com) or for all addresses at your domain. You can direct email to any account, local or remote. We can also handle sub-domains, so you can use addresses like ebay@john.example.com to more easily classify your email.

DNS: We can host DNS for your domain regardless of which registrar you’ve registered it with. We set up sane defaults so email & websites (in your file storage) just work. We give you a simple control panel to set A, MX, CNAME, etc records to any values you want. For more complex scenarios, there’s no silly arbitrary limits on how many of each record you can create.

Websites/photo galleries/redirects: You can publish websites to your whole domain, or a sub-domain, or an arbitrary path within a domain one of three ways:

  1. Host a simple website with static HTML pages & linked images from any directory in your file storage area
  2. Create an “instant photogallery” just by uploading photos into a directory of your file storage area (more details and an example)
  3. Create a redirect to another URL (either a regular redirect or a “cloaked” redirect inside a frame)

Webmail login: Our default DNS will point http://mail.example.com to a login screen where you can login to your email account.

Accounts in your domain (Family/Business only): Create accounts in your own domain, so rather than joecitizen@fastmail.fm, you can create an account directly in your domain as joecitizen@example.com

What we can’t do

Domain registration: Currently we can’t register domains for you. You have to do that using a domain registrar. There’s lots of domain registrars out there. If you don’t have a favourite, just do a google/yahoo/live search for one.

Webhosting with database/scripting/PHP/etc: You can publish any directory in your file storage area as a website, or have it display any uploaded photos as a photo gallery, or set up redirects to other sites. We can’t do any dynamic sites that require server side scripting (eg PHP, ASP.Net, etc) or databases.

Setting up your domain

Add domain at FastMail: The first thing to do is add the domain so FastMail knows about it. You can do that on the Options -> Virtual Domains screen (Enhanced), or Manage -> Domains screen (Family/Business).

Email: Next you setup the addresses you want to accept email for in that domain. On the same screen as above, add any addresses you want to capture (we call these domain aliases), and set the Target to the address you want the email to go to. If you want to capture email for all addresses in the domain, use the special value * (an asterisks) on it’s own. If you’re in a family/business, you can also use the Manage -> Add / change / delete users screen to add users in the domain.

Websites/photo galleries/redirects: You can set these up via the Options -> Websites screen (Enhanced), or Manage -> Websites screen (Family/Business). Just fill in the parts at the bottom of the screen, and make sure you choose the appropriate “Publish as” option. To make things a bit simpler, you don’t have to setup separate example.com and www.example.com sites, just setup example.com, and www.example.com will automatically work as well.

Custom DNS: If you’re happy for FastMail to handle email and websites/photo galleries/redirects for your domain, then you don’t need any custom DNS. Just proceed below with the “Full DNS” instructions. If you have a website hosted at an external webhosting service, then you need to set the correct IP address for this. To do that, go to the Options -> Custom DNS screen (Enhanced), or Manage -> Custom DNS screen (Family/Business), select the domain and click Select, then change the radio button to Custom DNS and click Change. After that, there will be a new section at the bottom where you can add/delete/change the DNS records for your domain. There will be two records with the Data STANDARD_WEB, change that to the IP address given to you by your web host. If your webhost gives you a CNAME record instead of an A record, see the “Extra tricks” section below.

Changing DNS to us

After you’ve done the above, you’re now ready to actually point the domain to us. There’s two main ways people might want to do this:

Full DNS

This is the easiest and most common. In this mode, we handle all the DNS for your domain. As mentioned above, the defaults are set so that email and websites/photo galleries/redirects will all just work, but you can use the Custom DNS settings as described above if you need to point to an external web host.

To use this mode, you just have to point the nameservers for your domain to us. Unfortunately we can’t do this for you. You have to login to the control panel provided by your domain registrar, and look for where you can change the “name servers” for your domain. You want to make sure there’s only two values, and that the two values are:

  • ns1.messagingengine.com
  • ns2.messagingengine.com

That’s it. After you make that change, and wait an hour or two usually, our servers will then be handling DNS for your domain.

Email only

In some cases, people only want email for their domain to come to us, and they want to handle DNS for their domain with an external DNS provider rather than with us. In that case, you want to change the MX records for your domain to:

  • in1.smtp.messagingengine.com
  • in2.smtp.messagingengine.com

That’s it. After you make that change, and wait an hour or two usually, email for your domain should start coming to our servers.

We recommend you consider hosting DNS for your domain with us (eg. the “Full DNS” instead of the “Email only” solution). Our nameservers are reliable and secure, and our web interface for managing DNS is simple, but powerful enough to allow any number of records of any type.

Extra tricks

CNAME only web hosts

Some website hosting places will only give you a CNAME record to point your domain to, not an IP address. This is a little trickier to setup and you have to be a bit careful. The core rule is: never set a CNAME record for your domain (example.com), this will break email delivery for your domain. Instead, do this:

This means users going to http://www.example.com will directly access your web hosts server, and users going to http://example.com will be redirected by us to http://www.example.com.

Mirrored domains

Sometimes people have multiple domains that they want to basically “mirror” each other. So you might have example.com and example.com.au, and you want any address for example.com to work for example.com.au as well. With email, that’s easy to do.

Just pick a “primary” domain (say example.com), and create all your domain aliases and accounts in that domain. Then create a single alias in your alternate domain with the special * value (eg *@example.com.au) and set the target of that alias to *@example.com. Then any valid email address at example.com, will also be valid at example.com.au. Note that this only works for email addresses, for websites, you have to explicitly create the website for each domain, or setup a redirect to the primary domain.

Existing users in FastMail domains

A number of people start off with individual accounts, and at a later stage want to move to a Family/Business account. That’s no problem. Just signup the Family/Business, and once you’ve done that, use the Manage -> Import User screen to import the users into the Family/Business. All the calculations are done on a pro-rata basis, so no money is ever “lost”.

You can also use the Manage -> Rename User screen to rename those users into your own domain (eg joecitizen@fastmail.fm -> joe@example.com), so then in the future they can go to http://mail.example.com and login with just the username part (eg “joe”) of their account name. After that, users will even be able to publish websites/photo galleries from their file storage area in their own sub-domain (eg for joe@example.com, they’ll be able to use http://joe.example.com)

Website only domains

In some rare cases, users want to host websites at FastMail, but want to host DNS and/or email for their domain elsewhere. In that case, you should preferably point your domain to the CNAME record “web.messagingengine.com”. Because of the issues with setting a CNAME record for your primary domain as described above, we recommend you set the CNAME for www.example.com. If you absolutely must use IP addresses, you can currently use 66.111.4.53 and 66.111.4.54, but we recommend against this where CNAME or hosting DNS with us is possible.

Posted in News, Technical. Comments Off

Improved spam filtering: avoiding email forwarding services

In a recent blog post, I mentioned that one of the ways to improve spam filtering is to avoid using a forwarding service, and especially if you have your own domain, to point the MX records for your domain directly to our servers (Enhanced or family/business accounts only).

The reason for this is that FastMail does a lot of work at the SMTP stage (when email is transferred from an external system to FastMail) to try and identify spam bots and block them while letting legitimate through. Unfortunately many of these processes can only work when the MX records for your domain point directly to our servers. If you point the MX records to another system, and forward from that system to FastMail, we can’t do the spam bot detection, and so more spam will get through.

To give you an idea of how effective this is, one of our users – the author, Simon Cann (http://simoncann.com/) – recently changed over the MX records for his Noise Sculpture domain (http://noisesculpture.com) to point to us instead of using a forwarding service. After doing so, he had the following to say:

As you suggested, I pointed the MX records for the Noise Sculpture domain at Fastmail and have been blown away by the results.  I’m now getting maybe three or four pieces of spam in a day and the few pieces of spam that are getting through are being correctly identified as spam and put in the Junk Mail folder.

Simon went on to explain why he hadn’t made the change sooner.

I came to Fastmail a year ago following an unexpected and highly inconvenient melt down of my web host who was also by default my email host for my simoncann.com domain.  This was embarrassing and potentially lost me business (I was in the middle of pitching a book proposal to a group of literary agents).

On signing up with Fastmail I immediately pointed the simoncann.com MX record at Fastmail.  My other website for my music-related books (Noise Sculpture) was hosted by a different web host and so I forwarded all of my email from that domain to Fastmail.  I didn’t see a need to change the MX record–the web host was working well (still is) and I couldn’t be bothered to start hacking the DNS records.

Long story short, as the simoncann.com and noisesculpture.com emails were sent to a single Fastmail account, I hadn’t realized how much simoncann.com spam was getting killed by Fastmail.  However, now that I have changed the MX record for noisesculpture.com, I can see just how much spam was getting through from that domain because I was using forwarding rather than pointing the MX records.

This change has stunned me (in a very good way).  I’m so impressed with the change (allied with your excellent service) that I’ve just renewed my Fastmail subscription to cover the next few years.

For the technically interested, the main things we do at the SMTP delivery stage to detect these spam bots are:

  • XBL blocking. The XBL is a highly accurate block list that lists the IPs of machines with known trojans and proxies. Independent testing shows that the ZEN RBL (of which XBL is a part) has a high block rate, with basically no false positives
  • Address enumeration detection. Spammers are always trying to find new addresses to send their spam to, and they do that by enumerating email addresses at hosts. We have a system that will automatically detect this behaviour, and block the spam bot from discovering any additional information about whether an address is valid or not
  • Greylisting. Greylisting is a process designed to detect poorly written sending email servers, of which almost all spam bots are. While greylisting is very useful, it is sometimes frowned upon because poorly implemented versions would delay email delivery. The FastMail implementation uses a number of tweaks to ensure that legitimate email is delivered with no delay, while spam email is delayed or stopped from delivery
  • RFC violations. When a server is transferring email from one system to another, there’s a series of conventions they have to follow defined in the SMTP RFC 2821. Again, spam bots are often poorly implemented, or try and use tricks to work around this conventions. By detecting some of the common violations, more spam bots can be detected and blocked
  • Custom spambot detection. Through our large and long term experience with running an email service, we’ve been able to closely analyse how spam bots try to deliver email to our servers. By using this data, we’re able to pre-emptively detect spam bots early only after they connect, and block them from delivering their spam email at all
Posted in News, Technical. Comments Off

Help test proxy to improve IMAP performance

Update: Due to a packaging error, the 1.0 proxy wouldn’t work on most machines. The new version 1.1 should work on XP/Vista machines. You need to uninstall the version 1.0 proxy first before installing version 1.1. You will also need to install the MS VS 2008 redistributable. See below for details.


IMAP is a great protocol that allows you to access your email via a wide variety of email clients (Outlook Express, Thunderbird, Outlook, etc), while keeping everything on the server and having a consistent synchronised view of your email at home, work, or when accessing webmail from anywhere.

The IMAP protocol does have an annoyance though. When you switch folders in an email client, the email client has to re-fetch a list of “flags” from the server on all messages in the folder to check which messages are flagged, seen/unseen, etc. On small folders, it’s not a problem, but on large folders, that can be quite a lot of data to transfer (several megabytes) just to make sure the flags list is consistent. If you have large folders (eg folders with >10,000 messages), you might notice this delay as the email client “pauses” after you select a folder before doing anything else.

The long term solution to this is a feature called MODSEQ/CONDSTORE, but this requires changes to IMAP clients and servers, and while FastMail’s server does support it, it’s not likely many clients will be supporting it soon.

There is another solution that can help. The data fetched when retrieving the flags data is very repetitive, which makes it easy to compress. There’s an extension to the IMAP protocol to allow compression of data. Now again, this requires client and server changes. The good news is that FastMail’s server now supports this. The bad news is that there’s no IMAP clients out there yet that do.

However, there is a work around for this, a “proxy” that compresses/expands the data between your email client and the FastMail server. For this to be useful, the proxy has to run on your local machine, and your email software has to be changed to talk to the proxy, which will then compress/decompress everything and talk to the FastMail server.

We’ve now developed such a proxy, and you can try it out today. Currently this is only for Windows (XP/Vista) users. Here’s what to do:

  1. You need the Visual Studio 2008 C++ redistributable (this is needed to make the OpenSSL encryption library work). Download it from http://www.microsoft.com/downloads/details.aspx?familyid=9B2DA534-3E03-4391-8A4D-074B9F2BC1BF (~1.7M) and follow the directions to install it
  2. Download the FastProxy installed from here:  http://robm.fastmail.fm/downloads/setup_fastproxy_1_1.exe (~5.4M)
  3. Run the setup_fastproxy_1_1.exe program, which will install FastProxy on your machine
  4. Run FastProxy from the Start menu
  5. You should see a new notification icon appear in the bottom right hand corner that looks look a small house. If you click on it, it will bring up an about window with some basic instructions. If you right-click on it, you’ll get an option to Quit if you want to stop it running. If you hover over it, it will display some statistics.

    fastproxy_snip

  6. Now you need to change your email client to access FastMail via the proxy. Doing that will depend on your email client, but the main things you have to change are:
    • Change the IMAP server from “mail.messagingengine.com” to “localhost”
    • Change the IMAP port number from 143/142/993/992 to 8143/8142/8993/8992 respectively (eg add an “8” on the front)
    • Turn off any SSL encryption. Don’t worry, if you’re using port 8993/8992, the connection and any data sent to/from FastMail will still be encrypted by the proxy, it’s just that you can’t use encryption when your email client talks to the proxy
  7. After you do that, save any changes. Your email client might ask you if you want to update your folder list. If possible, avoid doing that, because it might cause the client to throw away local cache information and it doesn’t need to do that.
  8. You should now be able to access your email account as usual, but hopefully you’ll find accessing large folders or downloading a number of emails faster than it was before.
  9. One more thing. When you shutdown/restart Windows, the proxy won’t start automatically. You can make it do that by putting a shortcut to FastProxy in your Startup folder. You can do that by doing the following:
    • Right-click on the Start menu and select “Explore all users”
    • Find the “fastproxy” folder in the tree on the left and click it
    • In the section on the right, right-click the “FastProxy – FastMail IMAP Compression Proxy” item and select “Copy”
    • Find the “Startup” folder in the tree on the left and click it
    • In the section on the right, right-click in a blank area and select “Paste”
    • That should create a link to the FastProxy program and make it start automatically at Windows startup time

If you hover over the notification icon, you should be able to get an idea of how much data the proxy is saving you. We’ve found that for everyday use (eg not bulk email downloading, but just reading new emails, switching between folders, etc), savings usually average around 80% (eg your IMAP sessions use 1/5th the bandwidth they previously did).

I’ll start a forum thread shortly where people can discuss the experiences with the proxy.

Posted in Technical. Comments Off
Follow

Get every new post delivered to your Inbox.

Join 5,149 other followers