Because some WeoGeo Market users reported that they were not receiving email notifications, we had a need for User Testing across a variety of email platforms. Enter Mechanical Turk:

We created a 6 question survey focused on our registration and email validation process and offered 2 cents per completed survey. After only 3 batches of 6 surveys (18 total for a whopping 36 cents!) we identified and confirmed problems with AOL, MSN Hotmail, and Yahoo! Mail blacklisting IPs owned by EC2. Other EC2 users had reported similar problems which quickly led us to a solution.

The average time from release to completion for all of the surveys was about 45 minutes and the quality of the responses is very good. No corners were cut by the Automatons. This has been such a success that we will look to make more tests focused on other aspects of our web site.

This entry was posted on Tuesday, July 15th, 2008 at 6:37 pm and is filed under AWS, Mechanical Turk. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.