Anyone who’s designed or coded a web site has faced the self-reckoning: the moment when you realize you’re not entirely sold on the design choices you’ve made. Would that icon look better on the left or right? Does the order of things in the sidebar make navigation confusing? Is that purple the ugliest purple possible?

There’s a new tool for those moments: Feedback Army. It uses Amazon’s Mechanical Turk to get 10 random people’s thoughts on the elements of your web site that you want feedback on. The cost: $7. The turnaround: minutes.

Admittedly, the responses are brief, and the quality may not match what you’d get from a focus group that has the time to dig more deeply into whatever questions you have. On the other hand, focus groups cost a lot more than $7. And there are certain small design or user-interface questions where a quick response may be exactly what you want.

To test it out, I asked Feedback Army to get some response to this site.

I didn’t have a particular design question to ask, so I went with a few generic queries:

What do you think about the top portion of the site, above the posts?

What do you think of the general appearance of posts?

What do you think of the layout and organization of the sidebar?

The first few responses were in within two minutes of submitting my questions. All 10 were in within 10 minutes. I have a feeling I would have gotten more valuable feedback with more specific, pointed questions, but the better ones I got were still interesting and useful:

…I like the dark red color [of the top], but it’s kind of plain. Maybe add a picture or two of something related.

…The blog posts look good. You might want to put more of an obvious break in between posts to tell them apart. They’re very organized and professional looking. They might be easier to read if they are not in a clump on the left side; if they are spread out across the page like normal writing on a website.

…I like the simplicity of the top portion. Visually, it could either be smaller eliminating space between the title and menu items OR add something in between to eat that space up.

…I also like the easy-to-find description of the site and its purpose. One small peeve: I don’t know why the words NiemanJournalismLab have to be all run together that way. I get it when it is the site’s name (e.g if the site’s URL was NiemanJournalismLab.com) but in this case it is not.

…I think the posts are visually “busy.” It’s hard to describe precisely. I think reducing the number of boxes, tables, and links that appear all together in one post would help. Increasing the space between DIV elements might break up posts a little.

Nothing groundbreaking in there, and a couple of the responses were duds, but it’s still useful feedback to hear. I wish Feedback Army had existed when I was designing the site a couple months ago — I might have used it to get input on a few choices I made. You can see all the feedback my $7 got me here.

Philipp Lenssen interviewed Feedback Army’s developer, grad student Raphael Mudge, over email a few days ago, who said the whole project was completed in a few days. Remarkable what today’s tools allow to be done in such a short time.

I’m just thinking out loud here, but wouldn’t a local version of this be an interesting service for newspapers to provide to their advertisers? The ability to have miniature focus groups to look at their ads, their marketing, or some other element of the way they present themselves to the public?