A decade ago, we conducted a series of experiments in email communication called The Nonprofit Email Study. For complex and disappointing reasons, the bulk of that research remains unpublished, although we did publish some compelling related research. (For example, Disconnected: The Nonprofit Email Survey and more pointedly, its related theoretical piece, The Nonprofit Email Marketing Model, were heralded as seminal documents that defined nonprofit email practice for some time.) As anyone who reads Nonprofit News would know, I am generally disappointed with our sector’s ambivalence about evidence and our anxiety-driven fondness for conventional hype.
Thus, I count myself as truly fortunate to be able to read and recommend this recent publication by the New Organizing Institute: Experiments in Online Advocacy: A Collection of Email Experiments, 2009-2011 (53 page PDF) by Nirmal Mankani and Ethan Roeder. While I could nitpick an experiment here or there, I have to say that this is the sort of work that should define nonprofit research. Honestly, in comparison, most nonprofit “research” barely deserves the name. What we have here are real experiments, solid design, and conclusions that are thought out with insight and humility.
The experiments explore some genuinely interesting questions: (1) How does the identity of the sender relate to email performance? (2) Is there a particular email format that optimizes opens, click-throughs, and actions? (3) Does asking individuals who have pledged to call Congress to specify what time they plan to make a call increase their response rate? (4) Are we sending too much email? (5) Is there an optimal time of day to send an email? (6) Can a live phone call increase the likelihood that an individual will take an action in an email? (7) Does embedding a chat room in an online call tool increase the likelihood that members will take an action?
Now, I’m the sort of person who gets tingles reading the list of proposed research questions at the end of the report. (Some of you might too, if you’ve seen how often our overworked peers want some sort of safe answer, without ever trying to ask the right question in the first place.) But even if those don’t excite you, how could you pass up a report that opens with this:
Your mother was right. Just because your friends are doing it doesn’t mean you should too. Experimentation and testing is a way to do business; it is in and of itself a best-practice; it is the way to run effective and efficient programs. Using the results of another organization’s experiments is like wearing your best friend’s glasses. Just because you both have eyes, brains, blonde hair, and progressive politics doesn’t mean you share the same prescription.