One of the most significant aspect of the last year for The Gilbert Center was working with one of the most difficult research clients we’ve experienced. I’ll save the long version of the story for another time (there are valuable lessons to share), but the polite short version is this: (a) They didn’t like the fact that there simply wasn’t compelling evidence for the kinds of conclusions they were seeking. (b) The conclusion for which there was at least some evidence — a higher-level insight related to the fact that we need to build learning loops into our tools — was deemed insufficiently sexy for the board. And thus (c) the final recommendations to the board bore only an indirect relationship to our actual findings.
There’s no reason to believe that research, evaluation, and program planning in civil society will be any better in 2013, but we’re still going to do our best to try to push it in that direction. So let’s start the year with a short piece that we can all use to help keep us honest: Neuroskeptic’s Nine Circles of Scientific Hell. We’re all guilty of these from time to time: Limbo, Overselling, Post-Hoc Storytelling, P-Value Fishing, Creative Use of Outliers, Plagiarism, Non-Publication of Data, Partial Publication of Data, Inventing Data.
We’ve been working with the Nonprofit Technology Network (NTEN) for the last couple of months on an online benchmarking tool designed to help nonprofits compare their technology investment decisions to those of their peers. It will be entering beta testing shortly and we would like to invite readers of Nonprofit News to participate. The first step is to register for NTEN’s Free Seminar on Nonprofit Spending Research and New Benchmarking Tool. As you can tell by the title, you’ll also get the first look at NTEN’s latest research results on nonprofit technology investment patterns.
Oh, Jason Mogus, thank you! This needed to be said! Although the point of his recent article on Why your non-profit won’t make a KONY 2012 has likely been obscured by other (weirder or more troublesome) stories about the most successful viral video of all time, that doesn’t make his points any less valid. Striking at the heart of the faddishness that drives much of what nonprofits take for innovation, here are his six points about why most organizations will never succeed in this way: (1) You’ve never met your supporters. (2) You don’t really have a twitter army. (3) You speak to too many audiences. (4) Your policy people would never let this get through. (5) You run 18 campaigns and your site has 35 calls to action. (6) Your organization isn’t aligned towards the social web.
I have mixed feelings about the Knight News Challenge. The grantmaking program is a strange mix: It’s brave, innovative (as a funding program), transparent but still opaque in odd ways, brilliant and clueless, democratic but maybe in the same way as a presidential election. (Disclosure: I worry that my feelings are colored by having had two proposals rejected. I hope not.) Because Knight’s process involves open publishing of submissions, it’s a very interesting source of information for analysts. One of these analysts is Seth Lewis, who published a paper on his results: The Knight News Challenge: How it works, what succeeds, and why that matters for the shaping of journalism innovation (42 page PDF). His method is as interesting as his results. Similar to a project we worked on for a while, he analyzed the textual content of the grant proposals (5000 of them) to help him determine what content factors made a proposal more likely to be funded.
At The Gilbert Center, we’ve always been interested in data. And we’ve always been interested in communication, of course. In the last several years, we’ve invested a lot of resources in how to bring those two together. Fortunately, so are a lot of other people. Even though he isn’t focusing on civil society, when Drew Conway asks What should a data visualization do?, that doesn’t make his answer any less applicable: Tell a Simple Small Truth. Read this (it’s short, but pithy) and if you like it, take a look at our Metrics Makeovers and tell us what you think.