 Start Here |
 Ch. 1: Funding Fit |
 Ch. 2: Impact |
 Ch. 3: Success |
 Search Database |
 See Videos |
Chapter 3: Measuring Success
New media makers often gauge how well they are doing by the impact their Web site has on civic life. But how should a foundation measure the success of its support for a news and information venture?
Metrics vary, depending as much on the aspirations of the funders as they do on the visions of the site operators.
Many of these projects are too new to crow about. “Experimental programs need new ways of evaluating, new ways to look at the success or failure or impact of what’s happening, and we’re in the process of working that out,” said Gary Kebbel, Knight Foundation’s journalism program director. “Maybe it’s the grantee that makes the success. It might be the audience or the technology.”
Foundation officials interviewed for the “New Media Makers” toolkit spoke of many benchmarks. They ranged from the quantity of news stories to the number of important eyeballs on the stories – to simply knowing whether anyone uses a news site. Sometimes foundation officials look at the very dynamics of the civic processes stirred by the news reports. Other times they log their own epiphanies about what journalism can do for their issues.
“What has been the traditional model is news from the few to the many, and we’re trying to change it from the many to the many.”
— Cheryl Phelps,
San Diego Foundation
When PlanPhilly.com videotapes the emotional arguments between irate stevedores and historic preservationists over the future of the city’s riverfront, “seeing that live makes the entire process a civic process in a way that it can’t possibly be if it’s left to those who have nothing to do but go to some meetings,” said Feather Houstoun, president of The William Penn Foundation, which helped create and still supports the Web initiative. “It’s really a very democratic process now.”
In funding the Voice of San Diego, The San Diego Foundation was looking to innovate in how it connects with people, said Cheryl Phelps, the director of civil society.
While she values sustainability and replicability, Phelps has other benchmarks: “community participation, the relevancy of the information people are exchanging with one another, the expansion of the readership of the grantee.”
She adds, though: “It’s not just readership, but it’s also how people are responding back, changing how information flows. … You have a lot of people who consider themselves just as invested in what news is out there as traditional journalists. … What has been the traditional model is news from the few to the many, and we’re trying to change it from the many to the many.”
Buzz Woolley, the venture philanthropist who launched the Voice of San Diego, says feedback is an important way to measure whether the site is meeting its goal of credible reporting for a wide variety of readers. “There was at one time within a week’s period … when we got an award from the League of Women Voters, which tends to be a fairly liberal organization. [Then] a local, very conservative talk-show host went on at some length on his program about how good Voice is. And a well-known Libertarian sent us some money. So if we can get those three entities to think that we’re good … I think we’ve accomplished that goal.”
In supporting the Center for Investigative Reporting’s push to step up state government coverage, the James Irvine Foundation wants to educate both the public and state legislators. But it also will be looking at “the volume of stories produced, the quality of people who have access to those stories, the extent to which the stories influence state agencies or lead to action on some of the topics being covered,” said California Democracy program director Amy Dominguez-Arms.
By the time it wrapped up its coverage of the 2007 Philadelphia mayoral race, TheNextMayor.com had “met every expectation we had,” said The William Penn Foundation’s Houstoun. “We had good journalism. We had a lot of eyeballs on the Web site. We had people talking about it. And … the right people were watching.”
She also credited the project with being a “learning experience” for the foundation. “I’m not sure, when this project started, we could have defined journalism other than [to say], ‘Journalists are doing it.’ … But as we got into it, I think we began to appreciate what it meant to have professional journalists covering an important story. … We’ve been very careful not to give editorial direction.”
The support of the Universal Health Care Foundation of Connecticut for the New Haven Independent was intended to counter the mainstream media’s oversimplification of health care issues. The resulting news coverage, said Kate Gervais, senior development officer, has “changed the view of what news is.”
The foundation might even reproduce some of the site’s articles on health care. The articles “create a great primer on the subject,” she said.
“Don’t bury your mistakes. Put the thing out there for an autopsy.”
— Ruth Ann Harnisch, president of The Harnisch Foundation
Knight Foundation President and CEO Alberto Ibargüen says that, for now, he’s looking to see if Knight-funded news initiatives are being used. “We’re going to measure success by whether it’s used and what we learned about why it’s used and how it’s used. In the long term it seems to me that we need to look at: And then what happened?
“If it’s not used, honestly, I hope we can learn something from failure. But it will be a failure in the sense that we learned that we couldn’t get news and information out to the community.
“Will it be a failure as a funding project? Maybe not. Because maybe we’ll learn why not, and the next time we try to do something we’ll know what to avoid.”
It’s OK to fail, said several funders who are backing cutting-edge projects. As traditional news coverage shrinks, Ruth Ann Harnisch, president of The Harnisch Foundation, has supported such things as MinnPost.com and Representative Journalism, a community-financed news experiment in Northfield, Minn.
“We have to throw spaghetti and see what sticks,” she said. “Don’t bury your mistakes. Put the thing out there for an autopsy.”
Concludes the Knight Foundation’s Kebbel: “It’s not a problem to do something and find out you should have done something else. It’s not a problem to do something and find out that that thing is a failure, because you’ve learned in both those cases.
“The greater problem,” he said, “is to do nothing in a world where we desperately need information and we’re getting less and less of it that’s useful for us or that helps us govern our lives.”