After I joined LION Publishers two years ago, I attended our annual conference for local independent news publishers. In the hallways, between sessions, funders asked me: “So, who should we be paying attention to? Who’s a success story?” I’d offer a few publications, usually ones I had seen named in an American Press Institute newsletter or written about by Nieman Lab or tweeted about by Important Journalism People.

But the truth is, I had absolutely no idea. And I still don’t.

That’s because the journalism industry is great at celebrating editorial accomplishments and revenue generation, but pretty terrible at defining what it means for an organization to be successful. I’ve seen “success stories” suddenly shut down sites or lay off half their staff in a day. And while “sustainability” is the word the industry throws around to describe long-term success, no one has clearly defined it.

So we decided to give it a shot.

Our hypothesis is that in order to be sustainable, independent local news businesses must be operationally resilient, financially healthy, and journalistically impactful. People are always excited to talk about journalistic impact and revenue — that’s the sexy stuff. But we don’t talk nearly enough about (the broad category of) operations, all the behind-the-scenes work it takes to make a workplace and business run smoothly.

To test this hypothesis, we created a Sustainability Audit. (Admittedly, the word “audit” isn’t doing much for the unsexiness of what we’re trying to measure.). Our Audit is a list of about 100 questions, documents, and metrics we ask news organizations to provide. Our goal is to look at this qualitative and quantitative data holistically and provide some analysis on the progress they’ve made so far toward sustainability.

Here are some examples of the questions we ask:

• On a scale of 1-10, how sustainable does the workload and pace feel for you and your staff?
• How do you collect from and give feedback to your employees?
• If you had to step away, what would need to happen to ensure a smooth transition?
• Have you taken steps to ensure your staff reflects the diversity of the community you’re serving? If so, what steps have you taken? What work still needs to be done?
• Have you taken steps to ensure your coverage and sources reflect the diversity of the community you’re serving? If so, what steps have you taken? What work still needs to be done?
• What are some of the most effective ways your audience is finding you? How do you know?
• What journalistic work has your team done recently that you’re most proud of and why?
• Can you provide an annual budget, profit and loss statement, current burn rate, and balance sheet?

The Audit will never be a magical algorithm that spits out a green sustainability checkmark or A–to-F letter grades. Rather, it’s a diagnostic tool, meant to introduce aspects of running a business that news leaders haven’t thought of and to highlight opportunities for development and growth.

In the long term, we want the audit to serve as a roadmap for independent news entrepreneurs trying to build a sustainable business by offering support and pointing folks to resources that can help their business become sustainable. And, once we have enough good data, we hope it can provide industry benchmarks for measuring sustainability.

As with any good attempts at measurement, our process has been imperfect, with lots of lessons learned. After completing 50 audits so far and listening to feedback from the news leaders filling them out and experts we hire to complete them, we’re drafting our third version of the audit, making more tweaks and improvements.

But here’s what we know for sure: By simply naming the things that are important for every independent news business to be thinking about, we’re closer to figuring out where organizations are on their path to sustainability. And that’s key to helping us fulfill our vision: a world where independent news businesses can sustain healthy, functional work cultures long-term while enabling a healthy, functioning democracy.

I predict 2022 is the year I can honestly answer the question: “Who’s a success story?” — and I’ll be able to tell you why.