Measuring success when you can’t track clicks
In our series of posts on Product Ideation, we have detailed the journey from defining audience value using the Jobs to Be Done (JTBD) framework to designing the Minimum Desirable Experience (MDE) under technical and security constraints.
The final step in this part of our process is to define impact. In a highly controlled and challenging information environment, traditional metrics like page views, ad revenue, or even subscription numbers are often insufficient, unsafe to collect, or simply not reflective of the true mission.
Therefore, we adopt a holistic framework that re-conceptualizes success not as producing content, but as enabling community information resilience.
JUMP: WHY redefine success | HOW to tailor indicators | HOW to apply the acts of journalism | WHAT we learned
Redefining success as functional resilience: The Acts of Journalism as a diagnostic tool
In traditional news media, impact is often measured by large-scale consumption metrics or by advocacy (e.g., an investigation leading to policy change). In our contexts, operating remotely to serve a constrained audience, we needed a measure of success that is subtler and deeper.
For this, we have integrated the Acts of Journalism framework developed by the Journalism + Design Lab. This framework was updated in December 2025 to identify eight fundamental ways that people inform their communities, moving beyond the definition of professional journalism to encompass practices performed by civic organizations and community members.
- Facilitators cultivate community listening, conversation, and connection.
- Documenters record what’s happening in a community.
- Commenters share their personal expertise or experience.
- Inquirers ask questions and dig deeper.
- Sensemakers help people understand and contextualize issues and events.
- Amplifiers curate, share and distribute news and information.
- Navigators assist others in accessing the information and services they need.
- Enablers share their time and resources to support local news ecosystems.
The J+D Lab framework serves as a powerful diagnostic tool. It allows us to:
- Identify disruption: Autocrats systematically target these functions. By recognizing that state-sponsored bot armies target the sensemaking function, and rapid post deletion attacks the documenting function, we can see precisely where the information system is being broken.
- Define our role: Our information product's ultimate job is to perform one of these acts (e.g., sensemaking) and, even more critically, enabling the other actors in the ecosystem.
- Measure resilience: Success is measured by the ability of the community to sustain all seven acts despite external pressure. If our product helps restore or strengthen a disrupted function (like enabling secure listening or facilitating accurate sensemaking) it is deemed a success, regardless of the view count.
By adopting this lens, we transform our product from a content source to an enabling node within a larger, resilient information process.
Measuring safety and reach through tailored indicators
Before assessing functional impact of a news product, we establish baseline indicators that focus on the project's viability and its penetration into the community without compromising user safety.
Our baseline indicators reflect our news product approach, including elements of trust and desirability. We selected operational security, reach, engagement and retention, and word-of-mouth amplification as our indicators:
How we applied this diagnostic tool and amplification indicators to one of our projects
In one of our projects that we have described throughout our process posts, we focused on people’s need for information about safe and stable jobs in a major city in Asia, and we worked to gather data and produce news products to meet this need.
Ultimately, we established a system for delivering verified, actionable job market intelligence to a constrained group of job seekers.
We identified disruption: that information about labor rights complaints, including strikes and protests, comes at a great cost to workers posting about it and is easily censored.
Our role takes a step back from those newsworthy moments and examines the broader ecosystem of sifting out stable and secure job roles in which workers are safe and have predictable incomes.
We produced a series of AI-generated animated videos in which fictional characters discuss the data we gathered in a way that is helpful for decision making about job seeking.
The core product is a centralized data service, but its true value is realized when a worker amplifies the wage data in a group chat, or when a labor organizer uses the trend analysis for mobilizing action.
Functional impact and measuring the acts
The truest measure of success lies in assessing whether the information enabled users to perform the fundamental "jobs" identified earlier and strengthened their capacity for journalistic acts.
1. Enabling sensemaking and inquiring
Our core information product is an act of sensemaking, transforming scraped data (raw facts) into actionable insights (understanding). It addresses the JTBD of financial predictability and minimizing exploitation risk by providing verified context.
- Indicator: Knowledge gain and disinformation countering.
- Proxy: Anecdotal evidence or anonymous surveys confirming that users possess knowledge from our analysis that they previously did not have (e.g., knowing the 20% wage difference between two districts).
- Indicator: Demand for personalized guidance.
- Proxy: Users showing a willingness to interact with the service through direct messages with specific questions, moving beyond public content to inquire about specific situations (e.g., asking for advice on a specific contract).
In our product thinking mindset, we explicitly evaluate our offerings against competing information sources, primarily state-aligned media and the misinformation that flourishes in information voids.
This means asking: When our audience encounters conflicting information from official sources, what specific product features help them evaluate credibility? How does our value proposition differentiate from state media that might address similar topics?
In our project about job data in a major Asian city, we thought about this in terms addressing needs. For blue collar workers, government platforms theoretically provide labor rights information and job listings.
Our product differentiation lies not in topic selection but in addressing the information those official sources fail to complete: providing wage transparency that employers prefer to obscure, connecting workers with peers who have experienced specific workplaces, and offering information in formats accessible to those with limited digital literacy.
We measure success partly by user retention and direct feedback, but ultimately by whether our products prove more valuable than alternatives for accomplishing the jobs audiences need done.
2. Enabling documenting and navigating
The ultimate goal of service journalism is to trigger a desirable behavior change or action. We focused on how the product’s insights could be immediately used in the labor ecosystem, which requires documenting evidence and, eventually, mobilizing action.
- Indicator: Behavior change (job search optimization).
- Proxy: Tracking which job types and districts the audience begins searching for, or asking questions about, in small groups after content about higher wages or better benefits in that location is released. This indicates that our sensemaking is translating into optimized job search behavior.
- Indicator: Transition to secure facilitation.
- Proxy: The rate of successful transition of public followers from easily-monitored feeds into small, intimate groups (typically 50-200 members) on more private platforms. This transition is a direct precursor to enabling safer documenting of labor abuses and facilitating peer support and collective problem-solving (mobilizing). This action itself signals a strategic success for our project.
The justification for users to make this leap is the exclusive additional value provided in the private groups: more detailed job market analysis, the ability to ask specific questions, and the establishment of peer support networks.
What we learned
The MDE of a news product is fundamentally designed to be an entry point. The strategic, long-term impact involves migrating the audience to independent platforms beyond the reach of platform censorship and surveillance.
Success in this final phase means the creation of a truly resilient information ecosystem, enabled by our information product.
- Product as enabler for migration: The product’s consistent delivery of value is the only asset that justifies the user’s extra effort and security risk of migrating to external platforms (Signal, Telegram, etc.). We use the accumulated trust to enable the next act: secure, private communication.
- Measuring ecosystem vitality: The highest level of success shifts from audience reach to ecosystem vitality. This is measured by observing if the small groups begin performing the full cycle of journalistic acts independently: Documenting abuse, Facilitating discussion, Inquiring into solutions, and successfully Mobilizing for mutual aid or protection.
This ultimate goal of functional resilience ensures that even if our initial social media account is shut down (as happened to us), the process of information sharing and collective action persists within the secure community we helped enable. This defines a genuine, sustainable impact that autocrats struggle to counteract.
The forthcoming phase of our Gazzetta process is Distribution, in which we explore and tackle the challenge of safely and reliably getting our news product in front of audiences we are working remotely from.
If you haven't already, sign up for our newsletter so you don't miss our discussions of research questions and how we're experimenting with solutions to address them.
If you have feedback or questions, don’t hesitate to get in touch at hello@gazzetta.xyz.