Themes & Takeaways

Cross-cutting themes that emerged across all three breakout groups — Responsible Research in Academia, Measuring Social Indicators, and Public Sector AI Procurement — during the Metrics Makerspace.

Theme 1: Perspective Shapes What Gets Measured

Across all three groups, a consistent finding was that the choice of metrics depends heavily on who is doing the choosing. Researchers, administrators, affected communities, procurers, and vendors each prioritize different principles and different forms of evidence. What counts as a "good" metric is rarely neutral.

Theme 2: The Gap Between Principles and Practice

Each group surfaced a tension between high-level principles (integrity, well-being, accountability) and the practical difficulty of operationalizing them. Academic institutions cite responsible research principles but may lack concrete measures. Social indicators may be technically sound but miss what communities actually care about. AI procurement standards may exist on paper but go unverified in practice.

Theme 3: Accountability Requires Ongoing Engagement

All three domains pointed toward accountability as a process rather than a checkbox — requiring sustained engagement with affected parties, not just upfront consultation.

Recurring Questions & Tensions

• Who gets to define what counts as a "good" metric — and whose perspective is missing from that decision?
• How do we balance standardized, comparable metrics with context-specific measures that better reflect lived experience?
• When principles like transparency or accountability are cited, what evidence actually demonstrates they are being upheld?
• How do power dynamics between institutions and affected communities shape which metrics are adopted?

Open Problems

These are areas where further work is needed:

• Developing shared frameworks for measuring principle adherence across different domains
• Building mechanisms for affected communities to participate in metric selection
• Bridging the gap between qualitative and quantitative approaches to well-being and impact