Net Beneficial
Influence
A 2×2 adjudication framework and five-metric interactive calculator for evaluating cooperative human-AI clinical decision-making. Asks not “Is AI better than clinicians?” but “Does using AI make clinicians better?”
Changed
Unchanged
Initially
wrong
change
resistance
Initially
right
change
resistance
Figure 1 · NBI 2×2 adjudication matrix
The clinical problem
Current AI evaluation methods ask, “Is AI better than clinicians?” While necessary, this is not sufficient. An accurate AI does not translate directly into improved diagnostic accuracy unless clinicians use it and change their decisions as a result.
Clinical decisions are implicitly Bayesian. AI is one input among many. Biases for and against AI have both been documented, and delivering the benefits of higher AI accuracy depends on baseline clinician accuracy and patterns of AI utilization.
The NBI framework reframes the question: “Does using AI make clinicians better?” Only cases where AI and clinician initially disagree offer incremental accuracy gains, so only those cases enter the calculation.
The adjudication logic
Each disagreement case is adjudicated by a 2×2 matrix crossing the direction of decision change (ΔD) against the initial clinician accuracy relative to a reference standard (R):
B (Beneficial change): AI prompted a correction when the clinician was initially wrong.
H (Harmful change): AI prompted an error when the clinician was initially right.
IR (Inappropriate resistance): Clinician stayed wrong despite correct AI input.
AR (Appropriate resistance): Clinician correctly ignored wrong AI input.
Explainable outputs
| Metric | Formula | Interpretation |
|---|---|---|
| NBI | (B − H) / Ndisagree × 100 | Net beneficial influence, primary outcome. Positive = net benefit; negative = net harm. |
| AIR | B / (B + H) | Appropriate Influence Ratio. Quality of decision changes, proportion that were beneficial. |
| ECR | B / (B + IR) × 100 | Error Correction Rate. Of cases where AI could have helped, how often it did. |
| EIR | H / (H + AR) × 100 | Error Induction Rate. Of cases where AI could have harmed, how often it did. |
| DIR | (B + H) / Ndisagree × 100 | Decision Influence Rate. Overall rate of AI-driven decision change in disagreement cases. |
Confidence intervals
Wilson / Multinomial
Wilson score for proportion metrics; multinomial-difference variance for NBI. All 95% CIs.
Input modes
3
Definitions view, per-case row entry (with CSV upload), and direct count entry.
Dependencies
Zero
No runtime dependencies beyond React 18. Fonts are self-hosted. Works fully offline.
Two views
The site has two views: Calculator (the default) and About.
The Calculator dashboard recomputes all five metrics live as inputs change. Each metric shows its value, 95% confidence interval, and a static interpretation note. An adjudication matrix detail panel is available via toggle.
The About view illustrates four signature patterns, combinations of high/low clinician trust and high/low AI accuracy, that produce characteristic NBI profiles, plus a limitations section.
-
Per-case entry
Add cases one row at a time (Di, A, Df, R, all binary 0/1). Each row auto-classifies into B, H, IR, or AR. CSV upload supported.
-
Direct count entry
Set the four cell counts (B, H, IR, AR) directly. Includes random sample generation and reset buttons.
-
Definitions mode
Variable definitions, the adjudication matrix structure, and a collapsible formulae section with all five metric definitions.
Architecture
Built with Vite + React 18, no runtime dependencies beyond React. Fonts (Source Serif 4, DM Sans, JetBrains Mono) are bundled via @fontsource so the deployed site has zero external network dependencies.
Core math is isolated in src/lib/nbi.js, pure functions with no UI coupling. The adjudication logic, all five metrics, and confidence interval calculations are independently testable.
Continuous deployment to GitHub Pages via GitHub Actions. Every push to main triggers a build and publishes to the custom domain via public/CNAME.
Citation
If you use this framework or calculator, please cite:
Gaiba, R. The Net Beneficial Influence Framework: Quantifying Cooperative Human-AI Decision-Making. Version 1.0. Zenodo. 2026.
https://doi.org/10.5281/zenodo.20043914
License
MIT + CC BY 4.0
Code: MIT. Methodology & written content: CC BY 4.0. Attribute when using the framework.