After the hearing, the mayor's office asked a deceptively simple question: if CivicLearn caused harm, who would be responsible?
The software vendor pointed to the city's policy choices.
The city pointed to the vendor's technical design.
Schools pointed to limited staffing and implementation guidance.
Reviewers pointed to the model score displayed on their screens.
The diffusion of responsibility was predictable precisely because the system crossed organizational boundaries.
Elias drew a responsibility map listing every actor who could alter outcomes.
Data officers determined what information entered the pipeline.
Developers chose objectives, features, training procedures, and evaluation criteria.
Managers decided where scores appeared and which programs they influenced.
Counsellors interpreted recommendations.
School leaders controlled resources.
Policy-makers determined funding, legal authority, and appeal rights.
Vendors maintained infrastructure and software updates.
No single actor controlled the whole process, but distributed control did not justify distributed evasion.
Priya proposed assigning explicit duties rather than searching for one person to blame after failure.
Data stewards would be responsible for provenance and retention.
Model owners would monitor performance, drift, and fairness.
Program managers would ensure that predictions were used only for authorized purposes.
Human reviewers would document departures from both model and policy.
An independent oversight group would investigate complaints and publish periodic findings.
The city also created an incident-response protocol for cases where harmful outcomes appeared systematic.
Accountability, Priya argued, requires the capacity to trace a decision backward through institutions, not merely through code.
Elias recognized that technical logs alone were insufficient.
A complete audit trail had to record who designed the rule, who approved it, who applied it, what information they had, and what alternatives were available.
Accountability required provenance not only for data but for authority: every consequential rule needed an identifiable origin, mandate, and mechanism of review.
Distributed systems become ethically opaque when responsibility fragments faster than traceability, allowing every participant to appear locally reasonable while collective harm remains ownerless.
Responsibility became clearer not by simplifying the system but by making its distributed structure explicit.