Once students learned that CivicLearn influenced access to support programs, rumours about the score spread rapidly.
Some students believed high scores guaranteed tutoring priority.
Others feared that the same scores would quietly affect university recommendations, although the city had prohibited such use.
A market for speculation emerged.
Online forums circulated advice about which behaviours supposedly raised or lowered risk.
Most of the advice was inaccurate, but accuracy was not necessary for it to influence conduct.
One student intentionally skipped an extracurricular program because she believed participation made the system classify her as overcommitted.
Another avoided requesting counselling because he feared the record would permanently increase his risk score.
The model had begun changing the behaviour it was designed to observe.
Priya called this feedback a form of performativity.
Once a metric affects opportunities, people adapt to the metric.
That adaptation can alter the meaning of the data used by the system.
If students strategically changed behaviour to manipulate scores, historical relationships could erode.
More seriously, students might avoid beneficial services because they misunderstood how data would be used.
The team responded by publishing clearer documentation about what CivicLearn could and could not influence.
They also prohibited secondary use of the risk score for admissions, discipline, or scholarship decisions.
Technical controls were added so those departments could not access the score through ordinary systems.
Elias argued that policy promises without access controls were too fragile.
Priya agreed and added that access controls without enforceable policy were equally incomplete.
The system needed both technical and institutional boundaries.
The team also examined whether staff might game performance targets by changing how cases were recorded.
An algorithm can create incentives for administrators as easily as for students.
Whenever a metric becomes consequential, the surrounding organization may begin optimizing for the metric rather than the underlying goal.
CivicLearn was supposed to help students receive support.
Performativity created a reflexive system in which predictions altered behaviour and altered behaviour changed the data-generating process on which future predictions depended.
Once this feedback begins, measurement no longer merely describes reality; it participates in producing the reality later treated as evidence.
The audit forced the city to ask repeatedly whether each new practice served that purpose or merely improved the appearance of the numbers.