Answer engine summary
AI decision-making is already routine in ranking, scoring and triage — and it becomes ethically serious the moment an output changes a person's realistic options.
It is already ordinary
Automated decisions are not a future scenario. Systems already influence which applications are read, which transactions are held, which cases are prioritised and which content is seen.
Most of these are described as decision support rather than decision making. That distinction is real on paper and thin in practice, because the supported human usually agrees.
Accuracy is not judgement
A system can be more accurate than a person and still be the wrong thing to use. Accuracy measures agreement with a labelled outcome. Judgement includes weighing values that were never in the labels.
Consider a decision that is statistically optimal and unconscionable in a particular case. The model has no access to the dimension on which it is failing, because that dimension was never encoded.
Why automated decisions resist appeal
Appealing a human decision means asking a person to reconsider. Appealing an automated one means asking an organisation to accept that its system was wrong, which is a much larger claim with budgetary implications.
Add the explanation problem — staff often cannot say why the output was what it was — and the appellant is arguing against a conclusion nobody can fully reconstruct.
The threshold worth defending
A workable line is this: automation is appropriate where the decision is reversible, the stakes are bounded, and a genuine appeal route exists with a human empowered to overturn the result.
Where any of those three is missing, automation is not a efficiency question. It is a transfer of authority, and it should be argued for on those terms.
The novel's version
In The Unread, the decisions that matter are not announced. They appear as absences: a record that no longer contains what it contained, a status that has quietly changed, a person nobody is looking for.
That is the form automated authority most often takes. Not a verdict, but a correction that arrives already applied.
Key definitions
- Decision support
- A system that recommends an outcome for a human to approve, which in practice is often approved.
Questions this page answers
- Is the decision reversible?
- Can anyone explain why the output was what it was?
- Is there a human empowered to overturn it?
“Useful was not the same as right.”
Questions
Are AI decisions less biased than human ones?
Sometimes, and sometimes not. A system can reproduce patterns in its training data at greater scale and with greater apparent authority than any individual.
Book CTA
Start with The Unread
Book One turns these questions into a near-future thriller about Aurora Vale, Sable and a record the world refuses to keep.