Introduction
Public authorities increasingly explore opaque artificial intelligence systems for tasks such as allocating welfare benefits, predicting recidivism and optimising urban services. Opaque systems, often described as ‘black boxes’, produce outputs without intelligible explanations of their internal processes. The normative question at stake is whether such delegation is morally acceptable. This essay defends the position that it is not, because opacity undermines accountability and equal respect for citizens. It first sets out the thesis with reference to deontological and capability-based considerations, then examines a consequentialist objection, and finally offers a reply that preserves the central claim while acknowledging limited exceptions.
Thesis
Delegation of public decisions to opaque AI systems is morally unacceptable because it violates duties of justification and impairs core human capabilities. From a deontological perspective, public institutions owe citizens reasons for coercive or allocative choices; when an algorithm cannot supply them, this duty is transferred to no one in particular (Floridi, 2019). The principle of responsibility, articulated by Jonas (1984), further requires identifiable agents who can be held to account when rights are affected. Opaque systems sever this link. In addition, Sen’s capability approach highlights that individuals are entitled to understand and contest decisions that shape their opportunities; unexplained scoring systems for housing or employment erode the capability for practical reason and affiliation (Sen, 2009). Empirical instances, such as the Dutch childcare-benefits scandal involving an opaque risk-classification model, illustrate how lack of transparency compounded systemic injustice (Dutch Data Protection Authority, 2021). These considerations support the claim that opacity renders delegation unjustifiable in principle.
Objection
A consequentialist critic might reply that opacity is acceptable whenever overall outcomes are superior. If an unexplainable model reduces crime or distributes limited resources more efficiently than human decision-makers, the moral gain in aggregate welfare outweighs the loss of individual explanation. Proponents of this view often cite predictive policing or medical triage algorithms whose accuracy demonstrably exceeds unaided professional judgement, arguing that citizens have a stronger interest in correct outcomes than in comprehensible reasons (Mayer-Schönberger and Cukier, 2013). On this account, demanding transparency would sacrifice real benefits for an abstract procedural ideal.
Reply
The objection overstates the inevitability of the accuracy–transparency trade-off and underestimates downstream harms to legitimacy. First, many high-stakes domains now possess explainable or inherently interpretable alternatives that achieve comparable performance; the trade-off is therefore contingent rather than necessary (Rudin, 2019). Second, even when predictive gains exist, they are frequently unevenly distributed: groups already marginalised may suffer higher false-positive rates, yet lack avenues to challenge the result precisely because the model is opaque. The resulting perception that decisions are arbitrary erodes the public trust required for legitimate governance, an effect not captured by narrow outcome metrics. Finally, partial concessions—such as allowing opacity only under strict independent audit and sunset clauses—can be accommodated without abandoning the thesis. Such safeguards restore minimal conditions of accountability while preserving consequentialist advantages where they genuinely arise. Thus the objection does not overturn the claim that routine delegation to opaque systems remains morally unacceptable.
Conclusion
Opaque AI systems pose a distinctive ethical problem for public decision-making because they obstruct justification and contestation. The thesis, grounded in deontological and capability considerations, withstands the strongest consequentialist objection by pointing to feasible alternatives and broader institutional costs. Future policy should therefore prioritise explainability requirements or prohibit opaque tools in rights-sensitive contexts. This stance does not reject AI assistance altogether; rather, it insists that technological adoption must remain subordinate to the moral preconditions of democratic authority.
References
- Dutch Data Protection Authority (2021) Report on the childcare benefits scandal. The Hague: Dutch Data Protection Authority.
- Floridi, L. (2019) ‘What the near future of artificial intelligence could be’, Philosophy & Technology, 32(1), pp. 1–15.
- Jonas, H. (1984) The imperative of responsibility: in search of an ethics for the technological age. Chicago: University of Chicago Press.
- Mayer-Schönberger, V. and Cukier, K. (2013) Big data: a revolution that will transform how we live, work, and think. London: John Murray.
- Rudin, C. (2019) ‘Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead’, Nature Machine Intelligence, 1(5), pp. 206–215.
- Sen, A. (2009) The idea of justice. Cambridge, MA: Harvard University Press.
