Introduction: The Inescapable Present
We live in a deeply unjust world. Across borders, life chances are hugely affected by where we are born. The median age for someone in Uganda or Niger is below 16 years while in Germany or Japan it is around 46.
Our conceptions of responsibility for realising justice are, however, under immense strain. In our world, causal connections to unjust harms are often highly diffuse and the effects of discrete actions on overall outcomes are increasingly hard to discern. Even as connections between deprivations and globalised social processes have intensified, moral implications for specific agents remain unclear. The standard view of “normal justice”, as the political theorist Judith Shklar called it, comes to represent the interests of some, often a relatively privileged few, while neglecting those of others.
This is the inescapable context in which artificial intelligence is surging as a global political, economic, and cultural force. Yet, this backdrop rarely makes it into the discussion of how machines ought to fit into our collective future. That is why Joi Ito’s Resisting Reduction: A Manifesto is such a timely and significant intervention. We cannot succumb to technological path dependencies. A defining question of our age is whether machines will exacerbate current injustices or contribute to rooting them out. Extended intelligence in a systems paradigm gives us realistic hope for the latter.
Reductionism and the Limits of Moral Mathematics
Reductionism has a firm grip on contemporary thinking about justice. At least since the influence of Descartes and Newton on scientific method in the seventeenth century, reductionism has been a powerful force on all aspects of Western thought.
To see what is at stake, contrast the reductionist view with the vision of responsibility articulated by Simone Weil, the great mystic and intellectual, in her 1949 masterpiece on the future of France, The Need for Roots:
Initiative and responsibility, to feel one is useful and even indispensable, are vital needs of the human soul.…For this need to be satisfied it is necessary that a man should often have to take decisions in matters great or small affecting interests that are distinct from his own, but in regard to which he feels a personal concern… [H]e requires to be able to encompass in thought the entire range of activity of the social organism to which he belongs... For that, he must be made acquainted with it, be asked to interest himself in it, be brought to feel its value, its utility and, where necessary, its greatness, and be made fully aware of the part he plays in it… Every social organism, of whatever kind it may be, which does not provide its members with these satisfactions, is diseased and must be restored to health.10Weil, S., The Need for Roots: Prelude to a Declaration of Duties Towards Mankind, Routledge,  2002, 15.
The liberal orthodoxy in contemporary philosophy, to say nothing of economics, does not promote this rich conception of human responsibility. It implicitly views the individual as an atomised, self-interested agent for whom moral obligations are a burden to be avoided unless and until there is a recognised violation of justice or harm.
This paradigm generates distinctive future risks. For technologists, investors, and geopolitical strategists alike, ideas of responsibility tend to be tied to notions of linear causality or narrow fiduciary roles such as the firm's responsibility to its shareholders. This approach weighs up moral costs and duties like on an accounting ledger. It focuses on interactions between agents or the rules governing such interactions but pays little attention to system goals or paradigms. In the context of profit-driven or geopolitical competition, individual advances in artificial intelligence can lead cumulatively to unanticipated harmful outcomes.
The pressing task for our collective future with machines is therefore not simply to predict the risks, however important that might be. Nor is it to usher in a utopian Singularity, as Joi Ito makes abundantly clear. Rather, it is to imagine and then continually make and remake a world where scientific discovery and emergent technologies deepen human flourishing. This involves discerning what we value most, individually and collectively, rather than simply adjudicating ethical dilemmas or structuring society to compensate for inequalities or harms after the market has produced them. It is hard to resist the force of John Palfrey’s argument that “It feels urgent that we examine what we care most about in humanity as we race to develop the science and technology of automation.”
The dominant views of harm, even in large-scale, highly diffuse cases of systemic injustice, is one of assigning blame for discrete wrongdoing. But this model of carving up responsibility struggles when an agent’s marginal contribution to a particular harm is almost unidentifiable. Moral philosophers such as Derek Parfit – a towering figure of contemporary analytical philosophy – have persuasively challenged our intuitions on this sort of problem.
The default becomes to blame a few exceptional wrongdoers, such as those most clearly linked to the end point of harm, rather than to see the wholeness of the situation. This is unsurprising given the grip of salience and the availability heuristic on human psychology.
A more useful way to address this challenge is the concept of “structural injustice” developed by the political theorist Iris Marion Young before her untimely passing in 2006. Structural injustice consists of social processes which “put large groups of persons under systematic threat of domination or deprivation of the means to develop and exercise their capacities, at the same time that these processes enable others to dominate or to have a wide range of opportunities for developing and exercising capacities available to them.”
Instead, Young’s model promotes political responsibility, which consists of an imperative to watch social institutions, monitor their effects “to make sure that they are not grossly harmful,” and maintain “organized public spaces where such watching and monitoring can occur and citizens can speak publicly and support one another in their efforts to prevent suffering.”
It is startling that the normative implications of complex adaptive systems have been almost totally neglected by political and moral philosophers. This vantage point connects agency with systemic change in a morally and psychologically compelling way with significant implications for the public sphere. I suggest that much of our thinking about justice has gone astray because it has failed to take account of system effects either in the perpetuation of injustice or the realisation of justice over time. By shifting the emphasis to system goals and paradigms, it brings political imagination centre stage. This opens quite radical departures from prevalent thinking with regard to who has responsibilities for justice, how these responsibilities are fulfilled, and the very nature of such responsibilities. Taken together, this encourages me to propose a new way of thinking about justice, which I call systems justice.
At the height of the Global Financial Crisis in November 2008, on a visit to the London School of Economics, Her Majesty Queen Elizabeth II of England asked a group of assembled academics a succinct but piercing question: “Why did no one see it coming?”
So in summary, Your Majesty, the failure to foresee the timing, extent and severity of the crisis and to head it off, while it had many causes, was principally a failure of the collective imagination of many bright people, both in this country and internationally, to understand the risks to the system as a whole.21Besley, T. and Hennessy, P., “The Global Financial Crisis – Why Didn’t Anybody Notice?,” British Academy Review 14 (November 2009): 8-10. Scott, J.C., Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale University Press, 1998.
Such failures of imagination not only afflict technocrats and economists. The inability to appreciate the “system as a whole” has deep intellectual and cognitive roots. People experience systems differently to interactions with other agents, rules, and material facts. James Scott, for instance, has documented the disastrous consequences when states seduced by “high modernism” have established schemes that render complex social realities legible to control according to scientific laws.
Recognising the ubiquity of system effects as the starting point for thinking about justice on a global scale, however, is a way out of this reductionist trap. It generates a new vantage point that unlocks the potential of each person to play their part in imagining, experimenting, and realising a more just world. To envision the possibilities, we can turn to the great historical struggles against slavery, racism, and patriarchy.
If it is to serve the cause of basic rights – a “morality of the depths”
Responsibility for justice is thus tied to our sense of self as moral beings: there is ultimately no meaningful distinction here, I suggest, between the interests and life projects of individual agents and affirmative responsibility for justice. This suggests the paradigm should therefore be one of coherence, rather than dividing the costs of action. This requires citizens – encompassing technologists, investors, and public leaders of various kinds – committed to human flourishing in a larger sense, beyond what is formally demanded of them. It is about virtue and integrity. Our theory of justice, if it is to seriously grapple with the complexity of harm in a globalised world, must make room for this constructive view of responsibility. It should encourage people to identify how they can positively use their talents, life projects, and social roles towards the transformation of unjust systems. Such a project must be reflected in the ethos of educational programs so as to enable all citizens to develop their capabilities to contribute to the process of structural transformation.
This task – one of empowerment and ennoblement – was already urgent given the challenges of global injustice and impending ecological crisis but the advancement of artificial intelligence accelerates this urgency further still. The interests driving advances in artificial intelligence can either join this agenda or they can ignore it. But if they choose the latter, the cumulative effect will likely end up moving closer to and eventually coinciding with a resounding answer to Kwame Anthony Appiah’s question “What will future generations condemn us for?”
By acknowledging the potential of all of us in our situated social roles – as, for example, technologists, artists, intellectuals, politicians, CEOs, investors, parents, citizens – to contribute towards a reimagined social ethos, this project deepens our sense of who we want to become as moral beings. It is a provocation and invitation to do our part – humbly and with openness – in an ongoing, dynamic process of social transformation. This approach pushes back against the premise of an atomised, self-interested agent and opens the way for alternative grounds for commitments to the common good.
The prevailing cost- or liability-based model suggests that some agents must be held responsible for a harm so that there is not an undue imposition on ordinary people going about their lives in morally legitimate ways. But this division is precisely what is at stake. What we need is a conception of responsibility that demands obligations of justice without undercutting individuals’ ability to pursue their own ends. Yet, those ends must themselves be conceptualised within a given social condition in which an agent finds herself. It is a moral mistake, I think, to suggest that we can partition ourselves from the world, demarcating one realm as that of personal freedom and autonomy and the other of social relations or the natural environment. In truth, the two are bound together. The reductionist account does not want to yield this.
Systems justice thus connects a bird’s eye view of justice – consistent with the “impartial spectator” deployed in Amartya Sen’s conception of justice
To enable this shift in perspective, systems justice must promote particular virtues of justice. If we are to foster and strengthen the social ethos – “a structure of response lodged in the motivations that inform everyday life” as the philosopher G.A. Cohen put it
So instead of viewing justice as a particular state of affairs, its conceptualisation should enable a way of seeing the world that is fit for the moral needs of the age. Systems justice is not an analytical category. Rather, it is a lens through which moral agents can see the world from different vantage points and motivate their distinctive contribution to global justice. It is a way to interpret our responsibilities given the exigencies of the multiple, nested social systems in which we live, rather than a formula for the design of perfect institutions. It is an ontological move which draws on complex systems as a recurring metaphor for imagining social and political life, and the relationships agents have to each other over time. It is a standpoint for discerning system-level obligations of justice. While most analytical philosophy seeks to contain our vision of social life to make it more manageable, systems thinking offers an accessible way to widen our vision. It generates a practical morality of coherence – one that connects small-scale action and commitments with large-scale transformation – over a compartmentalised morality.
My point is not to suggest that the direct responsibility of technologists or regulators, for example, is unimportant. Rather, it is say that both direct and system responsibility matter. We should think of direct liability and social connection as complementary elements of a far-reaching vision of responsibility. When we consider how feedback loops actually work, interim strategies focused on immediate accountability can be effectively linked to longer term outcomes. Incentive-based measures to promote direct liability, however, must be sensitive to inadvertently shifting the logic from one of morality to that of maximising gains, and thus crowding out moral motivations.
It seems we do not have well-developed conceptual resources for the ambiguous space – the transitional phase – between injustice and justice. But it is in this in-between space that political contestation is most intense, decisions are most ethically challenging, and where moral ideas and discourses do the most work. Incremental contributions are often an essential feature for these transitions. It is therefore more helpful when conceiving of system-level responsibility to think in terms of a continuum of fulfilling responsibility. This approach moves us away from blame for wrongdoing without sacrificing the language or force of responsibility. What is lost by way of analytical precision is surpassed by normative insights of a different kind, those needed to support paradigm transformation.
The reluctance to speak of responsibility in these more expansive ways is linked to the tendency for much of moral philosophy to focus on immediate scenarios and decisions. But when we turn to great historical struggles against injustice what we find are not only examples of lives of remarkable altruism and heroism but people and organisations who addressed injustice in a multitude of ways, at all levels, including through their ordinary lives and social roles, grounded in their sense of connection to the underlying social dynamics and culture they confronted. It is a great limitation of our current ideas of justice that they leave these agents out of the story or afford them a marginal role. Framing responsibility in this positive sense is controversial because it presses against our intuitions about who should bear cost. Indeed, by raising the prospect of responsibility for failing to prevent harm, it appears to stretch the concept so far as to render it meaningless, or at least indistinguishable from altruism and charity. This kind of response recalls Bernard Williams’ “negative responsibility” critique of utilitarianism, which resists the idea that we can be equally responsible for our actions and things we allow or fail to prevent.
But this response would mischaracterise the nature of responsibility as I have developed it here. Historically, structural injustice has tended to be overcome through many agents, feeling a sense of connection to harms (and the victims of harm) if not complicity in causing them, who have taken up the cause of normative change in fulfilment of their sense of responsibility. The reality of systemic injustices pushes back against the reductionist views of responsibility – ones limited to retracing causality or complicity – for the social system of which moral agents are a part. In this light, there must be a wider sense of responsibility for the system as a whole. It will be up to a wider set of agents, based on a wider moral vision, to generate and spread the attention, knowledge, and reimagined ethos required to transform such structures and the practices embedded in them. Admittedly, this notion of shared responsibility raises difficult questions about who, precisely, should be doing what, exactly. But such ambiguity must be tolerated: it is in the nature of systemic injustice that moving along the continuum of justice does not lend itself to clear-cut answers on the content of every obligation. In fact, diversity of action extends sources of renewal. The non-linearity of social change must be encompassed within our conceptual frame if it is going to address real-world injustice.
Butterfly Justice, Hope, and Experimentalism
The moral implications of artificial intelligence therefore require a new kind of responsibility. Agents – citizens, firms, states – must weigh their ethical responsibilities in relation to their connection to the entire system, not only discrete, linear interactions or even their simple aggregation. Systems justice subverts how we think about who has moral responsibility and how it is fulfilled. Since discourses and sentiments of the system shape it more than its mechanics, the role of artists and writers, for example, comes to the fore; small acts of moral courage become more powerful than they seem in our conventional ethical calculus.
Systems justice thus resists hasty conclusions on negligibility, the suggestion that small-scale contributions are largely irrelevant. Instead, it highlights the importance of ordinary attitudes and behaviours in sustaining – or transforming – social systems. As the Plato scholar Melissa Lane suggests in her brilliant book drawing on ancient ethics to respond to contemporary challenges of environmental sustainability: “The person who embodies a new outlook becomes in virtue of that very fact a node in a new political imagination, the first step to creating a new social ethos.”
While social scientists have sought to explain society-wide normative change through concepts such as norm, reputational and availability cascades and tipping points,
Since agents cannot predict or control the evolution of social systems ex ante, systems justice includes a normative commitment to democratic experimentalism. This principle draws on the tradition of American pragmatism found in the work of John Dewey and William James.
More fundamentally, systems justice starts by asking who we are and want to become as moral beings, rather than asking what costs we owe or burdens we should take on. This subverts the self-interested, atomistic conception of the individual.
Our future with machines will be defined by the cumulative, often unintended consequences of many agents working to advance the frontiers of intelligence within existing rules and norms. Traditional models of responsibility will be inadequate to channel these efforts towards “a culture of flourishing.”
For discussions on various themes and stray ideas related to this essay, I am grateful to William Butler, Daniel Butt, Janina Dill, Joi Ito, Cécile Laborde, Amartya Sen, Henry Shue, Kathryn Sikkink, Roberto Mangabeira Unger, Jonathan Zittrain and, especially, Jonathan Wolff.
 Chetty, R., Hendren, N., Jones, M. R., and Porter, S.R., Race and Economic Opportunity in the United States: An Intergenerational Perspective, NBER Working Paper No. 24441, 2018.
 Sen, A. Development as Freedom, Knopf, 1999.
 Shklar, J.N., The Faces of Injustice, Yale University Press, 1990.
 Nicholas Negroponte also wrestles with this point with his characteristic perceptiveness and iconoclasm: https://jods.mitpress.mit.edu/pub/issue3-negroponte. As Negroponte reminds us, the MIT Media Lab was itself founded on a model that fell outside prevailing market paradigm limitations.
 Descartes, R., A Discourse on the Method of Correctly Conducting One’s Reason and Seeking Truth in the Sciences, Oxford University Press,  2006, 17. Descartes summarized his approach in the following way: “I came to believe that in the place of the great number of precepts that go to make up logic, the following four would be sufficient for my purposes, provided that I took a firm and unshakeable decision never once to depart from them. The first was never to accept anything as true that I did not incontrovertibly know to be so; that is to say, carefully to avoid both prejudice and premature conclusions; and to include nothing in my judgements other than that which presented itself to my mind so clearly and distinctly, that I would have no occasion to doubt it. The second was to divide all the difficulties under examination into as many parts as possible, and as many as were required to solve them in the best way. The third was to conduct my thoughts in a given order, beginning with the simplest and most easily understood objects, and gradually ascending, as it were step by step, to the knowledge of the most complex; and positing an order even on those which do not have a natural order of precedence. The last was to undertake such complete enumerations and such general surveys that I would be sure to have left nothing out.”
 Weil, S., The Need for Roots: Prelude to a Declaration of Duties Towards Mankind, Routledge,  2002, 15.
 Parfit, D., Reasons and Persons, Clarendon Press, 1984, 78-84.
 See Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 375
 Tversky, A. and Kahneman, D., “Availability: A Heuristic for Judging Frequency and Probability,” Cognitive Psychology, 5:2 (1973): 207-232; Kahneman, D., Thinking, Fast and Slow, Farrar, Straus and Giroux, 2011, 129-136.
 Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 52.
 Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 96.
 Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 88.
 Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 179.
 Giles, C., “The economic forecasters’ failing vision,” Financial Times, The FT Year in Finance supplement, 16 December 2008, 5.
 Besley, T. and Hennessy, P., “The Global Financial Crisis – Why Didn’t Anybody Notice?,” British Academy Review 14 (November 2009): 8-10.
 Scott, J.C., Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale University Press, 1998.
 Watts, D.J., Everything is Obvious: Once You Know the Answer, Crown Business, 2011.
 On slavery (and racism), see for example Sinha, M., The Slave’s Cause: A History of Abolition, Yale University Press, 2017; Hochschild, A., Bury the Chains: The British Struggle to Abolish Slavery, Macmillan, 2005; Crawford, N.C., Argument and Change in World Politics: Ethics, Decolonization, and Humanitarian Intervention, Cambridge University Press, 2002. On women’s suffrage (as one illustration of confronting patriarchy), see for example Finnemore, M. and Sikkink, K., “International Norm Dynamics and Political Change,” International Organization 52:4 (1998): 887-917; Ramirez, F.O., Soysal, Y., and Shanahan, S., “The Changing Logic of Political Citizenship: Cross-National Acquisition of Women's Suffrage Rights, 1890 to 1990,” American Sociological Review 62:5 (1997): 735-745. One of the lessons from these historical cases is that the more granular perspective we take, the more it becomes apparent that ordinary citizens have been crucial to transformational change even if less valorized compared to a few exceptional leaders. For example, it might be easily forgotten that around the 1790s, 300,000 English people participated in a sugar boycott to abolish slavery (see Hochschild, Bury the Chains, 192-196).
 Shue, H., Basic Rights: Subsistence, Affluence, and U.S. Foreign Policy, Princeton University Press,  1996, 18.
 Appiah, K.A., “What will future generations condemn us for?,” The Washington Post, 26 September 2010: B01; Appiah, K.A., The Honor Code: How Moral Revolutions Happen, W.W. Norton and Company, 2010.
 Sen himself borrows this from Adam Smith. See Sen, A., The Idea of Justice, Harvard University Press, 2009, 124-152.
 Sen, A., The Idea of Justice, Harvard University Press, 2009, 131.
 Sen, A., The Idea of Justice, Harvard University Press, 2009, 134.
 Cohen, G.A., Rescuing Justice and Equality, Harvard University Press, 2008, 123.
 Bowles, S., The Moral Economy: Why Good Incentives Are No Substitute for Good Citizens, Yale University Press, 2016.
 Williams, B., “A Critique of Utilitarianism,” in J.J.C. Smart and Bernard Williams, Utilitarianism: For and Against, Cambridge University Press, 1973: 75-150.
 Arendt, H., “Personal Responsibility under Dictatorship,” in Responsibility and Judgment, ed. Kohn, J., Schocken Books, 2003, 28.
 Lane, M., Eco-Republic, Princeton University Press, 2012, 64.
 MacKinnon, C.A., Butterfly Politics, Harvard University Press, 2017, 1. MacKinnon draws on the concept to name her book, Butterfly Politics, compiling interventions made over 40 years. She uses the term “butterfly politics” as “an organizing metaphor and central conceit” for the volume.
 Meadows, D.H., Thinking in Systems: A Primer, Chelsea Green Publishing, 2008, 164.
 Seminal examples include Kuran, T., Private Truths, Public Lies: The Social Consequences of Preference Falsification, Harvard University Press, 1995; Sunstein, C.R., “Social Roles and Social Norms,” Columbia Law Review 96:4 (1996): 903-968; Kuran, T., and Sunstein, C.R., “Availability Cascades and Risk Regulation,” Stanford Law Review 51:4 (1999): 683-768.
 To illustrate this, consider how James Baldwin sought to resolve the dilemma he faced when deciding how to play his part in responding to racial injustice in the United States in the mid-twentieth century. After listing other avenues of black resistance and activism and how he did not feel personal compatibility with them, Baldwin writes: “This was sometimes hard on my morale, but I had to accept, as time wore on, that part of my responsibility – as a witness – was to move as largely and as freely as possible, to write the story, and to get it out.” See Baldwin, J., I Am Not Your Negro, texts compiled and edited by Peck, R., Vintage Books, 2017, 31.
 Ypi, L., Global Justice and Avant-Garde Political Agency, Oxford University Press, 2012.
 See, for example, Dewey, J., The Public and Its Problems, The Pennsylvania State University Press,  2012; James, W., The Will to Believe and Other Essays in Popular Philosophy, Harvard University Press,  1979. For more contemporary thinking in line with this approach, see Ansell, C.K., Pragmatist Democracy: Evolutionary Learning as Public Philosophy, Oxford University Press 2011; Dorf, M.C. and Sabel, C.F., “A Constitution of Democratic Experimentalism,” Columbia Law Review 98:2 (1998): 267-473; Sabel, C.F. and Zeitlin, J., “Experimentalist Governance,” in The Oxford Handbook of Governance, ed. Levi-Faur, D., Oxford University Press, 2012: 169-184.
 On the idea of an antidisciplinary research program, see https://jods.mitpress.mit.edu/pub/designandscience
 On the application of this to the public realm, I am inspired in part by the communitarian thinking of Benjamin Barber; see Barber, B., Strong Democracy: Participatory Politics for a New Age, University of California Press,  2003. For instance, on reductionist conceptions of freedom that pose it in opposition to power, Barber ( 2003, 36) remarks: “Rendering freedom and power in physical terms not only misconstrues them, it produces a conception of political liberty as entirely passive. Freedom is associated with the unperturbedness of the inertial body, with the motionlessness of the inertial frame itself. It stands in stark opposition to the idea of politics as activity, motion, will, choice, self-determination, and self-realization… The modern liberal appears to regard it [tranquility] as a republican ideal: man at rest, inactive, nonparticipating, isolated, uninterfered with, privatized, and thus free.”
 For one argument that offers this view from the perspective of psychology, see Seligman, M.E.P., Flourish: A Visionary New Understanding of Happiness and Well-being, Free Press, 2011.