Introduction: The Inescapable Present  

We live in a deeply unjust world. Across borders, life chances are hugely affected by where we are born. The median age for someone in Uganda or Niger is below 16 years while in Germany or Japan it is around 46.

1
http://apps.who.int/gho/data/view.main.POP2040ALL?lang=en
One in four children suffer stunted growth, rising to one in three in developing countries.
2
https://www.un.org/sustainabledevelopment/hunger/
Around four billion people don’t have access to the Internet.
3
https://www.itu.int/en/ITU-D/Statistics/Pages/facts/default.aspx
Within countries, social, gender, and racial disadvantages limit, often sharply, prospects for a flourishing life. As recent research by Raj Chetty and his collaborators has shown, for example, black boys from wealthy families in America are more likely to become poor in adulthood compared to their similarly wealthy white peers. Even when they grow up in the same neighbourhood with parents at the same income level, black boys end up with lower incomes than white boys in 99 percent of the country.
4
Chetty, R., Hendren, N., Jones, M. R., and Porter, S.R., Race and Economic Opportunity in the United States: An Intergenerational Perspective, NBER Working Paper No. 24441, 2018.
Arbitrary limitations on “development as freedom”, in Amartya Sen’s famous formulation, abound.
5
Sen, A. Development as Freedom, Knopf, 1999.
And beyond inequality and oppression, though not disconnected from it, is humanity’s reckoning with the Anthropocene and the “specter of growth-limiting resource constraints” illuminated so incisively by Anjali Sastry in this journal.
6
https://jods.mitpress.mit.edu/pub/issue3sastry

Our conceptions of responsibility for realising justice are, however, under immense strain. In our world, causal connections to unjust harms are often highly diffuse and the effects of discrete actions on overall outcomes are increasingly hard to discern. Even as connections between deprivations and globalised social processes have intensified, moral implications for specific agents remain unclear. The standard view of “normal justice”, as the political theorist Judith Shklar called it, comes to represent the interests of some, often a relatively privileged few, while neglecting those of others.

7
Shklar, J.N., The Faces of Injustice, Yale University Press, 1990.
Meanwhile, there appear to be few alternatives to dominant market or political dogmas that can adequately respond to these circumstances.
8
Nicholas Negroponte also wrestles with this point with his characteristic perceptiveness and iconoclasm: https://jods.mitpress.mit.edu/pub/issue3-negroponte. As Negroponte reminds us, the MIT Media Lab was itself founded on a model that fell outside prevailing market paradigm limitations.
Individual virtue seems insufficient but institutional responses to take up the slack are not in sight either.  

This is the inescapable context in which artificial intelligence is surging as a global political, economic, and cultural force. Yet, this backdrop rarely makes it into the discussion of how machines ought to fit into our collective future. That is why Joi Ito’s Resisting Reduction: A Manifesto is such a timely and significant intervention. We cannot succumb to technological path dependencies. A defining question of our age is whether machines will exacerbate current injustices or contribute to rooting them out. Extended intelligence in a systems paradigm gives us realistic hope for the latter.

 

Reductionism and the Limits of Moral Mathematics

Reductionism has a firm grip on contemporary thinking about justice. At least since the influence of Descartes and Newton on scientific method in the seventeenth century, reductionism has been a powerful force on all aspects of Western thought.

9
Descartes, R., A Discourse on the Method of Correctly Conducting One’s Reason and Seeking Truth in the Sciences, Oxford University Press, [1637] 2006, 17. Descartes summarized his approach in the following way: “I came to believe that in the place of the great number of precepts that go to make up logic, the following four would be sufficient for my purposes, provided that I took a firm and unshakeable decision never once to depart from them. The first was never to accept anything as true that I did not incontrovertibly know to be so; that is to say, carefully to avoid both prejudice and premature conclusions; and to include nothing in my judgements other than that which presented itself to my mind so clearly and distinctly, that I would have no occasion to doubt it. The second was to divide all the difficulties under examination into as many parts as possible, and as many as were required to solve them in the best way. The third was to conduct my thoughts in a given order, beginning with the simplest and most easily understood objects, and gradually ascending, as it were step by step, to the knowledge of the most complex; and positing an order even on those which do not have a natural order of precedence. The last was to undertake such complete enumerations and such general surveys that I would be sure to have left nothing out.”
It has also been a check on moral and political imagination. This generates a blindspot on harms which emerge from many people and organisations – most prominently corporations and states –  pursuing their goals and interests within the limits of accepted rules and norms. That is, there is a blindspot in ethics for harms which emerge from interconnected, complex systems. Liberal theories of justice are inadequate to solve this problem. By focusing on remediating rather than preventing injustice, they narrow imagination precisely where it needs to be expanded. This is a major deficiency since – within the status quo, baseline morality of the age – the injustices in question are either rendered invisible or correlative duties to rectify them cannot be located. In other words, no one appears to be responsible for systemic harms. After all, individual components of the system are not breaching a threshold of blameworthiness, and this is what counts in much cotemporary Anglo-American philosophy.

To see what is at stake, contrast the reductionist view with the vision of responsibility articulated by Simone Weil, the great mystic and intellectual, in her 1949 masterpiece on the future of France, The Need for Roots:

Initiative and responsibility, to feel one is useful and even indispensable, are vital needs of the human soul.…For this need to be satisfied it is necessary that a man should often have to take decisions in matters great or small affecting interests that are distinct from his own, but in regard to which he feels a personal concern… [H]e requires to be able to encompass in thought the entire range of activity of the social organism to which he belongs... For that, he must be made acquainted with it, be asked to interest himself in it, be brought to feel its value, its utility and, where necessary, its greatness, and be made fully aware of the part he plays in it… Every social organism, of whatever kind it may be, which does not provide its members with these satisfactions, is diseased and must be restored to health.

10
Weil, S., The Need for Roots: Prelude to a Declaration of Duties Towards Mankind, Routledge, [1949] 2002, 15.

The liberal orthodoxy in contemporary philosophy, to say nothing of economics, does not promote this rich conception of human responsibility. It implicitly views the individual as an atomised, self-interested agent for whom moral obligations are a burden to be avoided unless and until there is a recognised violation of justice or harm.

This paradigm generates distinctive future risks. For technologists, investors, and geopolitical strategists alike, ideas of responsibility tend to be tied to notions of linear causality or narrow fiduciary roles such as the firm's responsibility to its shareholders. This approach weighs up moral costs and duties like on an accounting ledger. It focuses on interactions between agents or the rules governing such interactions but pays little attention to system goals or paradigms. In the context of profit-driven or geopolitical competition, individual advances in artificial intelligence can lead cumulatively to unanticipated harmful outcomes. 

The pressing task for our collective future with machines is therefore not simply to predict the risks, however important that might be. Nor is it to usher in a utopian Singularity, as Joi Ito makes abundantly clear. Rather, it is to imagine and then continually make and remake a world where scientific discovery and emergent technologies deepen human flourishing. This involves discerning what we value most, individually and collectively, rather than simply adjudicating ethical dilemmas or structuring society to compensate for inequalities or harms after the market has produced them. It is hard to resist the force of John Palfrey’s argument that “It feels urgent that we examine what we care most about in humanity as we race to develop the science and technology of automation.”

11
https://jods.mitpress.mit.edu/pub/issue3-palfrey
An ethical framework based on backward-looking blame and guilt won’t be up to the great task before humanity to, as Ito puts it, “design systems that participate as responsible, aware and robust elements of even more complex systems.”
12
https://jods.mitpress.mit.edu/pub/resisting-reduction

The dominant views of harm, even in large-scale, highly diffuse cases of systemic injustice, is one of assigning blame for discrete wrongdoing. But this model of carving up responsibility struggles when an agent’s marginal contribution to a particular harm is almost unidentifiable. Moral philosophers such as Derek Parfit – a towering figure of contemporary analytical philosophy – have persuasively challenged our intuitions on this sort of problem.

13
Parfit, D., Reasons and Persons, Clarendon Press, 1984, 78-84.
Their arguments expose pitfalls of our “moral mathematics” and suggest culpability of some kind is called for even when actions appear to fall below a threshold for wrongdoing. But they do little to overcome the difficulty of distinctly systemic harm. Even in its aggregated form, the reductionist account depends on a linear model of causality and rectification of harm, with agents connected to outcomes in predictable ways. But as this type of connection diminishes or dissolves entirely through system effects which cannot be foreseen at the outset by the doers or enablers of harm, the limitations of the reductionist account become more apparent. The reductive account does not contain normative reasons to creatively transform the existing incentive structure or its compliance regime.
14
See Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 375.

The default becomes to blame a few exceptional wrongdoers, such as those most clearly linked to the end point of harm, rather than to see the wholeness of the situation. This is unsurprising given the grip of salience and the availability heuristic on human psychology.

15
Tversky, A. and Kahneman, D., “Availability: A Heuristic for Judging Frequency and Probability,” Cognitive Psychology, 5:2 (1973): 207-232; Kahneman, D., Thinking, Fast and Slow, Farrar, Straus and Giroux, 2011, 129-136.
But this leaves our notions of justice impotent when it comes to guiding and motivating systemic renewal. Rather than satisfying itself by closing in on a narrow set of moral duties, our theory of justice should therefore liberate the very imaginative context which dictates the limits of harm, ethics, and virtue. 

A more useful way to address this challenge is the concept of “structural injustice” developed by the political theorist Iris Marion Young before her untimely passing in 2006. Structural injustice consists of social processes which “put large groups of persons under systematic threat of domination or deprivation of the means to develop and exercise their capacities, at the same time that these processes enable others to dominate or to have a wide range of opportunities for developing and exercising capacities available to them.”

16
Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 52.
In response, Young advocated a “social connection model” of responsibility, in contrast to a legalistic liability model. According to this, “all those who contribute by their actions to structural processes with some unjust outcomes share responsibility for the injustice.”
17
Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 96.
Young argued that liability, deriving from notions of guilt or blame for wrongdoing, is an inappropriate framework for assigning responsibility in relation to structural injustice. Liability fixates on guilt which is unhelpful because it directs attention to some actors while absolving others, deflects attention from background conditions, and produces defensiveness, creating division where unified action is called for.

Instead, Young’s model promotes political responsibility, which consists of an imperative to watch social institutions, monitor their effects “to make sure that they are not grossly harmful,” and maintain “organized public spaces where such watching and monitoring can occur and citizens can speak publicly and support one another in their efforts to prevent suffering.”

18
Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 88.
The meaning of politics here is “public communicative engagement with others for the sake of organizing our relationships and coordinating our actions more justly.”
19
Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 179.
In thinking about the responsibility of specific agents to prevent structural harms, she argues we ought to weigh several parameters, including an agent’s power, privilege, interests, and ability to work collectively with others. I suggest something analogous is needed for developments in artificial intelligence. If Ito’s vision is to come to life, a wide array of individual and collective agents will need to take on responsibilities to monitor system effects, call out destructive dynamics, and help model and construct alternative practices and paradigms.

It is startling that the normative implications of complex adaptive systems have been almost totally neglected by political and moral philosophers. This vantage point connects agency with systemic change in a morally and psychologically compelling way with significant implications for the public sphere. I suggest that much of our thinking about justice has gone astray because it has failed to take account of system effects either in the perpetuation of injustice or the realisation of justice over time. By shifting the emphasis to system goals and paradigms, it brings political imagination centre stage. This opens quite radical departures from prevalent thinking with regard to who has responsibilities for justice, how these responsibilities are fulfilled, and the very nature of such responsibilities. Taken together, this encourages me to propose a new way of thinking about justice, which I call systems justice.

 

Systems Justice 

At the height of the Global Financial Crisis in November 2008, on a visit to the London School of Economics, Her Majesty Queen Elizabeth II of England asked a group of assembled academics a succinct but piercing question: “Why did no one see it coming?”

20
Giles, C., “The economic forecasters’ failing vision,” Financial Times, The FT Year in Finance supplement, 16 December 2008, 5.
In June the following year, a group of leading experts met at the British Academy to try and answer The Queen’s question. Their response, distilled in a letter, included this:

So in summary, Your Majesty, the failure to foresee the timing, extent and severity of the crisis and to head it off, while it had many causes, was principally a failure of the collective imagination of many bright people, both in this country and internationally, to understand the risks to the system as a whole.

21
Besley, T. and Hennessy, P., “The Global Financial Crisis – Why Didn’t Anybody Notice?,” British Academy Review 14 (November 2009): 8-10. Scott, J.C., Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale University Press, 1998.

Such failures of imagination not only afflict technocrats and economists. The inability to appreciate the “system as a whole” has deep intellectual and cognitive roots. People experience systems differently to interactions with other agents, rules, and material facts. James Scott, for instance, has documented the disastrous consequences when states seduced by “high modernism” have established schemes that render complex social realities legible to control according to scientific laws.

22
Scott, J.C., Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale University Press, 1998.
More generally, humans tend to demonstrate a low aptitude for learning from complex interactions, focusing on end-state outcomes rather than what might have been, the counterfactual; we rationalise how we got to where we are as a natural and inevitable product of past events.
23
Watts, D.J., Everything is Obvious: Once You Know the Answer, Crown Business, 2011.
It is unsurprising then that we reify and naturalise existing systems.  

Recognising the ubiquity of system effects as the starting point for thinking about justice on a global scale, however, is a way out of this reductionist trap. It generates a new vantage point that unlocks the potential of each person to play their part in imagining, experimenting, and realising a more just world. To envision the possibilities, we can turn to the great historical struggles against slavery, racism, and patriarchy.

24
On slavery (and racism), see for example Sinha, M., The Slave’s Cause: A History of Abolition, Yale University Press, 2017; Hochschild, A., Bury the Chains: The British Struggle to Abolish Slavery, Macmillan, 2005; Crawford, N.C., Argument and Change in World Politics: Ethics, Decolonization, and Humanitarian Intervention, Cambridge University Press, 2002. On women’s suffrage (as one illustration of confronting patriarchy), see for example Finnemore, M. and Sikkink, K., “International Norm Dynamics and Political Change,” International Organization 52:4 (1998): 887-917; Ramirez, F.O., Soysal, Y., and Shanahan, S., “The Changing Logic of Political Citizenship: Cross-National Acquisition of Women's Suffrage Rights, 1890 to 1990,” American Sociological Review 62:5 (1997): 735-745. One of the lessons from these historical cases is that the more granular perspective we take, the more it becomes apparent that ordinary citizens have been crucial to transformational change even if less valorized compared to a few exceptional leaders. For example, it might be easily forgotten that around the 1790s, 300,000 English people participated in a sugar boycott to abolish slavery (see Hochschild, Bury the Chains, 192-196).
Faced with structural injustice, the response of many agents throughout history has been to seek to undermine the future durability of the system of oppression – especially its discourse and sentiments – in ways that that cannot be reduced to a simple formula of obligations. Such responses do not necessarily resemble the repairing of harm to specific victims but rather an attack on background injustice through an expansion of collective moral imagination.  

If it is to serve the cause of basic rights – a “morality of the depths”

25
Shue, H., Basic Rights: Subsistence, Affluence, and U.S. Foreign Policy, Princeton University Press, [1980] 1996, 18.
– and certainly if it is to support a vision of human flourishing, our conception of justice must not be confined to the formulation of formal rules and institutions. Indeed, it should provide an animating impulse for the ethos which sustains such institutions, fostering citizens committed to imagining and working towards just ends in a deeply unjust world. Relying on a precise, measurable division of responsibility within the status quo may even inhibit our capacity to notice unjust system-level harm. Indeed, such duties can promote the illusion that we have done our fair share. 

Responsibility for justice is thus tied to our sense of self as moral beings: there is ultimately no meaningful distinction here, I suggest, between the interests and life projects of individual agents and affirmative responsibility for justice. This suggests the paradigm should therefore be one of coherence, rather than dividing the costs of action. This requires citizens – encompassing technologists, investors, and public leaders of various kinds – committed to human flourishing in a larger sense, beyond what is formally demanded of them. It is about virtue and integrity. Our theory of justice, if it is to seriously grapple with the complexity of harm in a globalised world, must make room for this constructive view of responsibility. It should encourage people to identify how they can positively use their talents, life projects, and social roles towards the transformation of unjust systems. Such a project must be reflected in the ethos of educational programs so as to enable all citizens to develop their capabilities to contribute to the process of structural transformation.  

This task – one of empowerment and ennoblement – was already urgent given the challenges of global injustice and impending ecological crisis but the advancement of artificial intelligence accelerates this urgency further still. The interests driving advances in artificial intelligence can either join this agenda or they can ignore it. But if they choose the latter, the cumulative effect will likely end up moving closer to and eventually coinciding with a resounding answer to Kwame Anthony Appiah’s question “What will future generations condemn us for?”

26
Appiah, K.A., “What will future generations condemn us for?,” The Washington Post, 26 September 2010: B01; Appiah, K.A., The Honor Code: How Moral Revolutions Happen, W.W. Norton and Company, 2010.

By acknowledging the potential of all of us in our situated social roles – as, for example, technologists, artists, intellectuals, politicians, CEOs, investors, parents, citizens – to contribute towards a reimagined social ethos, this project deepens our sense of who we want to become as moral beings. It is a provocation and invitation to do our part – humbly and with openness –  in an ongoing, dynamic process of social transformation. This approach pushes back against the premise of an atomised, self-interested agent and opens the way for alternative grounds for commitments to the common good.

The prevailing cost- or liability-based model suggests that some agents must be held responsible for a harm so that there is not an undue imposition on ordinary people going about their lives in morally legitimate ways. But this division is precisely what is at stake. What we need is a conception of responsibility that demands obligations of justice without undercutting individuals’ ability to pursue their own ends. Yet, those ends must themselves be conceptualised within a given social condition in which an agent finds herself. It is a moral mistake, I think, to suggest that we can partition ourselves from the world, demarcating one realm as that of personal freedom and autonomy and the other of social relations or the natural environment. In truth, the two are bound together. The reductionist account does not want to yield this.

Systems justice thus connects a bird’s eye view of justice – consistent with the “impartial spectator” deployed in Amartya Sen’s conception of justice

27
Sen himself borrows this from Adam Smith. See Sen, A., The Idea of Justice, Harvard University Press, 2009, 124-152.
– to the distinctive position of each agent in a social system. It encourages us to see the wholeness of the situation and to design institutions in light of the “admissibility of incompleteness.”
28
Sen, A., The Idea of Justice, Harvard University Press, 2009, 131.
The focus shifts from a perfectly just society derived from just institutions to a comparatively just society focused on social realisations.
29
Sen, A., The Idea of Justice, Harvard University Press, 2009, 134.
 

To enable this shift in perspective, systems justice must promote particular virtues of justice. If we are to foster and strengthen the social ethos – “a structure of response lodged in the motivations that inform everyday life” as the philosopher G.A. Cohen put it

30
Cohen, G.A., Rescuing Justice and Equality, Harvard University Press, 2008, 123.
– needed to tackle structural injustices and imagine alternatives, then our theoretical project cannot be limited to institutional arrangements or virtue but rather to their proper integration.

So instead of viewing justice as a particular state of affairs, its conceptualisation should enable a way of seeing the world that is fit for the moral needs of the age. Systems justice is not an analytical category. Rather, it is a lens through which moral agents can see the world from different vantage points and motivate their distinctive contribution to global justice. It is a way to interpret our responsibilities given the exigencies of the multiple, nested social systems in which we live, rather than a formula for the design of perfect institutions. It is an ontological move which draws on complex systems as a recurring metaphor for imagining social and political life, and the relationships agents have to each other over time. It is a standpoint for discerning system-level obligations of justice. While most analytical philosophy seeks to contain our vision of social life to make it more manageable, systems thinking offers an accessible way to widen our vision. It generates a practical morality of coherence – one that connects small-scale action and commitments with large-scale transformation – over a compartmentalised morality.

My point is not to suggest that the direct responsibility of technologists or regulators, for example, is unimportant. Rather, it is say that both direct and system responsibility matter. We should think of direct liability and social connection as complementary elements of a far-reaching vision of responsibility. When we consider how feedback loops actually work, interim strategies focused on immediate accountability can be effectively linked to longer term outcomes. Incentive-based measures to promote direct liability, however, must be sensitive to inadvertently shifting the logic from one of morality to that of maximising gains, and thus crowding out moral motivations.

31
Bowles, S., The Moral Economy: Why Good Incentives Are No Substitute for Good Citizens, Yale University Press, 2016.
Crucially, the manner in which perpetrators of injustice are held accountable should enrich and sharpen, rather than diminish, our sense of shared responsibility. Citizens become co-authors of justice, rather than passive recipients of arrangements enacted and enforced from the top down. Analogously, holding powerful technology companies accountable is a call to wider moral agency, not a formula for letting others off the hook.

It seems we do not have well-developed conceptual resources for the ambiguous space – the transitional phase – between injustice and justice. But it is in this in-between space that political contestation is most intense, decisions are most ethically challenging, and where moral ideas and discourses do the most work. Incremental contributions are often an essential feature for these transitions. It is therefore more helpful when conceiving of system-level responsibility to think in terms of a continuum of fulfilling responsibility. This approach moves us away from blame for wrongdoing without sacrificing the language or force of responsibility. What is lost by way of analytical precision is surpassed by normative insights of a different kind, those needed to support paradigm transformation.

The reluctance to speak of responsibility in these more expansive ways is linked to the tendency for much of moral philosophy to focus on immediate scenarios and decisions. But when we turn to great historical struggles against injustice what we find are not only examples of lives of remarkable altruism and heroism but people and organisations who addressed injustice in a multitude of ways, at all levels, including through their ordinary lives and social roles, grounded in their sense of connection to the underlying social dynamics and culture they confronted. It is a great limitation of our current ideas of justice that they leave these agents out of the story or afford them a marginal role. Framing responsibility in this positive sense is controversial because it presses against our intuitions about who should bear cost. Indeed, by raising the prospect of responsibility for failing to prevent harm, it appears to stretch the concept so far as to render it meaningless, or at least indistinguishable from altruism and charity. This kind of response recalls Bernard Williams’ “negative responsibility” critique of utilitarianism, which resists the idea that we can be equally responsible for our actions and things we allow or fail to prevent.

32
Williams, B., “A Critique of Utilitarianism,” in J.J.C. Smart and Bernard Williams, Utilitarianism: For and Against, Cambridge University Press, 1973: 75-150.
It also echoes Hannah Arendt’s concern that “where all are guilty, no one is.”
33
Arendt, H., “Personal Responsibility under Dictatorship,” in Responsibility and Judgment, ed. Kohn, J., Schocken Books, 2003, 28.

But this response would mischaracterise the nature of responsibility as I have developed it here. Historically, structural injustice has tended to be overcome through many agents, feeling a sense of connection to harms (and the victims of harm) if not complicity in causing them, who have taken up the cause of normative change in fulfilment of their sense of responsibility. The reality of systemic injustices pushes back against the reductionist views of responsibility – ones limited to retracing causality or complicity – for the social system of which moral agents are a part. In this light, there must be a wider sense of responsibility for the system as a whole. It will be up to a wider set of agents, based on a wider moral vision, to generate and spread the attention, knowledge, and reimagined ethos required to transform such structures and the practices embedded in them. Admittedly, this notion of shared responsibility raises difficult questions about who, precisely, should be doing what, exactly. But such ambiguity must be tolerated: it is in the nature of systemic injustice that moving along the continuum of justice does not lend itself to clear-cut answers on the content of every obligation. In fact, diversity of action extends sources of renewal. The non-linearity of social change must be encompassed within our conceptual frame if it is going to address real-world injustice.  

 

Butterfly Justice, Hope, and Experimentalism

The moral implications of artificial intelligence therefore require a new kind of responsibility. Agents – citizens, firms, states – must weigh their ethical responsibilities in relation to their connection to the entire system, not only discrete, linear interactions or even their simple aggregation. Systems justice subverts how we think about who has moral responsibility and how it is fulfilled. Since discourses and sentiments of the system shape it more than its mechanics, the role of artists and writers, for example, comes to the fore; small acts of moral courage become more powerful than they seem in our conventional ethical calculus.

Systems justice thus resists hasty conclusions on negligibility, the suggestion that small-scale contributions are largely irrelevant. Instead, it highlights the importance of ordinary attitudes and behaviours in sustaining – or transforming – social systems. As the Plato scholar Melissa Lane suggests in her brilliant book drawing on ancient ethics to respond to contemporary challenges of environmental sustainability: “The person who embodies a new outlook becomes in virtue of that very fact a node in a new political imagination, the first step to creating a new social ethos.”

34
Lane, M., Eco-Republic, Princeton University Press, 2012, 64.
Indeed, systems justice incorporates a moral version of the concept of the “butterfly effect” made famous by Edward Lorenz. As the feminist legal scholar Catherine MacKinnon recently described this idea, “some extremely small simple actions, properly targeted, can come to have highly complex and large effects in certain contexts.”
35
MacKinnon, C.A., Butterfly Politics, Harvard University Press, 2017, 1. MacKinnon draws on the concept to name her book, Butterfly Politics, compiling interventions made over 40 years. She uses the term “butterfly politics” as “an organizing metaphor and central conceit” for the volume.
This yields a deceptively simple normative insight: the power for large-scale transformation is already latent within the system. “If no paradigm is right,” as Donella Meadows puts it, it is up to each of us to identify one which achieves our deep purpose.
36
Meadows, D.H., Thinking in Systems: A Primer, Chelsea Green Publishing, 2008, 164.

While social scientists have sought to explain society-wide normative change through concepts such as norm, reputational and availability cascades and tipping points,

37
Seminal examples include Kuran, T., Private Truths, Public Lies: The Social Consequences of Preference Falsification, Harvard University Press, 1997; Sunstein, C.R., “Social Roles and Social Norms,” Columbia Law Review 96:4 (1996): 903-968; Kuran, T., and Sunstein, C.R., “Availability Cascades and Risk Regulation,” Stanford Law Review 51:4 (1999): 683-768.
these empirical phenomena have been neglected in the formulation of normative theory itself. Looking back does not yield fine-grained answers on what moral agents are obliged to do in the face of structural injustice. Rather, it suggests the wide scope moral agents have to reconceive their interests and moral identity, and the diverse ways they can direct their lives towards the realisation of justice.
38
To illustrate this, consider how James Baldwin sought to resolve the dilemma he faced when deciding how to play his part in responding to racial injustice in the United States in the mid-twentieth century. After listing other avenues of black resistance and activism and how he did not feel personal compatibility with them, Baldwin writes: “This was sometimes hard on my morale, but I had to accept, as time wore on, that part of my responsibility – as a witness – was to move as largely and as freely as possible, to write the story, and to get it out.” See Baldwin, J., I Am Not Your Negro, texts compiled and edited by Peck, R., Vintage Books, 2017, 31.
These studies reveal that the transformation of discursive and imaginative context lies within the grasp of moral agents, especially when acting in concert. They bring to light, for example, how the force of example and the generation of inertial momentum through incremental steps can reshape the social ethos and disrupt destructive path dependencies. This is most evident among an “avant garde” who promote a new standard of justice,
39
Ypi, L., Global Justice and Avant-Garde Political Agency, Oxford University Press, 2012.
but a wide array of agents, including ordinary citizens, play an indispensable role. Some of these contributions may not be salient or even visible, except perhaps retrospectively, yet this does not diminish their moral force.

Since agents cannot predict or control the evolution of social systems ex ante, systems justice includes a normative commitment to democratic experimentalism. This principle draws on the tradition of American pragmatism found in the work of John Dewey and William James.

40
See, for example, Dewey, J., The Public and Its Problems, The Pennsylvania State University Press, [1927] 2012; James, W., The Will to Believe and Other Essays in Popular Philosophy, Harvard University Press, [1897] 1979. For more contemporary thinking in line with this approach, see Ansell, C.K., Pragmatist Democracy: Evolutionary Learning as Public Philosophy, Oxford University Press 2011; Dorf, M.C. and Sabel, C.F., “A Constitution of Democratic Experimentalism,” Columbia Law Review 98:2 (1998): 267-473; Sabel, C.F. and Zeitlin, J., “Experimentalist Governance,” in The Oxford Handbook of Governance, ed. Levi-Faur, D., Oxford University Press, 2012: 169-184.
Under systems justice, institutions and citizens alike commit themselves to discovering harms and injustices and innovations to reduce them over time. It thus denaturalises current forms of the market and governance, leaving them open to revision. It harnesses people’s distinctive talents and capabilities in this enterprise, promotes unorthodox alliances and multi- and “antidisciplinary” innovations,
41
On the idea of an antidisciplinary research program, see https://jods.mitpress.mit.edu/pub/designandscience
and strengthens mechanisms for social learning about effective ways to reduce systemic harm and widen the development of human capabilities.  

More fundamentally, systems justice starts by asking who we are and want to become as moral beings, rather than asking what costs we owe or burdens we should take on. This subverts the self-interested, atomistic conception of the individual.

42
On the application of this to the public realm, I am inspired in part by the communitarian thinking of Benjamin Barber; see Barber, B., Strong Democracy: Participatory Politics for a New Age, University of California Press, [1984] 2003. For instance, on reductionist conceptions of freedom that pose it in opposition to power, Barber ([1984] 2003, 36) remarks: “Rendering freedom and power in physical terms not only misconstrues them, it produces a conception of political liberty as entirely passive. Freedom is associated with the unperturbedness of the inertial body, with the motionless of the inertial frame itself. It stands in stark opposition to the idea of politics as activity, motion, will, choice, self-determination, and self-realization… The modern liberal appears to regard it [tranquility] as a republican ideal: man at rest, inactive, nonparticipating, isolated, uninterfered with, privatized, and thus free.”
Since virtue is fortified in response to injustice, these are aspects of a coherent idea of the good life and human flourishing, rather than in permanent tension with each other. Our inner light shines most intensely when it contends with darkness. Human interests therefore should not be viewed as fixed. As we begin to engage in system transformation, our interests and those of others can transform. After all, a flourishing life depends at least in part on our sense of something bigger than ourselves.
43
For one argument that offers this view from the perspective of psychology, see Seligman, M.E.P., Flourish: A Visionary New Understanding of Happiness and Well-being, Free Press, 2011.

 

Conclusion

Our future with machines will be defined by the cumulative, often unintended consequences of many agents working to advance the frontiers of intelligence within existing rules and norms. Traditional models of responsibility will be inadequate to channel these efforts towards “a culture of flourishing.”

44
https://jods.mitpress.mit.edu/pub/resisting-reduction
The moral challenge in these circumstances is to foster citizens and design institutions that are responsive to these system-level effects. Beyond governments, agents such as individuals, civil society, and corporations can play a surprisingly constructive role drawing on their distinctive talents, knowledge, and capabilities. Since these agents cannot predict or control the evolution of social systems ex ante, citizens and institutions must remain open to discovering systemic injustice, innovating to root it out, and reconceiving their own interests in the process. Responsibility for systemic harm is fundamentally shared. Shirking this is not only a problem for potential victims but corrosive of our own virtue and moral identity. Ultimately, freedom for the self is connected to promoting justice for the whole since the two are permanently intertwined, part of a single moral life. Resisting reduction opens possibilities not only for extended intelligence but also extended morality. Obligations of justice in any given moment of history – and the values we embed in the process of technological evolution – are therefore not impositions or burdens, but rather the means to become who we aspire to be as moral beings.

 

 


Acknowledgements:

For discussions on various themes and stray ideas related to this essay, I am grateful to William Butler, Daniel Butt, Janina Dill, Joi Ito, Cécile Laborde, Amartya Sen, Henry Shue, Kathryn Sikkink, Roberto Mangabeira Unger, Jonathan Zittrain and, especially, Jonathan Wolff.


Notes:
[1] http://apps.who.int/gho/data/view.main.POP2040ALL?lang=en

[2] https://www.un.org/sustainabledevelopment/hunger/

[3] https://www.itu.int/en/ITU-D/Statistics/Pages/facts/default.aspx

[4] Chetty, R., Hendren, N., Jones, M. R., and Porter, S.R., Race and Economic Opportunity in the United States: An Intergenerational Perspective, NBER Working Paper No. 24441, 2018. 

[5] Sen, A. Development as Freedom, Knopf, 1999.

[6] https://jods.mitpress.mit.edu/pub/issue3sastry

[7] Shklar, J.N., The Faces of Injustice, Yale University Press, 1990.

[8] Nicholas Negroponte also wrestles with this point with his characteristic perceptiveness and iconoclasm: https://jods.mitpress.mit.edu/pub/issue3-negroponte. As Negroponte reminds us, the MIT Media Lab was itself founded on a model that fell outside prevailing market paradigm limitations.

[9] Descartes, R., A Discourse on the Method of Correctly Conducting One’s Reason and Seeking Truth in the Sciences, Oxford University Press, [1637] 2006, 17. Descartes summarized his approach in the following way: “I came to believe that in the place of the great number of precepts that go to make up logic, the following four would be sufficient for my purposes, provided that I took a firm and unshakeable decision never once to depart from them. The first was never to accept anything as true that I did not incontrovertibly know to be so; that is to say, carefully to avoid both prejudice and premature conclusions; and to include nothing in my judgements other than that which presented itself to my mind so clearly and distinctly, that I would have no occasion to doubt it. The second was to divide all the difficulties under examination into as many parts as possible, and as many as were required to solve them in the best way. The third was to conduct my thoughts in a given order, beginning with the simplest and most easily understood objects, and gradually ascending, as it were step by step, to the knowledge of the most complex; and positing an order even on those which do not have a natural order of precedence. The last was to undertake such complete enumerations and such general surveys that I would be sure to have left nothing out.”

[10] Weil, S., The Need for Roots: Prelude to a Declaration of Duties Towards Mankind, Routledge, [1949] 2002, 15.

[11] https://jods.mitpress.mit.edu/pub/issue3-palfrey

[12] https://jods.mitpress.mit.edu/pub/resisting-reduction

[13] Parfit, D., Reasons and Persons, Clarendon Press, 1984, 78-84. 

[14] See Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 375

[15] Tversky, A. and Kahneman, D., “Availability: A Heuristic for Judging Frequency and Probability,” Cognitive Psychology, 5:2 (1973): 207-232; Kahneman, D., Thinking, Fast and Slow, Farrar, Straus and Giroux, 2011, 129-136.

[16] Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 52.

[17] Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 96.

[18] Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 88.

[19] Young, I.M., Responsibility for Justice, Oxford University Press, 2004, 179.

[20] Giles, C., “The economic forecasters’ failing vision,” Financial Times, The FT Year in Finance supplement, 16 December 2008, 5. 

[21] Besley, T. and Hennessy, P., “The Global Financial Crisis – Why Didn’t Anybody Notice?,” British Academy Review 14 (November 2009): 8-10.

[22] Scott, J.C., Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale University Press, 1998.  

[23] Watts, D.J., Everything is Obvious: Once You Know the Answer, Crown Business, 2011.

[24] On slavery (and racism), see for example Sinha, M., The Slave’s Cause: A History of Abolition, Yale University Press, 2017; Hochschild, A., Bury the Chains: The British Struggle to Abolish Slavery, Macmillan, 2005; Crawford, N.C., Argument and Change in World Politics: Ethics, Decolonization, and Humanitarian Intervention, Cambridge University Press, 2002. On women’s suffrage (as one illustration of confronting patriarchy), see for example Finnemore, M. and Sikkink, K., “International Norm Dynamics and Political Change,” International Organization 52:4 (1998): 887-917; Ramirez, F.O., Soysal, Y., and Shanahan, S., “The Changing Logic of Political Citizenship: Cross-National Acquisition of Women's Suffrage Rights, 1890 to 1990,” American Sociological Review 62:5 (1997): 735-745. One of the lessons from these historical cases is that the more granular perspective we take, the more it becomes apparent that ordinary citizens have been crucial to transformational change even if less valorized compared to a few exceptional leaders. For example, it might be easily forgotten that around the 1790s, 300,000 English people participated in a sugar boycott to abolish slavery (see Hochschild, Bury the Chains, 192-196).

[25] Shue, H., Basic Rights: Subsistence, Affluence, and U.S. Foreign Policy, Princeton University Press, [1980] 1996, 18.

[26] Appiah, K.A., “What will future generations condemn us for?,” The Washington Post, 26 September 2010: B01; Appiah, K.A., The Honor Code: How Moral Revolutions Happen, W.W. Norton and Company, 2010.

[27] Sen himself borrows this from Adam Smith. See Sen, A., The Idea of Justice, Harvard University Press, 2009, 124-152.

[28] Sen, A., The Idea of Justice, Harvard University Press, 2009, 131.

[29] Sen, A., The Idea of Justice, Harvard University Press, 2009, 134.

[30] Cohen, G.A., Rescuing Justice and Equality, Harvard University Press, 2008, 123.

[31] Bowles, S., The Moral Economy: Why Good Incentives Are No Substitute for Good Citizens, Yale University Press, 2016.

[32] Williams, B., “A Critique of Utilitarianism,” in J.J.C. Smart and Bernard Williams, Utilitarianism: For and Against, Cambridge University Press, 1973: 75-150.

[33] Arendt, H., “Personal Responsibility under Dictatorship,” in Responsibility and Judgment, ed. Kohn, J., Schocken Books, 2003, 28.

[34] Lane, M., Eco-Republic, Princeton University Press, 2012, 64.

[35] MacKinnon, C.A., Butterfly Politics, Harvard University Press, 2017, 1. MacKinnon draws on the concept to name her book, Butterfly Politics, compiling interventions made over 40 years. She uses the term “butterfly politics” as “an organizing metaphor and central conceit” for the volume.

[36] Meadows, D.H., Thinking in Systems: A Primer, Chelsea Green Publishing, 2008, 164.

[37] Seminal examples include Kuran, T., Private Truths, Public Lies: The Social Consequences of Preference Falsification, Harvard University Press, 1995; Sunstein, C.R., “Social Roles and Social Norms,” Columbia Law Review 96:4 (1996): 903-968; Kuran, T., and Sunstein, C.R., “Availability Cascades and Risk Regulation,” Stanford Law Review 51:4 (1999): 683-768.

[38] To illustrate this, consider how James Baldwin sought to resolve the dilemma he faced when deciding how to play his part in responding to racial injustice in the United States in the mid-twentieth century. After listing other avenues of black resistance and activism and how he did not feel personal compatibility with them, Baldwin writes: “This was sometimes hard on my morale, but I had to accept, as time wore on, that part of my responsibility – as a witness – was to move as largely and as freely as possible, to write the story, and to get it out.” See Baldwin, J., I Am Not Your Negro, texts compiled and edited by Peck, R., Vintage Books, 2017, 31.

[39] Ypi, L., Global Justice and Avant-Garde Political Agency, Oxford University Press, 2012.

[40] See, for example, Dewey, J., The Public and Its Problems, The Pennsylvania State University Press, [1927] 2012; James, W., The Will to Believe and Other Essays in Popular Philosophy, Harvard University Press, [1897] 1979. For more contemporary thinking in line with this approach, see Ansell, C.K., Pragmatist Democracy: Evolutionary Learning as Public Philosophy, Oxford University Press 2011; Dorf, M.C. and Sabel, C.F., “A Constitution of Democratic Experimentalism,” Columbia Law Review 98:2 (1998): 267-473; Sabel, C.F. and Zeitlin, J., “Experimentalist Governance,” in The Oxford Handbook of Governance, ed. Levi-Faur, D., Oxford University Press, 2012: 169-184.

[41] On the idea of an antidisciplinary research program, see https://jods.mitpress.mit.edu/pub/designandscience

[42] On the application of this to the public realm, I am inspired in part by the communitarian thinking of Benjamin Barber; see Barber, B., Strong Democracy: Participatory Politics for a New Age, University of California Press, [1984] 2003. For instance, on reductionist conceptions of freedom that pose it in opposition to power, Barber ([1984] 2003, 36) remarks: “Rendering freedom and power in physical terms not only misconstrues them, it produces a conception of political liberty as entirely passive. Freedom is associated with the unperturbedness of the inertial body, with the motionlessness of the inertial frame itself. It stands in stark opposition to the idea of politics as activity, motion, will, choice, self-determination, and self-realization… The modern liberal appears to regard it [tranquility] as a republican ideal: man at rest, inactive, nonparticipating, isolated, uninterfered with, privatized, and thus free.”

[43] For one argument that offers this view from the perspective of psychology, see Seligman, M.E.P., Flourish: A Visionary New Understanding of Happiness and Well-being, Free Press, 2011.

[44] https://jods.mitpress.mit.edu/pub/resisting-reduction