The CRAFT program at the ACM Fairness, Accountability, and Transparency (FAT*) Conference was designed to welcome more holistic approaches to statistical and automated decision making that address critiques of the field of fairness, accountability, and transparency, be it by looking into its blind spots, omissions, or through exploring alternative possibilities.
It was an initiative by the FAT* community—which leads research on algorithmic bias worldwide—to formally acknowledge and welcome the critique of artists, activists, advocates, and engaged scholars from all disciplines focused on questioning and improving automated systems used in everyday life. CRAFT aimed to draw computer scientists, technology practitioners, and others in the FAT* community, as well as those who may benefit from holistic research on the topic to the CRAFT program. It was an ambitious and, at the same time, a modest proposal to open a reflexive conversation on the visions, objectives, and methods that FAT* intends to engage.
The questions we ask ourselves now are: Did we get where we want to go? Has CRAFT opened a space that allows reflexive practice within the algorithmic bias research community to flourish? While it may be too early to answer, going forward, we find it important to take a moment to reflect on our experience.
When we committed to co-chairing CRAFT, we came up with four themes, which resonated with questions raised by researchers and practitioners, and asked for submissions which addressed questions like:
What role can computer science play in developing processes for drawing red-lines that may be applied to current, legacy or future machine learning systems?)
How can FAT* learn from critics who point out that systems affect people in situated ways and that system design requires that designers understand power structures, exploitation, marginalization and other theories of subjugation?
How do we turn the FAT* process inwards on our own community and reflect on how, cognizant of power differentials, we might improve research processes internally as well as interactions between and engagement with different disciplines?
What are (a) unresolved questions in existing fields that need to be addressed, (b) desirable operating characteristics for FAT* frameworks in particular application areas that have yet to be explored, or (c) new frontiers of FAT approaches that require rethinking current practices?
To situate where our themes were coming from, we created a short blog post which pointed at some of the prominent critiques and engagements with FAT* visions, objectives and methods.1 For the selection of the submissions, we invited a group of reviewers that spanned computer scientists, critical race and technology scholars, practitioners working on technology for social change, artists, and more. Fortunately, most people we reached out to agreed to serve as reviewers of incoming CRAFT proposals (see “Selection Committee” here). They pored over fifty-two proposals.
Looking back at this pool of proposals reveals a lot about CRAFT and the fairness community. We now see that the diverse submissions we received brought cross-disciplinary and cross-practice approaches together in their proposals. The table provides an overview of the many backgrounds of the coordinators of the 52 proposals.
| Coordinators in Proposal included | # of Proposals |
|---|---|
| Academia | 20 |
| Industry | 18 |
| Advocacy Groups | 6 |
| Think tanks (corporate/philanthropy funded) | 6 |
| Government Entities | 5 |
| Independent Researchers and Artists | 5 |
| Professional Organizations | 1 |
Reviewing our proposal pool, we see that the most prominently represented themes (16/52) were sessions discussing the straightforward application of concepts like accountability, fairness, transparency or explainability frameworks—or approaches to them. This was followed by proposals to apply ethics frameworks that opened broader questions to which existing approaches (e.g., fairness measures) could be a response. Several proposals concerned the need for diversity and for Global South presence in fairness research and in technology governance structures. A good number of proposals wanted to bring in their fairness process or products to be tested with CRAFT participants. In other submissions, people made concrete suggestions for concepts or topics not addressed with fairness, accountability or transparency; for the introduction of new methods (e.g., qualitative methods, design thinking) into the toolbox of computer scientists; critique of existing approaches to fairness, accountability, or explainability, and a handful of proposals that looked into engaging affected communities. From a disciplinary lens, only two proposals came from computer scientists, and one proposal came from a self-organized community initiative.
The program that emerged from the review process yielded 13 sessions for the CRAFT program. We augmented this program by inviting three new sessions focused on: speculative AI futures; policing, technology, and minority groups in the European context; and visibility, surveillance, and marginality.2 We also added a “rump session”—which rallies conference participants to sign up and present new questions, reflections, projects, and more in five-minute bursts. In all, the final CRAFT program reflects an impressive group of presenters and session coordinators who have prepared sessions in a variety of formats for a whole plethora of important topics, including issues of modeling and non-deployment of automated systems. Before we look back, it is important to say that we are first and foremost grateful for all the sessions and for the fact that CRAFT was made possible through the generous support of ACM FAT* organizing team.
Looking back at the process and the first day of CRAFT, we heard positive feedback along three basic themes:
(Il)legibility and solidarity. During the distribution of the CRAFT call, we reached out to people working on relevant topics. Scholars and practitioners working on migration, labor, and critical geography, for example, gave us feedback that the call was not legible. Before the conference, we had to “make the road” and invent or adapt processes for session proposals, resulting in the discovery of many cultural differences among coordinators. (We threw hotcrp—a conference paper submission site designed for computer scientists but no one else—at CRAFT coordinators some of which had no familiarity with the site.) But coordinators were patient and (we think) forgave us. Even during the ACM FAT* conference, many conference attendees were still asking what CRAFT was. But participants showed up, listened in, and pushed back in constructive ways, and many sessions were oversubscribed. This feels like solidarity. We are thankful to everyone who beared with us as we pushed and pulled to make space for CRAFTers. We are especially thankful to the CRAFTers who are unused to this setting for their patience and generosity.
Welcoming format. The typical FAT* conference format consists of short and streamlined paper presentations in front of all 600-plus attendees. This year, the organizers squeezed the schedule further to make time for CRAFT and create space for more unorthodox (by computer science standards) interventions. Our sessions welcomed dialogue and opened the possibility for questions and challenges between presenter and participants. One session came to bear on concerns around justice and fairness through Black feminist thought and poetry. Others asked conference participants to share the stage with invited participants or to set the terms of small group discussions. Still yet others moved participants off site. However familiar or unfamiliar, these formats flaunted a less hierarchical style of engagement.
Main schedule vs. sideshow. Very early on, conference organizers acknowledged the importance of making CRAFT a part of the main conference schedule. Rather than being cordoned off to a pre- or post-event, CRAFT sessions featured as main events across two days of a four-day conference. The enthusiasm we received for the different formats and the high attendance (we were competing with the beautiful city of Barcelona) raises the question: should ACM FAT* make more time for alternative formats and approaches in the conference proper?
If critique and reflection is what we called for, then it goes without saying that we would also like to look critically at what the CRAFT program did and did not do:
Modeling participation. The ACM FAT* relies on an array of mostly corporate sponsors, and the pooled funds help to subsidize travel grants and fee waivers. Funding is necessary to make events like CRAFT to happen, and we are thankful for the efforts that go into making it available. At the same time, individuals had to make their case to a funding committee as to why they should receive such support. For many coming from outside of the academy, the registration and funding support model for individuals makes it feel like it’s a privilege to attend rather than a pleasure to host. It is no wonder that the CRAFT program featured a very small number of groups that directly work with marginalized communities, most of which work in a different time scale and capacity. In addition, the subsidy model raises concerns that it may give sponsors a type of legitimacy and credit for supporting a palatable form of diversity and inclusion. While good intentions may lie behind the current subsidization model, it approximates a policy of containment: allow a few outsiders into the conference in a safe and controlled manner.
Indeed, you can’t ignore the corporate leadership in the CRAFT program. Corporate-sponsored researchers coordinated sessions that address “red lines,” the need to pause, or reconsider the development and deployment of automated systems. These are exactly the kind of topics we had hoped for in CRAFT. However, taken alongside the inability of affected communities (or their representatives) to easily slot into the conference, these critical corporate-sponsored sessions beg the question: is this the shape of things to come? Are we going to see greater numbers of corporate-organized events reflecting critically on AI and what new technologies have to offer society? Who is nourished by this practice? And who is deprived?
Missing links. Computer scientists and engineers dedicated to fairness, accountability, and transparency were few and far between among the CRAFT coordinators. For comparison, take for example our selection committee. Twelve out of 43 reviewers on the CRAFT selection committee have a background in computer science or engineering, and at least some of these are publishing in the FAT* community. By contrast, only 2 (3 if you count engineering practitioners) of the 52 proposals came from computer scientists in the FAT* community. Most of the CRAFT program tries to surface values and assumptions underlying FAT* research. One session explicitly questions what computer scientists or engineers can to do look inwards, acknowledge power differentials, or transform research processes or interdisciplinary interactions.
We don’t know and are curious to hear why computer scientists and engineers largely stayed away from the CRAFT call. But the lack of representation by computer scientists and engineers in the submissions and final program of CRAFT is disappointing. While we wanted CRAFT to be a meeting point and an opening, our party ended up being incomplete. The CRAFT program features little of the evidence of the spark found of groups like Data4Black Lives or Free Radicals or self-organized movements coming out of computer science and engineering.3 Meanwhile, researchers who are “supportive of theoretical computer science… and other relevant analytical fields” are decamping to their own symposium that prioritizes mathematical work (versus the type of interdisciplinary thinking encouraged by CRAFT and FAT*). We acknowledge the immense pressure faced by PhD students and faculty to produce work, and the publish-or-perish hamster wheel may have made CRAFT submissions unviable. Given that, how can ensure that CRAFT or FAT* is also a place for self-reflection rather than a space predominantly to serve self-preservation through adherence to expected outputs and commitment to existing methods?
CRAFT has attracted a lot of people wanting to debate values and assumptions fundamental to research at FAT*. You could even say there is consensus around Kranzberg’s famous dictum that technology is never neutral. But what we feel CRAFT failed to adequately grapple with is the fact that math and formal models of computation is never apolitical.4
Despite claims that data sets can be debiased and algorithms neutralized, and despite their abstract (numeric) nature, machine learning, data mining, and artificial intelligence are inherently political methods. As many of our sessions intended to do, we need to address power asymmetries between people and institutions when such methods become priorities. However, we feel that the CRAFT program just scratched the surface when investigating the political power inherent in these methods which are so dear to technical researchers within the FAT* community. A few sessions questioned fundamentally the appropriateness of the application of concepts like utility metrics, optimization, accuracy to hard social justice problems, but these were exceptional among the applications. We would have liked CRAFT to play host to conversations about political alternatives, for example, to cloud computing facilities run by major global companies that power computational research. We feel it opened little space to consider concretely how these companies are building infrastructures of dispossession or predatory inclusion.
This is not to say those questions and provocations are absent at the conference or in the larger FAT* community. But we think these kinds of issues need a grander audience and a more committed approach. For that to happen, we plainly ask: should CRAFT continue? Or should it end? Could its aims and objectives be more centrally integrated into the conference’s main call for contributions?
Given the increasing interest in algorithmic bias, such questions need answers. We see a community that stands at a pivotal moment when it can choose to engage or to fragment.
The latter may lead us down a familiar path. As with privacy5, we can imagine the professionalization of fairness in institutions (e.g., “chief fairness officer”) and a banalization of fairness concerns. In the process, we may also witness a growing power differential between computational sciences versus social sciences and humanities, the latter two being only fungible when they serve to increase “trustworthiness” and thus acceptance of new technologies.
The former may take us into unchartered territory, but we think there are models from which to borrow inspiration, incomplete or flawed as they may be. These might include starting self-organized research networks, diverting funding to under-resourced research, developing adversarial strategies to circumvent or thwart power holders in the world of AI and automated systems, or taking the time to be reflexive rather than cherishing the current incentive structures which favor paper publication over engagement and collective reflection.
The immediate reactions we received demonstrate an enthusiasm and hunger for more of what was offered through CRAFT. And we see the value and urgency of finding ways to accommodate endeavors of the kind we propose above. ACM FAT* has done a tremendous job by enabling CRAFT to take place. We have faith that this support provides an opening for all conference organizers and participants to think about how their visions and, importantly, methods require constant and collective introspection and commitment. Some of these commitments include:
See CRAFT as a process through which to change conference practices and substance
Be transformative, rather than focusing on small add-ons that preserve computer science incentive structures and rewards
Take your time: introspection and change cannot be rushed
Make more space for uncomfortable encounters and political positions
We hope that whatever paths people take, spaces will be created not only for rethinking algorithms, or applying existing approaches to social justice, but also for reimagining computer science theory and practice as a whole.
Seeta Peña Gangadharan, Seda Gürses
Critiquing and Rethinking Fairness, Accountability and Transparency https://www.odbproject.org/2019/07/15/critiquing-and-rethinking-fairness-accountability-and-transparency/ ↩
This artistic performance unfortunately did not make it to Barcelona, but a modified version will take place here: https://www.aies-conference.com/2020/main-evening-event. ↩
Computer Professionals for Social Responsibility, Informatica Feminale, Chaos Computer Club, OHM, Hackers on Planet Earth serve as some examples. Note: none of these efforts is without criticism. But what sustains them is a sense of moral purpose and a sense of political possibility. ↩
We draw here from Phil Rogaway, whose writing on the moral character of cryptography says: “Cryptography rearranges power: it configures who can do what, from what. This makes cryptography an inherently political tool, and it confers on the field an intrinsically moral dimension.” ↩
The International Association of Privacy Professionals https://iapp.org ↩