Suresh, Harini, Emily Tseng, Meg Young, Mary Gray, Emma Pierson, and Karen Levy. 2024. “Participation in the Age of Foundation Models.” Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (New York, NY, USA), FAccT ’24, June 5, 1609–21. https://doi.org/10.1145/3630106.3658992.

Notes

In-text annotations

"Alongside these endeavors, scholars have also described how participatory efforts can fall short of meaningfully shifting power to the marginalized, such as through participation-washing or cooptation by powerful interests" (Page 1610)

"The framework extends the carpentry metaphor of a foundation—an unfinished base that can support many different kinds of structures. Built on top of the foundation, a subfloor layer provides a level and structurally-sound base to support the top-level surface layer. In our framework, the subfloor layer encompasses technical infrastructure, norms, and/or governance for a grounded domain (e.g., reproductive health)." (Page 1610)

"Participatory traditions have a long history in and outside of technology design [5, 42, 46, 69, 74, 75]. Specific motivations for participation vary, and include redistributing decision-making power to those who have less [75], learning from participants’ expertise or preferences [39, 87], or meeting epistemic goals such as procedural fairness" (Page 1610)

". Why are participatory approaches thus far limited to those that lend little power to stakeholders? We argue that the intersection of primarily corporate control and context-agnostic models leads to a participatory ceiling on what is possible when attempting to directly intervene on a foundation model." (Page 1613)

"Foundation model developers currently lack incentives to share control with communities. In our analysis, the stakes of participation are most often limited to consultation." (Page 1613)

"But the primacy of proprietary models managed by corporate actors creates an additional layer of detachment separating public stakeholders from decision-making processes. If meaningful participation requires the distribution of some decision-making power, this shift is not easily managed by firms, which are primarily constituted to protect shareholder interests rather than open collaboration to societal stakeholders." (Page 1613)

"Harrington et al. [51] argue, asking marginalized communities to engage in the “blue-sky ideation” of technology design risks ultimately frustrating underserved individuals. By focusing instead on the real harms and concerns people are presently experiencing, developers can limit the material and affective demands of participation [34], acknowledge participants as experts on local knowledge systems, and create outputs that more closely reflect actual downstream needs." (Page 1614)

"UbuntuAI [36, 77] proposes a system that responds to the expropriation by foundation models of African artists’ intellectual property by collaboratively creating a licenseable dataset of their work. While the project is still exploratory, its approach touches on the collaboration mode of participation, with ongoing co-creation of the dataset and a compensation structure that lends itself to community control." (Page 1614)

"these reporting and organizational infrastructures would create avenues for more robust, real-world, and variegated external input to shape surface-layer systems, in spite of a highly closed and rigid overall environment. It would also serve as an “early warning system” when cardholder communities or merchants face an uptick in declined transactions because of spurious or discriminatory correlations made by an LLM-backed product." (Page 1617)

"Our analysis identified the inherent centralization in foundation model development as a key bulwark of the participatory ceiling (Section 3.3). Whether a venture-backed startup, a technology giant, or a well-resourced academic or government institution, foundation model developers are an obligatory passage point for influencing the operation of a given foundation model." (Page 1617)