Evaluating Technology Policy

The Powerful and the Powerless: Reflections from FAccT 2020

By Zoe Kahn, CTSP Fellow 2019

Images of people soaring through space wearing astronaut helmets adorn the walls of a narrow hallway leading from the street-level entrance to the hotel lobby elevator bank. A life-sized model of an astronaut stands off to the side of the hotel registration desk. It’s safe to say… the conference hotel has an outer space theme. 

The words “What is fairness?” repeat on a large circular digital display at the center of the hotel lobby. I can’t decide if the display belongs in a newsroom, stock trade room, or train station. Perhaps art? Perhaps commentary? 

The word in question changes each day. What is Fairness? What is Accountability? What is Transparency? What questions are we missing? 

Image of digital sign reading "what is fairness?"

It is here—in Barcelona, Spain—that ~600 people gather for the ACM Fairness, Accountability, and Transparency (FAccT) conference to discuss efforts to design, develop, deploy, and evaluate machine learning models in ways that are more fair, accountable, and transparent. 

Despite the conference’s roots in computer science, there were several papers, tutorials, and CRAFT sessions that challenged the community to think beyond technical fixes to broader socio-technical contexts rife with power and complex social dynamics.

Let me begin with a discussion of power. The ACLU of Washington and the Media Mobilizing Project put on a terrific CRAFT session titled, Creating Community-Based Tech Policy: Case Studies, Lessons Learned, and What Technologist and Communities Can Do Together. The two-part session first documented the ACLU of Washington’s efforts, in partnership with academics primarily from the University of Washington (Katell et al., 2020), to include marginalized and vulnerable communities in the development of tech policy surrounding the acquisition and use of surveillance technologies in the City of Seattle. The Media Mobilizing Project then presented their work investigating the use of risk assessment tools in the criminal justice system across the country. And, in particular, their efforts with community groups to combat the use of criminal justice risk assessment tools in Philadelphia. Taken together, the session provided models for meaningfully engaging marginalized and vulnerable populations in algorithmic accountability, forming the foundation for workshop participants to reflect upon their own power.

READ MORE

Backstage Decisions, Front-stage Experts: Non-technical Experiences and the Political Engagement of Scientists

by Santiago Molina and Gordon PherriboCTSP Fellows

This is the second in a series of posts on the project “Democratizing” Technology: Expertise and Innovation in Genetic Engineering.

See the first post in the series: Backstage Decisions, Front-stage Experts: Interviewing Genome-Editing Scientists.

Since 2015, scientists, ethicists, and regulators have attempted address the ethical, moral, and social concerns involving genetic modifications to the human germline. Discourse involving these concerns focused on advancing a culture of responsibility and precaution within the scientific community, rather than the creation of new institutional policies and laws. Confidence in scientist’s ability to self-regulate has become increasingly tenuous with the recent news of the birth of genome-edited twins on November 26th, 2018, despite the scientific consensus that such experiments are medically and ethically unwarranted. In response, journalists, social scientists and critical researchers in the life sciences have posed the question: Who should be involved in deciding how genome-editing technologies should be used and for what aims?

In this post, we complicate the idea that technical expertise, which is usually narrowly defined on the basis of professional experience or knowledge, should be the main criteria for having a seat at the table during scientific decision-making. Drawing from eight interviews with scientists who participated in a small meeting held in Napa Valley in 2015, we highlight the role of non-technical experiences in shaping scientists’ views of decision-making about genome editing.

We identify three experiences that have influenced scientists’ views and deliberations about the application and potential consequences and benefits of genetic engineering technologies: 1) reading and group discussions outside of their academic disciplines, 2) direct engagement with patient communities, and 3) involvement in social movements. To wrap up, we make some modest suggestions for what these might mean in the context of STEM education.

1. Reading Outside of the Discipline and Group Discussions.

During our interviews we asked scientists how they shaped their viewpoints about biotechnology and its relationship to society. Respondents described their exposure to new viewpoints and reflected on the effect this exposure had on their decision-making. One of the sources of these exposures was reading outside of their academic discipline. We were surprised to hear about how the work of philosophers of science and sociologists of science did inform the decision making of one senior scientist at the Napa Valley meeting. This faculty member discussed their interest in finding opportunities to supplement their laboratory training with philosophical discussions about issues tangential to the science they were working on. With other graduate students, they created a small group that met regularly to discuss concepts and theories in philosophy of science, ethics and sociology of science.

We met- I don’t remember whether it was once a month or once every two weeks to discuss issues around the philosophy and societal issues of science. So we would find books, read books, um from you know – from Bertrand Russell, the philosopher, to Jacob Bronowski to Alfred Lord Whitehead, you know books on the philosophy and the applications of science,

The scientist described that this work added additional layers to their understanding of societal issues related to science. Even though this reading group was instrumental in developing his own awareness of the relationship between science and broader social, political and cultural issues, this respondent also lamented how the opportunity to delve into topics outside of a graduate student’s normal routine, “was not encouraged by any of [their] mentors.” This theme came up in several of our interviews, reinforcing the importance of mentors in shaping how scientists make meaning of their discipline in relation to society, and what educational and professional development opportunities graduate students feel comfortable pursuing outside of their formal training.

2. Direct engagement through service.

The most distinctly communicated experiences our interviewees engaged in outside of their formal training were service-related learning experiences that involved direct interaction with communities that would medically benefit from the technology. These experiences appeared to give individuals a greater sense of civic responsibility, and afforded them a more expansive understanding of the relationship between their work and broader communities. For genome-editing researchers, this crucially meant being aware of the social and medical realities of patients that might be research subjects in clinical trials for CRISPR-based therapies.

In our interviews, scientists had direct engagement with people outside of their discipline within national scientific boards, federal organizations, health clinics, and the biotech and pharmaceutical industry. These types of experiences provide an opportunity to collaborate with stakeholders on pressing issues, learn and benefit from industry and market knowledge, and ensure that the outcome of decisions are both relevant and meaningful to community stakeholders outside of the lab.

One of our respondents reflected on how they learned important skills, such as active listening, through professional experiences with indigenous patient communities–which helped this respondent better serve the community’s needs.

I’ve learned a whole lot from the patients I’ve taken care of and the people I’ve met. I certainly learned a great deal from going to the Navajo reservation. I’m – just to be able to sit down in a very different culture and listen and I think it’s very important for doctors to listen to their patients.

This interviewee was additionally committed to modeling the listening behavior of physicians and teaching these listening skills to others. When we further asked “What what do you think was specific about the way that [your mentors] spoke with patients and interacted with them?” the interviewee responded with clarity:        

Sitting back and not speaking and letting them talk about what’s important to them.

The interviewee conveyed that if you listen, people will tell you what is most important to them. They further argued that as decision-makers guiding the usage of far-reaching technologies, it is important to not make assumptions about what a particular community needs.

Similarly, in another interview, a molecular biologist described their experience setting up clinical trials and discussing the risks and benefits of an experimental treatment. This experience not only gave them a more concrete sense of what was at stake in the discussions held at the Napa Meeting, but also helped sensitize them towards the lived experiences of the patient communities that may be affected (for better or worse) by genome editing technology. When asked if experiences during their doctoral program, postdoc or work at a biotech firm, had prepared them for discussing genome editing and its implications, the molecular biologist responded:

Having been involved in therapeutic programs in which you’re discussing the pluses and minuses of therapies that can have side effects can prepare you for that. […] To me that was very helpful because it was a very concrete discussion. That conversation was not a like, “oh, I’m an academic and I wanna write a paper and someone’s going to read it and then enough.” […] [In a therapeutic program] the conversation was like, “we have a molecule, are we going to put it in people?” And if the answer is “yes,” like there is a living person on the other end that is going to take that molecule and [they are] going to have to live with the consequences positive and negative. […] 

            The distinction being drawn here between scientific work with concrete outcomes for people and work with solely academic outcomes, suggests that there are practical experiences that researchers at Universities may only have indirect knowledge of that are important for understanding how the products of science may affect others. As the interviewee further explained, the stakes of being unfamiliar with patient’s experiences are particularly high,

[My work at a biotech firm] has sort of prepared me at least a little bit for some of the discussion around therapeutic editing because different patient populations have wildly different ideas about gene editing. There are certain forms of inherited blindness where people are frankly insulted that you would call it a genetic disease, right? And I think rightly so. That’s their experience. It’s their disease. Why should we call this something that should be quote-unquote “corrected,” right?

In this case, prior experience with clinical trials alerted the researcher towards the heterogeneity of experiences of different patient populations. They further described how, in other interactions with patient advocates through public engagement, they were able to learn a great deal about the uniqueness of each patient group and their different views about genome editing. Here, the researcher additionally conveyed concern over the ableism that is often implicit in medical views of difference. They recounted how listening to perspectives from different patient communities led them to reflect on how procedurally safe genome editing can still cause harm in other ways.

3. Involvement in social movements.

The third non-technical form of expertise came from researchers’ political participation. While the recent fervor against the GOP’s “war on science” may give us ample evidence that politics and science don’t mix well, the role of social movements in the creation of scientific knowledge has been extensively documented by sociologists. For example, post World War II environmental movements changed the content, form and meaning of ecological research (Jamison 2006) and Gay Rights and AIDS activists helped steer the direction of biomedical research (Epstein 1996). What is less emphasized in these studies though, is how participation in social movements by scientists can impact their worldview and decision-making. When asked what personal experiences shaped how they thought of the process of decision-making around new biotech, one interviewee mentioned their engagement with political movements in the late 1960’s during anti-Vietnam War protests :

So I was in Berkeley in the late 60s…This is a time of a lot of social activity. Protests that went on against the Vietnam War in favor of civil rights. There was a lot of protest activity going on and I was involved in that to some extent, you know, I went on marches. I went door-to-door one summer in opposition to the Vietnam War…Um, so I had to you know- I had sort of a social equity outlook on life. All the way from my upbringing from college- and then at Berkeley you really couldn’t avoid being involved in some of these social issues.

This respondent went on to discuss how their commitments towards social equity shaped their decision-making around emerging technologies. In another interview, a respondent described how taking time off of their graduate program to work on a local election campaign motivated them to participate in science policy forums later in their career.

However, these example also suggests that how a scientist chooses to engage with social movements can have lasting effects on how they think of themselves as being a part of a larger community. If scientists participate unreflexively, social movements can fail to challenge individual’s to consider how the network building and activism they are doing affects themselves and may be excluding others from different communities.

To give a contemporary example, the March for Science (MfS) movement in January of 2017 protested against the Trump administration’s anti-science policies and actions. While the issues about science funding were urgent, MfS organizers failed to address language issues in MfS that were dismissive of the experience of marginalized communities in science. Whether or not a participant in MfS chose to critically engage in the movement, will influence how this individual sees the world and whether they intentionally or unintentionally reproduce inequities in science. By asking scientists to think about both their role in society and about the community of science itself, social movements provide a large quantity of knowledge and creativity that scientists can contribute to and use as a resource when making decisions and reflecting on the implications of emerging technologies.

The Value of Non-technical Expertise in Training

Many of the experiences that shaped our interviewees decision-making occurred during their early graduate and professional training. Despite the personal and professional value they found in these experiences, our interviewees noted the lack of support from their graduate mentors in their exploration of non-technical interests and a lack of incentives to participate more broadly in political endeavors during their training. While this may be changing for newer generations of scientists, this raises questions about how scientists in the natural and physical sciences are socialized into the broader scientific community, and the impact of that socialization on what they think of their political responsibilities are.

For example, a consensus study of the National Academies of Sciences, Engineering, and Medicine (2018) found that there is a lack of social and institutional support for activities located outside of the traditional realm of an individual’s discipline and argued for the creation of novel training pathways that could lead to holistic STEM research training. One way of creating more holistic STEM training programs noted by the study that our findings support would be to provide resources and structures to facilitate the connection between graduate training in the life sciences and fields, such as STS, sociology and philosophy. Exposure to these disciplines can help aspiring researchers grapple with the social interactions of their discipline and serve as additional tools for constructive debates around scientific issues. Promoting interdisciplinary collaboration may also help reduce stigma associated with non-traditional pathways to scientific training and provide easier channels to integrate professional development and internship opportunities into the curriculum.

The urgency of this current gap in training is apparent if you look at who is currently at the the decision making table. The committees and meetings for deliberation about the social and ethical issues of genome editing are almost exclusively constituted by senior scientists. These researchers are mainly conscripted into these roles because of their technical expertise and status in disciplinary networks. Historically, the academic institutions these scientists were trained in were not built to prepare scientists for making political decisions or for appreciating the social complexity and nuance that comes with the introduction of emergent technologies into society. In our third blog post we will explore the political stakes of this form of science governance, which are surprisingly high.


References:

Epstein, S. (1996). Impure science: AIDS, activism, and the politics of knowledge (Vol. 7). Univ of California Press.

Jamison, A. (2006). Social movements and science: Cultural appropriations of cognitive praxis. Science as Culture, 15(01), 45-59.

National Academies of Sciences, Engineering, and Medicine (2018) Graduate STEM Education for the 21st Century. Washington, DC: The National Academies Press. doi: https://doi.org/10.17226/25038.

Backstage Decisions, Front-stage Experts: Interviewing Genome-Editing Scientists

by Santiago Molina and Gordon PherriboCTSP Fellows

This is the first in a series of posts on the project “Democratizing” Technology: Expertise and Innovation in Genetic Engineering.

See the second post in the series: Backstage Decisions, Front-stage Experts: Non-technical Experiences and the Political Engagement of Scientists

When we think about who is making decisions that will impact the future health and wellbeing of society, one would hope that these individuals would wield their expertise in a way that addresses the social and economic issues affecting our communities. Scientists often fill this role: for example, an ecologist advising a state environmental committee on river water redistribution [1], a geologist consulting for an architectural team building a skyscraper [2], an oncologist discussing the best treatment options based on the patient’s diagnosis and values [3] or an economist brought in by a city government to help develop a strategy for allocating grants to elementary schools. Part of the general contract between technical experts and their democracies is that they inform relevant actors so that decisions are made with the strongest possible factual basis.

The three examples above describe scientists going outside of the boundaries of their disciplines to present for people outside of the scientific community “on stage” [4]. But what about decisions made by scientists behind the scenes about new technologies that could affect more than daily laboratory life? In the 1970s, genetic engineers used their technical expertise to make a call about an exciting new technology, recombinant DNA (rDNA). This technology allowed scientists to mix and add DNA from different organisms; later giving rise to engineered bacteria that could produce insulin and eventually transgenic crops. The expert decision making process and outcome, in this case, had little to do with the possibility of commercializing biotechnology or the economic impacts of GMO seed monopolies. This happened before the patenting of whole biological organisms [5], and the use of rDNA in plants in 1982. Instead, the emerging issues surrounding rDNA were dealt with as a technical issue of containment. Researchers wanted to ensure that anything tinkered with genetically stayed not just inside the lab, but inside specially marked and isolated rooms in the lab, eventually given rise to well-established institution of biosafety. A technical fix, for a technical issue.

Today, scientists are similarly engaged in a process of expert decision making around another exciting new technology, the CRISPR-Cas9 system. This technology allows scientists to make highly specific changes, “edits”, to the DNA of virtually any organism. Following the original publication that showed that CRISPR-Cas9 could be used to modify DNA in a “programmable” way, scientists have developed the system into a laboratory toolbox and laboratories across the life sciences are using it to tinker away at bacteria, butterflies, corn, frogs, fruit flies, human liver cells, nematodes, and many other organisms. Maybe because most people do not have strong feelings about nematodes, most of the attention in both popular news coverage and in expert circles about this technology has had to do with whether modifications that could affect human offspring (i.e. germline editing) are moral.  

We have been interviewing faculty members directly engaged in these critical conversations about the potential benefits and risks of new genome editing technologies. As we continue to analyze these interviews, we want to better understand the nature of these backstage conversations and learn how the experiences and professional development activities of these expects influenced their decision-making. In subsequent posts we’ll be sharing some of our findings from these interviews, which so far have highlighted the role of a wide range of technical experiences and skills for the individuals engaged in these discussions, the strength of personal social connections and reputation in getting you a seat at the table and the dynamic nature of expert decision making.

[1]  Scoville, C. (2017). “We Need Social Scientists!” The Allure and Assumptions of Economistic Optimization in Applied Environmental Science. Science as Culture, 26(4), 468-480.

[2] Wildermuth and Dineen (2017) “How ready will Bay Area be for next Quake?” SF Chronicle. Available online at: https://www.sfchronicle.com/news/article/How-ready-will-Bay-Area-be-for-next-big-quake-12216401.php

[3] Sprangers, M. A., & Aaronson, N. K. (1992). The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: a review. Journal of clinical epidemiology, 45(7), 743-760.

[4] Hilgartner, S. (2000). Science on stage: Expert advice as public drama. Stanford University Press.

[5] Diamond v Chakrabarty was in 1980, upheld first whole-scale organism patent (bacterium that could digest crude oil).

Preparing for Blockchain

by Ritt Keerati, CTSP Fellow | Permalink

Policy Considerations and Challenges for Financial Regulators (Part I)

Blockchain―a distributed ledger technology that maintains a continuously-growing list of records―is an emerging technology that has captured the imagination and investment of Silicon Valley and Wall Street. The technology has propelled the invention of virtual currencies such as Bitcoin and now holds promise to revolutionize a variety of industries including, most notably, the financial sector. Accompanying its disruptive potential, blockchain also carries significant implications and raises questions for policymakers. How will blockchain change the ways financial transactions are conducted? What risks will that pose to consumers and the financial system? How should the new technology be regulated? What roles should the government play in promoting and managing the technology?

READ MORE

Bodily Integrity in the Age of Dislocated Human Eggs

by Allyn Benintendi, CTSP Fellow | Permalink

In late October of 2012, soon after the American Society for Reproductive Medicine (ASRM) lifted the experimental label from human egg freezing, the good news spread like wildfire (Frappier 2012). Egg freezing is a medical procedure that harvests and removes a female’s mature oocytes (eggs) from her body for rapid freezing and storage for later use. Even though the ASRM report deliberately warned against healthy women freezing their eggs for the sole purpose of delaying childbearing, some saw with egg freezing a world-changing opportunity. This opportunity rested in the idea that the institutional failures that females faced as both laborers and eventual mothers could be relieved by a medical procedure. Bloomberg Businessweek aptly identified the solution and the problem in a 2014 headline, “Freeze Your Eggs, Free Your Career.” For tech giants Facebook and Apple, egg freezing is now a part of professional benefits packages.

READ MORE

Bug Bounty Programs as a Corporate Governance “Best Practice” Mechanism

by Amit Elazari Bar On, CTSP Fellow | Permalink

Originally posted on Berkeley Technology Law Journal Blog, on March 22, 2017

In an economy where data is an emerging global currency, software vulnerabilities and security breaches are naturally a major area of concern. As society produces more lines of code, and everything – from cars to sex toys is becoming connected: vulnerabilities are produced daily.[1]   Data breaches’ costs are estimated at an average of $4 million for an individual breach, and $3 trillion in total cost. While some reports suggest lower figures, there is no debate that such vulnerabilities could result in astronomically losses if left unattended. And as we recently learned from the Cloudflare breach, data breaches are becoming more prominent and less predictable,[2] and even security companies get hacked.

READ MORE