AI

Drug-discovery AI can be inverted to create chemical weapons, scientists find

It took researchers under six hours to coax a drug-discovery AI into producing 40,000 toxic molecules.
article cover

Dbenitostock/Getty Images

· less than 3 min read

Stay up to date on emerging tech

Drones, automation, AI, and more. The technologies that will shape the future of business, all in one newsletter.

On the one hand, harnessing the power of data-crunching algorithms to discover new drugs is extremely exciting—a tangible example of AI for good. On the other, those same algorithms can easily be inverted to develop bioweapons, according to a new article in Nature Machine Intelligence.

Wait, what? Collaborations Pharmaceuticals, a company that uses AI for drug discovery, was asked to prepare a conference presentation about the risks drug-discovery AI poses regarding dual use—a term describing the potential for ostensibly good scientific and technological developments to be used for not-good things.

Collaboration found that, in this case, the potential for dual use is alarmingly high. In under six hours—and with a couple tweaks to optimize for toxic compounds—a machine learning-powered molecule generator that the company had originally created to help guide drug discovery was spitting out deadly designs.

  • The system designed 40,000 molecules that scored within Collaboration’s “desired threshold” of toxicity, including known nerve agents, like VX, but also potentially new compounds that are even more toxic than well-known chemical agents.
  • The model is based on, and similar to, open-source software that is “readily available,” the authors wrote. The initial training data set did not include nerve agents.

To be clear…Collaboration did not actually synthesize any of these toxic compounds, but the researchers pointed out that “with a global array of hundreds of commercial companies offering chemical synthesis, that is not necessarily a very big step, and this area is poorly regulated.”

  • They also note that while expertise in chemistry or toxicology is still needed to wield these systems to such ends, ML-based systems like this “dramatically lower technical thresholds.”

The authors suggest a number of safeguards to protect against such misuse of commercial products, from API restrictions to reporting hotlines to more investment in ethics education for STEM students.

Zoom out: Concerns about dual-use technologies are nothing new: A compound used to prevent earwax is regulated as a chemical weapon because it is a key input to mustard gas, for example. But the burgeoning bioeconomy will present new iterations of the problem.

Stay up to date on emerging tech

Drones, automation, AI, and more. The technologies that will shape the future of business, all in one newsletter.