“We’ve rushed ahead, paying little attention to vulnerabilities inherent in machine learning platforms – particularly in terms of altering, corrupting or deceiving these systems,” explains a DARPA program manager.
Dr. Hava Siegelmann
“We must ensure machine learning is safe and incapable of being deceived”
Dr. Hava Siegelmann, program manager in the Defense Advanced Research Projects Agency‘s (DARPA) Information Innovation Office (I2O), introduced the Guaranteeing AI Robustness against Deception (GARD) program earlier this month to address vulnerabilities in machine learning (ML) platforms and to develop a new generation of defenses against adversarial deception attacks on ML models.
“There is a critical need for ML defense as the technology is increasingly incorporated into some of our most critical infrastructure,” said Siegelmann.
“The GARD program seeks to prevent the chaos that could ensue in the near future when attack methodologies, now in their infancy, have matured to a more destructive level. We must ensure ML is safe and incapable of being deceived,” she added.
GARD will focus on three main objectives:
“The kind of broad scenario-based defense we’re looking to generate can be seen, for example, in the immune system, which identifies attacks, wins and remembers the attack to create a more effective response during future engagements,” added Siegelmann.
Read More: ‘AI defense development may gain insight from biological immune systems’: DARPA
The GARD program will initially concentrate on state-of-the-art image-based ML, then progress to video, audio and more complex systems – including multi-sensor and multi-modality variations. It will also seek to address ML capable of predictions, decisions and adapting during its lifetime.
In a similar vein, DARPA is simultaneously funding research into making machines more trustworthy through the Competency-Aware Machine Learning (CAML) Program, which aims “to develop competence-based trusted machine learning systems whereby an autonomous system can self-assess its task competency and strategy, and express both in a human-understandable form, for a given task under given conditions.”
Read More: A program to keep Prometheus out of machine learning systems
Just as Prometheus was a liberator of humankind by bringing the flame of knowledge to humanity by defying the gods, DARPA wants to make sure that machine learning is trustworthy and doesn’t free itself and spread like an uncontrollable wildfire.
DARPA wants to make AI an collaborative partner for national defense, but at this early stage the language suggests that DARPA wants to make sure that machine learning doesn’t keep us in the dark about how it functions, why it behaves, and what it will do next.
Last year, DARPA announced that it was building an Artificial Intelligence Exploration (AIE) program to turn machines into “collaborative partners” for US national defense.
Read More: DARPA wants to make AI ‘collaborative partner’ for national defense
Programs like GARD and CAML aim to further that agenda.
Banks have moved from feast to famine amidst a spike in the number of financial…
Being a pet owner can be hard. For one, according to researchers from the University…
In this episode of Brains Byte Back, Erick Espinosa sits down with Sam Oliver, CEO…
Education technology (edtech) is once again among the top sectors in Southeast Asia (SEA) for…
There is no single right big data solution. Organizations are most likely to succeed with…
In 2017, a group of researchers (from Google and the University of Toronto) introduced a…
View Comments