NASA's New Self-Learning AI Could Save First Responders

NASA scientists are engineering a form of artificial intelligence (AI) that they hope will help firefighters and other first responders escape dangerous situations. Set to launch next year, the system will help first responders through unpredictable fires and chemical leaks by giving them advice based on machine learning of past emergencies.

The new system—called AUDREY—the Assistant for Understanding Data through Reasoning, Extraction and sYnthesis—is designed to be distributed to individual firefighters so it can collect a precise network of data directly from the field, and learn from that data for next time. No emergency is the same, which means first responders have to rely on extensive training and experience to stay safe in dangerous conditions that can change rapidly. The AUDREY system hopes to use distributed data collection and machine learning to better inform first responders about the situation at hand.

The AI system is under joint development by the Department of Homeland Security, which funded the project, and NASA’s Jet Propulsion Laboratory. It can analyze data and respond to human queries on demand, said Mark James, a supervisor and scientist at the lab. And it will communicate with those assigned to other first responders on the scene, creating a mesh network of AIs comprised of the police, firefighters and EMT.

Jet Propulsion Laboratory. Image: NASA

While digital assistants like Siri or Alexa are programmed to respond to language inputs, James explained, AUDREY won’t be working with a fixed set of rules. “Just like a person, AUDREY needs education before she can solve a problem,” he said.

Each first responder trying AUDREY out will get a version that has been pre-educated for their specific vocation, and customized to his or her own preferences. At first, these “mini-AUDREYs” will be accessed through mobile phone and internet, and eventually first responders will be able to access the program through special headgear, with voice command and LED lights.

Since AUDREY is educated before heading to the field with first responders, she will amass data from before, during and after an incident. “She fuses all this information together, understands the roles and training for each person and their equipment, and synthesizes a solution to problems posted by the first responder,” James explained.

And during the time of response, the systems will communicate. “If firefighters, EMT and police are all present, those disparate AUDREYs will talk to each other,” said Edward Chow, a manager from the Department of Homeland Security.

An emergency team can work without the cloud for indefinite amounts of time since AUDREY can run on regular LTE, like a phone, and doesn’t need to be connected to a tower to operate.

AUDREY workflow. Image: Jet Propulsion Laboratory

“When a local AUDREY loses connection to the cloud, they will work together to request and pass on requisite information for each first responder’s role via those AUDREYs still in contact with the cloud,” to maintain localized situational awareness, he said.

And the system will be useful even when it isn’t connected to a human responder. The team is building models that can be left in the cleared room of a burning building. These will act as sensors to monitor concentrations of flammable gas and temperature could be dropped behind the teams, keeping firefighters in-the-know about rooms in imminent danger of igniting or collapsing.

“For example, as a responder moves through a burning building, he could drop black boxes along his way,” and with on-board sensors, the data streaming from these breadcrumbs would help AUDREY let its users know if the way in is no longer such a good way out, James explained.

And at the end of a job AUDREY enters “dreaming mode”, during which all the problems solved and dangers curtailed in a day’s work are sifted and consolidated. The data adds to the system’s bank of experience, to be used for guiding a first responder’s through his or her next job.

Maybe that's something we can learn to do next.

from NASA's New Self-Learning AI Could Save First Responders