At the Intersection of Machine Learning and Disaster Response
AIVO recently spoke with Thomas Manzini, PhD candidate at Texas A&M University. Tom works with the NSF AI Institute for Societal Decision Making (NSF AI-SDM), the NSF-funded AI Institute that develops human-centric AI to enable effective, agile, resource-efficient and trustworthy decision making in uncertain and dynamic situations arising in societal use cases, such as disaster management and public health.
We discussed his work with AI-SDM, at the intersection of machine learning (ML) and disaster response. He works to improve the robustness and usability of automated systems in extreme environments.
Here’s the excerpt from our interview:
AIVO: What prompted you to delve into your current research area(s)?
Tom: Just for context, my research focuses at the intersection of disaster response aviation and computer science machine learning. That’s what brought me here. The thing that brought me into the space in general is, I like to think I have some operational experience in all three of those categories. My formal education is all in computer science and machine learning. But in addition to that, I worked for now-almost 10 years as an emergency medical technician; three years as a firefighter; and I’ve got a bunch of other certificates down the emergency response line. And in addition to that, I’m a commercial pilot for both crewed and uncrewed aviation platforms.
So I had this collection of experiences that I really wanted to focus my research at the intersection of. That’s what brought me to Texas A&M – to work on machine learning for drones that are flying in disaster areas, so that we can help improve decision making for emergency managers and personnel.
AIVO: Can you give us some details about your research?
Tom: The work that we’re doing is primarily focused on decision making for emergency managers, for when a disaster happens or when an emergency happens. There’s never enough information to make the decisions that you want. And one of the ways that we have seen this come to bear is when a large-scale disaster happens.
Imagine a hurricane or a major tornado. Drones get dispatched into the area, and they all are taking photos, building maps so that eventually they can make it back to the decision makers, so that they can get some idea of what’s going on.
But inevitably the disaster has destroyed a lot of things in the environment. And with that comes wireless, which is cellular or just connectivity in general. So the drones that are out there taking those images are capturing that data, and they can’t beam that data back to the decision maker in an easy way.
There is this issue of, we have the information; we know the information exists – the drones have captured it. But we can’t get it to the emergency managers in a timely way. You know, in practice, the way this is solved right now is literally just taking the SD card, putting it in a car, and driving it to the person who needs to make the decision.
But we think with artificial intelligence and the advancements in computer vision, we can speed up that cycle because there is some connectivity, just not enough to transmit all of the imagery. My work, our research in general, focuses on deploying artificial intelligence systems, computer vision systems into the wireless-denied environment so that they can look at that imagery first, and maybe not give a perfect answer as to what’s going on, but certainly tell the emergency manager which way the wind is blowing. Like, hey, the bad stuff is over here and not over here. Just getting that type of information to a decision maker 12 hours in advance doesn’t seem like a lot, but when the clock is ticking immediately after a disaster, those initial few hours are both incredibly impactful for the response and where we think we can make the greatest impact with our research.
AIVO: You’ve already touched on this next question. What is the goal of your current research project and/or its practical or real-world use?
Tom: The translation to practice is really critical here. It’s difficult to do research for disaster management or disaster response without that clear connection to practice, because otherwise you can be very, very quickly off solving problems that don’t exist. The goal for us is really to make technology, make computer vision machine learning systems that have a very clear path to practice and can be used readily.
We think there’s an opportunity to have a really virtuous cycle here where we are engaged. You know, we’re starting with the practitioners. We’re starting with their problems that they see in the field. And we’re using that to inform the fundamental research, which then feeds right back into practice.
I’m excited to say, we’ve had some successes here, at least in getting that virtuous cycle going. We had the chance to deploy our building damage assessment computer vision model last year (2024), during the hurricane seasons for Hurricanes Helene and Debby.
That meant that we could get the outputs of our model in front of the emergency managers. We could not only say, “Hey, there’s a clear translation to practice,” but also get feedback about where the problems are, where we can make things better, and use that to feed more foundational research.
AIVO: What’s the role of AI-SDM in your project?
Tom: The National Science Foundation (NSF) Institute for Societal Decision making is the primary funder for my research. I wouldn’t be able to do this research without them.
It’s been beneficial in two ways. One is helping to build those relationships with the practitioners. Like I mentioned in the beginning, I do have some operational experience. But having other people in the Institute who are focused on how decisions get made, how the integration should look, both from a computer science and from an ethics perspective, has been incredibly valuable.
Beyond that, we had this tabletop exercise a month ago now, where we took our technology and some of the other technologies that were being built throughout the Institute and put them in front of emergency managers in a tabletop setting.
Even when the sky is clear, and there’s no disaster going on we can get some feedback as to how this technology could work. And again, where those pain points might exist.
All of that, just to be explicit, was made possible through AI-SDM. That’s the sort of stuff that comes out of this multidisciplinary Institute that we wouldn’t be able to get on our own.
AIVO: How do you work with their team to move your project forward?
Tom: It’s absolutely a group effort. AI-SDM has a huge collection of individuals with a great collection of skills.
That spans from our partners at Florida State University. David Merrick manages the drone response for the State of Florida and occasionally participates in drone responses elsewhere. Having that operational experience in-house has been invaluable. Beyond that, the participation from people like Drs. Terri Adams and Coty Gonzalez, two PIs in the Institute, and having their expertise both in decision-making and in emergency management in-house, again has been really helpful – both for technical reasons, but also just to bounce ideas and say, “Hey, are we barking up the right tree here?”
AIVO: How did you find out about the opportunity to have a project with AI-SDM?
Tom: That was entirely through my advisor, Dr. Robin Murphy. She’s one of the senior personnel working on the disaster thrust within AI-SDM, and she was there through the inception of the Institute, so it was a very easy transition. I’m going to say transition, but it was easy to fold the work that I knew I was going to be doing as a part of my thesis, and build it along the lines that we thought would interest AI-SDM.
AIVO: You’re studying machine learning and robotics for disaster response. What are some ways you plan to use your PhD after you graduate?
Tom: It really comes back to this translation from theory to practice. Through my work and through some of the efforts that we’ve been putting together in the community, things like the Artificial Intelligence for Humanitarian Assistance and Disaster Response Workshop (AI+HADR) – we’ve gotten a chance to see a lot of these projects that are going from theory to practice and sometimes practice to theory. That relationship is so crucial here because disaster response is such an applied discipline.
I think the value that I’m going to bring to this community with my PhD is really making sure from the technical side, that there are foundational advancements that will be helpful to the community.
The foundational advancements that we’re working on will be helpful to the community. But the thing that I really want to push on as I look forward to the end of my PhD and beyond is really nailing that translation from theory to practice and back, getting that virtuous cycle. I like to think that’ll be helpful.
In a more practical answer, I’m looking to stay in academia, and that PhD is somewhat of a prerequisite.
AIVO: I’m sure your future students will benefit greatly!
You mentioned drones going into the disaster areas, and that probably happened before you started your research. To what extent, if you can maybe give some rough numbers, did that save the lives of emergency responders?
Tom: The nice part about being at Texas A&M and being under AI-SDM is that my advisor, Dr. Robin Murphy, was the person who used drones for the first time in a major disaster – during Hurricane Katrina. So there’s a lineage in the lab that goes back, not just for example, to Hurricane Ian, which I deployed to, and a couple of other disasters where drones have been used – but all the way back.
There are a number of instances where a first responder’s life has been saved by drones. Hurricane Ian is a great example where drones were used to point towards not just people who needed rescuing, but also to do interior searches of houses, getting the drones into the building and flying them because we knew they were struck. Either the buildings were structurally unstable, or they couldn’t be accessed from the ground level because of all the damage caused by storm surge.
There is a great website hosted by DJI, that counts the number of lives saved in general by drones across the victim side: https://enterprise.dji.com/drone-rescue-map/.
Many thanks to Tom for talking about your research with us and sharing your experiences! We’re anticipating many accomplishments throughout your career. We wish you great success!
About AI-SDM
Using AI, the NSF AI institute for Societal Decision Making (NSF AI-SDM) is improving decision making to benefit society. Based at Carnegie Mellon University in Pittsburgh, PA, NSF AI-SDM brings together AI and social sciences researchers to develop human-centric AI for societal good that harnesses the power of data and an improved understanding of human decisions to create better and more trusted choices. Learn more on the their website: https://www.cmu.edu/ai-sdm/
