American University of Beirut

How machine learning and social media can help first responders fight natural disasters

​​​​How many times have we heard of natural disasters that took the lives or homes of unfortunate people and wreaked havoc both physically and economically? Some of us may have even experienced them first-hand. Natural disasters are almost inevitable occurrences happening over 380 times every year, resulting in losses upwards of $150 billion. One major problem lies in the lack of precautionary warnings. All the while, with the rise of social media as means for instant news sharing, surges of images and help requests fill the internet, overwhelming rescue teams with information and impeding their ability to timely identify and utilize online information.

Researchers are tackling this issue through innovative approaches using machine learning, a subfield of artificial intelligence (AI), where systems can learn and improve simply from experience without human intervention. In a recent study developed by researchers at the Maroun Semaan Faculty of Engineering and Architecture (MSFEA) of the American University of Beirut (AUB), a new framework was proposed that can automatically analyze crisis data from Twitter images and texts and classify them based on graphics and semantics with the help of machine learning. This is expected to relieve the rescue teams and dedicated personnel from the wearying process of identifying the relevant data, and thus, result in a quicker response, improved management of resources, and proper prioritization. 

What Mariette Awad,​ associate professor at MSFEA, and her research team composed of Yara Rizk and Hadi S. Jomaa put forward in this study was an AI-based system that can go through crisis-related tweets, extract the images and texts separately, and generate relative feature groups. These included simple visual aspects such as shape, color, and texture, and linguistic elements that show the meaning behind the written words. These features allowed the framework to train on how to classify the information into two different sections: built-infrastructure damage, which included ruins of the built environment, and nature damage, which described natural environmental losses. The results showed a high level of accuracy in understanding the content of the tweets, especially when both visual and semantic features were combined, exceeding the 92 percent mark.

As an application in humanitarian computing, Awad and team were able to investigate their research question of how to make good use of image processing and natural language processing (NLP), creating a smart instant data classifier that can aid first responders in evaluating damage and reacting effectively. Indeed, future developments will improve upon this by enhancing the machine learning process and data understanding. Evidently, social media can be monumentally impactful when it comes to events such as natural disasters, and implementing relevant data science methodologies can bring up, as the researchers quoted, the traditional statement: “An annotated image is worth a thousand words."​

This article is based on: Rizk, Y. et al. (2019) “A Computationally Efficient Multi-modal Classification Approach of Disaster-related Twitter Images”, In The 34th ACM/SIGAPP Symposium on Applied Computing (SAC’19), Cyprus. DOI: 10.1145/3297280.3297481.

Other research in this area,​ by the same team, such as: "Damage Identification in Social Media Posts using Multimodal Deep Learning." In the ISCRAM 2018 Conference Proceedings – 15th International Conference on Information Systems for Crisis Response and Management.

Contact Us

For various questions, please try contacting us via social media first!
read more

Privacy Statement

We take data privacy seriously and adhere to all applicable data privacy laws and regulations.
read more

Copyright and Disclaimer

Written permission is needed to copy or disseminate all or part of the materials on the AUB website.
read more

Title IX, Non-Discrimination, and Anti-Discriminatory Harassment

AUB is committed to providing a safe, respectful, and inclusive environment to all members of its community.
read more