How many times have we heard of natural disasters that took the lives or homes of unfortunate people and wreaked havoc both physically and economically? Some of us may have even experienced them first-hand. Natural disasters are almost inevitable occurrences happening over 380 times every year, resulting in losses upwards of $150 billion. One major problem lies in the lack of precautionary warnings. All the while, with the rise of social media as means for instant news sharing, surges of images and help requests fill the internet, overwhelming rescue teams with information and impeding their ability to timely identify and utilize online information.
Researchers are tackling this issue through innovative approaches using machine learning, a subfield of artificial intelligence (AI), where systems can learn and improve simply from experience without human intervention. In a recent study developed by researchers at the
Maroun Semaan Faculty of Engineering and Architecture (MSFEA) of the
American University of Beirut (AUB), a new framework was proposed that can automatically analyze crisis data from Twitter images and texts and classify them based on graphics and semantics with the help of machine learning. This is expected to relieve the rescue teams and dedicated personnel from the wearying process of identifying the relevant data, and thus, result in a quicker response, improved management of resources, and proper prioritization.
What
Mariette Awad, associate professor at MSFEA, and her research team composed of Yara Rizk and Hadi S. Jomaa put forward in this study was an AI-based system that can go through crisis-related tweets, extract the images and texts separately, and generate relative feature groups. These included simple visual aspects such as shape, color, and texture, and linguistic elements that show the meaning behind the written words. These features allowed the framework to train on how to classify the information into two different sections: built-infrastructure damage, which included ruins of the built environment, and nature damage, which described natural environmental losses. The results showed a high level of accuracy in understanding the content of the tweets, especially when both visual and semantic features were combined, exceeding the 92 percent mark.
As an application in humanitarian computing, Awad and team were able to investigate their research question of how to make good use of image processing and natural language processing (NLP), creating a smart instant data classifier that can aid first responders in evaluating damage and reacting effectively. Indeed, future developments will improve upon this by enhancing the machine learning process and data understanding. Evidently, social media can be monumentally impactful when it comes to events such as natural disasters, and implementing relevant data science methodologies can bring up, as the researchers quoted, the traditional statement: “An annotated image is worth a thousand words."