Dr. Daniel Asmar, associate professor of mechanical engineering and Dr. Imad Elhajj, associate dean and professor of electrical and computer engineering at the Maroun Semaan Faculty of Engineering and Architecture (MSFEA) are part of a multi-institute team who have been awarded a Horizon Europe Grant worth almost €6 million (AUB's share €462,000) for their innovative Didymos project.
The project includes 14 partners from across Europe with groups such as IDEALworks (the robotics arm of BMW), Unity (the gaming engine and the world's leading platform for real-time content creation), and the Technical University of Berlin.
This Didymos (Greek word for twins) project aims to create digital twins and use them as high-fidelity models for long-term localization, which in turn serves to create more realistic XR applications in the wild.
The MSFEA team will be building the localization and mapping system for Didymos, based on the technology they have developed at the Vision and Robotics Lab (VRL) during the past decade. This project will kick off in January 2023.
What is a digital twin and why is it important?
The digital transformation and the availability of more diversified and cost-effective means for 3D capture have led to the development of Digital Twins, which are digital representations of sections of our public spaces, machinery, and processes.
Digital twins are noteworthy for various reasons, including analysis, storage, and preservation. Another major and modern application for digital twins is in Augmented, Virtual, and Mixed Reality (together referred to as eXtended reality or XR), where the XR system projects digital twins of the user's desired objects into a virtual world.
Accurate localization is a must for the successful deployment of these twins, so that the items are projected in the correct location within the virtual world (in sync with their real twins).
As the size of the virtual world expands, the localization issue becomes considerably more challenging (sometimes to the scale of neighborhoods or cities).
Didymos puts MSFEA in the forefront for AI and XR
The objective of DIDYMOS-XR is to enable more advanced, realistic, and dynamic extended reality (XR) applications that are powered by artificial intelligence. Thus, the project focuses on developing technology for producing synchronized, large-scale digital twins.
DIDYMOS-XR will conduct research and development on methods for reconstruction and mapping from diverse inputs, such as static and mobile sensors, AI-based data fusion, scene interpretation, and rendering. Digital twin-based XR applications also require precise positioning in the environment, aware of the scene's dynamics, which the project also addresses.
MSFEA's team has worked on Simultaneous Localization and Mapping (SLAM) for long time, be it from a localization or mapping standpoint. They have introduced two novel ways for SLAM, which distinguishes the Didymos systems from any other:
- A human-assisted SLAM, which permits a person to reduce errors in any SLAM system (which are unavoidable) by correcting mapping and localization issues on the fly.
- A hybrid SLAM system that was introduced this year, utilizing both direct and indirect SLAM in a complementary manner, resulting in a faster and more precise SLAM system. This feature has made Vision and Robotics Lab (VRL) appealing to teams seeking to apply localization and mapping solutions to their problems.