In a remarkable leap forward, Fujitsu Limited and Carnegie Mellon University jointly announced the development of an AI-powered Social Digital Twin technology, aimed at revolutionizing traffic analysis. The collaborative research, initiated in February 2022, has culminated in a transformative system capable of converting 2D scene images captured by a single monocular RGB camera into detailed 3D models. This innovative approach, relying on deep learning, marks a significant breakthrough in high-precision visualization and analysis of dynamic 3D traffic scenarios.
Fujitsu and Carnegie Mellon redefine traffic analysis with AI integration
Fujitsu Limited and Carnegie Mellon University have unveiled a groundbreaking technology that promises to reshape the landscape of traffic analysis and accident prevention. Commencing their joint research in February 2022, the collaboration aimed to develop a Social Digital Twin capable of dynamically replicating complex interactions within 3D spaces. The culmination of this research is an AI-powered system that transforms 2D scene images, captured by a monocular RGB camera, into digitalized 3D formats.
The system relies on two core technologies, each playing a pivotal role in its functionality. The first, 3D Occupancy Estimation Technology, employs deep learning networks to distinguish objects within 3D spaces from monocular RGB camera images. By representing these objects as Voxels in 3D space, the system achieves a detailed understanding of scenes, allowing for accurate 3D shape estimation of areas not visible in the input image. The second core technology, 3D Projection, builds a 3D digital twin based on the output of the Occupancy Estimation Technology. This step incorporates human behavior analysis know-how, ensuring movements are consistent with the real world and enabling precise position estimation even when parts of objects are obstructed.
To address concerns related to privacy, the technology automatically anonymizes faces and license plates, underscoring the commitment to responsible AI use. Fujitsu and Carnegie Mellon envision the commercialization of this technology by the fiscal year 2025, expanding its application beyond transportation to encompass smart cities and traffic safety.
Field trials in Pittsburgh – Validating the AI-powered technology’s potential
With the technology’s promising capabilities, Fujitsu and Carnegie Mellon initiated field trials in Pittsburgh, USA, starting on February 22, 2024, and running until May 31, 2024. The trials utilized data from intersections in the city, leveraging a monocular RGB camera installed on the campus of Carnegie Mellon University. The primary goal was to analyze crowd and traffic conditions, identifying potential accidents such as blind spots caused by buildings and temporary crowds. By reproducing this data on a Social Digital Twin, the effectiveness of the technology in real-world scenarios was thoroughly examined.
Assistant Research Professor, Prof. László A. Jeni, expressed his delight at the collaborative achievement, highlighting the ongoing commitment to advancing research on cutting-edge technologies. The collaborative effort between Fujitsu’s team and the academic experts at CMU played a pivotal role in reaching this milestone.
Daiki Masumoto, Fellow and Head of the Converging Technologies Laboratory at Fujitsu Research, emphasized the technology’s alignment with Fujitsu’s broader mission to make the world more sustainable through innovation. He expressed his thrill at the significant step towards achieving their goals through collaboration with Carnegie Mellon University.
As the field trials unfold, the implications of this AI-powered Social Digital Twin technology on traffic analysis and urban planning are becoming increasingly evident. The potential for this innovation to enhance safety, optimize traffic flow, and contribute to the development of smart cities is substantial. The question remains: How will this groundbreaking technology redefine our approach to urban planning, transportation, and societal challenges in the years to come?
