In a groundbreaking development, a group of researchers led by DPhil student Vít Růžička from the Department of Computer Science at the University of Oxford successfully trained a machine learning model in outer space. This milestone achievement took place on board the ION SCV004 satellite, launched in January 2022, as part of the Dashing through the Stars mission. The team’s innovation, named RaVAEn, utilized a few-shot learning approach and could enable real-time monitoring and decision-making capabilities for remote-sensing satellites. The implications of this breakthrough are significant, potentially transforming applications such as aerial mapping, weather prediction, and deforestation monitoring.
Challenges in traditional satellite data collection
Currently, most remote-sensing satellites are limited to passively collecting data and lack the capability to make decisions or detect changes. Consequently, data needs to be transmitted to Earth for processing, resulting in substantial delays ranging from several hours to days. This limitation hampers the ability to promptly respond to rapidly emerging events, such as natural disasters, which require swift action for effective management.
The idea and mission
The team’s ambition to overcome these restrictions led them to propose the idea of training a machine learning model directly onboard a satellite. Their proposal was accepted as part of the Dashing through the Stars mission, granting them the opportunity to experiment with the ION SCV004 satellite in outer space. In the autumn of 2022, the team uplinked the code for the RaVAEn model to the satellite, marking the initiation of their extraordinary endeavor.
Training the machine learning model in outer space
The researchers trained a simple yet powerful model to detect changes in cloud cover from aerial images directly on the satellite itself, diverging from the traditional approach of training models on the ground. The model was based on few-shot learning, allowing it to discern critical features with minimal training samples. The ability to compress data into smaller representations made the RaVAEn model faster and more efficient.
Vít Růžička explained the process: RaVAEn compresses large image files into 128-number vectors during the training phase. The model learns to retain only the informative values in this vector, specifically those related to the change it is trying to detect, such as the presence of clouds. This streamlined approach resulted in remarkably fast training times, a key advantage in the context of space missions.
Unprecedented speed and performance
While a typical machine learning model would require multiple rounds of training using a cluster of linked computers, the RaVAEn model accomplished its training phase using over 1300 images in just around one and a half seconds. The model’s performance was equally impressive when tested on novel data, with the ability to automatically detect the presence of clouds in around a tenth of a second for an area equivalent to about 4.8×4.8 km2.
Versatility and future applications
One of the most promising aspects of the RaVAEn model is its adaptability to diverse tasks and data types. The researchers believe that the model could be customized to differentiate between changes of interest, such as flooding, fires, and deforestation, and natural changes, like seasonal variations in leaf color. Moreover, the model’s potential expansion to handle more complex data, including hyperspectral satellite images, holds the key to detecting critical phenomena such as methane leaks, an essential capability in combating climate change.
Addressing sensor calibration challenges
In addition to its real-time monitoring capabilities, performing machine learning on satellites in outer space could offer solutions to the issue of sensor calibration in harsh environmental conditions. The proposed system could be utilized in constellations of non-homogeneous satellites, where reliable information from one satellite can be applied to train the others. This feature would aid in recalibrating sensors that have degraded over time or have experienced rapid changes in their operating environment.
Professor Andrew Markham’s perspective
Professor Andrew Markham, who supervised Vít Růžička’s DPhil research, emphasized the enormous potential of machine learning for remote sensing. By enabling space-based sensing to become increasingly autonomous, the inherent delays between data acquisition and action could be overcome, allowing satellites to learn from data onboard and take more informed and timely decisions. Vít’s groundbreaking work serves as a compelling proof-of-principle, paving the way for exciting future advancements in the field.
The successful training of a machine learning model on a satellite in outer space represents a remarkable achievement in the field of remote sensing. Led by the University of Oxford, the researchers’ groundbreaking project, RaVAEn, has the potential to revolutionize the capabilities of remote-sensing satellites by enabling real-time monitoring and decision-making for a wide range of applications. With the ability to autonomously detect changes and analyze complex data, this innovation holds promise in transforming how we observe, respond to, and manage our planet, ultimately contributing to a more sustainable and efficient future.
