Recently, Chinese researchers from Peking University and the Eastern Institute of Technology (EIT) have unveiled a groundbreaking framework designed to train machine learning models with prior knowledge, a departure from the conventional data-only approach. The researchers highlight the transformative impact of deep learning models on scientific research, emphasizing their ability to extract meaningful relationships from vast datasets. The framework addresses the limitations of existing models, such as OpenAI’s Sora, which struggles to simulate certain real-world interactions accurately due to a lack of understanding of physical laws like gravity.
Deep learning models, renowned for revolutionizing scientific research, primarily rely on extensive data for training rather than incorporating prior knowledge such as physical laws or mathematical logic. However, researchers from Peking University and the Eastern Institute of Technology propose a paradigm shift. The team suggests combining data with prior knowledge during training could result in more accurate and informed machine-learning models.
The challenge lies in determining which aspects of prior knowledge, including functional relationships, equations, and logic, should be integrated to facilitate better pre-learning without causing model collapse. Professor Chen Yuntian emphasizes that inundating models with a high volume of knowledge and rules often leads to struggles or failures in current informed machine learning models.
A framework for assessing rule importance
To overcome this challenge, the researchers developed a framework that evaluates the value of rules and determines optimal combinations that enhance the predictive capability of deep learning models. Xu Hao, the first author and a researcher at Peking University, explains that their framework calculates “rule importance” by analyzing how specific rules or combinations affect the predictive accuracy of a model.
This approach aims to balance the influence of data and knowledge, enhancing deep learning models’ efficiency and inference capabilities. The team envisions that embedding human knowledge into AI models could significantly improve their real-world reflectiveness, making them more applicable in scientific and engineering domains.
The researchers tested their framework by optimizing a model for solving multivariate equations and another for predicting the outcomes of a chemistry experiment. Chen suggests that, in the short term, this framework will find its most useful applications in scientific models where consistency with physics rules is vital to avoid potential adverse consequences.
Despite the promising results, the team acknowledges a challenge. As more data is added to a model, general rules become more significant than specific local rules. However, this observation does not provide significant benefits in fields like biology and chemistry, where general rules are often lacking.
Towards autonomous AI scientists
Looking ahead, the research team aspires to take its framework a step further, enabling AI to identify its knowledge and rules directly from data without human intervention. The ultimate goal is to create a closed loop, transforming the model into a genuine AI scientist. Chen envisions this development as a significant step towards autonomy in AI, and the team is actively working on an open-source plugin tool for AI developers to facilitate this transition.
