Artificial intelligence systems based on neural networks (such as ChatGPT, Claude, DeepSeek or Gemini) are extraordinarily powerful. But its inner workings remain largely a mystery. A group of physicists from Harvard University wants to unravel the black box of AI. To do this, they developed a simplified mathematical model of learning in neural networks. It can be analyzed mathematically using the tools of statistical physics.
It is a controlled theoretical laboratory to investigate the fundamental mechanisms of neural networks. Understanding these systems could contribute to the design of more efficient and reliable artificial intelligence systems. It is something similar to when Kepler described the laws that govern the movement of the planets. "Newton's laws of gravity were discovered by first identifying scaling laws between the orbital periods of the planets and their radii." It is explained by Alexander Atanasov, a theoretical physics student at Harvard.
Neural networks
Kepler formulated his laws by observing planetary motion, without fully understanding the mechanisms that explain it. However, that work proved crucial. It later allowed Newton to discover gravity. That led to a much deeper understanding of the universe.
There are several empirical laws that describe the behavior of neural networks. But we lack some kind of 'gravity theory' to explain why they behave that way.
Deep learning models are not handwritten algorithms like a set of rules. They are not designed manually. They look much more like an organism grown in a laboratory. Generative AI chatbots are based on neural networks. It is a technology that, in a very distant way, resembles the functioning of a biological brain. They are made up of numerous small processing units. They are artificial neurons connected to each other in a complex network. As the number of components increases, the complexity increases rapidly.
Understanding chaos
Systems like ChatGPT operate in spaces with thousands or even millions of variables. It makes an exact mathematical analysis extremely difficult. Ideas from statistical physics are useful here. In very high-dimensional data, small random variations, known as statistical fluctuations, appear naturally. Very complex systems can exhibit relatively simple large-scale behavior.
The mathematical model created could have another use. It would be a starting point to understand how learning behaves in very high-dimensional systems. You can now identify which aspects of learning are generic and which depend on the details of a specific model. The black box of AI has begun to be understood. What comes next could be even more surprising.
Leave a Reply