Researchers at Massachusetts Institute of Technology (MIT) have developed an AI system that learns to keep warehouse robot traffic running smoothly.

This approach adapts to decide which robots should get the right of way at every moment, avoiding congestion and increasing throughput.

Inside giant autonomous warehouses, hundreds of robots dart down aisles as they collect and distribute items to fulfil a steady stream of customer orders. In this busy environment, even small traffic jams or minor collisions can snowball into massive slowdowns.

To avoid such an avalanche of inefficiencies, researchers from MIT and the tech firm Symbotic developed a method that automatically keeps a fleet of robots moving smoothly. Their method learns which robots should go first at each moment, based on how congestion is forming, and adapts to prioritise robots that are about to get stuck. In this way, the system can reroute robots in advance to avoid bottlenecks.

The hybrid system uses deep reinforcement learning, an AI method for solving complex problems, to figure out which robots should be prioritised. Then, a fast and reliable planning algorithm feeds instructions to the robots, enabling them to respond rapidly in constantly changing conditions.

In simulations inspired by actual ecommerce warehouse layouts, this approach achieved about a 25 per cent gain in throughput over other methods. Importantly, the system can quickly adapt to new environments with different quantities of robots or varied warehouse layouts.

“There are a lot of decision-making problems in manufacturing and logistics where companies rely on algorithms designed by human experts,” said Han Zheng, a graduate student at MIT and lead author of a paper on this approach. “But we have shown that, with the power of deep reinforcement learning, we can achieve super-human performance. This is a very promising approach, because in these giant warehouses even a two or three per cent increase in throughput can have a huge impact.”

The research has been published in the Journal of Artificial Intelligence Research (jair.org/index.php/jair/article/view/20611).

Coordinating hundreds of robots in an ecommerce warehouse simultaneously is no easy task. The problem is especially complicated because the warehouse is a dynamic environment, and robots continually receive new tasks after reaching their goals. They need to be rapidly redirected as they leave and enter the warehouse floor. Companies often leverage algorithms written by human experts to determine where and when robots should move to increase the number of packages they can handle. But if there is congestion or a collision, a firm may have to shut down the entire warehouse for hours to sort the problem out manually.

“In this setting, we don’t have an exact prediction of the future,” said Zheng. “We only know what the future might hold, in terms of the packages that come in or the distribution of future orders. The planning system needs to be adaptive to these changes as the warehouse operations go on.”

The MIT researchers achieved this adaptability using machine learning. They began by designing a neural network model to take observations of the warehouse environment and decide how to prioritise the robots. They trained this model using deep reinforcement learning, a trial-and-error method in which the model learns to control robots in simulations that mimic actual warehouses. The model is rewarded for making decisions that increase overall throughput while avoiding conflicts. Over time, the neural network learns to coordinate many robots efficiently.

“By interacting with simulations inspired by real warehouse layouts, our system receives feedback that we use to make its decision-making more intelligent,” Zheng said. “The trained neural network can then adapt to warehouses with different layouts.”

It is designed to capture the long-term constraints and obstacles in each robot’s path, while also considering dynamic interactions between robots as they move through the warehouse. By predicting current and future robot interactions, the model plans to avoid congestion before it happens.

After the neural network decides which robots should receive priority, the system employs a tried-and-true planning algorithm to tell each robot how to move from one point to another. This algorithm helps the robots react quickly in the changing warehouse environment.

This hybrid approach builds on work on how to achieve the best of both worlds between machine learning and classical optimisation methods. Pure machine-learning methods still struggle to solve complex optimisation problems, and it is extremely time- and labour-intensive for human experts to design effective methods. But together, using expert-designed methods this can simplify the machine-learning task.

Once the researchers trained the neural network, they tested the system in simulated warehouses that were different than those it had seen during training. Since industrial simulations were too inefficient for this complex problem, the researchers designed their own environments to mimic what happens in actual warehouses.

On average, their hybrid learning-based approach achieved 25 per cent greater throughput than traditional algorithms as well as a random search method, in terms of number of packages delivered per robot. The approach also generated feasible robot path plans that overcame congestion caused by traditional methods.

“Especially when the density of robots in the warehouse goes up, the complexity scales exponentially, and these traditional methods quickly start to break down,” Zheng said. “In these environments, our method is much more efficient.”

In the future, the researchers want to include task assignments in the problem formulation, since determining which robot will complete each task impacts congestion. They also plan to scale up their system to larger warehouses with thousands of robots.

This MIT (www.mit.edu) research was funded by Symbotic (www.symbotic.com).