Credit...Video by Robert Rieger

A Warehouse Robot Learns to Sort Out the Tricky Stuff

At a facility near Berlin, a new kind of robot is automating tasks that until recently had been out of the reach of machines.

LUDWIGSFELDE, Germany — Inside a warehouse on the outskirts of Berlin, a long line of blue crates moved down a conveyor belt, carrying light switches, sockets and other electrical parts. As they came to a stop, five workers picked through the small items, placing each one in a cardboard box.

At Obeta, an electrical parts company that opened in 1901, it is the kind of monotonous task workers have performed for years.

But several months ago, a new worker joined the team. Stationed behind protective glass, a robot using three suction cups at the end of its long arm does the same job, sifting through parts with surprising speed and accuracy.

While it may not seem like much, this component-sorting robot is a major advance in artificial intelligence and the ability of machines to perform human labor.

As millions of products move through warehouses run by Amazon, Walmart and other retailers, low-wage workers must comb through bin after bin of random stuff — from clothes and shoes to electronic equipment — so that each item can be packaged and sent on its way. Machines had not really been up to the task, until now.

Image
Over 80,000 blue bins are stored in the Obeta warehouse outside Berlin.Credit...Robert Rieger for The New York Times

“I’ve worked in the logistics industry for more than 16 years and I’ve never seen anything like this,” said Peter Puchwein, vice president of Knapp, an Austrian company that provides automation technology for warehouses.

Standing nearby at the Obeta warehouse, the California engineers who made the robot snapped pictures with their smartphones. They spent more than two years designing the system at a start-up called Covariant.AI, building on their research at the University of California, Berkeley.

Their technology is an indication that, in the coming years, few warehouse tasks will be too small or complex for a robot. And as the machines master tasks traditionally handled by humans, their development raises new concerns about warehouse workers losing their jobs to automation.

Because the online retail business is growing so quickly — and most companies will be slow to adopt the latest robotic technologies — economists believe the advances will not cut into the overall number of logistics jobs anytime soon. But the engineers building these technologies admit that at some point most warehouse tasks will be done by machines. Human workers will need to find other things to do.

The engineers at Covariant specialize in a branch of artificial intelligence called reinforcement learning. The machines are wired to learn new tasks on their own through extreme trial and error. And the best place to learn is in the real world.

“If you want to advance artificial intelligence, you don’t just do it in a lab,” said Peter Chen, Covariant’s chief executive and co-founder. “There is a huge gap in bringing it to the real world.”

Video
A factory worker working alongside the Covariant robot. Currently, the robot is the only automated station in the packaging hall.CreditCredit...Video by Robert Rieger

Warehouses are already highly automated. At the facility outside Berlin, inside a fenced-off room larger than a football field, other robots have long been used to fetch large boxes from shelves several stories high.

But that is a relatively easy task for a machine. Engineers can program a robot to perform the same motion over and over again. The boxes are uniform. A robot can pick them up with the same motion every time.

Picking through a bin of random items is different. Shapes vary, as do surfaces. One light switch might be upside down, the other right-side up. The next electrical gadget might be in a plastic bag that reflects light in ways a robot has never seen. A human touch has been needed.

Programming a robotic arm to deal with every situation, one rule at a time, is impossible. At Knapp, Mr. Puchwein and his partners had tried and failed for years to create a robot with the dexterity and flexibility needed for the job.

Image
Peter Puchwein’s company, Knapp, had tried for years to create a sorting robot.Credit...Robert Rieger for The New York Times

Covariant, which is working with Knapp, built software that could learn through trial and error. First, the system learned from a digital simulation of the task — a virtual recreation of a bin filled with random items. Then, when Mr. Chen and his colleagues transferred this software to a robot, it could pick up items in the real world.

The robot could continue to learn as it sorted through items it had never seen before. Inside the German warehouse, the robot can pick and sort more than 10,000 different items, and it does this with more than 99 percent accuracy, according to Covariant.

This represents a significant change for the online retail and logistics industries.

Late last year, the international robot maker ABB ran a contest. It invited 20 companies to design software for its robot arms that could sort through bins of random items, from cubes to plastic bags filled with other objects.

Video
Robots on tracks searching for the right tote in the stock to send it to the package station.CreditCredit...Video by Robert Rieger

Ten of the companies were based in Europe, and the other half were in the United States. Most came nowhere close to passing the test. A few could handle most tasks but failed on the trickier cases. Covariant was the only company that could handle every task as swiftly and efficiently as a human.

“We were trying to find weaknesses,” said Marc Segura, managing director of service robotics at ABB. “It is easy to reach a certain level on these tests, but it is super difficult not to show any weaknesses.”

Knapp, which helped deploy the system outside Berlin, and ABB believe this technology can be used in similar warehouses.

Covariant engineers believe their robots will improve with practice. As a robot in one warehouse learns better ways for picking up certain items, the information feeds back to what is essentially a central brain run by Covariant that will help operate machines.

Dirk Jandura, the managing director of Obeta, said companies like his were under extreme pressure to be more efficient. Automation is a key way to keep costs low.

Image
Dirk Jandura, managing director of Obeta, at the company’s warehouse near Berlin.Credit...Robert Rieger for The New York Times

Like many warehouse operators, Obeta has trouble finding workers willing to do the monotonous work. Each picker handles about 170 orders an hour, or about three per minute, over an eight-hour day. In the summer, temperatures in the warehouse reach more than 100 degrees. It is hard to keep employees for longer than six months.

For Obeta, the new robot is an ideal solution. A job that requires three humans is done by one tireless robot.

“It doesn’t smoke, is always in good health, isn’t chatting with its neighbors, no toilet breaks,” Mr. Jandura said. “It’s more efficient.”

Knapp is also considering the design of warehouses staffed by robots rather than humans that would allow for packages to be more densely packed into spaces and retrieved by robots trained to perform the task.

“The new warehouses will be built around A.I. robots and not humans,” Mr. Puchwein said.

Knapp plans to make it hard for companies to say no to replacing human workers with robots. Mr. Puchwein said they would charge a fee that was always lower than what a company would pay a human. If a company paid $40,000 per year to a worker, Knapp would charge about $30,000, he said.

“We just go lower,” he said. “That’s basically the business model. For the customer, it’s not very hard to decide.”

Image
Peter Chen and Pieter Abbeel, founders of Covariant.AI.Credit...Robert Rieger for The New York Times

Beth Gutelius, associate director of the Center for Urban Economic Development at the University of Illinois at Chicago, who has studied the impact of automation on work, said this kind of technology was unlikely to shift the job market any time soon.

The greater problem, she said, is that as humans work alongside robots, they will be judged in new ways. “As we start to compare the speed and efficiency of humans to robots, there is a whole new set of health and safety issues that emerge,” she said.

Pieter Abbeel, a Berkeley professor who is a co-founder of Covariant as well as its president and chief scientist, said humans would continue to work alongside machines in these kinds of warehouses. But he acknowledged that the job market would significantly shift as machine learning improved.

Image
Trucks being loaded at the Obeta warehouse lot in Germany.Credit...Robert Rieger for The New York Times

“If this happens 50 years from now, there is plenty of time for the educational system to catch up to the job market,” he said.

At the German warehouse, a woman in a baggy T-shirt diligently sorted through the boxes, occasionally looking up at the English-speaking visitors who were taking pictures of the robot and were marveling at its effectiveness.

A Covariant engineer walked over to the group to share that the robot had filled more than 200 orders in the past hour, enough to receive a bonus if it were a human.

Adam Satariano reported from Ludwigsfelde, and Cade Metz from San Francisco.

Adam Satariano is a technology reporter based in London. More about Adam Satariano

Cade Metz is a technology correspondent, covering artificial intelligence, driverless cars, robotics, virtual reality, and other emerging areas. He previously wrote for Wired magazine.  More about Cade Metz

A version of this article appears in print on  , Section B, Page 1 of the New York edition with the headline: Little Is Out of Reach For This Robotic Arm. Order Reprints | Today’s Paper | Subscribe

Advertisement

SKIP ADVERTISEMENT