A consortium of German and Canadian researchers is working to refine the digital twin concept for warehousing.
The Kion Group and its partners LeddarTech, Karlsruhe Institute of Technology (KIT) and the STARS Lab at the University of Toronto have started a research project called Artificial Intelligence-Based Indoor Cartography (ARIBIC).
The idea for the research project originated in the summer of 2019 in the Technology & Innovation department of Kion Group. It is based on the idea that continuous data evaluation will make it possible to create a real-time digital twin of a warehouse or production environment.
Automated guided vehicles are already being used on a large scale in warehouses and production facilities. With sensor technology such as laser scanners and cameras, they find their way safely through racks, production lines, and warehouses. In the process, they generate a considerable amount of data about the environment in which they move.
However, this data is usually not yet systematically processed and remains a potential source of valuable information. The project is seeking to discover the full potential of these bits and bytes.
“The data collected by the sensors on the vehicles are used to create high-quality, high-resolution 3D maps of warehouses or production facilities. The objective is to create a digital twin of the environment, thus enabling relevant information to be displayed and shared in real time,” said Henry Puhl, chief technology officer of the Kion Group.
Creating a digital twin
Jonathan Kelly, head of the STARS Lab at University of Toronto, explained how it will work. The digital twin will be a continuously updated map of this highly dynamic world. “In the warehouse things are coming in and out, moving around. Lots of stuff happens all the time. Inventory tracking is really challenging,” he said. “But the more you know, the faster you can schedule things and get things out the door.”
To build the twin, KIT is creating some of the base level mapping software that lets you create a 3D reconstruction of the world. The STARS lab focuses on semantic mapping. It’s a buzzword in the robotics community, Kelly explained. “It means that you start with a map that contains maybe images, image data and geometry. And our job is to basically identify and label items like physical objects in the map. So, attach semantics, some meaning, to those objects.”
Kelly said the basic map would just show an object, like a pallet containing cartons of parts. But to catalogue inventory accurately, you need to know what’s in those boxes. “Our job is to take the raw map information and turn it into a product that says, ‘at these coordinates you’re looking at a pallet that is stacked with camshafts’, for example. It’s basically tagging everything in the world with an identifier that says what it is and where you would find it in inventory. So then, as a warehouse operator, you would query your digital twin and say, ‘Today we’re planning to ship 3,000 coolant hoses in boxes. Where are they in the warehouse?’”
Kelly added that the process would be completely automated beyond an operator entering the query, and “via the magic of this mapping framework, immediately all the instances in the warehouse of where those parts are, at that given time, would be displayed.”
The system would also have information about paths within the warehouse and access to the goods being sought, making it possible to simulate routing for automated vehicles tasked with fetching pallets.
Sensing specialist
LeddarTech, a Quebec-based specialist in environmental sensing solutions for autonomous vehicles and advanced driver assistance systems, is also working on the project. It will be working primarily on the sensor system, which employs the company’s expertise in sensing, perception, and sensor fusion for mobility applications.
“When Kion approached us and explained their vision for ARIBIC, it was clear from the beginning that we had to be part of this project,” said LeddarTech’s chief technology officer, Pierre Olivier.
“Not only does it allow us to collaborate with an industry leader as well as with two renowned labs, it also represents a perfect opportunity to leverage LeddarTech’s strong expertise in sensing, in integrating sensing platforms on vehicles, and in maximizing the potential from the available sensor data.”
The data generated by the sensor system is sent to the vehicle, processed there, and sent to the ARIBIC cloud platform. The sensor data is then processed further in the cloud.
Cooperation between industry and science
The Kion Group is hoping that the project will enhance its existing digital twin technology. “ARIBIC provides important progress in adding computational perception capabilities. Those leverage edge intelligence and open the door to many applications ranging from more efficient design of flexible automation and mobile robotic operations for the warehouse of tomorrow to inspection and detection of warehouse material placement and distribution that is critical to workflow optimization for many logistics operations,” said Hamid Montazeri, senior vice-president, software and digital solutions development at Kion subsidiary Dematic.
Kelly, speaking for the STARS lab, is enthused about the project’s potential to unlock efficiency for warehouse operations. “ARIBIC is a perfect project for us to collaborate on current research topics with international partners and to establish new industry relationships,” he said. “We’re really excited to be involved. This is the first project that my lab has done specifically in warehousing and logistics…There are huge opportunities just to make it much more efficient with technology.”
The project was approved at the beginning of 2021 by the National Research Council of Canada Industrial Research Assistance Program (NRC IRAP) and the German Federal Ministry for Economic Affairs and Energy (BMWi), which are also providing funding. The project is scheduled for completion in the fourth quarter of 2023.
Leave a Reply