Introduction

In a warehouse it is essential to keep track of where things are.

Usually for this purpose people use a warehouse management system (WMS) which is basically a big database. In this database are stored among other things the location of goods and all their movement history (i.e. when they entered the warehouse, where they were stored, etc.)

Being able to precisely locate the vehicles moving around in the warehouse (such as forklift trucks, small trucks, etc.) would be very beneficial for the warehouse managers as it allows them to consolidate their data and automatize some of the operator’s tasks.

In this small project, I am interested in localizing the vehicle in a fairly large warehouse: some 50 m x 60 m and height 14m. If we were outside we could use GPS for localizing but unfortunately we know that GPS signal is not available indoors.  Some systems available on the market use lasers. An attractive solution consists of using a camera embarked on the vehicle.

• cameras are cheap and low power
• cameras capture rich information : we don’t need to add beacons, or other infrastructure.

A fixed external camera could also be envisaged for tracking the vehicle but it is not as attractive because typically shelves will obstruct the camera view and the vehicle will only be visible occasionally. Hence localization is less effective in that case.

Typical warehouse with shelves and alleys between the shelves (c) jczyz.net

 Localizing with a camera embarked on the vehicle

How can we localize in an environment using a camera ?

One way to figure out global position is to use a visual map: a map with images associated with locations. Then, by matching the image we are getting from the camera to the visual map, we can determine where we are.

Some places will inevitably be ambiguous: they don’t contain enough information to uniquely determine where the camera is. Imagine your camera is seeing a long white wall. You cannot tell where you are along the wall using an image because the wall looks the same everywhere. However using the sequence of images coming from the camera, we can tell how much we have moved from a known location. In that case we say that we do tracking of our position (or odometry in the robotics jargon) as opposed to determine the global position.

Using ceiling images

In a warehouse everything is changing all the time. Goods come out and new goods come in to replace them. As we want to find visual features or keypoints and relate them to our current view, we need to find features that remain the same.  As everything changes this can be challenging.

Pointing the camera towards the ceiling has a few advantages

• the ceiling in not changing
• the ceiling of a warehouse has many features
• the ceiling is always at a more or less same distance to the camera (it is easier to deal with rapid rotation of the vehicle and motion blur)

However when looking at the ceiling we need to deal with difficult lighting conditions.

Results

In this project, I fixed a stereo camera on a fork lift truck connected to a small computer with some storage and WIFI. The forklift then operated normally.

I used a stereo camera because stereo cameras provide 3D positions which simplifies the positioning  problem.

A important part of the job was to create the visual map. I will explain that in another post.

Below is a video showing the forklift position in the warehouse. The forklift is represented as a small orange icon. The embedded image shows the ceiling images that are used for localization. The reddish dots are the walls and shelves of the warehouse.

The visual map can be registered with the warehouse map and therefore report to the warehouse management system the location of the vehicle. Below is a video showing the location of the forklift on the warehouse map (blue rectangles are the shelves locations)

welcome