ArduEye odometry example using Tam2 (16x16) chip and Rox1 board

OVERVIEW:

This is a quick start set of instructions to demonstrate simple odometry using an "ArduEye" and a Tam2 (16x16) chip. Note that in order to view images or sense optical flow, you need a lens on or above the Tam chip! Note that you need to provide the 5V Arduino board.

We will be updating this page as people ask more questions.

Pricing/Ordering: Refer to the products page on Centeye for pricing / ordering information. Look for the "ArduEye Shields".

FILES:

Below are the first release files for the Arduino:

Eagle CAD files for the Rox1 board: ArduEye_Rox1_v1.zip

Arduino script (compile Tam2_OdometryExample_r1): Tam2_OdometryExample_r1.zip

Tam series chip documentation (including chip schematic diagram!): Tam_Chip_Instructions_2011-02-28.pdf

Bonding diagram between Tam chip and Rox1 board: BondingDiagram_Rox1_Tam.pdf

HARDWARE INSTRUCTONS (FULL SIZED ARDUINO):

You will need to break apart the 5-pin header into three parts and solder them to the ArduEye Rox1 board. Please visit this forum post for instructions.

SOFTWARE INSTRUCTIONS:

0) Preparation:

First of all, since this posting deals with Arduinos, it assumes that you are familiar with both the Arduino hardware and the Arduino development environment. It is an easy platform to learn, so you should very easily master the preliminary material needed for this post. You should already have an Arduino board working with your computer, and be able to write programs that manipulate the Arduino's IO pins and send/receive data to the PC using the serial link.

Second, you should look through the above included files, in particular the instructions for using the Tam chip and the source code (e.g. "Arduino sketch") for this demonstration program. The code is quite extensively documented- many questions you may have can be answered in the code itself. You should also have a level of sophistication in C programming and elementary data processing that you can look at existing code and understand what is happening. An elementary background in image or signal processing is also helpful, though not required.

1) Hooking up the Rox1 board to a full-sized Arduino:

The board plugs into the Arduino as shown below. You will need to make 5 connections between the Rox1 board and the Arduino: VDD and GND which connect respectively to the 5V and GND Arduino lines on the lower left in the picture, ANA which connects to the Arduino ANALOG 0 line on the lower right, and IO6 and IO7 which connect respectively to DIGITAL 6 and DIGITAL 7 Arduino lines on the top. You will need a soldering iron to do this- use the 5 headers that came with the Rox1 board. You may now plug the Arduino into your computer. NOTE: These instructions assume use of a 5V Arduino board- the Tam chips need 5V for optimal operation.

2) Compiling and uploading the code:

Unpack Tam2_OdometryExample_r1 into it's own folder and compile the sketch file of the same name. (You will see two other sketch files as well.) Upload it to the Arduino.

3) Start the serial monitor:

Start the serial monitor in the Arduino environment. You should see the following welcome message appear:

Sample odometry using Tam2 16x16 chip

a - ASCII image
c - Calibrate
f - print frame counter
m - MATLAB image
r - read calibration from EEPROM
s - store calibration to EEPROM
0 (zero) - quiet output
1 (one) - output every frame
o (letter o) - print odometry info

At the top of the serial monitor is a command area where you can enter simple commands that get sent to Arduino via the USB link. To enter a command, click in the command area, press the letter corresponding to the command, and press <enter>.

4) Calibration:

The first thing you need to do is to calibrate the sensor and acquire a fixed pattern noise mask. The best way to do this is to expose the sensor to a uniform background- you can hold it right up to an LCD screen displaying a uniform color, or you could simply place a white sheet of paper over or on top of the lens.

Enter the command "c" (lower case) to calibrate the sensor. This causes the calibration mask to be grabbed. You can now stop covering the sensor with white paper or holding it up to a monitor or whatever you were doing to give it a uniform background.

Enter the command "s" to save the mask to the Arduino's EEPROM. The two commands produce the following output on the serial monitor. Note that the calibration mask is rendered as an ASCII image. (Your pattern will be different but generally noisy in the same way.)

Calibration pattern grabbed from image sensor
+ -*=-.+.*+
- -.,+*.,+.,
#.@%. @,*=-
+..* *-%-# +
=,--%-.,==.*
. %-. *.+.=
%=.=,==-=.
,*,.** @ =%.
@ %- +*+#+,
.- - -**%%,
+ * *+- +
+.* =
Done.
Writing calibration mask to EEPROM...
Done

Note that once you have saved the calibration mask to EEPROM, you can recall it at later times with the "r" command.

5) Grab and display an image

Now that the sensor is calibrated, you can grab and display an image. If you aim the sensor towards a bright light, or hold a flashlight high over it, and enter the "a" command, you might get something like this:

ASCII dump of image, after calibration


..
.**
,#%,
..=,
.





Done.

Note the location of the bright spot. Or if you aim it at some other texture in the environment you'll get a different image. Here's an image acquired by aiming the sensor in some random direction in my room:

ASCII dump of image, after calibration
==++,,------
==+, .,,,,--
=+., ,,,,.,+
=+,++-,,,.,+
===*=*=+=,-=
*===+@@@@*%%
=====%@@%*@#
***=+++++-=@
***=++++++=@
%**=+++--+*@
*%*=+==, ,=@
%*===*%+ ,%
Done.

For those of you with access to MATLAB, you can use the "m" command to dump the image in a format that can be copied into MATLAB.

%-----------------------------
Data = [
-5 -5 -4 -4 -2 0 -1 -1 0 -1 -1 -2
-5 -5 -5 -4 -3 -1 -1 -1 0 1 1 -1
-5 -5 -4 -4 -3 2 3 2 2 1 0 0
-6 -6 -5 -4 1 3 1 3 3 2 -1 -1
-5 -6 -6 -2 1 2 3 3 4 2 -2 -4
-6 -7 -6 -3 -3 -3 -4 -4 0 -5 -8 -7
-6 -7 -7 -6 -4 -6 -9 -9 -5 -8 -11 -9
-8 -7 -7 -6 -5 -4 -3 -3 -2 -2 -8 -10
-8 -8 -8 -7 -5 -4 -3 -3 -3 -3 -7 -10
-10 -9 -9 -7 -6 -5 -4 -4 -2 1 -6 -11
-10 -11 -9 -8 -6 -5 -5 -8 -4 5 0 -7
-14 -12 -9 -7 -6 -5 -5 -8 -9 -5 1 -3
];
%-----------------------------

6) Odometry

Let's now try simple odometry. Point the sensor in one direction and enter the "z" command. This resets the odometry measurement- This is equivalent to resetting the trip odometer in your car to 0.0. The serial monitor should reply that the odometry value was reset.

Next enter the "o" (lower case letter "o") command. The sensor will dump out an odometry value, which may look something like this:

X: accx + ofx = odox 0 + -1 = -1
Y: accy + ofy = odoy 0 + -4 = -4

To understand these values, you'll have to look at the code. But what you need to know is that the values at the end, the -1 and the -4 in this case, are the odometry values.

Now move the sensor around a bit, and enter the "o" command again. You'll see a different odometry value depending on how you move the sensor.

7) More odometry: output modes "1" and "0"

Right now the sensor is operating in mode zero, where it only outputs odometry values when you give it the "o" command. You can set the sensor to output odometry every frame by using the "1" command. This will slow the sensor down because of the additional data being sent to the serial monitor, but it is useful for observing the odometry values as you move the sensor around:

Output mode is 1
31 9
29 8
29 11
30 8
30 10
31 7
31 9

...

23 -14
18 -15
15 -11
15 -13
7 -14
1 -10
-2 -14
-8 -14
-12 -7
-16 -5
-21 -7
-29 -3
-32 -2
-38 -4
-41 1
-45 1
-54 1
-57 -1
-60 -8
-67 -9
-75 -7
-78 -11
-82 -15
-91 -14
-93 -14
-103 -17
-111 -18
-115 -22
-123 -24
-128 -27
-130 -30
-136 -31
-137 -33
-146 -32
-150 -33

You can set the output mode back to zero using the "0" (zero) command.

8) How do I integrate this with my application?

If you look at the source code for the Tam2_OdometryExample_r1 sketch, at around line 313 you'll see a comment just after the variable odox and odoy are computed. These variables are the resulting odometry information. In this code we simply output them back to the serial monitor, but in your application you may want to generate other signals or output these values using some other mechanism. This is the place where you could insert such code.

9) How do I get optical flow?

Odometry is essentially an integral of optical flow. So the best way to measure optical flow is to grab odometry values a one time instant t1, and again at a time instant t2, compute the difference between the odometry values, and divide the difference by (t2-t2).

10) How can I speed up the algorithm?

There is certainly some more optimization that can be made to speed up the code, in particular fixing my embarrassing floating-point kludge at lines 215 and 216. (Apologies to everyone- I am a better chip designer than a C programmer, and I just can't figure out variable casteing on an Atmel.) Another way is to decrease the image size- The current code uses a 12x12 pixel image from the 16x16 image. (The full 16x16 image is too large for the processor!) You can reduce this to 10x10, 8x8, or maybe even less. This will increase the frame rate, but at the expense of some accuracy. I've noted the code where such changes would need to be made.