Embedded Eye

Give your gizmo the gift of sight

Firmware Development for CYE8

In the current version of the firmware for the CYE8 sensor boards, the image arrays Pinhole_image, ImageX0, and ImageX1 are used for the computation of optic flow. When the main loop executes, Pinhole_image is copied over alternately either to ImageX0 or ImageX1 by the high pass filtering function Pix_HPF. Next, the fixed pattern noise is added to the copied image. Then, both ImageX0 and ImageX1 are sent to the optical flow computation function IIA2.

Since data memory on the microcontroller is at a premium (and is currently about 88% in use with revision 4), it is important to minimize the use of data memory, as taken up by these arrays. I think the optic flow computation can be performed with just 2 arrays instead of 3, and with no additional computational requirements. Actually, I think this implementation would save on some computation as well.

Basically, the modification to the firmware that I propose is as follows:
On each loop
- read in the Pinhole_image
- perform high pass filtering on Pinhole_image (no need to save to a new array, Pinhole_image can be overwritten)
- add the fixed pattern noise to Pinhole_image (perhaps this should be called AddFP instead of AddFPN, since it is not really noise but actually something we want to do)
- compute the optic flow by calling IIA2(Pinhole_image, Old_pinhole_image,...)
- copy Pinhole_image to Old_pinhole_image

I was also wondering if it is really necessary to have both the Pinhole_image array and the structure image_out_16x16, which contains an array called "read" for the pinhole image? On each loop, Pinhole_image is copied to image_out_16x16.read (actually, currently ImageX0 is copied). It seems that the pinhole image could be read directly into image_out_16x16.read, and I don't think there would be any performance penalty for writing to an array that is part of a global structure instead of simply writing to a global array. Similarly, the array Raw_image_16x16 and the structure raw_image_16x16 may be redundant.


Views: 80

▶ Reply to This

Replies to This Discussion

Permalink Reply by AJ on January 27, 2011 at 3:34pm

Chris,

 

I'm not familiar with SMBus, I'll have to check it out.

 

The only issue that I know of with version 6 is that I wasn't maintaining the functionality of optic flow mode 2, so that is probably broken, but it is probably also broken in version 5. I have actually fixed this on my local copy of the code, but I haven't committed it since I was also working on some other changes that aren't ready to commit. I think the most important improvement in version 6 was double buffering, which required some minor changes in I2Csupport. I think it would be safe to work with version 6, I just think it hasn't been evaluated by anyone else.

Permalink Reply by Chris Shake on January 27, 2011 at 4:46pm

I've just uploaded revision 7 with the added SMBus compatibility layer and a few other scattered updates, all documented in the changelog.

No changes to the flow computation except commenting out a few unused variables.

I added switches to pause and resume the image grabbing and computation, mainly to be sent just before and after an image read to ensure it all comes from one frame. I also added a command to change the of_scaling, which should be self-explanatory. Everything else was in I2CSupport for adding the SMBus things.

All updates in this revision were of course tested, except going in and actually changing the of_scale variable (but since that addition is so simple I can't see how it could not work).

 

Chris

 

  • Added a set of commands (10-13) and code to allow SMBus protocol compatibility.
    • COMM_CMD_SMBUS_READ = 10 - In a "read byte data" command, is silently ignored
    • COMM_CMD_SMBUS_READ_WORD = 11 - In a "read word data" command, basically ignored
    • COMM_CMD_SMBUS_READ_BLOCK = 12 - "read block data", makes the first byte of the following read command be the length of the total output array
    • COMM_CMD_SMBUS_READ_CONTINUE = 13 - continuation of "read block data". SMBus limits messages to 32 data bytes, so if the total array is longer the "read block data" command will return a 2-byte message (low high) with the overall length of the array, then the host should send the appropriate number of additional "read block data" transactions with READ_CONTINUE as the command and it will send the long array back in 32-byte chunks until finished.
    • variables added: unsigned char smbus_last_command, unsigned int smbus_start_partial, unsigned int smbus_partial_left. (5 bytes total)
    • All attempts made to not interfere with existing operation, so nothing changes unless one of these 4 commands is used.
  • Fixed the check for ensuring commands all have arguments (non-smbus indicator commands at least).
  • Added two commands for debugging:
    • COMM_CMD_PAUSE = 90 (keyed w/arg = 90) - pauses all work except command processing and I2C i/o
      • Idea is to ensure that an image being read out is all from one frame
    • COMM_CMD_RESUME = 91 - resumes computation
    • variables added: unsigned int control_flags - using bit 1 for pausing, leaving 7 open for other application control use. (1 byte)
  • Unkeyed COMM_CMD_LED_MODE to allow for adding numbered (above 2) flags throughout the program (debugging). The only existing use only triggers when it is set to 1 or 2 anyway.
  • Added support for dynamically changing the flow scaling:
    • COMM_CMD_OF_SCALE = 82 - set of_scale to cmd1 * 2 (cmd1 is unsigned char). The default value is 64 * 2 stored in a float, and we desire more sensitivity for our application. May change the scaling of the argument in the future to allow for values of more than 510, at the expense of precision. 2^arg1 would be the absolute highest we could go, but loses so much accuracy that it's undesired as of yet.
  • Removed declared but unused variables:
    • unsigned char i,j and signed char oldofx,oldofy in process_images()
  • Fixed xmtbuf.read_length assignment for COMM_ATT_IMAGE_OUT to the actual pinhole size, it had been using the max size.

Overall change: 2 additional bytes of RAM in use, a few more if statements and integer operations, a single flop added in changing the of_scale, so there should be no observable change in speed anywhere. All SMBus compatibility additions are in the ISR function switch statement inside if statements, with no change to the operation unless the command flags are used.

Permalink Reply by Geoffrey L. Barrows on January 30, 2011 at 2:28pm

Hi Chris,

This is great! Thank You for making that addition.

Out of curiosity- just to make sure have you verified that I2C still works or should one of us test that out (just to make sure).

Thanks!

Geof

Permalink Reply by Chris Shake on January 30, 2011 at 4:00pm

I haven't been able to test it myself, since I don't have an adapter (yet) that is capable of doing raw I2C. I can't see a reason it wouldn't work, as I only added some statements that trigger with the new commands and didn't change existing code.

Chris

Permalink Reply by Geoffrey L. Barrows on January 30, 2011 at 4:09pm

You're right it's probably OK. We'll try it out this week.

Geof

Permalink Reply by Chris Shake on February 25, 2011 at 4:27pm

As I was talking about in the other thread, I've added support for saving calibration values and state variables to EEPROM to save across power cycles, and just committed the change as Revision 8.

I've tested and built this version on two of my sensors and it works as expected, with no changes to any existing functionality or requirement to change existing code that anyone is using to talk to the sensors. The main additions:

* Saved all the variables in the BiasSettings struct to EEPROM along with Amp_Type, and they will be loaded after power cycles. Since this means that the values might not be the defaults when power is applied, I added ATT #30 as a 9 variable output array containing those values. Because of the array structure, the old variable names were converted to pointers that reference the new array locations, to retain readability.

* Saved to EEPROM everything on the application side that would be considered a state variable, i.e. everything that can be set by sending a byte with one of the application commands that isn't a routine or function. ATT was not included, since it changes too often. As with the VC variables, these were also added to an output array #72 that can be read to see what the current states are. Instead of converting the old variables into pointers, I decided to make them into #define "aliases" that can be used exactly as before, but transparently redirect to the array locations, so as to not increase memory use. It's an ugly hack, but it helps readability.

More details about the array orders and specific EEPROM memory locations used can be found in the revision notes. This should sync easily enough with custom versions that anyone may be using, the core functionality of the sensors hasn't been changed at all.

Permalink Reply by Geoffrey L. Barrows on March 2, 2011 at 1:33am

Chris,

Thank You again for doing this. I apologize I haven't had a chance to look at that code yet- I will do so when the new CYE8's come in...

Geof

Permalink Reply by Chris Shake on June 7, 2011 at 1:02pm

Geof,

I'm looking into modifying the firmware to use the vision chips with lenses, and have a question about binning on the F64. In F64PlusInterface.c/.h there's a function smooth_image(row,col) that says it bins the pixels, but when I look at the get_raw_image_16x16() in ApplicationPrototype.c/.h the setup lines of code send the VCF commands for no binning. If I were to want a 16x16 image that binned the extra pixels to create in effect a lower resolution image instead of skipping them, would I change those VCF lines to match those in the smooth_image code? The idea I'm going for is if there's a spot only visible on one pixel, but that pixel would be skipped over in the get_raw_image code, I want it to affect the nearest pixel that is read.

 

Chris

RSS

© 2013   Created by Geoffrey L. Barrows.   Powered by

Badges  |  Report an Issue  |  Terms of Service

Sign in to chat!