auto-exposure algorithm on CX3

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
alyo_4754581
Level 1
Level 1
10 likes given 5 likes given First like given

Hi,

My question is regarding running an auto exposure algorithm on the CX3 chip but only for images and not large video formats.

From my understanding as there is no ISP on the CX3, the only way to adjust the image parameters such as exposure  on the fly, is to send these data to the PC through the USB3.0 connection, do the processing there on a host application and then send back commands through the UVC to adjust the gain or other parameters to the CX3. Then the CX3 can write to the the image sensor these new parameters through the I2C connection and adjust the gain. so to summarize: 

Image sensor ------I2C read ------> CX3 ----- (USB 3.0) UVC connection ------> PC (process) ----> (USB 3.0) UVC connection -----> CX3 ------I2C write ------> Image sensor

Is this correct?

If so I have the following questions:

1) What is a typical round trip time for an update like this. From reading the values from the sensor to writing new ones to it given that the processing step in the PC is known and is not too large? If you don't know this can you please refer me to some documentation with which I could find out the timings of the I2C read/writes ans the UVC read/Writes?

2) Can the ARM9 processor on the CX3 not process the information such as exposure and automatically adjust and reply with a correct gain value. This will obviously take a lot of time but for single image this time might be acceptable. I am wondering if the FX3 SDK comes with any API functions to help with this.

3) the ultimate goal here is to have an automatic way of adjusting the exposure of the sensor based on the surrounding on the fly. I know that having and ISP is the easiest way but for our application we only care about still images so a high performance custom ISP is not required and we might be able to get by with the ARM9. Are there any resources that could help us calculate how performant the ARM9 could be? What is the performance overhead of all the UVC and other peripherals?

Thanks for your help in advance

Cheers

Ali

0 Likes
1 Solution
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Ali,

1) What is a typical round trip time for an update like this. From reading the values from the sensor to writing new ones to it given that the processing step in the PC is known and is not too large? If you don't know this can you please refer me to some documentation with which I could find out the timings of the I2C read/writes ans the UVC read/Writes?

>> The total time for changing the exposure as explained by you will depend on the I2C speed, DMA fabric speed, and USB.

I2C clocks supported by CX3: The I2C controller supports bus frequencies of 400 kHz and 1 MHz

DMA fabric: Distributed DMA controller that is capable of moving data at 800 MBps. This speed depends on different aspects like type of DMA channel, DMA buffer size, etc.

USB: You can refer to this app note https://www.cypress.com/file/125281/download  which mentions about the different aspects that impact the USB speed.

2) Can the ARM9 processor on the CX3 not process the information such as exposure and automatically adjust and reply with a correct gain value. This will obviously take a lot of time but for a single image, this time might be acceptable. I am wondering if the FX3 SDK comes with any API functions to help with this.

>> Yes, your understanding is right regarding the latency. Along with that CX3 has SRAM size of 512 KB and out of which 224 KB is assigned for DMA buffer. So storing a whole frame on CX3 would not be possible.

The idea of sending the frames to the host and doing the processing there and then sending back the data would be better if you are not planning to use an ISP

Please let me know if you have more queries on this

Regards,

Rashi

Regards,
Rashi

View solution in original post

8 Replies
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Ali,

1) What is a typical round trip time for an update like this. From reading the values from the sensor to writing new ones to it given that the processing step in the PC is known and is not too large? If you don't know this can you please refer me to some documentation with which I could find out the timings of the I2C read/writes ans the UVC read/Writes?

>> The total time for changing the exposure as explained by you will depend on the I2C speed, DMA fabric speed, and USB.

I2C clocks supported by CX3: The I2C controller supports bus frequencies of 400 kHz and 1 MHz

DMA fabric: Distributed DMA controller that is capable of moving data at 800 MBps. This speed depends on different aspects like type of DMA channel, DMA buffer size, etc.

USB: You can refer to this app note https://www.cypress.com/file/125281/download  which mentions about the different aspects that impact the USB speed.

2) Can the ARM9 processor on the CX3 not process the information such as exposure and automatically adjust and reply with a correct gain value. This will obviously take a lot of time but for a single image, this time might be acceptable. I am wondering if the FX3 SDK comes with any API functions to help with this.

>> Yes, your understanding is right regarding the latency. Along with that CX3 has SRAM size of 512 KB and out of which 224 KB is assigned for DMA buffer. So storing a whole frame on CX3 would not be possible.

The idea of sending the frames to the host and doing the processing there and then sending back the data would be better if you are not planning to use an ISP

Please let me know if you have more queries on this

Regards,

Rashi

Regards,
Rashi

Hi Rashi,

thanks for the information that will help us for sure.

Logically we also wanted to go with the CX3 with the ISP as that would give us the performance we are looking for but all the RDKs that have an ISP are End of life on e-con systems website and I could not find it anywhere else.

So regarding this do you by any chance know any other RDK or a source to order the Tania or the Ascella kits?

Also how hard is it to integrate our own ISP with the CX3? Is there any resource or application note from Cypress that tells you how to do this?

Cheers

Ali

0 Likes

Hello Ali,

Please refer to this weblink for the information on Tania and Ascella kits

Tania: https://www.cypress.com/documentation/development-kitsboards/tania-cypress-cx3-socionext-isp-referen...

Ascella: https://www.cypress.com/documentation/development-kitsboards/ascella-cypress-cx3-thine-isp-13mp-refe...

As of now, we don't have any application note to interface ISP to CX3 but we have seen customers building their custom design with the same configuration i.e. sensor + ISP +CX3. There are some KBAs that explains the interfacing of ISP to FX3, you can refer to them for some reference

KBA: FX3 + FPGA + HelionVision ISP-Based Industrial Camera Reference Design – KBA222700

UVC Firmware for FX3 FPGA Industrial Camera Demo Kit - KBA229407

You can also refer to FAQ FAQ: Ascella - 13MP Camera RDK for Cypress CX3 THine ISP

Please let me know if any queries on this

Regards,

Rashi

Regards,
Rashi

Hi Rashi,

Thanks for the info I did look at the Tania and Ascella, as I mentioned those are EOL though.

One last question. Is there a way to adjust the resolution of the image on the fly using CX3? I want to take the initial frame from the sensor at a lower resolution (so I have to process less data ) process the image in post in a PC and then send back the values to the sensor. When the sensor is adjusted for the new correct exposrue value I then want to take a full resolution image and send it to the PC to store. Is this possible to do on the fly using CX3?

Cheers

Ali

0 Likes
lock attach
Attachments are accessible only for community members.

Hi Ali,

You can add one more resolution to the firmware using the CX3 configuration tool and generate the firmware from the tool. You can refer to this KBA for adding one more resolution  Steps to Setup up MIPI CSI Camera Solution with CX3 – KBA225748

After the changes in the firmware, you can switch to the other (newly added resolution) by changing the resolution from the host application. The host will send SET_CUR request to the device which will be handled in the firmware and new resolution settings will be passed to the sensor and streaming will start.

Please refer to the attached snippet of CX3 configuration tool (highlighted newly added resolution)

Regards,

Rashi

Regards,
Rashi

Hi Rashi,

Amazing. Thanks a lot for the help. I knew that I can set multiple resolutions but being able to change the resolution on the fly is super useful for us. Just wondering this will not crop into the sensor correct? it will just give me a lower resolution over all correct?

Cheers,

Ali

0 Likes

Hello Ali,

I didn't get your question. Please explain.

When SET CUR request is sent from the host appropriate sensor settings will be passed to set the sensor to the required resolution

Here is the snippet from the firmware generated by the CX3 configuration tool

static void

CyCx3UvcAppImageSensorSetVideoResolution(

        uint8_t resolution_index

        )

{

CyU3PReturnStatus_t status = CY_U3P_SUCCESS;

switch (CyU3PUsbGetSpeed ())

{

case CY_U3P_SUPER_SPEED:

switch (resolution_index)

{

case 0x01:

/* Write Resolution0Settings */

#ifndef FX3_STREAMING

status = CyU3PMipicsiSetIntfParams (&null_YUY2_Resolution0, CyFalse);

if (status != CY_U3P_SUCCESS)

{

CyU3PDebugPrint (4, "\n\rUSBStpCB:SetIntfParams SS1 Err = 0x%x", status);

}

#endif

CyCx3_ImageSensor_Set_Resolution0 ();

break;

case 0x02:

/* Write Resolution1Settings */

#ifndef FX3_STREAMING

status = CyU3PMipicsiSetIntfParams (&null_YUY2_Resolution1, CyFalse);      // new resolutions MIPI structure

if (status != CY_U3P_SUCCESS)

{

CyU3PDebugPrint (4, "\n\rUSBStpCB:SetIntfParams SS1 Err = 0x%x", status);

}

#endif

CyCx3_ImageSensor_Set_Resolution1 ();     //new resolution sensor settings

break;

}

break;

}

}

Please let me know if more queries on this

Regards,

Rashi

Regards,
Rashi

All good Rashi thanks for the help!

0 Likes