8 Replies Latest reply on Aug 27, 2020 2:16 AM by alyo_4754581

    auto-exposure algorithm on CX3

    alyo_4754581

      Hi,

       

      My question is regarding running an auto exposure algorithm on the CX3 chip but only for images and not large video formats.

       

      From my understanding as there is no ISP on the CX3, the only way to adjust the image parameters such as exposure  on the fly, is to send these data to the PC through the USB3.0 connection, do the processing there on a host application and then send back commands through the UVC to adjust the gain or other parameters to the CX3. Then the CX3 can write to the the image sensor these new parameters through the I2C connection and adjust the gain. so to summarize: 

       

      Image sensor ------I2C read ------> CX3 ----- (USB 3.0) UVC connection ------> PC (process) ----> (USB 3.0) UVC connection -----> CX3 ------I2C write ------> Image sensor

       

      Is this correct?

       

      If so I have the following questions:

       

      1) What is a typical round trip time for an update like this. From reading the values from the sensor to writing new ones to it given that the processing step in the PC is known and is not too large? If you don't know this can you please refer me to some documentation with which I could find out the timings of the I2C read/writes ans the UVC read/Writes?

       

      2) Can the ARM9 processor on the CX3 not process the information such as exposure and automatically adjust and reply with a correct gain value. This will obviously take a lot of time but for single image this time might be acceptable. I am wondering if the FX3 SDK comes with any API functions to help with this.

       

      3) the ultimate goal here is to have an automatic way of adjusting the exposure of the sensor based on the surrounding on the fly. I know that having and ISP is the easiest way but for our application we only care about still images so a high performance custom ISP is not required and we might be able to get by with the ARM9. Are there any resources that could help us calculate how performant the ARM9 could be? What is the performance overhead of all the UVC and other peripherals?

       

      Thanks for your help in advance

       

      Cheers

      Ali

        • 1. Re: auto-exposure algorithm on CX3
          RashiV_61

          Hello Ali,

           

          1) What is a typical round trip time for an update like this. From reading the values from the sensor to writing new ones to it given that the processing step in the PC is known and is not too large? If you don't know this can you please refer me to some documentation with which I could find out the timings of the I2C read/writes ans the UVC read/Writes?

          >> The total time for changing the exposure as explained by you will depend on the I2C speed, DMA fabric speed, and USB.

          I2C clocks supported by CX3: The I2C controller supports bus frequencies of 400 kHz and 1 MHz

          DMA fabric: Distributed DMA controller that is capable of moving data at 800 MBps. This speed depends on different aspects like type of DMA channel, DMA buffer size, etc.

          USB: You can refer to this app note https://www.cypress.com/file/125281/download  which mentions about the different aspects that impact the USB speed.

           

          2) Can the ARM9 processor on the CX3 not process the information such as exposure and automatically adjust and reply with a correct gain value. This will obviously take a lot of time but for a single image, this time might be acceptable. I am wondering if the FX3 SDK comes with any API functions to help with this.

          >> Yes, your understanding is right regarding the latency. Along with that CX3 has SRAM size of 512 KB and out of which 224 KB is assigned for DMA buffer. So storing a whole frame on CX3 would not be possible.

          The idea of sending the frames to the host and doing the processing there and then sending back the data would be better if you are not planning to use an ISP

           

           

          Please let me know if you have more queries on this

           

          Regards,

          Rashi

          1 of 1 people found this helpful
          • 2. Re: auto-exposure algorithm on CX3
            alyo_4754581

            Hi Rashi,

             

            thanks for the information that will help us for sure.

             

            Logically we also wanted to go with the CX3 with the ISP as that would give us the performance we are looking for but all the RDKs that have an ISP are End of life on e-con systems website and I could not find it anywhere else.

             

            So regarding this do you by any chance know any other RDK or a source to order the Tania or the Ascella kits?

             

            Also how hard is it to integrate our own ISP with the CX3? Is there any resource or application note from Cypress that tells you how to do this?

             

            Cheers

            Ali

            • 3. Re: auto-exposure algorithm on CX3
              RashiV_61

              Hello Ali,

               

              Please refer to this weblink for the information on Tania and Ascella kits

              Tania: https://www.cypress.com/documentation/development-kitsboards/tania-cypress-cx3-socionext-isp-reference-design-kit-rdk-du…

              Ascella: https://www.cypress.com/documentation/development-kitsboards/ascella-cypress-cx3-thine-isp-13mp-reference-design-kit-rdk

               

              As of now, we don't have any application note to interface ISP to CX3 but we have seen customers building their custom design with the same configuration i.e. sensor + ISP +CX3. There are some KBAs that explains the interfacing of ISP to FX3, you can refer to them for some reference

              KBA: FX3 + FPGA + HelionVision ISP-Based Industrial Camera Reference Design – KBA222700

              UVC Firmware for FX3 FPGA Industrial Camera Demo Kit - KBA229407

               

               

              You can also refer to FAQ FAQ: Ascella - 13MP Camera RDK for Cypress CX3 THine ISP

               

              Please let me know if any queries on this

              Regards,

              Rashi

              1 of 1 people found this helpful
              • 4. Re: auto-exposure algorithm on CX3
                alyo_4754581

                Hi Rashi,

                 

                Thanks for the info I did look at the Tania and Ascella, as I mentioned those are EOL though.

                 

                One last question. Is there a way to adjust the resolution of the image on the fly using CX3? I want to take the initial frame from the sensor at a lower resolution (so I have to process less data ) process the image in post in a PC and then send back the values to the sensor. When the sensor is adjusted for the new correct exposrue value I then want to take a full resolution image and send it to the PC to store. Is this possible to do on the fly using CX3?

                 

                Cheers

                Ali

                • 5. Re: auto-exposure algorithm on CX3
                  RashiV_61

                  Hi Ali,

                   

                  You can add one more resolution to the firmware using the CX3 configuration tool and generate the firmware from the tool. You can refer to this KBA for adding one more resolution  Steps to Setup up MIPI CSI Camera Solution with CX3 – KBA225748

                   

                  After the changes in the firmware, you can switch to the other (newly added resolution) by changing the resolution from the host application. The host will send SET_CUR request to the device which will be handled in the firmware and new resolution settings will be passed to the sensor and streaming will start.

                   

                  Please refer to the attached snippet of CX3 configuration tool (highlighted newly added resolution)

                   

                   

                  Regards,

                  Rashi

                  1 of 1 people found this helpful
                  • 6. Re: auto-exposure algorithm on CX3
                    alyo_4754581

                    Hi Rashi,

                     

                    Amazing. Thanks a lot for the help. I knew that I can set multiple resolutions but being able to change the resolution on the fly is super useful for us. Just wondering this will not crop into the sensor correct? it will just give me a lower resolution over all correct?

                     

                    Cheers,

                    Ali

                    • 7. Re: auto-exposure algorithm on CX3
                      RashiV_61

                      Hello Ali,

                       

                      I didn't get your question. Please explain.

                      When SET CUR request is sent from the host appropriate sensor settings will be passed to set the sensor to the required resolution

                       

                      Here is the snippet from the firmware generated by the CX3 configuration tool

                       

                      static void

                      CyCx3UvcAppImageSensorSetVideoResolution(

                              uint8_t resolution_index

                              )

                      {

                      CyU3PReturnStatus_t status = CY_U3P_SUCCESS;

                      switch (CyU3PUsbGetSpeed ())

                      {

                      case CY_U3P_SUPER_SPEED:

                      switch (resolution_index)

                      {

                       

                       

                      case 0x01:

                      /* Write Resolution0Settings */

                      #ifndef FX3_STREAMING

                      status = CyU3PMipicsiSetIntfParams (&null_YUY2_Resolution0, CyFalse);

                      if (status != CY_U3P_SUCCESS)

                      {

                      CyU3PDebugPrint (4, "\n\rUSBStpCB:SetIntfParams SS1 Err = 0x%x", status);

                      }

                      #endif

                      CyCx3_ImageSensor_Set_Resolution0 ();

                      break;

                       

                      case 0x02:

                      /* Write Resolution1Settings */

                      #ifndef FX3_STREAMING

                      status = CyU3PMipicsiSetIntfParams (&null_YUY2_Resolution1, CyFalse);      // new resolutions MIPI structure

                      if (status != CY_U3P_SUCCESS)

                      {

                      CyU3PDebugPrint (4, "\n\rUSBStpCB:SetIntfParams SS1 Err = 0x%x", status);

                      }

                      #endif

                      CyCx3_ImageSensor_Set_Resolution1 ();     //new resolution sensor settings

                      break;

                       

                      }

                      break;

                       

                      }

                      }

                       

                      Please let me know if more queries on this

                       

                      Regards,

                      Rashi

                      1 of 1 people found this helpful
                      • 8. Re: auto-exposure algorithm on CX3
                        alyo_4754581

                        All good Rashi thanks for the help!