1 2 Previous Next 19 Replies Latest reply on Feb 7, 2020 2:26 AM by JayakrishnaT_76

    CSI-2 MIPI Issue

    HeHe_4094286

      Dear all,

       

      I am using the CX-3 controller interfaced to a ToF Image-Sensor using CSI-2. If the sensor sends the image data(which consists of a variable length of several same-sized sub-frames(640x480, RAW12)) as a superframe(=all the image data of a raw frame is transferred between a single "Start of Transmission" and "End of Transmission" on CSI-2 level) there is a limitation regarding the data-size. It is able to transfer 8 consecutive sub-frames having 640x480 resolution and 12bit per pixel. When transfering 9 consecuitve frames it stops working. However, if each sub-frame is transferred packed between "Start of Transmission" and "End of Transmission" packets, the image data can be transferred without a problem. I am somehow suspecting that the GPIFII interface causes these problems by maybe loosing track of data-counters due to an internal overflow. The firmware is mostly identical to the AN75779 UVC Firmware. Any help is kindly appreciated!

        • 1. Re: CSI-2 MIPI Issue
          JayakrishnaT_76

          Hello,

           

          According to my understanding, when you are trying to squeeze 8 sub frames between a SOT and EOT, it works fine. When you are trying to squeeze 9 sub frames, it is not working.

           

          Please let me know why you want to squeeze more and more sub frames between SOT and EOT. Also, how are you squeezing these sub frames between a single SOT and EOT? Also, what exactly is happening when you are trying to squeeze the 9th sub frame between SOT and EOT?

           

          Please share the following so that I can understand the problem better:

          1. MIPI Receiver Configuration Settings

          2. Please Probe the FV, LV and PCLK and share the traces.

           

          Best Regards,

          Jayakrishna

          • 2. Re: CSI-2 MIPI Issue
            HeHe_4094286

            Hi Jayakrishna,

             

            we are doing this for the post-processing software for data handling and it should make no difference concerning data-rates in my opinion.

            Is the FV signal on the parallel interface depending on EOT on CSI-2 level? Maybe it's causing a problem due to the GPIB software interrupt not happening at the right time. If we squeeze 9 raw frames together we are just delaying the EOT.

             

            The image sensor is connected to 2 lanes with 400MHz clock frequency.

            We are packing 2 Pixels of 12bits each into 3 bytes using this receiver configuration.

             

            CyU3PMipicsiCfg_t RAW12_Resolution0 =

            {

                CY_U3P_CSI_DF_RGB888,  /* CyU3PMipicsiDataFormat_t dataFormat */

                MIPICONFIG_NUM_OF_DATALANES, /* uint8_t numDataLanes */

                2,              /* uint8_t pllPrd */

                89,         /* uint16_t pllFbd */

                CY_U3P_CSI_PLL_FRS_250_500M, /* CyU3PMipicsiPllClkFrs_t pllFrs */

                CY_U3P_CSI_PLL_CLK_DIV_4,   /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

                CY_U3P_CSI_PLL_CLK_DIV_4,   /* CyU3PMipicsiPllClkDiv_t parClkDiv */

                0,                      /* uint16_t mClkCtl */

                CY_U3P_CSI_PLL_CLK_DIV_2,   /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

                640,            /* uint16_t hResolution */

                48                          /* uint16_t fifoDelay */

            };

             

            Is it even possible to probe these signals on the CX-3? -They are only available on the parallel GPIFII Interface and the image sensor is connected to 2 CSI-2 data-lanes of the CX-3 and the physical layer for this is D-PHY which is just serial.

             

            Thanks for your help in advance!

            • 3. Re: CSI-2 MIPI Issue
              JayakrishnaT_76

              Hello,

               

              Sorry for not being clear in my last response. Please share me a snapshot of the Image sensor configuration and CX3 Receiver Configuration Tabs of the MIPI configuration Utility.

               

              Regarding probing of FV, LV and PCLK, there are test pins on the CX3 chip to probe these signals. These are G6, H5 and H8 for HSYNC, VSYNC and PCLK respectively. Please probe the signals from these pins.

               

              Also, please refer to the following figure and answer my questions for better clarity of the problem:

               

               

              I understand that the video frame is of 640*480 resolution and each pixel is of 12 bits long. Please let me know which one among the following is the size of data packet between SOT and EOT in your test case:

              1. 640*12 bits.

              2. 640*480*12 bits

              3. 640*480*12*8 bits

               

              Best Regards,

              Jayakrishna

              • 4. Re: CSI-2 MIPI Issue
                HeHe_4094286

                image_sensor_configurations.PNG

                 

                 

                 

                 

                 

                 

                 

                 

                 

                Thanks for the fast response! Attached you find the image sensor configuration. The Receiver configuration is unfortunately only in available in the struct CyU3PMipicsiCfg_t I attached in the last answer..I will try to do the measurements in the beginning of next week. The size of the data packet depends is 640x480x12x8. If we increase this to 640x480x12x9 it stops working..

                Thanks for the help!

                • 5. Re: CSI-2 MIPI Issue
                  JayakrishnaT_76

                  Hello,

                   

                  Please try adjusting multiplier of unit clock setting in the CX3 Receiver Configuration tab of the MIPI Configuration Utility and let me know if there are any changes to the present behaviour.

                   

                  Please let me know the following:

                  1. The exact blanking time (the time difference between EOT of a packet and the SOT of the next packet).

                  2. I understand that each data packet is of size 640*480*12*8 bits. Please let me know the number of such packets you are sending when FV is HIGH.

                  3. When you are trying to change the number of sub frames in a packet, is there a change in fps or is it preserved.

                   

                  Best Regards,

                  Jayakrishna

                  • 6. Re: CSI-2 MIPI Issue
                    HeHe_4094286

                    1. The blanking time between two frames(consisting of several subframes as explained) depends on the exposure time for each subframe and the FPS set. So its varying between different measurements. Is the blanking time in the sensor settings an upper boundary?

                    What happens if this is violated?

                    2. Just to clarify on the MIPI side it looks like this: <SOT> var_frames x 640 x 480 x 12 bits <EOT>, where var_frames is the number of subframes

                    3. No there is no change in FPS, it changes the blanking time as explained above.

                     

                    Many thanks for your help!

                    • 7. Re: CSI-2 MIPI Issue
                      JayakrishnaT_76

                      Hello,

                       

                      Thanks for providing the details. But, I still didnt get a response for my second question in the previous reply. Please let me know the number of subframes in a frame. Or, the number of subframes that you are transmitting when the FV is HIGH  (when frame valid is high).

                       

                      Best Regards,

                      Jayakrishna

                      • 8. Re: CSI-2 MIPI Issue
                        HeHe_4094286

                        The number of subframes is variable(it changes depending on the sensor use-case) and if we increase the number of subframes to a specific value(e.g. 9) the transmission stops working. If we transmit every subframe as a whole frame we do not see the limitation.

                        e.g. <SOT>subframe1<EOT><SOT>subframe2<EOT><SOT>subframe3<EOT>...<SOT>subframe9<EOT> => working

                        <SOT>subframe1,subframe2,subframe3, ... , subframe9<EOT> => not working

                        Thanks for the response!

                        • 9. Re: CSI-2 MIPI Issue
                          JayakrishnaT_76

                          Hello,

                           

                          Thanks again for the response. We still dont have enough information to debug this problem. Please refer to the following snapshot and find my questions/comments below:

                           

                           

                          In between FS (Frame Start) and FE (Frame End), how many SOTs and EOTs do you have in your application? In between each SOT and EOT there are 640*480*12*var_frames bits. So, if there are multiple SOTs and EOTs (say x) in between FS and FE, then throughput will increase. This throughput will be (640*480*12*var_frames*x). If this throughput increase beyond 1Gbps on a lane, then CX3 cant support it as the max throughput on a lane supported by CX3 is 1Gbps. If there is only one SOT and EOT between FS and FE, then you need to ensure that the minimum frame blanking time (roughly the time between FE to the next FS) is 200us. This is because, as the active data is increased in a frame by keeping FPS constant, there is a possibility that the blanking time can decrease.

                          If you are using Microsoft UVC driver, even if one byte is missed per frame, the host application will show a blank image. Please share the debugprints using teraterm so that we can understand the amount of data transferred to the host. We can also verify whether the data is being sent to the host or not.

                           

                          Also, please let me know whether my suggestion of increasing the Multiplier of Unit clock setting in the CX3 Receiver Configuration tab of the MIPI Configuration Utility in response 5 had some impact or not? Also, please share the debugprints corresponding to different setting of Multiplier of Unit clock. In the each debugprint, please add a title indicating which setting of Multiplier of unit clock you have used.

                           

                          Best Regards,

                          Jayakrishna

                          • 10. Re: CSI-2 MIPI Issue
                            HeHe_4094286

                            Hi again,

                             

                            I just had a detailed look into the sensor behavior again and found out that SoT and EoT is issued for every row(480 rows in total) inside the subframes. So if the number of subframes increases so does the throughput. But still this does not reach 1Gbps as we have 480 x 640 pixels/col x 12 bits x num_of_subframes. num_of_subframes reaches 24 as a maximum and it stops working before this value. We have seen the behavior also if we operate on very low FPS(e.g.: 5 FPS)

                             

                            So to clarify again we are operating the sensor in two modes - 1. Single, 2. Superframe

                             

                            1. Single means that between every subframe a <Frame End> is sent at CSI2 level and additionally the Frame-Blank is set to low during

                            the transmission of the subframe.

                             

                            2. Superframe means that <FrameEnd> is suppressed between subframes and is only set after transmission of the whole frame. The Frame-Blank is set to low during the whole frame as well and set to high again after transmission of the whole frame.

                             

                            I already tried increasing the Multiplier of Unit clock. As the configuration GUI is not working I directly modified the CyU3PMipicsiCfg_t struct in the C code. Am I right that pllFbd field related to the Multiplier Unit clock? It looks like this:

                             

                            CyU3PMipicsiCfg_t IRS_RAW12_Resolution0 =

                            {

                                CY_U3P_CSI_DF_RGB888,  /* CyU3PMipicsiDataFormat_t dataFormat */

                                MIPICONFIG_NUM_OF_DATALANES, /* uint8_t numDataLanes */

                                2,              /* uint8_t pllPrd */

                                89,         /* uint16_t pllFbd */

                                CY_U3P_CSI_PLL_FRS_250_500M, /* CyU3PMipicsiPllClkFrs_t pllFrs */

                                CY_U3P_CSI_PLL_CLK_DIV_4,   /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

                                CY_U3P_CSI_PLL_CLK_DIV_4,   /* CyU3PMipicsiPllClkDiv_t parClkDiv */

                                0,                      /* uint16_t mClkCtl */

                                CY_U3P_CSI_PLL_CLK_DIV_2,   /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

                                640,            /* uint16_t hResolution */

                                48                          /* uint16_t fifoDelay */

                            };

                             

                            Thanks in advance for your help!

                            • 11. Re: CSI-2 MIPI Issue
                              JayakrishnaT_76

                              Hello,

                               

                              Please let me know if you are getting any MIPI errors when you are increasing the number of subframes. You can obtain this by enabling the macro CX3_DEBUG_ENABLED in cycx3_uvc.h. Also, please share the debug logs using teraterm which can give some information to understand where exactly the execution of code stops.

                               

                              Best Regards,

                              Jayakrishna

                              • 12. Re: CSI-2 MIPI Issue
                                HeHe_4094286

                                Hello again,

                                 

                                so I enabled Debugging and printed the Error-Counts of the GPIF-SM. When the no. of subframes is increased to a number where it stops working I am seeing the following errors on GPIF:

                                frmErrCnt: 1

                                crcErrCnt: 1

                                ctlErrCnt: 1

                                 

                                Those errors do not appear everytime it seems. So frmErrCnt is almost visible everytime but crcErrCnt and ctlErrCnt are not visible in every measurement. Additionally I printed the SMState variable which holds the current state of the GPIF-SM. I can see two different states

                                when increasing the no. of subframes to a number where it stops working.

                                SMState: 0x5 or 0x7

                                 

                                If transmission works normally I can see the SMState beeing 0x2.

                                 

                                Thanks for your help in advance!

                                • 13. Re: CSI-2 MIPI Issue
                                  JayakrishnaT_76

                                  Hello,

                                   

                                  Please find the following KBA which describes about the various MIPI errors.

                                  MIPI-CSI Protocol and Physical Layer Errors in CX3 (CYUSB3065 and CYUSB3064) – KBA228482

                                   

                                  From the above KBA, the field frmErrCnt is incremented by 1 when an unexpected Frame Start or Frame end short packet is received. Please verify this is not happening.

                                   

                                  Please let me know the DMA buffer size and count you are using for this application. Also, please provide the complete debug logs. In the previous response I could get information about MIPI errors and the SMState. Please share the complete logs so that I can get more information. Also, please share the USB trace using Wireshark. This can be used to find out how much data is actually sent to host.

                                   

                                  Best Regards,

                                  Jayakrishna

                                  • 14. Re: CSI-2 MIPI Issue
                                    HeHe_4094286

                                    Hi again,

                                     

                                    after investigating that those errors also sometimes appear while image data is received successfully I checked the USB Descriptor and changed the frame-size in bytes according to the requirements. The FW was using 1920x1080x2 Bytes as a maximum which explains the behavior of the limiting no. of subframes. So now the transmission works successfully. I have a final question. Which is the maximum number of bytes/UVC package which can be set by the probeCtrl? We are now using the following setting:

                                     

                                    uint8_t const glProbeCtrl[UVC_MAX_PROBE_SETTING] =

                                    {

                                        0x00, 0x00,                         /* bmHint : No fixed parameters */

                                        0x01,                               /* Use 1st Video format index */

                                        0x01,                               /* Use 1st Video frame index */

                                        0x15, 0x16, 0x05, 0x00,             /* Desired frame interval in 100ns = (1/30)x10^7 */

                                        0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

                                        0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

                                        0x00, 0x00,                         /* Compression quality control */

                                        0x00, 0x00,                         /* Window size for average bit rate */

                                        0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

                                        0x00, 0x80, 0x51, 0x01,                /* Max video frame size in bytes

                                        0x00, 0x40, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 16 KB */

                                        0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

                                        0x00,                               /* Framing Information - Ignored for uncompressed format*/

                                        0x00,                               /* Preferred payload format version */

                                        0x00,                               /* Minimum payload format version */

                                        0x00                                /* Maximum payload format version */

                                    };

                                     

                                    Thanks in advance for your help!

                                    1 2 Previous Next