CSI-2 MIPI Issue

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
HeHe_4094286
Employee
Employee

Dear all,

I am using the CX-3 controller interfaced to a ToF Image-Sensor using CSI-2. If the sensor sends the image data(which consists of a variable length of several same-sized sub-frames(640x480, RAW12)) as a superframe(=all the image data of a raw frame is transferred between a single "Start of Transmission" and "End of Transmission" on CSI-2 level) there is a limitation regarding the data-size. It is able to transfer 8 consecutive sub-frames having 640x480 resolution and 12bit per pixel. When transfering 9 consecuitve frames it stops working. However, if each sub-frame is transferred packed between "Start of Transmission" and "End of Transmission" packets, the image data can be transferred without a problem. I am somehow suspecting that the GPIFII interface causes these problems by maybe loosing track of data-counters due to an internal overflow. The firmware is mostly identical to the AN75779 UVC Firmware. Any help is kindly appreciated!

0 Likes
1 Solution

Hi again,

after investigating that those errors also sometimes appear while image data is received successfully I checked the USB Descriptor and changed the frame-size in bytes according to the requirements. The FW was using 1920x1080x2 Bytes as a maximum which explains the behavior of the limiting no. of subframes. So now the transmission works successfully. I have a final question. Which is the maximum number of bytes/UVC package which can be set by the probeCtrl? We are now using the following setting:

uint8_t const glProbeCtrl[UVC_MAX_PROBE_SETTING] =

{

    0x00, 0x00,                         /* bmHint : No fixed parameters */

    0x01,                               /* Use 1st Video format index */

    0x01,                               /* Use 1st Video frame index */

    0x15, 0x16, 0x05, 0x00,             /* Desired frame interval in 100ns = (1/30)x10^7 */

    0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

    0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

    0x00, 0x00,                         /* Compression quality control */

    0x00, 0x00,                         /* Window size for average bit rate */

    0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

    0x00, 0x80, 0x51, 0x01,                /* Max video frame size in bytes

    0x00, 0x40, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 16 KB */

    0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

    0x00,                               /* Framing Information - Ignored for uncompressed format*/

    0x00,                               /* Preferred payload format version */

    0x00,                               /* Minimum payload format version */

    0x00                                /* Maximum payload format version */

};

Thanks in advance for your help!

View solution in original post

0 Likes
19 Replies
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello,

According to my understanding, when you are trying to squeeze 8 sub frames between a SOT and EOT, it works fine. When you are trying to squeeze 9 sub frames, it is not working.

Please let me know why you want to squeeze more and more sub frames between SOT and EOT. Also, how are you squeezing these sub frames between a single SOT and EOT? Also, what exactly is happening when you are trying to squeeze the 9th sub frame between SOT and EOT?

Please share the following so that I can understand the problem better:

1. MIPI Receiver Configuration Settings

2. Please Probe the FV, LV and PCLK and share the traces.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hi Jayakrishna,

we are doing this for the post-processing software for data handling and it should make no difference concerning data-rates in my opinion.

Is the FV signal on the parallel interface depending on EOT on CSI-2 level? Maybe it's causing a problem due to the GPIB software interrupt not happening at the right time. If we squeeze 9 raw frames together we are just delaying the EOT.

The image sensor is connected to 2 lanes with 400MHz clock frequency.

We are packing 2 Pixels of 12bits each into 3 bytes using this receiver configuration.

CyU3PMipicsiCfg_t RAW12_Resolution0 =

{

    CY_U3P_CSI_DF_RGB888,  /* CyU3PMipicsiDataFormat_t dataFormat */

    MIPICONFIG_NUM_OF_DATALANES, /* uint8_t numDataLanes */

    2,              /* uint8_t pllPrd */

    89,         /* uint16_t pllFbd */

    CY_U3P_CSI_PLL_FRS_250_500M, /* CyU3PMipicsiPllClkFrs_t pllFrs */

    CY_U3P_CSI_PLL_CLK_DIV_4,   /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

    CY_U3P_CSI_PLL_CLK_DIV_4,   /* CyU3PMipicsiPllClkDiv_t parClkDiv */

    0,                      /* uint16_t mClkCtl */

    CY_U3P_CSI_PLL_CLK_DIV_2,   /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

    640,            /* uint16_t hResolution */

    48                          /* uint16_t fifoDelay */

};

Is it even possible to probe these signals on the CX-3? -They are only available on the parallel GPIFII Interface and the image sensor is connected to 2 CSI-2 data-lanes of the CX-3 and the physical layer for this is D-PHY which is just serial.

Thanks for your help in advance!

0 Likes

Hello,

Sorry for not being clear in my last response. Please share me a snapshot of the Image sensor configuration and CX3 Receiver Configuration Tabs of the MIPI configuration Utility.

Regarding probing of FV, LV and PCLK, there are test pins on the CX3 chip to probe these signals. These are G6, H5 and H8 for HSYNC, VSYNC and PCLK respectively. Please probe the signals from these pins.

Also, please refer to the following figure and answer my questions for better clarity of the problem:

pastedImage_0.png

I understand that the video frame is of 640*480 resolution and each pixel is of 12 bits long. Please let me know which one among the following is the size of data packet between SOT and EOT in your test case:

1. 640*12 bits.

2. 640*480*12 bits

3. 640*480*12*8 bits

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

image_sensor_configurations.PNG

Thanks for the fast response! Attached you find the image sensor configuration. The Receiver configuration is unfortunately only in available in the struct CyU3PMipicsiCfg_t I attached in the last answer..I will try to do the measurements in the beginning of next week. The size of the data packet depends is 640x480x12x8. If we increase this to 640x480x12x9 it stops working..

Thanks for the help!

0 Likes

Hello,

Please try adjusting multiplier of unit clock setting in the CX3 Receiver Configuration tab of the MIPI Configuration Utility and let me know if there are any changes to the present behaviour.

Please let me know the following:

1. The exact blanking time (the time difference between EOT of a packet and the SOT of the next packet).

2. I understand that each data packet is of size 640*480*12*8 bits. Please let me know the number of such packets you are sending when FV is HIGH.

3. When you are trying to change the number of sub frames in a packet, is there a change in fps or is it preserved.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

1. The blanking time between two frames(consisting of several subframes as explained) depends on the exposure time for each subframe and the FPS set. So its varying between different measurements. Is the blanking time in the sensor settings an upper boundary?

What happens if this is violated?

2. Just to clarify on the MIPI side it looks like this: <SOT> var_frames x 640 x 480 x 12 bits <EOT>, where var_frames is the number of subframes

3. No there is no change in FPS, it changes the blanking time as explained above.

Many thanks for your help!

0 Likes

Hello,

Thanks for providing the details. But, I still didnt get a response for my second question in the previous reply. Please let me know the number of subframes in a frame. Or, the number of subframes that you are transmitting when the FV is HIGH  (when frame valid is high).

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

The number of subframes is variable(it changes depending on the sensor use-case) and if we increase the number of subframes to a specific value(e.g. 9) the transmission stops working. If we transmit every subframe as a whole frame we do not see the limitation.

e.g. <SOT>subframe1<EOT><SOT>subframe2<EOT><SOT>subframe3<EOT>...<SOT>subframe9<EOT> => working

<SOT>subframe1,subframe2,subframe3, ... , subframe9<EOT> => not working

Thanks for the response!

0 Likes

Hello,

Thanks again for the response. We still dont have enough information to debug this problem. Please refer to the following snapshot and find my questions/comments below:

pastedImage_0.png

In between FS (Frame Start) and FE (Frame End), how many SOTs and EOTs do you have in your application? In between each SOT and EOT there are 640*480*12*var_frames bits. So, if there are multiple SOTs and EOTs (say x) in between FS and FE, then throughput will increase. This throughput will be (640*480*12*var_frames*x). If this throughput increase beyond 1Gbps on a lane, then CX3 cant support it as the max throughput on a lane supported by CX3 is 1Gbps. If there is only one SOT and EOT between FS and FE, then you need to ensure that the minimum frame blanking time (roughly the time between FE to the next FS) is 200us. This is because, as the active data is increased in a frame by keeping FPS constant, there is a possibility that the blanking time can decrease.

If you are using Microsoft UVC driver, even if one byte is missed per frame, the host application will show a blank image. Please share the debugprints using teraterm so that we can understand the amount of data transferred to the host. We can also verify whether the data is being sent to the host or not.

Also, please let me know whether my suggestion of increasing the Multiplier of Unit clock setting in the CX3 Receiver Configuration tab of the MIPI Configuration Utility in response 5 had some impact or not? Also, please share the debugprints corresponding to different setting of Multiplier of Unit clock. In the each debugprint, please add a title indicating which setting of Multiplier of unit clock you have used.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hi again,

I just had a detailed look into the sensor behavior again and found out that SoT and EoT is issued for every row(480 rows in total) inside the subframes. So if the number of subframes increases so does the throughput. But still this does not reach 1Gbps as we have 480 x 640 pixels/col x 12 bits x num_of_subframes. num_of_subframes reaches 24 as a maximum and it stops working before this value. We have seen the behavior also if we operate on very low FPS(e.g.: 5 FPS)

So to clarify again we are operating the sensor in two modes - 1. Single, 2. Superframe

1. Single means that between every subframe a <Frame End> is sent at CSI2 level and additionally the Frame-Blank is set to low during

the transmission of the subframe.

2. Superframe means that <FrameEnd> is suppressed between subframes and is only set after transmission of the whole frame. The Frame-Blank is set to low during the whole frame as well and set to high again after transmission of the whole frame.

I already tried increasing the Multiplier of Unit clock. As the configuration GUI is not working I directly modified the CyU3PMipicsiCfg_t struct in the C code. Am I right that pllFbd field related to the Multiplier Unit clock? It looks like this:

CyU3PMipicsiCfg_t IRS_RAW12_Resolution0 =

{

    CY_U3P_CSI_DF_RGB888,  /* CyU3PMipicsiDataFormat_t dataFormat */

    MIPICONFIG_NUM_OF_DATALANES, /* uint8_t numDataLanes */

    2,              /* uint8_t pllPrd */

    89,         /* uint16_t pllFbd */

    CY_U3P_CSI_PLL_FRS_250_500M, /* CyU3PMipicsiPllClkFrs_t pllFrs */

    CY_U3P_CSI_PLL_CLK_DIV_4,   /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

    CY_U3P_CSI_PLL_CLK_DIV_4,   /* CyU3PMipicsiPllClkDiv_t parClkDiv */

    0,                      /* uint16_t mClkCtl */

    CY_U3P_CSI_PLL_CLK_DIV_2,   /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

    640,            /* uint16_t hResolution */

    48                          /* uint16_t fifoDelay */

};

Thanks in advance for your help!

0 Likes

Hello,

Please let me know if you are getting any MIPI errors when you are increasing the number of subframes. You can obtain this by enabling the macro CX3_DEBUG_ENABLED in cycx3_uvc.h. Also, please share the debug logs using teraterm which can give some information to understand where exactly the execution of code stops.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hello again,

so I enabled Debugging and printed the Error-Counts of the GPIF-SM. When the no. of subframes is increased to a number where it stops working I am seeing the following errors on GPIF:

frmErrCnt: 1

crcErrCnt: 1

ctlErrCnt: 1

Those errors do not appear everytime it seems. So frmErrCnt is almost visible everytime but crcErrCnt and ctlErrCnt are not visible in every measurement. Additionally I printed the SMState variable which holds the current state of the GPIF-SM. I can see two different states

when increasing the no. of subframes to a number where it stops working.

SMState: 0x5 or 0x7

If transmission works normally I can see the SMState beeing 0x2.

Thanks for your help in advance!

0 Likes

Hello,

Please find the following KBA which describes about the various MIPI errors.

MIPI-CSI Protocol and Physical Layer Errors in CX3 (CYUSB3065 and CYUSB3064) – KBA228482

From the above KBA, the field frmErrCnt is incremented by 1 when an unexpected Frame Start or Frame end short packet is received. Please verify this is not happening.

Please let me know the DMA buffer size and count you are using for this application. Also, please provide the complete debug logs. In the previous response I could get information about MIPI errors and the SMState. Please share the complete logs so that I can get more information. Also, please share the USB trace using Wireshark. This can be used to find out how much data is actually sent to host.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hi again,

after investigating that those errors also sometimes appear while image data is received successfully I checked the USB Descriptor and changed the frame-size in bytes according to the requirements. The FW was using 1920x1080x2 Bytes as a maximum which explains the behavior of the limiting no. of subframes. So now the transmission works successfully. I have a final question. Which is the maximum number of bytes/UVC package which can be set by the probeCtrl? We are now using the following setting:

uint8_t const glProbeCtrl[UVC_MAX_PROBE_SETTING] =

{

    0x00, 0x00,                         /* bmHint : No fixed parameters */

    0x01,                               /* Use 1st Video format index */

    0x01,                               /* Use 1st Video frame index */

    0x15, 0x16, 0x05, 0x00,             /* Desired frame interval in 100ns = (1/30)x10^7 */

    0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

    0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

    0x00, 0x00,                         /* Compression quality control */

    0x00, 0x00,                         /* Window size for average bit rate */

    0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

    0x00, 0x80, 0x51, 0x01,                /* Max video frame size in bytes

    0x00, 0x40, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 16 KB */

    0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

    0x00,                               /* Framing Information - Ignored for uncompressed format*/

    0x00,                               /* Preferred payload format version */

    0x00,                               /* Minimum payload format version */

    0x00                                /* Maximum payload format version */

};

Thanks in advance for your help!

0 Likes

Hello,

Please let me know if you are able to stream multiple subframes now or not?

In the probe control structure shared above,

0x00, 0x80, 0x51, 0x01,                /* Max video frame size in bytes

This field represents the maximum size of frame that can be transferred.

0x00, 0x40, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 16 KB */

This specifies the maximum size of DMA buffers that can be used in the application.

Please let me know if this is what you need or not.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hi,

as I mentioned in the last comment everything works now as expected. Conclusively it is possible to transfer a large amount of subframes now. I know what the specified fields are doing as I changed them successfully. My question now is which "Max video frame size" can be set by the descriptor? The size / DMA Buffer is set to 0x3000 so it is well below the maximum payload size. Thanks in advance for your help!

0 Likes

Hello,

The Frame Descriptors under Video Streaming Interface Descriptors will contain the information regarding different resolutions supported by the device. This will also contain the bit-rate corresponding to each resolution. You can set the max video frame size inside the probe control structure with the byte equivalent of the greatest bit-rate among the Frame Descriptors.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Thanks,

I think the issue is resolved now.

BR

0 Likes

Hello,

Thank you for the update.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes