Raw10 to UVC using 24bit bus

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
RiBi_4316346
Level 1
Level 1

Hello,

I am trying to stream data from a 1008x800 raw10 ensor to standard UVC application (Amcap, VLC).

I can successfully stream if I use 16bit bus and I select CY_U3P_CSI_DF_YUV422_8_2 format in CyU3PMipicsiSetIntfParams, but I would like to make it work also using 24bit GPIF bus.

To set 24bit bus I changed the definition of GPIF_BUS_WIDTH from CY_U3P_MIPICSI_BUS_16 to CY_U3P_MIPICSI_BUS_24, but the streaming doesn't work, I don't understand why. I probed HSYNC and VSYNC and they are toggling with the correct timing.

The line size is a multiple of 24 bits and the buffer size is 36kB, so it's a multiple of 24 bits.

Here are my settings for 24bit case:

pastedImage_1.png

pastedImage_2.png

CyU3PMipicsiCfg_t OH02A10_RAW10_1008x800 = 

{

    CY_U3P_CSI_DF_YUV422_8_2,  /* CyU3PMipicsiDataFormat_t dataFormat */

    2,                          /* uint8_t numDataLanes */

    2,                /* uint8_t pllPrd */

    89,            /* uint16_t pllFbd */

    CY_U3P_CSI_PLL_FRS_250_500M, /* CyU3PMipicsiPllClkFrs_t pllFrs */ 

    CY_U3P_CSI_PLL_CLK_DIV_4,    /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

    CY_U3P_CSI_PLL_CLK_DIV_4,    /* CyU3PMipicsiPllClkDiv_t parClkDiv */

    0,                        /* uint16_t mClkCtl */

    CY_U3P_CSI_PLL_CLK_DIV_2,    /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

    1008,                /* uint16_t hResolution */

    50                            /* uint16_t fifoDelay */

};

and here is the SS descriptor:

/* GUID, globally unique identifier used to identify streaming-encoding format */

//rgb565

0X7B,0XEB,0X36,0XE4,

0X4F,0X52,0XCE,0X11,

0X9F,0X53,0X00,0X20,

0XAF,0X0B,0XA7,0X70,

0x10,  //Number of bits per pixel: 16

0x01, //Optimum Frame Index for this stream: 1 (1008x800)

0x00,  //X dimension of the picture aspect ratio; Non-interlaced

0x00,  //Y dimension of the pictuer aspect ratio: Non-interlaced

0x00,  //Interlace Flags: Progressive scanning, no interlace

0x00,  //duplication of the video stream restriction: 0 - no restriction

/* Class specific Uncompressed VS Frame Descriptor 1 - 1008x800 */

    0x1E,                                //Descriptor size

    CX3_CS_INTRFC_DESCR,                 //Descriptor type

    0x05,                                //Subtype:  frame interface

    0x01,                               // Frame Descriptor Index: 1

    0x00,                                //No Still image capture method supported

    0x76,0x02,                         // Width in pixel:  1008*10/16=630(0x276)

    0x20,0x03,                          //Height in pixel: 800

    0x00,0xc0,0x9b,0x1c,              //Min bit rate (bits/s): 630 x 800 x 16 x 59.52= 479969280(0x 1C9BC000)

    0x00,0xc0,0x9b,0x1c,              //Max bit rate (bits/s): Fixed rate so same as Min

    0x80,0x61,0x0f,0x00,              //Maximum video or still frame size in bytes(Deprecated): 630 x 800 x 2=1008000(0xF6180)

    0x4a,0x90,0x02,0x00,              //Default frame interval (in 100ns units): (1/59.52)x10^7

    0x01,                                //Frame interval type : No of discrete intervals

    0x4a,0x90,0x02,0x00,             // Frame interval 3: Same as Default frame interval

here is the probe control settings:

/*UVC Probe Control Settings */

uint8_t glProbeCtrl[CX3_UVC_MAX_PROBE_SETTING] = {

    0x00, 0x00,                         // bmHint : No fixed parameters

    0x01,                                //Use 1st Video format index

    0x01,                               // Use 1st Video frame index

    0x4a,0x90,0x02,0x00,             // Desired frame interval in 100ns = (1/30)x10^7

    0x00, 0x00,                          //Key frame rate in key frame/video frame units

    0x00, 0x00,                          //PFrame rate in PFrame / key frame units

    0x00, 0x00,                          //Compression quality control

    0x00, 0x00,                          //Window size for average bit rate

    0x00, 0x00,                          //Internal video streaming i/f latency in ms

    0x80,0x61,0x0f,0x00,              //Maximum video or still frame size in bytes(Deprecated): 630 x 800 x 2=1008000(0xF6180)

#ifdef CX3_UVC_1_0_SUPPORT

    0x00, 0x90, 0x00, 0x00              // No. of bytes device can rx in single payload: 32KB

#else

    /* UVC 1.1 Probe Control has additional fields from UVC 1.0 */

    0x00, 0x90, 0x00, 0x00,              //No. of bytes device can rx in single payload: 32KB

    0x00, 0x60, 0xE3, 0x16,              //Device Clock

    0x00,                                //Framing Information - Ignored for uncompressed format

    0x00,                                //Preferred payload format version

    0x00,                                //Minimum payload format version

    0x00                                 //Maximum payload format version

#endif

};

/* UVC Probe Control Setting - 1008x800 */

uint8_t const gl1008x800ProbeCtrl[CX3_UVC_MAX_PROBE_SETTING] = {

    0x00, 0x00,                         // bmHint : No fixed parameters

    0x01,                               // Use 1st Video format index

    0x01,                               // Use 1st Video frame index

    0x4a,0x90,0x02,0x00,             // Desired frame interval in 100ns = (1/30)x10^7

    0x00, 0x00,                         // Key frame rate in key frame/video frame units

    0x00, 0x00,                         // PFrame rate in PFrame / key frame units

    0x00, 0x00,                         // Compression quality control

    0x00, 0x00,                         // Window size for average bit rate

    0x00, 0x00,                         // Internal video streaming i/f latency in ms

    0x80,0x61,0x0f,0x00,             // Maximum video or still frame size in bytes(Deprecated): 630 x 800 x 2=1008000(0xF6180)

#ifdef CX3_UVC_1_0_SUPPORT

    0x00, 0x90, 0x00, 0x00              // No. of bytes device can rx in single payload: 36KB

#else

    /* UVC 1.1 Probe Control has additional fields from UVC 1.0*/

    0x00, 0x90, 0x00, 0x00,             // No. of bytes device can rx in single payload: 36KB

    0x00, 0x60, 0xE3, 0x16,             // Device Clock

    0x00,                               // Framing Information - Ignored for uncompressed format

    0x00,                               // Preferred payload format version

    0x00,                               // Minimum payload format version

    0x00                                // Maximum payload format version

#endif

};

Can you help me understand what is the issue?

Thanks,

Riccardo

0 Likes
1 Solution
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello,

The MIPI Receiver configuration settings seems to be fine. Please let me know if you have made any edits in the file cyu3mipicsi.c generated by the tool. If you have made any edits, please revert them back to the original settings generated by the tool.

Also, the UVC host applications do not support RGB565. Please do not change the GUID to RGB565 in the descriptors. You can keep the GUID of YUY2 itself. The bits per pixel should be 16 (0x10) itself. Both these are needed to use the UVC host applications like VLC or AMCap.

Please try the above modifications and let me know if you are able to see a preview on the AMCap. If this is not seen, please share the UART debug logs so that I can have a look at it.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna

View solution in original post

0 Likes
8 Replies
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello,

The MIPI Receiver configuration settings seems to be fine. Please let me know if you have made any edits in the file cyu3mipicsi.c generated by the tool. If you have made any edits, please revert them back to the original settings generated by the tool.

Also, the UVC host applications do not support RGB565. Please do not change the GUID to RGB565 in the descriptors. You can keep the GUID of YUY2 itself. The bits per pixel should be 16 (0x10) itself. Both these are needed to use the UVC host applications like VLC or AMCap.

Please try the above modifications and let me know if you are able to see a preview on the AMCap. If this is not seen, please share the UART debug logs so that I can have a look at it.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hello Jayakrishna,

thank you for your answer.

Actually I just managed to make it work: in cyu3mipicsi.c I had put CY_U3P_CSI_DF_YUV422_8_2 format but this is wrong for 24 bit bus.

Now I corrected it to CY_U3P_CSI_DF_RGB888 and the streaming works fine.

Best regards,

Riccardo

0 Likes

Hello Riccardo,

Please let me know if you have anymore questions on this thread?

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hello Jayakrishna,

yes I have one question more: on the host side I am trying to reconstruct the original raw10 image starting from the received "distorted" image.

If I understand correctly this is the conversion from MIPI raw10 to parallel 24 bit inside the cx3:

pastedImage_0.png

Then the data is sent to the host (using UVC) in 16bit format, so I expect the transmitted data to be ordered like this:

pastedImage_1.png

Am I right or I got the order wrong?

Thanks,

Riccardo

0 Likes

Hello Riccardo,

Your understanding on the conversion of MIPI data to 24 bit parallel data by CX3 is correct. The parallel data is sampled by CX3 and is stored inside the DMA buffers associated with the channel from P Port to U Port. Before sending the DMA buffer to the U Port, a 12 byte header is added at the start of each buffer. For storing the frame data, a certain amount of full buffers and one partial buffer is required. You can refer to the following KBA to understand the number of full and partial buffers required to send out the frame data.

UVC Troubleshooting Guide – KBA226722

The header information added at the start of a DMA buffer can be used to distinguish different frames and also to distinguish between a full and a partial buffer. Please refer to section 2.3.4 of AN75779 to understand more about the UVC header and its importance. The link to the same is given below:

https://www.cypress.com/file/123506/download

FX3 transfers the data filled in the DMA buffer to the host. As the start of each buffer (full or partial buffer), will have UVC header of 12 bytes, this will be transferred to the host first followed by the data that is present in the DMA buffer. The host application should decode the header and the data to reconstruct the image frame back.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hello Jayakrishna,

thank you for the clarifications.

I still am not sure about the bit and byte ordering: with respect to my previous post I made some corrections.

Here is the result:

A) MIPI serial raw10:

pastedImage_0.png

B) parallel 24 bit sampling:

pastedImage_1.png

C) yuy2 transmission:

pastedImage_2.png

Can you please confirm if the ordering of the bits is correct?

Regards,

Riccardo

0 Likes

Hello,

Please check the response 10 of the following thread and let me know if it answers your question.

Can the Denebola RDK handle 1920x1080@60fps in RAW10 ?

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hello Jayakrishna,

thank you for the suggestion: the data ordering explained in that thread is correct, now I can successfully reconstruct the correct image on the host pc.

Best regards,

Riccardo

0 Likes