8 Replies Latest reply on Aug 10, 2020 9:07 AM by RiBi_4316346

    Raw10 to UVC using 24bit bus

    RiBi_4316346

      Hello,

      I am trying to stream data from a 1008x800 raw10 ensor to standard UVC application (Amcap, VLC).

      I can successfully stream if I use 16bit bus and I select CY_U3P_CSI_DF_YUV422_8_2 format in CyU3PMipicsiSetIntfParams, but I would like to make it work also using 24bit GPIF bus.

       

      To set 24bit bus I changed the definition of GPIF_BUS_WIDTH from CY_U3P_MIPICSI_BUS_16 to CY_U3P_MIPICSI_BUS_24, but the streaming doesn't work, I don't understand why. I probed HSYNC and VSYNC and they are toggling with the correct timing.

      The line size is a multiple of 24 bits and the buffer size is 36kB, so it's a multiple of 24 bits.

       

      Here are my settings for 24bit case:

       

       

      CyU3PMipicsiCfg_t OH02A10_RAW10_1008x800 = 

      {

          CY_U3P_CSI_DF_YUV422_8_2,  /* CyU3PMipicsiDataFormat_t dataFormat */

          2,                          /* uint8_t numDataLanes */

          2,                /* uint8_t pllPrd */

          89,            /* uint16_t pllFbd */

          CY_U3P_CSI_PLL_FRS_250_500M, /* CyU3PMipicsiPllClkFrs_t pllFrs */ 

          CY_U3P_CSI_PLL_CLK_DIV_4,    /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

          CY_U3P_CSI_PLL_CLK_DIV_4,    /* CyU3PMipicsiPllClkDiv_t parClkDiv */

          0,                        /* uint16_t mClkCtl */

          CY_U3P_CSI_PLL_CLK_DIV_2,    /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

          1008,                /* uint16_t hResolution */

          50                            /* uint16_t fifoDelay */

      };

       

       

      and here is the SS descriptor:

       

      /* GUID, globally unique identifier used to identify streaming-encoding format */

      //rgb565

      0X7B,0XEB,0X36,0XE4,

      0X4F,0X52,0XCE,0X11,

      0X9F,0X53,0X00,0X20,

      0XAF,0X0B,0XA7,0X70,

       

      0x10,  //Number of bits per pixel: 16

      0x01, //Optimum Frame Index for this stream: 1 (1008x800)

      0x00,  //X dimension of the picture aspect ratio; Non-interlaced

      0x00,  //Y dimension of the pictuer aspect ratio: Non-interlaced

      0x00,  //Interlace Flags: Progressive scanning, no interlace

      0x00,  //duplication of the video stream restriction: 0 - no restriction

       

      /* Class specific Uncompressed VS Frame Descriptor 1 - 1008x800 */

       

          0x1E,                                //Descriptor size

          CX3_CS_INTRFC_DESCR,                 //Descriptor type

          0x05,                                //Subtype:  frame interface

          0x01,                               // Frame Descriptor Index: 1

          0x00,                                //No Still image capture method supported

          0x76,0x02,                         // Width in pixel:  1008*10/16=630(0x276)

          0x20,0x03,                          //Height in pixel: 800

          0x00,0xc0,0x9b,0x1c,              //Min bit rate (bits/s): 630 x 800 x 16 x 59.52= 479969280(0x 1C9BC000)

          0x00,0xc0,0x9b,0x1c,              //Max bit rate (bits/s): Fixed rate so same as Min

          0x80,0x61,0x0f,0x00,              //Maximum video or still frame size in bytes(Deprecated): 630 x 800 x 2=1008000(0xF6180)

          0x4a,0x90,0x02,0x00,              //Default frame interval (in 100ns units): (1/59.52)x10^7

          0x01,                                //Frame interval type : No of discrete intervals

          0x4a,0x90,0x02,0x00,             // Frame interval 3: Same as Default frame interval

       

      here is the probe control settings:

       

      /*UVC Probe Control Settings */

      uint8_t glProbeCtrl[CX3_UVC_MAX_PROBE_SETTING] = {

          0x00, 0x00,                         // bmHint : No fixed parameters

          0x01,                                //Use 1st Video format index

          0x01,                               // Use 1st Video frame index

          0x4a,0x90,0x02,0x00,             // Desired frame interval in 100ns = (1/30)x10^7

          0x00, 0x00,                          //Key frame rate in key frame/video frame units

          0x00, 0x00,                          //PFrame rate in PFrame / key frame units

          0x00, 0x00,                          //Compression quality control

          0x00, 0x00,                          //Window size for average bit rate

          0x00, 0x00,                          //Internal video streaming i/f latency in ms

          0x80,0x61,0x0f,0x00,              //Maximum video or still frame size in bytes(Deprecated): 630 x 800 x 2=1008000(0xF6180)

      #ifdef CX3_UVC_1_0_SUPPORT

          0x00, 0x90, 0x00, 0x00              // No. of bytes device can rx in single payload: 32KB

      #else

          /* UVC 1.1 Probe Control has additional fields from UVC 1.0 */

          0x00, 0x90, 0x00, 0x00,              //No. of bytes device can rx in single payload: 32KB

          0x00, 0x60, 0xE3, 0x16,              //Device Clock

          0x00,                                //Framing Information - Ignored for uncompressed format

          0x00,                                //Preferred payload format version

          0x00,                                //Minimum payload format version

          0x00                                 //Maximum payload format version

      #endif

      };

       

      /* UVC Probe Control Setting - 1008x800 */

      uint8_t const gl1008x800ProbeCtrl[CX3_UVC_MAX_PROBE_SETTING] = {

          0x00, 0x00,                         // bmHint : No fixed parameters

          0x01,                               // Use 1st Video format index

          0x01,                               // Use 1st Video frame index

          0x4a,0x90,0x02,0x00,             // Desired frame interval in 100ns = (1/30)x10^7

          0x00, 0x00,                         // Key frame rate in key frame/video frame units

          0x00, 0x00,                         // PFrame rate in PFrame / key frame units

          0x00, 0x00,                         // Compression quality control

          0x00, 0x00,                         // Window size for average bit rate

          0x00, 0x00,                         // Internal video streaming i/f latency in ms

          0x80,0x61,0x0f,0x00,             // Maximum video or still frame size in bytes(Deprecated): 630 x 800 x 2=1008000(0xF6180)

      #ifdef CX3_UVC_1_0_SUPPORT

          0x00, 0x90, 0x00, 0x00              // No. of bytes device can rx in single payload: 36KB

      #else

          /* UVC 1.1 Probe Control has additional fields from UVC 1.0*/

          0x00, 0x90, 0x00, 0x00,             // No. of bytes device can rx in single payload: 36KB

          0x00, 0x60, 0xE3, 0x16,             // Device Clock

          0x00,                               // Framing Information - Ignored for uncompressed format

          0x00,                               // Preferred payload format version

          0x00,                               // Minimum payload format version

          0x00                                // Maximum payload format version

      #endif

      };

       

      Can you help me understand what is the issue?

       

      Thanks,

      Riccardo

        • 1. Re: Raw10 to UVC using 24bit bus
          JayakrishnaT_76

          Hello,

           

          The MIPI Receiver configuration settings seems to be fine. Please let me know if you have made any edits in the file cyu3mipicsi.c generated by the tool. If you have made any edits, please revert them back to the original settings generated by the tool.

           

          Also, the UVC host applications do not support RGB565. Please do not change the GUID to RGB565 in the descriptors. You can keep the GUID of YUY2 itself. The bits per pixel should be 16 (0x10) itself. Both these are needed to use the UVC host applications like VLC or AMCap.

           

          Please try the above modifications and let me know if you are able to see a preview on the AMCap. If this is not seen, please share the UART debug logs so that I can have a look at it.

           

          Best Regards,

          Jayakrishna

          • 2. Re: Raw10 to UVC using 24bit bus
            RiBi_4316346

            Hello Jayakrishna,

            thank you for your answer.

            Actually I just managed to make it work: in cyu3mipicsi.c I had put CY_U3P_CSI_DF_YUV422_8_2 format but this is wrong for 24 bit bus.

            Now I corrected it to CY_U3P_CSI_DF_RGB888 and the streaming works fine.

             

            Best regards,

            Riccardo

            • 3. Re: Raw10 to UVC using 24bit bus
              JayakrishnaT_76

              Hello Riccardo,

               

              Please let me know if you have anymore questions on this thread?

               

              Best Regards,

              Jayakrishna

              • 4. Re: Raw10 to UVC using 24bit bus
                RiBi_4316346

                Hello Jayakrishna,

                yes I have one question more: on the host side I am trying to reconstruct the original raw10 image starting from the received "distorted" image.

                 

                If I understand correctly this is the conversion from MIPI raw10 to parallel 24 bit inside the cx3:

                Then the data is sent to the host (using UVC) in 16bit format, so I expect the transmitted data to be ordered like this:

                Am I right or I got the order wrong?

                 

                Thanks,

                Riccardo

                • 5. Re: Raw10 to UVC using 24bit bus
                  JayakrishnaT_76

                  Hello Riccardo,

                   

                  Your understanding on the conversion of MIPI data to 24 bit parallel data by CX3 is correct. The parallel data is sampled by CX3 and is stored inside the DMA buffers associated with the channel from P Port to U Port. Before sending the DMA buffer to the U Port, a 12 byte header is added at the start of each buffer. For storing the frame data, a certain amount of full buffers and one partial buffer is required. You can refer to the following KBA to understand the number of full and partial buffers required to send out the frame data.

                  UVC Troubleshooting Guide – KBA226722

                   

                  The header information added at the start of a DMA buffer can be used to distinguish different frames and also to distinguish between a full and a partial buffer. Please refer to section 2.3.4 of AN75779 to understand more about the UVC header and its importance. The link to the same is given below:

                  https://www.cypress.com/file/123506/download

                   

                  FX3 transfers the data filled in the DMA buffer to the host. As the start of each buffer (full or partial buffer), will have UVC header of 12 bytes, this will be transferred to the host first followed by the data that is present in the DMA buffer. The host application should decode the header and the data to reconstruct the image frame back.

                   

                  Best Regards,

                  Jayakrishna

                  • 6. Re: Raw10 to UVC using 24bit bus
                    RiBi_4316346

                    Hello Jayakrishna,

                    thank you for the clarifications.

                    I still am not sure about the bit and byte ordering: with respect to my previous post I made some corrections.

                    Here is the result:

                     

                    A) MIPI serial raw10:

                     

                    B) parallel 24 bit sampling:

                    C) yuy2 transmission:

                     

                    Can you please confirm if the ordering of the bits is correct?

                     

                    Regards,

                    Riccardo

                    • 7. Re: Raw10 to UVC using 24bit bus
                      JayakrishnaT_76

                      Hello,

                       

                      Please check the response 10 of the following thread and let me know if it answers your question.

                      Can the Denebola RDK handle 1920x1080@60fps in RAW10 ?

                       

                      Best Regards,

                      Jayakrishna

                      • 8. Re: Raw10 to UVC using 24bit bus
                        RiBi_4316346

                        Hello Jayakrishna,

                        thank you for the suggestion: the data ordering explained in that thread is correct, now I can successfully reconstruct the correct image on the host pc.

                         

                        Best regards,

                        Riccardo