6 Replies Latest reply on Oct 23, 2018 11:59 PM by KeerthyV_76

    Streaming Raw 12 as YUV422

      I have a Sony IMX 290 which supports full HD output, however the only output formats available to me are RAW 10 and RAW 12. I have set the sensor up so that it outputs RAW12 1948x1097@30 fps; I can see the buffers coming into the DMA with their corresponding prod/cons and partial buffer however I am unable to view the stream using a video streamer. The current settings I am using are as follows:

       

       

      IMX290_1080.PNG

      CyU3PMipicsiCfg_t IMX290_RAW12_1920x1080  =

      {

      CY_U3P_CSI_DF_RGB888,  /* CyU3PMipicsiDataFormat_t dataFormat */

      4,                          /* uint8_t numDataLanes */

      15, /* uint8_t pllPrd */

      494, /* uint16_t pllFbd */

      CY_U3P_CSI_PLL_FRS_250_500M, /* CyU3PMipicsiPllClkFrs_t pllFrs */

      CY_U3P_CSI_PLL_CLK_DIV_4, /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

      CY_U3P_CSI_PLL_CLK_DIV_4, /* CyU3PMipicsiPllClkDiv_t parClkDiv */

      0,                 /* uint16_t mClkCtl */

      CY_U3P_CSI_PLL_CLK_DIV_2, /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

      1948,         /* uint16_t hResolution */

      0                         /* uint16_t fifoDelay */

      };

       

       

       

      /* GUID, globally unique identifier used to identify streaming-encoding format*/

           0X55,0X59,0X56,0X59,

           0X00,0X00,0X10,0X00,

           0X80,0X00,0X00,0XAA,

           0X00,0X38,0X9B,0X71,

       

          0x10,                               /* Number of bits per pixel: 16*/

          0x01,                               /* Optimum Frame Index for this stream: 1 (1080p) */

          0x00,                               /* X dimension of the picture aspect ratio; Non-interlaced */

          0x00,                               /* Y dimension of the pictuer aspect ratio: Non-interlaced */

          0x00,                               /* Interlace Flags: Progressive scanning, no interlace */

          0x00,                               /* duplication of the video stream restriction: 0 - no restriction */

       

          /* Class specific Uncompressed VS Frame Descriptor 1 - 1080p */

          0x1E,                               /* Descriptor size */

          CX3_CS_INTRFC_DESCR,                /* Descriptor type*/

          0x05,                               /* Subtype:  frame interface*/

          0x01,                               /* Frame Descriptor Index: 1 */

          0x00,                               /* No Still image capture method supported */

          0xB5,0x05,                         /* Width in pixel:  1461 */

          0x49,0x04,                         /* Height in pixel: 1097 */

          0x60,0xa6,0xda,0x2d,             /* Min bit rate (bits/s): 1461 x 1097 x No Of Bits per Pixel x FrameRate =  769304160 */

          0x60,0xa6,0xda,0x2d,             /* Max bit rate (bits/s): Fixed rate so same as Min */

          0x3a,0xe9,0x30,0x00,             /* Maximum video or still frame size in bytes(Deprecated): 1461 x 1097 x 2 */

          0x15,0x16,0x05,0x00,             /* Default frame interval (in 100ns units): (1/30)x10^7 */

          0x01,                               /* Frame interval type : No of discrete intervals */

          0x15,0x16,0x05,0x00,             /* Frame interval 3: Same as Default frame interval */

       

       

      /* UVC Probe Control Settings */

      uint8_t glProbeCtrl[CX3_UVC_MAX_PROBE_SETTING] = {

          0x00, 0x00,                         /* bmHint : No fixed parameters */

          0x01,                               /* Use 1st Video format index */

          0x01,                               /* Use 1st Video frame index */

          0x2A,0x2C,0x0A,0x00,             /* Desired frame interval in 100ns */

          0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

          0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

          0x00, 0x00,                         /* Compression quality control */

          0x00, 0x00,                         /* Window size for average bit rate */

          0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

          0x3a,0xe9,0x30,0x00,                /* Max video frame size in bytes */

      #ifdef CX3_UVC_1_0_SUPPORT

          0x00, 0x90, 0x00, 0x00              /* No. of bytes device can rx in single payload: 32KB */

      #else

          /* UVC 1.1 Probe Control has additional fields from UVC 1.0 */

          0x00, 0x90, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 32KB */

          0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

          0x00,                               /* Framing Information - Ignored for uncompressed format*/

          0x00,                               /* Preferred payload format version */

          0x00,                               /* Minimum payload format version */

          0x00                                /* Maximum payload format version */

      #endif

      };

       

       

      /* UVC Probe Control Setting - 1080p*/

      uint8_t const gl1920x1080ProbeCtrl[CX3_UVC_MAX_PROBE_SETTING] = {

          0x00, 0x00,                         /* bmHint : No fixed parameters */

          0x01,                               /* Use 1st Video format index */

          0x01,                               /* Use 1st Video frame index */

          0x15,0x16,0x05,0x00,             /* Desired frame interval in 100ns = (1/30)x10^7 */

          0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

          0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

          0x00, 0x00,                         /* Compression quality control */

          0x00, 0x00,                         /* Window size for average bit rate */

          0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

          0x3a,0xe9,0x30,0x00,             /* Max video frame size in bytes = 1920 x 1080 x 2 */

      #ifdef CX3_UVC_1_0_SUPPORT

          0x00, 0x90, 0x00, 0x00              /* No. of bytes device can rx in single payload: 36KB */

      #else

          /* UVC 1.1 Probe Control has additional fields from UVC 1.0 */

          0x00, 0x90, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 36KB */

          0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

          0x00,                               /* Framing Information - Ignored for uncompressed format*/

          0x00,                               /* Preferred payload format version */

          0x00,                               /* Minimum payload format version */

          0x00                                /* Maximum payload format version */

      #endif

      };


      DMA UART DEBUG

      MIPI active = true

      Mipicsi active = 1

      Prod = 0 Cons = 0  Prtl_Sz = 2922 Frm_Cnt = 1 Frm_Sz = 2922 B

      0 0 0 6 0 0 0 0 11

      Prod = 1 Cons = 1  Prtl_Sz = 1554 Frm_Cnt = 2 Frm_Sz = 38370 B

      0 0 1 7 0 0 0 0 13

      frame count   2 received stopping app

      Prod = 87 Cons = 87  Prtl_Sz = 32046 Frm_Cnt = 3 Frm_Sz = 3235038 B

      0 0 2 2 0 0 0 0 2

      Prod = 87 Cons = 87  Prtl_Sz = 32046 Frm_Cnt = 4 Frm_Sz = 3235038 B

      0 0 0 0 0 0 0 0 0

      Prod = 87 Cons = 87  Prtl_Sz = 32046 Frm_Cnt = 5 Frm_Sz = 3235038 B

      0 0 2 0 0 0 0 0 0

      Prod = 87 Cons = 87  Prtl_Sz = 32046 Frm_Cnt = 6 Frm_Sz = 3235038 B

      0 0 0 0 0 0 0 0 0

        • 1. Re: Streaming Raw 12 as YUV422
          KeerthyV_76

          Hi,

           

          Can you please confirm the exact resolution that you are trying to stream?

          It looks like that the bytes received per frame 3235038 Bytes is not matching with the expected frame size 1948*1097*12/8 = 3205434 Bytes.

          The mismatch in the frame size may cause the frame to be dropped at the driver level and you may see a black screen.

          • 2. Re: Streaming Raw 12 as YUV422

            Hi,

             

            I am trying to stream 1948x1097@30fps; the full hd settings resolution according to the sony imx290 datasheet. I am uncertain why there is a mismatch in the frame size. How would I go about debugging/correcting this? 

            • 3. Re: Streaming Raw 12 as YUV422

              Hi,

               

              I have tried a number of different configurations on the sony IMX290 sensor and am unable to get the received frame size to match the expected one through calculations. Could this be due to value to which the PHY Time Delay is set?  The frame I receive to the CX3 is always returning longer than the expected frame for all tested resolutions, and I can't seem to find any relationship between the size being added to the frame. I have seen a number of similar threads where other people have used the YUV2 data format to stream RAW10 and RAW12 data formats however I am uncertain of the steps they have taken to reach the results discussed in the forum posts. I would highly appreciate if you could shed some light on this process as I can't seem to find any guidance anywhere else.

               

              All the best

              Armon Chojnacki

              • 4. Re: Streaming Raw 12 as YUV422
                KeerthyV_76

                Hi Armon,

                 

                If the bytes received/frame is constant then this is unlikely to be the issue due to Phy time delay parameter. Can you please check if there are any MIPI errors by using the API "CyU3PMipicsiGetErrors"?

                 

                Also please check if the sensor is sending any embedded or user defined data along with the video stream.

                • 5. Re: Streaming Raw 12 as YUV422

                  I have printed out the error counts in my DMA UART DEBUG section order underneath the produced, consumed line in the following order:

                       errCnts.crcErrCnt,errCnts.ctlErrCnt, errCnts.eidErrCnt, errCnts.frmErrCnt, errCnts.mdlErrCnt, errCnts.recSyncErrCnt, errCnts.recrErrCnt,      errCnts.unrSyncErrCnt, errCnts.unrcErrCnt.

                  Such that after the stream has started I receive 2 EID Errors every other frame until stopped via UVC command

                   

                  The IMX290 does send a line of embedded data before the frame. Does this mean I have to manually remove the embedded data from the sensor in the code before sending it through the dma as a stream?

                  • 6. Re: Streaming Raw 12 as YUV422
                    KeerthyV_76

                    The EID error counts unsupported Packet ID Errors.

                    Can you please check the MIPI data and let us know that data IDs that are present in the MIPI data?

                    The supported data formats are listed in the CX3 TRM.

                     

                    Does this mean I have to manually remove the embedded data from the sensor in the code before sending it through the dma as a stream?

                    >> If you don't want to send this embedded data to the USB host, then it is good to remove it from the sensor itself. If the data type is 0x12, then CX3 will send this data to the USB host along with the video data. In this case you may have to remove it in your host application.