1 2 Previous Next 17 Replies Latest reply on Nov 8, 2020 6:45 PM by RashiV_61

    Integrating OV2311 with EZ-USB SDK CX3 project

    NiNa_4798196

      I'm trying to bring up OV2311 sensor using CYUSB3065.

      I have the corresponding register settings for this sensor and set them accordingly in cyu3imagesensor.c file.

       

      I need help in understanding and setting CX3 receiver configuration, Image sensor configuration (1600x1300 @30fps) and UVC descriptors for RAW10 output format

      Please help and guide me in this regards.

        • 1. Re: Integrating OV2311 with EZ-USB SDK CX3 project
          RashiV_61

          Hello,

           

          Please let me know the following details:

          - MIPI data lanes:

          -  CSI clock (MHz):

          - THS -Prepare (ns):

          - THS Zero (ns):

          - H active(pixel):

          - H blanking(pixel):

          - V Active  (line):

          - V Blanking (line):

          - Data format: - RAW10

          - FPS: 30

           

          Please note the above details should be the same as the settings of the sensor.

          As the video format is RAW 10 (10bits/pixel) and if the GPIF bus width is configured for 16 bits then there are two options:

          - Padding of the pixel: non-image data (6 zeros) will be padded to the image data

          - Packing of the pixel: In packing, the image data will be packed to 16 bits

          Please refer to this KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387  and let me know which of the above two options you want in your application

           

           

          Please refer to this KBA for the steps Steps to Setup up MIPI CSI Camera Solution with CX3 – KBA225748

           

          Regards,

          Rashi

          • 2. Re: Integrating OV2311 with EZ-USB SDK CX3 project
            NiNa_4798196

            Hi RashiV_61,

             

            Sorry for the delayed reply. Here are the requested configurations :

            - MIPI data lanes: 2

            -  CSI clock (MHz): 400

            - THS -Prepare (ns): 60

            - THS Zero (ns): 120

            - H active(pixel): 1600

            - H blanking(pixel): 560

            - V Active  (line): 1200

            - V Blanking (line): 50

            - Data format: - RAW10

            - FPS: 30

            Screenshot from 2020-10-22 18-04-34.png

             

            For the output format we'll be going ahead with padding extra 6-bits and then streaming the output.

            • 3. Re: Integrating OV2311 with EZ-USB SDK CX3 project
              RashiV_61

              Hello,

               

              Thank you for the details.

              The CX3 MIPI receiver settings seems fine.

              But in your first post you mentioned the video resolution as "Image sensor configuration (1600x1300 @30fps) and UVC descriptors for RAW10 output format" but in the CX3 Configuration tool it says 1600*1200 @30fps and RAW 10 format.

              Please confirm that the settings under "MIPI CSI 2 Inputs (From Image sensor)" should be same as  the sensor settings.

               

              Regards,

              Rashi

              • 4. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                NiNa_4798196

                Hello,

                 

                Yes the sensor settings are set with respect to 1600*1200. I made a mistake while mentioning previously

                 

                The USB descriptors are set correctly now. The device is now enumerating correctly as a UVC camera.

                However, I have not tested streaming yet. I'll test and update on the same.

                 

                Regards,

                Ninad

                • 5. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                  NiNa_4798196

                  Hello,

                   

                  On connecting the camera to PC I'm getting following dmesg logs :

                   

                  [16958.784348] usb 3-6: USB disconnect, device number 15

                  [16961.755305] usb 3-6: new high-speed USB device number 16 using xhci_hcd

                  [16967.195397] usb 3-6: device descriptor read/64, error -110

                  [16967.457294] usb 3-6: New USB device found, idVendor=04b4, idProduct=00c3, bcdDevice= 0.00

                  [16967.457298] usb 3-6: New USB device strings: Mfr=1, Product=2, SerialNumber=0

                  [16967.457301] usb 3-6: Product: CX3-UVC

                  [16967.457303] usb 3-6: Manufacturer: Cypress

                  [16967.460001] uvcvideo: Found UVC 1.10 device CX3-UVC (04b4:00c3)

                  [16967.460637] uvcvideo: UVC non compliance - GET_DEF(PROBE) not supported. Enabling workaround.

                  [16967.460750] uvcvideo: Failed to query (129) UVC probe control : -32 (exp. 34).

                  [16967.460755] uvcvideo: Failed to initialize the device (-5).

                   

                   

                  Where I have gone wrong?

                   

                  This is my MIP CSI-2 configuration settings :

                   

                  /* OV2311_RAW10_2MP :  */

                  CyU3PMipicsiCfg_t OV2311_RAW10_2MP = 

                  {

                      CY_U3P_CSI_DF_RAW10,  /* CyU3PMipicsiDataFormat_t dataFormat */

                      2,                          /* uint8_t numDataLanes */

                      2, /* uint8_t pllPrd */

                      89, /* uint16_t pllFbd */

                      CY_U3P_CSI_PLL_FRS_250_500M, /* CyU3PMipicsiPllClkFrs_t pllFrs */ 

                      CY_U3P_CSI_PLL_CLK_DIV_4, /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

                      CY_U3P_CSI_PLL_CLK_DIV_4, /* CyU3PMipicsiPllClkDiv_t parClkDiv */

                      0,                 /* uint16_t mClkCtl */

                      CY_U3P_CSI_PLL_CLK_DIV_2, /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

                      1600,         /* uint16_t hResolution */

                      50                         /* uint16_t fifoDelay */

                  };

                   

                   

                  This is my USB descriptor :

                   

                      /* Class specific Uncompressed VS format descriptor */

                          0x1B,                               /* Descriptor size */

                          CX3_CS_INTRFC_DESCR,                /* Class-specific VS interface Type */

                          0x04,                               /* Subtype : VS_FORMAT */

                          0x01,                               /* Format desciptor index */

                          0x01,                               /* Number of Frame Descriptors that follow this descriptor: 1 */

                   

                          /* GUID, globally unique identifier used to identify streaming-encoding format*/

                          0x59, 0x55, 0x59, 0x32, /*MEDIASUBTYPE_YUY2 GUID: 32595559-0000-0010-8000-00AA00389B71 */

                  0x00, 0x00, 0x10, 0x00,

                  0x80, 0x00, 0x00, 0xAA,

                  0x00, 0x38, 0x9B, 0x71,

                   

                          0x10,                               /* Number of bits per pixel: 16*/

                          0x01,                               /* Optimum Frame Index for this stream: 1 (2MP) */

                          0x00,                               /* X dimension of the picture aspect ratio; Non-interlaced */

                          0x00,                               /* Y dimension of the pictuer aspect ratio: Non-interlaced */

                          0x00,                               /* Interlace Flags: Progressive scanning, no interlace */

                          0x00,                               /* duplication of the video stream restriction: 0 - no restriction */

                   

                          /* Class specific Uncompressed VS Frame Descriptor 1 - 2MP */

                          0x1E,                               /* Descriptor size */

                          CX3_CS_INTRFC_DESCR,                /* Descriptor type*/

                          0x05,                               /* Subtype:  frame interface*/

                          0x01,                               /* Frame Descriptor Index: 1 */

                          0x00,                               /* No Still image capture method supported */

                          0x40,0x06,                         /* Width in pixel:  1600 */

                          0xb0,0x04,                         /* Height in pixel: 1200 */

                          0x00,0x80,0xee,0x36,             /* Min bit rate (bits/s): 1600 x 1200 x No Of Bits per Pixel x FrameRate = 921600000 */

                          0x00,0x80,0xee,0x36,             /* Max bit rate (bits/s): Fixed rate so same as Min */

                          0x00,0x98,0x3a,0x00,             /* Maximum video or still frame size in bytes(Deprecated): 1600 x 1200 x 2 */

                          0x15,0x16,0x05,0x00,             /* Default frame interval (in 100ns units): (1/30)x10^7 */

                          0x01,                               /* Frame interval type : No of discrete intervals */

                          0x15,0x16,0x05,0x00,             /* Frame interval 3: Same as Default frame interval */

                   

                         /* Still image descriptor -YUV with QVGA resolution */

                      0x0b, /*Size of this descriptor*/

                      0x24, /*CS_INTERFACE descriptor type*/

                      0x03, /*VS_STILL_IMAGE_FRAME descriptor subtype*/

                      0x00, /*If method 3 is used, endpoint address is mentioned here*/

                      0x01, //No of frame Resolutions Follows

                   

                      0x40,0x06, /* Width in pixel:  1600=640*/ //1

                      0xb0,0x04, /* Height in pixel: 1200=4b0*/

                   

                      0x01, /*Compression patterns supported*/

                      0x00, /*No compression*/

                   

                   

                  And is my UVC probe control settings :

                   

                  /* UVC Probe Control Setting - 2MP*/

                  uint8_t const gl2MPProbeCtrl[CX3_UVC_MAX_PROBE_SETTING] = {

                      0x00, 0x00,                         /* bmHint : No fixed parameters */

                      0x01,                               /* Use 1st Video format index */

                      0x01,                               /* Use 1st Video frame index */

                      0x15,0x16,0x05,0x00,             /* Desired frame interval in 100ns = (1/30)x10^7 */

                      0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

                      0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

                      0x00, 0x00,                         /* Compression quality control */

                      0x00, 0x00,                         /* Window size for average bit rate */

                      0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

                      0x00,0x98,0x3a,0x00,             /* Max video frame size in bytes = 1920 x 1080 x 2 */

                  #ifdef CX3_UVC_1_0_SUPPORT

                      0x00, 0x90, 0x00, 0x00              /* No. of bytes device can rx in single payload: 36KB */

                  #else

                      /* UVC 1.1 Probe Control has additional fields from UVC 1.0 */

                      0x00, 0x90, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 36KB */

                      0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

                      0x00,                               /* Framing Information - Ignored for uncompressed format*/

                      0x00,                               /* Preferred payload format version */

                      0x00,                               /* Minimum payload format version */

                      0x00                                /* Maximum payload format version */

                  #endif

                  };

                   

                   

                  Please help in this regard as I have been stuck here for quite some time now.

                  • 6. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                    RashiV_61

                    Hello,

                     

                    Please let me know where in the firmware (or near which APIs) do you print these messages.

                    [16967.460637] uvcvideo: UVC noncompliance - GET_DEF(PROBE) not supported. Enabling workaround.

                    [16967.460750] uvcvideo: Failed to query (129) UVC probe control : -32 (exp. 34).

                    [16967.460755] uvcvideo: Failed to initialize the device (-5).

                     

                    If possible please share the firmware files (without the sensor configuration) for us to check. Also,. confirm if you use the default OV5640 project of the SDK as the base or generate the firmware using the CX3  MIPI configuration tool as per this KBA Steps to Setup up MIPI CSI Camera Solution with CX3 – KBA225748 or is it a custom firmware

                     

                     

                    Also, let me know which host application are you using for video streaming.

                     

                     

                     

                     

                    Regards,

                    Rashi

                    • 7. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                      NiNa_4798196

                      Hi,

                       

                      I'm creating firmware using CX3 MIPI configuration tool as per KBA listed above.

                       

                      I have managed to get past this error now. I added my 1600*1200 resolutions in switch cases in cycx3_uvc.c files.

                      Now I am able to see /dev/video2 and /dev/video3 device.

                       

                      However, eCam View shows a black screen.

                       

                      I tried using gstreamer to capture video as but the output got stuck as follows :

                       

                      cmd >gst-launch-1.0 v4l2src device=/dev/video2 num-buffers=100 ! video/x-raw,width=1600,height=1200 ! filesink location=~/test_h264.avi

                      o/p>

                      Setting pipeline to PAUSED

                      Pipeline is live and does not need PREROLL

                      Setting pipeline to PLAYING

                      New clock: GstSystemClock   (Got Stuck at this point)

                       

                      How do I move forward in debugging this issue?

                       

                      Regards,

                      Ninad

                      • 8. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                        RashiV_61

                        Hello,

                         

                        Please share the UART debug prints when the streaming is started and this error is seen on the host side.

                        Please enable the macro CX3_DEBUG_ENABLED and build the firmware again before starting the streaming

                         

                        Regards,

                        Rashi

                        • 9. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                          NiNa_4798196

                          Hi,

                           

                          These are my UART logs whenever I start streaming,

                           

                           

                           

                          AppInit:GpifSMStart passed

                          bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                          bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                          bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                          bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                          Sensor Initialization

                          Sensor Sleep

                          Event Flag = 0x20

                          Sensor Sleep

                          EnterÓÕÍÁ¹5½?ÑÕ?êÂÁbºÕÁ?¹êÂáRjTe¹Ñ2±êÂÉj¤iUËk×KÒ*«V.¡CQ·ºerSuspendMode Status =  0x0, Wakeup reason = 0x8

                          bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                          bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                          bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                          bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                          bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                          bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                          Event Flag ýÂÉj¤iUËk×KÒ*«V.¡CQ·ºerSuspendMode Status =  0x0, Wakeup reason = 0x8

                          bRType = 0xA1, bRequest = 0x81, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                          bRType = 0x21, bRequest = 0x1, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                          bRType = 0xA1, bRequest = 0x81, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                          bRType = 0xA1, bRequest = 0x83, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                          bRType = 0xA1, bRequest = 0x82, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                          bRType = 0x21, bRequest = 0x1, wValue = 0x200, wIndex = 0x1, wLength= 0x22

                          Sensor Initialization : 2MP

                          AplnStrt:SMState = 0x2

                          Sensor Wakeup

                          Event Flag = 0x10

                          CB failure : Event Flag = 16

                          glIsApplnActive = 1

                          Ninad : App Stop

                          Sensor Sleep

                          AplnStop:SMState = 0x5

                          Ninad : App Start

                          AplnStrt:SMState = 0x1

                          Sensor Wakeup

                          Event Flag = 0x10

                          CB failure : Event Flag = 16

                          glIsApplnActive = 1

                          Ninad : App Stop

                          Sensor Sleep

                          AplnStop:SMState = 0x5

                          Ninad : App Start

                          AplnStrt:SMState = 0x1

                          Sensor Wakeup

                          Event Flag = 0x10

                          CB failure : Event Flag = 16

                          glIsApplnActive = 1

                          Ninad : App Stop

                          Sensor Sleep

                          AplnStop:SMState = 0x5

                          Ninad : App Start

                          AplnStrt:SMState = 0x1

                          Sensor Wakeup

                          Event Flag = 0x10

                          CB failure : Event Flag = 16

                           

                          These keep on repeating continuously until I close the streaming app.

                          We probed the MIPI clock, and we're correctly getting 400MHz over there. And the sensor is streaming some data.

                          But on the Host side with eCam we can only see a black screen. As well, with v4l2 we're not able to get any dumps.

                          • 10. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                            RashiV_61

                            Hello,

                             

                            From the debug prints it seems that the CX3_DMA_RESET_EVENT event is triggered due to the failure of CyU3PDmaMultiChannelCommitBuffer API.

                            Can you please check the return value of this API so that we can know the error code? For checking the return value use a global variable to copy the return value f the API when it fails (doesn't return success) and print the global variable along with the print "Cb Failure"

                             

                            Also, to check if the frame size and frame rate received by the CX3 from the sensor is as expected, please enable the PRINT_FRAME_INFO macro and share the debug prints.

                             

                            Regards,

                            Rashi

                            • 11. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                              NiNa_4798196

                              Hello,

                               

                              This is the return value for CyU3PDmaMultiChannelCommitBuffer API.

                               

                              CB failure : CyU3PDmaMultiChannelCommitBuffer = 71

                               

                              Also, the 'PRINT_FRAME_INFO' is already enabled.

                               

                              Debug logs :

                               

                              AppInit:GpifSMStart passed

                              Chip ID High Read= 0x2311

                              bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                              bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                              bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100 ÒªëÂ)jIQåÁêÂÅbIÅÕÍÑêÂá

                                     bºY±ÕêÂÉbº%¹áêÂÁbº1¹Ñ¡õÂÅRjT?ÉMÕÍU¹5½?ÑÕ?êÂÁbºÕÁ?¹êÂáRjT?ÉMÕÍU¹5½?ÑÕ?êÂÁbºÕÁ?¹êÂáþ

                              bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                              bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                              bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                              bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                              bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

                              bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

                              EnterSuspendMode Status =  0x0, Wakeup reason = 0x8

                              bRType = 0xA1, bRequest = 0x81, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                              bRType = 0x21, bRequest = 0x1, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                              bRType = 0xA1, bRequest = 0x81, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                              bRType = 0xA1, bRequest = 0x83, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                              bRType = 0xA1, bRequest = 0x82, wValue = 0x100, wIndex = 0x1, wLength= 0x22

                              bRType = 0x21, bRequest = 0x1, wValue = 0x200, wIndex = 0x1, wLength= 0x22

                              Chip ID High Read= 0x2311

                              AplnStrt:SMState = 0x2

                              CB failure : CyU3PDmaMultiChannelCommitBuffer = 71

                              AplnStop:SMState = 0x5

                              AplnStrt:SMState = 0x1

                              CB failure : CyU3PDmaMultiChannelCommitBuffer = 71

                              AplnStop:SMState = 0x5

                              AplnStrt:SMState = 0x1

                              CB failure : CyU3PDmaMultiChannelCommitBuffer = 71

                              AplnStop:SMState = 0x5

                              AplnStrt:SMState = 0x1

                               

                               

                              Regards,

                              Ninad

                              • 12. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                                RashiV_61

                                Hello Ninad,

                                 

                                Please refer to this KBA Invalid Sequence Error in Multi-Channel Commit Buffer - KBA218830 which mentions the reason of commit buffer Api failure with error code 71.

                                To follow the first point for recovering from this error,  the DMA buffer size needs to be increased.

                                Please let me know what is current buffer size used in the firmware.

                                 

                                Also, 'PRINT_FRAME_INFO' is already enabled.

                                >> If the firmware is generated from the CX3 MIPI configuration tool, on enabling the PRINT_FRAME_INFO the below-mentioned prints will be enabled

                                CyU3PDebugPrint(4,"\n\rProd = %d Cons = %d  Prtl_Sz = %d Frm_Cnt = %d Frm_Sz = %d B", TxCountflag, RxCountflag, PartialBufSize, FrameCount, ((TxCountflag*CX3_UVC_DATA_BUF_SIZE)+PartialBufSize));

                                 

                                Please refer to the attached file generated from the tool which prints the information about the frames received and confirm if the same implementation is done in your project.

                                If the same implementation is being done in your firmware this means the end of the frame is not reached which prints the above-mentioned prints ( by enabling Printflag = 1).

                                 

                                In this case, please let me know whether the device enumerates as a USB 3.0 device or a USB 2.0 device.

                                You can call CyU3PUsbGetSpeed after CyCx3UvcAppInit or CyU3PConnectState to check the device connection speed.

                                 

                                Regards,

                                Rashi

                                • 13. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                                  NiNa_4798196

                                  Hello Rashi,

                                   

                                  I followed this (Invalid Sequence Error in Multi-Channel Commit Buffer - KBA218830) Still I could see

                                   

                                  CB failure : CyU3PDmaMultiChannelCommitBuffer = 71

                                   

                                  Current DMA buffer size is 36838

                                   

                                  The device enumerates as Full speed device

                                  I'm connecting 3.0 cable to 3.0 port. I have checked with both orientations

                                  • 14. Re: Integrating OV2311 with EZ-USB SDK CX3 project
                                    NiNa_4798196

                                    Hi Rashi,

                                     

                                    I have an update. We made some corrections on the hardware.

                                     

                                    The device is now detected as a USB2.0 device. But CyU3PUsbGetSpeed still return '1' as its return value.

                                     

                                    I have added sensor configuration for 640*480@30fps for UsbHS descriptor and on 'guvcview' on Ubuntu I am able to see some stream. But this stream hangs after some time and then the application itself crashes.

                                     

                                    But still, the UART logs are exactly same as above (during streaming) and this following line is still not getting printed

                                     

                                    CyU3PDebugPrint(4,"\n\rProd = %d Cons = %d  Prtl_Sz = %d Frm_Cnt = %d Frm_Sz = %d B", TxCountflag, RxCountflag, PartialBufSize, FrameCount, ((TxCountflag*CX3_UVC_DATA_BUF_SIZE)+PartialBufSize));

                                     

                                    The implementation is same as in the file shared by you.

                                    Please guide me in where should I debug the issues next.

                                     

                                    Regards,

                                    Ninad

                                    1 2 Previous Next