5 Replies Latest reply on Feb 15, 2019 11:47 AM by ScGr_289066

    Streaming RAW image data using Cypress Driver

    ScGr_289066

      Hi All,

       

      I have modified the original project's firmware (found here: FX3 / CX3 Firmware for Streaming RAW Image Data using Cypress Driver) to use the OV5647 sensor and am writing a simple PC app to receive and display the sensor data.  I have questions about the use of the CyAPI functions.

       

      After enumerating the USB bus and finding the Streamer Example Device, I am sending the vendor command 0x99 to start streaming.  My target device receives and acknowledges the 99, but the PC side reports an error and says no data was sent.  Also, what is the appropriate API call to receive the camera video data?  I assume it is a dev.BulkInEndPt -> XferData( fbuf, fbufsiz ) transfer to alternating 16K buffers (or sensor frame sized buffers).  Is that correct?

       

      Any comments on what's happening are most appreciated.

       

      Here's the code to send the vendor command:

       

      BOOL send_vendor( CCyControlEndPoint *ept, BYTE cmd )

      {

          UCHAR    buf[ 128 ], rbuf[ 128 ];

          LONG    buflen = sizeof( buf ), rbufsiz = sizeof( rbuf );

          BOOL    rls;

       

          memset( buf, 0, buflen );

       

          ept -> Target = TGT_DEVICE;

          ept -> ReqType = REQ_VENDOR;

          ept -> Direction = DIR_TO_DEVICE;

          ept -> ReqCode = cmd;                                                    //    vendor command

          ept -> Value = 0;

          ept -> Index = 0;

       

          ept -> TimeOut = 300;

      //    rls = ept -> XferData( (UCHAR *) buf, (LONG &) buflen );

          rls = ept -> Write( buf, buflen );

          printf( "Vendor %02X, status %d, sent %d bytes\n", cmd, rls, buflen );

       

          return( rls );

      }

       

      And the script of running the program:

       

      There are 3 devices attached

      Device 1, VID 4B4, PID 07: :USB-Serial (Dual Channel) Vendor 1

      0 alternate interfaces

      1 interfaces

      Device 2, VID 4B4, PID 07: :USB-Serial (Dual Channel) Vendor MFG

      0 alternate interfaces

      1 interfaces

      Device 3, VID 4B4, PID F1: SS:Cypress FX3 USB StreamerExample Device

      0 alternate interfaces

      1 interfaces

      Opening Cypress FX3 USB StreamerExample Device

      Bulk in endpoint

      Vendor 99, status 0, sent 0 bytes

       

      and the debug printout from the device:

       

      bRType = 0x40, bRequest = 0x99, wValue = 0x0, wIndex = 0x0, wLength= 0x80Start streaming

       

      AplnStrt:SMState = 0x2Stop streaming

       

      AplnStop:SMState = 0x5Start streaming

      This pattern continues to repeat.

       

      Thanks,

      Scott

        • 1. Re: Streaming RAW image data using Cypress Driver
          SrinathS_16

          Hello Scott,

           

          - The Write() API in turn calls the XferData() API. Set the timeout value of the endpoint to be higher or modify the vendor command handling in the firmware such that the CyU3PUsbAckSetup() statement is at the beginning.

          if  (bRequest == 0x99)

                  {

                  CyU3PUsbAckSetup();

                  ...

          - Also, modify the buflen parameter of the Write() API to be 0 as the device firmware is not configured to receive any data from the host using this vendor command.

          - When the streaming stops, please let me know if there is a failure on any of the DMA related APIs. Complete UART logs would also be useful.

           

          Best regards,
          Srinath S

          1 of 1 people found this helpful
          • 2. Re: Streaming RAW image data using Cypress Driver
            ScGr_289066

            Hi Srinath,

             

            You suggestion has helped.  I rearranged the firmware to acknowledge before starting the app (e.g. call CyU3PUsbAckSetup() first) and set the write buffer length to 0, now the API signals success in sending the start streaming command.

            However, I am still not receiving any video data from the camera.  Here's my brief code:

             

                send_vendor( dev.ControlEndPt, 0x99 );             this now succeeds

             

               unsigned char    fbuf[ 16384 ];
               LONG   fbufsiz = sizeof( fbuf );

             

               printf( "Read buffer %d, bytes %d\n", dev.BulkInEndPt -> XferData( fbuf, fbufsiz ), fbufsiz );

             

            The xferdata call times out (10 seconds) and sets fbufsiz to 0.  Is there something else I need to do?

             

            Thanks,

            Scott

            • 3. Re: Streaming RAW image data using Cypress Driver
              SrinathS_16

              Hello Scott,

               

              - Can you please use the Cypress USB Control Center to read data from the endpoint and check if there is any data received? This will help us isolate the problem.

              - Also, when issuing the XferData() API, please provide the UART logs from the device.

              - As mentioned earlier, please check if any of the DMA channel APIs are failing.

               

              Best regards,

              Srinath S

              • 4. Re: Streaming RAW image data using Cypress Driver
                ScGr_289066

                Hi Srinath,

                So using control center to send the vendor command 99, the app reacts:

                 

                bRType = 0x40, bRequest = 0x99, wValue = 0x0, wIndex = 0x0, wLength= 0x0Start streaming

                 

                AplnStrt:SMState = 0x2

                bRType = 0xC0, bRequest = 0xA0, wValue = 0xE600, wIndex = 0x0, wLength= 0x1

                bRType = 0x40, bRequest = 0x99, wValue = 0x0, wIndex = 0x0, wLength= 0x200Start streaming

                 

                AplnStrt:SMState = 0x2

                bRType = 0x40, bRequest = 0x99, wValue = 0x0, wIndex = 0x0, wLength= 0x200Start streaming

                 

                AplnStrt:SMState = 0x2

                 

                But control center reports a failure.

                 

                Then attempting to read using control center:

                 

                 

                Thanks,

                Scott

                • 5. Re: Streaming RAW image data using Cypress Driver
                  ScGr_289066

                  Hi Srinath,

                   

                  Here's an update.  Originally, the camera resolution wasn't matching the MIPI settings and that is what was causing all the problems.  Having ironed that out, I have successfully transferred frames of image data and been able to de-Bayer the data into viewable color-correct images by simply performing multiple XferData() calls until a full frame is received then de-Bayering it and rendering it with OpenCV.

                   

                  Today I have rewritten the app using two threads one to collect data from the CX3 and a second to render (de-Bayer & display) the resulting frames.  It is working pretty well in that it can keep up with the incoming data (1080p 30fps).  I am however having odd sync issues that I haven't yet resolved.

                   

                  I am in the process of adding more vendor commands to the PC app and firmware to control camera resolution, test patterns, etc.  No unexplained behavior that I can put my finger on, so this case is closed.

                   

                  Thanks,

                  Scott