Streaming RAW image data using Cypress Driver

[ScGr_289066]

Hi All,

I have modified the original project's firmware (found here: FX3 / CX3 Firmware for Streaming RAW Image Data using Cypress Driver to use the OV5647 sensor and am writing a simple PC app to receive and display the sensor data.  I have questions about the use of the CyAPI functions.

After enumerating the USB bus and finding the Streamer Example Device, I am sending the vendor command 0x99 to start streaming.  My target device receives and acknowledges the 99, but the PC side reports an error and says no data was sent.  Also, what is the appropriate API call to receive the camera video data?  I assume it is a dev.BulkInEndPt -> XferData( fbuf, fbufsiz ) transfer to alternating 16K buffers (or sensor frame sized buffers).  Is that correct?

Any comments on what's happening are most appreciated.

Here's the code to send the vendor command:

BOOL send_vendor( CCyControlEndPoint *ept, BYTE cmd )

{

    UCHAR    buf[ 128 ], rbuf[ 128 ];

    LONG    buflen = sizeof( buf ), rbufsiz = sizeof( rbuf );

    BOOL    rls;

    memset( buf, 0, buflen );

    ept -> Target = TGT_DEVICE;

    ept -> ReqType = REQ_VENDOR;

    ept -> Direction = DIR_TO_DEVICE;

    ept -> ReqCode = cmd;                                                    //    vendor command

    ept -> Value = 0;

    ept -> Index = 0;

    ept -> TimeOut = 300;

//    rls = ept -> XferData( (UCHAR *) buf, (LONG &) buflen );

    rls = ept -> Write( buf, buflen );

    printf( "Vendor %02X, status %d, sent %d bytes\n", cmd, rls, buflen );

    return( rls );

}

And the script of running the program:

There are 3 devices attached

Device 1, VID 4B4, PID 07: :USB-Serial (Dual Channel) Vendor 1

0 alternate interfaces

1 interfaces

Device 2, VID 4B4, PID 07: :USB-Serial (Dual Channel) Vendor MFG

0 alternate interfaces

1 interfaces

Device 3, VID 4B4, PID F1: SS:Cypress FX3 USB StreamerExample Device

0 alternate interfaces

1 interfaces

Opening Cypress FX3 USB StreamerExample Device

Bulk in endpoint

Vendor 99, status 0, sent 0 bytes

and the debug printout from the device:

bRType = 0x40, bRequest = 0x99, wValue = 0x0, wIndex = 0x0, wLength= 0x80Start streaming

AplnStrt:SMState = 0x2Stop streaming

AplnStop:SMState = 0x5Start streaming

This pattern continues to repeat.

Thanks,

Scott

[SrinathS_16]

Hello Scott,

- The Write() API in turn calls the XferData() API. Set the timeout value of the endpoint to be higher or modify the vendor command handling in the firmware such that the CyU3PUsbAckSetup() statement is at the beginning.

if  (bRequest == 0x99)

        {

        CyU3PUsbAckSetup();

        ...

- Also, modify the buflen parameter of the Write() API to be 0 as the device firmware is not configured to receive any data from the host using this vendor command.

- When the streaming stops, please let me know if there is a failure on any of the DMA related APIs. Complete UART logs would also be useful.

Best regards,
Srinath S

[ScGr_289066]

Hi Srinath,

You suggestion has helped.  I rearranged the firmware to acknowledge before starting the app (e.g. call CyU3PUsbAckSetup() first) and set the write buffer length to 0, now the API signals success in sending the start streaming command.

However, I am still not receiving any video data from the camera.  Here's my brief code:

send_vendor( dev.ControlEndPt, 0x99 );             this now succeeds

	unsigned char    fbuf[ 16384 ];
	LONG		fbufsiz = sizeof( fbuf );

printf( "Read buffer %d, bytes %d\n", dev.BulkInEndPt -> XferData( fbuf, fbufsiz ), fbufsiz );

The xferdata call times out (10 seconds) and sets fbufsiz to 0.  Is there something else I need to do?

Thanks,

Scott

[SrinathS_16]

Hello Scott,

- Can you please use the Cypress USB Control Center to read data from the endpoint and check if there is any data received? This will help us isolate the problem.

- Also, when issuing the XferData() API, please provide the UART logs from the device.

- As mentioned earlier, please check if any of the DMA channel APIs are failing.

Best regards,

Srinath S

[ScGr_289066]

Hi Srinath,

So using control center to send the vendor command 99, the app reacts:

bRType = 0x40, bRequest = 0x99, wValue = 0x0, wIndex = 0x0, wLength= 0x0Start streaming

AplnStrt:SMState = 0x2

bRType = 0xC0, bRequest = 0xA0, wValue = 0xE600, wIndex = 0x0, wLength= 0x1

bRType = 0x40, bRequest = 0x99, wValue = 0x0, wIndex = 0x0, wLength= 0x200Start streaming

AplnStrt:SMState = 0x2

bRType = 0x40, bRequest = 0x99, wValue = 0x0, wIndex = 0x0, wLength= 0x200Start streaming

AplnStrt:SMState = 0x2

But control center reports a failure.

Then attempting to read using control center:

Thanks,

Scott

[ScGr_289066 · Accepted solution]

Hi Srinath,

Here's an update.  Originally, the camera resolution wasn't matching the MIPI settings and that is what was causing all the problems.  Having ironed that out, I have successfully transferred frames of image data and been able to de-Bayer the data into viewable color-correct images by simply performing multiple XferData() calls until a full frame is received then de-Bayering it and rendering it with OpenCV.

Today I have rewritten the app using two threads one to collect data from the CX3 and a second to render (de-Bayer & display) the resulting frames.  It is working pretty well in that it can keep up with the incoming data (1080p 30fps).  I am however having odd sync issues that I haven't yet resolved.

I am in the process of adding more vendor commands to the PC app and firmware to control camera resolution, test patterns, etc.  No unexplained behavior that I can put my finger on, so this case is closed.

Thanks,

Scott