I've been developing a camera interface using Madhu Sudhan's RAW data driver firmware (FX3 / CX3 Firmware for Streaming RAW Image Data using Cypress Driver ) and have been making some progress. I have the OV5647 sensor streaming 5Mp, 1080p and now 720p RAW10 data to a PC running an application I've written using CyAPI and OpenCV. The PC app collects the streaming data and converts it to RGB format for display. I believe I have a post describing this in more detail.
I have some questions I'm hoping you can help me answer. I'll provide some background.
Of these three resolutions, I am getting two different behaviors. At 5Mp, the system (firmware and PC app) runs for a random period of time before CyAPI reports error 0x1F (31). Once this error ocurrs, firmware must be reloaded before any more image data can be transferred. Transfers typically proceed many tens of minutes before there's an error, and there are only a few detected DMA timeouts (error 0x3E3/995) at the onset of streaming, until the stream is established. This thread discusses the issue in detail: Streaming RAW image data using Cypress Driver part 2 and I've accepted the probably cause that was proposed in the thread that the app isn't grabbing the data quickly enough at times.
For all resolutions there are DMA timeouts (995) when the stream starts, but for 1080p and 720p these errors continue after the stream starts and continue during streaming. The result is very confusing. For 1080p, there seems to be no effect, the frame rate nearly matches the theoretical range from the sensor. But for 720p, the frame rate is seriously degraded by the constant errors & firmware recovery. These errors got me to wonder is the frames the CX3 is receiving from the sensor are corrupted. Since it's very difficult to probe the sensor's MIPI signals I began monitoring the HSYNC test point to count the number of scan lines in each frame. Using a deep-memory scope, I can monitor the HSYNC test point for many tens or hundreds of frames and see the individual horizontal transfer pulses with enough resolution to count and verify their numbers over the course of tens or hundreds of frames. This has led me to the inconsistency I mentioned in the title.
Given the behavior of the system for different resolutions, I'd expect to see perfect frames at 5Mp (since there are no errors I expect 1944 lines/frame) while the stream is running. Conversely, for 1080p and 720p resolutions, maybe there are incorrect numbers of scanned lines for these resolutions causing firmware to get out of sync and timeout. Since 1080p is better than 720p I'd expect its horizontal transfer counts to deviate from 1080 less often than 720p deviates from 720. This isn't what I found.
At all three resolutions, I am measuring 10 to 30% of the frames contain wrong numbers of scanned lines. This makes me wonder about the integrity of this test point. Is it reporting the wrong numbers of lines in each frame? Does it not match what firmware "sees"? If this high of a percentage of scanned lines is wrong, why don't ALL resolutions continually trigger error 995?
Can anyone explain what's wrong with my reasoning for this apparent paradox?
In order to confirm whether the sensor is streaming correctly, we can probe the HSYNC, VSYNC, and PCLK pins and calculate whether these signal are following proper timings (w.r.t resolution).
If you think that there is signal integrity on test pins, you can follow the other way.
i.e. Count the number of bytes received in DMA callback per frame.
--- Start a counter in first DMA Producer Event and increment the counter in consecutive DMA Prod. Events until you receive the partial buffer.
--- Once you receive the partial buffer, print the counter value in main thread (for loop).
In the DMA call back, discard the received buffer once you increment the counter. This avoids the dependancy of data flow between host and device. Hence, we can focus on the data flow between the sensor and device.
Once everything is working fine between device and sensor, commit the data to USB IN endpoint. Then, we shall debug on data flow between host and device.
Regarding the errors:
1. 31 (0x1F)
A device attached to the system is not functioning.
We need to know the state of the device and address this issue.
Use Software USB 3.0 analyzer USBlyzer - USB Protocol Analyzer and USB Traffic Sniffer for Windows and capture the trace.
2. 995 (0x3E3)
The I/O operation has been aborted because of either a thread exit or an application request.
Note that 995 is not a time out error. This is due to the operation is aborted.
Error Codes Reference: System Error Codes (500-999) - Windows applications | Microsoft Docs
I enabled printing frame information while capturing 5MP. There are 410 producer events and 409 consumer events with a partial buffer of 1536 bytes. This matches exactly the 2592x1944x2=410*24576+1536 byte frames I expect. Every frame reports the same values, except when the very occasional error 995 occurrs.
Unlike the other day, today when I collected HSYNC traces over several tens of seconds, I never see frames with incorrect numbers of scanned lines. I tried both 5MP and 1080p resolutions. I saw only a half dozen or so frames out of thousands that contained incorrect numbers of scanned lines, but now I believe they are the result of the random error 995 when firmware stops and restarts streaming.
I tried several times to discard DMA buffers without any luck. All my attempts caused the system to stop working. Can you be more specific how the code should be modified to toss the data?
I installed USBlyzer and collected a trace while running the camera at 5MP. It was most unhelpful. I have attached screen shots of the start of scan, and the end of scan caused when error 31 was reported along with my app's output.
the above frame is a capture at the start of scanning when several (3) error 995s are reported and firmware recovers.
Followed by many hundreds of successful buffer transfers:
Then finally the fatal error 31:
Finally, my App's output:
Opening Cypress FX3 USB StreamerExample Device
Version 2.000, Released 04/05/2019
Last error: 995
Last error: 31
Last error: 995
Stop CX3 1
Thanks for your help,
Can you please share UVC.c file in which you are handling discarding the DMA buffer?
I would like to see how you are handling the same.
Here's my DMA callback routine:
void CyCx3UvcAppDmaCallback ( CyU3PDmaMultiChannel *chHandle, CyU3PDmaCbType_t type, CyU3PDmaCBInput_t *input )
CyU3PReturnStatus_t status = CY_U3P_SUCCESS;
if (type == CY_U3P_DMA_CB_PROD_EVENT)
/* This is a produce event notification to the CPU. This notification is
* received upon reception of every buffer. The buffer will not be sent
* out unless it is explicitly committed. The call shall fail if there
* is a bus reset / usb disconnect or if there is any application error. */
/* Disable USB 3.0 LPM while Buffer is being transmitted out*/
if ((CyU3PUsbGetSpeed () == CY_U3P_SUPER_SPEED) && (doLpmDisable))
doLpmDisable = CyFalse;
CyU3PTimerStop( &UvcTimer );
CyU3PTimerModify( &UvcTimer, RESET_TIMER_ENABLE, 0 );
CyU3PTimerStart( &UvcTimer );
if (gettimeflag == 0)
time0 = CyU3PGetTime();
//framecount = FrameCount;
gettimeflag = 1;
status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);
while (status == CY_U3P_SUCCESS)
/* Add Headers*/
if( dmaBuffer.count < CX3_UVC_DATA_BUF_SIZE )
CyCx3UvcAppAddHeader ((dmaBuffer.buffer - CX3_UVC_PROD_HEADER), CX3_UVC_HEADER_EOF);
glHitFV = CyTrue;
PartialBufSize = dmaBuffer.count;
RxCountflag = RxCount;
TxCountflag = TxCount;
Printflag = 1;
if ((FrameCount % 30) == 0)
time1 = CyU3PGetTime();
fpsflag = 1;
gettimeflag = 0;
CyCx3UvcAppAddHeader ((dmaBuffer.buffer - CX3_UVC_PROD_HEADER), CX3_UVC_HEADER_FRAME);
// Commit Buffer to USB
// ******** modifed to discard USB data ***************
// status = CyU3PDmaMultiChannelCommitBuffer( chHandle, ( dmaBuffer.count ), 0 );
status = CyU3PDmaChannelDiscardBuffer( chHandle );
if ( status != CY_U3P_SUCCESS )
CyU3PDebugPrint ( 4, "DMA commit fail %x, size %d, count %d\r\n", status, dmaBuffer.size, dmaBuffer.count );
glActiveSocket ^= 1; /* Toggle the Active Socket */
status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);
else if(type == CY_U3P_DMA_CB_CONS_EVENT)
initio( 53, true, green1 );
green1 ^= 1;
/* Check if Frame is completely transferred */
glIsStreamingStarted = CyTrue;
if((glHitFV == CyTrue) && (glDmaDone == 0))
TxCount = 0;
RxCount = 0;
glHitFV = CyFalse;
CX3_INVALID_GPIF_STATE, ALPHA_CX3_START_SCK1, CX3_GPIF_SWITCH_TIMEOUT);
CX3_INVALID_GPIF_STATE, ALPHA_CX3_START_SCK0, CX3_GPIF_SWITCH_TIMEOUT);
doLpmDisable = CyTrue;
CyU3PTimerModify (&UvcTimer, RESET_TIMER_ENABLE, 0);
I modify it to toss (discard) the data by changing the CyU3PDmaMultiChannelCommitBuffer to CyU3PDmaChannelDiscardBuffer. As I said, this hadn't worked, so there must be more to it?
What's happening with this?
I have since implemented a new sensor resolution, 720p, which has a much more severe problem with halting than the other 5M and 1080p resolutions. The system typically runs for only a minute or two before reporting error 31. As before, firmware must be reloaded to restart image streaming.
While implementing 720p, I discovered that the DMA buffer size *must not* be a exact multiple of the total image byte size otherwise there are constant timer restarts (e.g. UvcAppProgressTimer() time-outs). This isn't mentioned in the documentation anywhere that I discovered.
Having corrected that issue to prevent streaming timeouts, I am now faced with the frequent error 31 issues. USBLyzer traces don't shed light on what's happening. All I see are "Unsuccessful (babble detected)" messages with no explanation of what babble is.
In a default UVC firmware, we are adding header (UVC Header, EOF header) in DMA Callback. Hence, we add EOF header upon detecting the partial buffer in the dma callback. Therefore, we need to to keep the buffer size in such a way that the frame size is not integer multiple of buffer size.
Upon detecting the partial buffer, we set glHitFV;
In the producer event, we look for glHitFV and dmadone == 0, then stop the timer; switch the GPIF state machine and modify the timer.
Yes, this is not documented in UVC appnote but implemented in AN7579. We will consider this in next revision.
Please let me know where exactly you are seeing this error in your host application.
Can you please use the streamer application and do streaming on IN endpoint then check whether there are 31 errors.
When you checked over web, the error 31 is related to host controller. Can you please have a look here? USB HOST CONTROLLER ERROR 31. - Microsoft Community
By the way what is the host controller and operating system you are using for the test? Can you please try with any other host controller and/or OS?