I have a sensor Aptina MT9V034 that is configured to run at 60 fps. I used a Cypress FX3 to control the sensor and communicate with an android application which uses UVCCam library.
I use the source code of AN75779 example which is compatible with UVC.
When using a software on pc to read the video from the camera it works fine without a problem.
However on my android application, after getting the frame of camera I do some image processing so my android application processes only around 30 fps (not fixed) and I get occasionally error code 71 which means CY_U3P_ERROR_INVALID_SEQUENCE after calling the function CyU3PDmaMultiChannelCommitBuffer. This error causes the frame sent by camera looks abnormally but until another error of invalid sequence occurs, the frame returns to normal.
As I understand the error is caused by the slow consumption of frame of my android application.
If that is the case which is the best solution to get rid of the error or to handle the error correctly so that I do not have to lower the frame rate of the sensor and the android application's slow consumption of frame doesn't affect the frame sent by the camera ?
Thank you very much.
Error code 71:CY_U3P_ERROR_INVALID_SEQUENCE corresponds to a scenario when the host is not reading the data fast enough. In this case, the DMA buffers gets filled and there is an overrun scenario created, and you loose data unless the buffer are free again ( CY_U3P_ERROR_INVALID_SEQUENCE can also lead to a commit buffer failure). You need to configure the sensor with a resolution less than the available USB bandwidth (Else there will always be a constant data loss like this). Can you check with any other standard video player for android for this issue? This way you can be sure if the application is making the transfer slow or the USB interface does not have enough bandwidth
Thank you for your response. I can't decrease the resolution of the sensor in my project. Can you explain more for me the USB bandwidth ? Can I do some thing with it to avoid the problem ?
I will try with a standard android application as you suggest.
Would the function "CyU3PDmaMultiChannelSetWrapUp" behave similar in this situation? Would it also return this error code in case of a slow consumer?
It depends on your host platform and your OS as well. What is the datarate your are trying to stream (mention your resolution/fps/ image format). Also mention the transfer type (BULK/OSOC). You can compare the throughput with any other similar device on the same OS (you will need a device of Video class. This is important as the UVC driver also does some work to remove and verify the hearder from every payload of image data).
Thank for your answer, I can manage to decrease the problem of frame corrupted when the rates of producing and consuming frame are mismatch by flowing the steps described in How_to_Implement_an_Image_Sensor_Interface_with_EZ-USB_FX3_in_a_USB_Video_Class_UVC_Framework.pdf (page 41):
"GPIF II Sockets can transfer video data only if the consuming side (USB) empties and releases the DMA buffers in time to receive the next chunk of video data from GPIF II. If the consumer is not fast enough, the sockets drop data because their DMA buffer writes are ignored. As a result, the byte counters lose sync with the actual transfers, which can propagate to the next frame. Therefore, a cleanup mechanism is required at the end of every frame. This mechanism is described in the Clean Up section."
And in the Clean Up section:
"At the end of a frame, the GPIF II state machine generates a CPU interrupt, which starts the chain of events
described above. The firmware waits for the USB side to drain the DMA buffer data using a loop in the
UVCAppThread_Entry function. As soon as hitFV (which is set by the GPIF callback function) is set, the last DMA buffer is committed to USB, and the prodCount equals the consCount, the firmware will do the following:
- Reset the DMA channel (which resets its FIFO)
- Toggle the UVC header FID bit
- Call the CyU3PGpifSMSwitch function in order to restart the GPIF II state machine at its START state
- Wait for FV to go HIGH again"
However, there are still one or two times the frame corrupted problems happen. Can anyone give me an example of code that do those steps so that I can verify if I implement them correctly ?
Thank you very much
What happens when you get corrupted frame? Please try to record the UART messsage to see if you are getting commitbuffer failure. Also do register for PIB callback to see if there is a backflow. You can refer AN75779 and enable BACKFLOW_DETECT. Please check and let me know if you are getting backflow errors.
There is no need resting the DMA channel after every frame. We have update our firmware example. If you download the new AN75779, you will see that we are not actually doing a DMA reset after every frame.
When I get corrupted frame, it lasts for several seconds (it means that I had corrupted frames for several seconds) and frames come back to normal which I don't understand why. I did check for BACKFLOW_DETECT but I didn't have it. I have another problem where my aptina is set to produce 60 fps by default but some times it gives a fluctuating fps (not stable like 13 then 30 then 23 ...). Can you tell me of this problem ?
Thank you very much
Please check with the latest AN75779 firmware (which is on the Website). The blackflow flag is names "back_flow_detected". This flag will be operational once you #define BACKFLOW_DETECT in the firmware.
If you are getting BACKFLOW, that can lead to frame loss because of bandwidth issue on host (and also it is a bulk endpoint). If there is no backflow then the host will be receiving all the frames, and the change is fps you see can be related to the host application and the UVC driver.
I found the solution for my problem so I will describe in depth here in case if someone has the same problem like me.
Problem: I had a cypress FX3 connect with an aptina sensor that is configured to give 60 fps. That cypress connects to my android phone by USB. On my phone I have an app that does image processing which processes only 30 fps. Due to some restrictions I can't reduce the fps on aptina sensor. When my app reads not fast enough the fps, sometimes I get 5 or more seconds of corrupting frames and then it becomes normal.
I did enable the BACK_FLOW flag but I didn't have any of this problem. The stream runs normally but only some moments the frame is corrupted and the comes back to normal.
To address this problem I implemented a reset procedure in the firmware of cypress each time I detect the send of the last buffer of a frame like indicated in the document of the example AN75779. However it gives another problem when the sensor act abnormally (it gives varying fps overtime). This problem makes my app can't get frame correctly and I see no image on my telephone.
I checked for the error on the buffer that causes the corrupted frames by checking the size of the end buffer and do a reset each time the size is not the correct number(I calculated that number, my buffer size is 16*1024 and my frame size is 640*480 so normally the ast buffer is the 19th one which has exactly 12576 bytes, in case of error in buffer the size is not 12576 bytes).
Another error is I have the correct size in the last buffer but somehow one frame occupies more or less than 19 buffers. So in that case I do another reset to correct the problem.
Thank you for your support Nishant.