UVC first time open problem

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
FeYu_1508691
Level 3
Level 3
First like received

Hello,

I have met a strange test result when using FX3 as UVC camera.

After I connect the usb cable to PC,

first time I open software (e.g. win: amcap/vdm, ubuntu: guvc) to watch the images,

there will always be black image(no data).

And then I close the software and reopen it,

images will be shown normally.

And after the first time, the camera always goes fine.

But if I disconnect the cable and re-plug in it, I will get black image at the first time again.

I have tried to used Directshow library in Windows to debug.

And I confirmed when the first time using the camera, callback function will never be called, which means no data is received.

Has anyone ever met problem like this?

Thank you.

0 Likes
1 Solution

CUSTOMER RESPONSE:

Hello Yashwant, Thanks for your help. Since we cannot find why 1.3.4 does not run well, we can use 1.3.3 to avoid the first open problem now. Is there any suggestion about lossing image data when using extension unit communication? 

Best regards,

Fei Yu

RESPONSE:

Hello Fei Yu,

Please refer to the following KBA: Implementing Extension Unit Control in AN75779 Example Project - KBA219280

This KBA explains about the use of extension unit in AN75779 to get a better understanding about the issue.


Regards,

Yashwant

View solution in original post

0 Likes
37 Replies
YashwantK_46
Moderator
Moderator
Moderator
100 solutions authored 50 solutions authored 50 likes received

Hi,


Can you please take a Wireshark USB trace and share with us?

It might help determine if there is no data coming to the host or is the data coming to host but the host application is not reading it.

Regards,

Yashwant

0 Likes
lock attach
Attachments are accessible only for community members.

Sorry for replying late.

It seems that Wireshark is not quite friendly on Windows.

And I used Microsoft Message Analyzer instead.

The attachment includes the first time with black screen, and then

re-plugin with normal image.

Best wishes,

Fei Yu

2019年10月2日(水) 20:01 YashwantK_46 <community-manager@cypress.com>:

<http://www.cypress.com>

Cypress Developer Community

<https://community.cypress.com/?et=watches.email.thread>

UVC first time open problem

reply from YashwantK_46

<https://community.cypress.com/people/YashwantK_46?et=watches.email.thread>

in USB Superspeed Peripherals - View the full discussion

<https://community.cypress.com/message/210848?et=watches.email.thread#210848>

0 Likes

Hi Fei Yu,

I did try to go through the trace that you have provided but it's a bit too difficult to understand exactly what's going on in that.

May i know what is the issue with using wireshark?

If you have a problem using wireshark on your PC, can you please try and use another PC to take the trace?


Regards,Yashwant

0 Likes
lock attach
Attachments are accessible only for community members.

Hello Yashwant,

Thanks for your reply.

The data file output of wireshark is so huge, and I didn't even know if it

was working normally.

Today I tried to do capturing with wireshark again, and do the operations

more quickly to have a smaller data file.

(about 100mb, and zip to 7z file about 24mb. The extension name of the attachment should be .7z, not .7z.zip)

The attachment data includes the first time with black screen, and then

re-plugin camera with normal image.

Best Wishes!

2019年10月11日(金) 21:29 YashwantK_46 <community-manager@cypress.com>:

<http://www.cypress.com>

Cypress Developer Community

<https://community.cypress.com/?et=watches.email.thread>

UVC first time open problem

reply from YashwantK_46

<https://community.cypress.com/people/YashwantK_46?et=watches.email.thread>

in USB Superspeed Peripherals - View the full discussion

<https://community.cypress.com/message/211686?et=watches.email.thread#211686>

0 Likes

Hi Fei Yu,

I tried to go through the wireshark traces that you have provided but couldn't really figure out the exact reason behind such behaviour.

Can you please share the UART debug logs for the same?

Please ensure you get the entire process where in you program the device for the first time, see the black screen and then reprogram the device to see the stream.

That can help me understand the cause of the board's behaviour.


Regards,
Yashwant

0 Likes

Sorry for replying late.

We are trying debugging on this issue.

UART debug log seems nothing strange.

We are using FPGA, not CMOS directly with FX3, and the initialization of

CMOS is all done by FPGA.

And the data is passed to GPIF when FPGA inits CMOS finished.

I think this may have some differences with the original code.

And we have found that the first time return value of SET_INTERFACE is not

correct (some image data is returned),

while the second / third... time is OK.

Thank you.

2019年10月22日(火) 17:58 YashwantK_46 <community-manager@cypress.com>:

<http://www.cypress.com>

Cypress Developer Community

<https://community.cypress.com/?et=watches.email.thread>

UVC first time open problem

reply from YashwantK_46

<https://community.cypress.com/people/YashwantK_46?et=watches.email.thread>

in USB Superspeed Peripherals - View the full discussion

<https://community.cypress.com/message/212723?et=watches.email.thread#212723>

0 Likes

Hi Fei Yu,


Can you please share if you have found any cause about the problem that you have been facing?


Also, can you please share the findings of your diagnostics?


Regards,

Yashwant

0 Likes

Hello Yashwant,

We are still working on this problem.

First we change FPGA program not to send image data so early.

And we have done more tests, now found that if we change back to SDK 1.3.3, the problem was fixed.

And if we use SDK 1.3.4, the problem would occur.

Best wishes

0 Likes

Hi Fei Yu,


Can you please share the findings of the issue?

Also, do you experience the same problem when using SDK 1.3.4 and not in 1.3.3?


Regards,
Yashwant

0 Likes

Hello Yashwant,

We are still having this issue. We cannot make sure where the problem is not.

Only we know is that if using 1.3.3, this problem disappears.

And if recompiled using 1.3.4, this problem occurs.

We have changed PC to compile the FX3 program, and everything went the same.

We have also tried Debug and Release version, they also went the same.

Best wishes.

0 Likes
lock attach
Attachments are accessible only for community members.

EDITED: ADDED SNAPSHOT

Hello Fei Yu,

I have analyzed the traces again and found the following:

1.) Your buffer count is set to 3 for each channel. Is this true?

2.) So, total buffer count for multi channels would be 6.

3.) When you open the application for the first time, the device starts streaming the data. I have checked this from the trace and found that 6 buffers are getting committed to the host. Check the snapshot attached.

4.) But, after that, there is a reset event or appstop event which can be caused by either of the two things:

          A.) There is a timer overflow which happens when the sensor is not sending the data to FX3.

          B.) There is a commit buffer failure which will make call appstop() and the sensor will stop streaming.

Can you please check the UART debug logs for the above and see if you see any RESET event?!

Also, can you please share a wireshark trace for the case when you are using 1.3.3 and also share the debug logs?

Regards,
Yashwant

0 Likes

Sorry for replying late.

the buffer count is 3.

Since we are using a edited version firmware by Keerthy in 2016, who is one of Cypress's staffs, I think.

I have created a Case in 2016, and then I got the help, which change DMA payload to 48KB and buffer count to 3.

After that time, we are continuing using the edited version firmware.

And we have down-graded to 1.3.3 now, we are still using SDK 1.3.3 to avoid the problem. And we have met another problem: when doing EP0 data transfer (e.g, extension unit), bulk transfer data (image data) will lost. This will make PC get incomplete image. Image data goes well when there is no EP0 data transfer.

I will try to re-install a PC with 1.3.4 to do the test.

Best wishes,

Fei Yu

0 Likes

Hello Fei Yu,

This is a known issue which was fixed with SDK version 1.3.2 and above.

Please refer to FX3_SDK_TroubleShooting_Guide provided with the FX3 SDK, Section 2.3, Part IV, Second bullet point.

pastedImage_0.png

You can go through the source of the CyU3PUsbSendEP0Data() and explore it to find the process of suspending the IN endpoints and then resuming them after the EP0-IN is finished in FX3 SDK version 1.3.2 or higher.

Regards,

Yashwant

0 Likes
lock attach
Attachments are accessible only for community members.

Some more test results on 1.3.4:

1) Debug with UART, and not goint to UvcAppProgressTimer(). (no timer overflow)

2) I have tried to re-capture data with Wireshark.

(about 70mb, and zip to 7z file about 24mb. The extension name of the attachment should be .7z, not .7z.zip)

The first time, PC app cannot show the image. Then I closed it, and re-opened the app.

The second time, everythig went well.

3) Also you can see from the data, some payload size is smaller than 49175, which means bulk data lost.

(USB payload is set to 48kB)

This usually happens after EP0 transfer.

(e.g. Line 1860)

Best wishes,

Fei Yu

0 Likes

Hello Fei Yu,


Apologies for the delayed response.

As I mentioned in response no: 11, I see the same pattern here, 6 full buffers begin committed to the host when you open the app for the first time and then nothing is committed to host.

When you open the application for the second time, the streaming starts as expected.

From the traces, Line: 1860, this can be due to a commit buffer failure and since the value is less than 48KB, the state machine will assert the end of frame flag which will make it commit as a partial buffer ( not expected) and the host will discard all the 17 full buffers before the 18th partial buffer ( 33947 length) since it didn't receive the expected number of full buffers before recieving the partial buffer.

So, can you share the UART debug logs along with the corresponding USB wireshark trace so that I can compare and check if there are any commit buffer failure events happening in the firmware that are causing this odd payload size?

Regards,

Yashwant

0 Likes

Sorry for replying late. I haven't seen the comment on page 2.

Actually there was not any special UART debug logs, only some printing of reading extension unit happened.

We were using UVC 1.1, but some of the information in UVC 1.5 was read. This is wierd, too.

And I think there is no commit buffer failure (or do not having enough log printing?)

DMA payload was set to 48k, because there was frame lost at long long ago.

Best regards,

Fei Yu

0 Likes

Hello Fei Yu,

Can you please share what are the incorrect debug logs that you are getting?

In the commitbuffer API, have you added any debug prints in the status != CY_U3P_SUCCESS? Are you able to see any prints in the commit buffer failure case?

only some printing of reading extension unit happened

--> Can you please share what extra information are you reading?

Regards,

Yashwant

0 Likes

Hello Yashwant,

We have add some extension unit communication of our own, for sending something like firmware version, etc.

Actually we do not have any incorrect debug logs now.

I will try to add some prints != CY_U3P_SUCCESS, and try to get some more logs.

Best regards,

Fei Yu

0 Likes

Hello Yashwant,

We have confirmed logs again today.

And there was no error logs in commit buffer api.

Best regards,

Fei Yu

0 Likes

Hello Fei Yu,

And there was no error logs in commit buffer api.

--> Good to hear that there are no commit buffer failures.

Does this mean that the issue of the black screen is fixed now or is it still present?

In your application, have you added the following definition in the uvc.h file as below:

#define FRAME_TIMER_ENABLE              /* Enable/Disable a timer that aborts an ongoing frame and restarts streaming

                                                                       * when the transfer is stalled. Default setting is to enable frame timer */

If you have it enabled, please comment it out and then see if you find any change?

Regards,

Yashwant

0 Likes

Hello Yashwant,

We have tried to comment frame timer today, and using 1.3.4, the result was the same.

The issue is still the same.

And we are using 1.3.3 to avoid this problem.

Best regards,

Fei Yu

0 Likes

CUSTOMER RESPONSE:

Hello Yashwant, Thanks for your help. Since we cannot find why 1.3.4 does not run well, we can use 1.3.3 to avoid the first open problem now. Is there any suggestion about lossing image data when using extension unit communication? 

Best regards,

Fei Yu

RESPONSE:

Hello Fei Yu,

Please refer to the following KBA: Implementing Extension Unit Control in AN75779 Example Project - KBA219280

This KBA explains about the use of extension unit in AN75779 to get a better understanding about the issue.


Regards,

Yashwant

0 Likes

Hello Yashwant,

Thanks for your reply.

We can use extension unit communication now.

But as the log I uploaded last time, when doing extension unit communication,

the image packet will be lost.  And that frame will be discarded.

Is this normal?

Best regards,

Fei Yu

0 Likes

Hello Fei Yu,


Can you please confirm if you are using USB 2.0 or USB 3.0?

There are known issues of image data loss when using SDK 1.3.3 and higher when there are simultaneous CONTROL-IN and BULK-IN transfers in USB 2.0.

This issue's workaround has been implemented with SDK 1.3.3 and higher and because of this, there would be bulk data loss when there is a simultaneous CONTROL request when streaming is going on.

The same has been mentioned on page no. 6 under the section USB Driver and APIs (Point No. 2) in Cypress EZ-USB FX3 SDK Release notes provided in the SDK path: C:\Program Files (x86)\Cypress\EZ-USB FX3 SDK\1.3\doc\firmware\FX3ReleaseNotes.pdf

Regards,
Yashwant

0 Likes

Hello Yashwant,

We are using USB 3.0.

Best regards,

Fei Yu

2020年4月8日(水) 17:03 YashwantK_46 <community-manager@cypress.com>:

<http://www.cypress.com>

Cypress Developer Community

<https://community.cypress.com/?et=watches.email.thread>

UVC first time open problem

reply from YashwantK_46

<https://community.cypress.com/people/YashwantK_46?et=watches.email.thread>

in USB Superspeed Peripherals - View the full discussion

<https://community.cypress.com/message/230180?et=watches.email.thread#230180>

0 Likes

Hello Fei Yu,

Please try the following and let me know:

1.) Can you please check if you see commit buffer failures when you are using extension unit communication?

2.) Can you see if there are PROD_EVENTS generated or not while using extension unit communication?

3.) Can you please confirm if you are using a DMA AUTO channel or MANUAL channel for the video streaming?

Regards,
Yashwant

0 Likes
lock attach
Attachments are accessible only for community members.

Hello Yashwant,

Thanks for your reply.

We will try to check item 1 and 2.

But I do not know about item 3. Could please tell me more about this?

And I have uploaded uvc.c and uvc.h here to see if you can find the answer for item 3 in the file.

When retrieving extension unit communication from PC,

FX3 will try to read data from another arm chip through IIC, then reply the extension unit command.

Best regards,

Fei Yu

0 Likes

Hello Fei Yu,

Thank you for the files.
I have checked them and see that you are using a MANUAL_MANY_TO_ONE channel for UVC streaming.

I also noticed that you are either committing the video data through callback or in the for ( ; ; ) loop using the macro UVC_COMMIT_IN_DMACB

During normal operation, are you committing the video data in CB or the main thread?

Please try the following to check if the I2C transfer during the extension unit communication is causing the issue by commenting out the i2c transfer function and checking if you see any image data loss or not.

This will help understand if the I2C read is causing the issue of image loss.


Please do the following and please confirm steps 1 and 2 in my previous responses.

Regards,

Yashwant

0 Likes

Hello Yashwant,

Thanks for your quick reply.

I have check the code again.

Since we have the defination UVC_COMMIT_IN_DMACB in uvc.h, I think we are doing committing in the callback.

(Actually we got the sample code from one of your staffs called Keerthy in 2016, and we have not change the part of DMA since then. We are not sure that we should change it or how to change it.)

1.png

We will try steps 1 and 2, and post the result later.

Best regards,

Fei Yu

0 Likes

Hello Yashwant,

0416.png

1. We tried to add print in place 1/2/3 in the screenshot.

All 1/2/3 could have debug print out, but PC could not show image when debug print was added.

Meanwhile, commit error print out (line 487) did not show.

2. If changing to comment #define UVC_COMMIT_IN_DMACB,

commit error would show occasionally.

Error in multichannelcommitbuffer: Code = 71, size = BFF0, dmaDone 6

Best regards,

Fei Yu

0 Likes

Hello Fei Yu,

1.) If changing to comment #define UVC_COMMIT_IN_DMACB,

commit error would show occasionally.

Error in multichannelcommitbuffer: Code = 71, size = BFF0, dmaDone 6

--> Does this mean that when the commit buffer action is happening in the main thread (not in DMA CB), you are receiving error code=71?

Erro code=71 corresponds to invalid sequence error.

A workaround to this issue is mentioned in the KBA article: Invalid Sequence Error in Multi-Channel Commit Buffer - KBA218830

Please try this and see if you face the error code again.

2.) Also, instead of adding the prints at 1/2/3 as above, please add the Debug print as mentioned in line 2331 as below:pastedImage_4.png

Please commit the video data through the DMA CB only and keep #define UVC_COMMIT_IN_DMACB as it is.

This will print the number of prod and cons events that are happening in the DMA CB mode and share the UART debug logs with me.

3.) Also, see if you get the commit buffer failure print as well while doing extension unit communication or not. Please confirm!

4.) As mentioned in my response no.24:

Please try the following to check if the I2C transfer during the extension unit communication is causing the issue by commenting out the i2c transfer function and checking if you see any image data loss or not.

This will help understand if the I2C read is causing the issue of image loss.


Please do the following and see if you face the issue.

Also, please share the debug log as requested with the prod and cons count.

Regards,

Yashwant

0 Likes
lock attach
Attachments are accessible only for community members.

Hello Yashwant,

We have done some tests, and I will write the result here.

1.txt is the log of 1), which edited as Invalid Sequence Error in Multi-Channel Commit Buffer - KBA218830 .

From the log we can see still there is code 71 error.

2_3. txt is the log of 2) and 3). #define UVC_COMMIT_IN_DMACB is keeped.

When PC did not open the camera, Prod = 0 Cons = 0. After PC opened the camera, Prod and Cons would get value bigger than 0 (after line 470).

We commented print in extension unit callback, and tried to do some extension unit communication. As you can see from log, there is no print for commit buffer failure.

4.txt is the log of 4).

We commented IIC communication in extension unit callback, and the log is similar with 2_3.txt.

We are still not sure about image data loss now, we will try to transfer more data with extension unit communication,

and try to get some log again.

Best regards,

Fei Yu

0 Likes

Hello Fei Yu,

Thank you for the logs.

1.) I have checked the logs where the error:71 is printed multiple times which indicates that the host application is unable to issue enough tokens to free up the buffers which lead to buffer overflow.
This can be validated by the difference between prod and cons count in the UART debug prints.
As you have mentioned that 1.txt is after implementing the workaround provided in KBA218830 but i am unable to see any of the prints added in the KBA and only see the commitbuffer failues.
Can you please share the updated firmware so that i can validate the modifications made to the firmware?

2.) The increase in prod and cons values is expected as it is based on the number of buffers filling on the PROD and CONS sockets but i am unable to see any of the commitbuffer failure prints as mentioned in 1.txt file shared by you.

Please do the following and share the UART debug logs again:

a.) Add a debug print like "This is extension unit request"  or something similar which can be used to get when you are using extension unit communication.

b.) Start streaming and share the entire UART debug logs including the commitbuffer failure prints (if any) with me so that i can see if the extension unit communication is causing any issue with the DMA channel between P-Port and U-Port.

3.) We commented IIC communication in extension unit callback, and the log is similar with 2_3.txt.

Does this mean that you have commented the CyU3PI2cTransmitBytes() and CyU3PI2cReceiveBytes() in the extension unit communication and you don't see any of the "Error in multichannelcommitbuffer: Code = 71, size = BFF0, dmaDone 6" as such?

4.) For 4.txt, please uncomment the I2C communication and then add a print as mentioned in (2 (a)) above and do an extension unit communication (only once) and then share the entire log of the process.
This will help in understanding if there is any issue in I2C causing halt in the DMA channel between the PIB and UIB.

Regards,
Yashwant

0 Likes

Hello Fei Yu,

Can you please let me know if you had any progress with the issue?
If not, can you please share the requested logs in the above interaction?

Regards.
Yashwant

0 Likes

Hello Yashwant,

Sorry for replying so late.

Actually we have not test on this issue yet, because we have met another big problem.

We have tried to add usb uart on this project these days.

And we can see the com port, and reading data through it on the PC now.

Uart works well, but the image only could transfer correctly several seconds,

and then it freezed.

We tried to add usbuart like this link:

UVC + USBUART Example

Do you have any idea about this?

Best regards,

Fei Yu

0 Likes

Hello Fei Yu,

Can you please share the logs and the updated firmware that I have requested in Response: 33?

Also, is the USB_UART Interface you had problems with, working well?

Please share any progress.

Regards,
Yashwant

0 Likes

Hello Yashwant,

Sorry for reply late again.

We can use usb_uart interface now, also there is image data lost while using uart.

But we are not sure image lost is caused by uart, or it is just the problem we have disscussed in this topic till now.

And I am sorry that we still have not done the test due to our development schedule.

Our engineers are all working on uart and something else these days.

You can close this disscusion and maybe I will open a new topic when we finish the test.

Sorry for taking you so much time.

Best regards,

Fei Yu

0 Likes