Data loss when streaming data via Bulk

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
jasony1
Level 1
Level 1
First question asked Welcome!

Hi all,

 

I am very new to the FX3 and I am trying to stream data from a 32 bit bus sampled at 10MHz to the computer. Data is sampled on the GPIF using two alternating threads. A program written on linux using libusb and cython simply reads 2MB of data (for now) and dumps it to the disk. 

Currently, I have a testbench set up to generate a 16-bit binary counter to be sampled. However, looking at the dumped data returned, there are (seemingly random) gaps in the data where the counter will jump values. I cant seem to find any consistency as to where the data drops and how much will be lost. 

Is there any way to figure out how to avoid packet loss?

Thanks!

0 Likes
1 Solution
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

EDITTED for better clarity

Hello Jason,

As the size of the DMA buffer used is 8KB , width of GPIF II data bus used is 32 bits and based on the implementation of GPIF II state machine, I feel that a count limit of 2046 should resolve the issue. Can you please try setting the counter limit to 2046 and let me know the results?

Best Regards,
Jayakrishna

View solution in original post

0 Likes
12 Replies
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello,

Please share the following details so that we can understand the problem better:

1. How many data words are missing in the stored dump?

2. Please share the GPIF II project with us so that we can understand how the data is sampled.

3. Are you seeing data loss at the same position (or index in the dump) if the experiment is repeated multiple times?

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

Hi Jayakrishna!

For some reason cypress forums isnt recognizing my original login, but I am the original poster.

When dumping files, it seems that 8192 bytes are transferred correctly, but then there is a gap of roughly 4kB. What is odd is that this gap isnt consistent and i cant seem to find a pattern to it. So far it seems that the gaps only appear in between 8KB 'chunks'. My current pclk is 1.25MHz and the FX3 is the only device on the USB root  so I believe the computer isnt the bottleneck.

I am attaching the project files, GPIF project, and a sample dump. 

 

Thanks for your help!

Jason Y. 

 

0 Likes
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello Jason,

As per the firmware, I understand that the size of the DMA buffer used is 8KB. Please confirm if you have made any changes on the DMA buffer size or if my understanding is not correct.

If my understanding is correct, then can you please change the counter limit value of DATA counter in GPIF II state machine from 4094 to 2095 and let us know the result? Also, after making this modification, please check if the GPIF callback function is invoked at any point.

Best Regards,
Jayakrishna
0 Likes

Hi Jayakrishna,

Thanks for your response. I will make that change to the GPIF SM. I have not made any significant changes to the firmware as this was borrowed from another project. Though i did change the data width in the GPIF II design from the orginal 16bits to 32 bits - is this the cause for the problems? 

 

Because i am new to the FX3 design flow, how do i check if the GPIF callback function is invoked? 

 

Also, would there be any advantage to increasing the DMA buffer size at all? from my understanding, the FX3 has 512KB of  RAM. If I increase the DMA buffer to 512KB, will this allow for more time between 'burst' reads from a computer?

 

Thanks for your quick reply!

Jason Y. 

0 Likes
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello,

Yes, the change in data bus width could be the cause of this issue. The counter is used for checking if the DMA buffer is full or not. The limit of the counter should be changed based on the width of the data bus.

The gpif callback function in your project is the function SDR_ApplnGPIFEventCB in the source file sdrx3.c. The API CyU3PDebugPrint() can be used to understand if this function is invoked or not. This API can be used by the application code to log free form message strings, and causes the output message to be written to the UART immediately. 

Increasing the size of DMA buffer should not have an impact on the data loss seen here. Also, the 512KB is the total size of FX3's SRAM. Out of this, by default only 224KB can be used for DMA buffers. Please refer to the fx3.ld file to understand more about the default memory map used for FX3 Application. This can be found in the following location:

C:\Program Files (x86)\Cypress\EZ-USB FX3 SDK\1.3\fw_build\fx3_fw

Also, the maximum size of a DMA buffer is limited to 0xFFF0 bytes. Please refer to page 66 of FX3 TRM which already documents about this upper limit on the DMA buffer size. This document can be found in the following location:

C:\Program Files (x86)\Cypress\EZ-USB FX3 SDK\1.3\doc\firmware

Please let me know if you have any queries on this.

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

Hi Jayakrishna,

I tried the fixes in your previous post. The counter limit is now 2095. additionally, i have checked the debugger output and it only prints a message that the GPIF has overflowed after i have stopped recording. Unfortunately, i am still seeing data loss. Now it seems that i can read 4904 bytes before data is 'lost'. 

Additionally, it seems that the contiguous chunks of data are not sequential - after a gap, the beginning of the next chunk is sometimes preceeding the last value of the first chunk. This is not due to a timer overflow as i am now testing with a 32 bit counter using a 1MHz clock and there isnt sufficient time for the counter to rollover. 

I am attaching a dump as well as the workspace files... Am I missing something obvious?

 

Thanks,

Jason Y. 

0 Likes
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello Jason,

Apologies for making a mistake in the count in my previous response. Can you please try changing the count to 2047 and let us know if you are seeing any improvements? Along with this, please ensure that the device is enumerating in USB 3.0 mode. Also, can you please test with the 16 bit counter that was used initially for the tests?

In addition to this, please confirm that the UART print "GPIF overflow INT received" is only seen when you are stopping the data recording. 

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

Hi Jayakrishna!

We're close! your suggestion for the counter value worked. I am running this in usb 3.0 mode and tested with both the 16 and 32 bit counters. The UART does print the overflow message only when stopping data recording. 

Everything is almost working - the only thing im seeing now is every 8KB, we are dropping a single 4 byte sample. Is this also because of the state machine? Im attaching a sample dump. 

Thanks,

Jason Y. 

0 Likes
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

EDITTED for better clarity

Hello Jason,

As the size of the DMA buffer used is 8KB , width of GPIF II data bus used is 32 bits and based on the implementation of GPIF II state machine, I feel that a count limit of 2046 should resolve the issue. Can you please try setting the counter limit to 2046 and let me know the results?

Best Regards,
Jayakrishna
0 Likes

Hi Jayakrishna!

Thanks for your help! it seems to work now!

Just following up on an earlier point, if we wanted to have a larger dma buffer, would I change the value passed into the function on line 354 in file 'sdrx3.c' while also changing the counter value accordingly? 

 

Thanks for your help!

Jason Y. 

0 Likes
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello Jason,

CY_FX_EP_BURST_LENGTH cannot be set to a value greater than 16. This is because CY_FX_EP_BURST_LENGTH is also used for setting the burstLen parameter of the superspeed endpoints. 

However, if you want to change the size of the DMA buffer, then you can simply assign the required size to dmaCfg.size as shown below:

dmaCfg.size = 2048;

Please make sure that the size of the DMA buffer used is a multiple of 16 bytes. In addition to this, to avoid any data loss, the counter limit value should also be modified.

Best Regards,
Jayakrishna
0 Likes

This is excellent! thank you for your help!

0 Likes