1 2 3 Previous Next 34 Replies Latest reply on Apr 14, 2017 12:12 PM by walter.britton

    SuperSpeed communication fails due to FX3 internal noise issues?

    kalev.sildaru

      Hi all,

         

      We have noticed that FX3 PHY errors count is in strong correlation with activity on GPIF interface. Until data is handled inside FX3, typically errors count stays 0. But if FX3 itself or external FPGA starts to output data to GPIF bus, PHY errors appear.

         

      The actual errors rate depends on host chipset type, USB cable length, GPIF interface voltage, etc. Definitely the host plays significant role here - I have a PC with Intel USB chipset on motherboard where the tests can run weeks without any PHY error (if I use short cable and plug the device into right port).

         

      But most important - with appearing PHY errors most probably sooner or later the SuperSpeed communication fails.

         

      According our experience, also FX3 clock signal quality has significant impact on errors count. You have to keep clock traces on PCB as short as possible (do not even think to clock two chips with same clock!). Otherwise the effect is similar - PHY errors and communication failure.

         

      Next is an excerpt from Cypress tech support response:

         

      >>>

         

      These errors are not part-part dependent, but channel, activity or noise dependent. A noisy set up will produce more of these errors compared to a quiet or less-noisy set up. More activity in the chip may lead to more noise and more of such errors. However, the layers of the communication protocol (USB) are designed to recover from such dynamic errors. IO toggling results in substrate noise. If you see there are 1336 PHY errors with 3.3V IO supply, 42 PHY errors with 1.8V IO supply. Lowering the supply voltage reduces the substrate noise.

         

      <<<

         

      I definitely agree with tech support. Few PHY errors can be considered natural at SuperSpeed rates. And USB protocol should recover from such errors. But my concern is that in practise USB communication with FX3 fails. Even with perfect host, if you lengthen the cable a bit so that PHY errors start to appear, also the communication starts to fail. I.e. in practise it does not recover from (FX3) PHY errors.

         

      dreitz posted in "SuperSpeed interoperability with USB 3.0 controllers" topic:

         

      >>>

         

      We are using a LeCroy AdvisorT3 to look at the USB 3.0.  We are seeing what they call Interpacket Symbols - IPS, but we never see any bad CRCs or other data.  It only shows Unknown Packets as a problem.  It's like the FX3 starts spewing garbage.

         

      When using our device and the data coming from the GPIF interface, it fails.  When we use our device and the USBBulkLoopAuto sample application, we do not see the garbage.

         

      <<<

         

      Taking all above into account, I start to doubt that FX3 itself fails USB communication due to its internal noise issues. 

         

      I attached a test for exploring the issue with Cypress FX3 DVK Device board (CYUSB3KIT-001).

         

      Test itself is quite simple - host sends 32-bit toggling (0x00000000/0xFFFFFFFF) data to FX3 and FX3 GPIF automata outputs this data to its pins, causing pins to toggle as well. 

         

      FX3 software is built by modifying Cypress Synchronous Slave FIFO (slfifosync) example. The original GPIF state machine is replaced with new one and a Device Vendor Request is implemented for querying FX3 error counters. 

         

      Designed GPIF state machine outputs all the host sent data to GPIF pins automatically, without any external control. Plus, it fills automatically IN pipe buffers for sending to host. As there is no external GPIF clock then the automata is modified to use FX3 internal clock. GPIF II project files are located in "FX3device\GPIF II" directory.

         

      The most of FX3 source modifications are placed between EXPLORE_GPIF_NOISE defines. Source files are in FX3device subdirectory.

         

      Three Windows command line utilities are supplied:

         

      1) FX3USBwrite - sends toggling data to FX3 OUT bulk pipe 0x01.

         

      2) FX3USBread  - can be used for reading data from FX3 IN bulk pipe 0x81.

         

      3) FX3USBerrors - reads FX3 PHY and LINK errors via Control Pipe 0.

         

      Utilities source codes are in relevant directories. 

         

      Testing scenario:

         

      1) On FX3 board, set VIO1..VIO5 to 3.3V (higher bank voltage causes more errors).

         

      2) Load FX3GPIFNoise.img to FX3.

         

      3) Run FX3USBerrors from command prompt with option "-clear".

         

            FX3USBerrors   -clear

         

      This reads FX3 error counters and resets them to 0.

         

      4) Read errors with FX3USBerrors several times without "-clear" option. Hopefully error counters stay 0.

         

      5) Start sending data to FX3 by launching FX3USBwrite.

         

      6) Read errors. Hopefully you will see errors appearing. Few errors per tens of minutes should guarantee USB failure. Leave FX3USBwrite running. After few minutes...days it will exit with error - USB communication has failed.

         

      7) Optionally you can launch FX3USBread concurrently as well, this may increase the probability of USB failure.

         

       If there are no errors, try to lengthen the cable or plug FX3 into another USB port. For example, if your PC has USB3.0 ports also at front, try these.

         

       You can also test FX3 with quiet GPIF. For that send constant data 0 to FX3 with command

         

          FX3USBwrite -data=0

         

      I expect that error counters remain 0, or at least there are significantly less errors. 

         

      Note about Etron chipset/driver. Etron and FX3 just do not cooperate. USB Control Transfers fail randomly at heavy USB throughput and therefore FX3USBerrors may exit immediately with error code 31. Just retry (of course, if FX3USBwrite still runs).

         

      Please, test your FX3 and host and give feedback. Especially if you have a set that works reliably with PHY errors. I have 4 different hosts and 2 Cypress FX3 DVK REV3 kits (+ several our own designed prototypes), but no one combination survives FX3 PHY errors.

         

      Thanks,

         

      kalev

        • 1. Re: SuperSpeed communication fails due to FX3 internal noise issues?
          rama.sai.krishna.vakkantula

          Hi Kalev,

             

           

             

          It looks like you already contacted Cypress tech support team regarding your issue and now you are looking to get suggestions/comments from the other customers.

             

           

             

          Thanks,

             

          Sai Krishna.

          • 2. Re: SuperSpeed communication fails due to FX3 internal noise issues?
            content.librarian

            Hi Kalev,

               

            we also have some noise issues. We recognized this during EMI tests  with external power supply and burst noise (
            EN 61000 4-4 ). We also used a USB3 Analyser watching the traffic between host and device (customer board) and have seen unkown packets during burst noise. Sometimes the connection could be reconnect with a new training sequence - often it could not reconnect. These could not see any problems with the same test enviorment during an USB2  connection.

               

            Do you have any Problems with USB2 connection? Currently we think it could be a FX3 Problem, cause all tries to reduce the noise with filter and or common mode choke in power supply could stop this problem. Maybe the FX3 has some Problem with its USB3 tranceiver and its PLL ?

               

            Question to the Support team:

               

            Are there any experience with your FX3 Evaluation board and EMI Burst test with external Power Supply (USB Device self powered) ?

               

            Until now, we did not start a start a support case to the tech support team.

               

            regards MPL

            • 3. Re: SuperSpeed communication fails due to FX3 internal noise issues?
              kalev.sildaru

              Hi RSKV,

                 

              I created this topic after I had contacted tech support first time.

                 

              Yes, I contacted tech support again by reopening the case. And they promised to explore the issue more.

                 

              Hopefully any feedback here could help me, tech support and whole developer community.

                 

              Thanks,

                 

              kalev

              • 4. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                kalev.sildaru

                Hi  MPL,

                   

                USB2 seems to work well, though I have not tested it as exhaustively as I test USB3.

                   

                > Maybe the FX3 has some Problem with its USB3 tranceiver and its PLL ?

                   

                We have found that FX3 USB3 is very sensitive to clock signal quality. We succeeded to suppress PHY errors significantly on our custom board by filtering clock chip power and shortening clock signal routes. Test results with FX3GPIFNoise.img:

                   

                Cypress FX3 DVK rev3 board - over 100 PHY errors/min.

                   

                Our custom board - about 0...1 PHY errors/min.

                   

                Clearly a noise on clock signal affects PHY errors count (and I guess because it affects PLL).

                   

                Why the USB3 communication breaks? I can only guess. Either it's HW issue, for example PLL output becomes so bad that USB3 state machine fails. Or, FX3 firmware just resets USB3 if it counts too much PHY errors. I really hope it to be a firmware issue because it seems to be in correlation with PHY errors rate. Note, according SDK version 1.2.2 FX3ReleaseNotes.pdf, similar issue was already fixed:

                   

                ...

                   

                2) Fixed a USB driver issue which could cause the device to re-enumerate occasionally due to
                USB 3.0 link errors that accumulate over a long period of time.

                   

                ...

                   

                According my tests, USB3 may break even if there is still 0 link errors even after device re-enumeration.

                   

                kalev

                • 5. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                  kalev.sildaru

                   Hi,

                  One more observation. Similar PHY errors and USB3 break effect can be achieved just by lengthening the USB cable. Therefore it seems that there is no direct relationship between FX3 external/internal noise and USB3 break, i.e. it's not PLL issue. Could it be that disconnecting USB3 causes "unkown packets"? If yes, then it's most probably software issue that just resets/disconnects USB3 when there appears enough PHY errors.

                  Let's see what the tech support will find.

                  Thanks,

                  kalev

                  • 6. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                    viktor.hildebrand

                    Any news on this topic?

                       

                    I think we have the same problem (I will open a case later). Depending on the Host-Controller, Cable, Cable length etc. we also observe connection aborts in superspeed mode. Sometimes after a few seconds and sometimes after hours. First we searched the problem in our software and the USB-3 Cables (no problem in USB-2 mode).

                       

                    To isolate the issue I used the bulksrcsink example from the SDK, modified it to include a sequence number in the data and wrote a testprogram to receive and check the data on the PC. When the problem occurs, the sequencenumber is not OK and/or the datatransfer stops. With shorter cables by trend the problem does not occur.

                       

                    For us this is a big issue, because we cannot risk this kind of problems at our customers, because at the moment there is no fall back (device has to be switch off/on) and we need 3m cables because ot the size of our device.

                       

                    I will try to monitor the PHY Error counter now...

                    • 7. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                      kalev.sildaru

                      Hi,

                         


                      There is little progress. Related to this SuperSpeed breaking issue, tech support reported that maybe they have found something unexpected happening on LINK layer and they are discussing issue with design team.

                         

                      Anyway. There are two problems involved:
                      1) FX3 noise sensitivity that causes PHY errors. You can increase the number of PHY errors dramatically on Cypress FX3 Kit just with outputting data onto GPIF pins.
                      2) Possible FX3 (or hosts) issue that breaks SuperSpeed connection when PHY errors appear.

                         

                      According USB 3.0 standard, SuperSpeed physical layer bit error rate is expected to be less than one in 1e12 bits (according my computations, 1 error in 200 seconds). So, anyway, if you want to build reliable device, you have to design a hardware where basically no PHY errors appear. And as you see, you must build better hardware that FX3 kit.

                         

                      Cypress is kindly agreed to review users PCB layouts, but I would appreciate if they could review FX3 kit first and tell what the users should pay attention in their designs to get less PHY errors compared to Cypress kit. I assume that Cypress has followed all their FX3 Hardware Design Guidelines but still the kit seems to be far from perfect.
                       

                         

                      Thanks,
                      kalev
                       

                      • 8. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                        content.librarian

                        Hi ,

                           

                        we made some progress with noise issue problem and SuperSpeed Mode. Now we  pass the burst test with modifikation at our case. This is just a little success. It would be better, if the hardware could pass the test without a case and special shielding cause we also like to sell our board without a case.

                           

                        We have seen other FX3 customer board without this problem. Cause of this, we made a physical layer compliance test without any issue. Currently we can use USB3 Cables up to 8 m long. We also made these test with the Cypress Evaluation Board and our customer evaluation board (Yes, we have two boards with this issue). Also no issue.

                           

                        I think there have to be a very sensitiv pin or signal at the FX3 with a non optimal circuit and/or layout. According to kalev, i think cypress have to find this issue at there own evalboard and tell us a solution :).

                           

                        Thanks, kalev, for your FX3FPIFnoise.zip. With our board we could not see any Problems during test with reading or writing.  The error counter stayed at zero, but ... if we do both at the same time, increasing the traffic at USB3, the counter increased.

                           

                        Now we will contact the tech support team. Hopefully, as more customer will start a support case, as faster a solution will be provided :)

                           

                        regards MPL

                        • 9. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                          kalev.sildaru

                          Hi,

                          I have a bit of information to share.

                          Tech support said that this is host that gives up and breaks the communication. To reduce noise, tech support suggested to try using of metallic shield over the FX3 (and use more realistic data pattern).

                             

                          It's clear that using GPIF increases noise and affects SuperSpeed communication. For users the biggest concern is probably that FX3 enumerates initially at SuperSpeed, but when the actual data transfer over GPIF starts, the communication with device may break. So, my suggestion is to check always for PHY/LINK errors, especially in your own hardware designs (NB! see my comments below about CyU3PUsbGetErrorCounts).

                             

                          Anyway, for those who are interested in playing with FX3, I attached updated FX3GPIFNoise2.zip. The changes are:

                             

                          1) I found that CyU3PUsbGetErrorCounts function may return incorrect error counter values, i.e. hide errors. As a work-around, I wrote my own GetErrorCounts implementation and changed a way it gets called. Unfortunately it's not possible to get 100% true counter values until Cypress does not fix the API. More comments in source.

                             

                          2) I replaced three separate host utilities with one single FX3USBtest - easier to use. Type

                             

                          FX3USBtest -help

                             

                          for help on its command line options.

                             

                          Below are screenshots from my tests with Renesas USB3.0 add-on adapter (bases on NEC D720200F1 chip):

                             

                          1) 1m USB3.0 cable is used for connecting FX3 DVK rev 3 to host Renesas adapter:

                             

                          C:\test>FX3USBtest
                          FX3USBtest version. 1.2
                          Press ESC for exit
                          00:00:28 Read/Write=196.2/108.5MB/s Errors PHY/LNK=5/0
                          Read from device failed (GetOverlappedResult error code=31)
                          A device attached to the system is not functioning.

                             

                          2) 1.8m cable. My 1.8m cable seems to be better than 1m cable, worked 9 minutes:

                             

                          C:\test>FX3USBtest
                          FX3USBtest version. 1.2
                          Press ESC for exit
                          00:09:42 Read/Write=199.2/ 99.1MB/s Errors PHY/LNK=62/0
                          Write to device failed (GetOverlappedResult error code=31)
                          A device attached to the system is not functioning.

                             

                          3) same 1m cable + 1.8m extension, total 2.8m cable. Data pattern all zeros. Has worked for now over 16 hours without errors:

                             

                          C:\test>FX3USBtest -data=0x0,0x0
                          FX3USBtest version. 1.2
                          Data pattern = 0x00000000,0x00000000
                          Press ESC for exit
                          16:40:51 Read/Write=200.0/109.0MB/s Errors PHY/LNK=0/0

                             

                          Br,
                          kalev

                          • 10. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                            content.librarian

                            Hi,

                               

                            any updates on that issue????

                               

                            thanks,

                               

                            lumpi

                            • 11. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                              kalev.sildaru

                              Hi,

                              On FX3 kit, to reduce noise, lower the VIO voltage. Set VIO1...VIO3 to 1.8V and you will get significantly less PHY errors. You can experiment also by removing VIO1...VIO3 jumpers at all after powering up and loading the program image into FX3. This way it's possible to reduce PHY errors rate to a level that is comparable with rates when processing data FX3 internally.

                              But when there appear PHY errors (with rate about 10 errors per minute) then you can be sure that at some point the SuperSpeed communication will break. I think, this is characteristic to USB3.0 (I can't understand why the host should give up so easily and not retry until it succeeds?). Anyway, keep the cable as short as possible. I have achieved to build a device that works with about 1...1.8m cable (tested with hosts I have available).

                              br,
                              kalev

                              • 12. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                                leonard.kobylenski

                                Hi All,

                                   

                                I wanted to jump in and follow this post. I have been facing a series of similar issues. Thanks kalev for the tester. We failed with error code 31.

                                • 13. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                                  joel.brenner
                                          We have the same problem. We designed an FPGA/FX3 based product which makes use of the GPIF configured for 32 bit synchronous slave fifo and we see a clear correlation between the reported USB physical errors and the activity on the 32 parallel slave fifo data bus. When we use the bus between the FPGA and the GIPIF (80 MHz pclk) at his maximal bandwidth the physical errors are increasing significantly up to brake the USB 3.0 link. Joel Brenner   
                                  • 14. Re: SuperSpeed communication fails due to FX3 internal noise issues?
                                    kalev.sildaru

                                    Hi,

                                    I attached FX3USBnoise3.zip that includes sources/images updated to FX3 SDK 1.3.0.

                                    As Cypress published API sources then I finally got a chance to fix HW/SW concurrency while updating "Error Count Register". Now the program displays the exact count of PHY/LINK errors as FX3 HW detects them.

                                    Of course, program works also when linked with original Cypress API libraries, but error counters displayed are then just less accurate.

                                    BTW, I fixed also CyU3PBusyWait function that was optimized out while compiling with gcc options -O2,-O3, -Os, etc. Prebuilt FX3GPIFnoise.img, provided in zip is linked with library compiled with -Os option.

                                    Modified API library sources are provided in FX3_lib_1.3.0.118_sources_updated directory.

                                    Br,
                                    Kalev

                                     

                                    1 2 3 Previous Next