1 2 Previous Next 23 Replies Latest reply on May 7, 2013 5:02 AM by prajith.cheerakkoda

    CyAPI.NET bug

    andrew.sobotka
              The C# .NET library that Cypress provides has a    catastrophic bug; it does not pin managed byte[] buffers before sending them to the driver, so when the .NET garbage collector runs and compacts memory, it will move the byte[] buffer while the driver is reading/writing to it. Any .NET application that uses the Cypress C# library will crash, given enough time, even if you're using XferData!   
         
      Myself and another poster here, andy, both experienced this problem and found solutions. Andy tried to contact Cypress and tell them about it, but he says they just ignored him. I posted here so that anyone else who runs into the problem might find a solution. We had a nice thread going three weeks ago, but it's not here anymore. I assume Cypress removed it, so I won't be surprised if this gets removed, too.   
         
      At least the C++ library and driver work.   
        • 1. Re: CyAPI.NET bug
          afasdf.asdgf.s.add
                  Yeah,   
             
          what was the reason of removing it?   
             
          I added that thread to my bookmarks and now wonder why it is unreachable...   
             
          I agree with you, there is a bug - when you not pin the byte buffers before pass them to the xfer functions of the lib, the application can freeze or end in mysterious exceptions.   
          • 2. Re: CyAPI.NET bug
            graa
                    andrewsobotka,   
            I would like to assure you that we did not remove the thread in question from the forum. In the past, I have faced a similar issue with a post in psoc forums, which was then restored by our web team.   
               
            peretus4,   
            Can you provide me with the link to the thread (from your bookmark) that disappeared so that I can try to locate it?   
               
            I would also like to assure you that I will bring up this issue with the concerned team and initiate corrective actions to fix any bug int he C#.NET library.   
            • 3. Re: CyAPI.NET bug
              andrew.sobotka
                      Hi Ganesh,   
                 
              I have an old response notification email. The URL was   
                 
                  http://www.cypress.com/forums/...readid=8246&forumid=1    
                 
              There was more to it than "need to pin buffers". There was an overlapped size allocation issue and an unnecessary ref issue.   
              • 4. Re: CyAPI.NET bug
                andrew.smolak
                        Sorry guys...I don't really have too much time to go into the issue right now. I also, sadly, don't have anything of use to cut and paste from my correspondences with Microsoft or Cypress. I LOT of my time was wasted chasing this bug (along with loads of help from MS engineers debugging in Windows source) So, I hope we can prevent any others from falling in the same trap...   
                   
                Basically, the Overlapped structure (wraps the buffers and commands to pass over the fence to unmanaged) gets moved or cleaned by the GC. It seems fairly rare but if you are pushing a lot of data through multiple endpoints, you WILL experience an ExecutionEngineException or NullReferenceException in Mscorlib.dll often enough to render your code "un-depoyable".   
                   
                My solution: Only use the Cyusb.dll to for initialization and housekeeping. Since I am almost saturating the bandwidth and doing a lot of SW processing, I chose a very asynchronous approach. Here is a VERY terse description (check out the "concurrent affairs" series of articles published in MSDN magazine by Jeffery Richter for details...I based my approach off of his model)   
                   
                Once you have an OS handle you can use Threadpool.bindhandle()   
                -this causes the threadpool to process the callbacks for overlappedIO operations   
                   
                Implement IAsyncResult to handle the Packing and unPacking of the NativeOverlapped structure and the basic async pattern   
                   
                Call DeviceIO Control with a callback function. I allocate a byte[] buffer inside my ReadData method and create a callback (using a lambda expression) that uses this same byte buffer to pass along to my DeviceIoControl method. The buffers get pinned using GCHandle.Alloc & get "Pack()"ed along with the commands and callback into the NativeOverlapped structure. All of these are passed into DeviceIoControl.   
                   
                This allows you to queue up multiple commands to the endpoints with pre-allocated buffers that are automatically called one-at-a-time by the OS and processed (by the callback) by the threadpool as it's scheduled. "Set it and forget it!" as Ron Popeil would say...   
                   
                Anyway, the gist is: Even the blocking "non asynchronous" XferData functions provided in the API are just blocking wrappers around the asynchronous calls. However, they are not implemented correctly and leave the door wide open for the GC to corrupt the heap.   
                   
                PM or e-mail me if you would like more info and I'll try to help as best I can.   
                • 5. Re: CyAPI.NET bug
                  andrew.sobotka
                          andy - it's not just the Overlapped object which is getting moved by the GC. ANY of the three byte[] buffers (Overlapped, SingleXfer, or the data buffer itself) can be moved by the GC. The error happened very quickly when I was writing 64k data buffers.   
                     
                  You don't need to write unmanaged code to solve the problem. Just use the    GCHandle class to pin all three byte[] buffers before passing them to any of the CyAPI Xfer functions. Cypress could easily pin the three buffers inside XferData before sending them to the driver, and un-pin before returning from XferData. Six lines of code and XferData would no longer cause crashes.   
                     
                     
                  GCHandle bufHandle1 = GCHandle.Alloc(data, GCHandleType.Pinned);   
                  GCHandle bufHandle2 = GCHandle.Alloc(singleXfer, GCHandleType.Pinned);   
                  GCHandle bufHandle3 = GCHandle.Alloc(overlap, GCHandleType.Pinned);   
                     
                  // send buffers to driver   
                  // wait for driver to finish using the buffers   
                  // get the results of the transfer   
                     
                  bufHandle1.Free();   
                  bufHandle2.Free();   
                  bufHandle3.Free();   
                  • 6. Re: CyAPI.NET bug
                    graa
                            Hi,   
                       
                    That thread had disappeared due to a bug. It has been restored. I have also written to the concerned team to have a look at this issue.   
                    • 7. Re: CyAPI.NET bug
                      andrew.sobotka
                              Awesome! Thanks Ganesh.   
                         
                      If you could, pass these on too.   
                         
                      1) OverlapSignalAllocSize should be a static member of CyConst, but instead it's a non-static member of CyUSBEndPoint. This is contrary to the documentation.   
                         
                      2) byte[] does not need to be passed as a ref parameter. Only value-types need to be passed as a ref parameter. When an array, or any other reference type, is passed as a non-ref parameter, the    reference (NOT the array!) is passed by value (i.e. a pointer is copied).   
                         
                      This is very different from C++, where an array must be passed by reference using & or the copy constructor would make a copy of the whole array. Using a ref parameter for byte[] is like passing a pointer by reference in C++, like say   
                         
                      C#   
                      bool XferData(ref byte[] buf, ref int len);   
                         
                      is like   
                         
                      C++   
                      bool XferData(PCHAR &data, LONG &len);   
                         
                      i.e. passing a pointer by reference.   
                         
                      ---   
                         
                      This does not match the actual C++ library,   
                         
                      bool XferData(PCHAR data, LONG &len);   
                         
                      Which looks like this in C#   
                         
                      bool XferData(byte[] buf, ref int len);   
                         
                      i.e. passing a pointer by value   
                         
                      Passing byte[] as a ref parameter adds unnecessary complexity to your customer's code by adding serious and unnecessary restrictions on what may be used as an argument.   
                      • 8. Re: CyAPI.NET bug
                        content.librarian
                                Hi Andrew   
                           
                        The software guys went through the XferData API implementation and did not find any loop hole in it. It does pining of all allocation properly.   
                        - XferData API is being used by all C# application(Bulk Loopback / Control Center / Screamer), which passed through the stress testing and did not find such issue.   
                           
                        Find the source code for the XferData API below.   
                           
                        public unsafe virtual bool XferData(ref byte[] buf, ref int len)   
                        {   
                        byte[] ovLap = new byte[OverlapSignalAllocSize];   
                           
                        fixed( byte *fixedOvLap = ovLap)   
                        {   
                        OVERLAPPED *ovLapStatus = (OVERLAPPED*) fixedOvLap;   
                        ovLapStatus->hEvent = PInvoke.CreateEvent(0, 0, 0, 0);   
                           
                        // This SINGLE_TRANSFER buffer must be allocated at this level.   
                        int bufSz = CyConst.SINGLE_XFER_LEN + ((XferMode == XMODE.DIRECT) ? 0 : len);   
                        byte[] cmdBuf = new byte[bufSz];   
                           
                        // These nested fixed blocks ensure that the buffers don't move in memory   
                        // While we're doing the asynchronous IO - Begin/Wait/Finish   
                        fixed (byte* tmp1 = cmdBuf, tmp2 = buf)   
                        {   
                        bool bResult = BeginDataXfer(ref cmdBuf, ref buf, ref len, ref ovLap);   
                        //   
                        // This waits for driver to call IoRequestComplete on the IRP   
                        // we just sent.   
                        //   
                        bool wResult = WaitForIO(ovLapStatus->hEvent);   
                        bool fResult = FinishDataXfer(ref cmdBuf, ref buf, ref len, ref ovLap);   
                           
                        PInvoke.CloseHandle(ovLapStatus->hEvent);   
                           
                        return wResult && fResult;   
                        }   
                        }   
                        }   
                           
                        Adding this API instead of callling CyUSB.dll XferData API will probably help.   
                           
                        Thanks   
                        VIVK   
                        • 9. Re: CyAPI.NET bug
                          andrew.sobotka
                                  Hi VIVK,   
                             
                          Thank you for the reply. I really appreciate the support that you are providing.   
                             
                          The fixed pointers tmp1 and tmp2 are not referenced in the fixed block. The compiler will warn you about this and potentially optimize the fixed block out. To make things more complicated, this optimization will only happen with Release build, so if your developers were testing a Debug build, you would not encounter this error.   
                             
                          To verify, I suggest using Reflector on the Release mode assembly and looking at XferData to see if the fixed block survived optimization.   
                             
                          Cheers,   
                          Andrew Sobotka   
                          • 10. Re: CyAPI.NET bug
                            content.librarian
                                    Hi Andrew,   
                               
                            I created a technical support case with you as a contact and routed it to the concerned engineer. Let me know if you are able to view the case.   
                               
                            Once the issue is resolved, I will post the same in the forum for the benefit of the others.   
                               
                            Cheers   
                            VIVK   
                            • 11. Re: CyAPI.NET bug
                              andrew.sobotka
                                      VIVK,   
                                 
                              I have looked in the list of support cases which I have previously had open, and all of the cases are closed. There are no new ones.   
                                 
                              By the way, I took the time to do a test on the XferData example you provided above. Using Reflector, I was able to verify that the problem only occurs for Release builds.   
                                 
                              Specifically, when the "Optomize Code" checkbox in the Project Properties' Build box is checked, like for Release builds, the compiler will optimize the fixed block away because the fixed pointers are unreferenced. If Optimize Code is not checked, like for Debug builds, the fixed block survives.   
                                 
                              I have a copy of the Reflector output to share with the engineer when I gain access to the technical support case.   
                                 
                              Cheers,   
                              Andrew   
                              • 12. Re: CyAPI.NET bug
                                andrew.smolak
                                        A new support case for this issue was opened for my account. (#1L2YGYZ) Being that Andrew and I have the same first name and similar last names, there might have been a mix-up.   
                                   
                                I don't mind helping. But, I'm probably gonna be too busy to contribute much over the next few months.   
                                • 13. Re: CyAPI.NET bug
                                  graa
                                          Andrew,   
                                     
                                  Can you please provide the case number of any of your closed cases? That will help us to locate your account and reassign this new case to your account so that you can view it.   
                                  • 14. Re: CyAPI.NET bug
                                    andrew.sobotka
                                            This one's my favorite. 13UULHN - USB Domain and Slave FIFO Peripheral Domain Losing Synchronization. It's from early 2007.   
                                    1 2 Previous Next