10 Replies Latest reply on Jun 17, 2020 6:56 PM by NCH_4488531

    FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback

    NCH_4488531

      Hi,

       

      Recently, we encounter a problem. When the traffic on USB bus is busy, EP_IN data transfer will stuck, while EP_OUT endpoints work OK.

       

      We were using BusHound to monitor USB communication.

      When this error happened, we got an incomplete data package (lost the leading part)  but no error message.

      After reset the EP_IN endpoint, the IN data flow will resume, but it will happen again.

      Also, we print DMA info from FX3 firmware, and got the "CY_U3P_DMA_CB_ERROR", the description of this error is:

      CY_U3P_DMA_CB_ERROR      = (1 << 6), /**< This event is generated when the hardware detects an error. */

       

      Why this error happened, and how to fix it?

        • 1. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
          RashiV_61

          Hello,

           

          As said by you CY_U3P_DMA_CB_ERROR  is generated when hardware detects the error.

          This event is generally set when CY_U3P_ERROR occurs which indicates the socket is suspended because of an error condition.

          To know exactly the cause of the problem we need to know the DMA channel configuration used in the firmware.

          Please share the DMA channel configuration which is passed to  the CyU3PDmaChannelCreate/CyU3PDmaMultiChannelCreate  API

           

          Regards,

          Rashi

          • 2. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
            NCH_4488531

            Endpoint config:

                CyU3PEpConfig_t epCfg;
                CyU3PMemSet ((uint8_t *)&epCfg, 0, sizeof (epCfg));
                epCfg.enable = CyTrue;
                epCfg.epType = CY_U3P_USB_EP_BULK;
                epCfg.burstLen = 1;
                epCfg.streams = 0;
                epCfg.pcktSize = 1024;
                apiRetStatus = CyU3PSetEpConfig(0x83, &epCfg);

             

            DMA config:

                CyU3PDmaChannelConfig_t dmaCfg;
                dmaCfg.size  = 1024;
                dmaCfg.count = 2;
                dmaCfg.prodSckId = CY_U3P_PIB_SOCKET_0;
               dmaCfg.consSckId = CY_U3P_UIB_SOCKET_CONS_3;
                dmaCfg.dmaMode = CY_U3P_DMA_MODE_BYTE;
                dmaCfg.prodHeader = 0;
                dmaCfg.prodFooter = 0;
                dmaCfg.consHeader = 0;
                dmaCfg.prodAvailCount = 0;
               dmaCfg.notification = ( /*CY_U3P_DMA_CB_CONS_EVENT |*/
                   CY_U3P_DMA_CB_ABORTED |
                   CY_U3P_DMA_CB_ERROR |
                   CY_U3P_DMA_CB_PROD_SUSP |
                   CY_U3P_DMA_CB_CONS_SUSP);
            apiRetStatus = CyU3PDmaChannelCreate(&glBulkLoop_p->m_ChHandleSlFifoPtoU, CY_U3P_DMA_TYPE_AUTO, &dmaCfg);

            • 3. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
              RashiV_61

              Hello,

               

              To find the cause of the problem can you try changing the DMA channel to CY_U3P_DMA_TYPE_AUTO_SIGNAL and track the CONS and PROD events in the DMA callback and get the values of the variable printed in the for {} loop in the thread entry function. Please share the debug prints (tracking the PROD and CONS event) when his error occurs.

               

                  if (type == CY_U3P_DMA_CB_PROD_EVENT)

                  {

               

              prod++;

                   }

               

               

                  if (type == CY_U3P_DMA_CB_CONS_EVENT)

                  {

                     cons ++;

                  }

               

               

              Also, try changing the DMA channel to CY_U3P_DMA_TYPE_MANUAL and commit the buffer when the PROD events occur in the DMA call back and let me know if there is some error returned by CyU3PDmaChannelCommitBuffer API or is the CY_U3P_DMA_CB_ERROR occurring when the DMA channel is changed to MANUAL

              }

               

                  if (type == CY_U3P_DMA_CB_PROD_EVENT)

                  {


              prod++;

                      status = CyU3PDmaChannelCommitBuffer (chHandle, input->buffer_p.count, 0);

                          

              if (status != CY_U3P_SUCCESS)

                      {

                        CyU3PDebugPrint (2, "Halting USB Streaming EP: %d\r\n", BulkRstCnt++);

                  }

               

               

                  if (type == CY_U3P_DMA_CB_CONS_EVENT)

                  {

                     cons ++;

                  }

               

              }

               

              Regards,

              Rashi

              • 4. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
                NCH_4488531

                Hi:

                 

                The tests are done. It seems like we lost a CY_U3P_DMA_CB_PROD_EVENT each time the problem occures.

                Any ideas why this happens?

                 

                Thanks!

                 

                Code in DMA callback when error happens:

                  status = CyU3PDmaChannelGetStatus(chHandle, &dma_state, &prod_xfer_cnt, &cons_xfer_cnt);

                  if(status == CY_U3P_SUCCESS) {

                   CyU3PDebugPrint(LOG_LEVEL_INFO, "[%s], dma_state=%d, prod_xfer_cnt=%d, cons_xfer_cnt=%d\n",

                    __FUNCTION__, dma_state, prod_xfer_cnt, cons_xfer_cnt);

                 

                Also added the following code:

                if (type == CY_U3P_DMA_CB_CONS_EVENT) {

                  cons++;

                }

                else if (type == CY_U3P_DMA_CB_PROD_EVENT) {

                  prod++;

                  status = CyU3PDmaChannelCommitBuffer(chHandle, input->buffer_p.count, 0);

                  if(status != CY_U3P_SUCCESS) {

                   ErrPrint("[%s], Halting USB Streaming EP: %d\n", __FUNCTION__, status);

                  }

                }

                 

                The first test:

                Change DMA_Mode to CY_U3P_DMA_TYPE_AUTO_SIGNAL;

                Add notification: CY_U3P_DMA_CB_CONS_EVENT | CY_U3P_DMA_CB_PROD_EVENT;

                Debug Info as follows:

                2020-06-15 15:52:50,062 - [INF] [SlFifoAppThread_Entry],cons_cnt=0, prod_cnt=2

                2020-06-15 15:53:02,052 - [INF] [SlFifoAppThread_Entry],cons_cnt=49, prod_cnt=49

                2020-06-15 15:53:05,048 - [INF] [SlFifoAppThread_Entry],cons_cnt=75, prod_cnt=75

                2020-06-15 15:53:08,043 - [INF] [SlFifoAppThread_Entry],cons_cnt=88, prod_cnt=88

                2020-06-15 15:53:11,054 - [INF] [SlFifoAppThread_Entry],cons_cnt=125, prod_cnt=125

                ......

                2020-06-15 15:56:58,994 - [INF] [SlFifoAppThread_Entry],cons_cnt=25746, prod_cnt=25746

                2020-06-15 15:57:02,004 - [INF] [SlFifoAppThread_Entry],cons_cnt=26079, prod_cnt=26079

                2020-06-15 15:57:05,000 - [INF] [SlFifoAppThread_Entry],cons_cnt=26419, prod_cnt=26419

                2020-06-15 15:57:07,527 - [CyFxSlFifoPtoUDmaCallback], CY_U3P_DMA_CB_ERROR(64)

                2020-06-15 15:57:07,542 - [CyFxSlFifoPtoUDmaCallback], dma_state=5, prod_xfer_cnt=2044768, cons_xfer_cnt=2044768

                2020-06-15 15:57:07,995 - [INF] [SlFifoAppThread_Entry],cons_cnt=26704, prod_cnt=26703

                 

                 

                The second test:

                Change DMA_Mode to CY_U3P_DMA_TYPE_MANUAL;

                Debug info as follows:

                2020-06-15 16:24:55,858 - [INF] [SlFifoAppThread_Entry],cons_cnt=0, prod_cnt=2

                2020-06-15 16:25:01,853 - [INF] [SlFifoAppThread_Entry],cons_cnt=42, prod_cnt=42

                2020-06-15 16:25:04,843 - [INF] [SlFifoAppThread_Entry],cons_cnt=166, prod_cnt=166

                2020-06-15 16:25:07,853 - [INF] [SlFifoAppThread_Entry],cons_cnt=586, prod_cnt=586

                2020-06-15 16:25:10,843 - [INF] [SlFifoAppThread_Entry],cons_cnt=1016, prod_cnt=1016

                2020-06-15 16:25:13,843 - [INF] [SlFifoAppThread_Entry],cons_cnt=1444, prod_cnt=1444

                2020-06-15 16:25:16,852 - [INF] [SlFifoAppThread_Entry],cons_cnt=1848, prod_cnt=1848

                2020-06-15 16:25:19,847 - [INF] [SlFifoAppThread_Entry],cons_cnt=2183, prod_cnt=2183

                2020-06-15 16:25:22,842 - [INF] [SlFifoAppThread_Entry],cons_cnt=2519, prod_cnt=2519

                2020-06-15 16:25:25,853 - [INF] [SlFifoAppThread_Entry],cons_cnt=2851, prod_cnt=2851

                ......

                2020-06-15 16:40:22,670 - [INF] [SlFifoAppThread_Entry],cons_cnt=104930, prod_cnt=104930

                2020-06-15 16:40:25,681 - [INF] [SlFifoAppThread_Entry],cons_cnt=105266, prod_cnt=105266

                2020-06-15 16:40:28,676 - [INF] [SlFifoAppThread_Entry],cons_cnt=105603, prod_cnt=105603

                2020-06-15 16:40:29,924 - [CyFxSlFifoPtoUDmaCallback], CY_U3P_DMA_CB_ERROR(64)

                2020-06-15 16:40:29,940 - [CyFxSlFifoPtoUDmaCallback], dma_state=5, prod_xfer_cnt=8065804, cons_xfer_cnt=8065800

                2020-06-15 16:40:31,671 - [INF] [SlFifoAppThread_Entry],cons_cnt=105746, prod_cnt=105746

                • 5. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
                  RashiV_61

                  Hi,

                   

                  - Please let me know the SDK version that you are using

                  - Are you doing only BULK IN transfers when this error occurs? Are there any concurrent transfers going on while doing BULK IN? If yes can you please do only BULK IN transfers and let me know the results

                  -  As mentioned by you " When this error happened, we got an incomplete data package (lost the leading part)  but no error message."

                  >> Do you mean that there is a data loss when this error occurs?

                  - Is the FX3 device connected as a Superspeed device (USB 3.0) or a Highspeed device (USB2.0)?

                   

                   

                  Regards,

                  Rashi

                   

                  • 6. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
                    NCH_4488531

                    HI,

                     

                    - Please let me know the SDK version that you are using

                       SDK Version: 1.3.4;

                       PC driver: Win_USB

                     

                    - Are you doing only BULK IN transfers when this error occurs? Are there any concurrent transfers going on while doing BULK IN? If yes can you please do only BULK IN transfers and let me know the results

                       One Bulk IN endpoint and 3 Bulk OUT endpoints work at the same time.

                       Our software created 4 threads to comminucate with the board:

                       Thread 1: EP3_OUT send a package (no more than 512B), EP3_In reply 2 packages (all of them are short packages);  Send interval: 50ms;

                       Thread 2: EP3_OUT send a package (no more than 512B), EP3_In reply 2 to 5 packages (all of them are short packages);  Send interval: 50ms;

                       Thread 3: EP10_OUT send 10MB size of packages continuously without sleep. We tried 2MB per package, got the same issue;

                       Thread 4: EP11_OUT send 10MB size of packages continuously without sleep. We tried 2MB per package, got the same issue;

                     

                        Thread 1 & 2, without 3 & 4 work OK in hours, we did not test long, because we need all 4 threads in our product;

                        Thread 1, 2 and one of Thread 3 or 4 work OK in hours, or days, but the problem will occure at last;    When all 4 threads work together, the problem will occure in minutes (1 to 20 minutes, mostly nearly 5 minutes);

                        And when this error happens, FLAG_A (dedicated to EP3_IN) keeps LOW (Busy) until RESET EP3_IN from the host;

                     

                    -  As mentioned by you " When this error happened, we got an incomplete data package (lost the leading part)  but no error message."

                    >> Do you mean that there is a data loss when this error occurs?

                       When DMA mode is Auto or Auto_Signal, BusHound can capture an incomplete package, lost the leading part, and always lost the leading part, but the number of Bytes that lost is uncertern.

                     

                    - Is the FX3 device connected as a Superspeed device (USB 3.0) or a Highspeed device (USB2.0)?

                       Superspeed device (USB 3.0);

                    • 7. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
                      RashiV_61

                      Hello,

                       

                      Please refer to this KBA Simultaneous IN/OUT USB Transfers in EZ-USB® FX3™ – KBA94607 which mentions that simultaneous data transfers on USB devices should be in a time-multiplexed fashion. As thread 3 and 4 runs continuously there might be a possibility that one of the transfers goes to suspend state while other is running (the possible reason of data loss)

                       

                      Regards,

                      Rashi

                      • 8. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
                        NCH_4488531

                        Hi,

                         

                        Thanks for your help!

                         

                        I have been thinking for a solution to fix this issue.

                         

                        From the HOST side, it's hard to know when the IN data will come; and we don't know when the FPGA can consume the OUT data either.

                        We also need to meet the requirment of band-width of OUT data. So we can not just wait a timeout of ReadPipe call.

                         

                        I think the best way to avoid simultaneously bi-directional data transfer is to do something in FX3 firmware.

                        Is there any way or API that can tell wether there is data transfer on-going on the USB bus (Sending or Receivingļ¼‰?

                        Or how to know FX3 received an OUT/IN request (the start of data transfer)?

                        • 9. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
                          RashiV_61

                          Hello,

                           

                          In the case of EP IN transfers, the CONS event will be generated when the data from the DMA buffer is consumed by the consumer (USB). So this can be used to send the status of USB transfer But to send the status of the USB transfer status from the device one more endpoint needs to be polled for the status which would increase the number of data transfers.

                          So, it would be good to handle the scheduling of the different threads on the host side as the requests are sent by the host. Depending on the status of one transfer the next transfer needs to be scheduled.

                           

                          Regards,

                          Rashi

                          • 10. Re: FX3 EP_IN data transfer stop and got CY_U3P_DMA_CB_ERROR in DMA Callback
                            NCH_4488531

                            OK! We will try to figure something out.

                             

                            Thanks again!