3 Replies Latest reply on Oct 19, 2020 4:35 AM by RashiV_61

    CyU3PDebugPrint glDebugLock deadlock

    user_4770066

      Hello!

       

      I'm building a UVC device based on the FX3 SDK 1.3.4.

      My firmware prints lots of debug output via UART and hangs after a while.

       

      Using the debugger, I found that several threads seem to be waiting for the glDebugLock mutex:

      Some thread owns the mutex, but it does not print any new output.

       

      I checked the source code of CyU3PDebugPrint and noticed that there is no CyU3PMutexPut call before the first return:

      CyU3PReturnStatus_t 
      CyU3PDebugPrint (
              uint8_t priority,
              char *message,
              ...)
      {
      ...
          CyU3PMutexGet (&glDebugLock, CYU3P_WAIT_FOREVER);
      ...
          stat = CyU3PDmaChannelCommitBuffer (&glDebugChanHandle, limit, 0);
          if (stat == CY_U3P_SUCCESS)
          {
              stat = CyU3PDmaChannelGetBuffer (&glDebugChanHandle, &glDebugBuf_p, glDbgTimeout);
          }
          if (stat != CY_U3P_SUCCESS)
          {
              CyU3PDebugChannelReset ();
              return stat;
          }
          glDebugBufOffset = 0;
          CyU3PMutexPut (&glDebugLock);
          return CY_U3P_SUCCESS;
      }
      

      If the CyU3PDmaChannelCommitBuffer or the CyU3PDmaChannelGetBuffer call fails, the function will exit early and the glDebugLock mutex will not be unlocked. Any future call to CyU3PDebugPrint from a different thread will block indefinitely.

       

      Is it actually intended to return early from that function? Other functions like CyU3PDebugLogFlush do not return after calling CyU3PDebugChannelReset.

       

      Regards,

      Tobias

        • 1. Re: CyU3PDebugPrint glDebugLock deadlock
          RashiV_61

          Hello Tobias,

           

          As you mentioned that CyU3PDebugPrint API is called at many instances in the firmware, please let me know if CyU3PDebugPrint is called inside the DMA callback or any other callbacks

           

          My firmware prints lots of debug output via UART and hangs after a while.

          >> Please let me know approx how many bytes are written using UART debug prints from one thread and how many such threads are running in your firmware.

          >> Also, let me know the return status of CyU3PDebugPrint when the UART prints hang.

           

          Regards,

          Rashi

           

           

           

           

          • 2. Re: CyU3PDebugPrint glDebugLock deadlock
            user_4770066

            Hello Rashi,

             

            CyU3PDebugPrint is called in the main thread, the DMA callback, the PIB callback and the UIB callback (mainly to report errors).

             

            The output is approximately less than 40 lines or 2kb per second

             

            If glDebugLock is locked, CyU3PDebugPrint won't return.

            I logged previous return values in an array: 0 (CY_U3P_SUCCESS), 19 (CY_U3P_ERROR_INVALID_CALLER) and 29 (CY_U3P_ERROR_MUTEX_FAILURE).

            CY_U3P_SUCCESS and CY_U3P_ERROR_INVALID_CALLER can be returned by CyU3PDebugPrint.

            But CY_U3P_ERROR_MUTEX_FAILURE does not appear in that function, so it must have been returned by CyU3PDmaChannelCommitBuffer or CyU3PDmaChannelGetBuffer. This means that "return stat" in line 18 (originally line 548) was executed and that glDebugLock was not released.

             

            Regards,

            Tobias

            • 3. Re: CyU3PDebugPrint glDebugLock deadlock
              RashiV_61

              Hello Tobias,

               

              We do not recommend to call CyU3PDebugPrint inside the callback function.

              fx3 DmaMultiChannel Manual_Many_to_one always hang in the DMA callback

              How to print debug message in timer call back function.

               

              As per section 5.17.2.2 of FX3 API guide (SDK), no blocking calls should be made from the DMA  callback function

              The CyU3PDebugPrint() API can encounter a failure when invoked from a callback function. You can check the return value of the API by setting it to a global variable and reading it in the thread main loop.

              Please let me know if the problem still occurs when the CyU3PDebugPrint is removed from the callback function. Meanwhile, I will try to reproduce the issue at my end.

               

               

              Regards,

              Rashi