1 2 Previous Next 24 Replies Latest reply on Mar 11, 2020 7:58 PM by SaLe_1201446

    FX3 data loss in thread switching

    SaLe_1201446

      Dear all,

       

      I am using the FX3-USB3014 to capture MPEG2 TS data from LG electronics demodulator IC.

      Because there is no flow control mechanism between FX3-GPIFII and demodulator output,

      so I adopted the 2 thread ping-pong DMA structure and similar GPIF II state machine as explained in application note AN-75779 UVC image sensor examples.

      I don't have any need to change the stream data, then I created AUTO Many-to-ONE DMA channels, and 4 buffers each thread, each buffer has 16KB(16384 bytes) size.

       

      This is my DMA channel creation,

       

      // Create a multi-DMA AUTO channel for the GPIF to USB transfer

      CyU3PMemSet((uint8_t *)&dmaMultiConfig, 0, sizeof(dmaMultiConfig));

      dmaMultiConfig.size            = 16384;

      dmaMultiConfig.count          = 4;

      dmaMultiConfig.validSckCount  = 2; // Number of producer sockets

      dmaMultiConfig.prodSckId[0]   = (CyU3PDmaSocketId_t)PING_PRODUCER_SOCKET;

      dmaMultiConfig.prodSckId[1]   = (CyU3PDmaSocketId_t)PONG_PRODUCER_SOCKET;

      dmaMultiConfig.consSckId[0]   = (CyU3PDmaSocketId_t)CONSUMER_ENDPOINT_SOCKET;

      dmaMultiConfig.dmaMode        = CY_U3P_DMA_MODE_BYTE;

      Status = CyU3PDmaMultiChannelCreate(&glDualGPIF2USB_Handle, CY_U3P_DMA_TYPE_AUTO_MANY_TO_ONE, &dmaMultiConfig);

       

      When I captured the packets through EP1-IN (BULK IN transfer),

      there is some data loss (or 1 byte more data) around each 16KB boundary of the captured data.

      Yet, this data loss result is different from which types of valid signal is coming. 

      As I have tested, it is depends on the signal pattern of incoming "VALID" signal.

       

      When the signal have 188-byte-length continuous style, like Fig. 1),  there is NO data loss in capturing.

      On the contrary, if the valid signal pattern is changed to like Fig 2),  it has 4 or 5 clocks intervals between each valid signal,

      every 16KB boundary , there is some data loss (about 6-bytes) or invalid data insertion in captured data.

       

      Except the valid signal format, all other test conditions are same .

       

      I have no idea why the data loss result is relevant to valid signal pattern.

      I tried several version of GPIF II , including almost same version of AN 75779 UVC example, but the result was same.

      Always data loss was occurered in Fig. 2 case in 16KB boundary.

      If I changed the DMA buffer size to 8KB,  then the data loss found in 8KB boundary.

       

      Attached is the GPIF II source which is used.

       

      I would appreciate any help I can get, so thanks in advance.

       

      Thank you.

       

      Sangcheol

       

       

        • 1. Re: FX3 data loss in thread switching
          JayakrishnaT_76

          Hello,

           

          Please answer my following questions so that I can understand the situation better:

          1. According to my understanding in Fig 1, you are trying to send 188 bytes within a single VALID signal and in Fig 2, you are trying to send 188 bytes within each VALID signals. That is, in fig 2 you are trying to send 188 bytes when each VALID pulse is issued by the demodulator. So, in Fig 1 you are sending 188 bytes of data only once where as in Fig 2, you are sending 188 bytes of data multiple times. Please correct me if Iam wrong.

          2. From your query, I understood that at times, about 6 bytes are lost and at times an unwanted byte is added at the boundary of DMA buffer. Please let me know whether you are losing exactly 6 bytes whenever you lose data and one unwanted byte whenever you get extra data.

          3. Also, you mentioned that the data loss is happening around each 16 KB boundary. Please let me know whether the change in data is seen to the end of a 16 KB buffer or at the start of the buffer.

           

          Also, please share the interface timings so that we can evaluate the shared GPIF II project and understand if it is matching the interface requirements.

           

          Best Regards,

          Jayakrishna  

          • 2. Re: FX3 data loss in thread switching
            SaLe_1201446

            Hi, Jayakrishna

             

            Here is my new drawings of Fig1, and Fig2

             

             

             

             

             

            1. All of these signals (CLK, VALID, SOP, DATA[7:0]) are issued from demodulator IC.

            Fig1 case, the valid signal maintains for 188 bytes clock length continuously,  data comes in every clock, and then some invalid period,  and then valid again, ...

            Fig 2 case, the valid signal maintains 1 clock period and 1 byte data comes in, and 4 or 5 clocks invalid period, the next valid byte comes in....

             

            2. As my test,

               at the end position of the first 16KB boundary, unwanted 1 byte added. (this error has occurred consistently in my test)

               but from the end of 2nd 16KB boundary, at times 6 bytes lost, at times 5 bytes lost, at times 4 bytes,  ... it is changed test by test , did not show any regular patterns.

             

            3. In my test, error is seen the end of 16KB.

              

            4. Here is clock and valid signal waveform captured of Fig 2 case.

            I have already checked timing of data, valid, clock of demodulator output and found nothing wrong in the timing requirements of FX3.

             

              clock freq. = 12.0MHz

             

             

            Best regards,

             

            Sangcheol

             

             

             

             

             

            • 3. Re: FX3 data loss in thread switching
              JayakrishnaT_76

              Hello Sangcheol,

               

              Please find the modified GPIF II project attached. Please use this and let me know if you find any difference in the present behaviour.

               

              Also, please let me know if after the sampling of VALID signal at a particular clock edge, the data remains same until next clock edge. That is the data whose VALID signal was sampled at a particular clock edge will remain unchanged until next clock edge.

               

              Best Regards,

              Jayakrishna

              • 4. Re: FX3 data loss in thread switching
                SaLe_1201446

                Hi Jayakrishna,

                 

                1. Your GPIF II does not work,  after apply it the DMA does not start.

                When I build it,  I got waring messages like this.

                 

                 

                 

                 

                2. Here is a waveform of valid and data, captured by logic analyzer tool.

                  CLK,  TP_SOP,  TP_VALID, TP_ERR, TP_DATA  order.

                 

                 

                 

                Best regards,

                 

                Sangcheol

                • 5. Re: FX3 data loss in thread switching
                  SaLe_1201446

                  Hi, Jayakrishna

                   

                  I have already tried to work around this issue by changing GPIF II design several times, but could not find any solution, the attached file is one of the GPIF II design that I tried to test. Basically the signal from demodulator is very similar to AN75779 image sensor case, therefore not much changed from the UVC example,  except changing signal name (LV-> TP_VALID ... etc) and deleting unneeded states.

                   

                  And I also attached 1 M bytes size captured data of my test result.

                  For easy to find error location, instead capturing real demodulator output,

                  I replaced the data with linear increasing number pattern.

                   

                  As you can see in that file,  one byte inserted at 0x3FFF position,  -- the nubmer 0x44  value --

                   

                   

                  and around 0x7FFF position,

                  there is some bytes lost between 0x02D244C9 and 0x02D244CD value.

                   

                   

                   

                   

                  I hope this is helpful to find the reason of error.

                   

                  Best regards,

                   

                  Sangcheol

                  • 6. Re: FX3 data loss in thread switching
                    JayakrishnaT_76

                    Hello,

                     

                    According to the interface timings that you shared before, when one VALID was sent from the demodulator, only one Byte is transferred to the FX3. So, please do the following modification to the state machine.

                    Please remove the direct transitions from state PUSH_DATA_SCK0 to PUSH_DATA_SCK1 and from PUSH_DATA_SCK1 to PUSH_DATA_SCK0. That is the transitions from PUSH_DATA_SCK0 to PUSH_DATA_SCK1 having transition equation TP_VALID&DATA_CNT_HIT and from PUSH_DATA_SCK1 to PUSH_DATA_SCK0 having transition equation TP_VALID&ADDR_CNT_HIT are to be removed from the state machine shared in your response 5.

                    Please let me know the result after making this change.

                     

                    Best Regards,

                    Jayakrishna

                    • 7. Re: FX3 data loss in thread switching
                      JayakrishnaT_76

                      Hello,

                       

                      Also, in addition to my previous response, you can uncheck the repeat Repeat Actions until next transition in States PUSH_DATA_SCK0 and PUSH_DATA_SCK1 of the same state machine. The snapshot of the same is provided below:

                       

                      Best Regards,

                      Jayakrishna

                      • 8. Re: FX3 data loss in thread switching
                        SaLe_1201446

                        Hi, Jayakrishna

                         

                        There is no difference in data error after applying your recommendation in response 6,7.

                         

                        Best Regards,

                         

                        Sangcheol

                        • 9. Re: FX3 data loss in thread switching
                          JayakrishnaT_76

                          Hello,

                           

                          Please share the data sampled by FX3 with the state machine you were using before and after the modifications that I mentioned in response 6 and 7 in .txt format. It seems like I cant open the .dat format that was shared before in your response 5.

                           

                          Best Regards,

                          Jayakrishna

                          • 10. Re: FX3 data loss in thread switching
                            JayakrishnaT_76

                            Hello,

                             

                            In addition to my previous response, please check that the demodulator IC is sending the 16384th byte properly. That is oxCA and not 0x44.

                             

                            Also, in the state machine, you can try modifying the transition equation from WAIT_FULL_SCK0 to PUSH_DATA_SCK1 by TP_VALID&DMA_RDY_TH1. For the transition from WAIT_FULL_SCK1 to PUSH_DATA_SCK0, the equation can be modified as TP_VALID&DMA_RDY_TH0.

                             

                            Best Regards,

                            Jayakrishna

                            • 11. Re: FX3 data loss in thread switching
                              SaLe_1201446

                              Hi,

                               

                              Please use a free hex data editor like HxD - Freeware Hex Editor and Disk Editor ,  https://mh-nexus.de/en/hxd/

                              you can open the response 5 data with HxD. please set the 'bytes per line 4' to see response 5 figure that I sent.

                               

                              The linear pattern data is generated by FPGA code to find out where the error is occurred,

                              but the used VALID signal is generated from demodulator itself. 

                              This is because it is hard to find where the error is occurred from the actual demodulaotr IC output,

                              The actual demodulator IC output is MPEG2-TS packets, so it has random-like data value.

                              Please refer to the following figure for my test configuration.

                              It is just to find out the error location purpose.

                               

                              And I am sure there is no error in either demodulator IC output or my FPGA linear pattern data.

                              As I already mentioned , if the TP_VALID style has changed to Fig 1 style in response 2,  there is no data loss in captured files.

                              Only Fig 2 style TP-VALID shows data error around 16KB boundary.

                               

                               

                               

                               

                              Best regards,

                               

                              Sangcheol

                              • 12. Re: FX3 data loss in thread switching
                                JayakrishnaT_76

                                Hello,

                                 

                                Please share the traces obtained by ILA probes when you are sending the 3FFFth byte and 7FFFth byte. This is to ensure that the problem is not caused by FPGA or demodulator so that we can check for issues on FX3 GPIF II.

                                In the first case, that is within one VALID, you were sending 188 bytes. So the implementation on FPGA side according to my understanding should be different compared to this case. In this case, you are sending one byte from FPGA per VALID from demodulator.

                                 

                                Also, please let me know if you are seeing exactly the same data when you changed the state machine based on my suggestions in response 6 and 7. You can verify this by taking a diff using Applications like Beyond Compare 4.

                                 

                                Also, please try modifying the transition equation from WAIT_FULL_SCK0 to PUSH_DATA_SCK1 by TP_VALID&DMA_RDY_TH1. For the transition from WAIT_FULL_SCK1 to PUSH_DATA_SCK0, the equation can be modified as TP_VALID&DMA_RDY_TH0. Please try this and let me know if you find any difference in the observations.

                                 

                                Best Regards,

                                Jayakrishna

                                • 13. Re: FX3 data loss in thread switching
                                  SaLe_1201446

                                  Hi,

                                   

                                  1. FPGA data pattern is used just to show you where the error is. 

                                     And I think comparing data has no meaning, even though the error location is always same but the content is different from capture by capture

                                     because FPGA pattern generation logic is not synchnized with the each capture start instant.

                                     so don't pay attention too much about linear pattern data and its value. 

                                     As I said, the purpose of linear pattern data is to find the error location only.

                                   

                                     I attached the real demodulator MPEG2-TS output captured.

                                     Please look inside it,  you can see 0x47 (TS sync byte) alignment error is found around every 16KB boundary.

                                     (If there is no data error in MPEG2-TS file, the TS sync byte(0x47) has to always show up at the first position of every 188 bytes,

                                      no alignment mismatch is expected for 0x47. )

                                   

                                   

                                  2. There is no difference even DMA_RDY_TH0/1 flag used. Changed also attached.

                                   

                                   

                                  Best regards,

                                   

                                  Sangcheol

                                  • 14. Re: FX3 data loss in thread switching
                                    JayakrishnaT_76

                                    Hello,

                                     

                                    It seems like the state machine is okay. Can you share the firmware so that we can have a better understanding?

                                     

                                    Best Regards,

                                    Jayakrishna

                                    1 2 Previous Next