1 2 3 Previous Next 32 Replies Latest reply on Mar 11, 2020 4:08 AM by YashwantK_46

    AN75779 modification - FX3 with specific image sensor

    RiFu_3006586

      Hi,

      i have a Enclustra PM3 with Mars ZX3 Zynq 7000 SoC

      On PM3 Board a FX3 3014  is assembled.

      Because of the given schematic I can't use GPIO29 als FV Pin.

      In GPIF II Designer I have changed FV to GPIO21

       

      The Zynq generate Video Timing for 720p Video Output

      The Pixel Clock is 100MHz, so I get ~80 fps

      The IIC Interface is connected to a AT24 EEPROM and the IIC address is changed.

      The FX3 configures the Eeprom instead of the image sensor, this works.

       

      But in AMCAP the is ablosutely no information shown aboaut the camera. The software says Acess denied.

      Wehe I activate the DEBUG_PRINT_FRAME_COUNT the console says: UVC: Completed 0 frames and 0 buffers

       

      What is going wrong?

        • 1. Re: AN75779 modification - FX3 with specific image sensor
          YashwantK_46

          Hello,

           

          Can you please probe the FV LV lines and share them?

          Also, which data format are you streaming from the sensor?

           

          Is your application still based on UVC?

           

          Regards,

          Yashwant

          • 2. Re: AN75779 modification - FX3 with specific image sensor
            RiFu_3006586

            The Sensor streames 8 bit uncompressed mono data.

            The application is the UVC example. Only the port is changed, the IIC address+Sensor Init and the framerate.

            • 3. Re: AN75779 modification - FX3 with specific image sensor
              YashwantK_46

              Hello,

               

              Can you please share the trace again with PCLK, FV and LV being shown?

              The pervious didn't have clear info about PCLK and FV.


              From the previous trace, this is what i can tell:

              1.) Please ensure that as soon as FV is asserted, LV should also get asserted right at that same time but there is a significant delay between FV assertion and LV assertion in the above trace.

              2.) Also, LV was stopped way before FV was de-asserted in the previous frame ( found from the trace)  - Before the FV for next frame is asserted.

               

              Please make sure that FPGA assert FV and LV at the same time and then only the device will start streaming.

               

              Also, can you please share the UART debug logs in the next interaction as well?

               

              Regards,
              Yashwant

              • 4. Re: AN75779 modification - FX3 with specific image sensor
                RiFu_3006586

                I think it works now.

                On a nother PC  with Linux and VLC I get an image.

                Every 10 seconds the live image hangs for 2 seconds and then it runs for 10 seconds.

                 

                I would upgrade the system to 24 Bit RGB interface. Is there a Windows compatible GUID for 24 Bit RGB Raw data which rund with amcap, vlc virtualdub ..?

                Thank you very much

                • 5. Re: AN75779 modification - FX3 with specific image sensor
                  RiFu_3006586

                  I changed from the working 16 bit YUV Design to a 24 bit interface, but  the design doesn't work.

                   

                  In GPIFII Designer --> Data Bus width from 16 to 24

                  LD Data Count and LD ADDR Count to 5455

                   

                  in cyfxuvcdscr :

                  /* Class specific Uncompressed VS format descriptor 24-bit RGB */
                  0x1B,                      /* Descriptor size */
                  0x24,                      /* Class-specific VS I/f Type */
                  0x04,                      /* Subtype : uncompressed format I/F */
                  0x01,                      /* Format desciptor index */
                  0x01,                      /* Number of frame descriptor followed */
                  /* GUID, globally unique identifier used to identify streaming-encoding format: YUY2  */
                  0x7D, 0xEB, 0x36, 0xE4,        /* MEDIASUBTYPE_RGB888 GUID: E436EB7D-524F-11CE-9F53-0020AF0BA770  */
                  0x4F, 0x52, 0xCE, 0x11,
                  0x9F, 0x53, 0x00, 0x20,
                  0xAF, 0x0B, 0xA7, 0x70,
                  0x18,                      /* Number of bits per pixel */
                  0x01,                      /* Optimum Frame Index for this stream: 1 */
                  0x10,                      /* X dimension of 10 the picture aspect ratio; Non-interlaced */
                  0x09,                      /* Y dimension of the pictuer aspect ratio: Non-interlaced */
                  0x00,                      /* Interlace Flags: Progressive scanning, no interlace */
                  0x00,                      /* duplication of the video stream restriction: 0 - no restriction */

                   

                  AND

                   

                  /* Class specific Uncompressed VS frame descriptor */
                  0x1E,                      /* Descriptor size */
                  0x24,                      /* Descriptor type*/
                  0x05,                      /* Subtype: uncompressed frame I/F */
                  0x01,                      /* Frame Descriptor Index */
                  0x01,                      /* Still image capture method 1 supported */
                  0x00, 0x05,                /* Width in pixel */
                  0xD0, 0x02,                /* Height in pixel */
                  0x00,0x00,0x78,0x69,       /* Min bit rate bits/s. 1280*720*24*80*/
                  0x00,0x00,0x78,0x69,       /* Max bit rate bits/s. */
                  //   0x00,0x00,0x50,0x46,       /* Min bit rate bits/s.1280*720*16*80 */
                  //   0x00,0x00,0x50,0x46,       /* Max bit rate bits/s. */
                  0x00,0x30,0x2A,0x00,       /* Maximum video or still frame size in bytes(Deprecated)1280*720*3*/
                  //   0x00,0x20,0x1C,0x00,       /* Maximum video or still frame size in bytes(Deprecated) 1280*720*2*/
                  0x48, 0xE8, 0x01, 0x00,    /* 80fps */
                  0x01,
                  0x48,0xE8,0x01,0x00,

                   

                  The Design does'nt work.

                  UART Print the followig message:

                   

                  .....

                  .....

                  Application Stopped

                  Application Started

                  UVC: Completed 0 frames and 0 buffers

                  DMA Reset Event: Frame timer overflow, time period = 76

                  Application Stopped

                  Application Started

                  UVC: Completed 0 frames and 0 buffers

                  DMA Reset Event: Frame timer overflow, time period = 76

                  Application Stopped

                  Application Started

                  UVC: Completed 0 frames and 0 buffers

                  DMA Reset Event: Frame timer overflow, time period = 76

                  Application Stopped

                  Application Started

                  UVC: Completed 0 frames and 0 buffers

                  DMA Reset Event: Frame timer overflow, time period = 76

                  Application Stopped

                  Application Stopped after 30 Commit buffer failures

                  UVC: Completed 0 frames and 0 buffers

                  • 6. Re: AN75779 modification - FX3 with specific image sensor
                    YashwantK_46

                    Hello,

                     

                    Can you please share the size and count of DMA buffers that you have allocated while creating the DMA channel?


                    Can you please share wireshark traces for the situation when you saw that the firmware was working on linux PC (Interaction No: 5)?
                    Also, can you please share wireshark traces for the above case when you are using 24-bit RGB stream?

                     

                    Regards,
                    Yashwant

                    • 7. Re: AN75779 modification - FX3 with specific image sensor
                      RiFu_3006586

                      Do you mean these DMA settings:

                      /* UVC Video Streaming Endpoint Packet Size */

                      #define CY_FX_EP_BULK_VIDEO_PKT_SIZE    (0x400)         /* 1024 Bytes */

                       

                      /* UVC Video Streaming Endpoint Packet Count */

                      #define CY_FX_EP_BULK_VIDEO_PKTS_COUNT  (0x10)          /* 16 packets (burst of 16) per DMA buffer. */

                       

                      /* DMA buffer size used for video streaming. */

                      #define CY_FX_UVC_STREAM_BUF_SIZE       (CY_FX_EP_BULK_VIDEO_PKTS_COUNT * CY_FX_EP_BULK_VIDEO_PKT_SIZE)  /* 16 KB */

                       

                      /* Maximum video data that can be accommodated in one DMA buffer. */

                      #define CY_FX_UVC_BUF_FULL_SIZE         (CY_FX_UVC_STREAM_BUF_SIZE - 16)

                       

                      /* Number of DMA buffers per GPIF DMA thread. */

                      #define CY_FX_UVC_STREAM_BUF_COUNT      (4)

                       

                      /* Low Byte - UVC Video Streaming Endpoint Packet Size */

                      #define CY_FX_EP_BULK_VIDEO_PKT_SIZE_L  (uint8_t)(CY_FX_EP_BULK_VIDEO_PKT_SIZE & 0x00FF)

                       

                      /* High Byte - UVC Video Streaming Endpoint Packet Size and No. of BULK packets */

                      #define CY_FX_EP_BULK_VIDEO_PKT_SIZE_H  (uint8_t)((CY_FX_EP_BULK_VIDEO_PKT_SIZE & 0xFF00) >> 8)

                       

                      /* Maximum commit buffer failures to detect a stop streaming event in a MAC OS */

                      #define CY_FX_UVC_MAX_COMMIT_BUF_FAILURE_CNT    (30)

                       

                       

                      I have no Wireshark on the Linux PC because it's a Knoppix, sorry,

                       

                      Thanks

                      Richard

                      • 8. Re: AN75779 modification - FX3 with specific image sensor
                        YashwantK_46

                        Hi Richard,

                         

                        We would firstly need to identify if the image sensor is streaming any data to FX3 or not and if it does, is FX3 not able to pass the data on to the host.

                         

                        So, for the first one, i would need you to share the FV, LV and PCLK traces from an oscilloscope and make sure that they are okay. If that is not proper, means that data from image sensor is not coming to FX3.

                         

                        After that, i would need you to share a wireshark trace of the working and non-working scenarios in a windows PC or wherever you can get a software analyzer trace from to understand if the data from image sensor is reaching the host or not or is the host not issuing any request to read data at all.

                         

                        The above things are necessary to identify what is causing the problem exactly and without them, its not possible to understand the source of the problem.

                         

                        Also, in order to stream RGB, you will have to write your own custom host application as there is no available host applications that can stream RGB raw data.

                         

                        Regards,
                        Yashwant

                        • 9. Re: AN75779 modification - FX3 with specific image sensor
                          RiFu_3006586

                          Thank you for your response.

                          I've captured now the data with wireshark,

                          With 16-bit Interface it is working fine, the image is shown in the application, so the fvalid lvalid interface works.

                          wireshark capture a lot of data in this working szenario, so the dump is very big. Which data do you need?

                           

                          In 24-bit mode, with is not working, i have capured some data too. This dump is attached.

                           

                          Thanks a lot

                          Richard

                          • 10. Re: AN75779 modification - FX3 with specific image sensor
                            YashwantK_46

                            Hello Richard,

                             

                            The wireshark only shows that one buffer is being committed to the host and nothing else.

                             

                            Can you do the following and then share the UART traces and the results:

                            1.) Comment #define FRAME_TIMER_ENABLE in uvc.h file so that the application won't have a DMA reset event when the timer overflows and share the UART logs.

                            2.) Please share the FV, LV and PCLK oscilloscope traces for the 24-bit interface scenario, i am still waiting on them.

                            3.) Please define a global variable and initialize to " 0 " in the firmware and increment the variable in DmaCallback when the CY_U3P_DMA_CB_PROD_EVENT occurs after the CommitBuffer API and print the variable in the for( ; ; ) loop and share the UART log.

                             

                            Please do the following and share the results.

                             

                            Regards,

                            Yashwant

                            • 11. Re: AN75779 modification - FX3 with specific image sensor
                              RiFu_3006586

                              YashwantK_46 schrieb:

                               

                              Hello Richard,

                               

                              The wireshark only shows that one buffer is being committed to the host and nothing else.

                               

                              Can you do the following and then share the UART traces and the results:

                              1.) Comment #define FRAME_TIMER_ENABLE in uvc.h file so that the application won't have a DMA reset event when the timer overflows and share the UART logs.

                              2.) Please share the FV, LV and PCLK oscilloscope traces for the 24-bit interface scenario, i am still waiting on them.

                              3.) Please define a global variable and initialize to " 0 " in the firmware and increment the variable in DmaCallback when the CY_U3P_DMA_CB_PROD_EVENT occurs after the CommitBuffer API and print the variable in the for( ; ; ) loop and share the UART log.

                               

                              Please do the following and share the results.

                               

                              Regards,

                              Yashwant

                              I've tested 1.) 2.) and 3.), there is no extra information in the uart log.

                              Is this the right count position fo 3.)?

                                 /* Commit Buffer to USB*/

                                          status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count + CY_FX_UVC_MAX_HEADER), 0);

                                          if (status == CY_U3P_SUCCESS)

                                          {

                              #ifdef DEBUG_PRINT_FRAME_COUNT

                                              glDmaDone++;

                              #endif

                                             commit++;

                                          }

                                          else

                                          {

                                              not_commit++;

                                              if(glDmaResetFlag == CY_FX_UVC_DMA_RESET_EVENT_NOT_ACTIVE)

                                              {

                                                  glDmaResetFlag = CY_FX_UVC_DMA_RESET_COMMIT_BUFFER_FAILURE;

                                                  CyU3PEventSet(&glFxUVCEvent, CY_FX_UVC_DMA_RESET_EVENT, CYU3P_EVENT_OR);

                                              }

                                              break;

                                          }

                               

                              to 2.) there is no other scope than shown on 

                              It's the same timing that works in 16 bit mode.

                              • 12. Re: AN75779 modification - FX3 with specific image sensor
                                YashwantK_46

                                Hello Richard,

                                 

                                1.) Please share the UART debug logs after commenting the #define FRAME_TIMER_ENABLE with me.

                                 

                                2.) Please share the oscilloscope traces with PCLK, FV and LV clrealy visible to me. The last trace that you shared isn't clear enough to understand.

                                 

                                For 3.), do it as shown below and see the logs

                                 

                                uint32_t Prod = 0;                                   //initializing variable for prod

                                uint32_t Cons = 0;                                   //initializing variable for cons

                                 

                                     In DmaCallback()
                                {

                                     if (type == CY_U3P_DMA_CB_PROD_EVENT)

                                      {  

                                               status = CyU3PDmaMultiChannelCommitBuffer(chHandle, dmaBuffer.count, 0);

                                               //increment the variable after the above line on Sucess

                                               Prod++;

                                     }

                                     else if (type == CY_U3P_DMA_CB_CONS_EVENT)

                                     {

                                          Cons++;

                                     }

                                }

                                 

                                in for (;;)

                                 

                                use the below line:

                                CyU3PDebugPrint(4,"\n\r Prod = %d Cons = %d", Prod, Cons);

                                 

                                Do the above steps and share the UART log with me so that I can compare it with the wireshark trace that you have shared.

                                 

                                Regards,
                                Yashwant

                                • 13. Re: AN75779 modification - FX3 with specific image sensor
                                  RiFu_3006586

                                  When I reduce the pixelclock from 100MHz to 66 MHz, it works fine in 24 bit.

                                  • 14. Re: AN75779 modification - FX3 with specific image sensor
                                    YashwantK_46

                                    Hello Richard,

                                     

                                    Changing the pixel clock would directly affect your fps. Can you check if the fps and throughput that you are getting is satisfactory to you?

                                     

                                    If you don't want the fps to be effected even after changing the pixel clock, you will have to make sure that the total frame time is similar to when the pixel clock was 100MHz. You can do this by reducing the H-blanking and V-blanking to get up to the satisfactory fps range that you want to achieve.

                                     

                                    So, please modify the blanking times so that you meet the required fps rate in your design.

                                     

                                    Regards,

                                    Yashwant

                                    1 2 3 Previous Next