14 Replies Latest reply on Mar 1, 2020 11:38 PM by YashwantK_46

    Dual Aptina MT9M114 MIPI stream @2560x720p

    MaSt_4567191

      I am driving the CX3 input by a MIPI video stream @2560x720p @25 fps (1 data lane, CSI clock @496MHz)

      The MIPI stream is generated by a pair of sensors @1280x720p merged by a Lattice FPGA with mip2csi IP to get the CX3 input.

      The configuration is successfully performed (with the obvious error that this will not work with USB 2.0) and the YVC CX3 is enumerated correctly. The FPGA is under debug so far ...

      I am using the UVC Troubleshooting Guide – KBA226722  as a reference, still I would like to know in advance if the community can suggest if there are any issue

       

      Thanks much

        • 1. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
          MaSt_4567191

          I read from https://community.cypress.com/docs/DOC-17227

          Since I cannot commet or rate this page I am here to ask clarifications

           

           

           

          //4. Make sure that the DMA buffer size is chosen in such a way that the total frame size in bytes is

          // not divisible by (DMA buffer size – 16). In the DMA buffer configuration, 12 bytes are reserved for

          // UVC Header and 4 bytes for footer and hence the image data would constitute (DMA buffer size – 16)

          // bytes. Reason being a partial DMA buffer in DMA callback would indicate the end of the frame where

          // ‘Frame end marker’ is inserted into the payload header.

           

          // 5. Calculate the number of full DMA buffers that would be required for each frame:

          // Total number of bytes per frame: S = (Image Width in Pixels) * (Image Height in Pixels) * (Pixel depth in bytes)

          // Total number of full DMA buffers per frame: F= Greatest integer less than ( S / (DMA buffer size – 16))

          // Number of bytes in the Partial buffer: P = DMA buffer size * fractional part of (S / (DMA buffer size – 16))

          // Note: This partial buffer size (P) is equal to the dmaBuffer.count when partial buffer is received in DMA callback.

           

          Where is exactly the code for checking this? I suppose there is a macro somewhere and I am checking the project now. Still, I am scary to mess up the operation ...

           

          Thanks much

          • 2. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
            MaSt_4567191

            About point #3 I am confused by endianess since no example is provided

            In my case

            nr.of bytes sent in a frame 1280x2x720x2  = 3686400 (YUY2)

            while

             

            0x00, 0x48, 0x3F, 0x00,             /* Max video frame size in bytes

             

            Is this 00483f00 003f4800 3f000048 ?

            None of this matches with my number

             

            //3. Make sure that maximum video frame size set in the probe control structure

            //(glProbeCtrl) is equal to or greater than the number of bytes sent in one frame.

            //Also, make sure that the maximum payload size set in the probe control structure

            //is more than the size sent in the payload (which is typically the DMA buffer size).

            // ...

            //

            /* UVC Probe Control Settings */

            uint8_t glProbeCtrl[CX3_UVC_MAX_PROBE_SETTING] = {

                0x00, 0x00,                         /* bmHint : No fixed parameters */

                0x01,                               /* Use 1st Video format index */

                0x01,                               /* Use 1st Video frame index */

                0x0A, 0x8B, 0x02, 0x00,             /* Desired frame interval in 100ns */

                0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

                0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

                0x00, 0x00,                         /* Compression quality control */

                0x00, 0x00,                         /* Window size for average bit rate */

                0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

                0x00, 0x48, 0x3F, 0x00,             /* Max video frame size in bytes endianess???*/

            #ifdef CX3_UVC_1_0_SUPPORT

                0x00, 0x90, 0x00, 0x00              /* No. of bytes device can rx in single payload: 32KB */

            #else

                /* UVC 1.1 Probe Control has additional fields from UVC 1.0 */

                0x00, 0x90, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 32KB */

                0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

                0x00,                               /* Framing Information - Ignored for uncompressed format*/

                0x00,                               /* Preferred payload format version */

                0x00,                               /* Minimum payload format version */

                0x00                                /* Maximum payload format version */

            #endif

            };

            • 3. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
              JayakrishnaT_76

              Hello Marco,

               

              Please find my comments to your following questions:

              4) DMA channel is created in cycx3_uvc.c file. The channel configurations can be found inside the same file. Typically this is of the form:

                

                  dmaCfg.size                 = CX3_APP_STREAM_BUF_SIZE;

                  dmaCfg.count                = CX3_APP_STREAM_BUF_COUNT;

                  dmaCfg.validSckCount        = CX3_APP_SOCKET_COUNT;

                  dmaCfg.prodSckId[0]         = CX3_PRODUCER_PPORT_SOCKET_0;

                  dmaCfg.prodSckId[1]         = CX3_PRODUCER_PPORT_SOCKET_1;

                  dmaCfg.consSckId[0]         = CX3_EP_VIDEO_CONS_SOCKET;

                  dmaCfg.dmaMode              = CY_U3P_DMA_MODE_BYTE;

                  dmaCfg.notification         = CY_U3P_DMA_CB_PROD_EVENT | CY_U3P_DMA_CB_CONS_EVENT;

                  dmaCfg.cb                   = CyCx3AppDmaCallback;

                  dmaCfg.prodHeader           = CX3_APP_PROD_HEADER;

                  dmaCfg.prodFooter           = CX3_APP_PROD_FOOTER;

                  dmaCfg.consHeader           = 0;

                  dmaCfg.prodAvailCount       = 0;

               

               

                  status = CyU3PDmaMultiChannelCreate (&glChHandleUVCStream, CY_U3P_DMA_TYPE_MANUAL_MANY_TO_ONE , &dmaCfg);

               

              Out of this, dmaCfg.size represents the size of the DMA channel. This size combines the actual payload size and the UVC header and trailer. We make use of 12 byte header and 4 byte trailer. So the total size of the DMA buffer is payload size + 16(for header and trailer) or the actual image data in a buffer is DMA buffer size - 16bytes.

               

              5) This calculation should be done by yourself. Please find how this calculation is done below:

              1. S is the number of bytes in a frame : S = Image width * image depth * Pixel depth in bytes

              This will be stored in DMA buffers and committed to the host. There will be certain number of full buffers and partial buffers for storing the image data.

               

              2. The number of full buffers is : F = Greatest integer less than (S/(DMA Buffer size -16))

              Please note that DMA Buffer size - 16 is again the actual video data present in a buffer.

               

              3. Number of bytes in partial buffer is computed as: P = DMA buffer size * fractional part of (S / (DMA buffer size – 16))

               

              Best Regards,

              Jayakrishna

              1 of 1 people found this helpful
              • 4. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                MaSt_4567191

                Thanks much, very helpful.

                I was assuming (hoping) the calculation would have been performed during the CX3 configuration process (why not? all the ingredients are in place and the dma configuration parameters are set as macro: am I supposed to edit these automatically generated macros or just check them?

                Where just I put / change S, F, P? In the dma descriptor?

                • 5. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                  JayakrishnaT_76

                  Hello Marco,

                   

                  The calculations that I mentioned in my previous response have to be done manually to ensure that the Frame always ends with a partial buffer. The video data is stored in DMA buffers and committed to host when they are full. The last buffer will be a partial buffer. At the beginning of each buffer, 12 bytes of header information is added. This header has some bits which conveys the End of Frame (EOF) information.

                  The bit fields in the header that are used for indicating EOF will be same for all the full buffers and will be different for the partial buffer. Actually this different bit pattern in the header of the partial buffer conveys EOF information to the host. So you need to ensure that the frame always ends with a partial buffer or the partial buffer has some bytes in it.

                   

                  Best Regards,

                  Jayakrishna 

                  • 6. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                    JayakrishnaT_76

                    Hello,

                     

                    Regarding your question in reply 4, your calculation on number of bytes in a frame is correct. But according to the KBA, maximum video frame size set in the probe control structure (glProbeCtrl) should be equal to or greater than the number of bytes sent in one frame.

                     

                    Regarding endianess,

                     

                    0x00, 0x48, 0x3F, 0x00,             /* Max video frame size in bytes

                     

                    This should be read as 0x 003F4800. This value is greater than 0x384000 in your case. So it will not matter.

                     

                    Also, the payload size set in the probe control structure is 32kB and the DMA Buffer size can be found in the DMA channel configuration settings as mentioned above. You need to make sure that the payload size set in the probe control structure is greater than the DMA buffer size.

                     

                    Best Regards,

                    Jayakrishna

                    1 of 1 people found this helpful
                    • 7. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                      MaSt_4567191

                      KBA says

                       

                      "Verify that GUID, mentioned in this section, is supported by Microsoft UVC Driver. (RAW formats are not supported by UVC driver. If the GUID is set to RAW format, UVC driver drops the data which is sent by FX3 and no video can be seen in Standard Video applications)."

                       

                      here is the code

                       

                          /* GUID, globally unique identifier used to identify streaming-encoding format*/

                          0X59,0X55,0X59,0X32,

                      0X00,0X00,0X10,0X00,

                      0X80,0X00,0X00,0XAA,

                      0X00,0X38,0X9B,0X71,

                       

                      how can I check this? I am digging the USB documents with no joy

                       

                      I find in section 2.2 of http://www.cajunbot.com/wiki/images/8/88/USB_Video_Payload_Uncompressed_1.1.pdf

                      YUY2 32595559-0000-0010-8000-00AA00389B71

                      so it looks good but still endianess is of concern if you llok carefully

                      • 8. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                        YashwantK_46

                        Hello,

                         

                        The order of interpreting the GUID in the firmware is as follows:

                        1.) The first 4 bytes are to be reversed: MSB becomes LSB and vice versa.

                        2.) The next 2 bytes are interchanged and the same happens for the next 2 bytes.

                        3.) The remaining 8 bytes are written as it is.

                         

                        So, you need not bother about the GUID since it is automatically added by the CX3 configuration utility when you select the streaming format.


                        As mentioned in the KBA, only YUY2 format is supported by UVC driver and if you want to use any other streaming format, you can select that in the tool and it will be updated in the firmware, but the UVC driver won't be able to take up this data. You will have to write a custom host application if you want to stream RAW formats.

                         

                        Regards,

                        Yashwant

                        1 of 1 people found this helpful
                        • 9. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                          MaSt_4567191

                          ok. A little sadistic to me

                           

                          On Tue, Jan 21, 2020 at 2:19 PM YashwantK_46 <community-manager@cypress.com>

                          • 10. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                            MaSt_4567191

                            I prepared a xsheet implementing the KBA checks as a funcrion of the image parameters (you may want to distribute something like this to cypress customers: I would be happy to attach it to the discussion but donno how)

                             

                            All checks seem ok

                             

                            "In the firmware, you can keep track of F and P (as mentioned above), and check if the values are as expected." which can be done in a debug session right?

                             

                            The KBA is not for MIPI so 8-9-10 are n/a. re there any specific steps for MIPI

                             

                            Thanks

                            • 11. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                              YashwantK_46

                              Hello,

                               

                              When posting a reply, a new text editor pane opens and if you check in the top right of this new text editor pane, there is an option "Use Advanced Editor".

                              Once you click that, a the page will reload with more options that can be used.

                              If you check at the bottom left corner, you can find "Attach", click on it and you can add your xsheet from there.

                               

                              "In the firmware, you can keep track of F and P (as mentioned above), and check if the values are as expected." which can be done in a debug session right?

                              -> Yes, you can keep track of the full and partial buffers in a debug session.

                               

                              You can go through the following link:Steps to Setup up MIPI CSI Camera Solution with CX3 – KBA225748

                              These are the steps on building a CX3 firmware from the cx3 config utility which takes care of the PCLK setting.

                               

                              Also, you can probe the PCLK, FV and LV in CX3 using the test points: H8 (TP16), G6 (TP18) and H5 (TP20) respectively.

                               

                              Regards,

                              Yashwant

                              • 12. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                                MaSt_4567191

                                Attache a validation spreadsheet I composed against the checklist requirements. It is a good frameworks but I have no room to add more cosmetics

                                All checks are OK with my system so USB parameters shall be fine

                                • 13. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                                  YashwantK_46

                                  Hello,

                                   

                                  I went through the excel sheet and all your calculations seems correct.


                                  Thanks for sharing it.

                                   

                                  Regards,

                                  Yashwant

                                  • 14. Re: Dual Aptina MT9M114 MIPI stream @2560x720p
                                    YashwantK_46

                                    Hello,


                                    This issue has been resolved in interaction: 44 of the following thread: Re: CyCx3UvcAppUSBEventCB never executed

                                     

                                    Regards,

                                    Yashwant