8 Replies Latest reply on Jul 11, 2018 6:47 PM by gary.lee_3399181

    How to create a faked line in start of every video frame in CX3 firmware

    gary.lee_3399181

      Assuming I have a 1920x1080 camera sensor connected to CX3, could it be possible to have a 1920x1081 UVC video streaming interface reported to PC by creating a faked line in start of every video frame in CX3 firmware?

       

      Apart from modifying VS Frame Descriptor, what else should I do in CX3 firmware to make it work? Many thanks!

       

      const uint8_t CyCx3USBSSConfigDscr[] =

      {

                :

          /* Class specific Uncompressed VS Frame Descriptor 1 - 1080p@30fps */

          0x1E,                               /* Descriptor size */

          CX3_CS_INTRFC_DESCR,                /* Descriptor type*/

          0x05,                               /* Subtype: Uncompressed frame interface*/

          0x01,                               /* Frame Descriptor Index: 1 */

          0x00,                               /* No Still image capture supported */

          0x80, 0x07,                         /* Width in pixel:  1920 */

          0x38, 0x39, 0x04,                         /* Height in pixel: 1080 1081 */

          0x00, 0x80, 0x53, 0x90, 0x61, 0x3B,             /* Min bit rate (bits/s): 1080 1081 x 1920 x 2 x 30 x 8 = 995328000 996249600*/

          0x00, 0x80, 0x53, 0x90, 0x61, 0x3B,             /* Max bit rate (bits/s): Fixed rate so same as Min */

          0x00, 0x48, 0x57, 0x3F, 0x00,             /* Maximum video or still frame size in bytes(Deprecated): 1920 x 1080 1081 x 2 */

          0x15, 0x16, 0x05, 0x00,             /* Default frame interval (in 100ns units): (1/30)x10^7 */

          0x01,                               /* Frame interval type : No of discrete intervals */

          0x15, 0x16, 0x05, 0x00,             /* Frame interval 3: Same as Default frame interval */

                :

      }

        • 1. Re: How to create a faked line in start of every video frame in CX3 firmware
          Madhu Lakshmipathy

          Hi

           

          Yes it is possible.

           

          Please do similar modifications in the glProbeCtrl.

           

          In the DMA Callback you can add the line, just like how headers are added in the firmware. Or you can send the header as a separate buffer to the USB, the fake line in a next separate buffer to the USB and the followed by the actual video data.

           

          Regards,

          - Madhu

          • 2. Re: How to create a faked line in start of every video frame in CX3 firmware
            gary.lee_3399181

            I've made some changes in the following 3 places as below.

            1. VS Frame Descriptor
            2. glProbeCtrl
            3. DMA call back

             

            However it doesn't work. I see lots of DMA reset events caused by the function call, CyU3PDmaMultiChannelCommitBuffer(), to commit the actual video data to USB.

            Any hints on that? Thanks!

             

            const uint8_t CyCx3USBSSConfigDscr[] =

            {

                      :

                /* Class specific Uncompressed VS Frame Descriptor 1 - 1081p@30fps */

                #if 0

                0x38, 0x04,                                 /* Height in pixel: 1080 */

                0x00, 0x80, 0x53, 0x3B,             /* Min bit rate (bits/s): 1080 x 1920 x 2 x 30 x 8 = 995328000 */

                0x00, 0x80, 0x53, 0x3B,             /* Max bit rate (bits/s): Fixed rate so same as Min */

                0x00, 0x48, 0x3F, 0x00,             /* Maximum video or still frame size in bytes(Deprecated): 1920 x 1080 x 2 */

                #else

                0x39, 0x04,                                 /* Height in pixel: 1081 */

               0x00, 0x90, 0x61, 0x3B,             /* Min bit rate (bits/s): 1081 x 1920 x 2 x 30 x 8 = 996249600 */

                0x00, 0x90, 0x61, 0x3B,             /* Max bit rate (bits/s): Fixed rate so same as Min */

                0x00, 0x57, 0x3F, 0x00,             /* Maximum video or still frame size in bytes(Deprecated): 1920 x 1081 x 2 */

                #endif

                      :

            };

             

             

            /* UVC Probe Control Setting - 1080p@30FPS */

            uint8_t const gl1080pProbeCtrl[CX3_APP_MAX_PROBE_SETTING] = {

                      :

                //0x00, 0x48, 0x3F, 0x00,             /* Max video frame size in bytes = 1920 x 1080 x 2 */

                0x00, 0x57, 0x3F, 0x00,             /* Max video frame size in bytes = 1920 x 1081 x 2 */

                      :

            };

             

            #define LINE_LENGTH (1920*2)

            uint8_t gfakeLine[LINE_LENGTH], gbackupLine[LINE_LENGTH];

             

            /* DMA callback function to handle the produce and consume events. */

            void

            CyCx3AppDmaCallback (

                    CyU3PDmaMultiChannel *chHandle,

                    CyU3PDmaCbType_t      type,

                    CyU3PDmaCBInput_t    *input)

            {

                CyU3PDmaBuffer_t dmaBuffer;

                CyU3PReturnStatus_t status = CY_U3P_SUCCESS;

             

             

                if (type == CY_U3P_DMA_CB_PROD_EVENT)

                {

                    /* This is a produce event notification to the CPU. This notification is

                     * received upon reception of every buffer. The buffer will not be sent

                     * out unless it is explicitly committed. The call shall fail if there

                     * is a bus reset / usb disconnect or if there is any application error. */

             

             

                    status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);

                    while (status == CY_U3P_SUCCESS)

                    {

                        /* Add Headers*/

                        if (dmaBuffer.count < CX3_APP_DATA_BUF_SIZE)

                        {

                            CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_EOF);

                            gIsSOF = CyTrue;

             

                        }

                        else

                        {

                            /* Start of Frame */

                            if (gIsSOF)

                            {

                                CyU3PMemSet(gfakeLine, 0x0, LINE_LENGTH);

                               CyU3PMemCopy(gbackupLine, dmaBuffer.buffer, LINE_LENGTH);   /* Backup the 1st line */

                                CyU3PMemCopy(dmaBuffer.buffer, gfakeLine, LINE_LENGTH);       /* Copy the faked line to DMA buffer */

                                CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_FRAME);                  

                               

                               /* Commit the faked line to USB */

                                status = CyU3PDmaMultiChannelCommitBuffer (chHandle, LINE_LENGTH + 12, 0);

             

                                if (status != CY_U3P_SUCCESS)

                                {

                                    CyU3PDebugPrint (4, "Commit faked line failed, status 0x%x \n\r", status);

                                }

             

                                /* Restore the original first line to DMA buffer */    

                                CyU3PMemCopy(dmaBuffer.buffer, gbackupLine, LINE_LENGTH);

             

                                gIsSOF = CyFalse;

                            }            

                            CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_FRAME);

                        }

                        /* Commit Buffer to USB */           

                        status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count + 12), 0);

                        if (status != CY_U3P_SUCCESS)

                        {

                            CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT, CYU3P_EVENT_OR);

                            break;

                        }

                        else

                        {

                            glDmaDone++;

                        }

                        status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);

                    }

                }

                       :

                       :

            }

            • 3. Re: How to create a faked line in start of every video frame in CX3 firmware
              srdr

              Gary,

               

              Can you please check whether the first commit buffer is failed? i.e. fake line commitbuffer. I guess it should commit successfully. Please confirm.

               

              Then the second commitbuffer (actuall video data buffer) is failing.

              Because, you can not do commitbuffer on the same buffer twice.

              • 4. Re: How to create a faked line in start of every video frame in CX3 firmware
                gary.lee_3399181

                You're right. The first commit is OK, however the second commit is failing.

                So the question becomes how could I send something extra (i.e. the faked line) to the consumer?

                Which APIs should I call? Thanks!

                • 5. Re: How to create a faked line in start of every video frame in CX3 firmware
                  abga

                  Hi,

                   

                  Here in this firmware you are adding fake line at the starting of the frame. You can add a line at the end of the frame as we have partial buffer there. Is it fine for your application?

                   

                  Thanks & regards

                  Abhinav

                  • 6. Re: How to create a faked line in start of every video frame in CX3 firmware
                    gary.lee_3399181

                    Adding the fake line at the end of the frame is also fine for my application.

                     

                    In theory, this approach works for Scenario 1, but it might not work for Scenario 2.

                    • Scenario 1: If dmaBuffer.size >= dmaBuffer.count + sizeof (fake_line)
                    • Scenario 2: If dmaBuffer.size < dmaBuffer.count + sizeof (fake_line)

                     

                    I tried that with CX3 RDK and got the following results.

                    • Resolution 640x481: Streaming OK, dmaBuffer.size = 24560, dmaBuffer.count = 400, sizeof(fake_line) = 640*2
                    • Resolution 1920x1081: Streaming NG, dmaBuffer.size = 24560, dmaBuffer.count = 21120, sizeof(fake_line) = 1920*2

                     

                    So it looks this solution depends on the resolution. May I have your suggestions for Scenario 2?

                    Perhaps one possible solution for Scenario 2 is to adjust DMA buffer size?

                     

                    /* DMA callback function to handle the produce and consume events. */

                    void

                    CyCx3AppDmaCallback (

                            CyU3PDmaMultiChannel *chHandle,

                            CyU3PDmaCbType_t      type,

                            CyU3PDmaCBInput_t    *input)

                    {

                        CyU3PDmaBuffer_t dmaBuffer;

                        CyU3PReturnStatus_t status = CY_U3P_SUCCESS;

                     

                     

                        if (type == CY_U3P_DMA_CB_PROD_EVENT)

                        {

                            /* This is a produce event notification to the CPU. This notification is

                            * received upon reception of every buffer. The buffer will not be sent

                            * out unless it is explicitly committed. The call shall fail if there

                            * is a bus reset / usb disconnect or if there is any application error. */

                     

                     

                            status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);

                            while (status == CY_U3P_SUCCESS)

                            {

                                /* Add Headers*/

                                if (dmaBuffer.count < CX3_APP_DATA_BUF_SIZE)

                                {

                                    CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_EOF);              

                                    CyU3PMemSet(dmaBuffer.buffer, 0xff, FAKE_LINE_LENGTH);

                                    dmaBuffer.count += FAKE_LINE_LENGTH;

                               }

                                else

                                {

                                    CyCx3AppAddHeader ((dmaBuffer.buffer - CX3_APP_PROD_HEADER), CX3_APP_HEADER_FRAME);

                                }

                     

                                /* Commit Buffer to USB */          

                                status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count + 12), 0);

                                if (status != CY_U3P_SUCCESS)

                                {

                                    CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT, CYU3P_EVENT_OR);

                                    break;

                                }

                                else

                                {

                                    glDmaDone++;

                                }

                                status = CyU3PDmaMultiChannelGetBuffer(chHandle, &dmaBuffer, CYU3P_NO_WAIT);

                            }

                        }

                                         :

                    }

                    • 7. Re: How to create a faked line in start of every video frame in CX3 firmware
                      srdr

                      Gary,

                       

                      Yes, you have to adjust the DMA Buffer size that meets your requirement - dmaBuffer.size > dmaBuffer.count + sizeof (fake_line). This is the only working case for you.

                       

                      In other case ( dmaBuffer.size < dmaBuffer.count + sizeof (fake_line)), buffer over flow occurs. Therefore, avoid this case.

                       

                      And in dmaBuffer.size = dmaBuffer.count + sizeof (fake_line) case, the host does not know when the frame ends for particular resolutions.

                      Becasue host expects a partial packet (less than 1024 bytes) to know that the transfer has ended. If the total frame including the fake line is exactly multiple of 1024, the host does not know the end of transfer. Hence, avoid this case too.

                      • 8. Re: How to create a faked line in start of every video frame in CX3 firmware
                        gary.lee_3399181

                        Thanks for all the comments. It's quite helpful!