1 2 Previous Next 27 Replies Latest reply on Jul 21, 2020 11:47 PM by JayakrishnaT_76

    Stream RAW10 with 24bit output format

    JaJo_4738236

      Hi,

       

      I developing firmware that can stream data at 720p@30 fps. Input format is RAW10. As RAW10 is not supported streaming for windows, I have experimented with two configurations using reference from the below link. As we want higher resolution as our next goal we want higher throughput. We want to go ahead with the second option. But we are unable to stream 720p@30fps.

      Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387 .

       

      1) Experiment 2:  16-bit output data format (CY_U3P_CSI_DF_YUV422_8_2)  in which Streaming is Working

      Here is MIPI receiver configuration.

       

      image.png

      Here is the descriptor and MIPI configuration.

      image.png

      image.png

      2) Example 3: 24-bit output data format (CY_U3P_CSI_DF_RGB888) in which Streaming is not Working

        Here is MIPI receiver configuration.

      image.png

      Here is the descriptor and MIPI configuration. 

      image.png

      image.png

        • 1. Re: Stream RAW10 with 24bit output format
          JayakrishnaT_76

          Hello,

           

          As per the KBA, please confirm that when you are packing the RAW 10 data into 24 bits format, the line size is a multiple of 24 bits.

           

          Also, please share the snapshots again as Iam not able to see them properly. You can share them as attachments also.

           

          Best Regards,

          Jayakrishna

          • 2. Re: Stream RAW10 with 24bit output format
            JaJo_4738236

            JayakrishnaT_76 Thanks for reply.

             

             

            As per my understaing line size means H-Active(Please correct my understating if  it is wrong) from configuration which is 1280 which is not multiple of 24bit. In that case what are changes required.

             

            Let me share you full configuration.
            H-Active : 1280.

            V-Active : 720.

            H-Blanking : 2340.

            V-Blanking  :2226

             

            Also I am attaching images. Please check names I have added 16bit and 24bit last name for different confgiuration.

            • 3. Re: Stream RAW10 with 24bit output format
              JayakrishnaT_76

              Hello,

               

              In your case, the number of pixels in a line is 1280.

              Bits per pixel is 10.

               

              So the Line size will be 1280*10 = 12800.

               

              This is divisible by 16. So you are able to stream the video data properly when you pack the incoming data to 16 bits.

              This is not divisible by 24. So it is not possible to stream the video data when you pack the incoming data to 24 bits.

               

              You should configure the sensor to output more pixels in a line so that the line size becomes divisible by 24. Only then it is possible to stream the data without any issues.

               

              Best Regards,

              Jayakrishna

              • 4. Re: Stream RAW10 with 24bit output format
                JaJo_4738236

                Hi JayaKrishna,

                 

                As number of pixel in line is not multiple of 24bit I have changes my resolution to 4908x3684.
                Now 4908x10bit = 49080 which is in multiple of 24bit.

                 

                And I have changed GUID in "cycx3_uvcdscr.c" as per below. I have not made any other chnages in MIPI configuration file or descriptor. Below changes are done in "CyCx3USBSSConfigDscr". I am using e-camview and amCap for testing I know that I am not going to clearer picture But I think should be able to see green color or some random picture if streaming is started from CX3 as I am seeing with 16bit YUV format. Do i need to make any other changes for streaming RAW10 data to RGB24 or streaming format which is supported by UVC.  

                 

                /* GUID, globally unique identifier used to identify streaming-encoding format: RGB888  */

                 

                  0x7D, 0xEB, 0x36, 0xE4,            /* MEDIASUBTYPE_RGB888 GUID: E436EB7D-524F-11CE-9F53-0020AF0BA770 */

                  0x4F, 0x52, 0xCE, 0x11,

                  0x9F, 0x53, 0x00, 0x20,

                  0xAF, 0x0B, 0xA7, 0x70,

                 

                  0x18,                              /* Number of bits per pixel: 24 */

                 

                I have also attached MIPI configuration snapshot.
                Thanks in Advance.

                 

                Regards,
                Jayesh.

                • 5. Re: Stream RAW10 with 24bit output format
                  JayakrishnaT_76

                  Hello,

                   

                  The MIPI configurations seems to be proper.

                   

                  UVC host applications do not support RGB888. Please do not change the GUID to RGB888. You can keep the GUID of YUY2 itself. The bits per pixel should be 16 itself. Both these are needed to use the UVC host applications like e-CamView or AMCap.

                   

                  Now, the frame descriptors need to be changed for supporting the bandwidth of the frame transferred.

                   

                  The input data rate to CX3 is 4608*3684*10. CX3 will pack the RAW10 data to 24 bits. However, UVC will consider each pixel to be of 16 bits each. So, the output data rate will be 4608*3684*16. Therefore, to match the input and output data rate, either h-active or v-active or both are to be changed in the UVC descriptors.

                   

                  4608*3684*10 = h_active * v_active * 16

                  So, an option can be:

                  h_active = 3456

                  v_active = 3070

                   

                  You need to change the UVC frame descriptor with these new values.

                   

                  Similarly, the Max video frame size field of probe control structure need to be changed to the hex equivalent of 3456*3070*2.

                   

                  Once this is done, please make sure that the first parameter of the API CyU3PMipicsiGpifLoad() is CY_U3P_MIPICSI_BUS_24. Also make sure that the DMA buffer size excluding size of header and footer is completely divisible by 3.

                   

                  Please let me know if you have any queries on this.

                   

                  Best Regards,

                  Jayakrishna

                  • 6. Re: Stream RAW10 with 24bit output format
                    JaJo_4738236

                    JayakrishnaT_76 Thanks for reply

                     

                    In above calculation H-Active is 4908 instead of 4608.
                    I have made calculation accordingly. Now H-Active x V-Active is 3681x3070. And made changes in descriptor for same As attached in file. GUID changed for YUV2 and bit rate set to 16.

                     

                    First parameter of the API CyU3PMipicsiGpifLoad() is CY_U3P_MIPICSI_BUS_24. I have confirmed that.
                    Also make sure that the DMA buffer size excluding size of header and footer is completely divisible by 3. :  It is OK as generated by tool.

                    I have one question in MIPI receiver configurator Pre Divder Value is 3 and Multiplier if Unit Clock is 90. But As I save "cyu3mipicsi.c" file in file it is changed to 2 and 89 as attached in images is it expected or is it bug in IDE.  I have tried with 3 and 90 also but still streaming is not started. Am I still missing something? Let me if you need any additional information.

                     

                    Regards Jayesh.

                    • 7. Re: Stream RAW10 with 24bit output format
                      JaJo_4738236

                      Max video frame size field of probe control structure need to be changed to the hex equivalent of 3681*3070*2.

                      I have attached probe control setting image

                       

                      • 8. Re: Stream RAW10 with 24bit output format
                        JayakrishnaT_76

                        Hello,

                         

                        Can you please share your firmware by removing the image sensor configuration settings so that I can have a look at it and understand better.

                        Also, 0x8FD0 is completely divisible by 3. So, please do not change it to 0x8FD1.

                         

                        Best Regards,

                        Jayakrishna

                        • 9. Re: Stream RAW10 with 24bit output format
                          JaJo_4738236

                          Also, 0x8FD0 is completely divisible by 3. So, please do not change it to 0x8FD1 : Changed to 0x8FD0.

                           

                          Also find attached firmware I have removed cyu3imagesensor.c and .h files As it has register information of camera  sensor so you might get after building firmware.

                           

                          CyCx3_ImageSensor_Set_4908x3684() is removed as Sensor resolution set during init sequence.

                          • 10. Re: Stream RAW10 with 24bit output format
                            JayakrishnaT_76

                            Hello,

                             

                            I found a small mistake in the min and max bit rates. Please find the snapshot below:

                             

                            The bit rate should be 3681*3070*16*5 which will be equal to 4908*3684*10*5. The result of this would be 904,053,600. Please change both the entries marked in the box to 0x60,0xc3,0xe2,0x35. Everything else looks fine in the project.

                             

                            Please try streaming the video after this modification is made. If you still dont get the video, please share the debug prints so that I can understand if any of the APIs are failing or not. While doing this, please uncomment the section of code that prints mipi errors also. This section of code can be found inside the infinite for loop. The snapshot of the same is given below:

                             

                            Best Regards,

                            Jayakrishna

                            • 11. Re: Stream RAW10 with 24bit output format
                              JaJo_4738236

                              JayakrishnaT_76 I have changed Min Rate Max Rate but still unable to start stream.

                               

                              As our board costume designed. We have not used UART pins(this are floating) Now I have developed one mechanism in which I can read data using UVC extension unit using another app (not e-camview) It send start stream command using UVC and it works as I have tested using 720p16bit firmware I have talked earlier. Now when I read this error this all error are zero. Please refer first FVtriggerCount and GRR Trigger Count. After two values are garbage.

                               

                              Is there anyway other than UART we can see realtime log? While using e-camview.
                              I have also tried EzUsbSuite debugging using Jlink but can not see debug print in Console. is there something missing while using debugger?

                              • 12. Re: Stream RAW10 with 24bit output format
                                JayakrishnaT_76

                                Hello,

                                 

                                Please let me know what does the prints for FVtriggerCount and GRR Trigger Count indicate. Do they indicate the values for MIPI errors?

                                JTAG cannot be used for realtime debugging. UART debugging is the best way for realtime logs.

                                 

                                However if you do not have a UART port on board, you can refer to the following blog post and try to implement a UVC+CDC interface for obtaining logs.

                                SlaveFiFoSync example with a CDC debug console

                                 

                                OR

                                 

                                You can refer to the following thread which does the same function:

                                FX3 - Send debug log/prints to USB-UART Bridge (using example)

                                 

                                You can refer to the following KBA to understand how to add CDC interface to UVC application:

                                Adding Communication Device Class Interface to FX3 Firmware - KBA229099

                                 

                                Please share the following so that I can debug this issue further:

                                1. Probe FV,LV and PCLK testpoints and share the traces. This can be used to understand if the data is coming from the sensor properly and is sampled by CX3 without any issues.

                                2. Using debug prints, check the size of each frame. For this, use a global variable and initialize it to 0. When a full buffer or partial buffer is received, add the size (dmaBuffer.count) to the global variable. At the end of the frame, a partial buffer will be received. At this time, the value of the global variable will indicate the total size of the frame (in bytes). Please note that the debug prints should not be used inside the DMA callback function as it can affect the streaming. You can use it in the infinite for loop. While using it in infinite for loop, make sure that the value of global variable is not changed upon reception of successive full buffers or partial buffers. Please share this size of frame with me.

                                3. Also, please confirm that in the cyu3mipicsi.c, the pre-divider setting is 2 and multiplier of unit clock setting is 89. These settings are proper and should not be changed.

                                 

                                Best Regards,

                                Jayakrishna

                                • 13. Re: Stream RAW10 with 24bit output format
                                  JaJo_4738236

                                  JayakrishnaT_76 Thanks for continous support.

                                   

                                  1) FVtriggerCount and GRRTriggerCount refers to all MIPI error.

                                   

                                  2)SlaveFiFoSync example with a CDC debug console example gives incriminating second value in teraterm(attached) so I am looking to integrate this project with my project. After adding the CDC_bulkloop descriptor in my project do I need to bind cdc_cypress driver or It will directly work same as slavefifo example?

                                   

                                  3)Adding Communication Device Class Interface to FX3 Firmware - KBA229099 Does for CX3 I need to follow same procedure and add descirptor information  of CDC from SlaveFiFoSync example with a CDC debug console with UVC.

                                   

                                  4)As I don't have oscilloscope nearby right now I am going ahead with UVC+CDC approach.

                                   

                                  5) I can confirmed that in cyu3mipicsi.c, the pre-divider setting is 2 and multiplier of unit clock setting is 89.

                                   

                                   

                                  Regards Jayesh.

                                  • 14. Re: Stream RAW10 with 24bit output format
                                    JayakrishnaT_76

                                    Hello,

                                     

                                    You can refer to the KBA Adding Communication Device Class Interface to FX3 Firmware - KBA229099  for adding CDC to UVC application. This already has an example project for CX3 which you can refer to add CDC to the CX3 firmware that you are using. You may need to bind the device manually to CDC driver.

                                    After the device is enumerated you can refer to the blog post SlaveFiFoSync example with a CDC debug console  OR the thread FX3 - Send debug log/prints to USB-UART Bridge (using example)  to implement the code to get debug logs.

                                     

                                    Best Regards,

                                    Jayakrishna

                                    1 2 Previous Next