9 Replies Latest reply on Sep 22, 2020 8:54 PM by RashiV_61

    CX3 vs Image Sensor PLL's - what's up?

    EtWh_2921621

      I'm trying to figure out some camera firmware that I inherited from somebody else. We're using a CX3 to control an Omnivision 4688. looking at the omnivision docs, it's got two PLL's with their own clock dividers and multipliers. The omnivision stuff has a SCLK_DEF, SCLK_OPT, a MIPI_PCLK, a MIPI_PHYS, a DAC_CLK, basically 5 clocks. When I go look at the CX3 documentation, it SEEMS like it has a couple PLLs along with associated clock multipliers and dividers as well. The CX3's clock params seem to go into cuy3mipicsi.c, and has values for the num of data lanes, the Prd, Fbd, pllFrs, csiRxClkDiv, and parClkDiv. The registers I send to the Omnivision 4688 have similar parameters passed to it for ITS PLL's!! What the heck?! How do I know which ones are the right ones to send!? Looking at the pinouts for the OV4688, there aren't many communication lines from/to the sensor chip and the CX3. It's unclear who is clocking who. Is the CX3 using the OV's pixel clock, or is the OV using the CX3's pixel clock? I'm not a firmware person, I'm a newbie. I don't understand who is generating the clock signals, and who is using the clock signals. Can somebody help explain?

        • 1. Re: CX3 vs Image Sensor PLL's - what's up?
          EtWh_2921621

          I'm reading through the CX3 docs and it appears it clocks in the data over the MIPI lanes based on the external REFCLK signal, which is between 6-40MHZ!? Okay, I know my image sensor is being driven at 24MHZ. I'm assuming it's the REFCLK that gets passed through the CX3's PLLs to get things to the right frequency, but why should the CX3 even need different PLLs? If there are PLLs on the image sensor itself, aren't those "enough", or does the CX3 need its own PLL circuits to be able to run at near or the same frequency as the sensor chip? One thing that is seriously bugging me, is if they both have PLLs, do both chips need to have exactly the same ultimate end frequencies for the MIPI and pixel clocks, or does the CX3 have to have clocks that are always FASTER than what is being output from the sensor chip? The way I figure it, the sensor chip only sends 2 clock signals to the CX3: REFCLK and possibly some kind of clock on the mipi lanes, CP/CN? (also called CLKp, CLKn)? I don't understand how the CLKp/n work: Who is the one who sets them? sensor chip or CX3? When the CLK p/n get signalled, there is a tsetup and thold time. What happens the the MIPI data is transmitted? 4 lanes of data transfer - what? How does the CX3 know what line of the sensor is being transmitted? Does the data xferred across the mipi lanes have some kind of format that indicates where it is in the capture? What is the frequency of the mipi data lane's transmission? Is it determined by the image sensor, or by the CX3? If it's determined by the image sensor, I'm assuming the CX3's "pixel clock" (does it have one of those?) has to be faster or exactly equal to, the image sensor's? My image sensor has PLLs that can be set up with a wide range of pre dividers and premultipliers. The clock can be divided by 1.5, 2, 2.5, 3, 6, etc. On the CX3, the PLLs have a very, very limited range of dividers... Do I need to set them up exactly the same or does the CX3 need to simply "run faster" than the image sensor?!

           

          I can't find a single doc on Cypress' site that adequately discusses clocks and how to set them up in great detail.

          • 2. Re: CX3 vs Image Sensor PLL's - what's up?
            RashiV_61

            Hello,

             

            You can use the CX3 Configuration tool, in the EzUSB Suite, which will generate the CX3  firmware according to the sensor settings.

            Please refer to this KBA for the steps Steps to Setup up MIPI CSI Camera Solution with CX3 – KBA225748

             

            Q Is the CX3 using the OV's pixel clock, or is the OV using the CX3's pixel clock

            >> The CSI clock (output from the sensor) is the input to CX3. The sensor part in the CX3 configuration tool needs to populated with the settings the same as the sensor settings (as shown below).

             

            cx3_image_sensor.PNG

            Please refer to this KBA for the understanding of the CX3 clocking parameter Analysis of CX3 Clocking Parameters – KBA226758

             

            Q If there are PLLs on the image sensor itself, aren't those "enough", or does the CX3 need its own PLL circuits to be able to run at near or the same frequency as the sensor chip?

            >> As mentioned in the KBA, the MIPI block generates all the clocks internally except the REFCLK which has to be provided from an external source.

             

            Q The way I figure it, the sensor chip only sends 2 clock signals to the CX3: REFCLK and possibly some kind of clock on the mipi lanes, CP/CN? (also called CLKp, CLKn)

            >> CP/CN is the MIPI CSI differential clock (CSI clock) which is fed to the CX3 from the sensor. The REFCLK need not be from the sensor. The REFCLK can be the same as the CX3 input clock (CLKIN). As shown in the below diagram the REFCLK is supplied using a clock source and not from the sensor.

            cx3_app.PNG

             

            The sensor only needs to provide the CSI clock which is a differential clock. This clock is a DDR clock.

             

            Q What is the frequency of the mipi data lane's transmission?

            >> In CX3, each MIPI lane supports up to 1 Gbps. This means the MIPI CSI clock frequency should not be more than 500 MHz (DDR clock)

             

            Q? How does the CX3 know what line of the sensor is being transmitted? Does the data xferred across the mipi lanes have some kind of format that indicates where it is in the capture?

            >> CX3 complies to MIPI CSI 2 specification which decides the format of the transfer

             

            Please let me know if any queries on this

             

            Regards,

            Rashi

             

            • 3. Re: CX3 vs Image Sensor PLL's - what's up?
              EtWh_2921621

              I understand a lot more, now. There is a MIPI preprocessor on the CX3 that turns the MIPI clock/data lanes into "parallel output" pixel data that goes across the GPIF bus. You must set the CX3's PLL to run the parallel copier as fast or faster than the incoming MIPI data, so it can store the pixel data to DMA. (this is what I think is happening).  It looks like the "output pixel clock" can only be 100Mhz max. I don't know what the PLL Out Clock affects, besides the output pixel clock. Does it affect anything else? In other words, what is the difference between (a) the PLL out clock being 800, then setting both dividers to be 8, and (b) the PLL out clock being 400, and setting both dividers to be 4? I guess it's not actually possible to do this.

               

              One thing that is making the MIPI receiver configuration tool not so useful is that we are trying to transmit 1920 x 1440 x 90FPS, RAW8. With blanking of 302 for H, and 59 for V. (don't know why, I inherited the project this way. I would have thought vblank should be higher) The data lanes are 4, and the GPIF has been set to 24 bits. We are using an OV4688 chip for input. For whatever reason, THS-prepare is set to 66.5, and THS-Zero is set to 152.75... That's the setup. Because we can't get the clocks to line up right, the output video format in the image sensor configuration has been set to 24-bit. The mipiconfig block struct has been set up to be RGB888. This means the MIPI preprocessor on the CX3 will be delivering "3 pixels at once" by lying and saying it's RGB24. I get it.

               

              The trouble is, when I set the Output Pixel Clock to 100Mhz, the CX3 is apparently happily receiving frames - I can show on the UART debug screen that the received PROD and CONS DMA events are being triggered just great. I'm receiving ~90FPS. There aren't any MIPI errors except ONE control error right at the beginning of startup (i don't know what it means or how to tell what the error is). All other mipi errors are 0. It's showing receiving 90 FPS, so I assume it's capturing... But the PC, in media foundation, we don't receive any frames! Nothing. Not bad frames, we don't receive any frames. I don't know what the deal is.

               

              But here is the mystery: If I increase the Output Pixel Clock to 113, it starts to deliver frames just great. I am also calling that function with the value of PHY Time Delay (of 14). I don't know where the values of THS-prepare and Zero came from, I don't know how close they are to the actual values...

               

              anyhow - how would I go about trying to figure out why the PC isn't consuming any UCV packets? clearly, if it's just the clock that makes it work or not, something is misconfigured, but I guess the MIPI is receiving just fine! Now what should I look at?!

               

               

              mipi config 1.JPGmipi config 2.JPG

              • 4. Re: CX3 vs Image Sensor PLL's - what's up?
                RashiV_61

                Hello,

                 

                From the CX3 configuration tool settings, I found that the CSI clock is 400 MHz when the data lanes are 4.

                The GPIF interface of CX3 supports the maximum data throughput of (24 bits * 100MHz) 2.4 Gbps. CSI clock is a DDR clock. On configuring the CSI clock as 400 MHz i.e. 800 MHz per lane (DDR), the data throughput on each CSI lane is 800 Mbps and the total data throughput becomes 800 *4 (lanes) = 3.2 Gbps which more than the maximum data throughput supported by the GPIF interface.

                 

                So, the CSI clock should be not more than 300 MHz (when using 4 lanes). Please try configuring the sensor as well as the CX3 MIPI configuration for CSI clock = 300MHz

                 

                Also, in the CX3 configuration tool settings that you have shared the CSI clock is 912 MHz and 744MHZ for other resolution. As mentioned in the CX3 datasheet, the maximum data throughput supported by each MIPI lane is 1Gbps, and the CSI clock is a DRR clock the maximum value of the clock can be 500 MHz

                Note: CSI clock can be 500MHz only when the data lanes used are 1 (1Gbps) or 2 (2 Gbps ) as the GPIF interface can support a maximum throughput of 2.4 Gbps.

                Maximum CSI Clock               Lanes used

                500 MHz                                   1/2 lanes

                400 MHz                                    3 lanes

                300 MHz                                    4 lanes

                 

                 

                I don't know where the values of THS-prepare and Zero came from, I don't know how close they are to the actual values...

                >> These values will be given by the sensor complaint to the MIPI spec. You can check the datasheet of the sensor or contact the sensor vendor for the value.

                The values provided by the sensor should be populated in the CX3 configuration tool settings.

                 

                 

                Please refer to this KBA for understanding the MIPI errors MIPI-CSI Protocol and Physical Layer Errors in CX3 (CYUSB3065 and CYUSB3064) – KBA228482

                 

                But the PC, in media foundation, we don't receive any frames! Nothing. Not bad frames, we don't receive any frames. I don't know what the deal is.

                >> Please modify the CSI clock to 300 MHZ as explained above.

                 

                 

                Because we can't get the clocks to line up right, the output video format in the image sensor configuration has been set to 24-bit.

                >> Please let me know if you want to configure the GPIF bus width as 24 bits or less?

                 

                It looks like the "output pixel clock" can only be 100Mhz max

                >> Yes, your understanding is correct.

                 

                 

                I don't know what the PLL Out Clock affects, besides the output pixel clock. Does it affect anything else? In other words, what is the difference between (a) the PLL out clock being 800, then setting both dividers to be 8, and (b) the PLL out clock being 400, and setting both dividers to be 4? I guess it's not actually possible to do this.

                >> Please refer to this KBA Analysis of CX3 Clocking Parameters – KBA226758 which mentioned that the parallel clock (output pixel clock)  is the parallel interface clock that runs the fixed GPIF state machine. With each edge of this clock, the number of bits (specified by the output data format of the config structure) will be sent by the MIPI block from the parallel bus. Other parameters /dividers need to be tuned to get the desired output pixel clock. To get the advantage of maximum throughput the dividers and the multipliers can be tuned to get the output pixel clock at 100MHz

                 

                With blanking of 302 for H, and 59 for V. (don't know why, I inherited the project this way. I would have thought vblank should be higher)

                >> I didn't understand your query. Please explain

                 

                Please refer to this KBA for a better understanding of the video timing parameters Analysis of CX3 Video Timing Parameters – KBA226779

                 

                 

                Please refer this KBA for understanding the packing and padding of RAW data Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387  The KBA is written for streaming  RAW10 format, this KBA also applies to RAW 8 format

                 

                 

                Please let me know if any queries on this

                 

                Regards,

                Rashi

                 

                • 5. Re: CX3 vs Image Sensor PLL's - what's up?
                  EtWh_2921621

                  Hi Rashi.. I understand the MIPI data is sampled twice per clock.. I'm confused, what is the calculation for "CSI" in the CX3 config? Is this the same as MIPI_PHY_CLK on the OV4688? (omnivision chip) Is CSI = Mipi Clock? When I set it to 300 instead of 600, no frames are received by the CX3! Plus I get a bunch of mipi errors. If I set it to 600, the CX3 happily receives frames. I am using 4 lanes and the incoming data is RAW8.

                   

                  Q: What is the calculation formula for CSI in the Mipi Received Config?

                  Q: What should it be, if it's wrong?

                   

                  This is the OV mipi clock diagram. I'm assuming MIPI_PHY_CLK is what the CX3 sees as the MIPI incoming data lines. I'm not quite sure what "PCLK" does. OV told me it should be set to 1/8 of MIPI_PHY_CLK.

                  ov mipi clock diagram.JPG

                  • 6. Re: CX3 vs Image Sensor PLL's - what's up?
                    RashiV_61

                    Hello,

                     

                    In the CX3 MIPI Receiver Configuration, the fields under MIPI CSI2 Input needs to be populated with the settings the same as the sensor settings

                    - CSI clock is the MIPI clock (DDR clock) which is fed to CX3 from the sensor. And this is the only clock which is coming from the sensor.

                     

                     

                    - What is the calculation formula for CSI in the Mipi Received Config?

                    >> There is no calculation formula for the CSI clock (MIPI clock) it completely depends on the MIPI clock output from the sensor. If the sensor is configured to output 300MHz MIPI clock, the same value needs to be populated in the CX3 MIPI Receiver Configuration.

                    But as explained earlier, the sensor (MIPI transmitter) should be configured such that it meets the maximum supported CSI clock (mipi clock)frequency of the MIPI receiver (CX3 in this case).

                     

                    I'm assuming MIPI_PHY_CLK is what the CX3 sees as the MIPI incoming data lines.

                    >>  Please confirm this with sensor vendor that MIPI_PHY_CLK is the MIPI clock (DDR clock)

                     

                    When I set it to 300 instead of 600, no frames are received by the CX3

                    >> Please confirm if the sensor settings were also modified to output the MIPI clock at 300 MHz

                     

                    Please let me know if further queries on this

                     

                    Regards,

                    Rashi

                    • 7. Re: CX3 vs Image Sensor PLL's - what's up?
                      EtWh_2921621

                      I was excited to wake up and see if you replied, Rashi, but no new information. :-(

                      I showed you the diagram for the OV chip. I don't understand it very well, but if PCLK's bandwidth is limited to 10-150Mhz, I can only assume that MIPI_PHY_CLK is the CSI clock.

                       

                      As I said before, if I calculate the OV's registers to output a MIPI_PHY_CLK @ 300Mhz, no frames are received by the CX3. If I calculate the registers to output @ 600Mhz, the CX3 shows them as being received just great. I have no idea what the actual output clock signal is, whether it is double what I tell it to be, or if it's exactly the frequency I told it to be. Since the OV chip came from Omnivision along with an associated "automatic frequency calculation spreadsheet", and there was no mention of frequency doubling, I'm assuming the MIPI_PHY_CLOCK that comes into the CX3 is half of the frequency I think I'm setting it to, and when I say "be 600Mhz", it comes out as 300, assuming the data is sampled on both edges of the clock.

                       

                      I asked what the calculation is for CSI. I meant the "min" value. It looks like Width * TotalHeight * Fps * BitsPerPixel / ( Mipi_Lanes * 2 )... Is this correct?

                       

                      So - yes, I am confirming - when I set it to 300 instead of 600, I get no frames from the CX3.

                       

                      I have a few different issues, I'd like to ask about. I'm afraid to ask them in a new question because you might not notice them or answer. if I write up a new question will you please look at them and help out? I will write the questions up now... they don't belong in this same thread...

                       

                      I have been working on this firmware issue for literally years. My boss is going to fire me if I spend too much more time on it.

                      • 8. Re: CX3 vs Image Sensor PLL's - what's up?
                        EtWh_2921621

                        I think this is the correct calculation for CSI Min:

                         

                        Width * TotalHeight * FPS * BitsPerPixel / ( MipiLanes * 2 )

                         

                        And Pixel Clock Min

                         

                        Width * TotalHeight * FPS * BitsPerPixel * GPIFWidth / 8

                         

                        comment: why is the CX3 MIPI Receiver Configuration tool still broken, at version 1.3.4, with respect to a proper CSI Min being displayed? Can't somebody fix it? :-)

                        • 9. Re: CX3 vs Image Sensor PLL's - what's up?
                          RashiV_61

                          Hello,

                           

                          I can only assume that MIPI_PHY_CLK is the CSI clock.

                          >> Yes, it seems that MIPI_PHY_CLK is the MIPI clock but it would be good if you get it confirmed.

                           

                           

                          I'm assuming the MIPI_PHY_CLOCK that comes into the CX3 is half of the frequency I think I'm setting it to, and when I say "be 600Mhz", it comes out as 300, assuming the data is sampled on both edges of the clock.

                          >> You can confirm this using an oscilloscope and measuring the frequency of the clock from sensor and check the changes on the clock based on the settings

                           

                           

                          I asked what the calculation is for CSI. I meant the "min" value. It looks like Width * TotalHeight * Fps * BitsPerPixel / ( Mipi_Lanes * 2 )... Is this correct?

                          >> I misunderstood your question earlier. For the minimum CSI clock value, you can refer to this Analysis of CX3 Video Timing Parameters – KBA226779

                          The minimum CSI clock required=  (H-Active (in Pixels) × Pixel Depth) ÷ H-Total (in µs) ÷2 ÷ Data Lanes

                           

                          I'm afraid to ask them in a new question because you might not notice them or answer. if I write up a new question will you please look at them and help out? I will write the questions up now...

                          >> Yes, you can create a new thread. We will be glad to help you.

                           

                          Regards,

                          Rashi