CX3 MIPI configuration

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
AiBe_4441001
Level 1
Level 1
First like given

Hello,

I am interfacing an Omnivision image sensor (OV12895) with the CX3 bridge (superspeed isochronous endpoint). The main problem I am facing is that frames are, on occasion, lost  (0x47 returned by CyU3PDmaMultiChannelCommitBuffer). After some researches and tests it looks like this kind of errors are likely to be due to host not retreiving data fast enough from the endpoint. Still I am wondering whether there could be a problem with the various throughputs in the chain (from image sensor to endpoint) not matching each other.

Alongside this issue some questions pops in my mind :

  • If the MIPI receives more data than GFIP can cope with, would it be possible to have this 0x47 (on CyU3PDmaMultiChannelCommitBuffer, in the GPIF thread callback) ?

  • What would be the impact of setting CX3_FIFO_CTRL register (actually fifoDelay member of CyU3PMipicsiCfg_t) to 1 ?

Or rather, would there be an interest in not setting it to 1 (e.g. example 0x100) ?

Concerning performances, are there other values for this register that could enable to take advantage of some (intrinsics?) mechanisms regarding DMA for example (word access, burst, ...) ?

  • Regarding memory storage and padding constraints in the MIPI receiver. In the CSI-2 specification (see chapter 12)

The CSI-2 data protocol requires certain behavior from the receiver connected to the CSI transmitter. The

following sections describe how different data formats should be stored inside the receiver. While

informative, this section is provided to ease application software development by suggesting a common

data storage format among different receivers.

What about the CX3 one ? Could there be other/extra constraints that could entail some padding ?

  • Last point : how is reacting the CX3 MIPI receiver if ever the amount of data indeed received through CSI lanes is less/greater than the value in CX3_BYTE_COUNT ?

Regards

0 Likes
1 Solution
Hemanth
Moderator
Moderator
Moderator
First like given First question asked 750 replies posted

Hi,

1. Fifo delay is helpful when your HActive time on the MIPI lanes is more than the HActive time

on the GPIF Interface. Whereas if the HActive time on GPIF interface is more than the HActive

time on the MIPI lanes, then Fifo delay is not necessary.

2. The bit rate difference between the MIPI and the GPIF interface in your case would need only small

part of the FIFO as you said. But please note that the entire line data of a particular line should

be moved from the MIPI receiver to GPIF Block within the line time of the MIPI interface (HActive + HBlanking).

In your case, you may increase the FIFO delay a bit without exceeding your HBlanking time -- so that you don't keep

(n-1)th line data in the FIFO while nth line starts coming in. This fifo delay given would not help

bring down the Total Horizontal time on the GPIF interface -- which is what is necessary to decrease

the rate of data flow into the GPIF block of FX3.

So, in order to decrease the rate of data flow into the GPIF Block of FX3, you will have to increase

the Total Horizontal time -- which can be done by increasing the HBlanking time and decreasing the VBlanking time

while maintaining the frame rate. If you cannot reduce the VBlanking too, then decrease of HBlanking time would drop the

frame rate.

3. It should be taken care that the total line data in bytes is a muliple of the number of bytes the

MIPI receiver is driving in one PCLK. In your case, the line data is 5100 Bytes which is divisible by 3.

If it is not divisible by 3, let's say, the line data is 5098 Bytes. Then at the last PCLK of the line,

2 unwanted bytes would be driven and GPIF would be sampling those.

Except this, no padding delay needs to be considered.

4. The effect would be from that of the MIPI packet header. HSYNC would be asserted for that many number

of bytes.

Regards,

Hemanth

Hemanth

View solution in original post

5 Replies
Hemanth
Moderator
Moderator
Moderator
First like given First question asked 750 replies posted

Hi,

1. The data coming from the GPIF interface from the MIPI Block of CX3 goes into the DMA buffer. If the rate of emptying those buffers slowed down (due to Host being slow), then buffer overflow occurs which leads to Commit Buffer Failure.

2. The value in CX3_FIFO_CTRL register is set with the value present in the member 'fifoDelay' of CyU3PMipicsiCfg_t structure that you link to the firmware. This value can be changed provided the CX3 configuration tool does not report any error in this field. But for your question: Changing this value does not help to slow down the rate at which data is sent over the GPIF interface to CX3.

3. Referring to the point 1 above, what we need to focus on is to reduce the rate at which the data is sent by the MIPI receiver block to CX3 GPIF interface. So, what you can try is:

- Try to reduce the value: X = (No. of bytes constituting one line of image) / (H-Total time)

To achieve the above, you will have to increase the H-Blanking time from the sensor.

4. If the amount of data received by the MIPI Receiver of CX3 in a particular line is more/less, then that is translated to H-active time being more/less on the parallel output interface of the MIPI Receiver.

Regards,

Hemanth

Hemanth
0 Likes

Hello Hemanth,

Thanks for answering. Still some points are puzzling me. Here are some more details about my "configuration" :

- Image Sensor => 4080x3072, 12 FPS, RAW10

- MIPI receiver => 240MHz, DDR, 4 lanes, RGB888, CX3_FIFO_CTRL = 1

- GPIF => 74.4MHz, 24-bit bus, 2 threads, 4 buffers (30Ko) / thread, manual DMA channel

  • as the GPIF bus is configured with a 24-bit width, is 1 for fifoDelay the better value we can set ?

  • GPIF throughput (1.785 Gbit/s) is less than the MIPI one (1.92 Gbit/s). Unfortunately it seems that (see https://community.cypress.com/thread/47390) we can't set this clock higher than this, othewise we get problem with HSYNC and we aren't able to retreive the frame). However even with these frequencies, if I am not mistaken and if I am properly assuming that MIPI receiver is able to buffer 512 bytes at most (maximum value of CX3_FIFO_CTRL), this FIFO should be large enough to cope with the clock difference.

t_line_mipi = 4080*10/1 920 000 000 = 21.25 µs

t_line_gpif  = 4080*10/1 785 600 000 ~ 22.85 µs

=> delta is ~ +1.6µs

With the given MIPI and GPIF clock rate, I would say that we could start having troubles after roughly :

        8 * |FIFO|

---------------------------- ~ 30.476µs of transmissions

Fclk_mipi - Fclk_gpif

Which is (way) less than the t_line_gpif. So it seems that even if GPIF is late compared to MIPI the FIFO should enbale GPIF to cope with this delay.

As the blanking duration on image sensor side is greater than the 1.6µs difference, this drift shouldn't be forwarded to the next frame.

  • Unfortunately modifying blanking is a bit tricky as the Omnivision datasheet doesn't detailed at all how this values can be comptued (according to the line width). Anyway may indeed let some room for DMA callback (required for us to add header to the data before commiting to USB)

  • Regarding my question about padding (previous message), I was wondering about this point because, for example, according to the line width (in pixels) we may not have a whole number of 32-bit words (on MIPI receiver side).

Mere example: if line width = 12 pixels (RAW10) => 15 bytes transimitted on MIPI lanes => storage on MIPI receiver = 16 bytes (?) which left 1 extra byte for every line (could 3 in worst case). Regarding this constraint (but is really one for CX3, hence my question : What about the CX3 one ?) is/are there, at your knowledge, any other(s) padding constraint(s) on CX3 side (due to for example the way data are stored after MIPI, DMA, .... whatever).

  • Regarding my last question (previous message), by :

    [..] the amount of data indeed received through CSI lanes is less/greater than the value in CX3_BYTE_COUNT ?

    I meant : what if the 16-bit WORD COUNT from the MIPI CSI-2 packet header is less/greater than the value set in the CX3_BYTE_COUNT register ? How the CX3 MIPI receiver would react ? (obviously this case shouldn't happen and we should take care of setting the proper value in CX3_BYTE_COUNT but I'd like to know what to expect if it is the case.

Regards

0 Likes
Hemanth
Moderator
Moderator
Moderator
First like given First question asked 750 replies posted

Hi,

1. Fifo delay is helpful when your HActive time on the MIPI lanes is more than the HActive time

on the GPIF Interface. Whereas if the HActive time on GPIF interface is more than the HActive

time on the MIPI lanes, then Fifo delay is not necessary.

2. The bit rate difference between the MIPI and the GPIF interface in your case would need only small

part of the FIFO as you said. But please note that the entire line data of a particular line should

be moved from the MIPI receiver to GPIF Block within the line time of the MIPI interface (HActive + HBlanking).

In your case, you may increase the FIFO delay a bit without exceeding your HBlanking time -- so that you don't keep

(n-1)th line data in the FIFO while nth line starts coming in. This fifo delay given would not help

bring down the Total Horizontal time on the GPIF interface -- which is what is necessary to decrease

the rate of data flow into the GPIF block of FX3.

So, in order to decrease the rate of data flow into the GPIF Block of FX3, you will have to increase

the Total Horizontal time -- which can be done by increasing the HBlanking time and decreasing the VBlanking time

while maintaining the frame rate. If you cannot reduce the VBlanking too, then decrease of HBlanking time would drop the

frame rate.

3. It should be taken care that the total line data in bytes is a muliple of the number of bytes the

MIPI receiver is driving in one PCLK. In your case, the line data is 5100 Bytes which is divisible by 3.

If it is not divisible by 3, let's say, the line data is 5098 Bytes. Then at the last PCLK of the line,

2 unwanted bytes would be driven and GPIF would be sampling those.

Except this, no padding delay needs to be considered.

4. The effect would be from that of the MIPI packet header. HSYNC would be asserted for that many number

of bytes.

Regards,

Hemanth

Hemanth

Hello Hemanth,

1. Ok thanks for your answer

Indeed in the meantime I found in this document "AN90369_How_to_Interface_a_MIPI_CSI-2_Image_Sensor_With_EZ-USB_CX3" (chapter 5.5.1 CX3 MIPI Receiver Configuration) and I made other tests as well that confirmed it (I increased GPIF clock one to be as close as possible to the MIPI throughput (1.92 Gb/s) => pllFbd=34 => 81.6 MHz which correspond to 1.9584 Gb/s) which confirm what you say.

So actually it worked on the contrary of what I thought it were in my previous message (initiallly I deemed this Fifo as some kind of elastic buffer, used to cope with a greater throughput coming from MIPI stream compared to GPIF). I suppose that this delay is necessary because GPIF can't try to pick data out of the MIPI parallel interface if there isn't any data to retreive.

Then what if MIPI throughput > GPIF throughput ? Can an entire line be buffered within the MIPI receiver ?

My guess would be no but looking at what you mentioned in 2. it looks like yes actually.

If so ok I got it. But if no how much faster can the MIPI interface be compared GPIF one without any data loss ?

(by data loss I mean data lost because there isn't enough space to buffer a whole line and GPIF not able to retreive data fast enough (so assuming that GPIF is at least fast enough to retreive data before the end of HBlanking)).

3. Yes I already took care of this (that's why we use 4080 instead of 4096). For the moment we set width so that :

  • w (in pixels) % 4 == 0 (due to RAW10 format we use on sensor side)
  • w (in bytes) % 3 == 0 (due to RBG888 format we use on CX3 side)

But my point was more focused on the MIPI receiver storage. Let's take for example width:4092px => 5115 bytes which is a multiple of 3. But only 3 bytes within the last 32-bit word is filled with useful data.

Do we have to add a third constraint "w (in bytes) % 4 == 0" on the witdh ? (CSI-2 specification (see chapter 12))

From what I understand, it seems that no, only the relevant bytes are retreived by GPIF is that correct ?

4. ok clear

0 Likes
Hemanth
Moderator
Moderator
Moderator
First like given First question asked 750 replies posted

Hi,

1 & 2: Your understanding about the FIFO is correct.

In the case of MIPI throughput > GPIF Throughout: Entire line need not be buffered in the MIPI receiver because,

it is assumed that GPIF statemachine has started already by the time you start streaming the image data (i.e by the time MIPI receiver starts taking data from the transmitter).

So, unless the entire line data comes instantaneously to the MIPI receiver and the GPIF is not yet running, entire line buffer is not necessary. The buffer required would only be to maintain the throughput difference.

I understand, that, now the question would be what can the difference be so as not to make the FIFO overflow (data loss). This can be known from the Max. FIFO delay which is displayed in the CX3 Configuration tool because:

When the FIFO is able to handle the throughput difference having some space still left, you would still have an option to provide some FIFO delay. But when the FIFO is full so as to maintain the throughput difference, then you would see that the Max FIFO delay that can be given will be displayed as 0 in the tool (And of course in this scenario the Min FIFO delay required is 0 - as we discussed earlier).

Note: In the above discussion, 'throughput' is considered to be 'throughput within Line Active time.

3. You are correct. The constraint "w (in bytes) % 4 == 0" is not necessary. Only the relevant data is retrieved by GPIF.

Regards,

Hemanth.

Hemanth