CX3 Mipi minimum Horizontal blanking (RAW12 format)

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
lock attach
Attachments are accessible only for community members.
Anonymous
Not applicable

Hi,

My issue:

I have a sensor streaming CSI2 at 1.2 Gbps, 120 fps, resolution 240x180, 2 mipi lanes, RAW12 format --> No problem in this case

I have the possibility to double the data of each pixel --> (480x180), keeping the line readout length fixed --> I cannot stream in this case

while I can correctly stream in the same situation with a mipi2usb done in FPGA instead of  CX3.

By artificially extending the readout time on the sensor of each line I can stream correctly from CX3.

My goal:

I want to stream through CX3 without slowing down the readout of the sensor (or at least understand the CX3 limitation)

My understanding of the issue:

I need 6.4 usec to readout 2 lines (in parallel) on the sensor and I need to committ all the data on the mipi bus during this time (before next 2 lines become available)

The time needed for the mipi to committ data is:

480* 2 *12 bits / 1.2 Gbps / 2 lanes = 4.8 usec + (THS + LineBlanking)

[By measurements I have: THS_Prepare = 220ns, THS_Trail=80ns, LineBlanking=460 ns] --> Total mipi time = 6.4 usec

This leaves 460 ns usec of line/horizontal blanking to the CX3 --> Is this enough (see attached image tek00024.png)?

Attempt#1

By relaxing the readout time we are virtually increasing horizontal blanking, and everything seem to work again (we need at least 2.6 usec of line blanking to stream)

Attempt#2:

What we have tried is to use RGB888 format instead of RAW12. In this case it seems we don't have the limitation, meaning we can stream

480 lines without reducing readout.

This is not acceptable since we have to manage externally the RAW12 data unpacking, and does not explain why the native RAW12 is not working.

This also leads me to think that CSI2 PHY layer may not be the problem but maybe how data is managed in the fixed GPIF.

My guesses:

- RAW12 "demosaicing" is introducing overhead

- Some insufficient buffering during data processing in (or before) the GPIF

- More bandwitdth (2.4 Gbps in case of RGB888 instead of 1.2 Gbps in case of RAW12 are "helping" to consume data faster and suffer less by previous point)

What do you think?

Is there a lower hard limit for line blanking in the Mipi receiver?

Here below the mipi configuration:

CyU3PMipicsiCfg_t RAW12_HQvga =

{

     CY_U3P_CSI_DF_RAW12,

     MIPI_LANES,                 /* uint8_t numDataLanes */

     0, /* uint8_t pllPrd */

     39, /* uint16_t pllFbd */

    CY_U3P_CSI_PLL_FRS_500_1000M, /* CyU3PMipicsiPllClkFrs_t pllFrs */

    CY_U3P_CSI_PLL_CLK_DIV_8, /* CyU3PMipicsiPllClkDiv_t csiRxClkDiv */

    CY_U3P_CSI_PLL_CLK_DIV_8, /* CyU3PMipicsiPllClkDiv_t parClkDiv */

     0x0707,

     CY_U3P_CSI_PLL_CLK_DIV_2, /* CyU3PMipicsiPllClkDiv_t mClkRefDiv */

     240,

     20,

};

Thank you,

Nico

0 Likes
1 Solution
KandlaguntaR_36
Moderator
Moderator
Moderator
25 solutions authored 10 solutions authored 5 solutions authored

Nico,

It looks like the buffer space in the CX3 MIPI Bridge is not enough when you set RAW 12 output format.

Here, you are using the RAW 12 input and RAW12 output format in the MIPI and you have set the GPIF Bus to 16 bit data bus using MipicsiGpifLoad API.

In this case, the GPIF will sample the 16 lines and store in the buffer as 16 bits per clock, but the MIPI block puts only 10 bits per clock, the rest 6 bit are sampled as zero or floating values. Hence, out 16 bits, only 10 bits belong to a single pixel, rest are in-valid data w.r.t image.

This takes much time to stream the data.

But when you set output format as RGB888, and GPIF is set to 24-bit format. Hence, 24 bits ( 2 x RAW 12 data) will be sent per clock. This takes less time to stream the same data compared to 16-bit GPIF case.

Regarding the unpacking of RGB888:

The MIPI bridge receives RAW 12 in the following format as per the MIPI CSI-2 Spec.

pastedImage_0.png

When you set RGB888 as output fornat, MIPI Bridge does not change the format of the received input data. It just converts the input 24-bit serial data into parallel 24-bit.

In other words, the CX3 MIPI will convert the FIRST 24 serial bits into parallel 16 bits. It does not bother about the data sequence.

So, the GPIF will receive these first 24-bit in the RAW 12 format converts into 24-bit parallel data.

i.e First clock cycle - P1[11:4] P2[11:4] P1[3:0] P2[3:0]

Second clock cycle - P3[11:4] P4[11:4] P3[3:0] P4[3:0]

Third clock cycle - P5[11:4] P6[11:4] P5[3:0] P6[3:0]

You can go with RGB888 format, if it solves your issue.

Please refer this thread. This may help you.

Streaming RAW8 or RAW10 using CX3

View solution in original post

9 Replies
KandlaguntaR_36
Moderator
Moderator
Moderator
25 solutions authored 10 solutions authored 5 solutions authored

Nico,

It looks like the buffer space in the CX3 MIPI Bridge is not enough when you set RAW 12 output format.

Here, you are using the RAW 12 input and RAW12 output format in the MIPI and you have set the GPIF Bus to 16 bit data bus using MipicsiGpifLoad API.

In this case, the GPIF will sample the 16 lines and store in the buffer as 16 bits per clock, but the MIPI block puts only 10 bits per clock, the rest 6 bit are sampled as zero or floating values. Hence, out 16 bits, only 10 bits belong to a single pixel, rest are in-valid data w.r.t image.

This takes much time to stream the data.

But when you set output format as RGB888, and GPIF is set to 24-bit format. Hence, 24 bits ( 2 x RAW 12 data) will be sent per clock. This takes less time to stream the same data compared to 16-bit GPIF case.

Regarding the unpacking of RGB888:

The MIPI bridge receives RAW 12 in the following format as per the MIPI CSI-2 Spec.

pastedImage_0.png

When you set RGB888 as output fornat, MIPI Bridge does not change the format of the received input data. It just converts the input 24-bit serial data into parallel 24-bit.

In other words, the CX3 MIPI will convert the FIRST 24 serial bits into parallel 16 bits. It does not bother about the data sequence.

So, the GPIF will receive these first 24-bit in the RAW 12 format converts into 24-bit parallel data.

i.e First clock cycle - P1[11:4] P2[11:4] P1[3:0] P2[3:0]

Second clock cycle - P3[11:4] P4[11:4] P3[3:0] P4[3:0]

Third clock cycle - P5[11:4] P6[11:4] P5[3:0] P6[3:0]

You can go with RGB888 format, if it solves your issue.

Please refer this thread. This may help you.

Streaming RAW8 or RAW10 using CX3

Anonymous
Not applicable

Dear Srdr,

Thank you very much for the answer, it is clear now.

I have an additional question on this topic:

I was quite surprised to be able to receive a csi2 @ 1.2Gbps/lane (600 MHz csi clock with ddr mode)

Even though cypress officially supports 1 Gbps/lane, actually 1.2Gbps/lane is ok for us.

Is the 1 Gbps (500MHz ckp/ckn in ddr mode) limit coming from analog design of the D-PHY?

Or is it coming from logic throuput limitation?

If it is analog limitation I can understand why with a good pcb design the CX3 can still work @ 600 MHz clock.

Can you please comment on this?

Thank you for your support.

Nico

0 Likes

Nico,

The CX3 MIPI may work beyond 500 MHz. But the guaranteed operation is upto 500 MHz.

However, the maximum bandwidth supported by CX3 is 2.4 Gbps.

Regards,

Sridhar

Anonymous
Not applicable

Hi,

Additionally:

Is it possible to decouple input output format and GPIF data bus width?

In practice can I:

Set GPIF Bus to 24 bit data bus using MipicsiGpifLoad, to exploit 2.4 Gbps

AND

Output RAW12 data (and not RGB888), so that I don't have to manage myself the pixel data reordering?

Thank you

Nico

0 Likes

NO,

If you set GPIF II to 24-bit and set MIPI output format to RAW12, you will receive RAW 12 bits (actual pixel data) + other 12 bits (0 or 1 - these are extra bits per clock) per clock.

Hence, this is not suggestible.

When you set RGB888, you do not need to do pixel re-ordering. right?

0 Likes
Anonymous
Not applicable

When I set RGB888 output I still have RAW12 input so I do need to reorder bits which are actually in RAW12 (de-interlacing of pixels following the format specified in Mipi standard)

0 Likes

As said, in case of RAW12 output and RAW12 input, the GPIF receive 16 bits (12 bits actual data + 4 additional bits per clock).

In case of RGB888 output and RAW12 input, your receive two pixel data (12 + 12) per clock.

Note that the RAW12 to RGB888 conversion does not change the pixel order. The MIPI just convers the serial data into parallel - whether it is RAW 10 or 12 or RAW8.

In case of RGB888 output format, you need to change the UVC descriptors, Probe Control structures.Becasue in earlier case, you are receiving 16 bits per pixel (4 bits are additional due to fixed GPIF II functionality). But in this case, it is only 12 bits per pixel.

0 Likes
Anonymous
Not applicable

Ok, this I understand.

I am a bit confused on when and where the RAW12 is "upacked" and pixels are put in the correct order:

My belief till now was:

Case#1

1- If we set input RAW12

2- If we set output RAW12

3- GPIF receives 16 bits (4'b0,PIX1[11:0])   --> reordered pixel data

Case#2

1- we set RAW12 input

2- We set RGB888 output

3- GPIF receives 24 bits: PIX1[11:4], PIX2[11:4],PIX1[3:0],PIX2[3:0]   --> NOT ordered pixel data

My question is:

Can we do it:

1- we set RAW12 input

2- We set CyU3PMipicsiGpifLoad(CY_U3P_MIPICSI_BUS_24, CX3_APP_DATA_BUF_SIZE);

3- GPIF receives 24 bits: PIX1[11:0], PIX2[11:0]  --> reordered pixel data

0 Likes

In case#1, GPIF receives 16 bits (4`b0, PIX1[11:4] PIX2[11:8] )

pastedImage_0.png

The above figure shows the RAW12 transmission.

The CX3 MIPI converts the serial input data  to parallel 12-bits without changing the order.

i.e First 12 bits will be transferred to GPIF II (PIX1[11:4] PIX2[11:8] ) + 4`b0

Next 12 bits in the order will be converted to parallel format. i.e PIX2[7:4] PIX1[3:0] PIX2[3:0] + 4`b0

Case#2, your understanding is correct

Answering your question:

>> Can we do it:

>> 1- we set RAW12 input

>> 2- We set CyU3PMipicsiGpifLoad(CY_U3P_MIPICSI_BUS_24, CX3_APP_DATA_BUF_SIZE);

In this case, GPIF receives 24 bits: P1[11:4] P2[11:4] P1[3:0] P2[3:0]

The pixel re-ordering usually to be done on the host application or some users use ISP which coverts RAW data into standard format (say, YUY2). Hence, YUY2 format images can be streamed by any generally available UVC host applications (say, E-cam). But RAW data can not be streamed by generally available host apps. If you use E-Cam for RAW data streaming, you may see some colors. This helps you to know that there is some data reaching to host end.