Can the Denebola RDK handle 1920x1080@60fps in RAW10 ?

[LaCh_4386416]

Hello,

Assuming we are using another camera than the OV5640.

Assuming we are using a camera that can do RAW10 spread on 4 MIPI lanes.

The short spec of the RDK says it can handle 1080p @ 30fps but does not precise the pixel format. 

Parsing docs, i have found, it might have been for YUY2 (16bits/pix)

According to the spec of the CX3, it looks like it could handle up to 2.4Gbits/s

1920x1080@60fps in RAW10 is 1.24Gbits/s, well below 2.4Gbits/s

even if it has to go by 16bits on the GPIFII bus, it would take only 1.99Gbits/s.

RAW10 could even be packed in 24 bits output format if i understand well, which would make it into 1.49Gbits/s

The highest bitrate that i found in e-con_CX3RDK_OV5640_Firmware_Package.zip/CX3RDKOV5640 Release Notes.txt

is 2592x1944@15fps in YUY2 : 1.209Gbits/s. A little bit lower, but now very far.

Then, am I right to assume that the Denebola RDK can handle the bitrate of a 1920x1080@60fps in RAW10 video stream ?

Thanks for your help.

[Rashi_Vatsa · Accepted solution]

Hi,

Yes, CX3 can handle 1920x1080@60fps in RAW10.

For streaming 1920x1080@60fps in RAW10, you need to change the Output Video Format in Image sensor Configuration in the CX3 configuration utility to 24 bit. And also change the GPIF bus width to 24 bit.

It is not recommended to use 16 bit bus width for streaming 1920x1080@60fps in Raw10 data.

Can you restate the sentence "RAW10 could even be packed in 24 bits output format if i understand well, which would make it into 1.49Gbits/s"

Note: Before changing the Output Video Format in Image sensor Configuration , make sure that line size is divisible by Output Video Format.  In this case the line size is divisible by 24. So changes can be done.

For more information you can refer section B of Q1 Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387

Regards,

Rashi

[LaCh_4386416]

Thanks.

>>Can you restate the sentence "RAW10 could even be packed in 24 bits output format if i understand well, which would make it into 1.49Gbits/s"

I just meant that I expected the GPIF bus contention to be reduced by using 24 bits packing 2 RAW10 pixels on the GPIF bus. (overhead 24/20 < 16/10)

Which matches your recommendation.

laurent

[LaCh_4386416]

How are the 10bitsx2 pixels packed into the 3 bytes.

This raises another question : what are the consequences of the bit packing on the PC side ?

I expect a performance penalty for de-shuffling the bits by CPU...

[Rashi_Vatsa]

Hi Laurent,

The 24 bits are packed as 10-10-4, without any junk.

Performance penalty will depend on the host application which will unpack the data. It may/may not affect the performance.

Regards,

Rashi

[LaCh_4386416]

When you say 10-10-4 without any junk, i don't understand. 

How are the 20 useful bits bits packed in 3 bytes in the memory of the CPU ?

For example, it could be 8-8-2+2 with 4 junk. Or 8-8-8-8-2+2+2+2 that's 0 junk.

regards,

laurent

[Rashi_Vatsa]

Hello Laurent,

The CX3 packs the input RAW10 format into the 24-bit output format

It will take data of three pixels to pack to 24 bit

------------------------------

1st pixel - 10 bits

2nd pixel - 10 bits

3rd pixel - 4 bits

-------------------------------

3rd pixel - 6 bits

4th pixel - 10 bits

5th pixel - 8 bits

and so on..

Regards,
Rashi

[LaCh_4386416]

Hello again,

You are still not telling how they get packed in bytes... you are telling me how the pixels get cut.

Since you say that there is no overhead, I assume these are packed as bytes and not 32 bits words.

So, I assume from your last answer that it is

[

first byte :

8 of first pixel

second byte

2 of first pixel + 6 of second pixel

third byte

4 of second pixel + 4 of third pixel

fourth byte

6 of third pixel + 2 of fourth pixel

fifth byte

8 of fourth pixel

]*

and so on...??? the cycle would be over five bytes every 4 pixels ?  can you confirm ?

laurent

[Rashi_Vatsa]

Hello Laurent,

I don't understand your statement " I assume these are packed as bytes and not 32 bits words" Can you restate it?

The overhead that you are referring to is for unpacking the data or for packing the data?

Setting 24-bit as the output data format in Image Sensor Configuration Tab enables the packing of incoming RAW10 data into 24-bit parallel data and sending it over the MIPI CSI2 parallel interface.

The input serial RAW 10 data will be converted to parallel 24-bit data as follows and this will be sampled by GPIF II.

i.e. For every clock cycle, GPIF II receives 24-bit data

• First clock cycle - P1[9:2], P2[9:2], P3[9:2],
• Second clock cycle - P4[9:2], P1[1:0], P2[1:0], P3[1:0], P4[1:0], P5[9:2]
• Third clock cycle –  and so on.

Regards,

Rashi

[LaCh_4386416]

Hello Again,

So that you understand,

Now that you have validated that the CX3 can indeed follow that speed, I am evaluating the CPU load at the end, in relation to the way pixels are put in memory.

This is for a camera.

How is data shuffled in the end in the memory of the CPU that will have to process it ?

This will also in the end incur more or less processing if it is organized in a bad way.

The CPU in the CX3 is an ARM9. That is a 32 bits processor.

The RAM is 512KB on a 32 bits bus.

001-91492_EZ-USB_CX3_Technical_Reference_Manual.pdf page 3.

How do these pixels end up in this memory, byte by byte.

If these do not end up on this memory but directly go into the USB stream (skipping the 512KB ram), then, the question is the same. How do these pixels get organized in the end into the memory of the PC (or in the USB frame, made of bytes I guess)?

Regards,

Laurent

[Rashi_Vatsa]

Hello Laurent,

As mentioned in my previous post, the data will be packed in 24 bit format. The GPIF II will transfer the data through the DMA channel to USB.

The data will be in little endian  format.

For First clock cycle - P1[9:2], P2[9:2], P3[9:2]

Pixel   GPIFII pins

P1[2]   - DQ0      LSB

P1[3]   - DQ1

....

....

P3[9]   - DQ23    MSB

Then this data will be put in little endian format.

Regards,

Rashi