Stream RAW10 with 24bit output format

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
jayesh_joshi
Level 4
Level 4
50 replies posted 10 questions asked 25 sign-ins

Hi,

I developing firmware that can stream data at 720p@30 fps. Input format is RAW10. As RAW10 is not supported streaming for windows, I have experimented with two configurations using reference from the below link. As we want higher resolution as our next goal we want higher throughput. We want to go ahead with the second option. But we are unable to stream 720p@30fps.

Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387 .

1) Experiment 2:  16-bit output data format (CY_U3P_CSI_DF_YUV422_8_2)  in which Streaming is Working

Here is MIPI receiver configuration.

image.png

Here is the descriptor and MIPI configuration.

image.png

image.png

2) Example 3: 24-bit output data format (CY_U3P_CSI_DF_RGB888) in which Streaming is not Working

  Here is MIPI receiver configuration.

image.png

Here is the descriptor and MIPI configuration. 

image.png

image.png

0 Likes
1 Solution

Hello,

The MIPI configurations seems to be proper.

UVC host applications do not support RGB888. Please do not change the GUID to RGB888. You can keep the GUID of YUY2 itself. The bits per pixel should be 16 itself. Both these are needed to use the UVC host applications like e-CamView or AMCap.

Now, the frame descriptors need to be changed for supporting the bandwidth of the frame transferred.

The input data rate to CX3 is 4608*3684*10. CX3 will pack the RAW10 data to 24 bits. However, UVC will consider each pixel to be of 16 bits each. So, the output data rate will be 4608*3684*16. Therefore, to match the input and output data rate, either h-active or v-active or both are to be changed in the UVC descriptors.

4608*3684*10 = h_active * v_active * 16

So, an option can be:

h_active = 3456

v_active = 3070

You need to change the UVC frame descriptor with these new values.

Similarly, the Max video frame size field of probe control structure need to be changed to the hex equivalent of 3456*3070*2.

Once this is done, please make sure that the first parameter of the API CyU3PMipicsiGpifLoad() is CY_U3P_MIPICSI_BUS_24. Also make sure that the DMA buffer size excluding size of header and footer is completely divisible by 3.

Please let me know if you have any queries on this.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna

View solution in original post

0 Likes
27 Replies
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello,

As per the KBA, please confirm that when you are packing the RAW 10 data into 24 bits format, the line size is a multiple of 24 bits.

Also, please share the snapshots again as Iam not able to see them properly. You can share them as attachments also.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

JayakrishnaT_76 Thanks for reply.

As per my understaing line size means H-Active(Please correct my understating if  it is wrong) from configuration which is 1280 which is not multiple of 24bit. In that case what are changes required.

Let me share you full configuration.
H-Active : 1280.

V-Active : 720.

H-Blanking : 2340.

V-Blanking  :2226

Also I am attaching images. Please check names I have added 16bit and 24bit last name for different confgiuration.

0 Likes

Hello,

In your case, the number of pixels in a line is 1280.

Bits per pixel is 10.

So the Line size will be 1280*10 = 12800.

This is divisible by 16. So you are able to stream the video data properly when you pack the incoming data to 16 bits.

This is not divisible by 24. So it is not possible to stream the video data when you pack the incoming data to 24 bits.

You should configure the sensor to output more pixels in a line so that the line size becomes divisible by 24. Only then it is possible to stream the data without any issues.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

Hi JayaKrishna,

As number of pixel in line is not multiple of 24bit I have changes my resolution to 4908x3684.
Now 4908x10bit = 49080 which is in multiple of 24bit.

And I have changed GUID in "cycx3_uvcdscr.c" as per below. I have not made any other chnages in MIPI configuration file or descriptor. Below changes are done in "CyCx3USBSSConfigDscr". I am using e-camview and amCap for testing I know that I am not going to clearer picture But I think should be able to see green color or some random picture if streaming is started from CX3 as I am seeing with 16bit YUV format. Do i need to make any other changes for streaming RAW10 data to RGB24 or streaming format which is supported by UVC.  

/* GUID, globally unique identifier used to identify streaming-encoding format: RGB888  */

  0x7D, 0xEB, 0x36, 0xE4,            /* MEDIASUBTYPE_RGB888 GUID: E436EB7D-524F-11CE-9F53-0020AF0BA770 */

  0x4F, 0x52, 0xCE, 0x11,

  0x9F, 0x53, 0x00, 0x20,

  0xAF, 0x0B, 0xA7, 0x70,

  0x18,                              /* Number of bits per pixel: 24 */

I have also attached MIPI configuration snapshot.
Thanks in Advance.

Regards,
Jayesh.

0 Likes

Hello,

The MIPI configurations seems to be proper.

UVC host applications do not support RGB888. Please do not change the GUID to RGB888. You can keep the GUID of YUY2 itself. The bits per pixel should be 16 itself. Both these are needed to use the UVC host applications like e-CamView or AMCap.

Now, the frame descriptors need to be changed for supporting the bandwidth of the frame transferred.

The input data rate to CX3 is 4608*3684*10. CX3 will pack the RAW10 data to 24 bits. However, UVC will consider each pixel to be of 16 bits each. So, the output data rate will be 4608*3684*16. Therefore, to match the input and output data rate, either h-active or v-active or both are to be changed in the UVC descriptors.

4608*3684*10 = h_active * v_active * 16

So, an option can be:

h_active = 3456

v_active = 3070

You need to change the UVC frame descriptor with these new values.

Similarly, the Max video frame size field of probe control structure need to be changed to the hex equivalent of 3456*3070*2.

Once this is done, please make sure that the first parameter of the API CyU3PMipicsiGpifLoad() is CY_U3P_MIPICSI_BUS_24. Also make sure that the DMA buffer size excluding size of header and footer is completely divisible by 3.

Please let me know if you have any queries on this.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

JayakrishnaT_76​ Thanks for reply

In above calculation H-Active is 4908 instead of 4608.
I have made calculation accordingly. Now H-Active x V-Active is 3681x3070. And made changes in descriptor for same As attached in file. GUID changed for YUV2 and bit rate set to 16.

First parameter of the API CyU3PMipicsiGpifLoad() is CY_U3P_MIPICSI_BUS_24. I have confirmed that.
Also make sure that the DMA buffer size excluding size of header and footer is completely divisible by 3. :  It is OK as generated by tool.

I have one question in MIPI receiver configurator Pre Divder Value is 3 and Multiplier if Unit Clock is 90. But As I save "cyu3mipicsi.c" file in file it is changed to 2 and 89 as attached in images is it expected or is it bug in IDE.  I have tried with 3 and 90 also but still streaming is not started. Am I still missing something? Let me if you need any additional information.

Regards Jayesh.

0 Likes
lock attach
Attachments are accessible only for community members.

Max video frame size field of probe control structure need to be changed to the hex equivalent of 3681*3070*2.

I have attached probe control setting image

0 Likes

Hello,

Can you please share your firmware by removing the image sensor configuration settings so that I can have a look at it and understand better.

Also, 0x8FD0 is completely divisible by 3. So, please do not change it to 0x8FD1.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

Also, 0x8FD0 is completely divisible by 3. So, please do not change it to 0x8FD1 : Changed to 0x8FD0.

Also find attached firmware I have removed cyu3imagesensor.c and .h files As it has register information of camera  sensor so you might get after building firmware.

CyCx3_ImageSensor_Set_4908x3684() is removed as Sensor resolution set during init sequence.

0 Likes

Hello,

I found a small mistake in the min and max bit rates. Please find the snapshot below:

pastedImage_0.png

The bit rate should be 3681*3070*16*5 which will be equal to 4908*3684*10*5. The result of this would be 904,053,600. Please change both the entries marked in the box to 0x60,0xc3,0xe2,0x35. Everything else looks fine in the project.

Please try streaming the video after this modification is made. If you still dont get the video, please share the debug prints so that I can understand if any of the APIs are failing or not. While doing this, please uncomment the section of code that prints mipi errors also. This section of code can be found inside the infinite for loop. The snapshot of the same is given below:

pastedImage_1.png

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

JayakrishnaT_76​ I have changed Min Rate Max Rate but still unable to start stream.

As our board costume designed. We have not used UART pins(this are floating) Now I have developed one mechanism in which I can read data using UVC extension unit using another app (not e-camview) It send start stream command using UVC and it works as I have tested using 720p16bit firmware I have talked earlier. Now when I read this error this all error are zero. Please refer first FVtriggerCount and GRR Trigger Count. After two values are garbage.

Is there anyway other than UART we can see realtime log? While using e-camview.
I have also tried EzUsbSuite debugging using Jlink but can not see debug print in Console. is there something missing while using debugger?

0 Likes

Hello,

Please let me know what does the prints for FVtriggerCount and GRR Trigger Count indicate. Do they indicate the values for MIPI errors?

JTAG cannot be used for realtime debugging. UART debugging is the best way for realtime logs.

However if you do not have a UART port on board, you can refer to the following blog post and try to implement a UVC+CDC interface for obtaining logs.

SlaveFiFoSync example with a CDC debug console

OR

You can refer to the following thread which does the same function:

FX3 - Send debug log/prints to USB-UART Bridge (using example)

You can refer to the following KBA to understand how to add CDC interface to UVC application:

Adding Communication Device Class Interface to FX3 Firmware - KBA229099

Please share the following so that I can debug this issue further:

1. Probe FV,LV and PCLK testpoints and share the traces. This can be used to understand if the data is coming from the sensor properly and is sampled by CX3 without any issues.

2. Using debug prints, check the size of each frame. For this, use a global variable and initialize it to 0. When a full buffer or partial buffer is received, add the size (dmaBuffer.count) to the global variable. At the end of the frame, a partial buffer will be received. At this time, the value of the global variable will indicate the total size of the frame (in bytes). Please note that the debug prints should not be used inside the DMA callback function as it can affect the streaming. You can use it in the infinite for loop. While using it in infinite for loop, make sure that the value of global variable is not changed upon reception of successive full buffers or partial buffers. Please share this size of frame with me.

3. Also, please confirm that in the cyu3mipicsi.c, the pre-divider setting is 2 and multiplier of unit clock setting is 89. These settings are proper and should not be changed.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

JayakrishnaT_76​ Thanks for continous support.

1) FVtriggerCount and GRRTriggerCount refers to all MIPI error.

2)SlaveFiFoSync example with a CDC debug console example gives incriminating second value in teraterm(attached) so I am looking to integrate this project with my project. After adding the CDC_bulkloop descriptor in my project do I need to bind cdc_cypress driver or It will directly work same as slavefifo example?

3)Adding Communication Device Class Interface to FX3 Firmware - KBA229099 Does for CX3 I need to follow same procedure and add descirptor information  of CDC from SlaveFiFoSync example with a CDC debug console with UVC.

4)As I don't have oscilloscope nearby right now I am going ahead with UVC+CDC approach.

5) I can confirmed that in cyu3mipicsi.c, the pre-divider setting is 2 and multiplier of unit clock setting is 89.

Regards Jayesh.

0 Likes

Hello,

You can refer to the KBA Adding Communication Device Class Interface to FX3 Firmware - KBA229099  for adding CDC to UVC application. This already has an example project for CX3 which you can refer to add CDC to the CX3 firmware that you are using. You may need to bind the device manually to CDC driver.

After the device is enumerated you can refer to the blog post SlaveFiFoSync example with a CDC debug console  OR the thread FX3 - Send debug log/prints to USB-UART Bridge (using example)  to implement the code to get debug logs.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
jayesh_joshi
Level 4
Level 4
50 replies posted 10 questions asked 25 sign-ins

Hi JayakrishnaT_76

This already has an example project for CX3 which you can refer to add CDC to the CX3 firmware that you are using : Which project are you reffering to

1)Adding Communication Device Class Interface to FX3 Firmware - KBA229099

2)Add CDC device to UVC

In first project device is enumarated as serial device and I can see log in teraterm but it is not detecting simultaneously as CX3_UVC. So I think it is not UVC+CDC project.

While in second project it has example for denebola kit. Project enumarates device as CX3_UVC and COM Port but I am unable to see log in teraterm. I have tried binding " C:\Program Files (x86)\Cypress\USB-Serial SDK\Cypress USB-Serial Driver\DriverBinary\CDC_Driver\bin\win10\x64". But No success. I have tried with denebola kit and I have win10-64bit PC.

Please attach firmware if I am referring to wrong firmware.

Thanks

0 Likes
lock attach
Attachments are accessible only for community members.
jayesh_joshi
Level 4
Level 4
50 replies posted 10 questions asked 25 sign-ins

JayakrishnaT_76

I am attaching firmware for denebola kit from Add CDC device to UVC I have made some changes for continuous log of "HI" at every  1 second. Device was detecteting in other device as Virtual Serial Port and I have bind driver as per above post and After I can see that in Ports(COM&LPT) as USB serial Port (COM17). But I am unable to see log in teraterm.

I am attaching firmware for reference. Let me know what could be the issue. Streaming works fine as normal I have also attached teraterm  configuration.

0 Likes
lock attach
Attachments are accessible only for community members.

Hello,

Please refer to the attached firmware. I have tested this with denebola kit and was able to get the logs without using UART port (through CDC interface). Please test this and implement the same in your firmware for streaming 4908*3684 video.

Password for the attachment : cypress

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

Thanks JayakrishnaT_76​,

UVC+CDC is working as expected from your reference project. Now I have printed 2 things on CDC serial 1) MIPI error and DMA reset event .

Now CyCx3UvcAppProgressTimer expires every time. I have set upto 5s but it still expires. And I am getting MIPI error. Please find below attachment. TIMER_PERIOD is 500ms for this experiment. Error sequence is written in txt file.

Regards Jayesh

0 Likes

Hello,

Are these the logs obtained from your project? The timer expires if the data is not obtained from the sensor.

Please let me know if you are receiving any data from the sensor by using a global variable. The value of this global variable will be increased if there is a full or partial buffer. At the end of the partial buffer, we can say that one frame of data is completed. At this time, the value of the global variable need to be print in the infinite for loop. You can refer to the project that I shared before to implement the same. Please refer to the check if(fcomplete) in the project that I shared before to understand how this is done. If no print is obtained, then it means that a full frame is not received by CX3. This can happen if sensor is not configured properly. This can be confirmed by probing FV,LV and PCLK test points on CX3.

Please try increasing Multiplier of unit clock setting in MIPI configuration utility to 100 and check if you are seeing any improvement in the MIPI errors. Also, make sure that the API CyU3PDebugPrint() is not used in any of the callback functions.

Please try the modifications and share the UART logs with me.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

JayakrishnaT_76​,

Are these the logs obtained from your project? : Yes

Please refer to the check if(fcomplete) in the project that I shared before to understand how this is done : I have taken all the changes from your project I have commented that portion for testing. I am getting value is global variable but it is inconsistent.

No there are not any debugprint as I have removed all print which can affect throughput.​ Please find attachment.

0 Likes

Hello,

Please let me know if there was any effect by changing Multiplier of Unit clock setting to 100 in MIPI configuration utility.

I find that the frame size is varying and less than the expected frame size in bytes.

The expected frame size (in bytes) = (4908*3684*10)/8 = 22,601,340

But the prints shows that the frame size is very less. This can mean that the sensor is not sending the data properly which may occur when the sensor configuration settings are not proper. Please refer to the following KBA to understand more on the different MIPI errors:

MIPI-CSI Protocol and Physical Layer Errors in CX3 (CYUSB3065 and CYUSB3064) – KBA228482

This also occurs due to the problems in data that is sent by the sensor.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
lock attach
Attachments are accessible only for community members.

Hi JayakrishnaT_76​ I think there is not much change after changing the Multiplier of Unit clock setting to 100 in MIPI configuration utility.

Find the log file attached.

0 Likes

Hello,

It seems like the sensor is not configured correctly. Can you please probe FV, LV and PCLK from the testpoints on CX3 and share the traces with us?

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

JayakrishnaT_76

It seems like the sensor is not configured correctly: You are right Line Length timing(ROW_TIME) was not correct.

Thank you for your continuous support.

0 Likes

Hello,

Is the issue solved? Are you able to stream the data properly now?

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

JayakrishnaT_76​ I am able to stream data properly

Thanks for your support.

Regards Jayesh

0 Likes

Hello,

Thanks for the update. Glad to hear that the issue is resolved!

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes