- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Team,
I'm trying to stream 1080P MIPI Raw 10 image sensor . I if configure the MIPI transmitter section and receiver section for 1080p 30 Fps There is no problem in receiver side and It streams correctly. When I done this for 60 FPS I cant change the the output pixel clock in receiver configuration and it shows minimum is 129 and maximum is 100 even if my output video format size is 24-bit.
my camera can support 8/10/20 bit raw output of Bayer format and both continues and gated MIPI clock mode.
Kindly suggest me some settings for 1080P 60 fps.
Thankyou
Solved! Go to Solution.
- Labels:
-
USB Superspeed Peripherals
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I still suspect that the sensor configuration settings are not proper. As mentioned before, CX3 is just a bridge between image sensor and host. It just forwards the data obtained from the image sensor to the host after adding UVC header. No additional frame data is added by CX3. The prints that you are seeing indicates the total amount of data sent by the image sensor. As mentioned before, the expected frame size is calculated as follows:
Frame size = (1920 x 1080 x 10) / 8 = 2592000 bytes
But, from the debug prints, the frame size received by CX3 is 3246480 bytes which is again greater than 2592000 bytes. Initially, you were receiving 3903348 bytes from the image sensor instead of 2592000 bytes. This was brought down to 3246480 bytes by changing the sensor configuration settings. I strongly suspect that the sensor configuration settings used currently is also not correct for your desired resolution. So, kindly check the configuration settings again with the image sensor vendor.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Please share the snapshot of CX3 Receiver configuration tab of MIPI configuration utility so that we can check if the settings are proper or not.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I tried with 16 bit output video format and 24 bit output video format, hear I'm sharing the screenshots for my configurations.
Image sensor configurations for 16 bit
Receiver configurations
Sensor configuration for 24 bit
Receiver configuration for 24 bit
Kindly check this and say what I should change.
*image sensor frame configurations are based on the image sensor that I have been using and it working for 30 fps Configurations.*
Thankyou
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
The configuration settings used for 24 bit output is correct. The error shown in the CX3 configuration Utility is due to a bug which will be fixed in the next release of FX3 SDK. You can ignore this error and go forward to build the project with the configuration settings used for 24 bit output. Please let me know if you face any difficulties while doing this.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I tried The same 24 bit configuration from the above images and build the project the in my camera app it says that media capture failed.
Should I change any format/descriptors internally or should I try using some other camera application.
*The camera application that I'm using is amcap, e-cam-view and default camera*
Kindly say what will be the problem
Thankyou
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Please refer to the following KBA for packing the incoming data into 24 bits:
The modifications mentioned in the KBA mentioned above should be added in your project also. If you are facing issues even after adding the modifications mentioned in the KBA, then kindly share the project after removing the sensor configuration settings so that I can check it.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I tried the KBA which u send previously and my cx3is got enumerated as UVC and There are some mismatch in the frame size
"these ware the frame size that I got in UART logs
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3325 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3326 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3327 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3328 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3329 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3330 Frm_Sz = 3903348 B
TimeDiff = 895 ms FPS = 33
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3331 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3332 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3333 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3334 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3335 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3336 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3337 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3338 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3339 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3340 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3341 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3342 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3343 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3344 Frm_Sz = 3903348 B
Prod = 106 Cons = 105 Prtl_Sz = 852 Frm_Cnt = 3345 Frm_Sz = 3903348 B
"
If I dint change anything in descriptor I got " media capture failed" but still its has been enumerated as UVC is there any option to view image in 24 bit.
If possible kindly suggest that.
hear I'm attached my project file kindly check for that.
Thankyou.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
The modifications mentioned in the descriptors needs to be done for streaming the video data properly. If the modifications are not done, then the descriptors will not be set correctly and can cause troubles to render the video stream.
From the UART debug logs, I understand that the frame size is constant but greater than the required size. The required frame size can be calculated as follows:
Frame size = (1920 x 1080 x 10) / 8 = 2592000 bytes
The data size shown in the UART debug log is 3903348 bytes
This means that the sensor is configured to send more data. CX3 is just a bridge that accepts the MIPI data from the sensor and forward the same to the host. If the sensor is sending more data in one frame, then the same will be forwarded to the host. In addition to this, you can also find that the fps is also shown as 33. This means that the fps set for the sensor is also not correct.
Kindly approach the image sensor vendor and check if the image sensor configuration settings are proper or not.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I tried connecting this UVC device in Linux I got "CB failure" in UART logs, Kindly clarify what is "CB failure".
In mean while I requested for new sensor configuration probably it may take 2 more days to verify that.
Thankyou
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
CB failure occurs when the API CyU3PDmaMultiChannelCommitBuffer() fails. We have seen this issue when the host is slow to issue IN tokens to the device. As you can see that the issue is seen only on Linux host, the same reason holds valid. Please let me know if the device is able to recover by itself when the CB failure is seen or not. That is, are you able to see the data being transferred in the UART logs after the CB failure is seen?
In mean while I requested for new sensor configuration probably it may take 2 more days to verify that
>> Sure, please let me know if you face any issues after trying with the new configurations.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
*are you able to see the data being transferred in the UART logs after the CB failure is seen?
No, I dint get any data transfer logs in UART.
Thankyou
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I would recommend you to create a new thread for discussing the CB failure issue. This could help other customers also who face the same issue. We can continue our discussion on the RAW10 60fps streaming issue in this thread.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I have been created a new thread for CB failure
https://community.cypress.com/t5/USB-Superspeed-Peripherals/CB-failure-while-
streaming-1080P-60fps/m-p/283731#M27629
hear is the link for new thread.
Thankyou
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
We got the new sensor configuration for 60 fps and still its is not streaming the data.
1. I tried generating code for 24 bit raw 10 60fps and replaced the dummy registers with our image sensor registers and tried to stream it says media capture fail in my camera app.
2. I tried replacing "Number of bits per pixel" in" Class specific Uncompressed VS format descriptor" from 10 t0 16. then I got below lines in UART logs.
"Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 869 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 870 Frm_Sz = 3246480 B
TimeDiff = 503 ms FPS = 59
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 871 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 872 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 873 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 874 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 875 Frm_Sz = 3246480 B"
3. I have tried streaming 1080p 60fps register in 1080p 30fps code, Which is streaming and got frame rate as 33fps.
according that KBA that you shared previously we are changing the descriptor form 1920X1080 to 1200x1080, so its not 1920x1080.
My question is can we stream 1080p 60fps directly or not ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
As you might be knowing, UVC applications will not support any formats that are not UVC compliant. RAW10 format is not UVC compliant. So, it is not supported by UVC applications.
Now, the video that you are trying to stream is 1080P RAW10 at 60FPS. Therefore, the bandwidth requirement can be found out as follows:
Data rate = 1920 * 1080 * 10 * 60 = 1.15Gbps
This is supported by CX3. But, CX3 needs to either pad this incoming data into 16 bits or it needs to pack the incoming data into 16/24 bits. If CX3 is configured to pack the incoming data, then the data rate calculated above will not change. But, if CX3 is configured to pad the incoming data into 16 bits, then the data rate will change. The new data rate is given below:
Data rate = 1920 * 1080 * 16 * 60 = 1.85Gbps
But, when a 16 bit parallel bus is used, the maximum bandwidth supported by CX3 is only 1.6Gbps. So, it is not possible to pad the incoming data into 16 bits for this case.
So, the only left over options are either to pack the incoming data into 16 or into 24 bits. For this, you need to change the descriptors as mentioned in the KBA. After these modifications are made, you can use a default UVC player to check if you are receiving any data or not. But, to obtain the original video stream, you need to develop a custom host application itself.
Please do not do any other modifications other than the ones given in the KBA. Please share the UART debug logs if you are facing issues after making the modifications mentioned in the KBA.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
>>regarding KBA that you shared
I tried to modify the descriptor , But still I get media capture failed in the default camera application.
In UART logs I get
"Sensor is in sleep
EnterSuspendMode Status = 0x0, Wakeup reason = 0x8
Sensor is in sleep
EnterSuspendMode Status = 0x0, Wakeup reason = 0x8"
I dint see any data in UART logs.
I do verified that MIPI is streaming.
Hear I'm uploading my project without sensor register configuration file Kindly check the UVC descriptors changes.
>> regarding custom host application.
I found this format in the KBA that you previously shared,
How to arrange these pixel data in rows and column of 1920x1080 frame.
Kindly explain how to split and re-arrange pixel data in my custom host application.
Thankyou.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Can you share the project again without removing the cyu3imagesensor.c file? You can remove the sensor configurations inside the structures declared as it is confidential. Basically, I would like to have a firmware that can be compiled without any errors for testing at my end. You can just remove the sensor configuration settings only from the project.
Please share the project with the modification mentioned above so that I can try to test it at my end.
Regarding the custom host application, please find my comments below:
1. Based on the KBA, when 24 bit output is used, the GPIF II block samples the data as follows:
- First clock cycle - P1[9:2], P2[9:2], P3[9:2],
- Second clock cycle - P4[9:2], P1[1:0], P2[1:0], P3[1:0], P4[1:0], P5[9:2]
- Third clock cycle – and so on.
2. For First clock cycle - P1[9:2], P2[9:2], P3[9:2], the data is presented to the GPIF II pins as follows:
Pixel GPIFII pins
P1[2] - DQ0 LSB
P1[3] - DQ1
....
....
P3[9] - DQ23 MSB
This data will then be put in little endian format into the DMA buffers. CX3 will add a 12 byte UVC header at the start of each DMA buffer. This gives information on the End of frame. The data that follows after this UVC header will be the frame data that was sampled and stored into the DMA buffers by the GPIF II block.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Thank you for sharing the project as requested. I tried to build the project that was shared in your previous response. After this, I downloaded the firmware into Denebola kit. I tested with the Camera Application of Windows 10 and also using AMCap. I did not find the media capture failed error that you saw while testing. Please refer to the snapshot of AMCap below:
Please try testing on a different host as I feel that this issue is specific to the host that you are using for testing.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Apology for that I tried the same thing but I dint get the below image only after switching the workspace and importing the same project I got some some UART logs but I don't see any image.
Snapshot :
UART log "Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 147 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 148 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 149 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 150 Frm_Sz = 3246480 B
TimeDiff = 504 ms FPS = 59
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 151 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 152 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 153 Frm_Sz = 3246480 B
Prod = 88 Cons = 87 Prtl_Sz = 6672 Frm_Cnt = 154 Frm_Sz = 3246480 B"
Is thare any option to pass the data to uvc ?
Thankyou
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I still suspect that the sensor configuration settings are not proper. As mentioned before, CX3 is just a bridge between image sensor and host. It just forwards the data obtained from the image sensor to the host after adding UVC header. No additional frame data is added by CX3. The prints that you are seeing indicates the total amount of data sent by the image sensor. As mentioned before, the expected frame size is calculated as follows:
Frame size = (1920 x 1080 x 10) / 8 = 2592000 bytes
But, from the debug prints, the frame size received by CX3 is 3246480 bytes which is again greater than 2592000 bytes. Initially, you were receiving 3903348 bytes from the image sensor instead of 2592000 bytes. This was brought down to 3246480 bytes by changing the sensor configuration settings. I strongly suspect that the sensor configuration settings used currently is also not correct for your desired resolution. So, kindly check the configuration settings again with the image sensor vendor.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I tried configuring the sensor register, finally I brought up to this in my UART,
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16302 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16303 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16304 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16305 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 18888 Frm_Cnt = 16306 Frm_Sz = 2596008 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16307 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16308 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 16512 Frm_Cnt = 16309 Frm_Sz = 2593632 B
Prod = 70 Cons = 69 Prtl_Sz = 16512 Frm_Cnt = 16310 Frm_Sz = 2593632 B
Prod = 70 Cons = 69 Prtl_Sz = 18924 Frm_Cnt = 16311 Frm_Sz = 2596044 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16312 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16313 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16314 Frm_Sz = 2593644 B
Prod = 70 Cons = 69 Prtl_Sz = 16512 Frm_Cnt = 16315 Frm_Sz = 2593632 B
Prod = 70 Cons = 69 Prtl_Sz = 16524 Frm_Cnt = 16316 Frm_Sz = 2593644 B
But this is not continues and also constant I don't know why, Each time I rewrite my image sensor configuration I got a single frame.
Sensor configuration flow
>>pause the streaming
>>writing the registers
>>Start streaming
This done by and external microcontroller and each time I pause and resume the image senor I t prints the frame size. but its not going continues
the same thing I did to get 1280X720 60fps It started streaming when it reaches almost the nearest value it started streaming, but the same thing is not working for 1920X1080 60fps.
For 1920X1080 30fps the frame size is(1920x1080x16)/8 = 4147200 Due to 6 more bit padded to this.
What should be the exact frame size if it is packed as 24 bit ?
Should I try increasing the frame size ?
Thankyou
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
The frame size when you pack the incoming data into 24 bits will be the same as the actual frame size. When you pack the incoming data, there are no additional padded bits. Please refer to the following KBA again as it explains in detail about how CX3 packs and pads the incoming data:
So, the frame size in this case will be:
frame size = (1920x1080x10)/8 = 2592000
For the UVC player to stream the data, the frame size should be equal to this value. I find that the frame size is still greater and is varying. Please check the sensor configuration settings again with the sensor vendor.
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Thanks for the support, As per your suggestion I tried the modifying the register configuration. This time instead of changing the frame size, I changed the MIPI CSI clock from 720Mhz to 1000Mhz which solves the problem, but still we dint developed any custom Host to view the proper image.
Due to packing 10 bits into 24 bits the image is look like this
Thus the data from cx3 is correct and only thing is, cx3 combines multiple frames in one image.
to get this image I made modifications according to this KBA .
Thankyou.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Thank you for the confirmation. We are glad to hear that the issue is resolved.
Jayakrishna