- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Few days ago I was asking questions regarding the debug interface through USB.
Thread:Debug interface through USB for CX3 + Image sensor application
From this, I was able to create my own firmware based on the example given.
In the mean time, I created the sensor library and digged into the uvc descriptor.
I can successfully talk to my sensor and I'm not getting any mipi error while setting it up.
Since the board used is a custom one, I'm sadly not able to probe the Mipi lanes to make sure they are behaving correctly.
However, I'm using the config given by the sensor manufacturer (with clock and fps modified), which should contain a correct setup.
The objective here is to stream a |1280x800 30fps RAW10| and a |640x400 30fps RAW10| through an USB 2.1 HyperSpeed interface.
I've tried to follow couple of threads available on the forum to fix my issue and get the stream working, without success.
Since I'm completely new in this firmware development world, I'm probably missing something but I can't find out what..
I've attached the project, without the sensor library, any comments are welcome!
This would be my first implementation of an image sensor, you can't imagine how excited I will be if I could have a stream working!
Thanks for your help,
Greg
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
From the firmware, I found that the descriptors and the probe control structure are not changed when the RAW 10 pixels are packed
Please refer Example 2 of this KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387
The frame size in your case is 640 * 400 *10 /8 = 320000 bytes but currently, in the descriptors and probe control structure, the frame size is (640*400*2 bytes ) more than the actual frame size (640*400*10/8 bytes) transferred by the sensor. So the host will expect more pixels in the frame.
So, the actual frame size needs to be mentioned in the probe control structure as well as in descriptors
a. Width in pixel : 640
b. Height in pixel : 400
c. Bits per pixel : 10
d. Frames per second : 30
e. Frame size : 640 x 400 x 10 bits = 2560000 bits
f. Bit rate : 640 x 400 x 10 x 30 bits per second
Re-writing the above calculations:
a. Width in pixel : 400 // Adjusted the width in pixel to match the frame size to 320000 bytes
b. Height in pixel : 400
c. Bits per pixel : 16 //This is done just to transfer RAW10 format as YUY2 format (which supports 16 bites per pixel)
d. Frames per second : 30
e. Frame size : 400 x 400 x 16 bits = 2560000 bits
f. Bit rate : 400 x 400 x 16 x 30 bits per second
So, now the new frame size (after adjusting the width in pixel and bits/pixel to transfer as YUY2 format) is 400*400*2 (bytes/pixel) = 320000 bytes
Similar changes need to be done for other resolution also.
Please refer to example 2 of the KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387 and make the necessary changes in the probe control structure gl400pProbeCtrl_HS/gl400pProbeCtrl and gl800pProbeCtrl_HS/gl800pProbeCtrl and build the firmware again and try streaming.
Please let me know if this works
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
To debug the problem we would need the debug prints when the video streaming starts or the host application is opened to view the stream.
From the firmware, I found that debug prints are sent over USB socket through CDC interface. So, can you share the debug prints after enabling the macro CX3_DEBUG_ENABLED. Please try printing the PROD and CONS events (glDmaDone) and the glHitFV variables in the CyCx3AppThread_Entry (for{} loop)
Please let me know if any queries on this
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Rashi,
Thanks for your help!
I've tried to activate CX3_DEBUG_ENABLED but it's making the cx3 to hang because of this part in CyCx3AppUSBSetupCB:
#if CX3_DEBUG_ENABLED
wLength = ((setupdat1 & CY_U3P_USB_LENGTH_MASK) >> CY_U3P_USB_LENGTH_POS);
CyU3PDebugPrint(4, "\n\rbRType = 0x%x, bRequest = 0x%x, wValue = 0x%x, wIndex = 0x%x, wLength= 0x%x",
bRType, bRequest, wValue, wIndex, wLength);
#endif
So I've commented out this part.
I've put debug prints in CONS_EVENT and PROD_EVENT, but it seems that these ones were never called.
Below is the outputs I obtained while opening eCamViewer and changing resolution.
I guess this is not what I should expect.
Mipicsi Set PHY Delay status = 0x0
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0 //Before resolution config
Writing OV9782 800p configuration done
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0 //After resolution config
AplnStrt:SMState = 0x2
Mipicsi Set PHY Delay status = 0x0
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0 //Before resolution config
Writing OV9782 400p configuration done
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0 //After resolution config
USB Setup CB:Call App stop is called
APP STOP
Writing OV9782 Sleep_Config size-10
OV9782 Sleep_Config--
AplnStop:SMState = 0x2
AplnStrt:SMState = 0x2
I had an App Stop called even if nothing was closed.
Any thought on what I should do?
Moreover a lot of the print statements are never sent. I'm guessing that this is due to some part of the CX3 not initialized yet, is that correct?
Is there a way to fix this?
Thanks,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
Please send the complete debug log so that I can know if there is something being missed. In the debug prints you shared, i have not seen the prints like "Setting MIPI interface" which are there in the firmware you shared.
You can comment out this print as of now
CyU3PDebugPrint(4, "\n\rbRType = 0x%x, bRequest = 0x%x, wValue = 0x%x, wIndex = 0x%x, wLength= 0x%x", bRType, bRequest, wValue, wIndex, wLength);
As the start and stop GPIF state are same it seems the data is not received by the CX3.
Please let me know if you have the test point on your board to probe the FV, LV, and PCLK and share the traces. Please refer Q10 of this KBA CX3 Hardware: Frequently Asked Questions - KBA91295 . From this we can confirm if the video data received by the MIPI receiver of CX3 is as expected.
Also, share the snippets of CX3 MIPI Receiver Configuration Settings for me to review
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Rashi,
Sorry for the delayed answer.
For some reason, the debug logs you have seen are the entire logs available. I canno't explain why at this stage, but as I mentioned, I'm guessing that part of the CYUSB3064 is not initialized/ready when the outputs are sent. Is there anyway to fix that?
Sadly this test points are not currently available on my board. However FSIN/VSYNC of image sensor is linked to GPIO23 if this can help..
Thanks for your help,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
As the endpoint related to the CDC interface (used for the debugging purpose)is configured after CyU3PConnectState API, so the debug prints before that won't be seen.
To check if the APIs before that pass successfully you can call CyU3PDeviceReset(CyFalse) in the case of the failures of API and check that device doesn't reset. If the device resets that means that particular API is not passing successfully.
If all the API before CyU3PConnectState API passes and still the CY_U3P_DMA_CB_PROD_EVENT events are not seen then please check if the sensor settings and the CX3 MIPI receiver are the same. Please share the CX3 MIPI receiver configuration toll settings for us to check.
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Rashi,
I've put CyU3PDeviceReset(CyFalse) in all the if(Failed) statements in the AppInit function.
None of them is making the device to reset. I'm guessing that everything is initialized correctly then.
Below are some more information about the clocks:
Configuration of the sensor's clocks
Configuration of the sensor:
CX3 Mipi receiver configuration tool:
Hopefully this can help.
Thanks,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
The CX3 configuration tool settings seem fine as all the parameters are in range.
In the firmware you shared I see the debug prints are used inside the DMA callback.
We do not recommend to use blocking calls (like CyU3PDebugPrint()) in the callbacks. Please use global variables to track he CY_U3P_DMA_CB_PROD_EVENT and CY_U3P_DMA_CB_CONS_EVENT events and then later get those variable printed in CyCx3AppThread_Entry for(;;) loop.
There are possibilities that the debug prints in the DMA callback do not print over UART. So, please try the above-mentioned suggestion and let me know if you get the CY_U3P_DMA_CB_PROD_EVENT when the host application is opened (CyCx3AppStart is called).
Also, try printing MIPI errors.
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Rashi,
I can confirm, the PROD and CONS Event are never called.
I've put a boolean variable in both of them to see if it's called at the some point and it is never called.
Does that mean that no data are received on the mipi lanes?
Should I try to scratch the silkscreen of the mipi lanes on one of the board to prob them?
Mipi Errors are still giving 0 for all the parameters when a camera resolution is called.
However I can see an AplnStr:SmState=0x2, is that an issue?
I also don't understand why AppStop is called when I'm changing the resolution used.
See logs:
Mipi sleep
Mipicsi Set PHY Delay status = 0x0
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0
Writing OV9782 800p configuration done
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0
AplnStrt:SMState = 0x2
Mipicsi Set PHY Delay status = 0x0
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0
Writing OV9782 400p configuration done
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0
USB Setup CB:Call App stop is called
APP STOP
Writing OV9782 Sleep_Config size-10
OV9782 Sleep_Config--
AplnStop:SMState = 0x2
AplnStrt:SMState = 0x2
Thanks,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
Does that mean that no data are received on the mipi lanes?
If the producer events are not seen that means the data is not being written to the DMA buffer. It seems that either the sensor is not giving the output or if the sensor s giving out the output the sensor settings and the CX3 MIPI receiver settings are not in sync.
Yes, you can try probing the MIPI clock and data lane and check if the sensor outputs the clock and data. Please check the MIPI clock frequency
However I can see an AplnStr:SmState=0x2, is that an issue?
>> This shows that the GPIF state machine is in state 0x02 which is the wait state. The GPIF state machine transitions to the next state only when the FV signal s high that is when frame data is being transmitted.
As AplnStop:SMState = 0x2 i.e the state doesn't transition to another sate which means that frame data is not being transferred.
When the resolution is changed on the fly, CX3_USB_UVC_SET_CUR_REQ request is sent by the host to set the new resolution by configuring the sensor.
From your firmware
/* Set Commit Control and Start Streaming*/
if (wValue == CX3_APP_VS_COMMIT_CONTROL) // SET_CUR COMMIT request
{
CyCx3AppImageSensorSetVideoResolution (glCommitCtrl[3]); // Set the sensor with new settings
if (glIsApplnActive)
{
CyU3PDebugPrint (4, "\n\rUSB Setup CB:Call AppSTOP1");
CyCx3AppStop(); // If the streaming is going on we need to stop the streaming with old resolution and restart the resolution with new resolution after resetting the DMA pipe
}
CyCx3AppStart(); // started streaming with new resolution
}
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Rashi!
Thanks again for you help and apologise for the delayed answer!
I've removed the correct answer since the problem isn't solved yet.
I was only able to probe the Mipi signals today.
I'm guessing that my oscilloscope isn't good enough to get any correct description of the signals and or the clock.
However, from what I can see with the signal probed, there is definitely data sent through the mipi routes.
I'm guessing a 33 ms period which should correspond to my 1280x800 | 30 FPS frame format. With signals sent for a period of around 8.33ms. Which correspond to a maximum of 120 fps for that format with a 2 mipi lanes interface.
I'm also seeing the corresponding changes while swapping to the 640x400 | 30 FPS format.
The MIPI-CP and MIPI-CN lanes are giving a 214mV visually continuous signal, nothing better can be obtain with the 110MHz of my oscilloscope.
But if I refer to the signals above, I'm guessing that the clock is working fine, correct?
Following your comments, it seems that the configuration of the sensor and the CX3 are not matching then.
Any hints on what I should modify/work on?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Rashi,
Since yesterday I spent a bit of time trying to figure out why. I wasn't receiving the PROD and CONS Event.
I found out that I didn't follow the Continuous clock mode process.
I did it a minute ago and I'm now getting the events as expected!
No images yet..
EDIT1:
Here is a set of logs, (but no images received this time):
Mipicsi Set PHY Delay status = 0x0
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0?
Writing OV9782 800p configuration done
crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0?
USB Setup CB:Call App start is called
AplnStrt:SMState = 0x2
Writing OV9782 Start_Config size-2
OV9782 Start_Config--
AppDmaCallBack CONS EVENT: glDmaDone:2 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:4 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:6 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:7 | glHitFV:0
ew Event AppThreadEntry
Frame timed out
ÿÿ
APP STOP
AppDmaCallBack CONS EVENT: glDmaDone:8 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:7 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:6 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:5 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:4 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:2 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:1 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:0 | glHitFV:0
Writing OV9782 Sleep_Config size-10
OV9782 Sleep_Config--
AplnStop:SMState = 0x7
AplnStrt:SMState = 0x1
Writing OV9782 Start_Config size-2
OV9782 Start_Config--
ew Event AppThreadEntry
Frame timed out
ÿÿ
APP STOP
Writing OV9782 Sleep_Config size-10
OV9782 Sleep_Config--
AplnStop:SMState = 0x1
AplnStrt:SMState = 0x1
Writing OV9782 Start_Config size-2
OV9782 Start_Config--
EDIT2:
The below logs are spam when the 640x400 resolution is used.
Images are apparently received but the usual cam apps (eCamviewer for example) are not showing any FPS.
Any reason for that?
AppDmaCallBack CONS EVENT: glDmaDone:6 | glHitFV:1
AppDmaCallBack CONS EVENT: glDmaDone:5 | glHitFV:1
AppDmaCallBack CONS EVENT: glDmaDone:4 | glHitFV:1
AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:1
AppDmaCallBack CONS EVENT: glDmaDone:2 | glHitFV:1
AppDmaCallBack CONS EVENT: glDmaDone:1 | glHitFV:1
AppDmaCallBack CONS EVENT: glDmaDone:0 | glHitFV:1
AppDmaCallBack CONS EVENT: glDmaDone:1 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:2 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:4 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:5 | glHitFV:0
AppDmaCallBack CONS EVENT: glDmaDone:5 | glHitFV:0
Size of frame is 320000
Have a nice day!
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
Thank you for the update.
For the first case, I can see the Frame timed out error which occurs when the video frame is not received before the the timeout value expires. You try increasing the timer's timeout value such that that the Timer value (TIMER_PERIOD - Maximum frame transfer time in milli-seconds) > V_total period
For second case, please find the attached file which calculates and prints the FPS (fps) and other parameters like prod, cons and frame size. Please refer to this and add the same debug prints to check if the fps coming from the sensor is as expected. Please share the UART debug prints for us to check
if (Printflag == 1)
{
/*For video streaming application of higher FPS refrain from using this debug print or try to reduce the print information*/
CyU3PDebugPrint(4,"\n\rProd = %d Cons = %d Prtl_Sz = %d Frm_Cnt = %d Frm_Sz = %d B", TxCountflag, RxCountflag, PartialBufSize, FrameCount, ((TxCountflag*CX3_UVC_DATA_BUF_SIZE)+PartialBufSize));
Printflag = 0;
if (fpsflag == 1)
{
fps = 30000/(time1 -time0); //FPS calculate using time difference for 30 frames
CyU3PDebugPrint(4,"\n\rTimeDiff = %d ms FPS = %d", (time1 -time0), fps);
fpsflag = 0;
}
Please let me know if any queries on this
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Rashi,
Thanks for your feedback!
Is the file missing in your message or is it just the lines of code below?
If so, where should I trigger fpsflag=1 ? and time0|1?
Thanks for your help,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
Please find the attached file(cycx3_uvc.c) which has the declaration and definition of these variables. these variables are triggered in the DMA callback (CyCx3UvcAppDmaCallback)
Please note that the FPS counted in the attached code is for 30 fps. Please modify the code snippet as per the settings of the fps in your application
- fpsflag ad time1
Please search for CyCx3UvcAppDmaCallback in the attached file.
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Rashi,
Sorry I missed the attached file in the first message.
EDIT: I've put the wrong resolution logs.
I did the changes and here is the output of it:
prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 511 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 512 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 513 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 514 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 515 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 516 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 517 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 518 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 519 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 520 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 521 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 522 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 523 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 524 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 525 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 526 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 527 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 528 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 529 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 530 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 531 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 532 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 533 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 534 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 535 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 536 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 537 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 538 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 539 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 540 Frm_Sz = 320000 B
TimeDiff = 1004 ms FPS = 29 C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 541 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 542 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 543 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 544 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 545 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 546 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 547 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 548 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 549 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 550 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 551 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 552 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 553 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 554 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 555 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 556 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 557 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 558 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 559 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 560 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 561 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 562 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 563 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 564 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 565 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 566 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 567 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 568 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 569 Frm_Sz = 320000 B C
Prod = 13 Cons = 7 Prtl_Sz = 720 Frm_Cnt = 570 Frm_Sz = 320000 B
TimeDiff = 1005 ms FPS = 29 C
It looks like the FPS are as expected.
So what could be the reason why my host is not receiving any images. I can't find anything in here saying that the USB transfer hasn't been completed.
Is it because my sensor is streaming RAW10 images? I did the changed to remove the 6 padded 0 as explained in:
Maybe I missed something? Also I'm guessing that Prod and Cons should be equal?
Thanks again,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
From the debug prints, it is seen that the frame size is not constant. This is the reason the frames are not seen on the host application
It is expected that the frame size received from the sensor is constant. Please let me know if the pixels are being packed or padded with non image data (zeros) Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387
Please let me know the changes that were done for continuous clock process. Are changes done as per Q13 of this KBA CX3 Hardware: Frequently Asked Questions - KBA91295
Please let me know if the custom board has the test point for probing the Hsync, Vsync and PCLK. Please refer to Q10of the above KBA
As per your previous reply "However FSIN/VSYNC of image sensor is linked to GPIO23 if this can help.." Please probe this pin for the VSYNC to check if the the FPS is as expected
Also, enable the MIPI errors in the firmware. Please refer to the same file shared earlier for the same
CyU3PMipicsiGetErrors( CyTrue, &errCnts);
CyU3PDebugPrint(4,"\n\r%d %d %d %d %d %d %d %d %d",errCnts.crcErrCnt,errCnts.ctlErrCnt, errCnts.eidErrCnt, errCnts.frmErrCnt, errCnts.mdlErrCnt, errCnts.recSyncErrCnt, errCnts.recrErrCnt, errCnts.unrSyncErrCnt, errCnts.unrcErrCnt );
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Rashi,
Sorry I just Edited my message, I was using the wrong resolution which hasn't been set correctly.
Could have a look at the new logs? Thanks.
I'm going to go through your message anyway
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Frames size seems consistent now.
It is expected that the frame size received from the sensor is constant. Please let me know if the pixels are being packed or padded with non image data (zeros) Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387
Yes, I have been using Example 2 and made the necessary adjustement.
Please let me know the changes that were done for continuous clock process. Are changes done as per Q13 of this KBA CX3 Hardware: Frequently Asked Questions - KBA91295
Yes I have done that too.
Please let me know if the custom board has the test point for probing the Hsync, Vsync and PCLK. Please refer to Q10of the above KBA
As per your previous reply "However FSIN/VSYNC of image sensor is linked to GPIO23 if this can help.." Please probe this pin for the VSYNC to check if the the FPS is as expected
Also, enable the MIPI errors in the firmware. Please refer to the same file shared earlier for the same
CyU3PMipicsiGetErrors( CyTrue, &errCnts);
CyU3PDebugPrint(4,"\n\r%d %d %d %d %d %d %d %d %d",errCnts.crcErrCnt,errCnts.ctlErrCnt, errCnts.eidErrCnt, errCnts.frmErrCnt, errCnts.mdlErrCnt, errCnts.recSyncErrCnt, errCnts.recrErrCnt, errCnts.unrSyncErrCnt, errCnts.unrcErrCnt );
Following the above new logs, is this still needed?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
From the new logs, the frame size is constant.
So what could be the reason why my host is not receiving any images. I can't find anything in here saying that the USB transfer hasn't been completed.
>> Please let me know the video frame size to calculate the frame size. We need to check if the frame size received in the firmware (seen in the logs) is same as h_active*v_active*10/8(RAW 10)
Maybe I missed something? Also I'm guessing that Prod and Cons should be equal?
>> Yes, it is expected PROD and CONS to be equal. Please let me know if CyU3PDmaMultiChannelCommitBuffer API fails. You can check this by setting an event in the CyCx3AppDmaCallback i.e. when this API fails and in CyCx3AppThread_Entry, CyU3PEventGet can be used to handle this event. Please refer the previous file for reference.
Please share the the probe control structure (UVC Probe Control Setting) for this resolution and the DMA buffer size used in your application
Probe control structure would be something like uint8_t const glVga60ProbeCtrl[CX3_APP_MAX_PROBE_SETTING] (example)
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Rashi,
Thanks for your message!
Please let me know the video frame size to calculate the frame size. We need to check if the frame size received in the firmware (seen in the logs) is same as h_active*v_active*10/8(RAW 10)
Frame size expected is 640*400 so the frame size given appears correct. 320000
Yes, it is expected PROD and CONS to be equal. Please let me know if CyU3PDmaMultiChannelCommitBuffer API fails. You can check this by setting an event in the CyCx3AppDmaCallback i.e. when this API fails and in CyCx3AppThread_Entry, CyU3PEventGet can be used to handle this event. Please refer the previous file for reference.
I've put a "Frame timed out" print in the case of this Event is called. It doesn't appear to be called.
Also I've noticed this in your example:
if(dmaBuffer.count < CX3_UVC_DATA_BUF_SIZE)
{
#ifdef UVC_APPLICATION
CyCx3UvcAppAddHeader ((dmaBuffer.buffer - CX3_UVC_PROD_HEADER), CX3_UVC_HEADER_EOF);
#endif
glHitFV = CyTrue;#ifdef PRINT_FRAME_INFO
FrameCount++;
PartialBufSize = dmaBuffer.count;
RxCountflag = RxCount;
TxCountflag = TxCount;
Printflag = 1; if ((FrameCount % 30) == 0)
{
time1 = CyU3PGetTime();
fpsflag = 1;
gettimeflag = 0;
}#endif
}
else
{
#ifdef UVC_APPLICATION
CyCx3UvcAppAddHeader ((dmaBuffer.buffer - CX3_UVC_PROD_HEADER), CX3_UVC_HEADER_FRAME);
#endif
} /* Commit Buffer to USB*/
#ifdef UVC_APPLICATION
status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count + 12), 0);
#else
status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count), 0);
#endif
if (status != CY_U3P_SUCCESS)
{
CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT,CYU3P_EVENT_OR);
break;
}
Should I be using the UVC application if statement or not, I'm not too sure anymore.
Please share the the probe control structure (UVC Probe Control Setting) for this resolution and the DMA buffer size used in your application
Probe control structure would be something like uint8_t const glVga60ProbeCtrl[CX3_APP_MAX_PROBE_SETTING] (example)
UVC Probe Control Settings
/* UVC Probe Control Setting - 400p*/
uint8_t const gl400pProbeCtrl_HS[CX3_APP_MAX_PROBE_SETTING] = {
0x00, 0x00, /* bmHint : No fixed parameters */
0x01, /* Use 1st Video format index */
0x01, /* Use 1st Video frame index */
0x15,0x16,0x05,0x00, /* Desired frame interval in 100ns = (1/30)x10^7 */
0x00, 0x00, /* Key frame rate in key frame/video frame units */
0x00, 0x00, /* PFrame rate in PFrame / key frame units */
0x00, 0x00, /* Compression quality control */
0x00, 0x00, /* Window size for average bit rate */
0x00, 0x00, /* Internal video streaming i/f latency in ms */
0x00,0xd0,0x07,0x00, /* Max video frame size in bytes = 640 x 400 x 2 */
0x00, 0x90, 0x00, 0x00, /* No. of bytes device can rx in single payload: 36KB */
0x00, 0x60, 0xE3, 0x16, /* Device Clock */
0x00, /* Framing Information - Ignored for uncompressed format*/
0x00, /* Preferred payload format version */
0x00, /* Minimum payload format version */
0x00 /* Maximum payload format version */
};
Function is:
extern const uint8_t gl400pProbeCtrl_HS[CX3_APP_MAX_PROBE_SETTING];
DMA buffer size
/* CX3 Application Buffer Parameters*/
#define CX3_APP_DATA_BUF_SIZE (0x5FF0) /* DMA Buffer Data Size Used: 12272 Bytes*/
Hopefully this is helpfull for you.
Thanks in advance,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
Frame size expected is 640*400 so the frame size given appears correct. 320000
>> Thank you for confirming
I've put a "Frame timed out" print in the case of this Event is called. It doesn't appear to be called.
>> Please do not use CyU3PDebugPrint inside the DMA callback. Instead you can use a global variable to track the failure of the CyU3PDmaMultiChannelCommitBuffer or use the SET/GET event API's as done in the shared cycx3_uvc.c file. Please let me know if
CyU3PDmaMultiChannelCommitBuffer fails. If yes, please share the return value of the API.
- status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count), 0);
- #endif
- if (status != CY_U3P_SUCCESS)
- {
- CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT,CYU3P_EVENT_OR);
- break;
- }
Should I be using the UVC application if statement or not, I'm not too sure anymore.
>> Please let me know which host application are you using for viewing the video stream and if your application requires to be a UVC compliant device then the macro UVC_APPLICATION needs to be enabled. Please share the DMA callback function from your firmware so that I can understand.
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Rashi,
>> Please do not use CyU3PDebugPrint inside the DMA callback. Instead you can use a global variable to track the failure of the CyU3PDmaMultiChannelCommitBuffer or use the SET/GET event API's as done in the shared cycx3_uvc.c file. Please let me know if
CyU3PDmaMultiChannelCommitBuffer fails. If yes, please share the return value of the API.
- status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count), 0);
- #endif
- if (status != CY_U3P_SUCCESS)
- {
- CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT,CYU3P_EVENT_OR);
- break;
- }
Sorry what I explained wasn't clear, I did the above and put the "frame timed out" in the for(;;) loop. So when the event is triggered the message frame timed out is printed. This isn't the case so the status of CyU3PDmaMultiChannelCommitBuffer is always
>> Please let me know which host application are you using for viewing the video stream and if your application requires to be a UVC compliant device then the macro UVC_APPLICATION needs to be enabled. Please share the DMA callback function from your firmware so that I can understand.
I've tried the native windows camera app, eCamviewer and I also wrote a small script in python using opencv. None can grab a frame and display. However there is definitely packets sent through the USB port since WireShark is going crazy when the stream is open.
I'm not good enough to understand more of it in wireShark.
For DMA callback function, see the file attached.
Thanks again for your help.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
If the Commit buffer API never fails, please share the Wireshark traces and use eCamView for streaming the video.
Please start capturing the USB traces when the device is plugged in, then program the CX3 with the .img file, and then start streaming by opening the eCamView
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Rashi,
I've attached the traces for the CX3 before upload of the firmware and the traces of the camera from flashing firmware to having the stream running.
I don't really know how to keep traces when the device is going from CX3 boot loader to actual USB capture device.
EDIT: The packages sent are only linked to the debug logs through the serial emulator. Nothing to do with images. If the stream is opened and the serial debugger is never called, no image packages are sent to the USB port...
I let you have a look at them, I'm not too sure what to look at here.
Thanks,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
From the traces, I found that USB packets are sent to the host. (24599 URB_BULK_IN). And it can be seen that 13 such packets are sent 13 - 24599 (24572 (24560 of data +12 bytes of UVC header) + 27 bytes of Wireshark header) packets and 1 partial packet of 759 (732 (720 bytes of data + 12 bytes of UVC header) + 27 bytes Wireshark header). So the full-frame 320000 is seen on the USB bus.
As the RAW format is not supported by UVC specification, the RAW video is transported as YUY2 to the UVC driver. It is expected that the video may not be seen or seen pinkish/greenish as UVC players will sample the data as per YUY2 format and not RAW 10. A custom host application needs to be developed for viewing the RAW video data
Can you please show the snippet of the eCamView where the video is to be seen?
Is it possible to use a different PC and try streaming? You can also try using AmCap or MPC-HC Player (UVC player) to view the video.
EDIT: The packages sent are only linked to the debug logs through the serial emulator. Nothing to do with images. If the stream is opened and the serial debugger is never called, no image packages are sent to the USB port...
>> I didn't understand this. Can you please explain this?
Also, from the traces, I see that the device enumerates with multiple interface UVC and CDC.
It seems that CDC interfaces used for debug prints. Is my understanding correct?
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Rashi,
Thanks for your feedback.
Yes I understand that, I'm expecting to see the weird behavior regarding the frames. But at least I would like to have frames
Here is the result obtained with eCamView
I also tried Amcap and this is the result:
I can't see any images going through either? Is there a setting that I missed to display the RAW10 format?
I'm trying now on another computer (Mac under Windows 10 bootcamp). Let's see if I'm more lucky.
>> I didn't understand this. Can you please explain this?
Also, from the traces, I see that the device enumerates with multiple interface UVC and CDC.
It seems that CDC interfaces used for debug prints. Is my understanding correct?
This is absolutely correct.
I was trying to say that if the serial debugger interface is not open prior to open the camera feed, then Wireshark do not display the (24599 URB_BULK_IN) packets as expected. Which seems weird in my opinion. This would say that the streaming can't start without the debug interface opened.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Rashi,
Exact same behavior on my Windows 10 bootcamp and eCamviewer.
By the way is there a way to export the wireShark packages and preview the image? So at least I could confirm that the data are correct.
Thanks,
Greg
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
was trying to say that if the serial debugger interface is not open prior to open the camera feed, then Wireshark do not display the (24599 URB_BULK_IN) packets as expected
>> This is not expected.
Can you share the firmware files for us to check. You can remove the sensor settings from the firmware and share the complete firmware.
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Greg,
From the firmware, I found that the descriptors and the probe control structure are not changed when the RAW 10 pixels are packed
Please refer Example 2 of this KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387
The frame size in your case is 640 * 400 *10 /8 = 320000 bytes but currently, in the descriptors and probe control structure, the frame size is (640*400*2 bytes ) more than the actual frame size (640*400*10/8 bytes) transferred by the sensor. So the host will expect more pixels in the frame.
So, the actual frame size needs to be mentioned in the probe control structure as well as in descriptors
a. Width in pixel : 640
b. Height in pixel : 400
c. Bits per pixel : 10
d. Frames per second : 30
e. Frame size : 640 x 400 x 10 bits = 2560000 bits
f. Bit rate : 640 x 400 x 10 x 30 bits per second
Re-writing the above calculations:
a. Width in pixel : 400 // Adjusted the width in pixel to match the frame size to 320000 bytes
b. Height in pixel : 400
c. Bits per pixel : 16 //This is done just to transfer RAW10 format as YUY2 format (which supports 16 bites per pixel)
d. Frames per second : 30
e. Frame size : 400 x 400 x 16 bits = 2560000 bits
f. Bit rate : 400 x 400 x 16 x 30 bits per second
So, now the new frame size (after adjusting the width in pixel and bits/pixel to transfer as YUY2 format) is 400*400*2 (bytes/pixel) = 320000 bytes
Similar changes need to be done for other resolution also.
Please refer to example 2 of the KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387 and make the necessary changes in the probe control structure gl400pProbeCtrl_HS/gl400pProbeCtrl and gl800pProbeCtrl_HS/gl800pProbeCtrl and build the firmware again and try streaming.
Please let me know if this works
Regards,
Rashi
Rashi
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Rashi,
Thanks for your feedback on this firmware.
As you probably noticed with the "correct answer" ticked, your solution solved my problem and I was able to acquire images.
Here is the result:
Now I need to reformat this stream.
Thanks a lot for your help on this project! It was super exciting to build my first camera and I'm looking forward to see the progress I will make on the firmware/SDK.
Have a great day!
Greg