Stream is not working: Custom board OV9782 / CYUSB3064

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
lock attach
Attachments are accessible only for community members.
GrAU_4638336
Level 4
Level 4
25 replies posted 10 replies posted 10 sign-ins

Hello,

Few days ago I was asking questions regarding the debug interface through USB.

Thread:Debug interface through USB for CX3 + Image sensor application

From this, I was able to create my own firmware based on the example given.

In the mean time, I created the sensor library and digged into the uvc descriptor.

I can successfully talk to my sensor and I'm not getting any mipi error while setting it up.

Since the board used is a custom one, I'm sadly not able to probe the Mipi lanes to make sure they are behaving correctly.

However, I'm using the config given by the sensor manufacturer (with clock and fps modified), which should contain a correct setup.

The objective here is to stream a |1280x800 30fps RAW10| and a |640x400 30fps RAW10| through an USB 2.1 HyperSpeed interface.

I've tried to follow couple of threads available on the forum to fix my issue and get the stream working, without success.

Since I'm completely new in this firmware development world, I'm probably missing something but I can't find out what..

I've attached the project, without the sensor library, any comments are welcome!

This would be my first implementation of an image sensor, you can't imagine how excited I will be if I could have a stream working!

Thanks for your help,

Greg

0 Likes
1 Solution

Hello Greg,

From the firmware, I found that the descriptors and the probe control structure are not changed when the RAW 10 pixels are packed

Please refer Example 2 of this KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387

The frame size in your case is  640 * 400 *10 /8 = 320000 bytes but currently, in the descriptors and probe control structure, the frame size is  (640*400*2 bytes ) more than the actual frame size (640*400*10/8 bytes) transferred by the sensor. So the host will expect more pixels in the frame.

So, the actual frame size needs to be mentioned in the probe control structure as well as in descriptors

a. Width in pixel                  : 640

b. Height in pixel                : 400

c. Bits per pixel                  : 10

d. Frames per second       : 30

e. Frame size                      : 640 x 400 x 10 bits =  2560000 bits

f. Bit rate                             : 640 x 400 x 10 x 30 bits per second

Re-writing the above calculations:

a. Width in pixel                  :  400     // Adjusted the width in pixel to match the frame size to 320000 bytes

b. Height in pixel                :   400

c. Bits per pixel                  :  16  //This is done just to transfer RAW10 format as YUY2 format (which supports 16 bites per pixel)

d. Frames per second       :  30

e. Frame size                      :  400 x 400 x 16 bits = 2560000 bits   

f. Bit rate                             :  400 x 400 x 16 x 30 bits per second

So, now the new frame size (after adjusting the width in pixel and bits/pixel to transfer as YUY2 format) is 400*400*2 (bytes/pixel) =  320000 bytes

Similar changes need to be done for other resolution also.

Please refer to example 2 of the KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387 and make the necessary changes in the probe control structure gl400pProbeCtrl_HS/gl400pProbeCtrl and gl800pProbeCtrl_HS/gl800pProbeCtrl  and build the firmware again and try streaming.

Please let me know if this works

Regards,

Rashi

Regards,
Rashi

View solution in original post

0 Likes
32 Replies
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Greg,

To debug the problem we would need the debug prints when the video streaming starts or the host application is opened to view the stream.

From the firmware, I found that debug prints are sent over USB socket through  CDC interface. So, can you share the debug prints after enabling the macro CX3_DEBUG_ENABLED. Please try printing the PROD and CONS events (glDmaDone) and the  glHitFV variables in the CyCx3AppThread_Entry (for{} loop)

Please let me know if any queries on this

Regards,

Rashi

Regards,
Rashi
0 Likes

Hello Rashi,

Thanks for your help!
I've tried to activate CX3_DEBUG_ENABLED but it's making the cx3 to hang because of this part in CyCx3AppUSBSetupCB:

#if CX3_DEBUG_ENABLED

    wLength  = ((setupdat1 & CY_U3P_USB_LENGTH_MASK)  >> CY_U3P_USB_LENGTH_POS);

    CyU3PDebugPrint(4, "\n\rbRType = 0x%x, bRequest = 0x%x, wValue = 0x%x, wIndex = 0x%x, wLength= 0x%x",

            bRType, bRequest, wValue, wIndex, wLength);

#endif

So I've commented out this part.

I've put debug prints in CONS_EVENT and PROD_EVENT, but it seems that these ones were never called.

Below is the outputs I obtained while opening eCamViewer and changing resolution.

I guess this is not what I should expect.

Mipicsi Set PHY Delay status = 0x0

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0 //Before resolution config

Writing OV9782 800p configuration done

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0 //After resolution config

AplnStrt:SMState = 0x2

Mipicsi Set PHY Delay status = 0x0

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0 //Before resolution config

Writing OV9782 400p configuration done

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0 //After resolution config

USB Setup CB:Call App stop is called

APP STOP

Writing OV9782 Sleep_Config size-10

OV9782 Sleep_Config--

AplnStop:SMState = 0x2

AplnStrt:SMState = 0x2

I had an App Stop called even if nothing was closed.
Any thought on what I should do?

Moreover a lot of the print statements are never sent. I'm guessing that this is due to some part of the CX3 not initialized yet, is that correct?

Is there a way to fix this?

Thanks,

Greg

0 Likes

Hello Greg,

Please send the complete debug log so that I can know if there is something being missed. In the debug prints you shared, i have not seen the prints like  "Setting MIPI interface" which are there in the firmware you shared.

You can comment out this print as of now

CyU3PDebugPrint(4, "\n\rbRType = 0x%x, bRequest = 0x%x, wValue = 0x%x, wIndex = 0x%x, wLength= 0x%x"bRType, bRequest, wValue, wIndex, wLength);

As the start and stop GPIF state are same it seems the data is not received by the CX3.

Please let me know if you have the test point on your board to probe the FV, LV, and PCLK and share the traces. Please refer Q10 of this KBA CX3 Hardware: Frequently Asked Questions - KBA91295 . From this we can confirm if the video data received by the MIPI receiver of CX3 is as expected.

Also, share the snippets of CX3 MIPI Receiver Configuration Settings for me to review

Regards,

Rashi

Regards,
Rashi
0 Likes
GrAU_4638336
Level 4
Level 4
25 replies posted 10 replies posted 10 sign-ins

Hello Rashi,

Sorry for the delayed answer.

For some reason, the debug logs you have seen are the entire logs available. I canno't explain why at this stage, but as I mentioned, I'm guessing that part of the CYUSB3064 is not initialized/ready when the outputs are sent. Is there anyway to fix that?

Sadly this test points are not currently available on my board. However FSIN/VSYNC of image sensor is linked to GPIO23 if this can help..

Thanks for your help,

Greg

0 Likes

Hello Greg,

As the endpoint related to the CDC interface (used for the debugging purpose)is configured after CyU3PConnectState API, so the debug prints before that won't be seen.

To check if the APIs before that pass successfully you can call CyU3PDeviceReset(CyFalse) in the case of the failures of API and check that device doesn't reset. If the device resets that means that particular API is not passing successfully.

If all the API before CyU3PConnectState API passes and still the CY_U3P_DMA_CB_PROD_EVENT events are not seen then please check if the sensor settings and the CX3 MIPI receiver are the same. Please share the CX3 MIPI receiver configuration toll settings for us to check.

Regards,

Rashi

Regards,
Rashi
0 Likes
GrAU_4638336
Level 4
Level 4
25 replies posted 10 replies posted 10 sign-ins

Rashi,

I've put CyU3PDeviceReset(CyFalse) in all the if(Failed) statements in the AppInit function.

None of them is making the device to reset. I'm guessing that everything is initialized correctly then.

Below are some more information about the clocks:

Configuration of the sensor's clocks

image (1).png

Configuration of the sensor:

image.png

CX3 Mipi receiver configuration tool:

Capture.PNG

Hopefully this can help.

Thanks,

Greg

0 Likes
lock attach
Attachments are accessible only for community members.
GrAU_4638336
Level 4
Level 4
25 replies posted 10 replies posted 10 sign-ins

I attached the last version of the code.

It has probably slightly changed since the first publication.

0 Likes

Hello Greg,

The CX3 configuration tool settings seem fine as all the parameters are in range.

In the firmware you shared I see the debug prints are used inside the DMA callback.

We do not recommend to use blocking calls (like CyU3PDebugPrint()) in the callbacks. Please use global variables to track he CY_U3P_DMA_CB_PROD_EVENT and CY_U3P_DMA_CB_CONS_EVENT events and then later get those variable printed in CyCx3AppThread_Entry for(;;) loop.

There are possibilities that the debug prints in the DMA callback do not print over UART. So, please try the above-mentioned suggestion and let me know if you get the CY_U3P_DMA_CB_PROD_EVENT when the host application is opened (CyCx3AppStart is called).

Also, try printing MIPI errors.

Regards,

Rashi

Regards,
Rashi
0 Likes
GrAU_4638336
Level 4
Level 4
25 replies posted 10 replies posted 10 sign-ins

Hi Rashi,

I can confirm, the PROD and CONS Event are never called.

I've put a boolean variable in both of them to see if it's called at the some point and it is never called.

Does that mean that no data are received on the mipi lanes?

Should I try to scratch the silkscreen of the mipi lanes on one of the board to prob them?

Mipi Errors are still giving 0 for all the parameters when a camera resolution is called.

However I can see an AplnStr:SmState=0x2, is that an issue?

I also don't understand why AppStop is called when I'm changing the resolution used.

See logs:

Mipi sleep

Mipicsi Set PHY Delay status = 0x0

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0

Writing OV9782 800p configuration done

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0

AplnStrt:SMState = 0x2

Mipicsi Set PHY Delay status = 0x0

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0

Writing OV9782 400p configuration done

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0

USB Setup CB:Call App stop is called

APP STOP

Writing OV9782 Sleep_Config size-10

OV9782 Sleep_Config--

AplnStop:SMState = 0x2

AplnStrt:SMState = 0x2

Thanks,

Greg

0 Likes

Hello Greg,

Does that mean that no data are received on the mipi lanes?

If the producer events are not seen that means the data is not being written to the DMA buffer. It seems that either the sensor is not giving the output or if the sensor s giving out the output the sensor settings and the CX3 MIPI receiver settings are not in sync.

Yes, you can try probing the MIPI clock and data lane and check if the sensor outputs the clock and data. Please check the MIPI clock frequency

However I can see an AplnStr:SmState=0x2, is that an issue?

>> This shows that the GPIF state machine is in state 0x02 which is the wait state. The GPIF state machine transitions to the next state only when the FV signal s high that is when frame data is being transmitted.

As AplnStop:SMState = 0x2 i.e the state doesn't transition to another sate which means that frame data is not being transferred.

When the resolution is changed on the fly, CX3_USB_UVC_SET_CUR_REQ request is sent by the host to set the new resolution by configuring the sensor.

From your firmware

   /* Set Commit Control and Start Streaming*/

        if (wValue == CX3_APP_VS_COMMIT_CONTROL)      // SET_CUR COMMIT request

        {

            CyCx3AppImageSensorSetVideoResolution (glCommitCtrl[3]);      // Set the sensor with new settings

            if (glIsApplnActive)

            {

                CyU3PDebugPrint (4, "\n\rUSB Setup CB:Call AppSTOP1");

                CyCx3AppStop();                                      // If the streaming is going on we need to stop the streaming with old resolution and restart the resolution with new resolution after resetting the DMA pipe

            }

            CyCx3AppStart();                                       // started streaming with new resolution

        }

Regards,

Rashi

Regards,
Rashi
0 Likes

Hello Rashi!

Thanks again for you help and apologise for the delayed answer!

I've removed the correct answer since the problem isn't solved yet.
I was only able to probe the Mipi signals today.

I'm guessing that my oscilloscope isn't good enough to get any correct description of the signals and or the clock.
However, from what I can see with the signal probed, there is definitely data sent through the mipi routes.
I'm guessing a 33 ms period which should correspond to my 1280x800 | 30 FPS frame format. With signals sent for a period of around 8.33ms. Which correspond to a maximum of 120 fps for that format with a 2 mipi lanes interface.

122749734_1221650618228042_4127355746942134154_n.jpg

I'm also seeing the corresponding changes while swapping to the 640x400 | 30 FPS format.

122669202_988741824964285_4708023288598232319_n.jpg

122532759_1254988394862819_562992160044303891_n.jpg

122678536_796954347531929_3842089393540946176_n.jpg

The MIPI-CP and MIPI-CN lanes are giving a 214mV visually continuous signal, nothing better can be obtain with the 110MHz of my oscilloscope.

122683797_350258859640854_1870097718184763510_n.jpg

But if I refer to the signals above, I'm guessing that the clock is working fine, correct?

Following your comments, it seems that the configuration of the sensor and the CX3 are not matching then.

Any hints on what I should modify/work on?

0 Likes

Hello Rashi,

Since yesterday I spent a bit of time trying to figure out why. I wasn't receiving the PROD and CONS Event.
I found out that I didn't follow the Continuous clock mode process.
I did it a minute ago and I'm now getting the events as expected!

No images yet..

EDIT1:
Here is a set of logs, (but no images received this time):

Mipicsi Set PHY Delay status = 0x0

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0?

Writing OV9782 800p configuration done

crcErrCnt:0, ctlErrCnt:0, eidErrCnt:0, frmErrCnt: 0?

USB Setup CB:Call App start is called

AplnStrt:SMState = 0x2

Writing OV9782 Start_Config size-2

OV9782 Start_Config--

AppDmaCallBack CONS EVENT: glDmaDone:2 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:4 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:6 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:7 | glHitFV:0

ew Event AppThreadEntry

Frame timed out

ÿÿ

APP STOP

AppDmaCallBack CONS EVENT: glDmaDone:8 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:7 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:6 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:5 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:4 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:2 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:1 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:0 | glHitFV:0

Writing OV9782 Sleep_Config size-10

OV9782 Sleep_Config--

AplnStop:SMState = 0x7

AplnStrt:SMState = 0x1

Writing OV9782 Start_Config size-2

OV9782 Start_Config--

ew Event AppThreadEntry

Frame timed out

ÿÿ

APP STOP

Writing OV9782 Sleep_Config size-10

OV9782 Sleep_Config--

AplnStop:SMState = 0x1

AplnStrt:SMState = 0x1

Writing OV9782 Start_Config size-2

OV9782 Start_Config--

EDIT2:

The below logs are spam when the 640x400 resolution is used.
Images are apparently received but the usual cam apps (eCamviewer for example) are not showing any FPS.

Any reason for that?

AppDmaCallBack CONS EVENT: glDmaDone:6 | glHitFV:1

AppDmaCallBack CONS EVENT: glDmaDone:5 | glHitFV:1

AppDmaCallBack CONS EVENT: glDmaDone:4 | glHitFV:1

AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:1

AppDmaCallBack CONS EVENT: glDmaDone:2 | glHitFV:1

AppDmaCallBack CONS EVENT: glDmaDone:1 | glHitFV:1

AppDmaCallBack CONS EVENT: glDmaDone:0 | glHitFV:1

AppDmaCallBack CONS EVENT: glDmaDone:1 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:2 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:3 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:4 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:5 | glHitFV:0

AppDmaCallBack CONS EVENT: glDmaDone:5 | glHitFV:0

Size of frame is 320000


Have a nice day!

Greg

0 Likes
lock attach
Attachments are accessible only for community members.

Hello Greg,

Thank you for the update.

For the first case, I can see the Frame timed out error which occurs when the video frame is not received before the the timeout value expires. You try increasing the timer's timeout value such that that the Timer value (TIMER_PERIOD - Maximum frame transfer time in milli-seconds) > V_total  period

For second case, please find the attached file which calculates and prints the FPS (fps) and other parameters like prod, cons and frame size. Please refer to this and add the same debug prints to check if the fps coming from the sensor is as expected. Please share the UART debug prints for us to check

if (Printflag == 1)

{

/*For video streaming application of higher FPS refrain from using this debug print or try to reduce the print information*/

CyU3PDebugPrint(4,"\n\rProd = %d Cons = %d  Prtl_Sz = %d Frm_Cnt = %d Frm_Sz = %d B", TxCountflag, RxCountflag, PartialBufSize, FrameCount, ((TxCountflag*CX3_UVC_DATA_BUF_SIZE)+PartialBufSize));

Printflag = 0;

if (fpsflag == 1)

{

fps = 30000/(time1 -time0); //FPS calculate using time difference for 30 frames

CyU3PDebugPrint(4,"\n\rTimeDiff = %d ms FPS = %d", (time1 -time0), fps);

fpsflag = 0;

}

Please let me know if any queries on this

Regards,

Rashi

Regards,
Rashi
0 Likes
GrAU_4638336
Level 4
Level 4
25 replies posted 10 replies posted 10 sign-ins

Hi Rashi,

Thanks for your feedback!


Is the file missing in your message or is it just the lines of code below?
If so, where should I trigger fpsflag=1 ? and time0|1?

Thanks for your help,

Greg

0 Likes
lock attach
Attachments are accessible only for community members.

Hello Greg,

Please find the attached file(cycx3_uvc.c) which has the declaration and definition of these variables. these variables are triggered in the DMA callback (CyCx3UvcAppDmaCallback)

Please note that the FPS counted in the attached code is for 30 fps. Please modify the code snippet as per the settings of the fps in your application

- fpsflag ad time1

fpsflag.PNG

time0.PNG

Please search for  CyCx3UvcAppDmaCallback in the attached file.

Regards,

Rashi

Regards,
Rashi
0 Likes

Hi Rashi,

Sorry I missed the attached file in the first message.

EDIT: I've put the wrong resolution logs.
I did the changes and here is the output of it:

prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 511 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 512 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 513 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 514 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 515 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 516 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 517 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 518 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 519 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 520 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 521 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 522 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 523 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 524 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 525 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 526 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 527 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 528 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 529 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 530 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 531 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 532 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 533 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 534 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 535 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 536 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 537 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 538 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 539 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 540 Frm_Sz = 320000 B

TimeDiff = 1004 ms FPS = 29 C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 541 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 542 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 543 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 544 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 545 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 546 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 547 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 548 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 549 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 550 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 551 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 552 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 553 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 554 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 555 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 556 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 557 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 558 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 559 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 560 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 561 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 562 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 563 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 564 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 565 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 566 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 567 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 568 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 569 Frm_Sz = 320000 B C

Prod = 13 Cons = 7  Prtl_Sz = 720 Frm_Cnt = 570 Frm_Sz = 320000 B

TimeDiff = 1005 ms FPS = 29 C

It looks like the FPS are as expected.

So what could be the reason why my host is not receiving any images. I can't find anything in here saying that the USB transfer hasn't been completed.

Is it because my sensor is streaming RAW10 images? I did the changed to remove the 6 padded 0 as explained in:

Maybe I missed something? Also I'm guessing that Prod and Cons should be equal?

Thanks again,

Greg

0 Likes

Hello Greg,

From the debug prints, it is seen that the frame size is not constant. This is the reason the frames are not seen on the host application

It is expected that the frame size received from the sensor is constant. Please let me know if the pixels are being packed or padded with non image data (zeros) Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387

Please let me know the changes that were done for continuous clock process. Are changes done as per Q13 of this KBA CX3 Hardware: Frequently Asked Questions - KBA91295

Please let me know if the custom board has the test point for probing the Hsync, Vsync and PCLK. Please refer to Q10of the above KBA

As per your previous reply "However FSIN/VSYNC of image sensor is linked to GPIO23 if this can help.." Please probe this pin for the VSYNC to check if the the FPS is as expected

Also, enable the MIPI errors in the firmware. Please refer to the same file shared earlier for the same

CyU3PMipicsiGetErrors( CyTrue, &errCnts);

CyU3PDebugPrint(4,"\n\r%d %d %d %d %d %d %d %d %d",errCnts.crcErrCnt,errCnts.ctlErrCnt, errCnts.eidErrCnt, errCnts.frmErrCnt, errCnts.mdlErrCnt, errCnts.recSyncErrCnt, errCnts.recrErrCnt, errCnts.unrSyncErrCnt, errCnts.unrcErrCnt );

Regards,

Rashi

Regards,
Rashi
0 Likes

Rashi,

Sorry I just Edited my message, I was using the wrong resolution which hasn't been set correctly.

Could have a look at the new logs? Thanks.

I'm going to go through your message anyway

0 Likes

Frames size seems consistent now.

It is expected that the frame size received from the sensor is constant. Please let me know if the pixels are being packed or padded with non image data (zeros) Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387

Yes, I have been using Example 2 and made the necessary adjustement.

Please let me know the changes that were done for continuous clock process. Are changes done as per Q13 of this KBA CX3 Hardware: Frequently Asked Questions - KBA91295

Yes I have done that too.

Please let me know if the custom board has the test point for probing the Hsync, Vsync and PCLK. Please refer to Q10of the above KBA

As per your previous reply "However FSIN/VSYNC of image sensor is linked to GPIO23 if this can help.." Please probe this pin for the VSYNC to check if the the FPS is as expected

Also, enable the MIPI errors in the firmware. Please refer to the same file shared earlier for the same

CyU3PMipicsiGetErrors( CyTrue, &errCnts);

CyU3PDebugPrint(4,"\n\r%d %d %d %d %d %d %d %d %d",errCnts.crcErrCnt,errCnts.ctlErrCnt, errCnts.eidErrCnt, errCnts.frmErrCnt, errCnts.mdlErrCnt, errCnts.recSyncErrCnt, errCnts.recrErrCnt, errCnts.unrSyncErrCnt, errCnts.unrcErrCnt );



Following the above new logs, is this still needed?

0 Likes

Hello Greg,

From the new logs, the frame size is constant.

So what could be the reason why my host is not receiving any images. I can't find anything in here saying that the USB transfer hasn't been completed.

>> Please let me know the video frame size  to calculate the frame size. We need to check if the frame size received in the firmware (seen in the logs) is same as h_active*v_active*10/8(RAW 10)

Maybe I missed something? Also I'm guessing that Prod and Cons should be equal?

>> Yes, it is expected PROD and CONS to be equal. Please let me know if CyU3PDmaMultiChannelCommitBuffer API fails. You can check this by setting an event in the CyCx3AppDmaCallback i.e. when this API fails and in  CyCx3AppThread_Entry, CyU3PEventGet can be used to handle this event. Please refer the previous file for reference.

Please share the the probe control structure (UVC Probe Control Setting) for this resolution and the DMA buffer size used in your application

Probe control structure would be something like uint8_t const glVga60ProbeCtrl[CX3_APP_MAX_PROBE_SETTING] (example)

Regards,

Rashi

Regards,
Rashi
0 Likes

Hello Rashi,

Thanks for your message!

Please let me know the video frame size  to calculate the frame size. We need to check if the frame size received in the firmware (seen in the logs) is same as h_active*v_active*10/8(RAW 10)

Frame size expected is 640*400 so the frame size given appears correct. 320000

Yes, it is expected PROD and CONS to be equal. Please let me know if CyU3PDmaMultiChannelCommitBuffer API fails. You can check this by setting an event in the CyCx3AppDmaCallback i.e. when this API fails and in  CyCx3AppThread_Entry, CyU3PEventGet can be used to handle this event. Please refer the previous file for reference.

I've put a "Frame timed out" print in the case of this Event is called. It doesn't appear to be called.

Also I've noticed this in your example:

if(dmaBuffer.count < CX3_UVC_DATA_BUF_SIZE)

            {

#ifdef UVC_APPLICATION

                CyCx3UvcAppAddHeader ((dmaBuffer.buffer - CX3_UVC_PROD_HEADER), CX3_UVC_HEADER_EOF);

#endif

                glHitFV = CyTrue;#ifdef PRINT_FRAME_INFO

                FrameCount++;

                PartialBufSize = dmaBuffer.count;

                RxCountflag = RxCount;

                TxCountflag = TxCount;

                Printflag = 1;                if ((FrameCount % 30) == 0)

                {

                 time1 = CyU3PGetTime();

                 fpsflag = 1;

                 gettimeflag = 0;

                }#endif

            }

            else

            {

#ifdef UVC_APPLICATION

                CyCx3UvcAppAddHeader ((dmaBuffer.buffer - CX3_UVC_PROD_HEADER), CX3_UVC_HEADER_FRAME);

#endif

            }            /* Commit Buffer to USB*/

#ifdef UVC_APPLICATION

            status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count + 12), 0);

#else

   status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count), 0);

#endif

            if (status != CY_U3P_SUCCESS)

            {

                   CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT,CYU3P_EVENT_OR);

                   break;

            }

Should I be using the UVC application if statement or not, I'm not too sure anymore.

Please share the the probe control structure (UVC Probe Control Setting) for this resolution and the DMA buffer size used in your application

Probe control structure would be something like uint8_t const glVga60ProbeCtrl[CX3_APP_MAX_PROBE_SETTING] (example)

UVC Probe Control Settings

/* UVC Probe Control Setting - 400p*/

uint8_t const gl400pProbeCtrl_HS[CX3_APP_MAX_PROBE_SETTING] = {

    0x00, 0x00,                         /* bmHint : No fixed parameters */

    0x01,                               /* Use 1st Video format index */

    0x01,                               /* Use 1st Video frame index */

    0x15,0x16,0x05,0x00,             /* Desired frame interval in 100ns = (1/30)x10^7 */

    0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

    0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

    0x00, 0x00,                         /* Compression quality control */

    0x00, 0x00,                         /* Window size for average bit rate */

    0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

    0x00,0xd0,0x07,0x00,             /* Max video frame size in bytes = 640 x 400 x 2 */

    0x00, 0x90, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 36KB */

    0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

    0x00,                               /* Framing Information - Ignored for uncompressed format*/

    0x00,                               /* Preferred payload format version */

    0x00,                               /* Minimum payload format version */

    0x00                                /* Maximum payload format version */

};


Function is:

extern const uint8_t gl400pProbeCtrl_HS[CX3_APP_MAX_PROBE_SETTING];

DMA buffer size

/* CX3 Application Buffer Parameters*/

#define CX3_APP_DATA_BUF_SIZE                   (0x5FF0) /* DMA Buffer Data Size Used: 12272 Bytes*/

Hopefully this is helpfull for you.

Thanks in advance,

Greg

0 Likes

Hello Greg,

Frame size expected is 640*400 so the frame size given appears correct. 320000

>> Thank you for confirming

I've put a "Frame timed out" print in the case of this Event is called. It doesn't appear to be called.

>> Please do not use CyU3PDebugPrint inside the DMA callback. Instead you can use a global variable to track the failure of the CyU3PDmaMultiChannelCommitBuffer or use the SET/GET event API's as done in the shared cycx3_uvc.c file. Please let me know if

CyU3PDmaMultiChannelCommitBuffer fails. If yes, please share the return value of the API.

  1. status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count), 0); 
  2. #endif 
  3.             if (status != CY_U3P_SUCCESS) 
  4.             { 
  5.                    CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT,CYU3P_EVENT_OR); 
  6.                    break
  7.             } 

Should I be using the UVC application if statement or not, I'm not too sure anymore.

>> Please let me know which host application are you using for viewing the video stream and if your application requires to be a UVC compliant device then the macro UVC_APPLICATION needs to be enabled. Please share the DMA callback function from your firmware so that I can understand.

Regards,

Rashi

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hello Rashi,

>> Please do not use CyU3PDebugPrint inside the DMA callback. Instead you can use a global variable to track the failure of the CyU3PDmaMultiChannelCommitBuffer or use the SET/GET event API's as done in the shared cycx3_uvc.c file. Please let me know if

CyU3PDmaMultiChannelCommitBuffer fails. If yes, please share the return value of the API.

  1. status = CyU3PDmaMultiChannelCommitBuffer (chHandle, (dmaBuffer.count), 0); 
  2. #endif 
  3.             if (status != CY_U3P_SUCCESS) 
  4.             { 
  5.                    CyU3PEventSet(&glCx3Event, CX3_DMA_RESET_EVENT,CYU3P_EVENT_OR); 
  6.                    break
  7.             } 

Sorry what I explained wasn't clear, I did the above and put the "frame timed out" in the for(;;) loop. So when the event is triggered the message frame timed out is printed. This isn't the case so the status of CyU3PDmaMultiChannelCommitBuffer is always

>> Please let me know which host application are you using for viewing the video stream and if your application requires to be a UVC compliant device then the macro UVC_APPLICATION needs to be enabled. Please share the DMA callback function from your firmware so that I can understand.

I've tried the native windows camera app, eCamviewer and I also wrote a small script in python using opencv. None can grab a frame and display. However there is definitely packets sent through the USB port since WireShark is going crazy when the stream is open.
I'm not good enough to understand more of it in wireShark.

For DMA callback function, see the file attached.

Thanks again for your help.

0 Likes

Hello Greg,

If the Commit buffer API never fails, please share the Wireshark traces and use eCamView for streaming the video.

Please start capturing the USB traces when the device is plugged in, then program the CX3 with the .img file, and then start streaming by opening the eCamView

Regards,

Rashi

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hi Rashi,

I've attached the traces for the CX3 before upload of the firmware and the traces of the camera from flashing firmware to having the stream running.

I don't really know how to keep traces when the device is going from CX3 boot loader to actual USB capture device.

EDIT: The packages sent are only linked to the debug logs through the serial emulator. Nothing to do with images. If the stream is opened and the serial debugger is never called, no image packages are sent to the USB port...

I let you have a look at them, I'm not too sure what to look at here.

Thanks,

Greg

0 Likes

Hello Greg,

From the traces, I found that USB packets are sent to the host. (24599 URB_BULK_IN). And it can be seen that 13 such packets are sent 13 - 24599 (24572 (24560 of data +12 bytes of UVC header) + 27 bytes of Wireshark header) packets and 1 partial packet of 759 (732 (720 bytes of data + 12 bytes of UVC header) + 27 bytes Wireshark header). So the full-frame 320000  is seen on the USB bus.

As the RAW format is not supported by UVC  specification, the RAW video is transported as YUY2 to the UVC driver.  It is expected that the video may not be seen or seen pinkish/greenish as UVC players will sample the data as per YUY2 format and not RAW 10. A custom host application needs to be developed for viewing the RAW video data

Can you please show the snippet of the eCamView where the video is to be seen?

Is it possible to use a different PC and try streaming? You can also try using AmCap or MPC-HC Player (UVC player) to view the video.

EDIT: The packages sent are only linked to the debug logs through the serial emulator. Nothing to do with images. If the stream is opened and the serial debugger is never called, no image packages are sent to the USB port...

>> I didn't understand this. Can you please explain this?

Also, from the traces, I see that the device enumerates with multiple interface UVC and CDC.

It seems that CDC interfaces used for debug prints. Is my understanding correct?

Regards,

Rashi

Regards,
Rashi
0 Likes

Hello Rashi,

Thanks for your feedback.

Yes I understand that, I'm expecting to see the weird behavior regarding the frames. But at least I would like to have frames

Here is the result obtained with eCamView

pastedImage_1.png

I also tried Amcap and this is the result:

pastedImage_0.png

I can't see any images going through either? Is there a setting that I missed to display the RAW10 format?

I'm trying now on another computer (Mac under Windows 10 bootcamp). Let's see if I'm more lucky.

>> I didn't understand this. Can you please explain this?

Also, from the traces, I see that the device enumerates with multiple interface UVC and CDC.

It seems that CDC interfaces used for debug prints. Is my understanding correct?

This is absolutely correct.

I was trying to say that if the serial debugger interface is not open prior to open the camera feed, then Wireshark do not display the (24599 URB_BULK_IN) packets as expected. Which seems weird in my opinion. This would say that the streaming can't start without the debug interface opened.

0 Likes

Rashi,

Exact same behavior on my Windows 10 bootcamp and eCamviewer.

By the way is there a way to export the wireShark packages and preview the image? So at least I could confirm that the data are correct.

Thanks,

Greg

0 Likes

Hello Greg,

was trying to say that if the serial debugger interface is not open prior to open the camera feed, then Wireshark do not display the (24599 URB_BULK_IN) packets as expected

>> This is not expected.

Can you share the firmware files for us to check. You can remove the sensor settings from the firmware and share the complete firmware.

Regards,

Rashi

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Rashi,

See the folder attached. Hopefully you will find something interresting!

EDIT: Please only consider High speed interface and 640*400 format, it's the only one I'm testing on at the moment.

Thanks again for you help,

Greg

0 Likes

Hello Greg,

From the firmware, I found that the descriptors and the probe control structure are not changed when the RAW 10 pixels are packed

Please refer Example 2 of this KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387

The frame size in your case is  640 * 400 *10 /8 = 320000 bytes but currently, in the descriptors and probe control structure, the frame size is  (640*400*2 bytes ) more than the actual frame size (640*400*10/8 bytes) transferred by the sensor. So the host will expect more pixels in the frame.

So, the actual frame size needs to be mentioned in the probe control structure as well as in descriptors

a. Width in pixel                  : 640

b. Height in pixel                : 400

c. Bits per pixel                  : 10

d. Frames per second       : 30

e. Frame size                      : 640 x 400 x 10 bits =  2560000 bits

f. Bit rate                             : 640 x 400 x 10 x 30 bits per second

Re-writing the above calculations:

a. Width in pixel                  :  400     // Adjusted the width in pixel to match the frame size to 320000 bytes

b. Height in pixel                :   400

c. Bits per pixel                  :  16  //This is done just to transfer RAW10 format as YUY2 format (which supports 16 bites per pixel)

d. Frames per second       :  30

e. Frame size                      :  400 x 400 x 16 bits = 2560000 bits   

f. Bit rate                             :  400 x 400 x 16 x 30 bits per second

So, now the new frame size (after adjusting the width in pixel and bits/pixel to transfer as YUY2 format) is 400*400*2 (bytes/pixel) =  320000 bytes

Similar changes need to be done for other resolution also.

Please refer to example 2 of the KBA Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387 and make the necessary changes in the probe control structure gl400pProbeCtrl_HS/gl400pProbeCtrl and gl800pProbeCtrl_HS/gl800pProbeCtrl  and build the firmware again and try streaming.

Please let me know if this works

Regards,

Rashi

Regards,
Rashi
0 Likes

Hello Rashi,

Thanks for your feedback on this firmware.

As you probably noticed with the "correct answer" ticked, your solution solved my problem and I was able to acquire images.

Here is the result:

pastedImage_0.png

Now I need to reformat this stream.

Thanks a lot for your help on this project! It was super exciting to build my first camera and I'm looking forward to see the progress I will make on the firmware/SDK.

Have a great day!

Greg

0 Likes