cancel
Showing results for 
Search instead for 
Did you mean: 

USB Superspeed Peripherals

ZaTu_4258396
New Contributor

Good afternoon,

 

I am currently working on getting a new camera working on the same Cypress CX3 chip. We have created a very similar board to the one we have been using for the IMX241 with changes only to the LED illumination system and the camera connection to accommodate the larger packaged sensor, the IMX412. 

 

This IMX412 is a 4056x3050 sensor. I have set this sensor up to output its central 5MP at 2592x1944 to match our previous camera’s setup. This streams at 43.4 fps with the current settings I have in place.

 

My current issue is that the device tends to reset quite often. And by reset I mean the following (from UART output):

 

EVENTFLAG Timer Reset Event, IsCameraSupposedtoBeRunning: 1, FrameIndexToSet: 2, FrameIndex: 2

APP STOP

 

This occurs just about 95% of the time within 60 seconds of streaming. My goal is to understand and mitigate this response as best as possible.

 

 

My understanding is that this occurs when the host has not pulled the data in the expected amount of time before the device is ready to output its’ next image. I am currently using Direct Show for my image capture. This is set up through a callback method and not doing any processing what-so-ever and just pulling the images into my program and releasing them after confirming a viable image in order to output FPS values.

 

My current Descriptor Settings:

ZaTu_4258396_7-1627318895583.png

ZaTu_4258396_8-1627318930186.png

My Hard set values:

ZaTu_4258396_9-1627318947571.png

 

My CX3 Clock:

ZaTu_4258396_10-1627318959714.png

My Camera Clock:

ZaTu_4258396_11-1627318971777.png

 

Continued:

 

ZaTu_4258396_12-1627318981484.png

 

 

Camera Lines: **I have not been able to effectively reduce the Total pixels per line without causing a “step response” seen below the values.** Different problem but need to solve after this. Will only get us a max of 3 more fps since the device cannot handle more than 47 fps from this camera.

 

 

ZaTu_4258396_13-1627318998873.png

 

 

 

 

Please let me know if I can provide any more additional preliminary information to aid your understanding of this current issue. Thank you for your time.

0 Likes
1 Solution
JayakrishnaT_76
Moderator
Moderator

Hello,

Thank you for the confirmation. It looks like the host is not pulling the data fast enough from CX3. As mentioned before, in your code, the timer is started when you get a CY_U3P_DMA_CB_PROD_EVENT in the DMA callback function and the timer is stopped when you get a CY_U3P_DMA_CB_CONS_EVENT in the DMA callback function. After stopping, the timer is reloaded with the timer period again for further use. The CY_U3P_DMA_CB_PROD_EVENT is generated when CX3 receives a DMA buffer from the producer and CY_U3P_DMA_CB_CONS_EVENT is generated when the consumer empties a DMA buffer. If the consumer is not emptying the buffer, then it can cause the timer to overflow. This was again verified by driving the GPIO in our experiment.

Can you please refer to the following example (for CX3) in FX3 SDK as a reference to understand how the timer is started and stopped inside the DMA callback function:

C:\Program Files (x86)\Cypress\EZ-USB FX3 SDK\1.3\firmware\cx3_examples\cycx3_uvc_ov5640

You can use the same implementation in your firmware too.

In addition to this, based on the following print, 

Prod = 171 Cons = 170 Prtl_Sz = 3024 Frm_Cnt = 4416 Frm_Sz = 6298560 B

and based on the fact that you are trying to stream 2592x1944, I think that the bits per pixel is 10bits. This is because the full frame size printed in the UART logs is 6298560 bytes. The frame size is calculated as follows:

frame size (in bytes) = h-active x v-active x bytes per pixel

This calculation would give bits per pixel as 10. Please correct me if my understanding is wrong.

If my understanding is correct, then I feel that the delay from the host occurs after the transmission of one complete frame. This is because soon after the following print:

Prod = 171 Cons = 170 Prtl_Sz = 3024 Frm_Cnt = 4416 Frm_Sz = 6298560 B

you can find that the timer is overflowing. I strongly suspect that the issue is related to the following statement in your original description as the delay is from the host side:

"I am currently using Direct Show for my image capture. This is set up through a callback method and not doing any processing what-so-ever and just pulling the images into my program and releasing them after confirming a viable image in order to output FPS values."

Best Regards,
Jayakrishna

View solution in original post

0 Likes
5 Replies
JayakrishnaT_76
Moderator
Moderator

Hello,

Please let me know the following details so that I can understand the issue better:

1. Are you able to preview the video stream using the host application?

2. Is the device able to recover by itself when this issue is seen?

3. What made the following print to occur in the UART debug logs:

"EVENTFLAG Timer Reset Event, IsCameraSupposedtoBeRunning: 1, FrameIndexToSet: 2, FrameIndex: 2"

Is it due to the failure of the API CyU3PDmaMultiChannelCommitBuffer() or is it due to the execution of the Timer callback function (UvcAppProgressTimer). Please let me know how this was confirmed.

4. Please share the snapshot of CX3 receiver configuration tab of MIPI configuration utility for us to check.

Best Regards,
Jayakrishna
0 Likes
ZaTu_4258396
New Contributor

Jayakrishna, 

 

Thank you for taking the time to look over this information.

1. Yes I am using Direct Show on a windows machine to capture images and I am outputting them to a viewing window on a per frame basis.

2. Yes, the device catches the timeout error, resets, and continues streaming. This reset sets all default configurations back to the camera. I could easily make sure that any changed configurations stay the same during this timeout event but I want to root cause the timeout event and not make a workaround.

3. This is due to the execution of the Timer callback function as mentioned.

4. I did not use the configuration tab at all to set this up. I have NEVER had success using this tool to actually output a viable solution for any configuration I have created or attempted to create. I did however fill out the applicable fields based on our cameras configuration.

ZaTu_4258396_0-1627400509612.png

-Zach

 

0 Likes
JayakrishnaT_76
Moderator
Moderator

Hello Zach,

Please let me know how did you confirm that the following prints in the UART debug logs came as part of the execution of timer callback function?

"EVENTFLAG Timer Reset Event, IsCameraSupposedtoBeRunning: 1, FrameIndexToSet: 2, FrameIndex: 2"

As you might be knowing, the Timer callback function will only be invoked when the timer overflows. In your code, the timer is started when you get a CY_U3P_DMA_CB_PROD_EVENT in the DMA callback function and the timer is stopped when you get a CY_U3P_DMA_CB_CONS_EVENT in the DMA callback function. After stopping, the timer is reloaded with the timer period again for further use. 

This could be the reason for the timer to overflow. This can be confirmed by driving a GPIO HIGH when the timer is started and driving it LOW when the timer is stopped. After this, you can check the time duration of HIGH pulse. If the GPIO remain HIGH for atleast 500ms, then it can trigger the timer callback function.

You can also use the following example (for CX3) in FX3 SDK as a reference to understand how the timer is started and stopped inside the DMA callback function:

C:\Program Files (x86)\Cypress\EZ-USB FX3 SDK\1.3\firmware\cx3_examples\cycx3_uvc_ov5640

Best Regards,
Jayakrishna
0 Likes
ZaTu_4258396
New Contributor

Jayakrishna,

Thank you for the suggestions. I went ahead and did exactly what you said. This actually caused the issue to happen more often. Actually back to back sometimes. Any time that it did happen the GPIO would extend out past 500 and cause a reset.

I wanted to gain a little more information during the trials and I had my UART connection to the device during this trial. I was able to notice that every time we had a reset, it would dump the prod/cons values and it was constantly at a 171 PROD and 170 CONS. Is this expected? 

My gut also tells me that there are some configurations that are specific to the 241 board that are implemented on this 412 board, like the interrupts for external power, CC lines, and LED manipulation that have not been removed prior to this testing. So I will be removing all of these additional configurations moving forward and awaiting your further instruction. Thank you for your help.

3 0 0 3 0 0 0 0 0
This was the MIPI error output. I usually dont hold that much significance to this output as it has not been fruitful for identifying anything in the past. However, it could be pointing towards something important that I do not see.

 

 

Prod = 171 Cons = 170 Prtl_Sz = 3024 Frm_Cnt = 4413 Frm_Sz = 6298560 B
4 0 0 4 0 0 0 0 0
EVENTFLAG Timer Reset Event, IsCameraSupposedtoBeRunning: 1, FrameIndexToSet: 1
Camera_Power_Down

AplnStop:SMState = 0xA
Inside esSetCameraResolution, FrameIndex: 1, SensorMode: 1
UpdatingSensorMode and Image Sensor Config, FrameIndex: 1, SensorMode: 1
imx_Config FULL RESOLUTION 30 FPS 1
CamRegSize = 128

APPINSTART
Camera_Power_Upe = 0x1

CC1: 0, CC2: 0, EXT_PWR: 1
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
Prod = 171 Cons = 170 Prtl_Sz = 3024 Frm_Cnt = 4416 Frm_Sz = 6298560 B
EVENTFLAG Timer Reset Event, IsCameraSupposedtoBeRunning: 1, FrameIndexToSet: 1
Camera_Power_Down

AplnStop:SMState = 0x5
Inside esSetCameraResolution, FrameIndex: 1, SensorMode: 1
UpdatingSensorMode and Image Sensor Config, FrameIndex: 1, SensorMode: 1
imx_Config FULL RESOLUTION 30 FPS 1
CamRegSize = 128

APPINSTART
Camera_Power_Upe = 0x1

CC1: 0, CC2: 0, EXT_PWR: 1
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
max power setting 132
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
AplnStrt:SMState = 0x1
Prod = 171 Cons = 170 Prtl_Sz = 3024 Frm_Cnt = 4445 Frm_Sz = 6298560 B
TimeDiff = -1 ms FPS = 0
EVENTFLAG Timer Reset Event, IsCameraSupposedtoBeRunning: 1, FrameIndexToSet: 1
Camera_Power_Down

AplnStop:SMState = 0xA
Inside esSetCameraResolution, FrameIndex: 1, SensorMode: 1
UpdatingSensorMode and Image Sensor Config, FrameIndex: 1, SensorMode: 1
imx_Config FULL RESOLUTION 30 FPS 1
CamRegSize = 128

0 Likes
JayakrishnaT_76
Moderator
Moderator

Hello,

Thank you for the confirmation. It looks like the host is not pulling the data fast enough from CX3. As mentioned before, in your code, the timer is started when you get a CY_U3P_DMA_CB_PROD_EVENT in the DMA callback function and the timer is stopped when you get a CY_U3P_DMA_CB_CONS_EVENT in the DMA callback function. After stopping, the timer is reloaded with the timer period again for further use. The CY_U3P_DMA_CB_PROD_EVENT is generated when CX3 receives a DMA buffer from the producer and CY_U3P_DMA_CB_CONS_EVENT is generated when the consumer empties a DMA buffer. If the consumer is not emptying the buffer, then it can cause the timer to overflow. This was again verified by driving the GPIO in our experiment.

Can you please refer to the following example (for CX3) in FX3 SDK as a reference to understand how the timer is started and stopped inside the DMA callback function:

C:\Program Files (x86)\Cypress\EZ-USB FX3 SDK\1.3\firmware\cx3_examples\cycx3_uvc_ov5640

You can use the same implementation in your firmware too.

In addition to this, based on the following print, 

Prod = 171 Cons = 170 Prtl_Sz = 3024 Frm_Cnt = 4416 Frm_Sz = 6298560 B

and based on the fact that you are trying to stream 2592x1944, I think that the bits per pixel is 10bits. This is because the full frame size printed in the UART logs is 6298560 bytes. The frame size is calculated as follows:

frame size (in bytes) = h-active x v-active x bytes per pixel

This calculation would give bits per pixel as 10. Please correct me if my understanding is wrong.

If my understanding is correct, then I feel that the delay from the host occurs after the transmission of one complete frame. This is because soon after the following print:

Prod = 171 Cons = 170 Prtl_Sz = 3024 Frm_Cnt = 4416 Frm_Sz = 6298560 B

you can find that the timer is overflowing. I strongly suspect that the issue is related to the following statement in your original description as the delay is from the host side:

"I am currently using Direct Show for my image capture. This is set up through a callback method and not doing any processing what-so-ever and just pulling the images into my program and releasing them after confirming a viable image in order to output FPS values."

Best Regards,
Jayakrishna

View solution in original post

0 Likes