How to set CX3 mipi receiver configuration?

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
Philip
Level 3
Level 3
50 sign-ins 25 replies posted 25 sign-ins

Hello,

We're trying to develop a custom module.
However, we're facing several problems.
let me explain our situation first.

Our sensor has EMB data so output data for one frame is as follows.

Philip_3-1622091481490.png

Also, the sensor has 4 types of frames and EMB has its own index so we can recognize the frames.
You can see that the four waveforms are clustered together and It has indexes 0, 1, 2, 3 from the beginning based on the grouped waveform as below the image. (One single waveform means one frame.)

Philip_4-1622093313540.png

We think the problem is that our module is different from a normal RGB sensor.
When we set MIPI Receiver Configuration, there are some obstructions because we have two types of V-blank as below.
So, the first question is how to set V-blank values in the above case.

Philip_7-1622094728419.png

The second question is how to set "CSI Clock" and "Frame Rate".
The sensor specs are
 - MIPI RATE: 800MHz
 - MIPI Lane: 2Lane
 - Frame rate: 30fps 
However, since one final output data is generated in 4 frames, Frame Rate should actually be set to 120 fps.
In this case, when we set CSI Clock to 400 and Frame Rate to 120, the minimum value of CSI Clock will be limited.
How can we handle this?

Philip_0-1622097357948.png

3. We tested by reducing the Frame Rate in question 2.
However, when CSI Clock is set to 400 and Frame Rate is set to 30, continuous frames weren't able to be received.
In this case, we can only get frames with an index of 0 or 2.
What could be the problem?

Philip_1-1622097422839.png

Philip_3-1622097958222.png


The timing of waveforms is below.
A Single Frame: 5.25ms
V-Blank1: 1.5ms
V-Blank2: 7.8ms
A Set of Final Output Frame: 33.33ms

Philip_1-1622090975323.png

Please let me know if there are any other problems.

Regards,
Philip

0 Likes
1 Solution
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Philip,

The datasheet says select ① or ② that suits the grabber board you are using.
On what basis should I choose this?

>> Is it possible to choose both V blanking values as same or a single value?

So, my question is how can we increase the FPS more?

>> I tried the CX3 receiver configuration at my end and couldn't see the error.  Please try the attached configuration and let me know if the error is seen

Finally,  I have one more question, we're wondering if CX3 can support ToF sensor?
If CX3 is not suitable for our sensor, let us know.

>> CX3 Configuration tool supports only one video blanking period (according to MIPI spec) for a video resolution if both the video blanking period can be configured as same then the sensor might be compatible to CX3.

We have seen some customers using TOF sensor with CX3 but we are not sure if they could get the expected outcome

Solved: CX3 with TOF sensor - Cypress Developer Community   Solved: cx3 tof issure - Cypress Developer Community  

Regards,
Rashi

View solution in original post

0 Likes
21 Replies
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Phillip,

When we set MIPI Receiver Configuration, there are some obstructions because we have two types of V-blank as below. So, the first question is how to set V-blank values in the above case.

>> I understand that for a particular resolution there can be single V blanking value. Please let me know what is the other V blanking .

The second question is how to set "CSI Clock" and "Frame Rate".

>> The CSI clock frequency set in the tool should be same as frequency from MIPI transmitter. Frame rate is generally calculated based on the V total period. As in you case the V total is 33.33ms, the frame rate will be 1000/33.33  = 30 FPS

However, when CSI Clock is set to 400 and Frame Rate is set to 30, continuous frames weren't able to be received. In this case, we can only get frames with an index of 0 or 2.
What could be the problem?

>> Do you mean that the frame rate received is half like 15 FPS? Can you please share the UART debug prints? Please confirm if MIPI transmitter outputs CSI clock with 400 MHz . Also let me know how is the V blanking ( lines) value set to 68. 

Regards,
Rashi
0 Likes

Hello Rashi,
Thank you for your response.

When we set MIPI Receiver Configuration, there are some obstructions because we have two types of V-blank as below. So, the first question is how to set V-blank values in the above case.

>> I understand that for a particular resolution there can be single V blanking value. Please let me know what is the other V blanking .

>> Actually, our sensor is a ToF sensor, not a typical RGB sensor.
The sensor has 4 types of frames and a V-blank 1 is between each frame.
There is a V-blank 2 for every 4 types of frames.
In the image below, ① corresponds to "V-blank 1" and ② corresponds to "V-blank 2".

Philip_0-1622423916354.png
The reason why there are two V-Blanks is that the frames are not all the same.
After we acquire the frames, generating a depth image with additional work.
In order to create a depth image, it is necessary to acquire consecutive frames of four types.
The datasheet says select ① or ② that suits the grabber board you are using. (Previously, we missed this part.)
On what basis should I choose this?

 

The second question is how to set "CSI Clock" and "Frame Rate".

>> The CSI clock frequency set in the tool should be same as frequency from MIPI transmitter. Frame rate is generally calculated based on the V total period. As in you case the V total is 33.33ms, the frame rate will be 1000/33.33  = 30 FPS

>> We think Frame Rate would be 120 because a depth image is created by a set of 4 types of frames.
Therefore, In the case of a single frame, fps would be 120 and in the case of a depth image, fps would be 30.
However,  when we set Frame Rate to 120, we cannot set the CSI Clock value because the minimum value of CSI Clock is limited as follows.
So, my question is how can we increase the FPS more?

Philip_1-1622166811438.png

 

However, when CSI Clock is set to 400 and Frame Rate is set to 30, continuous frames weren't able to be received. In this case, we can only get frames with an index of 0 or 2.
What could be the problem?

>> Do you mean that the frame rate received is half like 15 FPS? Can you please share the UART debug prints? Please confirm if MIPI transmitter outputs CSI clock with 400 MHz . Also let me know how is the V blanking ( lines) value set to 68. 

>> Above question, we cannot set a proper CSI Clock and Frame Rate.
Increasing the frame rate limits the CSI clock setting so we set Frame Rate to 30.
In this case, there are no error messages. (The hidden logs are the Sensor Configurations.)
Also, V-blanking values are calculated by the equation in the sensor datasheet.

Philip_3-1622169379009.png

Philip_2-1622169079069.png


Finally,  I have one more question, we're wondering if CX3 can support ToF sensor?
If CX3 is not suitable for our sensor, let us know.

Regards,
Philip

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Philip,

The datasheet says select ① or ② that suits the grabber board you are using.
On what basis should I choose this?

>> Is it possible to choose both V blanking values as same or a single value?

So, my question is how can we increase the FPS more?

>> I tried the CX3 receiver configuration at my end and couldn't see the error.  Please try the attached configuration and let me know if the error is seen

Finally,  I have one more question, we're wondering if CX3 can support ToF sensor?
If CX3 is not suitable for our sensor, let us know.

>> CX3 Configuration tool supports only one video blanking period (according to MIPI spec) for a video resolution if both the video blanking period can be configured as same then the sensor might be compatible to CX3.

We have seen some customers using TOF sensor with CX3 but we are not sure if they could get the expected outcome

Solved: CX3 with TOF sensor - Cypress Developer Community   Solved: cx3 tof issure - Cypress Developer Community  

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hello Rashi,
Thank you for your response.
We'll check your advice.

By the way, we've found something new that appears to be a problem.
There seems to be a problem with the DMA-related part of the firmware source code.
We think the DMA socket doesn't seem to be switching.
Even though we added some logs in CyCx3UvcAppDmaCallback() function, logs didn't appear.

CyU3PDebugPrint(4, "\n\rCX3_START_SCK");
if (glActiveSocket)
{
CyU3PDebugPrint(4, "\n\rCX3_START_SCK1");
CyU3PGpifSMSwitch(CX3_INVALID_GPIF_STATE, CX3_START_SCK1,
CX3_INVALID_GPIF_STATE, ALPHA_CX3_START_SCK1,
CX3_GPIF_SWITCH_TIMEOUT);
}
else
{
CyU3PDebugPrint(4, "\n\rCX3_START_SCK0");
CyU3PGpifSMSwitch(CX3_INVALID_GPIF_STATE, CX3_START_SCK0,
CX3_INVALID_GPIF_STATE, ALPHA_CX3_START_SCK0,
CX3_GPIF_SWITCH_TIMEOUT);
}

Could you check our source code?

Regards,
Philip

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Phillip,

We do not recommend to us CyU3PDebugPrint API in the DMA callback. You can try incrementing a global variable in the DMA callback and print the value in the for loop of thread entry function.

Regards,
Rashi
0 Likes

Hello Rashi,
Thank you for your help.

We checked it but I have a question.
We use two sockets but the frequency of use of the two sockets is different.
Doesn't it matter?

Philip_2-1622706765320.png

Regards,
Philip

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Philip,

When Multichannel DMA channel is used, the DMA Buffers are used in PING PONG manner as explained in section 3.3 of this app note https://www.cypress.com/file/123506/download . If the DMA buffers are not used in ping pong manner, the commit buffer (CyU3PDmaMultiChannelCommitBuffer) API will fail.

The default CX3 projects uses the DMA buffers in Ping Pong manner.

From the traces, I can see that CyU3PDmaMultiChannelCommitBuffer fails, please let us know what changes are done to the default firmware.

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hello Rashi,
Thank you for your help.

We added five things.

First, GPIO settings as below.
When we omit the below code, I2C communication doesn't work.

Philip_0-1623138692376.png

Second, the sensor register set and its related code.
We have attached the project except for the register set.

Third, code for Streaming On/Off as below:

Philip_1-1623138940542.png

Fourth,  GPIF BUS as below:
Philip_0-1623140475862.png

 



Fifth, MIPI Receiver Configuration as below:

Philip_2-1623139284939.png

Any comments or suggestions are welcome.

Regards,
Philip

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Philip,

It seems that you are not using the default OV5640 firmware from the SDK which is based on UVC but instead you are referring firmware from this thread  Solved: FX3 / CX3 Firmware for Streaming RAW Image Data us... - Cypress Developer Community   and have made only the changes mentioned your response . Is my understanding correct?

If yes, please let me know which host application is being used to stream the video. Are you using streamer application from the FX3 SDK.

Regards,
Rashi
0 Likes

Hello Rashi,

Yes, it is.
We use firmware from the above thread and the modifications are the same as those mentioned above.
Also, the host application is a streamer from the FX3 SDK.

Regards,
Philip

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Phillip, 

Please let me know the Packets per Xfer, Xfer per Queue and timeout settings of Streamer used for streaming the video.

Please try higher values of Packets per Xfer and Xfer per Queue to check if the CB failures are reduced. Also confirm that CyU3PDebugprint is not called in the DMA callback.

Can you please capture the USB traces using Wireshark and share the traces?

And also confirm if the failures are seen the very first time the device is plugged to the device and streaming is started by sending vendor command from control center

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hello Rashi,
Thank you for your response.

Please let me know the Packets per Xfer, Xfer per Queue and timeout settings of Streamer used for streaming the video.
>> below image is our settings:
Philip_0-1623218307237.png


Please try higher values of Packets per Xfer and Xfer per Queue to check if the CB failures are reduced. Also, confirm that CyU3PDebugprint is not called in the DMA callback.
>> We tested higher values of Packets per Xfer and Xfer per Queue. However, CyU3PDebugprint is not called in the DMA callback. We cannot see any error messages on the terminal. If no message is displayed on the terminal in the "Streaming On" state, should the Streamer be considered to be working fine? We suspect there may be a DMA-related issue with the firmware source code.

Can you please capture the USB traces using Wireshark and share the traces?
>> Please find the attached file. We captured it using the below settings:
 - Packets per Xfer: 32
 - Xfers to Queue: 32
 - Timeout Per Xfer(ms): 1500


And also confirm if the failures are seen the very first time the device is plugged to the device and streaming is started by sending vendor command from control center
>> When sending vendor command from control center, we can see the below messages. (We added DMA and SCK Count logs.) We think there are no failure messages. Is that correct?

Philip_1-1623220164248.png


Regards,
Philip

0 Likes
lock attach
Attachments are accessible only for community members.
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Philip,

In the firmware shared earlier, I couldn't find a debug print like " CB Failure" which was seen the UART logs shared earlier. Please let me know if there two different versions of firmware used for the tests.

I have modified the firmware shared earlier to add a debug print " Cb Failure" when CX3_DMA_RESET_EVENT is set. Please build the .img file for the attached firmware and program CX3 with the same. Please let me know if CB failures are seen when higher value of Packets per Xfer (128) and Xfer per Queue (64) are used.

Also, confirm if you are using the default Streamer application from the FX3 SDK and have not modified the streamer application code.

When sending vendor command from control center, we can see the below messages. (We added DMA and SCK Count logs.) We think there are no failure messages. Is that correct?

>> Can you please do the same test with the firmware attached. We need to check what causes the AppStart and Appstop to be called so frequently. In the firmware shared by you, I could see that CyCx3UvcApplnStart is called twice - when vendor command is received and when CX3_DMA_RESET_EVENT is set. From the USB trace, I couldn't find vendor commands, so it is likely that the CX3_DMA_RESET_EVENT is causing  frequent Appstop and Appstart calls.

Regards,
Rashi
0 Likes

Hello Rashi,

In the firmware shared earlier, I couldn't find a debug print like " CB Failure" which was seen the UART logs shared earlier. Please let me know if there two different versions of firmware used for the tests.
>> First of all, Sorry for the confusion. We made a new project from the default firmware to reduce change points. We forgot to add debug print.


I have modified the firmware shared earlier to add a debug print " Cb Failure" when CX3_DMA_RESET_EVENT is set. Please build the .img file for the attached firmware and program CX3 with the same. Please let me know if CB failures are seen when higher value of Packets per Xfer (128) and Xfer per Queue (64) are used.
>> We tested it but we cannot set any errors.

Philip_0-1623303201168.png



Also, confirm if you are using the default Streamer application from the FX3 SDK and have not modified the streamer application code.
>> We've not modified Streamer application.


Can you please do the same test with the firmware attached. We need to check what causes the AppStart and Appstop to be called so frequently. In the firmware shared by you, I could see that CyCx3UvcApplnStart is called twice - when vendor command is received and when CX3_DMA_RESET_EVENT is set. From the USB trace, I couldn't find vendor commands, so it is likely that the CX3_DMA_RESET_EVENT is causing  frequent Appstop and Appstart calls.
>> We tested it using the new version.
1. Use Control Center to send vendor command (0x99) to CX3.
2. As soon as you send the command, you can see the logs for AppStart and Appstop in the terminal.
3. Then press the Start button in the Streamer application.
4. The AppStart and Appstop messages disappear and the Streamer is working properly.

Anything strange above? We need to check why The AppStart and Appstop are called frequently?

Regards,
Philip

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Phillip,

Thank you for the details

Anything strange above? We need to check why The AppStart and Appstop are called frequently?

>> YEs, can you please check from where is Appstart and appstop called. 

As you mentioned that while streaming no errors are seen, which means the DMA over flow condition is solved when the requests from Streamer application are faster. 

Regards,
Rashi
0 Likes

Hello Rashi,

I'd like to ensure several things.
First of all, I think I'd better share my state.
Now, I can see the below logs when sending "Streaming On" vendor command without pushing the start button on ControlCenter.(However, the log is not output after the start button is pressed.)
Philip_0-1623657048908.png

My previous question:
Anything strange above? We need to check why The AppStart and Appstop are called frequently?
Your answer:
>> YEs, can you please check from where is Appstart and appstop called. 

>> This is because CyU3PEventGet function does not return when the Start button is pressed in the streamer. Because of this, no error message is displayed.
1. In this case, there is a possibility that a problem is occurring even if the streamer receives data without error. am I right?
2. If the above could happen, how do I know if an error has occurred? and how to handle it?
3. Is it normal not to return from CyU3PEventGet() function?

Logs are output at 1, 3, and 4 as shown below.
Philip_2-1623659345601.png


Regards,
Philip

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Phillip,

I understand that the logs are from the scenario when streamer application is not started and 0x99 is sent from control center. Is that correct?

 From the snippets shared by you the CB Failure prints will be seen only when CX3_DMA_RESET_EVENT is set. This event will be set if streaming is started i.e. 0x99 is received but  streamer application is not opened and start transfer is not clicked. And when streamer application is started, the CB Failure error should not be seen.

If the CB Failure is not seen when streamer application is working , then the application is working fine.

Regards,
Rashi
0 Likes

Hello Rashi,
Thank you for your response.

I have several additional questions.
Now, the CB Failure is not seen when streamer application is working. It means the application is working fine as you mentioned above. However, I'm not still able to receive the whole data. Below is my MIPI output waveform.
Philip_3-1623719975124.png

In the above image, you can see that the four waveforms are paired. Each of them is a single image and not equal so I need to get successive data to process the next step.
From now on, each waveform in the waveform group will be named image 0 through image 3.

The current problem is that I can only receive images 0 and 2. For this reason, I first doubted MIPI output data so I captured MIPI output signal using a scope but there is nothing strange. Next, I doubted the CX3 output data and the Streamer and verified the above questions. However, you answered me streamer is working fine. Now, I think there is no problem with CX3 output and Streamer.

Now, if there is a problem, I think there may be a problem when sending data from CX3. This is because both input data (MIPI Waveform) and output data (Streamer) are verified.

So I want to check the DMA-related part of the Firmware.
First, I checked socket count as follows:

Philip_1-1623719272303.png

Philip_0-1623719192394.png


1. In the image above, the counts on the two sockets don't match, could this be a problem?

2. If it could be a problem, what should I look into?

3. Currently, I'm using Manual DMA. However, I want to change to Auto DMA.
Simply, I changed the following two parts but it doesn't seem to work.
Which part do I need to check?

Philip_5-1623723752071.png


Regards,
Philip

0 Likes
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Phillip,

I understand that the MIPI output trace that you have shared is captured from the MIPI lanes. Is that correct?

If yes, please let me know if your hardware has the test pins as in Q10 of this KBA CX3 Hardware: Frequently Asked Questions - KBA9129... - Cypress Developer Community . If yes, please probe PCLK, HSYNC, VSYNC and share the traces. Please note that we want to check the H blanking, h total, v blanking , v total time of the traces captured

Kindly let me know if the frame size of 0 and 2 are as expected. Please check the producer and consumer events for one frame. We do not recommend to use CyU3PDebugPrint API in the DMA callback. Please trace the producer and consumer events using global variables.

1. In the image above, the counts on the two sockets don't match, could this be a problem?

>> The socket count mismatch can be due to the repeated Appstart and AppStop being called.

Regards,
Rashi
0 Likes

Hello Rashi,
Thank you for your help.

I've used Denebola RDK Board but I cannot find the test pins on the board. There seem to be no test points that you mentioned. How can I handle it?

I added a counter variable for consumer and producer events. When data is received from the streamer, the log does not appear because the CyU3PEventGet() function does not return. The image below is right after pressing the Stop button while receiving data from the stream normally.

Philip_0-1623811917794.png

I think both events seem to be running normally. Is that correct?

Regards,
Philip

0 Likes
lock attach
Attachments are accessible only for community members.
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Philip,

Please build the attached firmware and program FX3. Please share the UART debug prints with the attached firmware.

 

Regards,
Rashi
0 Likes