FX3 non-uvc camera streamer

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
lock attach
Attachments are accessible only for community members.
omco_594686
Level 3
Level 3
Welcome!

Hi all,

First, my goal is a streamer application (non-uvc) which capable to stream RAW data.

The HW setup - I'm using the CYUSB3KIT-003 with the Aptina sensor (According to an75779).

About the firmware:

1. I follow this post FX3 / CX3 Firmware for Streaming RAW Image Data using Cypress Driver for non-uvc firmware.

2. I import the sensor.c and sensor.h from an75779 to be able to work with the Aptina sensor.

3. I use the streamer application from the SDK (The C# project)

Now, when I send vendor command 0x99 and run the streamer I see lots of 0x71 errors (screenshot is attached).

The DMA and GPIF configuration remains the same, right?

Also, I tried to follow KBA218830 but I don't see the point of this when I'm using non-uvc FW.

How can I solve this error?

Thanks!

0 Likes
1 Solution
lock attach
Attachments are accessible only for community members.

Hello,

Apologies for making a mistake in the GPIF project attached with my previous response. Please find the latest project with the fixed GPIF Project. As mentioned in my previous response, this project also works fine with streamer and provided a throughput of 54MBps.

Please find my comments for your questions n the previous response:

1. Please check the snapshot of Wireshark trace attached below:

pastedImage_0.png

As you can see there are lot of IN transfers of length 32791 bytes (actual data is 32764 and the remaining is wireshark header). These are the full buffers sent out from FX3. Also, you can see IN transfer of length 9127 (actual data is 9100 excluding wireshark header). This represents the partial buffers committed by FX3 to the host. As you might be knowing FX3 sends out a set of full buffers followed by one partial buffer to the host to transfer the video frame.

In the data section of full and partial buffers, first 12 bytes represents the header added by the firmware. The header is added in accordance with the UVC specification. Please find a snapshot of the data section of a full and partial buffer below:

a. For full buffer

pastedImage_2.png

b. For partial buffer:

pastedImage_5.png

You can see that the second byte of the data is 0x8c for a full buffer and 0x8e for partial buffer. If you check the second byte of all the full full buffers before a partial buffer, they will be the same. This is because the EOF buffer is to be distinguished from the full buffers.

Now, after a partial buffer with second byte as 0x8e is send out, then next set of full buffers will have the second byte as 0x8d. This will continue until the next partial buffer for which the second byte will be 0x8f. This pattern repeats until the streaming is stopped.

2. The GPIF Designer project is added along with the attached project. inside the folder fx3_uvc.cydsn.

Please let me know if you need further clarifications on this.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna

View solution in original post

0 Likes
16 Replies
lock attach
Attachments are accessible only for community members.
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello,

I have tested the firmware (with some modifications) given in the post FX3 / CX3 Firmware for Streaming RAW Image Data using Cypress Driver with streamer application.

Streamer  settings

Packets per Xfer : 128

Xfer per Queue: 64

Timeout: 100 ms

As per the KBA I have increased the DMA buffer size to 32 KB and also made the modification to counter of GPIF state machine

The modified firmware is attached to the post. The test results are also attached.

Regards,

Rashi

Regards,
Rashi
0 Likes
lock attach
Attachments are accessible only for community members.

Hi  RashiV_61​,

Thanks for your reply.

I tested the firmware you attached and I don't see any errors/failures of buffer commits.

I have few more questions:

1. I'm interested to see the hitFV variable when CyFxGpifCB pops, but I see no prints (I added debug prints)

2. I also can't see the print of CyU3PDebugPrint (4, "frame %d\r\n", frameCnt++); . Why is that? I want to make sure I get the full frame and the SM goes to FRAME_END_SCK0 or FRAME_END_SCK1 states.

3. I run tests with different "Packets per Xfer" and "Xfer per Queue" and see no errors. How this parameters effect the data-rate?

0 Likes

Hello,

I'm interested to see the hitFV variable when CyFxGpifCB pops, but I see no prints (I added debug prints)

>> We recommend not use CyU3PDebugPrint API inside the callback function as blocking functions should not be called inside the callback functions. To detect the Frame start/end you can set an event when the callback for frame end is triggered.

CyU3PEventSet API can be used to set the event and CyU3PEventGet in the UVCAppEP0Thread_Entry for{}  to print the status of the frame start/end. You can refer to AN75779 firmware which uses the CyU3PEventSet/CyU3PEventGet API

I run tests with different "Packets per Xfer" and "Xfer per Queue" and see no errors. How this parameters effect the data-rate?

>> As you were seeing the commit buffer failures, which generally occurs due to slow host (when the host doesn't consume the data as fast as the sensor pumps the data to FX3). So we tried increasing the Packets per Xfer and  Xfer per Queue so that the host sends larger requests to the device to get more data. This will also depend on the rate at which the sensor pumps the data to FX3.

Please let me know if you have any queries on this

Regards,

Rashi

Regards,
Rashi
0 Likes

Hi RashiV_61​,

Great!

I removed the debug prints from the CB and able to "catch" the FV with events (and count the frames).

Last thing that bothers me - The Streamer shows data-rate of ~4000KBps, which is strange..

The sensor configuration from AN75779 is: 1280 * 720 * 30fps = ~27 mega pixels per second.

When using YUY2 each 2 pixels represented by 4 bytes.

What am I missing?

Why's the difference is so big between my calculation and the data-rate in practice?

Omri.fx3_streamer_02.jpg

0 Likes

Hello,

Can you try increasing the Packets per Xfer value (128) and Xfer (64) per Queues value and let me know the results.

You can also try reducing the timeout value (100ms)  such that failures are not seen.

In the streamer application, BeginDataXfer API is used to start the streaming. Xfer length parameter is passed to this API which is

endpoint size * Packets per Xfer = 16384*32  (in your case).So (16384*32 bytes) of data will be requested for Xfer per Queue  (16) times in your case.

Xfer rate is bytes transferred/ time taken for the transfer.

Increasing the Packets per Xfer value and Xfer per Queue will improve the Xfer rate

Regards,

Rashi

Regards,
Rashi
0 Likes

Hi RashiV_61​,

I tried the settings you suggest - Packets per xfer = 128 and xfer per queue = 64.

I notice that when I set the timeout to 100ms I got only failures on the PC side (no commit errors on the FW side)

When I set the timeout to 1000ms I got successes (not a single failure) on the PC side and no commit errors on the FW (and slow data-rate)

The results are attached

streamre_100ms_TO.jpgstreamre_1000ms_TO.jpg

Thanks,

Omri.

0 Likes

Hello Omri,

Please let me know the GPIF bus width that you are using to send the data from the sensor to FX3. Is it 8 bits or 16 bits? And the DMA buffer size that is being used in the firmware. This is to confirm that the GPIF state machine used matches the configuration that is used in your firmware.

Regards,

Rashi

Regards,
Rashi
0 Likes

Hi RashiV_61​,

I used the project you sent me in your first comment.

I use the exact hardware from the an75779 - CYUSB3KIT-003 with the Aptina sensor and CYUSB3ACC-004A (the data bus is 8 bit)

Omri.

0 Likes

Dear RashiV_61​,

Any update?

Omri

0 Likes
lock attach
Attachments are accessible only for community members.

Hello,

Apologies for the delayed response.

Please try the attached firmware. We were able to get a throughput of 54MBps as shown in the attached snapshot. You can use the same vendor commands used before i.e 0x99 to start the streaming and 0x88 to terminate the streaming.

pastedImage_0.png

You may see commit buffer failures in between. This is because the host is slow to issue the IN packets. The workaround for this i.e restart the streaming is already provided in the firmware.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hi JayakrishnaT_76​,

First, thanks for you reply.

The FW you attach works as expected! I can see few commit buffer failures.. which is good.

My only concern is the data-lose:

I want to add a headers to each frame, before I commit it to USB - Start-of-frame header AND End-of-frame header.

I use the CyFxUVCAddHeader function and call it depends on if (dmaBuffer.count == CY_FX_UVC_BUF_FULL_SIZE)

(In your FW this code is commented out) but I can't see any difference on the incoming data (I use Wireshark + USBPcap)

What is the right way to do it?

In addition, Can you please attach the GPIFII-Designer project files?

Thanks,

Omri

0 Likes
lock attach
Attachments are accessible only for community members.

Hello,

Apologies for making a mistake in the GPIF project attached with my previous response. Please find the latest project with the fixed GPIF Project. As mentioned in my previous response, this project also works fine with streamer and provided a throughput of 54MBps.

Please find my comments for your questions n the previous response:

1. Please check the snapshot of Wireshark trace attached below:

pastedImage_0.png

As you can see there are lot of IN transfers of length 32791 bytes (actual data is 32764 and the remaining is wireshark header). These are the full buffers sent out from FX3. Also, you can see IN transfer of length 9127 (actual data is 9100 excluding wireshark header). This represents the partial buffers committed by FX3 to the host. As you might be knowing FX3 sends out a set of full buffers followed by one partial buffer to the host to transfer the video frame.

In the data section of full and partial buffers, first 12 bytes represents the header added by the firmware. The header is added in accordance with the UVC specification. Please find a snapshot of the data section of a full and partial buffer below:

a. For full buffer

pastedImage_2.png

b. For partial buffer:

pastedImage_5.png

You can see that the second byte of the data is 0x8c for a full buffer and 0x8e for partial buffer. If you check the second byte of all the full full buffers before a partial buffer, they will be the same. This is because the EOF buffer is to be distinguished from the full buffers.

Now, after a partial buffer with second byte as 0x8e is send out, then next set of full buffers will have the second byte as 0x8d. This will continue until the next partial buffer for which the second byte will be 0x8f. This pattern repeats until the streaming is stopped.

2. The GPIF Designer project is added along with the attached project. inside the folder fx3_uvc.cydsn.

Please let me know if you need further clarifications on this.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hi Jayakrishna,

I failed to build the GPIF Designer project:

gpif_build_failed.jpg

It looks like there's a local path which defines on your environment.. how can I change it?

Besides, The data rate calculation is:

(32764 - 12) * 56 + 9100 - 12 = 1843200

12 - the header size

56 - number of full frame

which is equivalent to:

1280 * 720 * 2 (bpp) =  1843200

Great. I just need to compile the GPIF project on my PC to see if it works.

Omri.

0 Likes

Hello,

Please go to Build->Build Settings. After this, Change the Output Location in the popup. You can set it to the project directory directly. After this, if you build the project, the .h file will be generated in the project location. Please refresh the project in eclipse to use the generated .h after the GPIF Designer project is built.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes

Hi,

Thanks. it works.

Can you please explain the GPIF changes from the last FW?

Omri.

0 Likes

Hello,

In the states WAIT_FOR_FRAME_START_0 and WAIT_FOR_FRAME_START_1, we load 2 counters (LD_DATA_COUNT and LD_ADDR_COUNT). The Counter limit values in the project shared in response 10 had a mistake. This is corrected in the project shared in response 12.

Please refer to section 3.5 of AN75779 to understand how to set the counter limit values. The link to the same is given below:

https://www.cypress.com/file/123506/download

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes