How to support different video resolutions - AN75779

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
bojankoce
Level 3
Level 3
25 replies posted 10 replies posted 5 replies posted

Hello guys,

We were able to successfully benefit from the application note AN75779 and use FX3 to transfer video streams from our image sensors (IMX250, IMX253, IMX420) to the PC. VLC player has been used to play the streams. Thanks for your support in that sense (thread 1, thread 2)! It is really appreciated.

We would now like to replace VLC player by our custom software using UVC class drivers. We would like to be able to change video resolution on the fly (by sending appropriate set_resolution commands from PC). On the page 65 (Section 9.2) of AN75779 document, it is explained that new frame descriptor should be added to available format descriptor.

How can we actually do it?

Does that mean we need to have multiple /* Class specific Uncompressed VS frame descriptors */ within CyFxUSBSSConfigDscr[] Super Speed Configuration Descriptor - one frame descriptor for every resolution supported?

Similarly, if we want to support different UVC RAW formats (e.g. YUY2 16 bits per pixel, some custom 32-bit per pixel format...), does that mean we need to have multiple /* Class specific Uncompressed VS format descriptors*/ within CyFxUSBSSConfigDscr[] Super Speed Configuration Descriptor - one for every VS format supported?

For the case of affirmative response on above questions, do we place multiple frame and format descriptors one beneath the other within CyFxUSBSSConfigDscr[] Super Speed Configuration Descriptor? Is it necessary to change any other field of CyFxUSBSSConfigDscr[] Super Speed Configuration Descriptor in order to support different video resolutions?

Thank you very much for your time and efforts!

Sincerely,

Bojan.

0 Likes
1 Solution
lock attach
Attachments are accessible only for community members.

Hello Bojan,

Within glProbeCtrl[CY_FX_UVC_MAX_PROBE_SETTING] Probe Control Settings we set desired frame interval to be 60fps (max). Similarly, Max Video frame size in bytes parameter is set to be equal to the size of our max resolution. Is that OK?

>> You have create separate  probe control structures  for every resolution (in your case 7). These probe control structures should be passed to the host as per the request from the host.

At first the glProbeCtrl[CY_FX_UVC_MAX_PROBE_SETTING] should contain a default resolution that you want to stream lets take 1920 *1080 *60 fps (frameindex 1).

You have to create multiple probe control structures for each resolution (like 1920*1080*60 fps)

For example:

uint8_t glProbeCtrl_1920x1080_60fps[FX3_UVC_MAX_PROBE_SETTING];

uint8_t glProbeCtrl_1280x720_60fps[FX3_UVC_MAX_PROBE_SETTING];

uint8_t glProbeCtrl_640x640_60fps[FX3_UVC_MAX_PROBE_SETTING];

uint8_t glProbeCtrl_640x480_60fps[FX3_UVC_MAX_PROBE_SETTING];

This is  for one setting (one resolution)

case CY_FX_USB_UVC_GET_DEF_REQ: /* There is only one setting per USB speed. */

                    if (usbSpeed == CY_U3P_SUPER_SPEED)

                    {

                    

                        CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *)glProbeCtrl);

                    }

                    else

                    {

                        CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *)glProbeCtrl20);

                    }

                    break;

For multiple resolution

you can parse the frame index number and send the probe control data of the appropriate  resolution (as per the request from the host).

case CY_FX_USB_UVC_GET_DEF_REQ: /* There is only one setting per USB speed. */

                    if (usbSpeed == CY_U3P_SUPER_SPEED)

                    {

                      if(frame_index == 2)

                        CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *glProbeCtrl_1280x720_60fps));

                    }... so on

However, when I generate image file and load it to FX3 RAM, I receive notification that USB device is not recognized! Normally, the device should be visible as FX3 device under Cameras in Windows OS.

>> Apologies,  i had missed mentioning about the configuration descriptor size. Please refer to the attached modified file which includes calculation of configuration descriptor. It is as per the descriptors i shared. You can calculate the descriptor size as per your descriptor file. (as mentioned in the comments)

Please let me know if you have query on the calculation part.

We removed SS_YUY2_STILL_DESCR_SIZE + 0x06 from the formula because AN75779 firmware does not contain Still image descriptor  and Colour matching descriptor fields. Is that OK?

>> Yes, it is fine. Please confirm that you don't add these descriptors in the calculation of class descriptors

Regards,

Rashi

Regards,
Rashi

View solution in original post

4 Replies
lock attach
Attachments are accessible only for community members.
Rashi_Vatsa
Moderator
Moderator
Moderator
5 likes given 500 solutions authored 1000 replies posted

Hello Bojan,

1) Does that mean we need to have multiple /* Class specific Uncompressed VS frame descriptors */ within CyFxUSBSSConfigDscr[] Super Speed Configuration Descriptor - one frame descriptor for every resolution supported

>> Yes your understanding is correct

2) Yes, for multiple formats different formats needs to be added under /* Class-specific Video Streaming Input Header Descriptor */

The field    0x01,              /* Format descriptor follows */ need to be modified as per the number of formats that would be supported. Also the total descriptor size of Class-specific Video Streaming descriptor should be modified in this field

SS_FC_TOTAL_SIZE_CLASS_DSCR_L,        /* Total size of Class specific VS descr -Low Byte*/

SS_FC_TOTAL_SIZE_CLASS_DSCR_H,        /* Total size of Class specific VS descr -High Byte*/

Please confirm that  Class specific Uncompressed VS format descriptor  should have GUID of frame format supported by UVC driver.

3) For a particular format the total number of video resolutions supported is to be modified in /* Class specific Uncompressed VS format descriptor */. Every format will have an unique index associated with it.

SS_FC_UYVY_FORMAT_INDEX,              /* Format desciptor index */

SS_FC_UYVY_TOTAL_NO_OF_RES,            /* Number of Frame Descriptors that follow this descriptor */

The class specific video streaming descriptors can be in this format

Class-specific Video Streaming Input Header Descriptor

            Class specific Uncompressed VS format descriptor

                      Class specific Uncompressed VS Frame Descriptor - UYVY_1920x1080 at 30 FPS*/

                      Class specific Uncompressed VS Frame Descriptor - UYVY_1280x720 at 30 FPS

                      Class specific Uncompressed VS Frame Descriptor - UYVY_640x480 at 30 FPS*/

          Class specific Uncompressed VS format descriptor */

                    Class specific Uncompressed VS Frame Descriptor - _1920x1080 at 30 FPS*/

                    ....

For better understanding, please refer to the attached file  (with multiple resolutions (one frame format))

Every frame resolution supported should have an associated UVC probe control settings also.

Please let me know if any queries on this

Regards,

Rashi

Regards,
Rashi

Hello, RashiV_61​.

Thank you very much for your useful feedback!

We would like to go step by step and first implement the case with one VS format (YUY2) and multiple resolutions (VS frame descriptors). Based on your feedback and multi_reslution_0 file you shared with me, here is how we formatted Input header descriptor as well as VS format descriptor:

/* Class-specific Video Streaming Input Header Descriptor */

        0x0E,                           /* Descriptor size */

        0x24,                           /* Class-specific VS I/f Type */

        0x01,                           /* Descriptotor Subtype : Input Header */

        0x01,                           /* 1 format desciptor follows */

        SS_FC_TOTAL_SIZE_CLASS_DSCR_L,         /* Total size of Class specific VS descr -Low Byte*/

        SS_FC_TOTAL_SIZE_CLASS_DSCR_H,         /* Total size of Class specific VS descr -High Byte*/

        CY_FX_EP_BULK_VIDEO,            /* EP address for BULK video data */

        0x00,                           /* No dynamic format change supported */

        0x04,                           /* Output terminal ID : 4 */

        0x01,                           /* Still image capture method 1 supported */

        0x00,                           /* Hardware trigger NOT supported */

        0x00,                           /* Hardware to initiate still image capture NOT supported */

        0x01,                           /* Size of controls field : 1 byte */

        0x00,                           /* D2 : Compression quality supported */

        /* Class specific Uncompressed VS format descriptor */

        0x1B,                           /* Descriptor size */

        0x24,                           /* Class-specific VS I/f Type */

        0x04,                           /* Subtype : uncompressed format I/F */

        0x01,                           /* Format desciptor index */

        SS_FC_UYVY_TOTAL_NO_OF_RES,            /* Number of Frame Descriptors that follow this descriptor */

        /* GUID, globally unique identifier used to identify streaming-encoding format: YUY2  */

        0x59,0x55,0x59,0x32,            /* GUID used to identify streaming-encoding format: YUY2  */

        /*'Y' = 0x59, 'U' = 0x55, 'V' = 0x56,'2' = 0x32 */

        0x00,0x00,0x10,0x00,

        0x80,0x00,0x00,0xAA,

        0x00,0x38,0x9B,0x71,

        0x10,                           /* Number of bits per pixel - 16*/

        0x01,                           /* Optimum Frame Index for this stream: 1 */

        0x10,                           /* X dimension of the picture aspect ratio; Non-interlaced */

        0x09,                           /* Y dimension of the pictuer aspect ratio: Non-interlaced */

        0x00,                           /* Interlace Flags: Progressive scanning, no interlace */

        0x00,                           /* duplication of the video stream restriction: 0 - no restriction */

The above fields are followed by VS Frame descriptors for every resolution we want to have (SS_FC_UYVY_TOTAL_NO_OF_RES is equal to 7).

Within glProbeCtrl[CY_FX_UVC_MAX_PROBE_SETTING] Probe Control Settings we set desired frame interval to be 60fps (max). Similarly, Max Video frame size in bytes parameter is set to be equal to the size of our max resolution. Is that OK?

However, when I generate image file and load it to FX3 RAM, I receive notification that USB device is not recognized! Normally, the device should be visible as FX3 device under Cameras in Windows OS.

Did we miss something!?

Total size of Class specific VS descriptor is calculated as:

#define SS_FC_TOTAL_SIZE_CLASS_DSCR   (0x0E + 0x1B + (SS_FC_UYVY_TOTAL_NO_OF_RES * 0x1E))

This differs from the file you shared with me. There this parameter is calculated as:

#define SS_FC_TOTAL_SIZE_CLASS_DSCR   (0x0E + 0x1B + SS_YUY2_STILL_DESCR_SIZE + 0x06 +\
(SS_FC_UYVY_TOTAL_NO_OF_RES * 0x1E))

We removed SS_YUY2_STILL_DESCR_SIZE + 0x06 from the formula because AN75779 firmware does not contain Still image descriptor  and Colour matching descriptor fields. Is that OK?

Thank you very much for your time and efforts!

Sincerely,

Bojan.

0 Likes
lock attach
Attachments are accessible only for community members.

Hello Bojan,

Within glProbeCtrl[CY_FX_UVC_MAX_PROBE_SETTING] Probe Control Settings we set desired frame interval to be 60fps (max). Similarly, Max Video frame size in bytes parameter is set to be equal to the size of our max resolution. Is that OK?

>> You have create separate  probe control structures  for every resolution (in your case 7). These probe control structures should be passed to the host as per the request from the host.

At first the glProbeCtrl[CY_FX_UVC_MAX_PROBE_SETTING] should contain a default resolution that you want to stream lets take 1920 *1080 *60 fps (frameindex 1).

You have to create multiple probe control structures for each resolution (like 1920*1080*60 fps)

For example:

uint8_t glProbeCtrl_1920x1080_60fps[FX3_UVC_MAX_PROBE_SETTING];

uint8_t glProbeCtrl_1280x720_60fps[FX3_UVC_MAX_PROBE_SETTING];

uint8_t glProbeCtrl_640x640_60fps[FX3_UVC_MAX_PROBE_SETTING];

uint8_t glProbeCtrl_640x480_60fps[FX3_UVC_MAX_PROBE_SETTING];

This is  for one setting (one resolution)

case CY_FX_USB_UVC_GET_DEF_REQ: /* There is only one setting per USB speed. */

                    if (usbSpeed == CY_U3P_SUPER_SPEED)

                    {

                    

                        CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *)glProbeCtrl);

                    }

                    else

                    {

                        CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *)glProbeCtrl20);

                    }

                    break;

For multiple resolution

you can parse the frame index number and send the probe control data of the appropriate  resolution (as per the request from the host).

case CY_FX_USB_UVC_GET_DEF_REQ: /* There is only one setting per USB speed. */

                    if (usbSpeed == CY_U3P_SUPER_SPEED)

                    {

                      if(frame_index == 2)

                        CyU3PUsbSendEP0Data (CY_FX_UVC_MAX_PROBE_SETTING, (uint8_t *glProbeCtrl_1280x720_60fps));

                    }... so on

However, when I generate image file and load it to FX3 RAM, I receive notification that USB device is not recognized! Normally, the device should be visible as FX3 device under Cameras in Windows OS.

>> Apologies,  i had missed mentioning about the configuration descriptor size. Please refer to the attached modified file which includes calculation of configuration descriptor. It is as per the descriptors i shared. You can calculate the descriptor size as per your descriptor file. (as mentioned in the comments)

Please let me know if you have query on the calculation part.

We removed SS_YUY2_STILL_DESCR_SIZE + 0x06 from the formula because AN75779 firmware does not contain Still image descriptor  and Colour matching descriptor fields. Is that OK?

>> Yes, it is fine. Please confirm that you don't add these descriptors in the calculation of class descriptors

Regards,

Rashi

Regards,
Rashi

Hello, RashiV_61​!

You were right, it was indeed necessary to properly calculate total size of the Super Speed Configuration Descriptor! Thanks for the explanations on how to do that!

We are now able to see from VirtualDub player that the player can play multiple resolutions. I also noticed that the last Class specific Uncompressed VS frame descriptor in the list is the default one!

We will now dive deeper into the code and try to figure out how the code is organized as well as how FX3 firmware communicates with the code on PC side.

Cheers,

Bojan.

0 Likes