OV2740 UVC Request to set Exposure time

[PhMi_4136011]

Hi,

We are working with an 10 bit RAW gray scale sensor and established a Video stream (1080p 30fps, WIn10). SeeRe: Setup MIPI connection between CX3 and OV2740

1. Do we need to care about differences in UVC1.1 and 1.5, since the generated Code seems to be 1.1.

2. We want to develop a Host application based ob Python3 and Opencv. If we use the Videocapture function, the frames are interpreted as 8bit, but the sensor is set to 16 bit mode with 10 bits used and 6 bits zeros. It follows, that intensitys higher than 255 are displayed black. Is there a way to capute a 16 bit image and normalize the gray values to 10 bit?

We want to set USB request via Python and Opencv to contoll exposure time (and other controls) (via cap.set(cv2.CAP_PROP_EXPOSURE, xyz)). Therefor we edited the cycx3_uvcdscr.c and the cycx3_uvc.c .

cycx3_uvcdscr.c:

/* Input (Camera) Terminal Descriptor */

    0x2A,0x00,0x02,                        /*  @kolja: found this in physical-hasher/bitmark-microscope at master · bitmark-inc/physical-hasher · GitHub */ 

//    0x00, 0x00, 0x00,                   /* No controls supported */

                                        /*A bit set to 1 indicates that the

                                        mentioned Control is supported for

                                        the video stream.

                                        D0: Scanning Mode, D1: Auto-Exposure Mode, D2: Auto-Exposure Priority, D3: Exposure Time(Absolute)

                                        D4: Exposure Time(Relative) ........D18: Reserved, set to zero */

    /* Processing Unit Descriptor */

    0x01, 0x00, 0x00,                   /* No controls supported */

                                        /*@kolja: edited this to 1 from 0

                                         A bit set to 1 indicates that the mentioned

                                            Control is supported for the video stream.

                                            D0: Brightness, D1: Contrast, D2: Hue, D3: Saturation, D4: Sharpness....D18..(n*8-1): Reserved. Set to zero*/

3. For a start we want to control exposure. Which Descriptor needs to be set in  /* Input (Camera) Terminal Descriptor */ and /* Processing Unit Descriptor */ ? I tried several, but the debug prints in Console does not mention the cap.set(cv2.CAP_PROP_EXPOSURE, xyz) request.

cycx3_uvc.c

[PhMi_4136011]

Debug prints with the attached firmware and some custom debug prints:

CyCx3UvcAppUSBSetupCB bRType = 0x81, bRequest = 0x0, wValue = 0x0, wIndex = 0x0, wLength= 0x2

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x86, wValue = 0x400, wIndex = 0x100, wLength= 0x1

USB UVC Control Interface request for EXPOSURE

USB UVC Control Interface request for case GET_INFO_REQ

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x82, wValue = 0x400, wIndex = 0x100, wLength= 0x4

USB UVC Control Interface request for EXPOSURE

USB UVC Control Interface request for case GET_MIN_REQ

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x83, wValue = 0x400, wIndex = 0x100, wLength= 0x4

USB UVC Control Interface request for EXPOSURE

USB UVC Control Interface request for case GET_MAX_REQ

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x84, wValue = 0x400, wIndex = 0x100, wLength= 0x4

USB UVC Control Interface request for EXPOSURE

USB UVC Control Interface request for case GET_RES_REQ

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x87, wValue = 0x400, wIndex = 0x100, wLength= 0x4

USB UVC Control Interface request for EXPOSURE

USB UVC Control Interface request for case GET_DEF_REQ

Get Exposure Request

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x82, wValue = 0x400, wIndex = 0x100, wLength= 0x4

USB UVC Control Interface request for EXPOSURE

USB UVC Control Interface request for case GET_MIN_REQ

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

AppInit:GpifSMStart passed

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x86, wValue = 0x900, wIndex = 0x100, wLength= 0x1

Video COntroll Request but no case

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x86, wValue = 0x600, wIndex = 0x100, wLength= 0x1

Video COntroll Request but no case

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x86, wValue = 0x200, wIndex = 0x200, wLength= 0x1

Video COntroll Request but no case

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

CyCx3UvcAppUSBSetupCB bRType = 0x1, bRequest = 0x3, wValue = 0x0, wIndex = 0x100, wLength= 0x0

StpCB:In SET_FTR 0::1

CyCx3UvcAppUSBSetupCB bRType = 0x1, bRequest = 0x3, wValue = 0x0, wIndex = 0x0, wLength= 0x0

StpCB:In SET_FTR 0::1

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA

Video COntroll Request but no case

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x200, wIndex = 0x0, wLength= 0x1

CyCx3UvcAppUSBSetupCB bRType = 0x1, bRequest = 0x3, wValue = 0x0, wIndex = 0x100, wLength= 0x0

StpCB:In SET_FTR 0::1

KOLJA: Sensor reseted.

Sensor Initialization

Writing Configuration Settings:

Register = 0x103: Value = 0x1

..

...

...

Register = 0x5901: Value = 0x0

OV2740-Sensor put to sleep

Writing Configuration Settings:

Register = 0x100: Value = 0x0

OV2740-Sensor put to sleep

Writing Configuration Settings:

Register = 0x100: Value = 0x0

EnterSuspendMode Status =  0x0, Wakeup reason = 0x8

CyCx3UvcAppUSBSetupCB bRType = 0x1, bRequest = 0x3, wValue = 0x0, wIndex = 0x0, wLength= 0x0

StpCB:In SET_FTR 0::1

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x100, wIndex = 0x1, wLength= 0x22

CyCx3UvcAppUSBSetupCB bRType = 0x21, bRequest = 0x1, wValue = 0x100, wIndex = 0x1, wLength= 0x22

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x81, wValue = 0x100, wIndex = 0x1, wLength= 0x22

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x83, wValue = 0x100, wIndex = 0x1, wLength= 0x22

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x82, wValue = 0x100, wIndex = 0x1, wLength= 0x22

CyCx3UvcAppUSBSetupCB bRType = 0x21, bRequest = 0x1, wValue = 0x200, wIndex = 0x1, wLength= 0x22

Writing Configuration Settings:

Register = 0x100: Value = 0x1

AplnStrt:SMState = 0x2

OV2740-Sensor wake uppppp

Writing Configuration Settings:

Register = 0x100: Value = 0x1

Prod = 113 Cons = 113  Prtl_Sz = 35120 Frm_Cnt = 1 Frm_Sz = 4195328 B

Prod = 113 Cons = 113  Prtl_Sz = 35120 Frm_Cnt = 2 Frm_Sz = 4195328 B

Prod = 113 Cons = 113  Prtl_Sz = 35120 Frm_Cnt = 3 Frm_Sz = 4195328 B

Prod = 113 Cons = 113  Prtl_Sz = 35120 Frm_Cnt = 4 Frm_Sz = 4195328 B

4. Is there an overview for the shown wValue and wIndex combination? Are the following from git correct? physical-hasher/cycx3_uvc.c at master · bitmark-inc/physical-hasher · GitHub I couldnt find an overview.

[Hemanth]

There are few changes that needs to be done in descriptor file as well as firmware file to migrate from UVC 1.1 to 1.5

Regards,

Hemanth

[PhMi_4136011]

Hi Hemanth,

Do we need to do this changes to control the camera via OpenCV or will the camera control work with UVC 1.1 specifications?

Where do I find an example or overview for the needed changes?

Kind regards,

Philipp

[Rashi_Vatsa]

Hello Philipp,

The changes from UVC 1.1 to UVC 1.5 needs to be done in descriptors and probe control structure.

For example glProbeCtrl_1280x720_60fps, refer to this for adding the fields to every probe control settings is using UVC 1.5 including glProbeCtrl[FX3_UVC_MAX_PROBE_SETTING]

Probe Control:

/* UVC Probe Control Settings */

uint8_t glProbeCtrl_1280x720_60fps[FX3_UVC_MAX_PROBE_SETTING] = {

0x00, 0x00,                         /* bmHint : No fixed parameters */

SS_FC_UYVY_FORMAT_INDEX,            /* Corresponding Video format index */

SS_YUY2_1280x720_INDEX,          /* Corresponding Video frame index */

0x0A, 0x8B, 0x02, 0x00,             /* Frame interval (in 100ns units): (1/60)x10^7  */

0x00, 0x00,                         /* Key frame rate in key frame/video frame units */

0x00, 0x00,                         /* PFrame rate in PFrame / key frame units */

0x00, 0x00,                         /* Compression quality control */

0x00, 0x00,                         /* Window size for average bit rate */

0x00, 0x00,                         /* Internal video streaming i/f latency in ms */

0x00, 0x20, 0x1C, 0x00,             /* Maximum video or still frame size in bytes: 1280 x 720 x 2 */

#ifdef FX3_UVC_1_0_SUPPORT

0x00, 0x90, 0x00, 0x00,             /* No. of bytes device can rx in single payload: 56KB */

#else

0x00, 0xE0, 0x00, 0x00,             /* No. of bytes device can rx in single payload */

#endif

0x00, 0x60, 0xE3, 0x16,             /* Device Clock */

0x00,                               /* Framing Information - Ignored for uncompressed format */

0x00,                               /* Preferred payload format version */

0x00,                               /* Minimum payload format version */

0x00,                               /* Maximum payload format version */

#ifdef FX3_UVC_1_5_SUPPORT

0x00,                               /* Current bUsage */

0x00,                               /* bBitDepthLuma */

0x00,                               /* bmSettings */

0x00,                               /* bMaxNumberOfRefFramesPlus1 */

0x00, 0x00,                         /* bmRateControlModes*/

0x00, 0x00, 0x00, 0x00,             /* bmLayoutPerStream */

0x00, 0x00, 0x00, 0x00,             /* No. of bytes device can rx in single payload */

#endif

#endif

};

This will also change the values of following MACRO

#define FX3_UVC_MAX_PROBE_SETTING               (48)            /* UVC 1.5 Maximum number of bytes in Probe Control */

#define FX3_UVC_MAX_PROBE_SETTING_ALIGNED       (64)            /* Maximum number of bytes in Probe Control aligned to 32 byte */

In descriptors:

Under /* Class specific VC Interface Header Descriptor */, need to mention the UVC version as 1.5

0x50, 0x01,                         /* Revision of UVC class spec: 1.5

To set he exposure time, please refer to UVC spec 1.5 which mentions, D3 and D4 bit need to be set in the  Camera Terminal Descriptor (bmControl field)

D3: Exposure Time (Absolute)

D4: Exposure Time (Relative)

You can refer to AN75779 firmware (UVCHandleCameraTerminalRqts)  to handle the camera terminal requests (set exposure time) https://www.cypress.com/documentation/application-notes/an75779-how-implement-image-sensor-interface...

Let me know if any queries on this

Regards,

Rashi

[PhMi_4136011]

Hi Rashi,

we are struggeling to aply the UVCHandleCameraTerminalRqts to our code for CX3.

1. Which functions do we need to merge ? Do we need the to use the uvcAppEP0Thread / Thread_Entry and stream/control events since we can address all interfaces without? We modifyed the CyCx3UvcAppUSBSetupCB function and added cases within the control interface. Example:

else if (CY_U3P_GET_LSB(wIndex) == CX3_UVC_CONTROL_INTERFACE))

switch ((wIndex >> 8))

            {

                case CX3_UVC_CAMERA_TERMINAL_ID:

                        UVCHandleCameraTerminalRqts ();

                break;

}

2. Debugging shows, that the UVCHandleCameraTerminalRqts function is called, but non of our cases are hit. Is there a better way to compare wValue?

UVCHandleCameraTerminalRqts (void)
{
    CyU3PReturnStatus_t status = CY_U3P_SUCCESS;
    uint16_t readCount;
    uint16_t ExposureVal = 0;
    CyBool_t sendData = CyFalse;

#ifdef CX3_DEBUG_ENABLED
    CyU3PDebugPrint(4, "\n\r UVCHandleCameraTerminalRqts, with wValue %x ", wValue);
#endif
     switch (wValue)
        {
            case CX3_UVC_CT_EXPOSURE_TIME_ABSOLUTE_CONTROL:

                #ifdef CX3_DEBUG_ENABLED
                    CyU3PDebugPrint(4, "\n\r UVCHandleCameraTerminalRqts -> CX3_UVC_CT_EXPOSURE_TIME_ABSOLUTE_CONTROL");
                #endif

Debug:

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x86, wValue = 0x400, wIndex = 0x100, wLength= 0x1
UVC APP Thread case CX3_UVC_CAMERA_TERMINAL_ID ....
UVCHandleCameraTerminalRqts, with wValue 0
UVCHandleCameraTerminalRqts else case

..

.

CyCx3UvcAppUSBSetupCB bRType = 0xA1, bRequest = 0x87, wValue = 0x1400, wIndex = 0x100, wLength= 0xA
UVC APP Thread case CX3_UVC_CAMERA_TERMINAL_ID ....
UVCHandleCameraTerminalRqts, with wValue 0
UVCHandleCameraTerminalRqts else case

3. is the bmControl field within the Camera Terminal Descriptor correct ? 0x18, 0x00, 0x00,

[Rashi_Vatsa · Accepted solution]

Hello Phillip,

Please share the firmware in .zip format, as i am not able to extract the .rar file you shared.

- Please confirm that you are not calling the UVCHandleCameraTerminalRqts function inside the CyCx3UvcAppUSBSetupCB function itself.

Instead you can check for CY_FX_UVC_CONTROL_INTERFACE by parsing the LSB of wIndex. On getting the CY_FX_UVC_CONTROL_INTERFACE set the event CY_FX_UVC_VIDEO_CONTROL_REQUEST_EVENT which can be latter handled in the for{} loop (as done in AN75779).

- The MSB of wIndex is parsed to get the terminal ID which should be 1 i.e. CY_FX_UVC_CAMERA_TERMINAL_ID in your case

- UVCHandleCameraTerminalRqts should be called when MSB of wIndex is CY_FX_UVC_CAMERA_TERMINAL_ID

- In UVCHandleCameraTerminalRqts, wValue decides the control CY_FX_UVC_CT_EXPOSURE_TIME_ABSOLUTE_CONTROL         (uint16_t)(0x0400)

- As per the debug prints you shared, the wValue is 0x400 when printed in CyCx3UvcAppUSBSetupCB and changes to 0 when printed from UVCHandleCameraTerminalRqts. Please confirm the scope of the wValue variable. It should be declared as global variable

uint16_t wValue;

- Also check that value of wValue is not modified elsewhere in the firmware.

- is the bmControl field within the Camera Terminal Descriptor correct ? 0x18, 0x00, 0x00,

>> Yes, the bmControl field seems correct as it enables Exposure Time (absolute) control and exposure time (relative) control.

To confirm that the host sends the control request correctly. Please probe the usb line using wireshark. Also let me know which host application are you using.

Regards,

Rashi

[PhMi_4136011]

Hi Rashi,

Thank you for the support !! Set exposure Time is now working and other controls should be similar.

But we still struggle with the pixle format. Since we got a 10 bit raw gray scale sensor, we operate gpif in 16 bit mode (YUV2 with 0x59, 0x55, 0x59, 0x32 in descriptor file). Our sensor got an unusal active pixel array of 1928 x 1088. Does this mean  Streaming RAW10 Format Input Data to 16/24-bit Output Format in CX3 MIPI CSI-2 - KBA224387  won't work for us? The document says, the values need to be a multiple of 16...1928/1088 can't be divided by 16.

Therefore we are currently developing a host application which avoids the problem and erases the unused 6 bits. I attached a two images in .txt format to show asystematic error within the camera picture. Is this error due to an expected bayer pattern ? We don't know how to handle this problem, is this know to you?

Kind regards,

Philipp