Lattice Crosslink FPGA configuration and UVC Color bar video streaming using Cypress FX2LP – KBA30768

Version 6

    Translation - Japanese: サイプレスFX2LPを使用したラティスクロスリンクFPGA構成とUVCカラーバービデオストリーミング –  KBA30768 - Community Translated (JA)

     

    Version: **

     

    This KBA documents the details about implementation of FPGA configuration and UVC framework on FX2LP firmware. It uses Lattice Crosslink FPGA to generate and stream color bar video. The firmware example attached with this KBA binds with the standard UVC driver of the Host PC / mobile phone. The color bar is generated on FPGA and streamed by FX2LP firmware. The firmware is tested using FX2LP Low Cost Board (Figure 2) and MIPI Camera Crosslink FPGA Addon Board (Figure 3). The following block diagram shown in Figure 1, provides an overview of the high-level design of interfacing a camera using FX2LP and Crosslink FPGA.

     

    Figure 1. System Block Diagram

     

    Figure 2. FX2LP Low Cost Board

     

    Figure 3. MIPI Camera Crosslink FPGA Add-on Board

     

    FX2LP Firmware features

    This firmware consists of slave FIFO interface, UVC framework and Vendor class interface to support FPGA configuration and video streaming. UVC framework used in this project is described in Cypress Application note AN75779. Please note that this application note implements USB Video Class (UVC) Framework for an FX3 device. But this can be used as a reference for understanding the details about UVC framework.

    This firmware supports video streaming in YUY2 format with 640 x 480 resolution. The slave FIFO interface of the FX2LP device gets the  640 x 480 resolution color bar video from the FPGA and loads it onto FX2LP’s FIFO buffer. FX2LP sends this video data to the host over the UVC interface. UVC-related requests and enumeration are handled by the FX2LP device. Crosslink FPGA starts streaming according to FIFO buffer empty control signals sent from FX2LP.

    The USB Endpoint 2 (EP2) size is configured as 512 bytes, quad buffered to function as a Slave FIFO Bulk IN receiver. Flag A is configured as FIFO empty flag signaling Crosslink FPGA to begin loading data onto buffer and FLAG B is configured as FIFO full flag. FPGA asserts PKTEND signal to send short packet data of 1280 (640 x 2) bytes to send one line of video stream, buffer is loaded again when empty flag is asserted and  UVC request reads and clears data in EP2.

    Crosslink FPGA is configured (programmed) via I2C by FX2LP. FX2LP receives bitstream from PC through vendor class interface as vendor commands. Vendor commands use control endpoint to receive the bitstream data. A custom vendor class host application is built and provided along with this KBA which takes in generated FPGA bitstream and breaks it into multiple blocks of 4096 bytes and transmits it repeatedly as a vendor request. Commands to configure and write to Crosslink FPGA over I2C has been implemented and can be found in td_init() function. FX2LP switches to Slave FIFO upon assertion of CDONE from Crosslink indicating completion of FPGA configuration.

    Latest revision of MIPI Camera Crosslink FPGA Add-on Board includes an I2C - 2Mbit EEPROM, which can be used to load the bitstream directly to FPGA. This will eliminate the need to download bitstream from HOST PC to Crosslink every time device is reset. This hasn’t been tested and will be updated accordingly.

    Testing the Implementation

    The attached project includes Crosslink FPGA code, which implements a color bar video generation and stream using FX2LP’s slave FIFO Master interface. YUY2 640 x 480 video resolution was streamed with a 12 MHz[NB1] clock, leading to data speed of 192 Mbps (12M X 16 bits). This was tested on FX2LP Low cost board, schematic files for which can be found on KBA229176 and MIPI Camera Crosslink FPGA Add-on Board, schematic files of this board is attached with this KBA. The UVC application used to stream video color bar is MPC-HC video player. Generated color bar with rendering statistics are as shown in Figure 4.

    Figure 4. Streamed YUV Color bar
    Pin Mappings made for this implementation as per Table  1.
    Table 1. Pin Mapping

    Serial number

    Signal Name on MIPI Camera Crosslink FPGA ADD-ON Board

    FX2LP PIN Name

    1

    PA0

    PA0

    2

    SLWR

    SLWR

    3

    PKTEND

    PA6

    4

    FLAG A

    CTL0

    5

    FD0

    PB0

    6

    FD1

    PB1

    7

    FD2

    PB2

    8

    FD3

    PB3

    9

    FD4

    PB4

    10

    FD5

    PB5

    11

    FD6

    PB6

    12

    FD7

    PB7

    13

    FD8

    PD0

    14

    FD9

    PD1

    15

    FD10

    PD2

    16

    FD11

    PD3

    17

    FD12

    PD4

    18

    FD13

    PD5

    19

    FD14

    PD6

    20

    FD15

    PD7

    21

    ADR0

    PA4

    22

    ADR1

    PA5

    23

    CDONE

    PA1

    24

    CRESET

    PA3

     

    Additional Modifications Possible

    You can further modify the attached firmware to include more resolutions, formats, and frame rates. You can also add standard UVC video controls and custom extension unit controls.

    “MIPI CAMERA FPGA ADD-ON BOARD FOR FX2LP” also has an image sensor connector and an image sensor module can be connected to stream video instead of color bar.

    Files enclosed along with KBA

    1. “MIPI CAMERA FPGA ADD-ON BOARD FOR FX2LP” Schematic
    2. Crosslink FPGA logic project files
    3. FX2LP firmware project files
    4. Custom Application to send Generated Bitstream over Control Endpoint to FX2LP