Translation - Japanese: サイプレスFX2LPを使用したラティスクロスリンクFPGA構成とUVCカラーバービデオストリーミング – KBA30768 - Community Translated (JA)
Version: **
This KBA documents the details about implementation of FPGA configuration and UVC framework on FX2LP firmware. It uses Lattice Crosslink FPGA to generate and stream color bar video. The firmware example attached with this KBA binds with the standard UVC driver of the Host PC / mobile phone. The color bar is generated on FPGA and streamed by FX2LP firmware. The firmware is tested using FX2LP Low Cost Board (Figure 2) and MIPI Camera Crosslink FPGA Addon Board (Figure 3). The following block diagram shown in Figure 1, provides an overview of the high-level design of interfacing a camera using FX2LP and Crosslink FPGA.
Figure 1. System Block Diagram
Figure 2. FX2LP Low Cost Board
Figure 3. MIPI Camera Crosslink FPGA Add-on Board
FX2LP Firmware features
This firmware consists of slave FIFO interface, UVC framework and Vendor class interface to support FPGA configuration and video streaming. UVC framework used in this project is described in Cypress Application note AN75779. Please note that this application note implements USB Video Class (UVC) Framework for an FX3 device. But this can be used as a reference for understanding the details about UVC framework.
This firmware supports video streaming in YUY2 format with 640 x 480 resolution. The slave FIFO interface of the FX2LP device gets the 640 x 480 resolution color bar video from the FPGA and loads it onto FX2LP’s FIFO buffer. FX2LP sends this video data to the host over the UVC interface. UVC-related requests and enumeration are handled by the FX2LP device. Crosslink FPGA starts streaming according to FIFO buffer empty control signals sent from FX2LP.
The USB Endpoint 2 (EP2) size is configured as 512 bytes, quad buffered to function as a Slave FIFO Bulk IN receiver. Flag A is configured as FIFO empty flag signaling Crosslink FPGA to begin loading data onto buffer and FLAG B is configured as FIFO full flag. FPGA asserts PKTEND signal to send short packet data of 1280 (640 x 2) bytes to send one line of video stream, buffer is loaded again when empty flag is asserted and UVC request reads and clears data in EP2.
Crosslink FPGA is configured (programmed) via I2C by FX2LP. FX2LP receives bitstream from PC through vendor class interface as vendor commands. Vendor commands use control endpoint to receive the bitstream data. A custom vendor class host application is built and provided along with this KBA which takes in generated FPGA bitstream and breaks it into multiple blocks of 4096 bytes and transmits it repeatedly as a vendor request. Commands to configure and write to Crosslink FPGA over I2C has been implemented and can be found in td_init() function. FX2LP switches to Slave FIFO upon assertion of CDONE from Crosslink indicating completion of FPGA configuration.
Latest revision of MIPI Camera Crosslink FPGA Add-on Board includes an I2C - 2Mbit EEPROM, which can be used to load the bitstream directly to FPGA. This will eliminate the need to download bitstream from HOST PC to Crosslink every time device is reset. This hasn’t been tested and will be updated accordingly.
Testing the Implementation
The attached project includes Crosslink FPGA code, which implements a color bar video generation and stream using FX2LP’s slave FIFO Master interface. YUY2 640 x 480 video resolution was streamed with a 12 MHz[NB1] clock, leading to data speed of 192 Mbps (12M X 16 bits). This was tested on FX2LP Low cost board, schematic files for which can be found on KBA229176 and MIPI Camera Crosslink FPGA Add-on Board, schematic files of this board is attached with this KBA. The UVC application used to stream video color bar is MPC-HC video player. Generated color bar with rendering statistics are as shown in Figure 4.
Serial number | Signal Name on MIPI Camera Crosslink FPGA ADD-ON Board | FX2LP PIN Name |
1 | PA0 | PA0 |
2 | SLWR | SLWR |
3 | PKTEND | PA6 |
4 | FLAG A | CTL0 |
5 | FD0 | PB0 |
6 | FD1 | PB1 |
7 | FD2 | PB2 |
8 | FD3 | PB3 |
9 | FD4 | PB4 |
10 | FD5 | PB5 |
11 | FD6 | PB6 |
12 | FD7 | PB7 |
13 | FD8 | PD0 |
14 | FD9 | PD1 |
15 | FD10 | PD2 |
16 | FD11 | PD3 |
17 | FD12 | PD4 |
18 | FD13 | PD5 |
19 | FD14 | PD6 |
20 | FD15 | PD7 |
21 | ADR0 | PA4 |
22 | ADR1 | PA5 |
23 | CDONE | PA1 |
24 | CRESET | PA3 |
Additional Modifications Possible
You can further modify the attached firmware to include more resolutions, formats, and frame rates. You can also add standard UVC video controls and custom extension unit controls.
“MIPI CAMERA FPGA ADD-ON BOARD FOR FX2LP” also has an image sensor connector and an image sensor module can be connected to stream video instead of color bar.
Files enclosed along with KBA
- “MIPI CAMERA FPGA ADD-ON BOARD FOR FX2LP” Schematic
- Crosslink FPGA logic project files
- FX2LP firmware project files
- Custom Application to send Generated Bitstream over Control Endpoint to FX2LP