Please find the answers to your question inline:
1.when the os is invoked , what will the os do?
The RTOS kernel is invoked next from the main(). This is a non-returning call and sets up the ThreadX kernel. At the end of the RTOS initialization, all the driver threads (one for DMA, GPIF II port, another serial interfaces etc..) are created. These driver threads are part of FX3 library.
2.when I want to transfer date to peripheral by FIFO ,if the os will take part in ? which example introduce this function?
If you are interested to connect some peripheral (FPGA or an image sensor) to GPIF II of FX3 then please refer to slavefifosync and USBVideoClass examples.
Also the following application notes will be helpful to get started: