Please find my comments below:
1. Your understanding that AN75779 project make use of DMA buffers of 16K is correct. But the number of DMA buffers allocated is actually 8. This is because for each GPIF socket, 4 buffers are allocated. So, the total DMA buffer size used is around 16K*8 = 128K.
2. It is possible to support sensors having row size greater than Buffer size. Please refer to the section 3.4 of AN75779 document. This shows the thread switching procedure that is implemented in the state machine. From this, when the state machine is sampling data into DMA buffers in a thread and the buffer becomes full, it will switch to the next thread and sample data into the DMA buffers allocated with that thread. The same is implemented in the state machine of AN75779 project. Hence it is possible to use a sensor having line size greater than buffer size.
3. The maximum resolution that can be supported depends on practical USB bandwidth and GPIF II bandwidth. The bandwidth of the video is calculated as
Bandwidth = h-active * v-active * bits per pixel * fps.
Note that h-active is in number of pixels unit and v-active is in number of lines in the above calculation.
If a 32 bit GPIF II interface is used with the image sensor, then a maximum of 3.2Gbps(400MBps) can be supported by FX3. This is because the maximum supported GPIF II clock frequency is 100MHz. So, the bandwidth of the sensor should be less than this. Note that if you are using a 16 bit GPIF II interface with the sensor, then the maximum bandwidth further reduces to 1.6Gbps(200MBps).
Example : For a 64MP sensor if the number of bits per pixel is 16(2 Bytes), and if a 32 bit GPIF II interface is used, then
You can find that from the above calculation, it may support a max of 3fps.
Using the above calculation, you can find the maximum fps for 48MP also depending on the parameters that you wish to use.
Please let me know if you have further queries on this.