You could use DMA to transfer from the FIFO buffers to somewhere in the SRAM. You can trigger the DMA to transfer only twice, then disable it. It triggers when there is data available in the counter FIFO. Before enabling the DMA, make sure to clear the counter FIFO.
Thank you! I ended up just never allowing the buffers to overflow. They should only be accessed at around 500Hz, which is okay and a lot less frequent than I thought.