Accessing the buffer while using GPIF has to do with the mode of operation you're in (auto mode or manual mode). Even in that case you won't be able to play with the buffer in the way you're suggesting. GPIF will start writing from location 0.
What is the transfer speed you're looking at?
Go through the GPIF chapter for more details