Hi, all
I would like to confirm about P of example Calculation in KBA226722 as follows:
S = 640*480*2 = 614400 bytes
F = Greatest integer less than (614400 / (16384 – 16)) = 37
P = DMA buffer size * fractional part of (S / (DMA buffer size – 16)) = 8784 bytes
However, P is:
P = 16384 * fractional part of (614400 / (16384 – 16))
= 16384 * 0.536656891495601173020527859238
= 8792.5865102639296187683284457478 bytes
The calculation does not mach. The answer is correct if (DMA buffer size – 16) instead of multiplying by "DMA buffer size".
P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))
= 16368 * 0.536656891495601173020527859238
= 8784 bytes
Which is the correct calculation for the Number of bytes in the Partial buffer?
Best Regards,
Kenshow
Solved! Go to Solution.
Hello,
Your understanding is correct. The calculation should be:
P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))
Thank you for pointing out this mistake. We will correct this mistake and update the KBA soon.
Best Regards,
Jayakrishna
Hello,
Your understanding is correct. The calculation should be:
P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))
Thank you for pointing out this mistake. We will correct this mistake and update the KBA soon.
Best Regards,
Jayakrishna
Hi Jayakrishna,
I will modify my Japanese translation, too.
Thanks you.
Best Regards,
Kenshow