- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, all
I would like to confirm about P of example Calculation in KBA226722 as follows:
S = 640*480*2 = 614400 bytes
F = Greatest integer less than (614400 / (16384 – 16)) = 37
P = DMA buffer size * fractional part of (S / (DMA buffer size – 16)) = 8784 bytes
However, P is:
P = 16384 * fractional part of (614400 / (16384 – 16))
= 16384 * 0.536656891495601173020527859238
= 8792.5865102639296187683284457478 bytes
The calculation does not mach. The answer is correct if (DMA buffer size – 16) instead of multiplying by "DMA buffer size".
P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))
= 16368 * 0.536656891495601173020527859238
= 8784 bytes
Which is the correct calculation for the Number of bytes in the Partial buffer?
Best Regards,
Kenshow
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Your understanding is correct. The calculation should be:
P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))
Thank you for pointing out this mistake. We will correct this mistake and update the KBA soon.
Best Regards,
Jayakrishna
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Your understanding is correct. The calculation should be:
P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))
Thank you for pointing out this mistake. We will correct this mistake and update the KBA soon.
Best Regards,
Jayakrishna
Jayakrishna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Jayakrishna,
I will modify my Japanese translation, too.
Thanks you.
Best Regards,
Kenshow