Example Calculation in KBA226722

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
Kenshow
Level 8
Level 8
Distributor - Marubun (Japan)
50 solutions authored 25 solutions authored 10 solutions authored

Hi, all

I would like to confirm about P of example Calculation in KBA226722 as follows:

S = 640*480*2 = 614400 bytes

F = Greatest integer less than (614400 / (16384 – 16)) = 37

P = DMA buffer size * fractional part of (S / (DMA buffer size – 16)) = 8784 bytes

However, P is:

P = 16384 * fractional part of (614400 / (16384 – 16))

  = 16384 * 0.536656891495601173020527859238

  = 8792.5865102639296187683284457478 bytes

The calculation does not mach. The answer is correct if (DMA buffer size – 16) instead of multiplying by "DMA buffer size".

P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))

  = 16368 * 0.536656891495601173020527859238

  = 8784 bytes

Which is the correct calculation for the Number of bytes in the Partial buffer?

Best Regards,

Kenshow

0 Likes
1 Solution
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello,

Your understanding is correct. The calculation should be:

P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))

Thank you for pointing out this mistake. We will correct this mistake and update the KBA soon.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna

View solution in original post

0 Likes
2 Replies
JayakrishnaT_76
Moderator
Moderator
Moderator
First question asked 1000 replies posted 750 replies posted

Hello,

Your understanding is correct. The calculation should be:

P = (DMA buffer size – 16) * fractional part of (S / (DMA buffer size – 16))

Thank you for pointing out this mistake. We will correct this mistake and update the KBA soon.

Best Regards,

Jayakrishna

Best Regards,
Jayakrishna
0 Likes
Kenshow
Level 8
Level 8
Distributor - Marubun (Japan)
50 solutions authored 25 solutions authored 10 solutions authored

Hi Jayakrishna,

I will modify my Japanese translation, too.

Thanks you.

Best Regards,

Kenshow

0 Likes