The device descriptor read back from the device should be word-aligned. As an Endpoint occupies only 7 bytes of memory, so with odd number of endpoints the descriptor boundary will not be word-aligned and it will mess up with other descriptors. The work around for this is insert 0x00 at the end of any of the endpoint descriptor. This will solve the issue.
This issue can also be solved by following the method described in the knowledge base article in the link here