Depending on the voltage range and the CPU frequency desired, there are two options to run the USB. The D+/D- lines swing between 0 and 3.3V and needs a 3.3V reference for this. When the operating voltage is selected as 5V, an internal voltage regulator in the USB PHY is enabled that takes care of regulating the 5V to 3.3V.
When the operating voltage is selected as 3.3V, this internal voltage regulator is bypassed and the VDD is directly provide VDD as the reference to the D+/D- lines. As the D+/D- lines can have a tolerance of 10%, the device can operate from 3.0V to 3.6V.
But when the Vdd is above 3.6V, this goes above the 10% tolerance of the D+ / D- lines, but the internal voltage regulator does not have enough regulation to provide a 3.3V reference. The regulator can provide a 3.3V reference only when Vdd is greater than 4.35V. Because of this, when VDD is between 3.6V and 4.35V, the USB lines will not be operating per spec.
If the device has to operate in the full 3.0V to 5.25V voltage range, then an external level shifter with an external 3.3V supply has to be used.