Appreciate if someone shed lights on this.
No, the peripherals as such do not show different behavior or performance depending on the core they are interacting with. The CPU's power mode does not impact the peripheral's performance as long as the system is in Active mode (i.e. at least one of the CPU is not in DeepSleep power mode). Even when both the CPUs are in Sleep mode, the peripherals remain active and function to their configuration/performance.
That said, M4 and M0+ use different interface (rather infrastructure) to interact with the peripherals (the MMIO register interface). M4 uses fast infrastructure and M0+ (and other bus masters like DMA, Crypto) use slow infrastructure for interfacing to the peripherals. In order to facilitate multiple master interface, arbiters (select the master interface to talk to) and bridges (sync's the clock to the peri clock domain) are provided in the device. Because of this you may observe some (read 'expected') performance reduction (depending on the number of active bus masters at any given time).
Let me know if this helps or if you need further clarification.
Meenakshi Sundaram R