Could you please enable the Model and Core traces in the applications and share the logs with us?
-> Please try to delete all the temp files generated by the provisioner and try it again freshly.
-> May I know from which SDK you have taken lightness server/client applications? Also which is your provisioner?
-> Is there any node configured to work as Low Power Node?
-> Are you trying to remove any nodes from the network?
This can lead to initiate the key refresh procedure, and if the key refresh procedure stuck at phase 2 can actually cause the similar issue which you have described. At phase 2, the nodes will transmit using the new keys but can receive using the old keys and the new keys.