“he says that he’s hearing his playing coming back to him delayed” - the path from the Performer input back to cue (what Performer hears) is only possible if you have either engaged Track Monitor for a Performer track - but afaik this is blocked for tracks where VST Connect is inserted in the input channel - or, that you have some pre fader send or plugin in that path which routes to the Performer cue.
Pls check: when your drummer speaks into his mic, do you see the level of the VST Connect plugins’ CUE meter follow his signal? Then you definately have a routing issue.
If not, it is a problem on the Performer side, such as listening to direct monitoring at the same time, usually that’s a mix dial on the audio interface.