I’m not sure why you are setting ‘Direct Monitoring’ in Nuendo.
I’m using an RME UFX/Totalmix with Nuendo and it works fine. All direct monitoring of inputs is set in headphone outputs in TotalMix. The pre-recorded audio in Nuendo goes through TotalMix to the cans along with the audio channels being recorded. No latency. All this can be done in TotalMix with no special settings in Nuendo.
If you are new to TotalMix there is a bit of a learning curve, but it’s great when you get it - it took me a while.