Latency regarding M-Audio Audiophile 2496 and Line 6 POD 2.0

Strange one this… have you tried uninstalling the SB drivers first before you physically remove it?
Also i used to have a delta 1010… same family same drivers… Are you using your pod with the pod ASIO i.e. physically switching the ASIO driver over and using the pod as an actual interface or just connecting the pod to your Audiophile?
You would do well to DUMP the SB ASAP mate… they are dreadful! the Audiophile is a pretty decent card for the money!
Avoid using the pod as an actual interface too… they are hopeless as interfaces! i have an X3Pro and it is useless as an interface, quite comical to watch though lol. If you have SP/dif out on your pod i would connect it to your Audiophile that way… is the way i do it here and it works beautifully!
That interface should run at 128 samples standing on it’s head! I think the latency you were getting sounds like as has been suggested using the wrong ASIO driver OR possibly the latency you could expect from the pod if using it as your ASIO interface… they really are hopeless as an interface… brilliant as an amp/fx sim which is what their main function is.