I did some investigation into this, and it appears that when you set a destination, it does some smoke & mirrors - it still creates a completley new track in the project; it simply tries to place it in the same location - however it’s underlying ID is different, and that totally messes with all visibility configs etc.
So even though you pick a destination track/bus, the process doesn’t apply settings to that track, it imports a new one and then tries to make that imported track look like it’s the same - the channel color not being applied is just one of the things (along with visibility assignments / generic remote and probably others) that is not correctly retained from the destination track you selected.
It makes import to destination even worse than new - it’s actually easier to import new tracks, move all the audio and settings into the destination, then removew the imported tracks IMO, as at least you know exactly what has been done.