Thanks everyone for the quick response.
A good part of my collection is 320kbps mp3 files. So what I understand is it might make a significant SQ difference if I use a USB DAC like dragonfly or schiit modi.
BTW, I do have a lot of old CDs, especially western classical. So the 6003 player may not be a bad idea :-$