You can use around 6-8 NiMH batteries in series behind a low noise LPS like LM317 regulator. It is just Rs 210 or so. Replace the capacitor with a better one from Keltron, adjust the trimmer to give you 5.2 v and you will be done.
Regarding Shanti, you many be getting more noise compared to NiMH because Shanti is connected to Mains. The Shanti has better low noise regulator than LM317. Try connecting a thick wire from the Shanti earth terminal to the earth terminal of your usbridge (both have earth terminals). That will eliminate any ground loop that you may be having in your setup.
I'm using shanti with a thick wire connecting my shanti, usbridge, allo revolution dac (all have the earth terminal) and my Preamp. It totally eliminates ground noise.
There is another option. Open your Shanti. Connect your NiMH battery pack output to the rectified output inside Shanti. There will be lot of molten wax inside Shanti. You will have to remove it if you want to attach the battery pack on the top. But the bottom of the PCB is accessible. You could connect your battery pack to the bottom. With a bit of ingenuity, you could make a hole in the face plate, put a DC socket that connects to the rectified output. In that case you can connect the battery plug to the shanti using a DC barrel plug allowing you to use the SHanti without the mains.
You will require some DIY skills, circuit tracing skills, Drilling skill and soldering skill to pull this off. Your post has inspired me to try this. Just need to kick my lazy ass into action. From your replies in other posts, I believe you have good electrical knowledge
View attachment 88032