I've done it twice (my dad got me some records from Scotland). London I don't know but i can tell you you a few shops in Edinburgh (If you're going to London, then it might be worth visiting Edinburgh. It takes 4-5 hours by train. Amazing place apparently!)
-Unknown Pleasures, 110 Canongate, EH8 8DD
-Vinyl Villians, 5 Elm Row, EH7 4AA
-Backtracks Music, 17 Brougham Street, EH3 9SJ.
-Fopp Records, Rose Street, EH2 2PR
These places have an amazing collection of jazz, pop, hard rock, progressive rock, jazz fusion, etc. Fopp has a good collection of metal as well. The prices are very reasonable (except for Fopp maybe which mainly sells new sealed LPs).
Apart from these there are many other shops that sell second hand records (in great condition!) in Edinburgh.