In some ways, it's more satisfying to put together an inexpensive but musical system than it is with $5K! Just looking at the Audio Advisor catalog, here's a system that would satisfy:

Speakers - Wharfdale Diamond 8.3 - $350
Amp - Cambridge Azur integrated - $350
CD/SACD/DVD - Pioneer DV-563A -$179

That's $879 and you can use the cables that come with the components or spend $30 or so on speaker wire, etc. As critical as I think phono playback is, I wouldn't recommend it in new components for under $1K. With used gear, it's more than possible. But the above system would play music and most people would not find themselves wanting for anything... until they went to a friends house and heard vinyl! Then they'd have to at minimum invest $60 for a phono board on the amp and $300 for a Music Hall MMF-2 if they insisted on new gear. Total would be $1269 with phono capabilities.