Well, with that budget you could build a great 2-channel system or you could begin building a good 5.1 system that would also do a fabulous job with 2-channel audio. If you want to build a separate 2-channel system, I'd recommend getting an integrated amp (for under $1k), which is like a receiver for 2-channel audio but without the tuner which you could always add on via the used market. And then you'd have enough money to get a good CD player and quality kick-ass speakers. Are you looking to get bookshelf or floorstanding speakers?

For a 5.1 system, you could get a good receiver in the neighboorhood of $750, and you may wish to build the speakers a bit at a time, finding those speakers that please you best for music and movies and then find a matching center (the 3 front speakers being the most important). Quality subwoofers usually begin at $500 and can be added last, depending on how far your budget can be stretched. A good universal DVD player can be had between $100 and $300 and will all usually do a good job with CDs.

The most exciting part is getting out there and familiarizing yourself with quality audio brands outside the big box stores and give a listen to what it all sounds like.