Tough call - if u plan on using your system for music more than surround sound movies - get the sub obviously, unless you have big, bass heavy towers. . If its 50/50 music and movies, and you are only worried about how it sounds to u, get the surronds next. Phantom mode is pretty shoddy at times when you aren't perfectly seated in the sweet spot, and even then it's not all that great.
That said, I believe the center channel will deliver the most benefit to sound quality, most of the time for multi-channel sources. Subs and surrounds don't receive nearly as muany signals as a center. It really takes the soundstage and imaging to a level that's not possible with 2 speakers.
I'm just not sure there's as much "fun" factor as a sub shaking the room or surrounds whizzing objects past your head. I suspect that's something you'll have to decide for yourself - there isn't a right or wrong - just different priorities.