Auric, that was my exact first thought.

If those speakers aren't powered you probably aren't getting much sound from the headphone jack. If you have a stereo you are using the best way to get both sets of speakers working, assuming they are passive, is to get yourself a speakers switch box. If the speakers are nominal 8 ohm and the stereo amp section is capable of driving low impedance you could just hook both sets to the speaker terminals.

If the speakers are amplified just use a Y adaptor off the low level output.