Unless you know for a fact that plugging this port is encouraged (preferrably by the manufacturer), I'd take it out....you can do it on most speakers and get away with it until you crank the volume up really high, and on some even that won't hurt anything, but if you don't know 100%, I'd err on the side of caution. Besides, the speaker wasn't meant to be listened to with a port plugged, or it would have had a sealed cabinet to begin with.

Can you get inside the cabinet? Fixing a port isn't that hard.