The B&W's you're using aren't the most sensitive speakers out there (I would describe B&W as "creative" when it comes to giving sensitivity measurements) and what you are describing sounds a normal enough for the 540A on speaker of that type. Do remember;
1) The 540A is happy to about 12 ish on the dial
2) If it does get flustered, it will simply turn itself down.