Mixed impedances aren't going to create any specific problem, the real problem will be either with the power supply not having enough "oomph", or the output devices/heat sink not keeping up.

As long as you are expecting reasonable volumes, start with the switch in 8 ohm mode. If you have problems, switch to the 4 ohm mode. Please turn it off before making the switch over. All the switch does is limit the voltage to the output devices to keep them within their Safe Operating Area (SOA), the lower impedances will increase current draw, offsetting most of what would be "lost".