Play a CoreMIDI instrument without using VirtualMIDI?

When using a MIDI controller (via the camera connector) I’m not able to play a CoreMIDI instrument running in the background unless I use the “Virtual MIDI” port entry. Instruments such as iMini, Nave, DM1, Galileo, etc all show up in the output port of the MIDI connections – but no MIDI data gets to them. I can only hear them if I choose Virtual MIDI…which sucks because then it triggers every instrument in the background. Really we should be able to choose the instrument by name, and the track should trigger only the selected instrument.

Is this another MIDI oversight? Or do we always have to select Virtual MIDI and trigger everything at once?