The PLUGIN button (M3) toggles the instrument GUI when CHANNEL is active.
To toggle LEARN mode, you must NOT be in channel mode. It works with insert FX plugins and the relevant slot button must be active. You need to hold SHIFt when you press the data knob, otherwise it just toggles the button label view.
That said, I tend to use Quick Learn and avoid learn mode until I’m saving a map. Quick learn is achieved by holding SHIFT, moving a plugin parameter with the mouse, then turning a dial on CS12 to map (then turn data/Menu to color the dial).
If you are trying to map instrument parameters it’s a little different currently, the easiest way is to use the quick controls for which there are 8 automatically assigned controls on the CHANNEL home page. More detailed instrument mapping without using Nektarine is coming (soon!).