Is therere no way to use my own VST library/3rd party plugins in Groove Agent inserts?

As the title asks

You load the inserts e.g. into the instrument track that is created when you load GA. If you load GA as a rack instrument, there is a folder track called Groove Agent and in there you load the inserts into the Master track. This is the same for all VST instruments.