Unfortunately at this present time, any GPU that would make a significant difference is likely to cost you much more than the price of SpectraLayers itself, so almost everything would be “too expensive”. While SpectraLayers does work with multiple brands, the leader is NVIDIA, and I’ve had good experience with even the RTX 3060 with 6GB (a laptop chip with no fans) using older versions (SL10, SL11).
With the current version (SL13) I’m using an RTX 5060 Ti 16GB which I was lucky enough to snag last year at ~$400, and I don’t think anything with less than 12GB would be even usable (some say 24GB). Unless you can find an NVIDIA card used (and not for mining!), you’re pretty much out of luck in terms of cost – at least, until the so-called “AI” bubble bursts, at which point they’ll be giving them away free.
FWIW, that is the exact card I’m using on my i9 285k system. I got it for $300 when I built my Windows 11 system last year. It seems to work well with the song unmix in SpectraLayers Pro 12, typically unmixing a song in roughly real time on my system. I’ve never tried doing that in ARA mode, only the standalone version.