There are not a ton of classes with spellcasting restrictions, and those are either less mana or higher spell cost...
What if there were a monster class with a different casting system. Maybe a monster that uses spell gems, that you produce when you level up or convert items. They would either fill your inventory (small, non-stacking or low-stack-capacity items) or a fixed number of slots that are in the place of your mana bar. To cast a spell, you use up one spell gem.