Leveling/Unleveling speed of Specializations

So went to go play around, picked up an obsidian potion of guidance and leveled two specializations right quick. Cool. Went to go swap one for a different one (to go play with), and im going to be beating on a training dummy for the next hour+. Not sure if the obsidian potion should also increase delevel speed, if the delevel speed in general for these needs to be tweaked (as players level/delevel and find the ones they actually want).

I went from 80-75 just autoattacking in the time it took for the game to think I was idle and put my weapon away. Doesn't feel good imo.