Crooked Shank - Main Quest does not progress

I found the key to the elven cave and opened the gate there before I returned the house key to the mayor.

Now the task "Enter Locked Elven Tunnel" won't progress after I enter the tunnel, even after speaking to the elf children. I am unable to use the Brittle Crook Shank Elven Key in my inventory as the gate is already unlocked.

 

After relogging the tunnel door is locked again and I can use the key to unlock it. However, there is still no quest progression happening after I enter the tunnels and talk to the elf children.