When I run word_language_model with python main.py --cuda --nlayers 1 I am getting:

 | epoch   1 |   400/ 1327 batches | lr 20.00 | ms/batch 25.91 | loss  6.07 | ppl   431.54

While when I run it with the CuDNN RNN I get:

| epoch   1 |   400/ 1327 batches | lr 20.00 | ms/batch  9.39 | loss  6.06 | ppl   429.87

This is more than 2x faster. Without JIT enabled, I get:

| epoch   1 |   400/ 1327 batches | lr 20.00 | ms/batch 31.51 | loss  6.07 | ppl   431.49

So JIT is faster but not by much. I remember us having better results a few weeks ago on essentially the same problem. What is going on?