Can confirm this! I've tried training a couple of times with the above and it seems convergence is significantly slower than advertised with an avg. reward of around 50 @ 50K episodes.