If you encounter any issues encountered with using GPUs in env simulations, please bring it up on the developer forums at https://www.ray.io/

So much of this work with Ray is based on leveraging GPUs :)