No NVIDIA Runtime Found in Docker on Ubuntu 24.04
I wanted to run a local LLM using Docker to deploy Ollama and Open WebUI. Since my machine has a GPU, I documented the steps to integrate Docker (containers) with the GPU. Conclusion Avoid using Docker Desktop. Switching to Docker Engine resolved the issue. This post skips over the installation of NVIDIA drivers and the NVIDIA Container Toolkit (formerly NVIDIA Container Runtime). Failure Case: Using Docker Desktop Initially, I installed Docker Desktop. ...