[How To] CUDA 12.8 Ubuntu Setup for AI Workloads on Ubuntu 24.04

Performing a cuda 12.8 ubuntu setup on Ubuntu 24.04 is a critical first step for developers building high-performance AI and machine learning applications. NVIDIA’s CUDA (Compute Unified Device Architecture) provides a parallel computing platform and API that allows software to leverage the immense power of NVIDIA GPUs. In this guide, we will walk through the complete process of installing CUDA 12.8 on the latest Ubuntu LTS release, ensuring your system is optimized for AI workloads.

Table of Contents

• Prerequisites for CUDA Installation
• Step 1: System Preparation and Cleanup
• Step 2: Configure the NVIDIA Repository
• Step 3: Install CUDA 12.8 Toolkit
• Step 4: Configure Environment Variables
• Step 5: Verify the Installation
• Step 6: Install cuDNN (Optional)
• Best Practices for CUDA Development
• Conclusion

Prerequisites for CUDA Installation

Before you begin the installation process, ensure your hardware and software meet the following requirements to avoid common setup errors.

• NVIDIA GPU: A CUDA-capable GPU from NVIDIA (Pascal architecture or newer is recommended for AI).
• Ubuntu 24.04: A clean installation of Ubuntu 24.04 (Noble Numbat).
• Internet Access: Required to download the repository keys and toolkit packages.
• Sudo Privileges: You must have administrative access to the system.

Step 1: System Preparation and Cleanup

First, it is essential to prepare your system by removing any existing NVIDIA drivers or CUDA versions. Conflicts between old and new versions often lead to broken installations. Use the following commands to purge old NVIDIA-related packages.

lc-root@ubuntu:~$ sudo apt-get purge nvidia* -y
lc-root@ubuntu:~$ sudo apt-get autoremove -y
lc-root@ubuntu:~$ sudo apt-get autoclean -y

This ensures a clean slate for the cuda 12.8 ubuntu setup. After purging, update your local package list to ensure all dependencies are current.

Step 2: Configure the NVIDIA Repository

NVIDIA provides a dedicated repository for Ubuntu that includes the latest CUDA versions and compatible drivers. To use it, you must download a pin file to prioritize NVIDIA’s packages over the default Ubuntu ones and then install the keyring.

lc-root@ubuntu:~$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-ubuntu2404.pin
lc-root@ubuntu:~$ sudo mv cuda-ubuntu2404.pin /etc/apt/preferences.d/cuda-repository-pin-600
lc-root@ubuntu:~$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
lc-root@ubuntu:~$ sudo dpkg -i cuda-keyring_1.1-1_all.deb
lc-root@ubuntu:~$ sudo apt-get update

By using the official NVIDIA repository, you ensure that you receive the latest updates and security patches directly from the source.

Step 3: Install CUDA 12.8 Toolkit

Now that the repository is configured, you can install the specific CUDA 12.8 package. This meta-package includes the CUDA toolkit, the necessary libraries, and the latest compatible NVIDIA driver for your hardware.

lc-root@ubuntu:~$ sudo apt-get -y install cuda-12-8

The installation process may take several minutes depending on your internet speed. Once the process completes, it is highly recommended to restart your system to load the newly installed NVIDIA driver.

lc-root@ubuntu:~$ sudo reboot

Step 4: Configure Environment Variables

After the system reboots, you must add the CUDA binary and library paths to your environment variables. Without this step, your system will not recognize the nvcc compiler or the CUDA libraries when you attempt to run AI workloads.

lc-root@ubuntu:~$ echo 'export PATH=/usr/local/cuda-12.8/bin${PATH:+:${PATH}}' >> ~/.bashrc
lc-root@ubuntu:~$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc
lc-root@ubuntu:~$ source ~/.bashrc

These commands append the paths to your .bashrc file, making the changes persistent across all future terminal sessions.

Step 5: Verify the Installation

Verification is a critical part of the cuda 12.8 ubuntu setup. You should verify both the driver status and the CUDA compiler version to ensure everything is working correctly.

lc-root@ubuntu:~$ nvidia-smi
lc-root@ubuntu:~$ nvcc -V

The nvidia-smi command should display your GPU details and the driver version, while nvcc -V should confirm that CUDA release 12.8 is installed and ready for use.

Step 6: Install cuDNN (Optional)

For most AI workloads, such as running DeepSeek-R1 locally with Ollama or training models, you will also need cuDNN (CUDA Deep Neural Network library). This library provides highly tuned implementations for standard routines such as convolution and pooling.

lc-root@ubuntu:~$ sudo apt-get install libcudnn9-cuda-12

Integrating cuDNN ensures your AI models run with maximum efficiency on NVIDIA hardware. For more information on AI distributions, see our guide on top Linux distributions for AI and machine learning in 2026.

Best Practices for CUDA Development

To maintain a stable development environment, consider the following best practices for your CUDA and AI projects.

• Version Management: Use environment modules or Docker containers if you need to switch between different CUDA versions for different projects.
• Stay Updated: Regularly check for NVIDIA driver updates using the apt-get update command to ensure compatibility with new AI libraries.
• Testing: Always verify your setup with a simple CUDA sample or by running a basic AI model on Linux.
• Documentation: Refer to the official NVIDIA CUDA Installation Guide for advanced configuration options.

Conclusion

Successfully completing a cuda 12.8 ubuntu setup empowers you to utilize the full potential of your NVIDIA GPU for advanced AI development. By following this guide, you have established a robust foundation for building, training, and deploying sophisticated machine learning models on Ubuntu 24.04. Whether you are working with PyTorch, TensorFlow, or custom CUDA kernels, your system is now ready for the most demanding AI workloads.