A graduate student who wired 32 GeForce cards into a gaming rig at Stanford ended up building the software layer the AI industry now runs on. Ian Buck went from that experiment to a programming language called Brook, funded in part by DARPA, and then to NVIDIA, which hired him in 2004 and paired him with architecture director John Nickolls. The result shipped publicly in 2007, according to the Wikipedia article on CUDA, which also records the first SDK release on 15 February 2007 for Windows and Linux.

What CUDA actually is

NVIDIA’s CUDA C++ Programming Guide calls CUDA a parallel computing platform and programming model that lets developers accelerate compute-intensive applications on the GPU. It is not one library. Wikipedia describes it as both a software layer giving direct access to the GPU and a set of APIs for parallel computation, shipped alongside drivers, compilers, libraries and developer tools. There are two API levels, a lower-level driver API and a higher-level runtime API.

The language is C, with support designed in for C++, Fortran, Python and Julia. That mattered early. Writing general-purpose GPU code before CUDA meant going through Direct3D or OpenGL, which assumed you knew graphics programming.

The libraries are the real lock-in

Writing a kernel in CUDA is portable in principle, and projects like HIPIFY exist to translate it. The harder dependency is the library stack above it, because framework authors call into it rather than writing their own kernels.

cuDNN is the clearest case. NVIDIA’s documentation describes it as a GPU-accelerated library of primitives for deep neural networks, with tuned implementations of the operations that turn up repeatedly in neural network code. NCCL, pronounced “Nickel”, provides inter-GPU communication primitives that are topology-aware. Its overview page is explicit that it is not a full parallel programming framework but a library focused on accelerating inter-GPU communication, implementing each collective in a single kernel that handles both communication and computation. It supports PCIe, NVLink, InfiniBand Verbs and IP sockets. TensorRT sits at the deployment end: NVIDIA describes it as an SDK for optimising deep learning inference that takes trained models from frameworks such as PyTorch and ONNX and compiles them into engines, with mixed precision from FP32 down to FP4 and INT4.

Replacing CUDA therefore means replacing cuDNN, NCCL and TensorRT too, or convincing every framework to call something else. NCCL is at least open source, mostly under Apache 2.0 with some parts retaining their original BSD licence, per its repository.

What the licence says

The CUDA Toolkit EULA, last updated 26 January 2026, grants a non-exclusive, non-transferable licence without the right to sublicense, covering installation and use, modification of sample source code, and distribution of the portions identified as distributable in object code inside an application. Distribution carries conditions. Your application must have material additional functionality beyond the included SDK portions, the distributable portions must only be accessed by your application, and modified sample code must carry the notice “This software contains source code provided by NVIDIA Corporation.”

Two restrictions matter for anyone thinking about compatibility layers. The agreement says you may not reverse engineer, decompile or disassemble any portion of the SDK. It then adds a second clause: you may not reverse engineer, decompile or disassemble any portion of the output generated using SDK elements for the purpose of translating such output artifacts to target a non-NVIDIA platform. The EULA also forbids using the SDK in a way that would subject it to an open source licence, and caps NVIDIA’s total cumulative liability at US$10.00.

Separately, the agreement permits using the SDK to develop and test applications released under licences approved by the Open Source Initiative.

This summarises the licence text. It is not legal advice.

FAQ

Is CUDA open source?

No. Wikipedia lists CUDA as proprietary, and CUDA-capable GPUs come only from NVIDIA. Some components around it are open, including NCCL, which its repository licenses mostly under Apache 2.0.

Does the CUDA licence ban porting my kernels to AMD?

The EULA restricts reverse engineering of SDK output in order to target a non-NVIDIA platform. It says nothing that stops you rewriting your own source for another platform, which is what HIPIFY-style porting does. Read the clause and take your own legal advice.

Which NVIDIA GPUs does CUDA support?

Wikipedia states that CUDA works with NVIDIA GPUs from the G8x series onwards, covering GeForce, Quadro and Tesla lines. Individual frameworks set higher floors, so check the runtime’s own requirements.

Do I need TensorRT to serve a model?

No. TensorRT is an inference compiler for squeezing more out of a deployment. Runtimes such as vLLM and llama.cpp use the CUDA toolkit and their own kernels instead.