Use NVIDIA GPUs directly from Go.

In development. Source is private, available for technical review on request. The next public release is in preparation.

• Compile and launch kernels. Use the CUDA driver for memory, streams, events, and kernel launches. NVRTC compiles CUDA code at runtime; nvJitLink links device code.
• Call the math libraries. Work with cuBLAS, cuBLASLt, cuDNN, cuFFT, cuSPARSE, and cuSOLVER through Go bindings.
• Connect GPUs. Use NCCL collectives and related communication libraries for multi-GPU workloads.
• Measure the work. Collect device telemetry with NVML and profiling data with CUPTI. Higher-level packages support capture, analysis, and comparison.

The bindings are generated with libgen and load native libraries dynamically. The Go build needs no C toolchain or vendor headers. Running a GPU operation uses the installed NVIDIA libraries, driver, and hardware.

Direct bindings follow the native APIs. Higher-level packages under nvidia/x provide helpers for device contexts, memory, graphs, and profiling. Their APIs are still evolving.

A slow GPU workload can spend its time waiting for the host, executing kernels, or waiting for another rank. The diagnostic packages follow Go workload identity into device observations and keep those timing channels separate.

• Retain the incident. A flight recorder keeps recent host and GPU observations in memory, then captures a bundle within configured byte, duration, and concurrency limits.
• Explain the delay. Trace contracts check the observations required for a finding. Missing scheduler events, dropped records, or uncertain clocks can leave the cause unresolved.
• Inspect collective stalls. The NCCL Inspector reader retains rank, communicator, and operation identity to compare host enqueue, device activity, and completion for two-rank observations.
• Reconstruct a trial. Regression plans pin source, binaries, libraries, and inputs by digest. Interleaved baseline and candidate runs use null controls to estimate the noise floor.

Development branches now connect incident capture, diagnostic findings, and regression planning. These experimental packages are still being qualified on their intended hardware; their APIs may change.

Read the build note: from a GPU incident to a repeatable test ↗

mlx-go provides the array runtime for inference and training, including selected Linux CUDA workflows. gputrace brings trace inspection and comparison to Apple Metal.