EADST

Building for CPU

The CPU build is straightforward and works on any system with a modern C++ compiler. Here's how to do it:

cmake -B build
cmake --build build --config Release

Building with CUDA

If you have an NVIDIA GPU, you can build llama.cpp with CUDA support for significantly faster inference:

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j 32

-DGGML_CUDA=ON enables CUDA support

-j 32 enables parallel compilation with 32 threads to speed up the build process

Reference

Build llama.cpp locally