From 4f8cf52a38761f8cd611d3f65f07b6fe382445a9 Mon Sep 17 00:00:00 2001 From: Andrei Betlen Date: Sun, 7 May 2023 05:20:04 -0400 Subject: [PATCH] Update README --- README.md | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9fa3bed..b7772d9 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ This package provides: - OpenAI-like API - LangChain compatibility -## Installation +## Installation from PyPI (recommended) Install from PyPI (requires a c compiler): @@ -26,8 +26,30 @@ pip install llama-cpp-python The above command will attempt to install the package and build build `llama.cpp` from source. This is the recommended installation method as it ensures that `llama.cpp` is built with the available optimizations for your system. -This method defaults to using `make` to build `llama.cpp` on Linux / MacOS and `cmake` on Windows. -You can force the use of `cmake` on Linux / MacOS setting the `FORCE_CMAKE=1` environment variable before installing. + +### Installation with OpenBLAS / cuBLAS / CLBlast + +`llama.cpp` supports multiple BLAS backends for faster processing. +Use the `FORCE_CMAKE=1` environment variable to force the use of `cmake` and install the pip package for the desired BLAS backend. + +To install with OpenBLAS, set the `LLAMA_OPENBLAS=1` environment variable before installing: + +```bash +LLAMA_OPENBLAS=1 FORCE_CMAKE=1 pip install llama-cpp-python +``` + +To install with cuBLAS, set the `LLAMA_CUBLAS=1` environment variable before installing: + +```bash +LLAMA_CUBLAS=1 FORCE_CMAKE=1 pip install llama-cpp-python +``` + +To install with CLBlast, set the `LLAMA_CLBLAST=1` environment variable before installing: + +```bash +LLAMA_CLBLAST=1 FORCE_CMAKE=1 pip install llama-cpp-python +``` + ## High-level API