baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
1145 commits 1 branch 0 tags 2.1 MiB
Commit graph

1145 commits

This branch
This branch
All branches
Author SHA1 Message Date
Billy Cao
c471871d0b
make n_gpu_layers=-1 offload all layers 2023-08-13 11:21:28 +08:00
Billy Cao
d018c7b01d
Add doc string for n_gpu_layers argument 2023-08-12 18:41:47 +08:00
Hannes Krumbiegel
17dd7fa8e0 Add py.typed 2023-08-11 09:58:48 +02:00
Pradyumna Singh Rathore
d010ea70d2
Fixed Cuda Dockerfile 
Previously models produced garbage output when running on GPU with layers offloaded.

Similar to related fix on another repo: 331326a0e3
 2023-08-10 20:41:34 +05:30
MeouSker77
88184ed217 fix CJK output again 2023-08-09 22:04:35 +08:00
Andrei Betlen
66fb0345e8 Move grammar to function call argument 2023-08-08 15:08:54 -04:00
Andrei Betlen
1e844d3238 fix 2023-08-08 15:07:28 -04:00
Andrei Betlen
843b7ccd90 Merge branch 'main' into c0sogi/main 2023-08-08 14:43:02 -04:00
Andrei Betlen
bf0c603c51 Merge branch 'main' into fix-on-m1 2023-08-08 14:38:35 -04:00
Andrei Betlen
36041c8bec Merge branch 'main' of github.com:abetlen/llama_cpp_python into main 2023-08-08 14:35:10 -04:00
Andrei Betlen
d015bdb4f8 Add mul_mat_q option 2023-08-08 14:35:06 -04:00
Andrei
dcc26f7f78
Merge pull request #573 from mzen17/spelling-error-patch 
Fix typo "lowe-level API" to "low-level API" in the README
 2023-08-08 14:32:34 -04:00
Andrei Betlen
f6a7850e1a Update llama.cpp 2023-08-08 14:30:58 -04:00
Andrei
1dd4774ca9
Merge pull request #583 from abetlen/dependabot/pip/mkdocs-1.5.2 
Bump mkdocs from 1.5.1 to 1.5.2
 2023-08-08 14:24:40 -04:00
Andrei
03e575f6a5
Merge pull request #584 from abetlen/dependabot/pip/fastapi-0.101.0 
Bump fastapi from 0.100.1 to 0.101.0
 2023-08-08 14:24:31 -04:00
dependabot[bot]
83f8438c4f
Bump fastapi from 0.100.1 to 0.101.0 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.100.1 to 0.101.0.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.100.1...0.101.0)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-07 20:10:12 +00:00
dependabot[bot]
4cf2fc7d3d
Bump mkdocs from 1.5.1 to 1.5.2 
Bumps [mkdocs](https://github.com/mkdocs/mkdocs) from 1.5.1 to 1.5.2.
- [Release notes](https://github.com/mkdocs/mkdocs/releases)
- [Commits](https://github.com/mkdocs/mkdocs/compare/1.5.1...1.5.2)

---
updated-dependencies:
- dependency-name: mkdocs
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-07 20:09:55 +00:00
c0sogi
0d7d2031a9 prevent memory access error by llama_grammar_free 2023-08-07 17:02:33 +09:00
c0sogi
b07713cb9f reset grammar for every generation 2023-08-07 15:16:25 +09:00
c0sogi
418aa83b01 Added grammar based sampling 2023-08-07 02:21:37 +09:00
Mike Zeng
097fba25e5
Fixed spelling error "lowe-level API" to "low-level API" 2023-08-05 02:00:04 -05:00
c0sogi
ac188a21f3 Added low level grammar API 2023-08-05 14:43:35 +09:00
bretello
9f499af6b0
Update llama.cpp 2023-08-03 18:25:28 +02:00
bretello
39978ccaf5
add mul_mat_q parameter 
This also fixes a crash when loading the 70b llama2 model on MacOS with
metal and `n_gpu_layers=1`
 2023-08-03 18:24:50 +02:00
Ihsan Soydemir
a5bc57e279
Update README.md 2023-08-03 16:49:45 +02:00
Ihsan Soydemir
d4844b93ae
Merge pull request #2 from Isydmr/isydmr/docker-improvements 
Docker improvements
 2023-08-03 16:42:27 +02:00
Ihsan Soydemir
cdab73536b
Docker improvements 2023-08-03 16:36:50 +02:00
Huge
60e85cbe46
Fix dev setup in README.md so that everyone can run it 2023-08-02 12:27:08 +02:00
Andrei
91bf8fac1b
Merge pull request #550 from abetlen/dependabot/pip/uvicorn-0.23.2 
Bump uvicorn from 0.23.1 to 0.23.2
 2023-08-01 17:39:25 -04:00
dependabot[bot]
0cc8d8282a
Bump uvicorn from 0.23.1 to 0.23.2 
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.23.1 to 0.23.2.
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/uvicorn/compare/0.23.1...0.23.2)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-01 21:25:18 +00:00
Andrei
5011ef4fc6
Merge pull request #549 from abetlen/dependabot/pip/mkdocs-material-9.1.21 
Bump mkdocs-material from 9.1.19 to 9.1.21
 2023-08-01 17:24:04 -04:00
Andrei
7fb4f0bda2
Merge pull request #548 from abetlen/dependabot/pip/fastapi-0.100.1 
Bump fastapi from 0.100.0 to 0.100.1
 2023-08-01 17:23:45 -04:00
dependabot[bot]
ecb72cc0a2
Bump mkdocs-material from 9.1.19 to 9.1.21 
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.19 to 9.1.21.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.19...9.1.21)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-31 20:32:04 +00:00
dependabot[bot]
4a2f41a803
Bump fastapi from 0.100.0 to 0.100.1 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.100.0 to 0.100.1.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.100.0...0.100.1)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-31 20:31:16 +00:00
Andrei Betlen
ce57920e60 Suppress llama.cpp output when loading model. 2023-07-28 14:45:18 -04:00
Andrei Betlen
a9b9f0397c Format 2023-07-28 01:53:08 -04:00
Andrei Betlen
abc538fcd5 fix: annoying bug where attribute exceptions were droining out file not found exceptions 2023-07-28 01:43:00 -04:00
Andrei
d89bf8c072
Merge pull request #536 from Isydmr/patch-2 
Fix OpenBLAS Docker build
 2023-07-28 01:37:23 -04:00
Ihsan Soydemir
3e77eea7ec
Fix OpenBLAS Docker build 
Current build produces the following:
`RuntimeError: Failed to load shared library '/usr/local/lib/python3.11/site-packages/llama_cpp/libllama.so': /usr/local/lib/python3.11/site-packages/llama_cpp/libllama.so: undefined symbol: cblas_sgemm`
 2023-07-27 19:44:15 +02:00
Andrei
ffa81218f6
Merge pull request #531 from abetlen/dependabot/pip/uvicorn-0.23.1 
Bump uvicorn from 0.22.0 to 0.23.1
 2023-07-26 23:46:57 -04:00
dependabot[bot]
583d63351a
Bump uvicorn from 0.22.0 to 0.23.1 
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.22.0 to 0.23.1.
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/uvicorn/compare/0.22.0...0.23.1)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-27 03:33:07 +00:00
Andrei
468fccbed7
Merge pull request #532 from abetlen/dependabot/pip/black-23.7.0 
Bump black from 23.3.0 to 23.7.0
 2023-07-26 23:31:48 -04:00
dependabot[bot]
ecdfe4fbd3
Bump black from 23.3.0 to 23.7.0 
Bumps [black](https://github.com/psf/black) from 23.3.0 to 23.7.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/23.3.0...23.7.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-26 19:08:28 +00:00
Andrei
6e6c275c94
Merge pull request #530 from abetlen/dependabot/pip/mkdocs-material-9.1.19 
Bump mkdocs-material from 9.1.18 to 9.1.19
 2023-07-26 15:07:21 -04:00
dependabot[bot]
bfbbc8db14
Bump mkdocs-material from 9.1.18 to 9.1.19 
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.18 to 9.1.19.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.18...9.1.19)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-26 17:54:39 +00:00
Andrei
7debb505ab
Merge pull request #516 from charles-dyfis-net/poetry-no-versions-in-extras 
pyproject.toml: extras list should contain only package list, not versions
 2023-07-26 13:53:48 -04:00
Andrei
e665b557fd
Merge pull request #523 from shouyiwang/tensor_split 
Update tensor_split to match llama.cpp's change
 2023-07-26 13:53:02 -04:00
Andrei
d3bf7db2db
Merge pull request #525 from Isydmr/patch-1 
Fix typo in model path
 2023-07-25 16:31:02 -04:00
Ihsan Soydemir
0687a3092b
Fix typo in 70B path 2023-07-25 20:49:44 +02:00
Shouyi Wang
426dbfe3f4 Change tensor_split from array to pointer 2023-07-25 18:29:59 +10:00