baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
1092 commits 1 branch 0 tags 2.1 MiB
Commit graph

1092 commits

This branch
This branch
All branches
Author SHA1 Message Date
Pradyumna Singh Rathore
d010ea70d2
Fixed Cuda Dockerfile 
Previously models produced garbage output when running on GPU with layers offloaded.

Similar to related fix on another repo: 331326a0e3
 2023-08-10 20:41:34 +05:30
MeouSker77
88184ed217 fix CJK output again 2023-08-09 22:04:35 +08:00
Andrei Betlen
66fb0345e8 Move grammar to function call argument 2023-08-08 15:08:54 -04:00
Andrei Betlen
1e844d3238 fix 2023-08-08 15:07:28 -04:00
Andrei Betlen
843b7ccd90 Merge branch 'main' into c0sogi/main 2023-08-08 14:43:02 -04:00
Andrei Betlen
bf0c603c51 Merge branch 'main' into fix-on-m1 2023-08-08 14:38:35 -04:00
Andrei Betlen
36041c8bec Merge branch 'main' of github.com:abetlen/llama_cpp_python into main 2023-08-08 14:35:10 -04:00
Andrei Betlen
d015bdb4f8 Add mul_mat_q option 2023-08-08 14:35:06 -04:00
Andrei
dcc26f7f78
Merge pull request #573 from mzen17/spelling-error-patch 
Fix typo "lowe-level API" to "low-level API" in the README
 2023-08-08 14:32:34 -04:00
Andrei Betlen
f6a7850e1a Update llama.cpp 2023-08-08 14:30:58 -04:00
Andrei
1dd4774ca9
Merge pull request #583 from abetlen/dependabot/pip/mkdocs-1.5.2 
Bump mkdocs from 1.5.1 to 1.5.2
 2023-08-08 14:24:40 -04:00
Andrei
03e575f6a5
Merge pull request #584 from abetlen/dependabot/pip/fastapi-0.101.0 
Bump fastapi from 0.100.1 to 0.101.0
 2023-08-08 14:24:31 -04:00
dependabot[bot]
83f8438c4f
Bump fastapi from 0.100.1 to 0.101.0 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.100.1 to 0.101.0.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.100.1...0.101.0)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-07 20:10:12 +00:00
dependabot[bot]
4cf2fc7d3d
Bump mkdocs from 1.5.1 to 1.5.2 
Bumps [mkdocs](https://github.com/mkdocs/mkdocs) from 1.5.1 to 1.5.2.
- [Release notes](https://github.com/mkdocs/mkdocs/releases)
- [Commits](https://github.com/mkdocs/mkdocs/compare/1.5.1...1.5.2)

---
updated-dependencies:
- dependency-name: mkdocs
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-07 20:09:55 +00:00
c0sogi
0d7d2031a9 prevent memory access error by llama_grammar_free 2023-08-07 17:02:33 +09:00
c0sogi
b07713cb9f reset grammar for every generation 2023-08-07 15:16:25 +09:00
c0sogi
418aa83b01 Added grammar based sampling 2023-08-07 02:21:37 +09:00
Mike Zeng
097fba25e5
Fixed spelling error "lowe-level API" to "low-level API" 2023-08-05 02:00:04 -05:00
c0sogi
ac188a21f3 Added low level grammar API 2023-08-05 14:43:35 +09:00
bretello
9f499af6b0
Update llama.cpp 2023-08-03 18:25:28 +02:00
bretello
39978ccaf5
add mul_mat_q parameter 
This also fixes a crash when loading the 70b llama2 model on MacOS with
metal and `n_gpu_layers=1`
 2023-08-03 18:24:50 +02:00
Ihsan Soydemir
a5bc57e279
Update README.md 2023-08-03 16:49:45 +02:00
Ihsan Soydemir
d4844b93ae
Merge pull request #2 from Isydmr/isydmr/docker-improvements 
Docker improvements
 2023-08-03 16:42:27 +02:00
Ihsan Soydemir
cdab73536b
Docker improvements 2023-08-03 16:36:50 +02:00
Huge
60e85cbe46
Fix dev setup in README.md so that everyone can run it 2023-08-02 12:27:08 +02:00
Andrei
91bf8fac1b
Merge pull request #550 from abetlen/dependabot/pip/uvicorn-0.23.2 
Bump uvicorn from 0.23.1 to 0.23.2
 2023-08-01 17:39:25 -04:00
dependabot[bot]
0cc8d8282a
Bump uvicorn from 0.23.1 to 0.23.2 
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.23.1 to 0.23.2.
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/uvicorn/compare/0.23.1...0.23.2)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-01 21:25:18 +00:00
Andrei
5011ef4fc6
Merge pull request #549 from abetlen/dependabot/pip/mkdocs-material-9.1.21 
Bump mkdocs-material from 9.1.19 to 9.1.21
 2023-08-01 17:24:04 -04:00
Andrei
7fb4f0bda2
Merge pull request #548 from abetlen/dependabot/pip/fastapi-0.100.1 
Bump fastapi from 0.100.0 to 0.100.1
 2023-08-01 17:23:45 -04:00
dependabot[bot]
ecb72cc0a2
Bump mkdocs-material from 9.1.19 to 9.1.21 
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.19 to 9.1.21.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.19...9.1.21)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-31 20:32:04 +00:00
dependabot[bot]
4a2f41a803
Bump fastapi from 0.100.0 to 0.100.1 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.100.0 to 0.100.1.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.100.0...0.100.1)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-31 20:31:16 +00:00
Andrei Betlen
ce57920e60 Suppress llama.cpp output when loading model. 2023-07-28 14:45:18 -04:00
Andrei Betlen
a9b9f0397c Format 2023-07-28 01:53:08 -04:00
Andrei Betlen
abc538fcd5 fix: annoying bug where attribute exceptions were droining out file not found exceptions 2023-07-28 01:43:00 -04:00
Andrei
d89bf8c072
Merge pull request #536 from Isydmr/patch-2 
Fix OpenBLAS Docker build
 2023-07-28 01:37:23 -04:00
Ihsan Soydemir
3e77eea7ec
Fix OpenBLAS Docker build 
Current build produces the following:
`RuntimeError: Failed to load shared library '/usr/local/lib/python3.11/site-packages/llama_cpp/libllama.so': /usr/local/lib/python3.11/site-packages/llama_cpp/libllama.so: undefined symbol: cblas_sgemm`
 2023-07-27 19:44:15 +02:00
Andrei
ffa81218f6
Merge pull request #531 from abetlen/dependabot/pip/uvicorn-0.23.1 
Bump uvicorn from 0.22.0 to 0.23.1
 2023-07-26 23:46:57 -04:00
dependabot[bot]
583d63351a
Bump uvicorn from 0.22.0 to 0.23.1 
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.22.0 to 0.23.1.
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/uvicorn/compare/0.22.0...0.23.1)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-27 03:33:07 +00:00
Andrei
468fccbed7
Merge pull request #532 from abetlen/dependabot/pip/black-23.7.0 
Bump black from 23.3.0 to 23.7.0
 2023-07-26 23:31:48 -04:00
dependabot[bot]
ecdfe4fbd3
Bump black from 23.3.0 to 23.7.0 
Bumps [black](https://github.com/psf/black) from 23.3.0 to 23.7.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/23.3.0...23.7.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-26 19:08:28 +00:00
Andrei
6e6c275c94
Merge pull request #530 from abetlen/dependabot/pip/mkdocs-material-9.1.19 
Bump mkdocs-material from 9.1.18 to 9.1.19
 2023-07-26 15:07:21 -04:00
dependabot[bot]
bfbbc8db14
Bump mkdocs-material from 9.1.18 to 9.1.19 
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.18 to 9.1.19.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.18...9.1.19)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-26 17:54:39 +00:00
Andrei
7debb505ab
Merge pull request #516 from charles-dyfis-net/poetry-no-versions-in-extras 
pyproject.toml: extras list should contain only package list, not versions
 2023-07-26 13:53:48 -04:00
Andrei
e665b557fd
Merge pull request #523 from shouyiwang/tensor_split 
Update tensor_split to match llama.cpp's change
 2023-07-26 13:53:02 -04:00
Andrei
d3bf7db2db
Merge pull request #525 from Isydmr/patch-1 
Fix typo in model path
 2023-07-25 16:31:02 -04:00
Ihsan Soydemir
0687a3092b
Fix typo in 70B path 2023-07-25 20:49:44 +02:00
Shouyi Wang
426dbfe3f4 Change tensor_split from array to pointer 2023-07-25 18:29:59 +10:00
Andrei Betlen
078902a6fe Add llama_grammar_accept_token 2023-07-24 15:55:26 -04:00
Andrei Betlen
bf901773b0 Add llama_sample_grammar 2023-07-24 15:42:31 -04:00
Andrei Betlen
1b6997d69f Convert constants to python types and allow python types in low-level api 2023-07-24 15:42:07 -04:00