baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
802 commits 1 branch 0 tags 2.1 MiB
Commit graph

802 commits

This branch
This branch
All branches
Author SHA1 Message Date
Andrei Betlen
11eae75211 perf: avoid allocating new buffers during sampling 2023-07-07 19:28:53 -04:00
Andrei Betlen
7887376bff Update llama.cpp 2023-07-07 19:06:54 -04:00
Andrei Betlen
a14d8a9b3f perf: assign to candidates data structure instead 2023-07-07 18:58:43 -04:00
wu-qing-157
9e61661518 fix indexing token_logprobs after sorting 2023-07-07 10:18:49 +00:00
Audrey Roy Greenfeld
d270ec231a Update macOS Metal GPU step 4 
* Update "today" to version 0.1.62
* Fix numbering (there were 2 step 4's)
 2023-07-07 11:15:04 +01:00
Andrei Betlen
ca11673061 Add universal docker image 2023-07-07 03:38:51 -04:00
Andrei Betlen
57d8ec3899 Add setting to control request interruption 2023-07-07 03:37:23 -04:00
Andrei Betlen
cc542b4452 Update llama.cpp 2023-07-07 03:04:54 -04:00
Andrei Betlen
4c7cdcca00 Add interruptible streaming requests for llama-cpp-python server. Closes #183 2023-07-07 03:04:17 -04:00
Andrei Betlen
98ae4e58a3 Update llama.cpp 2023-07-06 17:57:56 -04:00
Andrei Betlen
a1b2d5c09b Bump version 2023-07-05 01:06:46 -04:00
Andrei Betlen
b994296c75 Update llama.cpp 2023-07-05 01:00:14 -04:00
Andrei
058b134ab6
Merge pull request #443 from abetlen/dependabot/pip/mkdocs-material-9.1.18 
Bump mkdocs-material from 9.1.17 to 9.1.18
 2023-07-05 00:40:46 -04:00
dependabot[bot]
9261a52916
Bump mkdocs-material from 9.1.17 to 9.1.18 
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.17 to 9.1.18.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.17...9.1.18)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-04 21:21:09 +00:00
Andrei
5e0a6b664d
Merge pull request #442 from abetlen/dependabot/pip/typing-extensions-4.7.1 
Bump typing-extensions from 4.6.3 to 4.7.1
 2023-07-04 17:19:36 -04:00
dependabot[bot]
f1b442337d
Bump typing-extensions from 4.6.3 to 4.7.1 
Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.6.3 to 4.7.1.
- [Release notes](https://github.com/python/typing_extensions/releases)
- [Changelog](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/python/typing_extensions/compare/4.6.3...4.7.1)

---
updated-dependencies:
- dependency-name: typing-extensions
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-04 18:22:53 +00:00
Andrei
2379ed5809
Merge pull request #444 from abetlen/dependabot/pip/fastapi-0.99.1 
Bump fastapi from 0.98.0 to 0.99.1
 2023-07-04 14:21:44 -04:00
Andrei
bf1dc6693b
Merge pull request #436 from mikeyang01/main 
Update README.md
 2023-07-04 14:21:30 -04:00
dependabot[bot]
fb02077e3f
Bump fastapi from 0.98.0 to 0.99.1 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.98.0 to 0.99.1.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.98.0...0.99.1)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-03 20:55:32 +00:00
Mike
c8d0647caa
Update README.md 
prevent not found errors
 2023-06-30 16:42:13 +08:00
Andrei
28ec88cfb4
Merge pull request #435 from vladkens/patch-1 
Update README.md
 2023-06-29 21:49:39 -04:00
vladkens
485eee7bef
Update README.md 
Fix installation link in readme
 2023-06-30 00:48:21 +03:00
Andrei Betlen
c67f786360 Update llama.cpp 2023-06-29 01:08:15 -04:00
Andrei Betlen
e34f4414cf Hotfix: logits_all bug 2023-06-29 00:57:27 -04:00
Andrei Betlen
4d1eb88b13 Bump version 2023-06-29 00:46:15 -04:00
Andrei Betlen
a2ede37bd5 Load logits directly into scores buffer 2023-06-29 00:45:46 -04:00
Andrei Betlen
b95b0ffbeb Use pre-allocated buffers to store input_ids and scores 2023-06-29 00:40:47 -04:00
Andrei Betlen
a5e059c053 Free model when llama is unloaded. Closes #434 2023-06-28 23:58:55 -04:00
Andrei Betlen
442213b070 Add stopping criteria and logits processor to docs 2023-06-28 21:07:58 -04:00
Andrei Betlen
a3766591bb Update docs 2023-06-27 13:02:30 -04:00
Andrei Betlen
530599a467 Merge branch 'main' of github.com:abetlen/llama_cpp_python into main 2023-06-27 12:45:33 -04:00
Andrei Betlen
dae983342a Update docs 2023-06-27 12:45:31 -04:00
Andrei
e2a251352c
Merge pull request #432 from abetlen/dependabot/pip/pytest-7.4.0 
Bump pytest from 7.3.2 to 7.4.0
 2023-06-26 23:40:48 -04:00
dependabot[bot]
89f9e435ba
Bump pytest from 7.3.2 to 7.4.0 
Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.3.2 to 7.4.0.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.3.2...7.4.0)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-06-26 23:26:11 +00:00
Andrei
2718dfbe3c
Merge pull request #430 from abetlen/dependabot/pip/fastapi-0.98.0 
Bump fastapi from 0.97.0 to 0.98.0
 2023-06-26 19:25:09 -04:00
dependabot[bot]
dcf1834294
Bump fastapi from 0.97.0 to 0.98.0 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.97.0 to 0.98.0.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.97.0...0.98.0)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-06-26 21:23:22 +00:00
Andrei
7d01749c8c
Merge pull request #429 from abetlen/dependabot/pip/mkdocs-material-9.1.17 
Bump mkdocs-material from 9.1.16 to 9.1.17
 2023-06-26 17:22:23 -04:00
dependabot[bot]
e03c3806f8
Bump mkdocs-material from 9.1.16 to 9.1.17 
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.16 to 9.1.17.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.16...9.1.17)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-06-26 21:16:53 +00:00
Andrei
e18fe74bd7
Merge pull request #431 from abetlen/dependabot/pip/numpy-1.24.4 
Bump numpy from 1.24.3 to 1.24.4
 2023-06-26 17:15:51 -04:00
dependabot[bot]
c9a8b7eb43
Bump numpy from 1.24.3 to 1.24.4 
Bumps [numpy](https://github.com/numpy/numpy) from 1.24.3 to 1.24.4.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](https://github.com/numpy/numpy/compare/v1.24.3...v1.24.4)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-06-26 21:03:34 +00:00
Andrei Betlen
452929404f Updated docs link 2023-06-26 16:35:38 -04:00
Andrei Betlen
66b8b979a5 Update readthedocs setup 2023-06-26 16:31:16 -04:00
Andrei Betlen
155dedf28f Add readthedocsc config 2023-06-26 16:25:17 -04:00
Andrei Betlen
5193af297b Bump version 2023-06-26 08:53:54 -04:00
Andrei Betlen
3379dc40a1 Merge branch 'main' of github.com:abetlen/llama_cpp_python into main 2023-06-26 08:50:48 -04:00
Andrei Betlen
952228407e Update llama.cpp 2023-06-26 08:50:38 -04:00
Andrei Betlen
b4a3db3e54 Update type signature 2023-06-26 08:50:30 -04:00
Andrei
628e3fb3df
Merge pull request #370 from Okabintaro/fix-state-pickle 
fix: Make LLamaState pickleable for disk cache
 2023-06-26 08:46:59 -04:00
Andrei
5eb4ebb041
Merge branch 'main' into fix-state-pickle 2023-06-26 08:45:02 -04:00
Andrei
04d9218b92
Merge pull request #420 from samfundev/main 
Only concatenate after all batches are done
 2023-06-26 08:10:43 -04:00