shimmy

Michael-A-Kuykendall/shimmy

Python-free Rust inference server for NLP models with OpenAI API compatibility and hot model swapping.

llama
llamacpp
llm-inference
ollama-api
command-line-tool
gguf
inference-server
local-ai

More in Rust

shimmytokMichael-A-Kuykendall/shimmytok

Pure-Rust tokenizer for GGUF models, compatible with llama.cpp tokenization.

LLMsNLP

25 updated 6d ago

huggingface/tokenizershuggingface/tokenizers

Fast State-of-the-Art Tokenizers optimized for Research and Production

Deep learningNLP

11.2k updated 1d ago

rust-bertguillaume-be/rust-bert

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

Deep learningNLP

3.1k updated 9mo ago

candlehuggingface/candle

Candle is a minimalist ML framework for Rust with a focus on performance (including GPU support) and ease of use.

General

21.2k updated 10d ago

burntracel-ai/burn

Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals

Deep learning

16.2k updated today

leafautumnai/leaf Deprecated

Open source framework for machine intelligence, sharing concepts from TensorFlow and Caffe. Available under the MIT license.

General

5.5k updated 2y ago

tch-rsLaurentMazare/tch-rs

Rust bindings for the C++ API of PyTorch

Deep learning

5.5k updated 2mo ago

linfarust-ml/linfa

A comprehensive toolkit to build Machine Learning applications with Rust

General

4.8k updated 2mo ago

All 17 in Rust