A Rust, Python and gRPC server for text generation inference. Used in production at Hugging Face to power Hugging Chat, the Inference API and Inference Endpoint.

Extendable version manager with support for Ruby, Node.js, Elixir, Erlang & more ...