TensorRT-LLM license, foundation and stars

Model serving and local AI · Custom or mixed (see the LICENSE file) (source-available) · backed by Nvidia · 15,033 stars

GitHubSuggest a correction

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

More in model serving and local ai

MIT186k starsOllamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.Ollama · +684 a week · v0.40.14, Nov 3, 2026MIT134k starsllama.cppLLM inference in C/C++.+655 a week · v0.5.0, Sep 23, 2026Apache 2.096.6k starsvLLMA high-throughput and memory-efficient inference and serving engine for LLMs.PyTorch Foundation · +562 a week · v0.30.0, Sep 22, 2026MIT77.4k starsgpt4allGPT4All: Run Local LLMs on Any Device.-5 a weekMIT55k starswhisper.cppPort of OpenAI's Whisper model in C/C++.+160 a weekMIT50.1k starsLocalAILocalAI is the open-source AI engine.+145 a weekApache 2.045.7k starsJanJan is an open source alternative to ChatGPT that runs 100% offline on your computer.Menlo Research · +146 a week · relicensed May 2025Apache 2.041.9k starsairllmAirLLM 2.8T LLM inference with single 4GB GPU.+1,139 a week

Project	License	Stars	Per week	Latest release
OllamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.OllamaMIT	MIT	186k	+684	v0.40.14Nov 3, 2026
llama.cppLLM inference in C/C++.MIT	MIT	134k	+655	v0.5.0Sep 23, 2026
vLLMA high-throughput and memory-efficient inference and serving engine for LLMs.PyTorch FoundationApache 2.0	Apache 2.0	96.6k	+562	v0.30.0Sep 22, 2026
gpt4allGPT4All: Run Local LLMs on Any Device.MIT	MIT	77.4k	-5	-
whisper.cppPort of OpenAI's Whisper model in C/C++.MIT	MIT	55k	+160	-
LocalAILocalAI is the open-source AI engine.MIT	MIT	50.1k	+145	-
JanJan is an open source alternative to ChatGPT that runs 100% offline on your computer.Menlo ResearchApache 2.0	Apache 2.0changed May 2025	45.7k	+146	-
airllmAirLLM 2.8T LLM inference with single 4GB GPU.Apache 2.0	Apache 2.0	41.9k	+1,139	-