Local Deployment of Open Source Large Modeling Tools

General Introduction llm.pdf is an open source project that allows users to run Large Language Models (LLMs) directly in PDF files. Developed by EvanZhouDev and hosted on GitHub, this project demonstrates an innovative approach: by Em...

General Introduction Aana SDK is an open source framework developed by Mobius Labs, named after the Malayalam word "ആന" (elephant). It helps developers quickly deploy and manage multimodal AI models, supporting processing of text, images, audio and video, and other digital...

General Introduction BrowserAI is an open source tool that lets users run native AI models directly in the browser. It was developed by the Cloud-Code-AI team and supports language models like Llama, DeepSeek, Kokoro...

Comprehensive Introduction LitServe is an open source AI model service engine from Lightning AI, built on FastAPI, focused on rapidly deploying inference services for general-purpose AI models. It supports everything from large language models (LLMs), visual...

Comprehensive Introduction Nexa AI is a platform focused on multimodal AI solutions that run locally. It offers a wide range of AI models, including Natural Language Processing (NLP), Computer Vision, Speech Recognition and Generation (ASR and TTS), all of which can be used on local devices...

Comprehensive Introduction vLLM is a high-throughput and memory-efficient reasoning and service engine designed for Large Language Modeling (LLM). Originally developed by the Sky Computing Lab at the University of California, Berkeley, it has become an academic and industry-driven...

General Introduction Transformers.js is a JavaScript library provided by Hugging Face designed to run state-of-the-art machine learning models directly in the browser without server support. The library works in conjunction with the Hugging Fa...

Comprehensive Introduction Harbor is a revolutionary containerized LLM toolset focused on simplifying the deployment and management of local AI development environments. It enables developers with a clean command line interface (CLI) and companion application to launch and manage with a single click, including LLM backends, API interfaces, front...

Comprehensive Introduction Xorbits Inference (or Xinference for short) is a powerful and versatile library focused on providing distributed deployment and services for language models, speech recognition models, and multimodal models. With Xorbits Inference...

Comprehensive Introduction AI Dev Gallery is an AI development tools application from Microsoft (currently in public preview) designed for Windows developers. It provides a comprehensive platform to help developers easily integrate AI features into their Windo...

Comprehensive Introduction LightLLM is a Python-based Large Language Model (LLM) inference and service framework known for its lightweight design, ease of extension, and efficient performance. The framework leverages a variety of well-known open source implementations, including FasterTransfor...

General Introduction Transformers.js is a JavaScript library developed by Hugging Face to enable users to run state-of-the-art machine learning models directly in the browser without server support. The library works in conjunction with Hugg...

Comprehensive Introduction GLM-Edge is a series of large language models and multimodal understanding models designed for end-side devices from Tsinghua University (Smart Spectrum Light Language). These models include GLM-Edge-1.5B-Chat, GLM-Edge-4B-Chat, GLM...

综合介绍 Exo是一个开源项目，旨在利用日常设备（如iPhone、iPad、Android、Mac、Linux等）运行自己的AI集群。通过动态模型分区和自动设备发现，Exo能够将多个设备统一成一个强大...

综合介绍 LocalAI 是一个开源的本地AI替代方案，旨在提供与OpenAI、Claude等兼容的API接口。它支持在消费级硬件上运行，不需要GPU，能够进行文本、音频、视频、图像生成和语音克隆等多...

General Introduction llamafile is a tool from the Mozilla Builders project designed to simplify the deployment and operation of the Large Language Model (LLM). By combining llama.cpp with Cosmopolitan Libc ...

综合介绍 Petals 是一个由 BigScience Workshop 开发的开源项目，旨在通过分布式计算的方式运行大型语言模型（LLMs）。用户可以在家中使用消费级 GPU 或 Google Co...

综合介绍 Aphrodite Engine是PygmalionAI的官方后端引擎，旨在为PygmalionAI网站提供推理端点，并支持Hugging Face兼容模型的快速部署。该引擎利用vLLM的P...

General Introduction llama.cpp is a library implemented in pure C/C++ designed to simplify the inference process for Large Language Models (LLMs). It supports a wide range of hardware platforms, including Apple Silicon, NVIDIA GPUs, and AMD GPUs, and provides a variety of quant...