An open-weights model is a Large Language Model (LLM) whose parameters (or "weights") are publicly accessible and can be used and modified without restriction.
Unlike closed or Proprietary Models, open-weights models are often shared within the AI community for research, educational purposes, or to foster innovation. They can be fine-tuned or adapted to specific tasks and contribute to the transparency and collaborative advancement of AI technology. Open-weights models can also facilitate reproducibility in AI research, allowing others to validate and build upon existing work.
Open weights also allow independent deployment optimizations such as quantization, which can reduce memory requirements for local or lower-cost inference.
In contrast to Open-source Models, open-weights models do not provide access to the model architecture or source code for the training pipeline.
In an interview with Sequoia Capital, Andrej Karpathy used a nice analogy to software, saying "an open-weights model is a little bit like tossing over a binary for an operating system."
Examples of open-weights models are the LLaMA series by Meta AI and Mistral 7B / Mixtral 8x7B by Mistral AI.
The LLM Knowledge Base is a collection of bite-sized explanations for commonly used terms and abbreviations related to Large Language Models and Generative AI.
It's an educational resource that helps you stay up-to-date with the latest developments in AI research and its applications.