An open-weights model is a Large Language Model (LLM) whose parameters (or "weights") are publicly accessible and can be used and modified without restriction.
Unlike closed or Proprietary Models, open-weights models are often shared within the AI community for research, educational purposes, or to foster innovation. They can be fine-tuned or adapted to specific tasks and contribute to the transparency and collaborative advancement of AI technology. Open-weights models can also facilitate reproducibility in AI research, allowing others to validate and build upon existing work.
In contrast to Open-source Models, open-weights models do not provide access to the model architecture or source code for the training pipeline.
In an interview with Sequoia Capital, Andrej Karpathy used a nice analogy to software, saying "an open-weights model is a little bit like tossing over a binary for an operating system."
Examples of open-weights models are the LLaMA series by Meta AI and Mistral 7B / Mixtral 8x7B by Mistral AI.
Recent posts