LLM Index

Overview and comparison of available providers and models

You can find an overview of model parameter support for each available model in the LiteLLM docs.

If you are missing a provider or model that you would like to use inside PROMPTMETHEUS, please don't hesitate to request it. The goal is to support all relevant foundation models and eventually even custom models. Shoutout to Krrish and Ishaan from LiteLLM who make this possible.

Disclaimer: Even though we try to keep this list accurate and up-to-date, please do not rely on the presented information for critical use cases and always double check official sources.

How to choose the right LLM for my use case?

Unfortunately, there is no universal answer to this question. It depends on your use case, your budget, and your personal preferences. A good strategy to go about it is to do the following:

If you are new to the game, start with the industry standard, GPT-3.5 for simple tasks and GPT-4 for more complex ones. Use PROMPTMETHEUS to experiment until you get satisfying and reproducable results. If you do not get anywhere with OpenAI, try different providers. Once you have a working prompt, try to optimize it for performance, speed, reliability, and cost by comparing different LLMs and model parameters. As a rule of thumb, the cheapest model which is fast enough and does the job is the best one.

If you are more experienced with Prompt Engineering and AI Development, start with the model that historically worked best for use cases similar to the one you have at hand. But before you go into fine-tuning of your prompt, take an early version and execute it with a few different LLMs to see which one is most promising. Use that one to fine-tune your prompt. Once you achieve great results, revisit your model choice and optimize for performance, speed, and reliability.

Please also check the "Building a Prompt Engineering IDE" and "Prompt Engineering Tips & Tricks" posts for more information.