Platform aggregators
Whenever a new digital technology enters the stage, the same scenario plays out.
- There are no service providers at all and — if you want to join the party — you have to deal with the technology directly. As a business that usually means hiring domain experts and engineers, high barrier of entry, high risk, etc. but also IP, competitive advantage, and first-mover advantage.
- Service providers appear, abstracting away technical challenges and offering convenient access to the technology without the hassle, leaving you with the agony of choice, or with the burden to deal with all of them. That usually also means more competition and lower margins.
- Finally, a platform emerges that aggregates all the service providers and gives you convenient access to all their products and services through a universal interface. Even more competition and even lower margins. The end users win big.
We all have seen this process unfold many times, with Amazon for online shopping, Netflix for video streaming, Stripe for online payments, Twilio for text messaging, and Alchemy for web3. These service providers are usually referred to as aggregators.
Generative AI and Large Language Models (LLMs) are in stage 2 right now (as of October 2023).
OpenAI was the first player to establish a viable end consumer product with ChatGPT and to onboard a large audience to their product (>$1B in annual revenue according to Reuters). By offering access to their flagship LLMs via API request to the OpenAI platform, they also opened the door for startups to build on top of their game-changing technology.
And building they did. If you want to get a grasp of how many GPT-powered AI products are already out there, just open one of the gazillion AI app marketplaces or listing sites, e.g. futuretools.ai.
That was a few month ago. Now, we have the choice between multiple providers, Anthropic, Cohere, Gemini, Hugging Face, and Replicate, just to name a few. It seems like a new AI platform is popping out of the ground every other week, each with their own strengths and weaknesses, each with their own set of features, and each with their own quirks and problems to deal with.
When building an AI-powered product today, you likely want to leverage multiple platforms to get the best performance, or at least experiment with different options to see which one works best for your use case. Chances also are that whatever provider or model you choose today, a better one will likely come around in a couple of weeks or months and it is always a smart move to avoid vendor lock-in.
It's time for stage 3.
Cross-platform solutions for Generative AI
Let's look at some of the solutions that are already out there. They each tackle a different aspect of the problem and focus on a different target audience.
Langchain: Libraries for Prompt Templating and LLM App Development
Langchain is a set of open-source Python- and JS/TS libraries that abstract away the differences between the various LLM platforms on a code-level and give users a unified developer experience when compiling prompt templates into API calls for different providers.
The startup implemented the first viable solution to reduce the challenges posed by phase 2 of the scenario outlined above and now powers many higher-level applications in the field of generative AI.
Langchain is a great tool for AI engineers and software developers that know how to code and feel home in Jupyter notebooks.
LiteLLM: A Python SDK for LLM API Abstraction and Monitoring
LiteLLM takes the abstraction a step further and maps the different LLM SDKs (including model settings, aso.) into a single open-source Python SDK compatible with the OpenAI standard. This makes integration of multiple AI platforms into an application a whole lot easier. Simply select the name of the model you want to use for prompt completion and you're good to go.
LiteLLM doesn't implement its own templating helpers, but integrates seamlessly with Langchain. On top of the SDK abstraction, LiteLLM also offers request monitoring and budgeting features.
The premise of LiteLLM is to help developers build cross-platform LLM applications faster and cheaper. Just like Langchain, LiteLLM is aimed at AI engineers and software developers.
Promptmetheus: A cross-platform Prompt Engineering IDE
Promptmetheus is a no-code prompt engineering IDE that leverages LiteLLM to connect to most of the established LLM providers. You can find a detailed list of model availability, token pricing, and relevant properties in our LLM Index. In contrast to Langchain and LiteLLM, Promptmetheus is not aimed at developers, but at Prompt Engineers and non-technical users that want to automate their processes and workflows with AI. That is not to say though, that if you know how to code, that it will not be useful to you.
Next to a meticulously designed UI, our cloud-native app features
- Prompt composability via blocks
- Prompt variables
- Test datasets
- Prompt history and full traceability
- Cost estimation for completions
- Output evaluation and prompt performance statistics
- Real-time collaboration
- and much more...
The goal of Promptmetheus is to segregate Prompt Engineering from software development and to open the field up to a broader audience.
Take a look at the "Building a Prompt Engineering IDE (VS Code for AI)" post for a more detailed introduction, or just give it a try. And if you do, don't forget to send some feedback our way.
Thanks for reading, catch you later...