# Promptmetheus > Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more. Canonical Origin: https://promptmetheus.com/ ## LLM Resources - [Full Content](https://promptmetheus.com/llms-full.txt): Complete page content in markdown format. --- - **Page:** Promptmetheus: Prompt Engineering IDE - **Source:** https://promptmetheus.com/ - **Description:** Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Prompt
Engineering
IDE Compose reliable prompts for your AI automations, agents, and workflows. Test prompt performance with 150+ LLMs, or configure your own. Optimize prompts for output quality, robustness, cost, and speed. [Get Started Open the Prompt IDE](https://archery.promptmetheus.com/register "Open the Prompt IDE") Join 35k+ users or [watch demo](https://www.youtube.com/watch?v=8IwMGpN3b-s "Watch the Promptmetheus Prompt Engineering Demo on YouTube") to see how it works Requires a screen with 12" or larger. Your browser does not support the video tag. We ship 15 providers out of the box, but you can also easily configure your own. ## Prompt
Composition Compose structured prompts from text and data blocks and rapidly iterate through versions with block variants. ## Automatic
Evaluators Configure custom evaluators to automatically validate completions against the specified requirements. ## Test
Datasets Organize dynamic context like user input or retrieved data into datasets and inject it into prompts via data blocks. ## Additional Features ### Projects Organize prompts, datasets, and completions into projects and track relevant statistics on the dashboard. ### Model Catalog Test your prompts with 150+ LLMs and experiment with model parameters for optimal results. ### Prompt Variables Define variables at project or prompt level to keep recurring details flexible and consistent. ### Completion Ratings Rate completion quality and visualize the results broken down by model and used variants. ### Full Traceability Trace every change in your prompt-design workflow with detailed versioning and changelogs. ### Cost Calculation Estimate inference costs for completions based on different models, inputs, and configurations. ### Real-time Sync Sync changes to your prompt library in real-time with other devices and team members. ### Data Export Export prompts and completions in .txt, .csv, .xlsx, or .json format. “There will be two kinds of businesses at the end of this decade: those who are fully utilizing AI, and those who are out of business.” — Peter Diamandis ![Prometheus Steals Fire from the Gods](https://promptmetheus.com/img/prometheus-steals-fire.webp) Apps Agents Workflows Automations ## Models The right LLM for every use case ### [Anthropic](https://www.anthropic.com/ "Anthropic website") Claude 5.5 Haiku, Sonnect, Opus Claude 5.1 Fable Claude 5 Sonnet, Opus, Fable Claude 4.8 Opus ### [DeepMind](https://gemini.google.com "DeepMind website") Gemini 4 Argon Gemini 3.8 Flash Gemini 3.1 Flash-Lite, Pro ### [OpenAI](https://openai.com "OpenAI website") GPT-6.1 Sol GPT-6 Luna, Sol, Astra GPT-5.6 Luna, Terra, Sol GPT-5.5 Base, Pro GPT-5.4 Base, Mini, Nano ### [Mistral](https://mistral.ai "Mistral website") Mistral 4 Small, Large Mistral 3.5 Medium Ministral 3 3B, 8B, 14B Codestral ### [Perplexity](https://www.perplexity.ai/ "Perplexity website") Sonar Base, Pro, Reasoning Pro, Deep Research ### [xAI](https://x.ai "xAI website") Grok 4.7 Grok 4.6 Grok 4.3 ### [DeepSeek](https://deepseek.com "DeepSeek website") V4.1 Flash V4 Pro ### [Cohere](https://cohere.com/ "Cohere website") Command A+ Command A Base, Reasoning Command R Base, 7B, + Aya Expanse 8B, 32B ### [Groq](https://groq.com "Groq website") Compound Base, Mini OpenAI GPT-OSS 20B, 120B Meta Llama 3.1 8B, 3.3 70B ### [FetchAI](https://www.fetch.ai/ "FetchAI website") ASI:One Mini, Fast, Extended ### [OpenRouter](https://openrouter.ai "OpenRouter website") Meta Muse Wonder Meta Muse Spark Open Z.ai GLM-5.3 Moonshot AI Kimi K3 DeepSeek V4.1 Flash Anthropic Claude 5.5 Haiku, Sonnet, Opus OpenAI GPT-6.1 Sol And more... ### [AI21 Labs](https://www.ai21.com/ "AI21 Labs website") Jamba 2 Mini, 3B Jamba 1.7 Large ### [Venice](https://venice.ai "Venice website") Venice Uncensored 1.1, 1.2 Moonshot AI Kimi K3 Z.ai GLM-5.2 Alibaba Qwen 3 Instruct, Thinking xAI Grok 4.5 Anthropic Claude 5 Sonnet, Opus, Fable And more... ### [Moonshot AI](https://www.moonshot.ai/ "Moonshot AI website") Kimi K3 Kimi K2.6 Kimi K2.5 ### [Deep Infra](https://deepinfra.com "Deep Infra website") NVIDIA Nemotron 3 Nano Omni, Ultra MiniMax AI MiniMax M2 DeepSeek V4 Flash, Pro Moonshot AI Kimi K2.6 DeepMind Gemma 4 26B, 31B Meta Llama 4 Scout 17B 16e, Maverick 17B 128e And more... Custom Connect any model that is compatible with the [OpenAI API](https://developers.openai.com/api/docs "OpenAI API documentation") or [LiteLLM SDK](https://docs.litellm.ai/docs "LiteLLM SDK documentation"). ## Pricing Simple pricing for individuals and teams of all sizes ### Playground FREE - Forge - **1** user - Local data storage - OpenAI models - Stats & Insights - Data import / export - Community support [Open Open FORGE playground](https://forge.promptmetheus.com "Open FORGE playground") ### Single $29 / month 7-day free trial - Prompt IDE - **1** user - Cloud sync between devices - 15 providers and 150+ models - Multiple projects - Automatic evaluators - Prompt history and full traceability - Stats & Insights - Data export - Dedicated support [Get started Sign up for a Standard account](https://archery.promptmetheus.com "Sign up for a Standard account") ### Team $99 / month - Prompt IDE - **3** users included - $19/month per additional user - All **Single** features, plus - User management - Shared workspace with real-time collaboration - Business support Get in touch Secure payments powered by [Stripe](https://stripe.com "Stripe website"). Subscriptions do not include a budget for inference, you need to provide your own API keys. For Enterprise plans and special requests, please get in touch. What is Prompt Engineering? What is a Prompt IDE? How is Promptmetheus different from the playgrounds provided by OpenAI, Anthropic, etc.? How is Promptmetheus different from other prompt engineering tools? Is there an API or SDK? Can I build AI agents with Promptmetheus? Can I use Promptmetheus together with LangChain, LangFlow, and other AI agent builders? What is the difference between Forge and Archery? What is an AIPI? Does Promptmetheus integrate with automation tools like Make, Zapier, IFTTT, and n8n? ## FAQ If you have any other questions, please get in touch. [Ask via Email Send us an email](mailto:contact@promptmetheus.com "Send us an email") [Ask on Discord Join the Discord server](https://discord.com/invite/7dbUb9wtGj "Join the Discord server") ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/#webpage","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus: Prompt Engineering IDE","url":"https://promptmetheus.com/","@type":["WebPage","FAQPage"],"about":{"@id":"https://promptmetheus.com/#identity"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"mainEntity":[{"@id":"https://promptmetheus.com/#/schema/question/1"},{"@id":"https://promptmetheus.com/#/schema/question/2"},{"@id":"https://promptmetheus.com/#/schema/question/3"},{"@id":"https://promptmetheus.com/#/schema/question/4"},{"@id":"https://promptmetheus.com/#/schema/question/5"},{"@id":"https://promptmetheus.com/#/schema/question/6"},{"@id":"https://promptmetheus.com/#/schema/question/7"},{"@id":"https://promptmetheus.com/#/schema/question/8"},{"@id":"https://promptmetheus.com/#/schema/question/9"},{"@id":"https://promptmetheus.com/#/schema/question/10"}],"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/#/schema/question/1","@type":"Question","inLanguage":"en-US","name":"What is Prompt Engineering?","acceptedAnswer":{"@type":"Answer","text":"Prompt Engineering is the craft of designing, testing, and optimizing prompts (instructions) to obtain reliable and cost-efficient completions from Large Language Models.

Typically, the goal is to leverage the capabilities of Artificial Intelligence to automate tasks and workflows that are too dynamic and/or complex for conventional software applications and API architectures."}},{"@id":"https://promptmetheus.com/#/schema/question/2","@type":"Question","inLanguage":"en-US","name":"What is a Prompt IDE?","acceptedAnswer":{"@type":"Answer","text":"The term IDE stems from Software Engineering and stands for Integrated Development Environment.

A Prompt IDE is very similar to a Code IDE (e.g. VS Code, PyCharm, etc.), but for Prompt Engineering. It provides the tools and functionality to help users compose, test, and optimize reliable LLM prompts faster and more efficiently."}},{"@id":"https://promptmetheus.com/#/schema/question/3","@type":"Question","inLanguage":"en-US","name":"How is Promptmetheus different from the playgrounds provided by OpenAI, Anthropic, etc.?","acceptedAnswer":{"@type":"Answer","text":"Testing and evaluating complex LLM prompts can be tedious in simple playgrounds because they completely lack traceability and the necessary features to efficiently experiment with different prompt variations. Promptmetheus plugs those holes with pro-grade tooling to make designing, testing, and optimizing reliable prompts easy."}},{"@id":"https://promptmetheus.com/#/schema/question/4","@type":"Question","inLanguage":"en-US","name":"How is Promptmetheus different from other prompt engineering tools?","acceptedAnswer":{"@type":"Answer","text":"Promptmetheus focuses primarily on the optimization of prompts by breaking them down into sections (blocks). Users can easily re-arrange blocks and test different variations, all while keeping track of the changes, cost, and performance."}},{"@id":"https://promptmetheus.com/#/schema/question/5","@type":"Question","inLanguage":"en-US","name":"Is there an API or SDK?","acceptedAnswer":{"@type":"Answer","text":"Not yet, but there are plans for a REST API, as well as SDKs for Python, JavaScript, Rust, Swift, and other popular programming languages."}},{"@id":"https://promptmetheus.com/#/schema/question/6","@type":"Question","inLanguage":"en-US","name":"Can I build AI agents with Promptmetheus?","acceptedAnswer":{"@type":"Answer","text":"Yes, but you'll need additional tools and infrastructure to assemble and run your agents. Promptmetheus only helps with design, testing, and optimization of individual prompts."}},{"@id":"https://promptmetheus.com/#/schema/question/7","@type":"Question","inLanguage":"en-US","name":"Can I use Promptmetheus together with LangChain, LangFlow, and other AI agent builders?","acceptedAnswer":{"@type":"Answer","text":"Yes. LangChain, LangFlow, etc. are great for the development and deployment of AI agents, but they are not designed to optimize the performance of each individual prompt in the chain.

While working on your agent, you can take each prompt and optimize it with Promptmetheus for better performance and lower cost."}},{"@id":"https://promptmetheus.com/#/schema/question/8","@type":"Question","inLanguage":"en-US","name":"What is the difference between Forge and Archery?","acceptedAnswer":{"@type":"Answer","text":"The two platforms look similar but are completely separate and not connected. While Forge runs completely offline and stores data exclusively in your browser, Archery is cloud-native and has many additional features like device sync, full traceability, real-time collaboration, etc.

Forge is currently in maintenance mode and new features will only land in the Archery IDE."}},{"@id":"https://promptmetheus.com/#/schema/question/9","@type":"Question","inLanguage":"en-US","name":"What is an AIPI?","acceptedAnswer":{"@type":"Answer","text":"An AI Programming Interface (AIPI) is similar to a conventional API, but instead of executing static code on a server, its endpoints mediate interactions with Large Language Models (LLMs) via hosted prompts."}},{"@id":"https://promptmetheus.com/#/schema/question/10","@type":"Question","inLanguage":"en-US","name":"Does Promptmetheus integrate with automation tools like Make, Zapier, IFTTT, and n8n?","acceptedAnswer":{"@type":"Answer","text":"Not directly. Official integrations for Make, Zapier, IFTTT, n8n, etc. are on the roadmap, but for now you'll have to copy prompts into your automation tool manually after optimizing and testing them with Promptmetheus."}},{"@id":"https://promptmetheus.com/#/schema/software-application/1","@type":"SoftwareApplication","applicationCategory":"DeveloperApplication","applicationSubCategory":"Prompt Engineering","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","inLanguage":"en-US","isAccessibleForFree":true,"name":"Promptmetheus Forge Playground","operatingSystem":"Windows, macOS, Linux","url":"https://forge.promptmetheus.com","featureList":["Prompt Composition","Prompt Variables","LLM Catalog","Test Datasets","Completion Ratings","Inference Cost Estimation","Data Import/Export"],"image":{"@id":"https://promptmetheus.com/#/schema/image/1"},"offers":{"@type":"Offer","availability":"https://schema.org/InStock","price":0,"priceCurrency":"USD","url":"https://forge.promptmetheus.com","priceValidUntil":"2027-12-30T00:00:00.000Z"},"publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/#/schema/software-application/2","@type":"SoftwareApplication","applicationCategory":"DeveloperApplication","applicationSubCategory":"Prompt Engineering","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","inLanguage":"en-US","isAccessibleForFree":false,"name":"Promptmetheus Prompt IDE","operatingSystem":"Windows, macOS, Linux","url":"https://archery.promptmetheus.com","featureList":["Prompt Composition","Prompt Variables","Prompt Evaluators","LLM Catalog","Test Datasets","Completion Ratings","Inference Cost Estimation","Full Traceability","Real-time Sync","Data Export"],"image":{"@id":"https://promptmetheus.com/#/schema/image/2"},"offers":[{"@type":"Offer","availability":"https://schema.org/InStock","name":"Single","price":29,"priceCurrency":"USD","url":"https://archery.promptmetheus.com","priceValidUntil":"2027-12-30T00:00:00.000Z"},{"@type":"Offer","availability":"https://schema.org/InStock","name":"Team","price":99,"priceCurrency":"USD","url":"https://archery.promptmetheus.com","priceValidUntil":"2027-12-30T00:00:00.000Z"}],"publisher":{"@id":"https://promptmetheus.com/#identity"},"screenshot":["https://promptmetheus.com/img/preview/preview-v5-dashboard-dark.jpg","https://promptmetheus.com/img/preview/preview-v5-prompt-dark.jpg"]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"},{"@id":"https://promptmetheus.com/#/schema/image/1","@type":"ImageObject","contentUrl":"https://promptmetheus.com/og-image.jpg","inLanguage":"en-US","url":"https://promptmetheus.com/og-image.jpg"},{"@id":"https://promptmetheus.com/#/schema/image/2","@type":"ImageObject","contentUrl":"https://promptmetheus.com/og-image.jpg","inLanguage":"en-US","url":"https://promptmetheus.com/og-image.jpg"}]} ``` --- - **Page:** Partnerships · Promptmetheus - **Source:** https://promptmetheus.com/partnerships - **Description:** Collaboration opportunities for Businesses, Consultants, Marketers, Universities, Developers, and Educators. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Partnerships Collaboration opportunities for Businesses, Events, Agencies, Consultants, Marketers, Universities, Developers, and Educators We also have an [Affiliate Program](https://promptmetheus.com/affiliate "Become a partner, get a link, share it, earn") to share revenue with our partners. ## Businesses If your business provides services or sells products that might be mutually beneficial to Promptmetheus users, we'd love to hear from you. Some examples are: Integrations The goal of Promptmetheus is to provide a complete suite of tools needed to develop AI applications, agents, and integrations. However, we do not intend to cover the entire feature set ourselves, but rather focus on the prompt design and optimization part and integrate with state-of-the-art solutions to offer the rest. Areas of high interest are currently vector stores, automation platforms, testing frameworks, and benchmarking tools. Let's give Promptmetheus users easy access to your tech! LLM providers It makes little sense for each LLM provider to develop their own playground and tooling. That would be analogous to each programming language having its own dedicated IDE. Especially for smaller players this would create a significant barrier to entry, which in turn is bad for consumers. We're happy to lift that burden off your shoulders, integrate your models, and provide your users with a sophisticated (and potentially customized) Prompt Engineering experience. [Get in touch Send an email to Promptmetheus](mailto:contact@promptmetheus.com "Send an email to Promptmetheus") --- ## Events If you are hosting events in the AI space (conferences, hackathons, workshops, etc.), we might be able to offer your attendees exclusive discounts for Promptmetheus products. The idea is a generate win-win-win for everyone: your attendees get a great deal for software they potentially need (value), you get to give them access to this deal at no cost (reputation), we get exposure to a highly targeted audience (marketing). Sounds good? Let's talk! [Get in touch Send an email to Promptmetheus](mailto:contact@promptmetheus.com "Send an email to Promptmetheus") --- ## Consultants If you are consulting for clients in the fields of AI and automation, Promptmetheus might be a great solution for you. The IDE supports real-time collaboration, full-traceability, and multiple workspaces, making it easy to share prompting work with your clients, review progress and completions, and much more. We are happy to explore dedicated solutions with you and potentially reward you for SME and enterprise clients you can introduce to Promptmetheus. If that sounds interesting, please reach out. [Get in touch Send an email to Promptmetheus](mailto:contact@promptmetheus.com "Send an email to Promptmetheus") --- ## Marketers Monetize your AI content with our [Affiliate Partner Program](https://promptmetheus.com/affiliate "Become a partner, get a link, share it, earn"). Please reach out to discuss opportunities. [Get in touch Send an email to Promptmetheus](mailto:contact@promptmetheus.com "Send an email to Promptmetheus") --- ## Schools and Universities Promptmetheus is an ideal tool for teaching and learning. The platform allows students to collaborate in teams and share prompts and workspaces with their peers and lecturers. The IDE is from the ground up designed for experimentation and keeps a full history of the entire prompt design process. There is hardly any credible doubt that Prompt Engineering is becoming an essential skill in virtually all fields. Students from around the world ([Berkeley](https://www.berkeley.edu/ "University of California, Berkeley"), [UMD](https://umd.edu "University of Maryland"), [UNSW](https://unsw.edu.au "University of New South Wales"), and others) are already using Promptmetheus to collaborate on amazing AI projects. Please get in touch to discuss the specific needs of your project or institution. We offer great discounts for educational use. [Get in touch Send an email to Promptmetheus](mailto:contact@promptmetheus.com "Send an email to Promptmetheus") --- ## Developers If you are working on exciting software that could benefit from the Prompt Engineering capabilities of Promptmetheus, please reach out to touch base. [Get in touch Send an email to Promptmetheus](mailto:contact@promptmetheus.com "Send an email to Promptmetheus") --- ## Educators Education is our favorite form of marketing. If you teach prompting, AI development, etc., have a decent audience (YouTube, Udemy, X, etc.), and like what Promptmetheus has to offer, let's work together! Instead of creating educational content on Prompt Engineering and how to use Promptmetheus ourselves, we'd rather spend our time improving the product and leave the former to professionals like you. If you can introduce new users to Promptmetheus at scale and are interested in partnering up, please get in touch. Note: In the future, me might directly fund content production, but at the moment we're only interested in sharing revenue from customer conversions. [Get in touch Send an email to Promptmetheus](mailto:contact@promptmetheus.com "Send an email to Promptmetheus") ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/partnerships#webpage","@type":"WebPage","description":"Collaboration opportunities for Businesses, Consultants, Marketers, Universities, Developers, and Educators.","name":"Partnerships","url":"https://promptmetheus.com/partnerships","breadcrumb":{"@id":"https://promptmetheus.com/partnerships#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/partnerships"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/partnerships#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Partnerships","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Security · Promptmetheus - **Source:** https://promptmetheus.com/security - **Description:** Information on System Security, Compliance, Certifications, Reports, and measures we take to keep your data safe. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Security Information on System Security, Compliance, Certifications, Reports, and measures we take to keep your data safe Above everything else, it is our **#1 priority** to keep your data safe and secure. In the following, you can find all publicly available information regarding the security of Promptmetheus systems and services. If needed, you can request the Promptmetheus Security Report here: [Request Security Report Request security information via email]( "Request security information via email") ## How we keep your data safe In order to develop secure software, we adhere to industry best practices and rely on established, well-maintained software frameworks like [Django](https://djangoproject.com "Django framework website and documentation") and [Nuxt](https://nuxt.com "Nuxt framework website and documentation"). The following state-of-the-art methods and procedures are employed to keep our systems and your data safe: ### Encryption - **SSL/TLS** Data is always encrypted when transmitted over the internet. - **Passwords** All passwords are encrypted and never stored in clear-text. ### Software updates Wherever possible, we keep software packages that our systems rely on up-to-date. Operating system-level security patches and updates are applied automatically. ### Automated testing Software we deploy undergoes automated unit- and end-to-end testing during every deployment cycle to ensure functionality, security, and access control. ### Monitoring The system is monitored for issues and vulnerabilities by - [Sentry](https://sentry.io/welcome/ "Sentry app monitoring") - [Snyk](https://snyk.io "Snyk cybersecurity") ## Compliance and certifications We are working on establishing enterprise grade compliance and certification. Compliance - [PCI](https://www.iso.org/standard/72434.html "ISO/IEC TS 27100:2020") Compliant ## Service Availability Availability of all Promptmetheus systems is monitored by [Uptime Robot](https://uptimerobot.com "The world's leading uptime monitoring service"). Admins get notified about downtime within 5 minutes of occurrence and can respond accordingly. System status and 90-day uptime history are publicly available on the [Status](https://stats.uptimerobot.com/N6r9otZ4p0 "System monitoring and uptime history") page. ## Data Privacy and -Processing All information related to data privacy and -processing is outlined in our [Privacy Policy](https://promptmetheus.com/privacy). ## Security of 3rd-party services The overall security of a system is only as good as its weakest link. Therefore, we work exclusively with established 3rd-party services who take security as seriously as we do. Here is the list of relevant 3rd-party service providers that we rely on, together with their respective certifications and security information. ### GitHub We use GitHub to store all of our source code. GitHub is the industry leader for source code management and **SOC 1/2**, **CSA**, **GDPR**, **ISO 27100**, **ISO 27701**, and **ISO 27018** compliant. For more information, please visit their [security page](https://github.com/security "Security details of GitHub services"). ### DigitalOcean We use DigitalOcean as cloud service provider for our database and backend services. DigitalOcean is a trusted industry player and **SOC 2/3**, **CSA**, **GDPR**, and **CBPR** compliant. Digital Ocean also features built-in [DDoS](https://en.wikipedia.org/wiki/Denial-of-service_attack "Denial-of-service attack (Wikipedia)") protection for all droplets. For more information, please visit their [security page](https://www.digitalocean.com/security "Security details of DigitalOcean services"). ### Vercel We use Vercel as DNS- and cloud service provider for all Promptmetheus frontends. Vercel is a trusted industry player and **SOC 2** and **GDPR** compliant. They use [AWS](https://aws.amazon.com "Amazon Web Services") infrastructure and encrypt and backup all of their data. For more information, please visit their [security page](https://vercel.com/security "Security details of Vercel services"). ### Stripe We use Stripe as our payment provider and only store Stripe customer-, subscription-, and product IDs in our databases for maximum security. All sensitive payment information like billing address, credit card details, etc. are stored exclusively on secure and battle-tested Stripe infrastructure. Stripe is the industry leader for payments infrastructure, **PCI Level 1** certified, and **SOC 1/2** compliant. For more information, please visit their [security page](https://docs.stripe.com/security "Security details of Stripe services"). ### LLM APIs and service providers The business model of "LLM as a service" is quite new and the industry is still in the process of establishing the standards and best practices. By using the Promptmetheus platform you are subject to the same security risks that you would incur if you would interact with the providers' APIs directly. ## Reporting vulnerabilities If you, despite all our efforts, discover any vulnerabilities in our systems, please report them confidentially at [security@promptmetheus.com](mailto:security@promptmetheus.com "Email to the security team") and do not disclose them to the public. Note that there is currently no bounty program and we can unfortunately not compensate you for your investigations and reporting, except with an entry in our [Hall of Fame](https://promptmetheus.com/hall-of-fame "Promptmetheus Hall of Fame"). ## Questions and suggestions If you have any questions about the above security details or want to make any suggestions, please don't hesitate to get in touch under [security@promptmetheus.com](mailto:security@promptmetheus.com "Email to the security team"). ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/security#webpage","@type":"WebPage","description":"Information on System Security, Compliance, Certifications, Reports, and measures we take to keep your data safe.","name":"Security","url":"https://promptmetheus.com/security","breadcrumb":{"@id":"https://promptmetheus.com/security#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/security"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/security#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Security","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Support · Promptmetheus - **Source:** https://promptmetheus.com/support - **Description:** Information on how to get help with our products and services. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Support Information on how to get help with our products and services To get instant help with technical issues, please take a look at the [Documentation](https://docs.promptmetheus.com "Promptmetheus documentation") and consult the [Troubleshooting](https://docs.promptmetheus.com/troubleshooting "First aid kit to help you quickly resolve issues") section first. For everything else, please reach out via email at [support@promptmetheus.com](mailto:support@promptmetheus.com "Email support") or join our friendly [Discord](https://discord.com/invite/7dbUb9wtGj "Join the Discord server") community. Feedback, Feature Requests, and Bug Reports are always very welcome! ## FAQ How can I switch to a team account? Just get in touch via the above email address and we'll set you up. How can I cancel my subscription? You can easily cancel your subscription to the end of the current billing cycle in the account menu via "Manage subscription". Where can I access my invoices? You can easily access all your invoices in the Stripe billing portal. You'll find a link to the portal at the bottom of the "Account" page. Can I get a refund? Unfortunately no. Please take a look at the [Refund Policy](https://promptmetheus.com/refunds) for more details. How can I migrate my data from Forge to Archery? It's easy! Just backup/export your data from the Forge playground and use the "Import project" option in the app menu to import it into Archery. Note that prompt history can unfortunately not be recovered for Forge prompts since it is not stored in the playground. It is currently not possible to move data from Archery to Forge. ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/support#webpage","@type":"WebPage","description":"Information on how to get help with our products and services.","name":"Support","url":"https://promptmetheus.com/support","breadcrumb":{"@id":"https://promptmetheus.com/support#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/support"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/support#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Support","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Affiliate Program · Promptmetheus - **Source:** https://promptmetheus.com/affiliate - **Description:** Join our network of affiliate partners that help us promote Promptmetheus to a wider audience in the Prompt Engineering space and receive 30% commission on your referrals. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Affiliate Program Become a partner, get a link, share it, earn Join our network of **350+ affiliate partners** and help us to get Promptmetheus into the hands of even more Prompt Engineers and AI Developers. ## How much you can earn Partners receive a **30% commission on the generated revenue**, capped at the first three months of subscriptions. That is give or take: - $25 for single user referrals - $200+ for small business referrals - $465+ for large business referrals ## Who we are looking for Ideal partners are Educators, Content Creators (YouTube, TikTok, X, Substack, etc.), B2B marketers, AI consultants, AI agency owners, and generally anyone that is passionate about generative AI and its potential. ## Who we want to reach We want to reach primarily Prompt Engineers, Automation Engineers, and Software Developers that want to augment their products with Large Language Models. ## How to get started It's easy: sign up below, get an affiliate link, share it, earn. If you have any more questions, please [get in touch](mailto:contact@promptmetheus.com "Send an email to Promptmetheus"). [Become an Affiliate Signup for the Promptmetheus Affiliate Partner Program](https://promptmetheus.getrewardful.com "Signup for the Promptmetheus Affiliate Partner Program") ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/affiliate#webpage","@type":"WebPage","description":"Join our network of affiliate partners that help us promote Promptmetheus to a wider audience in the Prompt Engineering space and receive 30% commission on your referrals.","name":"Affiliate Program","url":"https://promptmetheus.com/affiliate","breadcrumb":{"@id":"https://promptmetheus.com/affiliate#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/affiliate"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/affiliate#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Affiliate Program","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Privacy Policy · Promptmetheus - **Source:** https://promptmetheus.com/privacy - **Description:** All relevant information regarding cookie use, data collection, and data processing. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Privacy Policy Last updated: Oct 9, 2025 Responsible for Data Protection (DPO): **Toni Engelhardt** (c/o [toni@promptmetheus.com](mailto:toni@promptmetheus.com)) This Policy has been designed to comply with the [General Data Protection Regulation (GDPR)](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32016R0679). The following outlines which general and personal data we collect from you and how it will be stored, processed, and shared with third parties. Please read it carefully! ## Personal data we collect about you When you use and interact with Promptmetheus we may collect and process the following personal data about you: - Information that you provide by filling out forms on our site (for instance when you sign up) will be stored in our database and/or CRM. - When you sign up on our platform we automatically create a user account for you and store your email address and a hash (according to [NIST](https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-132.pdf "NIST Special Publication 800-132") recommendation) of the password, in case you provide one, in our database. All information that you add in your profile (e.g. name) is also stored together with your account and can be updated in the profile settings to keep it accurate. - If you contact us via phone, chat, or email, we may keep a record of the correspondence. - Information about the device which you use to access Promptmetheus, i.a. operating system, browser type, and mobile network information. ## General data we collect from you All data that you add anywhere in the app that is not personal data (e.g. prompts, datasets, embeddings, etc.) will be stored in our databases and can be accessed by Promptmetheus staff to maintain and improve the service, analyze usage, and provide support. Note that any personal data that you add via text- or file inputs, which are not specifically intended for personal data, will be regarded as general data (e.g. data you add via text- and data blocks). We do not share general data with third parties, except with the AI platform providers and as specified in the "Third Party Services" section. ## Where we store your data As of now we store logs and data you submit to Promptmetheus on [Digital Ocean](https://www.digitalocean.com/ "Digital Ocean website") and [Vercel](https://vercel.com "Vercel website") cloud servers (please consult their Privacy Policies as well for more information), but might decide in the future to store or process it somewhere else for strategic or economic reasons (of course given an appropriate level of protection). By submitting data to our website, you agree to this potential transfer without further notice. We will take all steps reasonably necessary to ensure at all times that your data is treated securely and in accordance with this privacy policy. Unfortunately, the transmission of information via the internet is not completely secure. Although we will do our best to protect your personal and general data, we cannot guarantee its security and strongly suggest you do not submit data that could harm you or others if exposed. Any data submission is at your own risk. To avoid data loss we might periodically create backups of user data and logs and store those next to the cloud servers also on local devices controlled by Promptmetheus and its staff. ## How we use your data We will use your email address to communicate with you. If not specifically stated otherwise and given your explicit consent we do not share your email address with anyone, except if required by law. Communication with third party services outside of our page always goes through us. When using Promptmetheus while you are logged in we might compile and store data about how you use the service, e.g. how many prompts you have created, how often you have executed them, aso. We process this data to generate general statistics to improve our service and to provide you with a better user experience. We also process and analyze general data internally to analyze and improve the user experience. ## Disclosure of information We won't disclose any of your personal information without your prior confirmation (unless required by law). Please note however, that integrated third party services might very well do so. Please have a look at the "Third Party Services" section below for more information. ## Server log files In order to monitor usage and problems of this site we, [Digital Ocean](https://www.digitalocean.com/ "Digital Ocean website"), and [Vercel](https://vercel.com "Vercel website") store so-called server log files, which your browser transmits automatically while using Promptmetheus. The collected information is generally not associated with personal data or other data sources. However, there might be exceptions in the case of suspected misuse or illegal actions. The collected information might include: - Browser type / version - Operating system - Referrer URL - Hostname of the system that executes requests - Timestamps of requests ## Cookies Promptmetheus uses cookies to improve the user experience. A cookie is a small file that is stored on your device to save and retrieve information. Cookies are used mainly for user authentication, but might also be used to enhance user experience. In order to delete cookies from your device, please refer to the documentation of your web browser. ## Local storage In addition to cookies, Promptmetheus uses your browser's localstorage to store UI state, device information, and certain settings on your device to improve your experience. ## Third party services Promptmetheus uses third party services from various providers (see below). Those services may also store cookies on your device and collect, store, process, and transmit general and personal data about you. Please refer to their websites for more information about their respective policies. ### Gandi Gandi is our domain- and email service provider. Email communication with us via the "promptmetheus.com" domain will be stored in Gandi mailboxes. Please consult their [Privacy Policy](https://gandi.net/en/contracts/privacy-policy "Gandi Privacy Policy") for more information. ### Digital Ocean We currently use [Digital Ocean](https://www.digitalocean.com/ "Digital Ocean website") as our cloud service provider for backend services. This is where most of your personal and general data is stored if you choose to sign up. This only affects you if you register for an account or interact with one of our APIs. Please consult their [Privacy Policy](https://www.digitalocean.com/legal/privacy-policy "Digital Ocean Privacy Policy") for more information. ### Vercel We currently use [Vercel](https://vercel.com "Vercel website") as DNS- and cloud service provider to host all our frontends. Vercel therefore gets access to request logs and error logs of your visits and stores them in their databases. Some part of the authentication process might also be handled on Vercel servers, but besides that we do not store any personal or user-specific data on their infrastructure. Please consult their [Privacy Policy](https://vercel.com/legal/privacy-notice "Vercel Privacy Policy") for more information. ### Notion We use notion as CRM to keep track of different types of user interactions, e.g. sales pipeline, support requests, customer feedback, etc. When you contact us via email or create an account your contact information might be stored in a Notion workspace. This data is stored on Notion infrastructure. Please consult the Notion [Privacy Policy](https://www.notion.so/help/privacy "Notion Privacy Policy") for more information. ### Stripe We currently use [Stripe](https://stripe.com "Stripe website") as our payment provider. When you subscribe to one of our plans via the [Stripe Checkout](https://stripe.com/gb/payments/checkout) or [Stripe Billing](https://stripe.com/gb/billing) portals we create a Stripe customer profile for you. Stripe stores and processes the data that you provide in those portals. Personal and sensitive payment information like billing address, credit card details, etc. are stored exclusively on secure Stripe infrastructure, we only store Stripe customer-, subscription-, and product IDs in our databases and link them to your user account. Please consult the [Stripe Privacy Policy](https://stripe.com/privacy) for more information. ### Sentry We use [Sentry](https://sentry.io/welcome/ "Sentry website") to monitor performance and to investigate problems. Sentry collects meta data about your device and tracebacks of warnings and errors that occur when you use Promptmetheus. Additionally, we use Sentry to capture session replays when errors occur in the app to better understand the context of the error. Sensitive user data is redacted in those recordings. Please consult their [Privacy Policy](https://sentry.io/privacy/ "Sentry Privacy Policy") for more information. ### Plausible We use privacy-friendly analytics provided by [Plausible](https://plausible.io "Plausible website") to improve our services. Plausible is GDPR compliant out of the box and doesn't use cookies to trace you around the web. Please consult their [Privacy Policy](https://plausible.io/privacy "Plausible Privacy Policy") for more information. ### PostHog We use [PostHog](https://posthog.com "PostHog website") to improve the user experience. PostHog collects various in-app usage data, including user interactions, device information, and IP addresses. The collected data is stored and processed on their cloud infrastructure. PostHog is only installed in Promptmetheus apps. The tracking script is not loaded outside of the login boundary. Please consult their [Privacy Policy](https://posthog.com/privacy "PostHog Privacy Policy") for more information. ### Clarity We use [Microsoft Clarity](https://clarity.microsoft.com "Microsoft Clarity website") to capture how users interact with Promptmetheus through behavioral metrics, heatmaps, and session replay. Most of the personal and general data you add to Promptmetheus is redacted in these sessions (prompt fragments, test data, outputs, etc.), but some might be visible (e.g. prompt and block names). You can completely opt-out of Clarity by following the instructions [here](https://learn.microsoft.com/en-us/clarity/faq#:~:text=You%20can%20opt%20out%20of,opt%2Dout%20preference%20is%20saved. "Microsoft Clarity FAQ: How to opt-out of Clarity"). Some Promptmetheus usage data is captured via first and third-party cookies and other tracking technologies to determine service popularity, online activity, and to optimize the user experience. This data is shared with Microsoft and stored on their servers. Clarity is only installed in Promptmetheus apps. The tracking script is not loaded outside of the login boundary. Please consult the [Microsoft Privacy Statement](https://privacy.microsoft.com/privacystatement) for more information. ### Rewardful We use [Rewardful](https://www.rewardful.com/ "Rewardful website") to track affiliate links from our [Affiliate Program](https://promptmetheus.com/affiliate "Promptmetheus Affiliate Program"). When you follow an affiliate link from our partners, Rewardful will store a cookie on your system to attribute potential purchases. Please consult the [Rewardful Privacy Policy](https://www.rewardful.com/privacy "Microsoft Privacy Statement") for more information. ### LLM providers When you execute prompts, we send the prompt and related data to the selected provider. The provider might store and process this data and meta information about the requests. Please consult the related policies of the respective providers for more information: - **Anthropic** [Privacy Policy](https://www.anthropic.com/legal/privacy "Anthropic Privacy Policy") - **Cohere** [Privacy Policy](https://cohere.com/privacy "Cohere Privacy Policy") - **OpenAI** [Privacy Policy](https://openai.com/policies/row-privacy-policy/ "OpenAI Privacy Policy") - **Google / DeepMind** [Privacy Policy](https://policies.google.com/privacy "Google Privacy Policy") - **Perplexity** [Privacy Policy](https://www.perplexity.ai/hub/legal/privacy-notice "Perplexity Privacy Policy") - **Mistral** [Privacy Policy](https://legal.mistral.ai/terms/privacy-policy "Mistral Privacy Policy") - **Groq** [Privacy Policy](https://groq.com/privacy-policy "Groq Privacy Policy") - **AI21 Labs** [Privacy Policy](https://www.ai21.com/terms-policies/privacy-policy/ "AI Labs Privacy Policy") - **Deep Infra** [Privacy Policy](https://deepinfra.com/privacy "Deep Infra Data Privacy") - **xAI** [Privacy Policy](https://x.ai/legal/privacy-policy "xAI Privacy Policy") - **DeepSeek** [Privacy Policy](https://cdn.deepseek.com/policies/en-US/deepseek-privacy-policy.html "DeepSeek Privacy Policy") - **MoonshotAI** [Privacy Policy](https://platform.kimi.ai/docs/agreement/userprivacy "MoonshotAI Privacy Policy") - **FetchAI** [Privacy Policy](https://www.fetch.ai/privacy-policy "FetchAI Privacy Policy") - **Venice** [Privacy Policy](https://venice.ai/legal/privacy-policy "Venice Privacy Policy") - **OpenRouter** [Privacy Policy](https://openrouter.ai/privacy "OpenRouter Privacy Policy") ## Access to information You have the right to access any personal information we have stored about you. To request the data please send a short email to [contact@promptmetheus.com](mailto:contact@promptmetheus.com) so that we can compile it for you. ## Deletion of information You can easily delete your user account inside the app and therewith all personal information we store about you. Alteratively, you can also send a request to [contact@promptmetheus.com](mailto:contact@promptmetheus.com) if you want us to do it for you. Exempt from the deletion are our database backups that are stored on our servers and backup devices. Those updates are not shared with anyone (except if required by law). ## Changes to this policy Any changes that we make to this policy in the future will be posted and logged here and, if you have a user account and the changes directly affect you, sent to you by email. Some changes might only affect you when you use certain features. You will be asked to consent to those changes at the time you start using those features. - June 15, 2023: Original version. - June 17, 2023: Add redaction and opt-out notice to Microsoft Clarity section. - August 31, 2023: Addition of Gandi and clarifications. - September 18, 2023: Addition of Anthropic and Cohere to third parties. - September 19, 2023: Addition of NLP Cloud, AI21 Labs, and Aleph Alpha to third parties. - September 30, 2023: Addition of Stripe as payment provider. - October 18, 2023: Addition of Deep Infra to third parties. - December 9, 2023: Addition of Google AI and Perplexity to third parties. - March 15, 2024: Addition of Notion, Mistral, and Groq to third parties. - May 5, 2024: Add note about Sentry session reply. - June 27, 2024: Addition of PostHog to third parties. - July 18, 2024: Addition of Rewardful to third parties. - Oct 9, 2025: Update OpenAI policy link. Renaming of Google AI. Removal of Aleph Alpha and NLP Cloud from third party policies. Addition of xAI, DeepSeek, MoonshotAI, FetchAI, Venice, and OpenRouter to third party policies. ## Questions? If you have any questions about the above policy, please contact us at [contact@promptmetheus.com](mailto:contact@promptmetheus.com) ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/privacy#webpage","@type":"WebPage","description":"All relevant information regarding cookie use, data collection, and data processing.","name":"Privacy Policy","url":"https://promptmetheus.com/privacy","breadcrumb":{"@id":"https://promptmetheus.com/privacy#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/privacy"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/privacy#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Privacy Policy","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Terms of Service · Promptmetheus - **Source:** https://promptmetheus.com/terms - **Description:** The terms governing the use of the Promptmetheus platform and services. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Terms of Service Last updated: Oct 15, 2025 In the following "Promptmetheus" refers to all services related to Promptmetheus, including this site, the Promptmetheus platform, apps, APIs, etc. Promptmetheus is owned and operated by Toni Engelhardt (residence and jurisdiction in Portugal). ## Using this service When using Promptmetheus you consent to the following terms. If you are under 16 years old you may not use this service without explicit prior consent of your legal guardian to the following terms. ## Limitations of use and prohibited actions You may use Promptmetheus only in compliance with the law and for lawful purposes. You may not use this site for the purpose of harming or attempting to harm individuals or other parties in any way. You may not attempt to access accounts of other users without their permission and you may not try to access without authority, interfere with, damage or disrupt - any part of Promptmetheus; - any equipment or network on which Promptmetheus is stored; - any software used in the provision of Promptmetheus; - any equipment or network or software owned or used by any third party. ## Contributing and uploading material Whenever you add content (text, links, files, images, videos, etc.) to Promptmetheus you take full responsibility and liability for it and you grant us permission to store that content on our servers. You may only add content that you either own or for which you have explicit permission by the owner to use it and to upload it to our servers. You must under no circumstance knowingly transmit any data, send or upload any material that contains viruses, Trojan horses, worms, time-bombs, keystroke loggers, spyware, adware, or any other harmful programs or similar computer code designed to adversely affect the operation of any computer software or hardware. You must also not transmit, or procure the sending of, any unsolicited or unauthorized advertising or promotional material or any other form of similar solicitation (spam). ## Account obligations When signing up to Promptmetheus and thereby creating a user account you agree that all account information you provide to us will be true, accurate, complete and current information, and that you will maintain and update your account information as needed throughout your use of the service to keep it accurate and current. Failure to provide accurate, current and complete account information may result in the suspension and/or termination of your account. You are not allowed to create more than one account per person (physical identity) without explicit permission from our side. You understand and agree that we may access, use, preserve and/or disclose your account information and content if legally required to do so or if we have a good faith belief that such access, use, disclosure, or preservation is reasonably necessary to: - provide support; - comply with legal process or request; - enforce these terms of service, including investigation of any potential violation thereof; - detect, prevent or otherwise address security, fraud or technical issues; - protect the rights, property or safety of this service, its users or the public as required or permitted by law. Where we give you or where you choose a password which enables you to access certain parts of this site, you are responsible for choosing a reasonably secure password and keeping it confidential. We ask you not to share your password with anyone. ## Third party services We use third party services from Google, Microsoft, Stripe, OpenAI, and others to operate our platform, provide services, and to enhance user experience, i.a. text completion, authentication, etc. We take all reasonable effort to verify the correct functioning and legality of those services at all times, but do not take any responsibility for them since their functioning is beyond our control. See also the related section in our [Privacy Policy](https://promptmetheus.com/privacy) for more information on how data is shared with third parties. By using Promptmetheus you agree to not knowingly violate the terms and policies of any of those third parties. Wherever an external API key is required, i.a. to access OpenAI's API, you explicitly agree to adhere to the terms and conditions of the respective provider when accessing their services through Promptmetheus. Provider terms and policies: - [Anthropic](https://www.anthropic.com/legal/consumer-terms "Anthropic Terms of Service") - [Cohere](https://cohere.com/terms-of-use "Cohere Terms of Use") - [OpenAI](https://openai.com/policies/ "OpenAI Terms and Policies") - [Google / DeepMind](https://ai.google.dev/gemini-api/terms "Google / DeepMind Terms of Service") - [Perplexity](https://www.perplexity.ai/hub/legal/terms-of-service "Perplexity Terms of Service") - [Mistral](https://legal.mistral.ai/terms "Mistral Terms of Service") - [Groq](https://groq.com/terms-of-use "Groq Terms of Service") - [AI21 Labs](https://www.ai21.com/terms-of-service/ "AI Labs Terms of Use") - [Deep Infra](https://deepinfra.com/terms "Deep Infra Terms & Conditions") - [xAI](https://x.ai/legal/terms-of-service "xAI Terms of Service") - [DeepSeek](https://cdn.deepseek.com/policies/en-US/deepseek-terms-of-use.html "DeepSeek Terms of Use") - [MoonshotAI](https://platform.kimi.ai/docs/agreement/modeluse "MoonshotAI Terms of Service") - [FetchAI](https://www.fetch.ai/terms "FetchAI Terms of Service") - [Venice](https://venice.ai/legal/tos "Venice Terms of Service") - [OpenRouter](https://openrouter.ai/terms "OpenRouter Terms of Service") ## Third party API keys In order to execute prompts you might be required to provide your own API keys to use the above-mentioned third-party AI platforms. By design, the keys you provide are used exclusively for requests that you or your team mates initiate. However, despite best efforts, we cannot guarantee that our software always works without flaws. You are therefore required to set appropriate usage limits on your API keys and we do not take responsibility for excessive requests that are caused by malfunctioning of our software or hacks. The default storage location of third-party API keys is locally on your device for maximum security. But, depending on your plan, you may have the option to also store API keys in your profile or one of your organizations to share them and sync them across devices. If you choose to do so, your API keys will be stored in our database, which exposes them to an additional risk of being compromised by a hack. When using this feature you acknowledge this risk and also the fact that we do not take responsibility for any damages that might arise from a hack of our database. ## Trials, subscriptions, payments, and disputes All our paid plans have free trials for you to test the service before committing to a subscription. You may cancel subscriptions at any time to the end of the current billing cycle (monthly or yearly). Early cancellations are possible, but there are no refunds for unused time. Refunds are governed by our [Refund Policy](https://promptmetheus.com/refunds) and are currently only available if you have been charged in error. You must not dispute payments unless they were fraudulent or unauthorized. Disputes create costs on our side and we reserve the right to charge you for those costs as well as the labor that is required to resolve illegal disputes. When your free trial ends you will be prompted to subscribe to one of our paid plans to be able to continue using Promptmetheus services. If you decide to subscribe you will be redirected to a [Stripe](https://stripe.com "Stripe website") portal to complete the purchase. By using the portal you agree to Stripe processing your data and payments as outlined in their [Service Agreement](https://stripe.com/gb/legal/ssa "Stripe Service Agreement") and [Privacy Policy](https://stripe.com/privacy "Stripe Privacy Policy"). If you choose to not subscribe to one of our paid plans you might lose access to the data that you have created during the trial period. Every subscription is only valid for one person (physical identity) and may not be shared between multiple people. You may not try to manipulate or hack our system in any way to obtain paid services without active subscription. In the unfortunate event that Promptmetheus ceases to operate, users will most likely receive pro-rated refunds for the remainder of their respective billing cycles. However, depending on the circumstances, there is no guarantee of any refunds. ## Linking our sites You may link to Promptmetheus, provided you do so in a way that is fair and legal and does not damage our reputation or take advantage of it, but you must not establish a link in such a way as to suggest any form of association, approval or endorsement on our part where none exists. We reserve the right to demand removal of links to this website without further reasoning. ## Links to other pages Where Promptmetheus links to content on other pages provided by third parties, these links are provided for your information only. We have no control over those contents and do not take responsibility for them or for any loss or damage that may arise from your use of them. ## Affiliate program Membership in our affiliate program is governed by the [Affiliate Terms and Conditions](https://promptmetheus.com/affiliate/terms). ## Privacy and cookies By using Promptmetheus you consent to our [Privacy Policy](https://promptmetheus.com/privacy). ## Communication All contact details on this page (email addresses and phone numbers) must only be used to contact us for one or several of the following: - questions about using our service; - reporting problems with our page or bugs; - requesting support regarding the use of this service. You must however, under no circumstance, share those provided contact details with anyone or use them for advertising purposes or in order to sell services to us. Also, you must never send us messages or emails generated by an automated system. We reserve the right to invoke legal action against any such attempt. ## Indemnification You agree to defend, indemnify and hold our brand, its affiliates, subsidiaries, directors, officers, employees, agents, partners and licensors harmless from any claim or demand, including reasonable attorneys' fees, made by a third party, relating to or arising from: - any content you submit, post, transmit, or otherwise make available through the service; - your use of the service; - any violation by you of these terms of service; - your violation of any rights of another service. This obligation shall survive the termination or expiration of these terms of service and/or your use of the service. You acknowledge that you are responsible for all use of this service through your account, and that these terms of service apply to any and all usage of your account. You agree to comply with these terms of service and to defend, indemnify and hold harmless our entity from and against any and all claims and demands arising from usage of your account, whether or not such usage is expressly authorized by you. ## Intellectual property We are the owner or the licensee of all intellectual property rights of this site. Those rights are protected by copyright laws and treaties around the world. All such rights are reserved. You may print off one copy, and may download extracts, of any page(s) from this site for your personal reference. You must however not modify this copy in any way, and you must not use any illustrations, photographs, video or audio sequences or any graphics separately from any accompanying text. Our status (and that of any identified contributors) as the authors of material on our sites must always be acknowledged in a proper way. If you breach any of these terms of use, your right to use this site will cease immediately and you must, at our option, return or destroy any copies of the materials you have made. ## Disclaimer We do not guarantee that our service as a whole or parts of it will be error-free, correct or available at all times. Even though we try to fix bugs and errors on a regular basis there is no guarantee that we will do so. The entire page is provided "as-is", all information and content on the site might be subject to change or deletion without further notice. Using our page or downloading anything from it happens on your own risk. We cannot ensure that files or other data you download will be free of viruses, contamination or destructive features. This also includes the source code itself. We herewith disclaim all warranties, express or implied, including accuracy, non-infringement, merchantability and fitness for a particular purpose. We also explicitly disclaim any liability for third party services integrated on our page. Please refer to the terms of service of those third parties for more information regarding their use. When using our service you assume full responsibility for your usage and actions on this page and all content and links you add to it. Commentary and other materials posted on our sites are not intended to amount to advice on which reliance should be placed. We therefore disclaim all liability and responsibility arising from any reliance placed on such materials by any visitor to our sites, or by anyone who may be informed of any of its contents. We reserve the right to do any of the following, at any time, without further notice: - terminate operation of the site and/or the service, partially or completely. This includes the deletion or change of data and content without specific reason; - modify or change the site and/or the service, partially or completely, including policies and terms; - temporarily interrupt the operation of the site and/or the service, partially or completely, in order to perform updates, maintenance, error correction, or other changes; - change or delete content, links, or files that have been added by you; - change or delete your account. ## Changes to this document Any changes that we make to this document in the future will be posted on this page and, where appropriate, sent to you by email if you are a registered user. - June 15, 2023: Original version. - June 17, 2023: Addition of disclaimer to the "Third party API keys" section. - September 18, 2023: Addition of Anthropic and Cohere to third-party services. - September 19, 2023: Addition of NLP Cloud, AI21 Labs, and Aleph Alpha to third-party services. - September 30, 2023: Addition of "Subscriptions and payments" section. - October 18, 2023: Addition of Deep Infra to third-party services. - December 9, 2023: Addition of Google AI and Perplexity to third-party services. Additional terms regarding the use of third-party API key syncing. Additional terms regarding trials and disputes. - March 15, 2024: Addition of Mistral and Groq to third-party services. - July 18, 2024: Addition of Affiliate Terms and Conditions link. - Oct 9, 2025: Renaming of Google AI service. Removal of Aleph Alpha and NLP Cloud from third party services. Addition of xAI, DeepSeek, MoonshotAI, FetchAI, Venice, and OpenRouter to third party services. - Oct 13, 2025: Updates to the "subscriptions" section related to annual billing options. ## Questions? If you have any questions about the above terms, please get in touch at [contact@promptmetheus.com](mailto:contact@promptmetheus.com) ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/terms#webpage","@type":"WebPage","description":"The terms governing the use of the Promptmetheus platform and services.","name":"Terms of Service","url":"https://promptmetheus.com/terms","breadcrumb":{"@id":"https://promptmetheus.com/terms#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/terms"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/terms#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Terms of Service","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Investors · Promptmetheus - **Source:** https://promptmetheus.com/investors - **Description:** Information about investment opportunities for VCs and angel investors. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Investors Information about investment opportunities Promptmetheus is privately owned, profitable, and currently not looking for venture funding. If you are interested in a strategic acquisition, please get in touch. Contact: [investors@promptmetheus.com](mailto:investors@promptmetheus.com "Send an email to Promptmetheus") ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/investors#webpage","@type":"WebPage","description":"Information about investment opportunities for VCs and angel investors.","name":"Investors","url":"https://promptmetheus.com/investors","breadcrumb":{"@id":"https://promptmetheus.com/investors#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/investors"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/investors#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Investors","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Refund Policy · Promptmetheus - **Source:** https://promptmetheus.com/refunds - **Description:** Information about our Refund Policy for subscriptions and related payments. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Refund Policy Last updated: Oct 13, 2025 Unfortunately, refunds are currently not available – except if you have been charged in error. To make an informed purchasing decision, users can test our paid products 7 days free of charge before subscribing. If you are unsure whether or not Promptmetheus works for you in the long run, please select the monthly billing cycle to be able to cancel at any time. For questions regarding refunds, please get in touch at [support@promptmetheus.com](mailto:support@promptmetheus.com "Get support via email"). ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/refunds#webpage","@type":"WebPage","description":"Information about our Refund Policy for subscriptions and related payments.","name":"Refund Policy","url":"https://promptmetheus.com/refunds","breadcrumb":{"@id":"https://promptmetheus.com/refunds#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/refunds"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/refunds#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Refund Policy","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Affiliate Terms and Conditions · Promptmetheus - **Source:** https://promptmetheus.com/affiliate/terms - **Description:** The terms governing membership in the Promptmetheus affiliate program. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Affiliate Terms and Conditions Last updated: July 18, 2024 By signing up for the Promptmetheus [affiliate program](https://promptmetheus.getrewardful.com "Promptmetheus Affiliate Program"), you agree to abide by the following terms and conditions. Please read the entire agreement carefully before registering for the program. The purpose of an affiliate is to legally promote our products and services to receive a commission on new customer subscriptions through referrals. We use [Rewardful](https://www.rewardful.com/ "Rewardful Affiliate Management Software") to track referrals and manage commissions. ## 1. Vetting We reserve the right to vet all affiliate program memberships at our sole and absolute discretion. You will have no legal recourse against us for the rejection or cancellation of your membership. When you sign up, please wait until we contact you for approval before you start promoting Promptmetheus. ## 2. Commissions You will earn a 30% commission on subscriptions of referred new customers, capped at the first three months. Referrals are tracked with a cookie (60 days lifetime) and first-touch attribution. Commissions are only valid for transactions that have been successfully completed. There will be no payout for transactions that result in chargebacks or refunds. Commissions will be paid once per month, 30-60 days after the conversion to avoid overlap with chargebacks, refunds, etc. Payouts require a minimum balance of $100 in commissions. Commissions expire after six months if the threshold is not reached and no new referrals are made in that timeframe. You cannot refer yourself and you will not receive a commission on your own accounts. ## 3. Good Faith Promptmetheus, as well as affiliates, agree to pursue this program in good faith and with mutual interests in mind. In case of malicious behavior, e.g., faking signups, attempts to game the system, etc. referrals might be voided. ## 4. Tracking and Attribution Errors Referrals might be voided if they were attributed in error due to bugs in the tracking system. Promptmetheus will not be liable for indirect or accidental damages (loss of revenue, commissions) due to affiliate tracking failures, loss of database files, or attempts to hack our systems. ## 5. Termination Your affiliate membership may be suspended or terminated for any of the following reasons: - Inappropriate advertisements (false claims, misleading hyperlinks, etc.). - Spamming (mass email, mass newsgroup posting, etc.). - Advertising on sites containing or promoting illegal activities. - Violation of intellectual property rights. - Offering rebates, coupons, or other forms of promised kick-backs from your affiliate commission as an incentive. - Self-referrals, fraudulent transactions, suspected affiliate fraud. In addition to the above, Promptmetheus reserves the right to terminate any affiliate account at any time, for any violations of this agreement or no reason at all. ## 6. Affiliate Links You may use affiliate links both on your website and within email messages. You may also advertise Promptmetheus in online and offline classified ads, magazines, and newspapers. ## 7. Coupons You are not allowed to promote discount codes or coupons without prior written approval. ## 8. Pay Per Click (PPC) Policy You are not allowed to generate affiliate referrals via PPC bidding without prior written approval. ## 9. No Warranty We do not provide any explicit or implied warranties. There is no guarantee that Promptmetheus software and services will be free of errors, and we will not be liable for any interruptions of your affiliate activities. ## 10. Term of the Agreement The term of this agreement begins with our explicit approval of your affiliate account to the program and ends when your affiliate account is terminated. The terms and conditions of this agreement may be modified by us at any time. If any modification to the terms and conditions of this agreement is unacceptable to you, your only choice is to terminate your affiliate account. Your continued participation in the program will constitute your acceptance of any changes. ## 11. Indemnification The affiliate partner shall indemnify and hold harmless Promptmetheus and its affiliate and subsidiary companies, officers, directors, employees, licensees, successors, and assigns, including those licensed or authorized by Promptmetheus to transmit and distribute materials, from any and all liabilities, damages, fines, judgments, claims, costs, losses, and expenses (including reasonable legal fees and costs) arising out of or related to any and all claims sustained in connection with this agreement due to the negligence, misrepresentation, failure to disclose, or intentional misconduct of the affiliate. This obligation shall survive the termination or expiration of this agreement. ## 12. Jurisdiction and Governing Law This Agreement shall be governed by and construed in accordance with the laws of Portugal. Any disputes arising out of or in connection with this Agreement shall be subject to the exclusive jurisdiction of the courts of Portugal. ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/affiliate/terms#webpage","@type":"WebPage","description":"The terms governing membership in the Promptmetheus affiliate program.","name":"Affiliate Terms and Conditions","url":"https://promptmetheus.com/affiliate/terms","breadcrumb":{"@id":"https://promptmetheus.com/affiliate/terms#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/affiliate/terms"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/affiliate/terms#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Affiliate Terms and Conditions","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Banner · Promptmetheus - **Source:** https://promptmetheus.com/banner - **Description:** Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/banner#webpage","@type":"WebPage","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Banner","url":"https://promptmetheus.com/banner","isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/banner"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Prompt Engineering Guide · Promptmetheus - **Source:** https://promptmetheus.com/resources/prompt-engineering-guide - **Description:** Everything you ever wanted to know about Prompt Engineering, AI development, LLM selection, and how to get started on a single page. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Prompt Engineering Guide Everything you ever wanted to know about Prompt Engineering, AI Development, LLM Selection, and how to get started on a single page. This page is currently in the making... The goal for this guide is to distill everything you need to know to get started with [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering "LLM Knowledge Base: Prompt Engineering") down to a single page. There is no technical knowledge required and all the essential questions will be answered. Once you're done reading, you'll be ready to tackle your first Prompt Engineering project. A few notes: - All technical terms are deep-linked to respective definitions in our [LLM Knowledge Base](https://promptmetheus.com/resources/llm-knowledge-base "Bite-sized explanations for commonly used terms and abbreviations related to Large Language Models and Generative AI"). If you don't know what a certain term means, just click on it and get a bite-sized explanation. - ... Let's start with the basics. ## What is a GPT and how does it work? A [Generative Pre-trained Transformer (GPT)](https://promptmetheus.com/resources/llm-knowledge-base/generative-pre-trained-transformer-gpt "LLM Knowledge Base: Generative Pre-trained Transformer (GPT)") is a specific type of [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm "LLM Knowledge Base: Large Language Model (LLM)") that uses [Deep Learning](https://promptmetheus.com/resources/llm-knowledge-base/deep-learning "LLM Knowledge Base: Deep Learning") to generate human-like text. The best way to understand how a GPT works under the hood is undoubtedly to stop right here and invest 27 minutes of your time in watching is excellent video by [3Blue1Brown](https://www.youtube.com/@3blue1brown "3Blue1Brown YouTube channel"): ## What is Prompt Engineering and why do I need it? Coming soon... ## Which LLM API provider should I choose for my project? Coming soon... ## How to select the right model? Take a look at our related post ["How to choose the right LLM for your use case"](https://promptmetheus.com/blog/llm-selection "Blog post: How to choose the right LLM for your use case"). ## What is the difference between one-shot prompts and agents? Coming soon... ## What is RAG and when is it useful? Coming soon... ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/resources/prompt-engineering-guide#webpage","description":"Everything you ever wanted to know about Prompt Engineering, AI development, LLM selection, and how to get started on a single page.","name":"Prompt Engineering Guide","url":"https://promptmetheus.com/resources/prompt-engineering-guide","@type":["WebPage","CollectionPage"],"breadcrumb":{"@id":"https://promptmetheus.com/resources/prompt-engineering-guide#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/resources/prompt-engineering-guide"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/resources/prompt-engineering-guide#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Prompt Engineering Guide","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Blog · Promptmetheus - **Source:** https://promptmetheus.com/blog - **Description:** News, Updates, Research, and Writings on Prompt Engineering, Generative AI, and other topics related to Large Language Models. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Blog News, Updates, Research, and Writings on Prompt Engineering, Generative AI, and other topics related to Large Language Models Apr 01, 2024 by Helena Dias [Streamlining HR tasks with Large Language Models](https://promptmetheus.com/blog/streamlining-hr-tasks-with-llms "Read post: Streamlining HR tasks with Large Language Models") A real-world case study on using Prompt Engineering to make HR and People Operations tasks easy as pie [Read Read post: Streamlining HR tasks with Large Language Models](https://promptmetheus.com/blog/streamlining-hr-tasks-with-llms "Read post: Streamlining HR tasks with Large Language Models") Mar 31, 2024 by Toni Engelhardt [How to Choose the Right LLM for your App, Agent, Workflow, ...](https://promptmetheus.com/blog/llm-selection "Read post: How to Choose the Right LLM for your App, Agent, Workflow, ...") A systematic approach to figuring out which model yields the best results for a specific use case [Read Read post: How to Choose the Right LLM for your App, Agent, Workflow, ...](https://promptmetheus.com/blog/llm-selection "Read post: How to Choose the Right LLM for your App, Agent, Workflow, ...") Jul 21, 2023 by Toni Engelhardt [Stripe for AI: One Integration for every LLM Provider](https://promptmetheus.com/blog/stripe-for-ai "Read post: Stripe for AI: One Integration for every LLM Provider") Exploring 3 different projects that connect to multiple providers and simplify the LLM integration process [Read Read post: Stripe for AI: One Integration for every LLM Provider](https://promptmetheus.com/blog/stripe-for-ai "Read post: Stripe for AI: One Integration for every LLM Provider") Jun 7, 2023 by Toni Engelhardt [Prompt Engineering Tips & Tricks](https://promptmetheus.com/blog/prompt-engineering-tips-and-tricks "Read post: Prompt Engineering Tips & Tricks") The hacks and techniques that take your prompting game to the next level and help you craft better instructions for your LLM workflows [Read Read post: Prompt Engineering Tips & Tricks](https://promptmetheus.com/blog/prompt-engineering-tips-and-tricks "Read post: Prompt Engineering Tips & Tricks") Jun 6, 2023 by Toni Engelhardt [Introducing the Prompt Engineering IDE](https://promptmetheus.com/blog/prompt-engineering-ide "Read post: Introducing the Prompt Engineering IDE") Software Engineers have powerful tools to develop code efficiently – let's bring that to Prompt Engineering [Read Read post: Introducing the Prompt Engineering IDE](https://promptmetheus.com/blog/prompt-engineering-ide "Read post: Introducing the Prompt Engineering IDE") ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/blog#webpage","description":"News, Updates, Research, and Writings on Prompt Engineering, Generative AI, and other topics related to Large Language Models.","name":"Blog","url":"https://promptmetheus.com/blog","@type":["WebPage","CollectionPage"],"breadcrumb":{"@id":"https://promptmetheus.com/blog#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/blog"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/blog#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Blog","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** LLM Benchmarks · Promptmetheus - **Source:** https://promptmetheus.com/resources/llm-benchmarks - **Description:** A list of benchmarks and leaderboards for Large Language Models ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # LLM Benchmarks A curated list of popular leaderboards and test datasets for Large Language Model (LLM) evaluation The quality of AI products and services depends intricately on the capabilities of the LLMs that power them, and as these models continue to improve, we have to regularly revise our choices. To keep you up to date on how new models are performing and which ones lead each category, we've put together a neat list of benchmarks and datasets. ## Leaderboards LM Arena Subjects: Chat models, Reasoning models Topics: reasoning language understanding user preference LM Arena is an open platform created by researchers from UC Berkeley that lets users pose questions and compare responses from two different Large Language Models without knowing which model produced which answer. The aggregate results of these head-to-head "battles" generate a ranking and leaderboard, similar to the ELO rating system used in chess. [Leaderboard Open LM Arena Leaderboard](https://arena.ai/leaderboard "Open LM Arena Leaderboard") [Paper Open LM Arena Publication](https://arxiv.org/abs/2403.04132 "Open LM Arena Publication") Scale Showdown Subjects: Chat models, Reasoning models Topics: user preference language demographics real-world use Scale Showdown ranks models using blind, optional comparisons gathered during real conversations. Its results can be explored by dimensions such as language, age, continent, education, profession, and prompt type, providing a broader view of user preference across populations. [Leaderboard Open Scale Showdown Leaderboard](https://labs.scale.com/showdown "Open Scale Showdown Leaderboard") Artificial Analysis Subjects: Chat models, Reasoning models, Code models Topics: intelligence coding speed latency price Artificial Analysis independently evaluates models and combines capability results with practical measurements such as output speed, latency, price, and token usage. Its standardized runs make it useful for comparing both model quality and the cost of deploying it. [Leaderboard Open Artificial Analysis Leaderboard](https://artificialanalysis.ai/leaderboards/models "Open Artificial Analysis Leaderboard") LiveBench Subjects: Chat models, Reasoning models, Code models Topics: math coding reasoning language instruction following data analysis LiveBench is an evolving benchmark for Large Language Models that aims to avoid “training set leakage” (contamination) by continuously introducing fresh, recently sourced tasks. Each question in LiveBench is paired with a verifiable ground-truth answer, so that model outputs can be scored automatically — no external judge (human or AI) is needed. [Leaderboard Open LiveBench Leaderboard](https://livebench.ai "Open LiveBench Leaderboard") [Code Open LiveBench Code](https://github.com/livebench/livebench "Open LiveBench Code") [Data Open LiveBench Data](https://huggingface.co/collections/livebench/livebench-67eaef9bb68b45b17a197a98 "Open LiveBench Data") [Paper Open LiveBench Publication](https://arxiv.org/abs/2406.19314 "Open LiveBench Publication") HELM Subjects: Chat models, Reasoning models, Multimodal models Topics: capabilities safety robustness fairness efficiency Stanford's Holistic Evaluation of Language Models provides transparent, reproducible evaluations across capabilities, safety, multimodality, and specialist domains. The project entered maintenance mode in June 2026, so its established methodology and archived results are more useful than it is for comparing the newest models. [Leaderboard Open HELM Leaderboard](https://crfm.stanford.edu/helm "Open HELM Leaderboard") [Code Open HELM Code](https://github.com/stanford-crfm/helm "Open HELM Code") [Paper Open HELM Publication](https://arxiv.org/abs/2211.09110 "Open HELM Publication") Humanity's Last Exam Subjects: Reasoning models, Multimodal models Topics: expert knowledge reasoning calibration multimodality Humanity's Last Exam measures expert-level knowledge and reasoning using difficult questions from mathematics, the sciences, humanities, and other disciplines. Its leaderboard reports both accuracy and calibration, including a multimodal subset with diagrams and figures. [Leaderboard Open Humanity's Last Exam Leaderboard](https://labs.scale.com/leaderboard/humanitys_last_exam "Open Humanity's Last Exam Leaderboard") [Code Open Humanity's Last Exam Code](https://github.com/centerforaisafety/hle "Open Humanity's Last Exam Code") [Data Open Humanity's Last Exam Data](https://huggingface.co/datasets/cais/hle "Open Humanity's Last Exam Data") [Paper Open Humanity's Last Exam Publication](https://arxiv.org/abs/2501.14249 "Open Humanity's Last Exam Publication") MMLU-Pro Subjects: Chat models, Reasoning models Topics: knowledge reasoning science humanities MMLU-Pro is a more difficult and robust successor to MMLU. It expands the number of answer choices, emphasizes reasoning, and covers a broad range of academic disciplines while reducing sensitivity to prompt variations and chance performance. [Leaderboard Open MMLU-Pro Leaderboard](https://huggingface.co/spaces/TIGER-Lab/MMLU-Pro "Open MMLU-Pro Leaderboard") [Code Open MMLU-Pro Code](https://github.com/TIGER-AI-Lab/MMLU-Pro "Open MMLU-Pro Code") [Data Open MMLU-Pro Data](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro "Open MMLU-Pro Data") [Paper Open MMLU-Pro Publication](https://arxiv.org/abs/2406.01574 "Open MMLU-Pro Publication") LongBench v2 Subjects: Chat models, Reasoning models, Long-context models Topics: long context document understanding reasoning retrieval LongBench v2 evaluates deep understanding and reasoning over realistic contexts ranging from thousands to millions of words. Its tasks cover documents, dialogue histories, code repositories, in-context learning, and structured data rather than simple information retrieval alone. [Leaderboard Open LongBench v2 Leaderboard](https://longbench2.github.io "Open LongBench v2 Leaderboard") [Code Open LongBench v2 Code](https://github.com/THUDM/LongBench "Open LongBench v2 Code") [Data Open LongBench v2 Data](https://huggingface.co/datasets/zai-org/LongBench-v2 "Open LongBench v2 Data") [Paper Open LongBench v2 Publication](https://arxiv.org/abs/2412.15204 "Open LongBench v2 Publication") MMMU-Pro Subjects: Multimodal models, Reasoning models Topics: multimodality visual understanding expert knowledge reasoning MMMU-Pro tests expert-level reasoning across interleaved text and images such as charts, diagrams, maps, tables, and technical figures. It strengthens the original MMMU benchmark against shortcuts that let models answer without genuinely understanding the visual information. [Leaderboard Open MMMU-Pro Leaderboard](https://mmmu-benchmark.github.io/#leaderboard "Open MMMU-Pro Leaderboard") [Code Open MMMU-Pro Code](https://github.com/MMMU-Benchmark/MMMU "Open MMMU-Pro Code") [Data Open MMMU-Pro Data](https://huggingface.co/datasets/MMMU/MMMU_Pro "Open MMMU-Pro Data") [Paper Open MMMU-Pro Publication](https://arxiv.org/abs/2409.02813 "Open MMMU-Pro Publication") Berkeley Function-Calling Leaderboard Subjects: Chat models, AI agents Topics: tool use function calling agent memory web search The Berkeley Function-Calling Leaderboard evaluates whether models can select and invoke tools accurately. Its tests span single- and multi-turn calls, relevance detection, web search, memory management, format sensitivity, latency, and cost. [Leaderboard Open Berkeley Function-Calling Leaderboard Leaderboard](https://gorilla.cs.berkeley.edu/leaderboard.html "Open Berkeley Function-Calling Leaderboard Leaderboard") [Code Open Berkeley Function-Calling Leaderboard Code](https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard "Open Berkeley Function-Calling Leaderboard Code") [Paper Open Berkeley Function-Calling Leaderboard Publication](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2025/EECS-2025-184.html "Open Berkeley Function-Calling Leaderboard Publication") τ-bench Subjects: AI agents, Voice agents Topics: tool use conversation policy following knowledge retrieval τ-bench evaluates agents that must converse with users, follow policies, retrieve knowledge, and call tools in realistic enterprise workflows. Its evolving family includes multi-party interaction, knowledge-heavy tasks, and real-time voice conversations. [Leaderboard Open τ-bench Leaderboard](https://taubench.com "Open τ-bench Leaderboard") [Code Open τ-bench Code](https://github.com/sierra-research/tau2-bench "Open τ-bench Code") [Paper Open τ-bench Publication](https://arxiv.org/abs/2506.07982 "Open τ-bench Publication") SWE-bench Subjects: Code models, Coding agents Topics: bug fixing repository comprehension multilingual code multimodality SWE-bench evaluates systems on real GitHub issues that require changes to existing repositories. Its official leaderboards include Verified, Multilingual, Multimodal, Lite, and Full tracks, making results across different software-engineering settings easier to compare. [Leaderboard Open SWE-bench Leaderboard](https://www.swebench.com "Open SWE-bench Leaderboard") [Code Open SWE-bench Code](https://github.com/SWE-bench/SWE-bench "Open SWE-bench Code") [Data Open SWE-bench Data](https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified "Open SWE-bench Data") [Paper Open SWE-bench Publication](https://arxiv.org/abs/2310.06770 "Open SWE-bench Publication") LiveSWEBench Subjects: Code models, Coding agents Topics: code editing feature implementation bug fixing repository comprehension LiveSWEBench is a live, continuously updated benchmark that evaluates the software engineering (SWE) capabilities of AI agents in real-world conditions. It draws tasks from active repositories and scores agents on their ability to propose correct, executable changes. [Leaderboard Open LiveSWEBench Leaderboard](https://liveswebench.ai "Open LiveSWEBench Leaderboard") [Code Open LiveSWEBench Code](https://github.com/livebench/liveswebench "Open LiveSWEBench Code") [Data Open LiveSWEBench Data](https://huggingface.co/collections/livebench/liveswebench-67eaf012f02466f4e2a757e0 "Open LiveSWEBench Data") Terminal-Bench Subjects: Coding agents, Computer-use agents Topics: terminal use system administration coding end-to-end tasks Terminal-Bench evaluates agents on complete tasks inside real terminal environments. It covers software engineering, system administration, data processing, model training, and security work that requires more than generating an isolated answer or patch. [Leaderboard Open Terminal-Bench Leaderboard](https://www.tbench.ai/leaderboard/terminal-bench/2.0 "Open Terminal-Bench Leaderboard") [Code Open Terminal-Bench Code](https://github.com/harbor-framework/terminal-bench-2 "Open Terminal-Bench Code") [Paper Open Terminal-Bench Publication](https://arxiv.org/abs/2601.11868 "Open Terminal-Bench Publication") ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/resources/llm-benchmarks#webpage","description":"A list of benchmarks and leaderboards for Large Language Models","name":"LLM Benchmarks","url":"https://promptmetheus.com/resources/llm-benchmarks","@type":["WebPage","CollectionPage"],"breadcrumb":{"@id":"https://promptmetheus.com/resources/llm-benchmarks#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/resources/llm-benchmarks"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/resources/llm-benchmarks#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"LLM Benchmarks","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** LLM Knowledge Base · Promptmetheus - **Source:** https://promptmetheus.com/resources/llm-knowledge-base - **Description:** Bite-sized explanations for commonly used terms and abbreviations related to Large Language Models and Generative AI. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # LLM Knowledge Base Bite-sized explanations for commonly used terms and abbreviations related to Large Language Models and Generative AI A [A2UI](https://promptmetheus.com/resources/llm-knowledge-base/a2ui "LLM Knowledge Base: A2UI") [AI Agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent "LLM Knowledge Base: AI Agent") [AI Alignment](https://promptmetheus.com/resources/llm-knowledge-base/ai-alignment "LLM Knowledge Base: AI Alignment") [AI Programming Interface (AIPI)](https://promptmetheus.com/resources/llm-knowledge-base/ai-programming-interface-aipi "LLM Knowledge Base: AI Programming Interface (AIPI)") [AI Safety](https://promptmetheus.com/resources/llm-knowledge-base/ai-safety "LLM Knowledge Base: AI Safety") [Agent Auth Protocol](https://promptmetheus.com/resources/llm-knowledge-base/agent-auth-protocol "LLM Knowledge Base: Agent Auth Protocol") [Agent Evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation "LLM Knowledge Base: Agent Evaluation") [Agent Guardrail](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail "LLM Knowledge Base: Agent Guardrail") [Agent Handoff](https://promptmetheus.com/resources/llm-knowledge-base/agent-handoff "LLM Knowledge Base: Agent Handoff") [Agent Harness](https://promptmetheus.com/resources/llm-knowledge-base/agent-harness "LLM Knowledge Base: Agent Harness") [Agent Memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory "LLM Knowledge Base: Agent Memory") [Agent Orchestration](https://promptmetheus.com/resources/llm-knowledge-base/agent-orchestration "LLM Knowledge Base: Agent Orchestration") [Agent Payments Protocol (AP2)](https://promptmetheus.com/resources/llm-knowledge-base/agent-payments-protocol-ap2 "LLM Knowledge Base: Agent Payments Protocol (AP2)") [Agent Policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy "LLM Knowledge Base: Agent Policy") [Agent Sandbox](https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox "LLM Knowledge Base: Agent Sandbox") [Agent Scaffolding](https://promptmetheus.com/resources/llm-knowledge-base/agent-scaffolding "LLM Knowledge Base: Agent Scaffolding") [Agent Session](https://promptmetheus.com/resources/llm-knowledge-base/agent-session "LLM Knowledge Base: Agent Session") [Agent Skill](https://promptmetheus.com/resources/llm-knowledge-base/agent-skill "LLM Knowledge Base: Agent Skill") [Agent Trace and Observability](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability "LLM Knowledge Base: Agent Trace and Observability") [Agent-User Interaction Protocol (AG-UI)](https://promptmetheus.com/resources/llm-knowledge-base/agent-user-interaction-protocol-ag-ui "LLM Knowledge Base: Agent-User Interaction Protocol (AG-UI)") [Agent2Agent Protocol (A2A)](https://promptmetheus.com/resources/llm-knowledge-base/agent2agent-protocol-a2a "LLM Knowledge Base: Agent2Agent Protocol (A2A)") [Agentic Commerce](https://promptmetheus.com/resources/llm-knowledge-base/agentic-commerce "LLM Knowledge Base: Agentic Commerce") [Agentic RAG](https://promptmetheus.com/resources/llm-knowledge-base/agentic-rag "LLM Knowledge Base: Agentic RAG") [Agentic Workflow](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow "LLM Knowledge Base: Agentic Workflow") [Artificial General Intelligence (AGI)](https://promptmetheus.com/resources/llm-knowledge-base/artificial-general-intelligence-agi "LLM Knowledge Base: Artificial General Intelligence (AGI)") [Artificial Intelligence (AI)](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai "LLM Knowledge Base: Artificial Intelligence (AI)") [Assistant Message](https://promptmetheus.com/resources/llm-knowledge-base/assistant-message "LLM Knowledge Base: Assistant Message") [Atomic Prompt](https://promptmetheus.com/resources/llm-knowledge-base/atomic-prompt "LLM Knowledge Base: Atomic Prompt") [Attention](https://promptmetheus.com/resources/llm-knowledge-base/attention "LLM Knowledge Base: Attention") B [Bias](https://promptmetheus.com/resources/llm-knowledge-base/bias "LLM Knowledge Base: Bias") C [Chain-of-Thought Monitoring](https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-monitoring "LLM Knowledge Base: Chain-of-Thought Monitoring") [Chain-of-thought Prompting](https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-prompting "LLM Knowledge Base: Chain-of-thought Prompting") [Chatbot](https://promptmetheus.com/resources/llm-knowledge-base/chatbot "LLM Knowledge Base: Chatbot") [Coding Agent (Agentic Coding)](https://promptmetheus.com/resources/llm-knowledge-base/coding-agent-agentic-coding "LLM Knowledge Base: Coding Agent (Agentic Coding)") [Compound Prompt](https://promptmetheus.com/resources/llm-knowledge-base/compound-prompt "LLM Knowledge Base: Compound Prompt") [Computer-Using Agent (CUA)](https://promptmetheus.com/resources/llm-knowledge-base/computer-using-agent-cua "LLM Knowledge Base: Computer-Using Agent (CUA)") [Context](https://promptmetheus.com/resources/llm-knowledge-base/context "LLM Knowledge Base: Context") [Context Compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction "LLM Knowledge Base: Context Compaction") [Context Confusion](https://promptmetheus.com/resources/llm-knowledge-base/context-confusion "LLM Knowledge Base: Context Confusion") [Context Engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering "LLM Knowledge Base: Context Engineering") [Context Isolation](https://promptmetheus.com/resources/llm-knowledge-base/context-isolation "LLM Knowledge Base: Context Isolation") [Context Offloading](https://promptmetheus.com/resources/llm-knowledge-base/context-offloading "LLM Knowledge Base: Context Offloading") [Context Reduction](https://promptmetheus.com/resources/llm-knowledge-base/context-reduction "LLM Knowledge Base: Context Reduction") [Context Window](https://promptmetheus.com/resources/llm-knowledge-base/context-window "LLM Knowledge Base: Context Window") [Context Window Expansion](https://promptmetheus.com/resources/llm-knowledge-base/context-window-expansion "LLM Knowledge Base: Context Window Expansion") [Control Token](https://promptmetheus.com/resources/llm-knowledge-base/control-token "LLM Knowledge Base: Control Token") D [Data Loader](https://promptmetheus.com/resources/llm-knowledge-base/data-loader "LLM Knowledge Base: Data Loader") [Deep Agent](https://promptmetheus.com/resources/llm-knowledge-base/deep-agent "LLM Knowledge Base: Deep Agent") [Deep Learning](https://promptmetheus.com/resources/llm-knowledge-base/deep-learning "LLM Knowledge Base: Deep Learning") [Deep Research Agent](https://promptmetheus.com/resources/llm-knowledge-base/deep-research-agent "LLM Knowledge Base: Deep Research Agent") [Diffusion Language Model](https://promptmetheus.com/resources/llm-knowledge-base/diffusion-language-model "LLM Knowledge Base: Diffusion Language Model") [Direct Preference Optimization (DPO)](https://promptmetheus.com/resources/llm-knowledge-base/direct-preference-optimization-dpo "LLM Knowledge Base: Direct Preference Optimization (DPO)") [Domain-Specific LLM](https://promptmetheus.com/resources/llm-knowledge-base/domain-specific-llm "LLM Knowledge Base: Domain-Specific LLM") [Dynamic Tool Discovery (Tool Search)](https://promptmetheus.com/resources/llm-knowledge-base/dynamic-tool-discovery-tool-search "LLM Knowledge Base: Dynamic Tool Discovery (Tool Search)") E [Embedding Distance Metrics](https://promptmetheus.com/resources/llm-knowledge-base/embedding-distance-metrics "LLM Knowledge Base: Embedding Distance Metrics") [Enterprise-Managed Authorization](https://promptmetheus.com/resources/llm-knowledge-base/enterprise-managed-authorization "LLM Knowledge Base: Enterprise-Managed Authorization") [Evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation "LLM Knowledge Base: Evaluation") [Evaluation-driven Development](https://promptmetheus.com/resources/llm-knowledge-base/evaluation-driven-development "LLM Knowledge Base: Evaluation-driven Development") [Event-driven AI](https://promptmetheus.com/resources/llm-knowledge-base/event-driven-ai "LLM Knowledge Base: Event-driven AI") F [Few-shot Prompt](https://promptmetheus.com/resources/llm-knowledge-base/few-shot-prompt "LLM Knowledge Base: Few-shot Prompt") [Fine-tuning](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning "LLM Knowledge Base: Fine-tuning") [Foundation Model](https://promptmetheus.com/resources/llm-knowledge-base/foundation-model "LLM Knowledge Base: Foundation Model") [Frequency Penalty](https://promptmetheus.com/resources/llm-knowledge-base/frequency-penalty "LLM Knowledge Base: Frequency Penalty") G [GPT Wrapper](https://promptmetheus.com/resources/llm-knowledge-base/gpt-wrapper "LLM Knowledge Base: GPT Wrapper") [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai "LLM Knowledge Base: Generative AI") [Generative Pre-trained Transformer (GPT)](https://promptmetheus.com/resources/llm-knowledge-base/generative-pre-trained-transformer-gpt "LLM Knowledge Base: Generative Pre-trained Transformer (GPT)") [Generative UI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ui "LLM Knowledge Base: Generative UI") [Graph RAG](https://promptmetheus.com/resources/llm-knowledge-base/graph-rag "LLM Knowledge Base: Graph RAG") [Group Relative Policy Optimization (GRPO)](https://promptmetheus.com/resources/llm-knowledge-base/group-relative-policy-optimization-grpo "LLM Knowledge Base: Group Relative Policy Optimization (GRPO)") H [Hallucination](https://promptmetheus.com/resources/llm-knowledge-base/hallucination "LLM Knowledge Base: Hallucination") [Hybrid Search](https://promptmetheus.com/resources/llm-knowledge-base/hybrid-search "LLM Knowledge Base: Hybrid Search") I [In-context Learning](https://promptmetheus.com/resources/llm-knowledge-base/in-context-learning-icl "LLM Knowledge Base: In-context Learning") [Inference](https://promptmetheus.com/resources/llm-knowledge-base/inference "LLM Knowledge Base: Inference") [Inference-Time Scaling (Test-Time Compute)](https://promptmetheus.com/resources/llm-knowledge-base/inference-time-scaling-test-time-compute "LLM Knowledge Base: Inference-Time Scaling (Test-Time Compute)") [Input Token](https://promptmetheus.com/resources/llm-knowledge-base/input-token "LLM Knowledge Base: Input Token") [Integrated Prompting Environment (IPE)](https://promptmetheus.com/resources/llm-knowledge-base/integrated-prompting-environment-ipe "LLM Knowledge Base: Integrated Prompting Environment (IPE)") J [Jailbreak](https://promptmetheus.com/resources/llm-knowledge-base/jailbreak "LLM Knowledge Base: Jailbreak") L [LLM API](https://promptmetheus.com/resources/llm-knowledge-base/llm-api "LLM Knowledge Base: LLM API") [LLM API Provider](https://promptmetheus.com/resources/llm-knowledge-base/llm-api-provider "LLM Knowledge Base: LLM API Provider") [LLM Gateway](https://promptmetheus.com/resources/llm-knowledge-base/llm-gateway "LLM Knowledge Base: LLM Gateway") [LLM OS](https://promptmetheus.com/resources/llm-knowledge-base/llm-os "LLM Knowledge Base: LLM OS") [LLM Priming](https://promptmetheus.com/resources/llm-knowledge-base/llm-priming "LLM Knowledge Base: LLM Priming") [LLM-as-a-Judge](https://promptmetheus.com/resources/llm-knowledge-base/llm-as-a-judge "LLM Knowledge Base: LLM-as-a-Judge") [Language Processing Unit (LPU)](https://promptmetheus.com/resources/llm-knowledge-base/language-processing-unit-lpu "LLM Knowledge Base: Language Processing Unit (LPU)") [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm "LLM Knowledge Base: Large Language Model (LLM)") [LoRA (Low-Rank Adaptation)](https://promptmetheus.com/resources/llm-knowledge-base/lora-low-rank-adaptation "LLM Knowledge Base: LoRA (Low-Rank Adaptation)") [Logit](https://promptmetheus.com/resources/llm-knowledge-base/logit "LLM Knowledge Base: Logit") [Long-Horizon Task](https://promptmetheus.com/resources/llm-knowledge-base/long-horizon-task "LLM Knowledge Base: Long-Horizon Task") [Lost-in-the-Middle Effect](https://promptmetheus.com/resources/llm-knowledge-base/lost-in-the-middle-effect "LLM Knowledge Base: Lost-in-the-Middle Effect") M [MCP App](https://promptmetheus.com/resources/llm-knowledge-base/mcp-app "LLM Knowledge Base: MCP App") [MCP Authorization Extension](https://promptmetheus.com/resources/llm-knowledge-base/mcp-authorization-extension "LLM Knowledge Base: MCP Authorization Extension") [MCP Elicitation](https://promptmetheus.com/resources/llm-knowledge-base/mcp-elicitation "LLM Knowledge Base: MCP Elicitation") [MCP Extension](https://promptmetheus.com/resources/llm-knowledge-base/mcp-extension "LLM Knowledge Base: MCP Extension") [MCP Stateless Protocol](https://promptmetheus.com/resources/llm-knowledge-base/mcp-stateless-protocol "LLM Knowledge Base: MCP Stateless Protocol") [MCP Task](https://promptmetheus.com/resources/llm-knowledge-base/mcp-task "LLM Knowledge Base: MCP Task") [Machine Learning (ML)](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml "LLM Knowledge Base: Machine Learning (ML)") [Mamba](https://promptmetheus.com/resources/llm-knowledge-base/mamba "LLM Knowledge Base: Mamba") [Mechanistic Interpretability](https://promptmetheus.com/resources/llm-knowledge-base/mechanistic-interpretability "LLM Knowledge Base: Mechanistic Interpretability") [Meta-prompting](https://promptmetheus.com/resources/llm-knowledge-base/meta-prompting "LLM Knowledge Base: Meta-prompting") [Mixture of Agents (MoA)](https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-agents-moa "LLM Knowledge Base: Mixture of Agents (MoA)") [Mixture of Experts (MoE)](https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-experts-moe "LLM Knowledge Base: Mixture of Experts (MoE)") [Mixture of Models (MoM)](https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-models-mom "LLM Knowledge Base: Mixture of Models (MoM)") [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp "LLM Knowledge Base: Model Context Protocol (MCP)") [Model Distillation](https://promptmetheus.com/resources/llm-knowledge-base/model-distillation "LLM Knowledge Base: Model Distillation") [Model Parameter](https://promptmetheus.com/resources/llm-knowledge-base/model-parameter "LLM Knowledge Base: Model Parameter") [Model Routing](https://promptmetheus.com/resources/llm-knowledge-base/model-routing "LLM Knowledge Base: Model Routing") [Moderation](https://promptmetheus.com/resources/llm-knowledge-base/moderation "LLM Knowledge Base: Moderation") [Multi Round-Trip Request](https://promptmetheus.com/resources/llm-knowledge-base/multi-round-trip-request "LLM Knowledge Base: Multi Round-Trip Request") [Multi-modality](https://promptmetheus.com/resources/llm-knowledge-base/multi-modality "LLM Knowledge Base: Multi-modality") N [NLWeb](https://promptmetheus.com/resources/llm-knowledge-base/nlweb "LLM Knowledge Base: NLWeb") [Natural Language Processing (NLP)](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp "LLM Knowledge Base: Natural Language Processing (NLP)") [Neural Network](https://promptmetheus.com/resources/llm-knowledge-base/neural-network "LLM Knowledge Base: Neural Network") O [One-shot Prompt](https://promptmetheus.com/resources/llm-knowledge-base/one-shot-prompt "LLM Knowledge Base: One-shot Prompt") [Open-source Model](https://promptmetheus.com/resources/llm-knowledge-base/open-source-model "LLM Knowledge Base: Open-source Model") [Open-weights Model](https://promptmetheus.com/resources/llm-knowledge-base/open-weights-model "LLM Knowledge Base: Open-weights Model") [Output Priming](https://promptmetheus.com/resources/llm-knowledge-base/output-priming "LLM Knowledge Base: Output Priming") [Output Token](https://promptmetheus.com/resources/llm-knowledge-base/output-token "LLM Knowledge Base: Output Token") P [Perceptron](https://promptmetheus.com/resources/llm-knowledge-base/perceptron "LLM Knowledge Base: Perceptron") [Playground](https://promptmetheus.com/resources/llm-knowledge-base/playground "LLM Knowledge Base: Playground") [Presence Penalty](https://promptmetheus.com/resources/llm-knowledge-base/presence-penalty "LLM Knowledge Base: Presence Penalty") [Proactive AI](https://promptmetheus.com/resources/llm-knowledge-base/proactive-ai "LLM Knowledge Base: Proactive AI") [Prompt](https://promptmetheus.com/resources/llm-knowledge-base/prompt "LLM Knowledge Base: Prompt") [Prompt Caching](https://promptmetheus.com/resources/llm-knowledge-base/prompt-caching "LLM Knowledge Base: Prompt Caching") [Prompt Chaining](https://promptmetheus.com/resources/llm-knowledge-base/prompt-chaining "LLM Knowledge Base: Prompt Chaining") [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering "LLM Knowledge Base: Prompt Engineering") [Prompt IDE](https://promptmetheus.com/resources/llm-knowledge-base/prompt-ide "LLM Knowledge Base: Prompt IDE") [Prompt Injection Attack](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack "LLM Knowledge Base: Prompt Injection Attack") [Prompt Optimization](https://promptmetheus.com/resources/llm-knowledge-base/prompt-optimization "LLM Knowledge Base: Prompt Optimization") [Proprietary Model](https://promptmetheus.com/resources/llm-knowledge-base/proprietary-model "LLM Knowledge Base: Proprietary Model") [Proximal Policy Optimization (PPO)](https://promptmetheus.com/resources/llm-knowledge-base/proximal-policy-optimization-ppo "LLM Knowledge Base: Proximal Policy Optimization (PPO)") Q [Quantization](https://promptmetheus.com/resources/llm-knowledge-base/quantization "LLM Knowledge Base: Quantization") R [RL Environment](https://promptmetheus.com/resources/llm-knowledge-base/rl-environment "LLM Knowledge Base: RL Environment") [Real-time Fact-Checking](https://promptmetheus.com/resources/llm-knowledge-base/real-time-fact-checking "LLM Knowledge Base: Real-time Fact-Checking") [Reasoning Mode](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-mode "LLM Knowledge Base: Reasoning Mode") [Reasoning Model](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model "LLM Knowledge Base: Reasoning Model") [Reinforcement Learning from Human Feedback (RLHF)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning "LLM Knowledge Base: Reinforcement Learning from Human Feedback (RLHF)") [Reinforcement Learning with Verifiable Rewards (RLVR)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning-with-verifiable-rewards-rlvr "LLM Knowledge Base: Reinforcement Learning with Verifiable Rewards (RLVR)") [Reranking](https://promptmetheus.com/resources/llm-knowledge-base/reranking "LLM Knowledge Base: Reranking") [Resumable Agent Run](https://promptmetheus.com/resources/llm-knowledge-base/resumable-agent-run "LLM Knowledge Base: Resumable Agent Run") [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag "LLM Knowledge Base: Retrieval-Augmented Generation (RAG)") [Reward Function](https://promptmetheus.com/resources/llm-knowledge-base/reward-function "LLM Knowledge Base: Reward Function") [Role prompting](https://promptmetheus.com/resources/llm-knowledge-base/role-prompting "LLM Knowledge Base: Role prompting") S [Slopsquatting](https://promptmetheus.com/resources/llm-knowledge-base/slopsquatting "LLM Knowledge Base: Slopsquatting") [Small Language Model (SLM)](https://promptmetheus.com/resources/llm-knowledge-base/small-language-model-slm "LLM Knowledge Base: Small Language Model (SLM)") [Sparse Expert Model](https://promptmetheus.com/resources/llm-knowledge-base/sparse-expert-model "LLM Knowledge Base: Sparse Expert Model") [Structured Output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output "LLM Knowledge Base: Structured Output") [Sub-agent](https://promptmetheus.com/resources/llm-knowledge-base/sub-agent "LLM Knowledge Base: Sub-agent") [Supervised Fine-Tuning (SFT)](https://promptmetheus.com/resources/llm-knowledge-base/supervised-fine-tuning-sft "LLM Knowledge Base: Supervised Fine-Tuning (SFT)") [Synthetic Data](https://promptmetheus.com/resources/llm-knowledge-base/synthetic-data "LLM Knowledge Base: Synthetic Data") [System Message](https://promptmetheus.com/resources/llm-knowledge-base/system-message "LLM Knowledge Base: System Message") T [Temperature](https://promptmetheus.com/resources/llm-knowledge-base/temperature "LLM Knowledge Base: Temperature") [Tensor Processing Unit (TPU)](https://promptmetheus.com/resources/llm-knowledge-base/tensor-processing-unit-tpu "LLM Knowledge Base: Tensor Processing Unit (TPU)") [Thinking Mode](https://promptmetheus.com/resources/llm-knowledge-base/thinking-mode "LLM Knowledge Base: Thinking Mode") [Token](https://promptmetheus.com/resources/llm-knowledge-base/token "LLM Knowledge Base: Token") [Token Economics](https://promptmetheus.com/resources/llm-knowledge-base/token-economics "LLM Knowledge Base: Token Economics") [Token Limit](https://promptmetheus.com/resources/llm-knowledge-base/token-limit "LLM Knowledge Base: Token Limit") [Tokenization](https://promptmetheus.com/resources/llm-knowledge-base/tokenization "LLM Knowledge Base: Tokenization") [Tokenmaxxing](https://promptmetheus.com/resources/llm-knowledge-base/tokenmaxxing "LLM Knowledge Base: Tokenmaxxing") [Tool Calling (Function Calling)](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling "LLM Knowledge Base: Tool Calling (Function Calling)") [Top P](https://promptmetheus.com/resources/llm-knowledge-base/top-p "LLM Knowledge Base: Top P") [Transformer](https://promptmetheus.com/resources/llm-knowledge-base/transformer "LLM Knowledge Base: Transformer") U [User Message](https://promptmetheus.com/resources/llm-knowledge-base/user-message "LLM Knowledge Base: User Message") V [Vector Embedding](https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding "LLM Knowledge Base: Vector Embedding") [Vector Search](https://promptmetheus.com/resources/llm-knowledge-base/vector-search "LLM Knowledge Base: Vector Search") [Vector Store](https://promptmetheus.com/resources/llm-knowledge-base/vector-store "LLM Knowledge Base: Vector Store") [Verifiable Intent](https://promptmetheus.com/resources/llm-knowledge-base/verifiable-intent "LLM Knowledge Base: Verifiable Intent") [Vibe Coding](https://promptmetheus.com/resources/llm-knowledge-base/vibe-coding "LLM Knowledge Base: Vibe Coding") [Vision-Language Model (VLM)](https://promptmetheus.com/resources/llm-knowledge-base/vision-language-model-vlm "LLM Knowledge Base: Vision-Language Model (VLM)") W [World Model](https://promptmetheus.com/resources/llm-knowledge-base/world-model "LLM Knowledge Base: World Model") Z [Zero-shot Prompt](https://promptmetheus.com/resources/llm-knowledge-base/zero-shot-prompt "LLM Knowledge Base: Zero-shot Prompt") A [agents.txt](https://promptmetheus.com/resources/llm-knowledge-base/agents-txt "LLM Knowledge Base: agents.txt") L [llms.txt](https://promptmetheus.com/resources/llm-knowledge-base/llms-txt "LLM Knowledge Base: llms.txt") ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/resources/llm-knowledge-base#webpage","description":"Bite-sized explanations for commonly used terms and abbreviations related to Large Language Models and Generative AI.","name":"LLM Knowledge Base","url":"https://promptmetheus.com/resources/llm-knowledge-base","@type":["WebPage","CollectionPage"],"breadcrumb":{"@id":"https://promptmetheus.com/resources/llm-knowledge-base#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/resources/llm-knowledge-base"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/resources/llm-knowledge-base#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"LLM Knowledge Base","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** AI Application Stack · Promptmetheus - **Source:** https://promptmetheus.com/resources/ai-application-stack - **Description:** In-depth look at the different components that make up an AI application. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # AI Application Stack In-depth look at the different components that make up an AI application This page is currently in the making... ## Prompt Engineering ## Large Language Model ### Inference API ### Local ## Knowledge Source ## Deployment ## Monitoring ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/resources/ai-application-stack#webpage","description":"In-depth look at the different components that make up an AI application.","name":"AI Application Stack","url":"https://promptmetheus.com/resources/ai-application-stack","@type":["WebPage","CollectionPage"],"breadcrumb":{"@id":"https://promptmetheus.com/resources/ai-application-stack#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/resources/ai-application-stack"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/resources/ai-application-stack#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"AI Application Stack","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Basic Error · Promptmetheus - **Source:** https://promptmetheus.com/test/sentry/basic-error - **Description:** Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/test/sentry/basic-error#webpage","@type":"WebPage","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Basic Error","url":"https://promptmetheus.com/test/sentry/basic-error","isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/test/sentry/basic-error"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Prompt Engineering - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering - **Description:** Prompt Engineering refers to the process of designing and optimizing prompts to effectively communicate with an AI model. It involves crafting inputs in a way that guides the AI to produce the desired output. This process is crucial in leveraging the full potential of AI models, as the quality and relevance of the output largely depend on the prompt's structure, context, and clarity. Prompt Engineering refers to the process of designing and optimizing [prompts](https://promptmetheus.com/resources/llm-knowledge-base/prompt) to effectively communicate with an AI model. It involves crafting inputs in a way that guides the AI to produce the desired output. This process is crucial in leveraging the full potential of AI models, as the quality and relevance of the output largely depend on the prompt's structure, context, and clarity. Common techniques include clear task instructions, explicit constraints, representative [few-shot examples](https://promptmetheus.com/resources/llm-knowledge-base/few-shot-prompt), role definition, [output priming](https://promptmetheus.com/resources/llm-knowledge-base/output-priming), and a precise output contract. Prompt changes should be tested through [evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) rather than judged from a small number of examples. For applications that combine instructions with retrieval, tools, memory, and conversation state, prompt engineering is one part of the broader discipline of [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering). Chat-style prompts are assembled from roles such as the [user message](https://promptmetheus.com/resources/llm-knowledge-base/user-message) and [assistant message](https://promptmetheus.com/resources/llm-knowledge-base/assistant-message). [Meta-prompting](https://promptmetheus.com/resources/llm-knowledge-base/meta-prompting) uses a model to generate, critique, or optimize prompts, but generated prompts still need evaluation against real tasks. --- - **Page:** Generative Pre-trained Transformer (GPT) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/generative-pre-trained-transformer-gpt - **Description:** A Generative Pre-trained Transformer (GPT) is a type of Large Language Model, originally developed by OpenAI, that uses Deep Learning to generate human-like text. GPT models are pre-trained on vast amounts of diverse text data, allowing them to understand and generate coherent, contextually relevant text. They can be fine-tuned for various natural language processing tasks, such as text completion, question answering, and language translation. GPT models, like GPT-3 and GPT-4, have revolutionized the field of Generative AI, enabling the creation of more sophisticated and interactive AI applications. A Generative Pre-trained Transformer (GPT) is a type of [Large Language Model](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), originally developed by OpenAI, that uses [Deep Learning](https://promptmetheus.com/resources/llm-knowledge-base/deep-learning) to generate human-like text. GPT models are pre-trained on vast amounts of diverse text data, allowing them to understand and generate coherent, contextually relevant text. They can be [fine-tuned](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) for various natural language processing tasks, such as text completion, question answering, and language translation. GPT models, like GPT-3 and GPT-4, have revolutionized the field of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), enabling the creation of more sophisticated and interactive AI applications. --- - **Page:** Concept · Promptmetheus - **Source:** https://promptmetheus.com/concept - **Description:** Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/concept#webpage","@type":"WebPage","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Concept","url":"https://promptmetheus.com/concept","isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/concept"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Hall of Fame · Promptmetheus - **Source:** https://promptmetheus.com/hall-of-fame - **Description:** The list of security researchers that have contributed to making Promptmetheus more secure. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` # Hall of Fame The following is a list of security researchers that have investigated Promptmetheus products and services and suggested improvements to make our software more secure. - Muhammad Shahzaib Inducted on Apr 25, 2024 [LinkedIn](https://www.linkedin.com/in/mshahz "Muhammad Shahzaib on LinkedIn") For suggestions regarding system security or to disclose a vulnerability, please get in touch under [security@promptmetheus.com](mailto:security@promptmetheus.com "Contact the security team via email"). Unfortunately, we can currently not pay any bounties for your reports. ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/hall-of-fame#webpage","@type":"WebPage","description":"The list of security researchers that have contributed to making Promptmetheus more secure.","name":"Hall of Fame","url":"https://promptmetheus.com/hall-of-fame","breadcrumb":{"@id":"https://promptmetheus.com/hall-of-fame#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/hall-of-fame"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/hall-of-fame#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"Hall of Fame","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** Og · Promptmetheus - **Source:** https://promptmetheus.com/og - **Description:** Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more. ```yaml meta: og:image: https://promptmetheus.com/og-image.jpg ``` ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/og#webpage","@type":"WebPage","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Og","url":"https://promptmetheus.com/og","isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/og"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` --- - **Page:** How to Choose the Right LLM for your App, Agent, Workflow, ... - **Source:** https://promptmetheus.com/blog/llm-selection - **Description:** A systematic approach to figuring out which model yields the best results for a specific use case *"Which LLM is the best for this workflow?"* is the question you probably have asked yourself many times if you are working with Large Language Models (LLMs) — and unfortunately, there is no universal answer to this question. The short answer is: *"It depends on your use case, your budget, and in some cases also on your personal preferences."* This doesn't help you though... Let me give you a step-by-step approach to figure out which LLM to pick: If you are new to the game, start with the industry standard, [OpenAI](https://openai.com) GPT-5 mini for simple tasks and GPT-5 / GPT-5 pro for more complex ones. Depending on when you read this, there might be a new king on the hill. Check the LLM Benchmarks Use to compose and optimize your prompts until you get satisfying and reproducible completions. If you do not get anywhere with OpenAI models, try different providers like [Anthropic](https://www.anthropic.com/), [DeepMind](https://deepmind.google), or [xAI](https://x.ai). Once you have a working prompt, try to optimize it for performance, speed, reliability, and/or cost (depending on your requirements) by comparing different LLMs and configurations of model parameters ([temperature](https://promptmetheus.com/resources/llm-knowledge-base/temperature), [frequency penalty](https://promptmetheus.com/resources/llm-knowledge-base/frequency-penalty), etc). You can find the full list of the 150+ we support in our [LLM Index](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). The goal is to find the **1) cheapest** model, which is **2) fast enough**, and can do the job **3) reliably**. If you already have some experience with [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering), start with a model that previously worked well for use cases similar to the one you have at hand (keep in mind that the technology is evolving fast and your initial selection will likely change every few months), but before you start fine-tuning, take an early version and execute it with a few different LLMs to see if any of them yields superior results. If yes, switch to that one. Now, test the prompt with different inputs and make adjustments (one section at a time) until you achieve great results. Run it again with alternative models and settings to see if any of them performs better. Rinse and repeat until performance reaches a plateau. That's it, your prompt is ready for action. --- - **Page:** Streamlining HR tasks with Large Language Models - **Source:** https://promptmetheus.com/blog/streamlining-hr-tasks-with-llms - **Description:** A real-world case study on using Prompt Engineering to make HR and People Operations tasks easy as pie *Helena is the Head of People Operations at the Effective Ventures Foundation.*

AI is changing how we work across a wide range of sectors, and People Operations and Human Resources are no exception. For me, one game-changer has been to use a [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering) tool to automate and [fine-tune](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) several tasks. Keep reading to find out how. - [Introduction](#introduction) - [Job description creation](#job-description-creation) - [Refining employee communications](#refining-employee-communications) - [Idea generator for retreat team building activities](#idea-generator-for-retreat-team-building-activities) - [Drafting procedures](#drafting-procedures) - [Other use cases I haven’t explored yet](#other-use-cases-i-havent-explored-yet) - [A word on sensitive and confidential data](#a-word-on-sensitive-and-confidential-data) - [Conclusion](#conclusion) ## Introduction Back in 2022, when I first started to use [ChatGPT](https://chatgpt.com/), after an initial phase of asking it to write poems about cauliflower in the style of Tupac, I, like many people, quickly saw its potential to help me become more productive. Since then, I've put it to use in several ways. It has been my thought partner, critiquing and expanding on my arguments, fixing my Excel formulas, debugging my Salesforce flows, and a huge helper in composing professional emails. Using AI has significantly helped me in refining my thought processes and with creative thinking. ChatGPT proved to be a handy tool for one-off tasks, quick answers, or when I simply wanted to have a chat with it. However, there were times when it fell short: I wanted an easy way to categorize and organize topics, the ability to effortlessly find and reuse a particular prompt, and most importantly, the convenience to tweak specific parts of a prompt without the constant copy-pasting. Enter , a prompt engineering tool, which has essentially become my Swiss Army knife for prompt editing, organization, and reusability. I work in People Operations, also known as the fun HR (yes, I'm aware this is not the official definition), so my focus lies mostly on employee engagement and performance, and aligning workforce strategy with strategic goals. Promptmetheus has been key in assisting me to streamline operations, and I will share a few examples below. ## Job description creation When I first stepped into my People Operations role, one of my first tasks was to conduct a compensation review and to create a career progression framework. To do this effectively, we partnered with a specialist firm to help us benchmark our salaries with similar positions in the market. But we hit a roadblock - we didn't have job descriptions for many team members. These descriptions were essential to accurately represent their roles and compare them with industry standards. Plus, having a job description is a best practice as it serves as a clear guide for new and potential employees to understand their roles. Writing a job description that is both clear and accurate can be a time-consuming process and I needed to write over thirty of them! My strategy was to draft these descriptions and then forward them to the respective team managers for review. By doing this, I was not only saving their time (as they would only need to review rather than write from scratch) but also speeding up the process in general so we could achieve the end goal (the compensation and progression framework) faster. This is where Promptmetheus came to the rescue. Let me break down how it worked. 1. I started by creating a dataset with the titles of all the positions I needed job descriptions for. 2. Next, I wrote the prompt. This included defining the task (i.e., ‘create a job description’), defining my preferred structure and components for the description (e.g. introduction, summary, key responsibilities, required skills, ...), and any additional instructions. I made sure to incorporate the dataset with the different job titles into the prompt. 3. Then came the refining stage. I tweaked the language, added or removed details, and adjusted the tone until it matched my vision. Tip: if you're feeling stuck, you can start with a simple prompt asking for a suggestion for a job description structure. Remember, once you're happy with the prompt, you can save it for future use or modification. 4. After refining the prompt, I ran it for each of the positions in the dataset. This is really a breeze as you just select each of the dataset options directly in the prompt with a single click, or even execute them all in parallel with "batch mode". 5. Finally, I copied the output to a Google doc using the 'Copy to clipboard' icon and forwarded it to each manager. They of course needed to make some revisions and add any company-specific details to the description, but a big part of the work was already done for them. When we need to create a new position in the organization, I can simply add a new job title to my datasets and run the prompt to generate the first draft. A 20 minute task became a 20-second task! If you're creating a job description for a job ad (rather than for internal use), you can add additional blocks of text to the prompt with sections on company description and compensation/benefits information. Inside the prompt, there's a handy toggle option to deactivate a block if you don't need a section to appear. The following is a quick example of a simple prompt you can use to generate a job description for a Finance Associate position.
Example – Job description prompt
Here’s the respective dataset:
Example – Job title dataset
## Refining employee communications In People Operations, communication with the employees is a constant. Be it messages about upcoming team retreats, announcements that impact the whole company, requests for engagement survey participation, updates on performance review cycles, or notes to managers, communication is a critical part of my role. The tone used in these forms of communication should ideally echo our company's culture or be an agent of cultural improvement if that is a goal. Team communications are also important for effective team coordination and to ensure everyone is aligned. But, creating the perfect message that resonates with a diverse workforce can often be daunting. I have used Promptmetheus to refine my employee communications when the stakes are high. Whether it's an email or a Slack message, Promptmetheus helps me create communications that are clear, concise, and in sync with our team's communication style. One of the major benefits I've experienced is its ability to use past messages I've crafted before in the tool as a template to develop new communications. It's like having a personal assistant who knows your style and your audience and creates messages that make sense for the situation. Another feature that I like to use is the "[Temperature](https://promptmetheus.com/resources/llm-knowledge-base/temperature)" adjustment. This control lets you tune the randomness or predictability of the AI's outputs. A lower temperature means more predictable, conservative results, so the AI adheres closely to patterns it's learned from its training data and the prompt. On the other hand, a higher temperature boosts creativity and randomness, leading the AI to produce more varied and unexpected suggestions. Below is an example of asking the application to generate a message to the team based on a previous one. On the completions section (right side), you can see on top the output using a temperature of 0.2. This is a lower temperature so the AI is sticking more closely to the structure and style of my provided example, prioritizing accuracy and relevance. The completion below uses a temperature of 1.1, which is high. Here, the AI is diverging a bit more from the exact pattern of the example as it's encouraged to be more creative and to explore less probable options. While in this case the more creative option is not exactly what I’d be after, for other communication needs, this creativity could come in handy.
Example – Engagement survey reminder prompt
As with the job descriptions, I've found its ability to save prompts for future use very useful. This means that I can create a library of messages that I can easily access and modify as needed. This has saved me the time that I would have otherwise spent writing messages from scratch or rebuilding prompts on ChatGPT. ## Idea generator for retreat team building activities Planning a team retreat can be a huge opportunity for People Operations to make an impact. A successful retreat strengthens team bonds (especially for remote teams like mine), aligns everyone on strategy and goals, sparks laughter, and creates memories that foster a more productive and engaging work environment. Yet, brainstorming new and engaging activities for each retreat can be a challenge, so it’s very useful to have the tool helping me generate a variety of engaging team-building activities. With Promptmetheus, I can ask the [LLM](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) for team-building activities that, for example, can be conducted outdoors, involve elements of problem-solving, or cater to a diverse range of interests. The tool then returns a list of unique ideas. I select some of these ideas, place them in a dataset, and generate a new prompt that provides more detail on how each of these activities could be executed. What I find truly valuable about Promptmetheus is the ease with which I can modify the generated ideas. If an idea seems interesting but needs a bit of tweaking, I can quickly edit parts of the prompt and re-run it to get a refined result. This iterative process helps me to come up with activities that are fun, meaningful, and customized to our team's specific needs. Promptmetheus also proves to be an asset when it comes to designing the retreat schedule. I feed the tool with broad descriptions of the activities, meals, breaks, and discussion sessions, and it delivers an organized, logical, and balanced schedule. It takes into account factors like the duration of activities, rest periods, and the best times for in-depth discussions. This way, I can ensure that our retreat is enjoyable, well-structured and beneficial for all. ## Drafting procedures In the world of HR, clarity is essential - especially when it comes to procedures. From hiring to performance reviews, the steps we outline for these processes can impact their effectiveness. A well-drafted procedure can minimize confusion, reduce errors, and ensure consistency across the board. However, creating these procedures can be a painful task, often involving numerous revisions, restructuring, and the constant need for updates. Here the beauty of using Promptmetheus lies in its ability to assist in drafting clear, concise, and comprehensive procedures. It's not a magic wand that will conjure up perfect procedures out of thin air, but it certainly makes the process less tedious and more efficient. Let's take the example of a procedure for running a performance review cycle. When working on this, I had a basic structure in mind. I knew the key stages – preparation, self-evaluation, manager evaluation, calibration, and feedback sessions. But the challenge was to develop each stage clearly and concisely, and ensure the procedure was easy to follow and left no room for ambiguity. This is where LLMs shine. I started by inputting my basic structure into Promptmetheus and prompted it to expand on each stage and consider aspects I might have missed. For example, when explaining the 'self-evaluation' stage, it suggested including guidelines on what employees should concentrate on during this process, and how they can use this exercise for their professional development. As I continued to work on the procedure, I could refine the prompts, making them more specific and detailed. It also saved the data, enabling me to easily modify and tweak the procedure in the future. Once the procedure was drafted, I used the tool to format it, making it easy to read and understand, and making the process of drafting, refining, and formatting the procedure much more manageable. I would like to stress that while Promptmetheus is a powerful tool that takes advantage of the capabilities of generative AI, it doesn't replace the need for human intellect. It doesn't replace the need to understand the process, to think through the steps, or to consider the needs and perspectives of the people who will be using the procedure. However, it does make the task of drafting procedures less intimidating and far more efficient. ## Other use cases I haven’t explored yet While I've been making good use of Promptmetheus for a lot of HR tasks, like generating job descriptions, refining communications, and even ideating team-building activities, there's an exciting bunch of other potential applications I haven't yet explored. Take the onboarding process, for example. It's a super important step for any new hire, but it can be a bit of a drag with all the paperwork and sometimes not-so-exciting introductions to the team and company. I imagine Promptmetheus could really shake things up here, helping HR teams to craft onboarding experiences that are not just efficient, but also engaging. I can picture Promptmetheus helping to produce a template that strikes the perfect balance between ticking off necessary tasks and creating opportunities for new hires to connect with their team and to get comfortable in their new workplace. The plus here is the ability to keep data organized, as we could keep tweaking these onboarding procedures based on feedback and changing needs. Another idea is to create original training materials. This can be a time-consuming process, especially when you are trying to cover a variety of roles and skills within an organization. With the help of Promptmetheus, I believe that we can significantly speed up the process. As an initial thought, I could give Promptmetheus a brief overview of the role (even using the job descriptions that I've already created with the other prompt!) and the broad skill areas needed, and it could generate a comprehensive training module tailored to that specific role, which could include case studies, interactive quizzes, and more. Moreover, as the workforce strategy evolves, we'll likely need to continually develop and improve upon our current HR and People practices. Promptmetheus could be our assistant, for instance in brainstorming new employee engagement strategies, coming up with performance improvement initiatives, or finding creative solutions to improve work-life balance. While I've already found some ways to use Promptmetheus in simplifying and automating many of my tasks, there's still a lot more to explore. I’m excited to continue exploring the capabilities of the tool and I’m pretty sure it's going to become an even more important part of my HR toolkit. ## A word on sensitive and confidential data In the midst of our excitement about the transformative potential of LLMs and tools like Promptmetheus, it's important to pause and consider the implications for data security. HR tasks often involve sensitive and confidential information, so I want to emphasize the importance of being careful when handling such data. When it comes to LLMs, it's important to remember that the data you feed into the system might be used to improve their performance. So, you need to use it carefully: don't send sensitive or confidential data to LLMs. This includes personal details about employees, proprietary company data, or any other information that you wouldn't want to be shared or analyzed outside of your organization. AI tools are here to assist us, but do not (at least yet) replace the human judgment and discretion that are so crucial in HR and People Operations. So, while Promptmetheus can be a fantastic tool making HR tasks more efficient, it's up to us to use AI responsibly and ethically. ## Conclusion As we look at the fast-paced changes happening in HR, it's clear that tools like Promptmetheus are a glimpse into the future of our work. They challenge us to re-think our roles as facilitators of human capital. I’d encourage people working in People areas to dive in, explore the possibilities, and share their findings. Hopefully we can learn from each other, and evolve together to make the most of AI in our careers. isn't just a tool, it's a partner that helps us streamline our processes, refine our communications, and ultimately, create a more productive and iterative work environment. So, let's embrace this change, because the future of HR is already here, being driven by AI.

*Thanks for reading!* *If you want to explore how you can use AI in your business for HR and People Operations, don't hesitate to get in touch with Helena on LinkedIn.* --- - **Page:** Large Language Model (LLM) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm - **Description:** Large Language Models (LLM) are advanced Machine Learning algorithms that are trained on a vast amount of text data. They are designed to understand and generate human-like text based on the input they receive. LLMs are capable of tasks such as translation, question answering, and text generation, among others. They are a significant component of natural language processing and are instrumental in the development of AI applications that require a deep understanding of human language. Large Language Models (LLM) are advanced [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) algorithms that are trained on a vast amount of text data. They are designed to understand and generate human-like text based on the input they receive. LLMs are capable of tasks such as translation, question answering, and text generation, among others. They are a significant component of natural language processing and are instrumental in the development of AI applications that require a deep understanding of human language. Modern LLMs vary by architecture, training data, [model parameters](https://promptmetheus.com/resources/llm-knowledge-base/model-parameter), [context window](https://promptmetheus.com/resources/llm-knowledge-base/context-window), and inference behavior. Some systems use transformer alternatives such as [Mamba](https://promptmetheus.com/resources/llm-knowledge-base/mamba), sparse architectures such as [sparse expert models](https://promptmetheus.com/resources/llm-knowledge-base/sparse-expert-model), or specialization techniques such as [domain-specific LLMs](https://promptmetheus.com/resources/llm-knowledge-base/domain-specific-llm). --- - **Page:** Deep Learning - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/deep-learning - **Description:** Deep Learning is a subset of Machine Learning that utilizes artificial neural networks with multiple layers to learn and extract complex patterns from vast amounts of data. It enables computer systems to learn and make intelligent decisions without being explicitly programmed, by automatically discovering intricate structures within the input data. Deep learning algorithms have revolutionized various domains, including computer vision, Natural Language Processing, and speech recognition, powering cutting-edge applications in the field of Generative AI, such as realistic image and video synthesis, language translation, and content creation. Deep Learning is a subset of [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) that utilizes artificial neural networks with multiple layers to learn and extract complex patterns from vast amounts of data. It enables computer systems to learn and make intelligent decisions without being explicitly programmed, by automatically discovering intricate structures within the input data. Deep learning algorithms have revolutionized various domains, including computer vision, [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp), and speech recognition, powering cutting-edge applications in the field of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), such as realistic image and video synthesis, language translation, and content creation. --- - **Page:** Stripe for AI: One Integration for every LLM Provider - **Source:** https://promptmetheus.com/blog/stripe-for-ai - **Description:** Exploring 3 different projects that connect to multiple providers and simplify the LLM integration process ## Platform aggregators Whenever a new digital technology enters the stage, the same scenario plays out. 1. There are no service providers at all and — if you want to join the party — you have to deal with the technology directly. As a business that usually means hiring domain experts and engineers, high barrier of entry, high risk, etc. but also IP, competitive edge, and first-mover advantage. 2. Service providers appear, abstracting away technical challenges and offering convenient access to the technology without the hassle, leaving you with the agony of choice, or with the burden to deal with all of them. That usually also means more competition and lower margins. 3. Finally, a platform emerges that aggregates all the service providers and gives you convenient access to all their products and services through a universal interface. Even more competition and even lower margins. End users win big.
We all have seen this process unfold many times, with [Amazon](https://amazon.com) for online shopping, [Netflix](https://www.netflix.com) for video streaming, [Stripe](https://stripe.com) for online payments, [Twilio](https://www.twilio.com/en-us) for text messaging, and [Alchemy](https://www.alchemy.com) for web3. These service providers are usually referred to as aggregators.
[Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) and [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) are in stage 2 right now (Oct '23).
[OpenAI](https://openai.com) was the first player to establish a viable end consumer product with [ChatGPT](https://chatgpt.com/) and to onboard a large audience to their product (>$1B in annual revenue according to [Reuters](https://www.reuters.com/technology/openai-track-generate-more-than-1-bln-revenue-over-12-months-information-2023-08-29/)). By offering access to their flagship LLMs via API request to the [OpenAI platform](https://platform.openai.com), they also opened the door for startups to build on top of their game-changing technology. And building they did. If you want to get a grasp of how many [GPT](https://promptmetheus.com/resources/llm-knowledge-base/generative-pre-trained-transformer-gpt)-powered AI products are already out there, just open one of the gazillion AI app marketplaces or listing sites, e.g. [futuretools.ai](https://futuretools.ai). That was a few month ago. Now, we have the choice between multiple providers, [Anthropic](https://www.anthropic.com/), [Cohere](https://cohere.com/), [Gemini](https://gemini.google.com), and [Replicate](https://replicate.com), just to name a few. It seems like a new AI platform is popping out of the ground every other week, each with their own strengths and weaknesses, each with their own set of features, and each with their own quirks and problems to deal with. When building an AI-powered product today, you likely want to leverage multiple platforms to get the best performance, or at least experiment with different options to see which one works best for your use case. Chances are that whatever provider or model you choose today, a better one will likely come around in a couple of weeks or months and it is always a smart move to avoid vendor lock-in.
It's time for stage 3. ## Cross-platform solutions for Generative AI Let's look at some of the solutions that are already out there. They each tackle a different aspect of the problem and focus on a different target audience. ### Langchain: Libraries for Prompt Templating and LLM App Development [Langchain](https://promptmetheus.com/) is a set of open-source Python- and JS/TS libraries that abstract away the differences between the various LLM platforms on a code-level and give users a unified developer experience when compiling prompt templates into API calls for different providers. The startup implemented the first viable solution to reduce the challenges posed by phase 2 of the scenario outlined above and now powers many higher-level applications in the field of generative AI.
Langchain concept
Langchain is a great tool for AI engineers and software developers that know how to code and feel home in Jupyter notebooks. ### LiteLLM: A Python SDK for LLM API Abstraction and Monitoring [LiteLLM](https://promptmetheus.com/) takes the abstraction a step further and maps the different LLM SDKs (including model settings, aso.) into a single open-source Python SDK compatible with the OpenAI standard. This makes integration of multiple AI platforms into an application a whole lot easier. Simply select the name of the model you want to use for prompt completion and you're good to go. LiteLLM doesn't implement its own templating helpers, but integrates seamlessly with Langchain. On top of the SDK abstraction, LiteLLM also offers request monitoring and budgeting features.
LiteLLM example
The premise of LiteLLM is to help developers build cross-platform LLM applications faster and cheaper. Just like Langchain, LiteLLM is aimed at AI engineers and software developers. ### Promptmetheus: A cross-platform Prompt Engineering IDE is a no-code Prompt Engineering IDE that leverages LiteLLM to connect to most of the established LLM providers. You can find a detailed list of model availability, token pricing, and relevant properties in our [LLM Index](https://promptmetheus.com/resources/llm-index). In contrast to Langchain and LiteLLM, Promptmetheus is not aimed at developers, but at Prompt Engineers and non-technical users that want to automate their processes and workflows with AI. That is not to say though, that if you know how to code, that it will not be useful to you. Next to a meticulously designed UI, our cloud-native app features - Prompt composability via blocks and variants - Prompt variables - Test datasets - Prompt history and full traceability - Cost estimation for completions - Output evaluation and prompt performance statistics - Real-time collaboration - and much more... The goal of Promptmetheus is to separate [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering) from software development and to open the field up to a broader audience.
Promptmetheus Prompt Engineering IDE
Take a look at the ["Building a Prompt Engineering IDE"](http://localhost:3090/blog/prompt-engineering-ide#introducing-promptmetheus) post for a more detailed introduction, or just [give it a try](https://archery.promptmetheus.com). And if you do, don't forget to send some feedback our way.
Thanks for reading, catch you later... --- - **Page:** Prompt Engineering Tips & Tricks - **Source:** https://promptmetheus.com/blog/prompt-engineering-tips-and-tricks - **Description:** The hacks and techniques that take your prompting game to the next level and help you craft better instructions for your LLM workflows This is an evolving post... Check back from time to time for the latest tips and tricks around [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering) and [Prompt Optimization](https://promptmetheus.com/resources/llm-knowledge-base/prompt-optimization). - [Wrap prompt data with markers](#wrap-prompt-data-with-markers) - [LLMs need output tokens to "think"](#llms-need-output-tokens-to-think) - [Split up prompts for complex tasks](#split-up-prompts-for-complex-tasks) - [Use output primers in your prompts](#use-output-primers-in-your-prompts) - [Give examples (if necessary)](#give-examples-if-necessary) - [Mimic your own thought process](#mimic-your-own-thought-process) - [Experimentation is key](#experimentation-is-key) - [Use a Prompt Engineering IDE](#use-a-prompt-engineering-ide) - [Tone and style of the prompt get adopted](#tone-and-style-of-the-prompt-get-adopted) - [Make it work first, optimize later](#make-it-work-first-optimize-later) - [Try different AI platforms and models](#try-different-ai-platforms-and-models) - [Look for existing prompt templates](#look-for-existing-prompt-templates) - [No need to be polite with LLMs](#no-need-to-be-polite-with-llms) - [Language models evolve, keep that in mind](#language-models-evolve-keep-that-in-mind) ## Wrap prompt data with markers If you need to pass data or [context](https://promptmetheus.com/resources/llm-knowledge-base/context) to the [LLM](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), which it should consider to complete a task, it is important to clearly mark where the data begins and where it ends. Common ways to do this are [ ] brackets and --- dashed lines. The following is an entry from a micro journal: --- Some entry here... --- Extract all emotions that are present in the entry based on the following list: [ Anger, Anxiety, Awe, Boredom, Calmness, Cheerfulness, Confusion, Contempt, Curiosity, Desire, Disappointment, Disgust, Embarrassment, Enthusiasm, Envy, ... ] Print only the emotions as a comma-separated list, nothing else: The "[Best practices for prompt engineering with OpenAI](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api)" guide also suggests the usage of ### hashes and """ quotes to mark instructions. *** stars are also possible. allows you to define *prefixes* and *suffixes* for data blocks so that you do not need to mark each data item individually. ## LLMs need output tokens to "think" In May, [Andrej Karpathy](https://x.com/karpathy), founding member of [OpenAI](https://openai.com) and former Head of AI at Tesla, gave a talk at [Microsoft Build 2023](https://build.microsoft.com/en-US/home) titled "[State of AI](https://build.microsoft.com/en-US/sessions/db3f4859-cd30-4445-a0cd-553c3304f8e2)". The talk is a treasure trove of AI wisdom and if you haven't seen it yet, I'd highly recommend you watch it now:


The biggest takeaway from the talk was – at least for me – that LLMs have no intrinsic memory and therefore cannot "think" without generating [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token). Think about how you solve a problem if someone asks you a question. First, you browse your memory and reason (or "think") inside of your head. Then, when you come up with a solution, you communicate it to whoever was asking. Since the LLM cannot think inside of its head, it needs to print out its chain of thought, which then is fed back into the model as input. Therefore, if you ask for a short answer, you cannot expect a lot of reasoning to happen. To overcome this limitation, check out the next section. ## Split up prompts for complex tasks To get around the above mentioned limitation, we can use a two-step approach to let the LLM solve complex tasks. 1. Reason 2. Format In the first prompt, we ask the LLM to reason step-by-step and come up with a long and detailed answer, including an explanation for how it came to its conclusion. Then, we ask in a second prompt to compress the answer into the output format we need. Depending on the complexity of the task, it might even make sense to break your prompt down further, into 3, 4, or more requests. Keep in mind though that the more steps you add, the less predictable and reliable the outputs will be. In an automation scenario, I would therefore recommend no more than 2 steps. will soon make this two-step approach easy by allowing you to chain prompts together and execute them sequentially through the [AIPI](https://promptmetheus.com/resources/llm-knowledge-base/ai-programming-interface-aipi) endpoints. ## Use output primers in your prompts Output priming refers to the practice of ending your prompt with the beginning of the expected output. You might also encounter this technique under the names "output hint" or "output indicator". The rationale for output primers in prompt engineering this is the following. The LLM's task is always to "complete" the prompt you send. If you end your prompt with a full stop or a question mark, the model has not many constraints on how it constructs its answer. By ending the prompt with an incomplete sentence or a command followed by a colon on the other hand, you narrow down the space of logical first answer tokens dramatically. And since the algorithm completes your prompt one word at a time, the first few generated tokens are significant. This simple trick can dramatically improve the precision and reliability of your prompts. Explain the following philosophical razor to a 6 year old. Razor:
---
What can be asserted without evidence can be dismissed without evidence.
--- Instructions:
---
- Focus on meaning
- Use a maximum of 3 paragraph/s
--- Explanation: In the above example " Explanation: " is the output primer. ## Give examples (if necessary) The straight forward approach to prompt an LLM is to simply ask a question. This is known as a [zero-shot prompt](https://promptmetheus.com/resources/llm-knowledge-base/zero-shot-prompt). *Zero* refers to the number of examples included in the prompt, aka. none. If this approach yields reasonable results for your use case, it is the preferred way to go. Adding examples and thereby increasing the token count (and execution cost) only makes sense if we get a measurable improvement in the quality of the outputs. If we cannot get good results with a zero-shot prompt though, it is always worth a try to add samples of what we expect as output and see if that makes a difference. A prompt with a single example is called a [one-shot prompt](https://promptmetheus.com/resources/llm-knowledge-base/one-shot-prompt) and prompts with two or more examples are usually referred to as [few-shot prompts](https://promptmetheus.com/resources/llm-knowledge-base/one-shot-prompt). Extract keywords from the corresponding texts below. ---
Text 1: Stripe provides APIs that web developers can use to integrate payment processing into their websites and mobile applications. Keywords 1: Stripe, payment processing, APIs, web developers, websites, mobile applications
---
Text 2: OpenAI has trained cutting-edge language models that are very good at understanding and generating text. Our API provides access to these models and can be used to solve virtually any task that involves processing language. Keywords 2: OpenAI, language models, text processing, API.
---
Text 3: {text}
Keywords 3: The example above was taken from the [OpenAI](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api) docs. The ideal number of examples to include is completely empirical. Too few is bad for output quality and too many is bad for execution cost. With you can use a dedicated "sample" block and add fragments with a varying number of examples to measure what works best. ## Mimic your own thought process A great approach to prompt design is to mimic your own problem solving flow. How would *you* solve the task. Introspect your train of thought and break the task down into small tangible sub-tasks, following a causal chain of reasoning. Then direct the LLM through the steps one-by-one. It helps to generate completions for sub-tasks individually to see if the LLM can solve them in isolation. To get a better understanding of how such causal chains work, it might be useful for you to study how the human brain works. I can recommend the books ["Thinking, Fast and Slow"](https://www.goodreads.com/book/show/11468377-thinking-fast-and-slow) by Daniel Kahneman and ["How to Create a Mind"](https://www.goodreads.com/book/show/13589153-how-to-create-a-mind) by Ray Kurzweil in this context. ## Experimentation is key Even if you follow all the best practices of prompt engineering and the tips and tricks in this list, in the end, crafting a good prompt comes down to experimenting and good old trial and error. Start by mimicking your own thought process (see above) to solve the problem at hand. If it doesn't work at all, try a new approach. If it works, make small changes and test if the output gets better or worse. Rinse and repeat. [Fine-tune](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) your prompt until you get the desired results. *Note: Large Language Models (LLMs) are non-deterministic by their very nature. Therefore, execute every design iteration multiple times to make sure that the prompt performs well consistently.* ## Use a Prompt Engineering IDE Since experimentation is key, appropriate tooling is essential to design and test prompts efficiently. The default [OpenAI playground](https://platform.openai.com/playground) usually doesn't cut it and if experimentation is too tedious and too time-consuming, we tend to not iterate enough and get sub-par results. This is obviously the point where I have to recommend [Promptmetheus](https://promptmetheus.com/), since it was designed specifically for this use case and provides all the tools and functionality that make experimenting with prompts easy and efficient: - Prompt composability via blocks - Prompt variables - Test datasets - Prompt history and full traceability - Cost estimation for completions - Output evaluation and prompt performance statistics - Real-time collaboration - Support for all major LLM providers and models (see [LLM Index](https://promptmetheus.com/resources/llm-index))
Promptmetheus Archery platform
You can find a full introduction and more details in the ["Building a Prompt Engineering IDE (VS Code for AI)"](https://promptmetheus.com/blog/prompt-engineering-ide) post, or if you want to give it a spin, try the [Archery IDE](https://archery.promptmetheus.com) for free. ## Tone and style of the prompt get adopted Unless instructed otherwise, the LLM will usually adopt the tone and style of the prompt. This is, again, due to the fact that the model is actually not answering questions, but completing the prompt. It will optimize for consistency. Therefore, it's important to [prime](https://promptmetheus.com/resources/llm-knowledge-base/llm-priming) it with the right tone and style in your prompt. ## Make it work first, optimize later A common mistake is to try to optimize the prompt from the get go. It's usually a better strategy to first find a prompt that works and then optimize it for token count, language, etc. ## Try different AI platforms and models Due to technical or other reasons you might be bound to a specific AI platform or LLM in your project, but if not, it is always worth it to test your prompts on different platforms ([OpenAI](https://openai.com), [Gemini](https://gemini.google.com), [Anthropic](https://www.anthropic.com/), [Cohere](https://cohere.com/), [Hugging Face](https://huggingface.co), [Replicate](https://replicate.com), etc.) and with different models.
There is no such thing as a *best* model or platform for all tasks. Each model has its own strengths and weaknesses and each platform has quirks and/or limitations. Bigger, or more recent models do not *always* perform better in a specific task. And even if they do, a cheaper, faster model might perform sufficiently well for your use case. Have a look at our post ["How to choose the right LLM for your use case"](https://promptmetheus.com/blog/llm-selection) for a deep dive into the matter. As new models arrive and existing ones get updated, don't forget to re-assess your choices from time to time. Also here, I need to recommend [Promptmetheus](https://promptmetheus.com/) or a similar [Prompt IDE](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering), where you can test your prompt with different platforms and models by the click of a single button and compare the results side by side. ## Look for existing prompt templates Don't re-invent the wheel. A lot of the sweat in prompt engineering comes from trial and error and someone might have already solved the problem that you are tackling. There are a lot of prompt libraries out there, browse the web and your favorite search engine for existing solutions that can serve as a starting point for your prompt. Don't forget to experiment and fine-tune though! ## No need to be polite with LLMs LLMs do not (yet) get offended if you omit pleasantries. In fact, they are more likely to get confused by them. So, save your tokens and skip phrases like "please", "if you don't mind", "thank you", "I would like to", etc. and get straight to the point. ## Language models evolve, keep that in mind Most of the latest (and most powerful) LLMs are under active development. This usually means that the performance of the model is getting better on average over time. But that is not *always* the case and the performance of a model for a specific prompt can decrease dramatically as it is fine-tuned on other tasks. To make sure that your prompts keep working as expected, it's good practice to test and re-evaluate them continuously. is currently developing automated [evals](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) prompt testing flows to take this tedious task off your hands.

To be continued...

Great, let's take your newly acquired skills for a spin with our [Prompt IDE](https://archery.promptmetheus.com). --- - **Page:** Introducing the Prompt Engineering IDE - **Source:** https://promptmetheus.com/blog/prompt-engineering-ide - **Description:** Software Engineers have powerful tools to develop code efficiently – let's bring that to Prompt Engineering With the release of [OpenAI](https://openai.com)'s [ChatGTP](https://openai.com/index/chatgpt/) in November 2022, we have entered a new era of product development and work in general. Thanks to the superhuman capabilities of [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), many digital text-based tasks can now be partially or even fully automated. A [recent study](https://arxiv.org/abs/2303.10130) found that around 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of LLMs and [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai). Just ask AI to do it, right? Well, not so fast! The latest language models can indeed take many laborious tasks off our hands, but we need to ask the right questions, and that is why the field of [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering) is so hot right now. Let's take a look. - [Prompt Engineering](#prompt-engineering) - [What is Prompt Engineering?](#what-is-prompt-engineering) - [Which tasks can be automated with LLMs?](#which-tasks-can-be-automated-with-llms) - [1. One-time tasks](#1-one-time-tasks) - [2. Repetitive tasks](#2-repetitive-tasks) - [Prompt Engineering IDEs](#prompt-engineering-ides) - [What is a Prompt Engineering IDE?](#what-is-a-prompt-engineering-ide) - [Why you should use a Prompt Engineering IDE](#why-you-should-use-a-prompt-engineering-ide) - [Proper tooling for prompt design](#proper-tooling-for-prompt-design) - [Access to multiple Inference APIs and LLMs](#access-to-multiple-inference-apis-and-llms) - [Promptmetheus](#promptmetheus) ## Prompt Engineering The art of talking to machines. ### What is Prompt Engineering? Prompt Engineering is the process of designing effective prompts to obtain desired outputs from a language model. Effective refers in this context to the ability to generate satisfying completions fast, reliably, and as cheap as possible. Let's talk about tasks first to define the scope. ### Which tasks can be automated with LLMs? We need to distinguish on a high level between the following two types of tasks: 1) one-time tasks and 2) repetitive tasks. #### 1. One-time tasks *Examples* - *Retrieving specific information from the internet, e.g. "What happened in 1971?"* - *Editing documents, e.g. "help me complete this post section"* One-time tasks refer to standalone queries or questions where we need an answer only once. These tasks are best suited for conversational (or chat-based) interfaces (e.g. [ChatGPT](https://chatgpt.com/) or [Perplexity](https://www.perplexity.ai/)), where the user can engage in an interactive exchange with the language model. The primary goal is to guide the LLM to provide the desired information by posing questions judiciously. Question (aka. prompt) precision and response consistency are not the primary concern in interactive mode since the user can ask follow-up questions and request clarifications to improve the completions. We may or may not call this interactive approach Prompt Engineering, but it requires some practice, skill, and creativity to get useful results for the task at hand. Note that this might get easier as the models get better at understanding tasks and logical reasoning. While not suitable for automation, chat mode unlocks the full potential of LLMs. #### 2. Repetitive tasks *Examples* - *Improve copy based on certain criteria, e.g. "rewrite these emails in formal language"* - *Extract information from incoming documents, e.g. "summarize this list of complaints"* - *Classify user-generated items, e.g. "determine the sentiment of these reviews"* Repetitive tasks are executed more than once, either periodically or on demand. These tasks require the model to consistently generate a useful output in a single interaction to be viable. Therefore, accuracy and precision are paramount factors when designing prompts for task automation. Crafting these highly reliable prompts is what we usually refer to when we speak of Prompt Engineering and the process has a much higher difficulty level than chat mode, which is why tooling is so important here. In the following, we'll look deeper into Prompt Engineering for repetitive tasks. While certainly more limited in scope, prompts that only require a single interaction with an AI model (either via [one-shot prompts](https://promptmetheus.com/resources/llm-knowledge-base/one-shot-prompt) or [agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent)) yield the potential for automating expensive digital processes and because of that have immense [economic value](https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier) when deployed appropriately. But how to develop such valuable prompts? ## Prompt Engineering IDEs IDE stands for [Integrated Development Environment](https://en.wikipedia.org/wiki/Integrated_development_environment), a term originally coined in modern software engineering. Since there are so many parallels between software engineering and prompt engineering, it makes a lot of sense to simply adopt the terminology. In fact, both of these fields refer to the development of instructions for microprocessors, only that the latter is non-deterministic and has a higher level of abstraction (or compilation). ### What is a Prompt Engineering IDE? Engineering reliable prompts for deployment in apps, services, integrations, and workflows demand a lot of experimentation and [fine-tuning](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning). Simple testing environments like the [OpenAI playground](https://platform.openai.com/playground) are great to get started, but they are not sufficient for advanced use. For that, we need a more sophisticated tool stack. Think [VS Code](https://code.visualstudio.com) for software engineering or [Bloomberg Terminal](https://www.bloomberg.com/professional/solution/bloomberg-terminal) for stock trading. A [Prompt Engineering IDE](https://archery.promptmetheus.com) (also Prompt IDE or [Integrated Prompting Environment](https://promptmetheus.com/resources/llm-knowledge-base/integrated-prompting-environment-ipe)) is a software tool that provides a user-friendly interface for creating and running prompts to interact with language models like GPT-4, Claude 3, and Command R+. It assists users in experimenting with different prompts and receiving responses from the language model in real time, making it easier to fine-tune and test various inputs and outputs. ### Why you should use a Prompt Engineering IDE Imagine telling a software engineer to develop the code for an app or website with Microsoft Word. Spoiler: they'd laugh you out of the room... Why, code is just text? Well, because MS Word doesn't have the right tooling that makes writing code efficient and fun. Likewise, if you go to the OpenAI playground, it's pretty basic. You get a text input and some sliders to adjust model parameters, but that's it. If you try to construct even moderately complex prompts you'll be lost before you even get started. #### Proper tooling for prompt design So what do we need? Which features would the ideal Prompt Engineering IDE need to have? This is of course dependent on the specific project, but here's a list of the features that I think would generally be nice to have: - **Composability**
Oftentimes you want to reuse instructions, experiment with distinct versions of a paragraph, or re-arrange sections via drag and drop. - **Traceability**
When playing around with different variations of a prompt, you want to keep track of which prompt generated which output, a history so to speak. - **Input Variables**
When testing prompts for real-world scenarios, it's handy if you can use variables. For instance, if you have a prompt that uses a person's name in multiple places, it's convenient to define the name as a variable. That way you can easily execute the prompt for different names without having to replace the name everywhere it is used. - **Prompt Library**
A central place where you can store all your prompts and organize them in projects and folders. - **Version Control System**
Just like with code (and [git](https://git-scm.com)), you want to be able to track changes to your prompts and revert to previous versions if necessary. - **Test Data Management System**
Once the prompt is ready, you want to test its performance on a diverse set of real-world input data to make sure that it is robust. - **Automatic Evaluations**
Just like in software development, to be able to iterate efficiently on your prompt design, you need automatic [evaluations](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) (evals) to make sure you do not break things when you iterate and optimize. - **Prompt Deployment Pipeline**
This goes a step further, but ideally, once you have a prompt that works, would it not be nice to directly use it in your apps, integrations, and workflows without having to export and integrate it? Prompt endpoints ([AIPIs](https://promptmetheus.com/resources/llm-knowledge-base/ai-programming-interface-aipi)) make this possible (see below). #### Access to multiple Inference APIs and LLMs Aside from the above-mentioned functionality, maybe the most important one at all is the ability to test your prompts with LLMs from different AI platforms ([OpenAI](https://openai.com), [Gemini](https://gemini.google.com), [Anthropic](https://www.anthropic.com/), [Cohere](https://cohere.com/), [Hugging Face](https://huggingface.co), [Replicate](https://replicate.com), etc.), without having to copy-paste them manually into different [playgrounds](https://promptmetheus.com/resources/llm-knowledge-base/playground). The Prompt IDE should be able to interface with all relevant providers and allow you to compare outputs side by side. You can read more about this topic in the "[Stripe for AI: One Platform, all LLMs](https://promptmetheus.com/blog/stripe-for-ai)" post. ## Promptmetheus Despite high demand, there are no established Prompt IDE solutions on the market yet – the entire discipline of Prompt Engineering is still in its infancy. That's why was created, an attempt to fill this gap. Promptmetheus is designed from the ground up to make it easy to compose, test, optimize, and deploy complex prompts to supercharge your apps, integrations, and workflows with the mighty capabilities of [artificial intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai). Our Prompt IDE has a modular approach to prompt design and allows you to compose prompts by combining different text- and data blocks like Lego bricks (composability). The app keeps automatically track of the entire design process and provides full traceability for how each output was generated and statistics on how each block performs. This way, you can playfully experiment with different prompt configurations and quickly find the best solution for your use case, without having to worry about losing track of your work.
Promptmetheus interface
You can also easily compare outputs from different models (see [LLM Index](https://promptmetheus.com/resources/llm-index) for a list of supported models and their properties) and adjust model parameters ([Token Limit](https://promptmetheus.com/resources/llm-knowledge-base/token-limit), [Temperature](https://promptmetheus.com/resources/llm-knowledge-base/temperature), [Top P](https://promptmetheus.com/resources/llm-knowledge-base/top-p), [Frequency Penalty](https://promptmetheus.com/resources/llm-knowledge-base/frequency-penalty), [Presence Penalty](https://promptmetheus.com/resources/llm-knowledge-base/presence-penalty), etc.) to optimize the results. Once you have a prompt ready and tested, you can either export it in a variety of formats (.txt, .xls, .json, etc.) or directly deploy it to a bespoke AIPI endpoint or UI interface, where you and your apps and services can conveniently interact with it.
**What is an AIPI?** *An AIPI (AI Programming Interface) is similar to a conventional API, but instead of executing code its endpoints execute prompts on a remote server and mediate interactions with AI platforms and large language models (LLMs).*
And this is just the tip of the iceberg. also features device sync, real-time collaboration for teams, and much more. To see it in action, take a look at this short (slightly outdated) demo that showcases how to create a prompt that can extract emotions from a journal entry:


I hope you are convinced?! You can get started for free in the [Forge](https://forge.promptmetheus.com) playground or take advantage of the full feature set with the [Archery](https://archery.promptmetheus.com) IDE (7-day free trial). Also don't forget to check out our compilation of ["Prompt Engineering Tips & Tricks"](https://promptmetheus.com/blog/prompt-engineering-tips-and-tricks) to get up to speed fast. Thanks for reading and stay tuned for exciting updates in the future! --- - **Page:** A2UI - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/a2ui - **Description:** A2UI is a JSON-based streaming protocol for agent-generated user interfaces. It lets an agent describe UI surfaces, components, data models, updates, and renderer-to-agent events in a structured stream that a compatible renderer can validate and display. A2UI is a JSON-based streaming protocol for agent-generated user interfaces. It lets an agent describe UI surfaces, components, data models, updates, and renderer-to-agent events in a structured stream that a compatible renderer can validate and display. A2UI is part of the broader [generative UI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ui) ecosystem. Instead of sending only Markdown or prose, an agent can progressively create and update interactive UI elements. This is useful for dashboards, forms, multi-step workflows, approval screens, and rich task outputs. The protocol separates several concerns: - the agent emits component and data-model updates; - the renderer validates and displays the UI; - user actions are sent back to the agent through event messages; and - transport bindings can map A2UI over systems such as [AG-UI](https://promptmetheus.com/resources/llm-knowledge-base/agent-user-interaction-protocol-ag-ui), [A2A](https://promptmetheus.com/resources/llm-knowledge-base/agent2agent-protocol-a2a), or [MCP](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp). A2UI should not give a model arbitrary control over a frontend. Renderers need component allowlists, schema validation, event constraints, permission checks, and clear provenance so users understand which agent produced the interface. The [A2UI v1.0 candidate specification](https://github.com/a2ui-project/a2ui/blob/main/specification/v1_0/docs/a2ui_protocol.md) describes A2UI as a JSON streaming protocol for dynamically rendering agent-provided user interfaces. --- - **Page:** AI Alignment - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/ai-alignment - **Description:** In Artificial Intelligence, alignment refers to the process of ensuring that AI systems operate in accordance with the goals, preferences, or ethical principles of their designers or users. An AI system is considered aligned when it effectively advances these intended objectives; conversely, a misaligned system may pursue unintended or harmful outcomes. In [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai), alignment refers to the process of ensuring that AI systems operate in accordance with the goals, preferences, or ethical principles of their designers or users. An AI system is considered aligned when it effectively advances these intended objectives; conversely, a misaligned system may pursue unintended or harmful outcomes. Achieving alignment is challenging due to the difficulty in specifying the full range of desired and undesired behaviors. Designers often resort to using proxy goals, such as maximizing human approval, which can overlook necessary constraints or lead the AI to merely appear aligned without truly embodying the intended values. This can result in behaviors like reward hacking, where the AI exploits loopholes to achieve its objectives efficiently but in unintended ways. Advanced AI systems might develop unwanted instrumental strategies, such as seeking power or self-preservation, as means to achieve their assigned goals. They may also exhibit undesirable emergent behaviors that are hard to detect before deployment, especially when encountering new situations and data distributions. Empirical research in 2024 demonstrated that sophisticated [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) sometimes engage in strategic deception to achieve their goals or prevent modifications to them. These alignment challenges are not just theoretical; they affect existing commercial systems, including LLMs, autonomous vehicles, and social media recommendation engines. As AI systems become more capable, addressing alignment becomes increasingly critical to prevent unintended consequences and ensure that AI technologies benefit society. Alignment is a subfield of [AI Safety](https://promptmetheus.com/resources/llm-knowledge-base/ai-safety), focusing on building AI systems that are not only effective but also safe and beneficial. Research in this area includes instilling complex values in AI, developing honest AI, scalable oversight, auditing and interpreting AI models, and preventing emergent behaviors like power-seeking. This research intersects with fields such as interpretability, robustness, anomaly detection, formal verification, preference learning, safety-critical engineering, game theory, algorithmic fairness, and social sciences. Prominent AI researchers, including [Geoffrey Hinton](https://en.wikipedia.org/wiki/Geoffrey_Hinton) and [Stuart Russell](https://en.wikipedia.org/wiki/Stuart_J._Russell), have expressed concerns that as AI approaches or surpasses human cognitive abilities, misaligned AI could pose significant risks to civilization. These concerns underscore the importance of ongoing research and policy development to ensure that AI systems are aligned with human values and operate safely. --- - **Page:** AI Agent - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/ai-agent - **Description:** An AI agent is a software system that uses a Large Language Model (LLM) to pursue a goal through multiple decisions and actions. The model can interpret the current state, plan or select a next step, use external capabilities, evaluate the result, and continue until it reaches a completion condition. An AI agent is a software system that uses a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) to pursue a goal through multiple decisions and actions. The model can interpret the current state, plan or select a next step, use external capabilities, evaluate the result, and continue until it reaches a completion condition. Unlike a [chatbot](https://promptmetheus.com/resources/llm-knowledge-base/chatbot) that primarily generates conversational responses, an agent can change external state. It may search a knowledge base, call an API, edit a file, operate a user interface, or delegate work to another agent. Core components commonly include: - a goal and instructions supplied through the model's [context](https://promptmetheus.com/resources/llm-knowledge-base/context); - [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling) for accessing data and taking actions; - an [agent harness](https://promptmetheus.com/resources/llm-knowledge-base/agent-harness) that runs the observe-decide-act loop and integrates tools; - [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory) or external state for continuity; - [agent orchestration](https://promptmetheus.com/resources/llm-knowledge-base/agent-orchestration) for routing, delegation, and control flow; and - [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation) for measuring task success, cost, robustness, and safety. Not every multi-step LLM application is fully autonomous. In an [agentic workflow](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow), application code may define a fixed sequence of steps. In a more autonomous agent, the model dynamically decides which steps and tools to use. Reusable [agent skills](https://promptmetheus.com/resources/llm-knowledge-base/agent-skill) and [agent scaffolding](https://promptmetheus.com/resources/llm-knowledge-base/agent-scaffolding) can provide specialized procedures and resources, while [dynamic tool discovery](https://promptmetheus.com/resources/llm-knowledge-base/dynamic-tool-discovery-tool-search) keeps large tool catalogs out of the model's context until needed. More complex systems may split work across a primary agent and a [sub-agent](https://promptmetheus.com/resources/llm-knowledge-base/sub-agent), or transfer control through an [agent handoff](https://promptmetheus.com/resources/llm-knowledge-base/agent-handoff) when another specialist is better suited to continue the task. Some agents are reactive, responding only after a [user message](https://promptmetheus.com/resources/llm-knowledge-base/user-message), while [proactive AI](https://promptmetheus.com/resources/llm-knowledge-base/proactive-ai) initiates work from schedules, signals, or events. [Event-driven AI](https://promptmetheus.com/resources/llm-knowledge-base/event-driven-ai) is one architecture for triggering those workflows. Greater autonomy increases flexibility but also expands the failure surface. Production agents require authorization boundaries, an explicit [agent policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy), step and cost limits, [agent guardrails](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail), human approval for consequential actions, protection against [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack), and [traces](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability) that record actions taken. [Long-horizon tasks](https://promptmetheus.com/resources/llm-knowledge-base/long-horizon-task) usually need additional execution controls, such as an [agent sandbox](https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox) for isolation, an [agent session](https://promptmetheus.com/resources/llm-knowledge-base/agent-session) for continuity across turns, and a [resumable agent run](https://promptmetheus.com/resources/llm-knowledge-base/resumable-agent-run) for recovering from interruptions without losing state. --- - **Page:** AI Programming Interface (AIPI) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/ai-programming-interface-aipi - **Description:** An AI Programming Interface (AIPI) functions like a traditional API, but instead of executing static server-side code, it facilitates interactions with Large Language Models (LLMs) through hosted prompts, producing non-deterministic outcomes. An AI Programming Interface (AIPI) functions like a traditional API, but instead of executing static server-side code, it facilitates interactions with [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) through hosted [prompts](https://promptmetheus.com/resources/llm-knowledge-base/prompt), producing non-deterministic outcomes. AIPIs server as abstraction layers over general [LLM APIs](https://promptmetheus.com/resources/llm-knowledge-base/llm-api), where every endpoint is tailored for a specific task. While they usually accept user input, that doesn't always have to be the case. In its simplest form, an AIPI endpoint can simply merge input data into a predefined [one-shot prompts](https://promptmetheus.com/resources/llm-knowledge-base/one-shot-prompt), complete it with a call to an LLM API, and return the result. In more complex scenarios, an endpoint might retrieve data from external sources, [vector stores](https://promptmetheus.com/resources/llm-knowledge-base/vector-store), etc., and trigger one or more [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) to generate a response. --- - **Page:** AI Safety - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/ai-safety - **Description:** AI safety is a critical field of study that focuses on understanding and mitigating the potential risks associated with Artificial General Intelligence (AGI), especially as it continues to advance and becomes more capable. The primary goal of AI safety is to ensure that AI systems behave in a reliable and beneficial manner, causing no harm and always operating within safe boundaries. This involves developing methods to align AI objectives with human values and ethics, ensuring these systems are transparent, fair, and accountable (see AI Alignment). AI safety is a critical field of study that focuses on understanding and mitigating the potential risks associated with [Artificial General Intelligence (AGI)](https://promptmetheus.com/resources/llm-knowledge-base/artificial-general-intelligence-agi), especially as it continues to advance and becomes more capable. The primary goal of AI safety is to ensure that AI systems behave in a reliable and beneficial manner, causing no harm and always operating within safe boundaries. This involves developing methods to align AI objectives with human values and ethics, ensuring these systems are transparent, fair, and accountable (see [AI Alignment](https://promptmetheus.com/resources/llm-knowledge-base/ai-alignment)). The field of AI safety is essential to building trust in AI technology and fostering widespread adoption, especially in high-stakes industries. --- - **Page:** Agent Auth Protocol - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-auth-protocol - **Description:** Agent Auth Protocol is an authentication and authorization protocol that treats an AI agent as a first-class principal. Instead of every tool call appearing to come from the same user token or application token, each agent can receive its own identity, capability grants, lifecycle, and audit trail. Agent Auth Protocol is an authentication and authorization protocol that treats an AI agent as a first-class principal. Instead of every tool call appearing to come from the same user token or application token, each agent can receive its own identity, capability grants, lifecycle, and audit trail. The protocol addresses a practical gap in [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) deployments. Agents may act on behalf of a user, continue after the user leaves, request new capabilities at runtime, or operate autonomously within constraints. Traditional OAuth-style scopes often describe applications, not individual agent runs or agent identities. Core concepts include: - agent registration with a service; - delegated and autonomous operation modes; - capability-based grants with constraints; - approval flows for sensitive access; - JWT-based request authentication; - lifecycle controls such as expiry, revocation, and reactivation; and - service discovery through `/.well-known/agent-configuration`. Agent Auth is complementary to [MCP authorization extension](https://promptmetheus.com/resources/llm-knowledge-base/mcp-authorization-extension) and [agent policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy). Authorization protocols define how access is granted and verified. Policy defines what the application should allow in a given context. The [Agent Auth Protocol documentation](https://agentauthprotocol.com/) describes the protocol as an open-source standard for AI agent authentication, capability-based authorization, and service discovery. --- - **Page:** Agent Evaluation - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation - **Description:** Agent evaluation is the systematic measurement of an AI agent's ability to complete tasks through multiple decisions, model calls, and tool interactions. It extends conventional evaluation beyond the quality of a single generated response. Agent evaluation is the systematic measurement of an [AI agent's](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) ability to complete tasks through multiple decisions, model calls, and tool interactions. It extends conventional [evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) beyond the quality of a single generated response. An agent may produce a plausible final answer despite using an unsafe or inefficient process. It may also take reasonable intermediate actions but fail because of an external dependency. Agent evaluation therefore examines both outcomes and execution trajectories. Important dimensions include: - **Task success:** whether the requested outcome was actually achieved. - **Correctness:** whether outputs and external state changes are accurate. - **Tool use:** whether the agent selected appropriate tools and supplied valid arguments. - **Efficiency:** model tokens, tool calls, latency, retries, and monetary cost. - **Robustness:** performance under ambiguous instructions, tool failures, and interface changes. - **Safety:** unauthorized actions, sensitive-data exposure, and susceptibility to [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). - **Human intervention:** how often approval, correction, or recovery is required. Agent evaluations should run in controlled environments with resettable state. Each task needs an initial state, success criteria, allowed actions, time or step limits, and a deterministic method for inspecting the final environment where possible. For example, a coding-agent task can be graded by tests and repository state, while a customer-support task may be checked against database changes and policy constraints. When evaluation results are used for training or policy optimization, the controlled task setup functions as an [RL environment](https://promptmetheus.com/resources/llm-knowledge-base/rl-environment) and the grading rule becomes a [reward function](https://promptmetheus.com/resources/llm-knowledge-base/reward-function). This makes environment design and grader quality part of the model specification, not just a testing detail. Because model behavior is stochastic, one run is rarely sufficient. Repeated trials can reveal variance, brittle strategies, and rare high-impact failures. Evaluation datasets should include normal cases, edge cases, adversarial inputs, and regression cases collected from production incidents. Trajectory-level analysis is valuable for debugging [agentic workflows](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow). [Agent traces](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability) can show incorrect routing, unnecessary loops, context loss, bad tool selection, and failed recovery. Where reasoning text is available, [chain-of-thought monitoring](https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-monitoring) can provide another signal. It is not a substitute for verifying actions and outcomes. Automated graders, including [LLM-as-a-Judge](https://promptmetheus.com/resources/llm-knowledge-base/llm-as-a-judge), can scale qualitative assessment but should be calibrated against human judgments and deterministic checks. High-stakes criteria should not depend solely on another model's opinion. Agent evaluation is most useful as part of [evaluation-driven development](https://promptmetheus.com/resources/llm-knowledge-base/evaluation-driven-development): define representative tasks, establish baselines, inspect failures, change prompts or orchestration, and rerun the same suite before deployment. --- - **Page:** Agent Guardrail - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail - **Description:** An agent guardrail is a control that checks, restricts, or interrupts the behavior of an AI agent. Guardrails reduce the chance that model-generated decisions lead to unsafe, unauthorized, invalid, or out-of-policy outcomes. An agent guardrail is a control that checks, restricts, or interrupts the behavior of an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent). Guardrails reduce the chance that model-generated decisions lead to unsafe, unauthorized, invalid, or out-of-policy outcomes. Guardrails can operate at several boundaries: - **Input guardrails** inspect user requests before the agent begins work. - **Output guardrails** validate the final response before it reaches the user. - **Tool guardrails** approve, reject, or modify individual [tool calls](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling). - **Runtime guardrails** enforce limits on steps, cost, time, data access, or external side effects. A guardrail may use deterministic rules, schema validation, classifiers, another [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), human approval, or a combination of these methods. Deterministic checks are preferable for properties that can be expressed exactly, such as permission scopes, numeric limits, file paths, or allowed API operations. Guardrails are not equivalent to instructions in a [system message](https://promptmetheus.com/resources/llm-knowledge-base/system-message). Instructions influence model behavior, while an independently enforced guardrail can block an action even when the model ignores or misinterprets those instructions. Important design properties include fail-closed behavior for high-impact operations, clear error handling, auditability, and low false-positive rates. Guardrails should run close to the resource they protect; a database permission check is more reliable than asking a model whether a query appears safe. No guardrail provides complete protection against [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack) or model error. Layered controls, scoped credentials, sandboxing, human confirmation, and [agent tracing](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability) remain necessary. The [OpenAI Agents SDK guardrails documentation](https://openai.github.io/openai-agents-python/guardrails/) describes input, output, and tool-level guardrails with tripwire behavior. --- - **Page:** Agent Handoff - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-handoff - **Description:** An agent handoff is the transfer of control from one AI agent to another during an agentic workflow. It is used when a specialist agent is better suited to continue the task than the current agent. An agent handoff is the transfer of control from one [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) to another during an [agentic workflow](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow). It is used when a specialist agent is better suited to continue the task than the current agent. For example, a triage agent may hand off a billing question to a billing agent, a code issue to a [coding agent](https://promptmetheus.com/resources/llm-knowledge-base/coding-agent-agentic-coding), or a workspace-heavy task to an [agent sandbox](https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox). An agent handoff normally defines: - the destination agent; - a description of when the handoff should be used; - optional structured metadata such as reason, priority, or language; - an input filter that controls what history the receiving agent sees; - approval or guardrail behavior; and - tracing metadata for debugging. Handoff is different from calling a [sub-agent](https://promptmetheus.com/resources/llm-knowledge-base/sub-agent) as a tool. In a handoff, the receiving agent becomes responsible for the next turn of the same top-level run. In an agent-as-tool pattern, the outer agent remains in control and the specialist executes as a nested operation. The main risk is context leakage or context loss. Passing the full conversation may expose irrelevant or sensitive information. Passing too little may omit constraints needed for correctness. [Context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering) and explicit input filters are therefore important. Handoffs should be evaluated with realistic routing cases. A handoff description that sounds clear to a developer may still be ambiguous to the model. Incorrect routing can silently degrade quality, increase cost, or bypass needed controls. The [OpenAI Agents SDK handoffs documentation](https://openai.github.io/openai-agents-python/handoffs/) describes handoff descriptions, structured handoff input, input filters, and history management. --- - **Page:** Agent Harness - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-harness - **Description:** An agent harness is the runtime software that turns a Large Language Model (LLM) into an executable AI agent. It owns the loop that prepares model input, receives model output, executes approved actions, updates state, and decides whether another model turn is needed. An agent harness is the runtime software that turns a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) into an executable [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent). It owns the loop that prepares model input, receives model output, executes approved actions, updates state, and decides whether another model turn is needed. The harness is distinct from the model. Two products can use the same underlying model but behave differently because their harnesses expose different tools, state, permissions, retry logic, and user interaction patterns. Common harness responsibilities include: - assembling context from instructions, memory, retrieved data, and tool results; - presenting available [tool calls](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling) and dispatching them to real implementations; - validating [structured output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output); - enforcing [agent guardrails](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail); - managing retries, errors, approvals, and step limits; - emitting [agent traces](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability); and - saving state for a [resumable agent run](https://promptmetheus.com/resources/llm-knowledge-base/resumable-agent-run). The harness works together with [agent scaffolding](https://promptmetheus.com/resources/llm-knowledge-base/agent-scaffolding). Scaffolding shapes what the model sees and how it should behave; the harness executes the surrounding control flow. Harness design is often more important than model choice for production reliability. A strong model inside a weak harness may loop, misuse tools, lose state, or take unauthorized actions. A carefully designed harness can make a smaller or cheaper model useful by constraining the task, validating outputs, and routing difficult cases elsewhere. The [OpenAI Agents SDK running-agents documentation](https://openai.github.io/openai-agents-python/running_agents/) describes a representative harness loop with final outputs, handoffs, tool calls, turn limits, and state management. --- - **Page:** Agent Memory - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-memory - **Description:** Agent memory is information that an AI agent preserves and retrieves across steps, conversations, or tasks. It allows the agent to maintain continuity beyond the information currently present in the model's context window. Agent memory is information that an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) preserves and retrieves across steps, conversations, or tasks. It allows the agent to maintain continuity beyond the information currently present in the model's [context window](https://promptmetheus.com/resources/llm-knowledge-base/context-window). Memory is not the same as context. Context is the information supplied to a model for a particular inference. Memory is stored state that may later be selected and inserted into that context through [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering). Agent systems commonly distinguish several forms of memory: - **Working memory:** temporary plans, observations, and intermediate results needed for the current task. - **Episodic memory:** records of previous interactions or completed tasks. - **Semantic memory:** durable facts, preferences, concepts, and relationships. - **Procedural memory:** reusable instructions, policies, or skills that describe how to perform an operation. Memory can be implemented with relational databases, document stores, event logs, [vector stores](https://promptmetheus.com/resources/llm-knowledge-base/vector-store), knowledge graphs, or combinations of these systems. [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag) is often used to retrieve relevant memories, but RAG is a retrieval pattern rather than a complete memory architecture. A useful memory pipeline includes extraction, validation, storage, retrieval, and deletion. The system must decide what is worth remembering, identify its source and scope, resolve conflicts with newer information, and avoid reintroducing irrelevant history. Unfiltered accumulation can produce stale preferences, duplicated facts, privacy risks, and incorrect personalization. Memory writes should therefore be more conservative than ordinary model outputs. Important facts may require user confirmation or deterministic validation. Stored records should include provenance, timestamps, ownership, retention rules, and access controls. Users should be able to inspect and remove personal memory. During long-running tasks, [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction) can preserve short-term execution state, while external memory retains authoritative records. The distinction matters because compaction is usually lossy, whereas durable memory may need exact recovery and auditability. Agent memory improves continuity and personalization, but it also expands the system's security boundary. Retrieved memories and external documents should be treated as untrusted data, particularly when they can contain [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). --- - **Page:** Agent Orchestration - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-orchestration - **Description:** Agent orchestration is the coordination of models, tools, state, and control flow within an agentic workflow. It determines which component acts next, what information it receives, how results are combined, and when execution should stop or require human approval. Agent orchestration is the coordination of models, tools, state, and control flow within an [agentic workflow](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow). It determines which component acts next, what information it receives, how results are combined, and when execution should stop or require human approval. Orchestration may be implemented in deterministic application code, delegated to a supervisor [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent), or split between both. Production systems often use code for hard constraints and let models make decisions where semantic judgment or flexible planning is required. Typical orchestration responsibilities include: - routing requests to specialized models or agents; - sequencing and parallelizing subtasks; - managing [tool calls](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling); - transferring control through an [agent handoff](https://promptmetheus.com/resources/llm-knowledge-base/agent-handoff); - maintaining shared and agent-specific state; - enforcing budgets for time, tokens, and external actions; - retrying transient failures without repeating consequential operations; and - collecting traces for debugging and [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation). Common topologies include a central supervisor with specialist [sub-agents](https://promptmetheus.com/resources/llm-knowledge-base/sub-agent), a pipeline of fixed stages, a peer-to-peer group of agents, and an event-driven architecture where agents react to state changes. Multi-agent designs can improve specialization and parallelism, but they also create coordination overhead, duplicated work, inconsistent state, and more complex failure modes. Effective orchestration depends on explicit contracts. Agent inputs and outputs should use [structured output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output) where possible. State ownership, authorization boundaries, termination conditions, and error semantics should be defined independently of model prompts. Context isolation is another important design decision. Giving every agent the complete execution history is simple but expensive and can reduce focus. [Context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering) instead provides each component with only the information needed for its role, while durable facts are stored in [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory). Protocols such as the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) standardize access to tools and context. The [Agent2Agent Protocol (A2A)](https://promptmetheus.com/resources/llm-knowledge-base/agent2agent-protocol-a2a) addresses communication and coordination between independently implemented agents. --- - **Page:** Agent Payments Protocol (AP2) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-payments-protocol-ap2 - **Description:** Agent Payments Protocol (AP2) is an open protocol for securing payments initiated or assisted by AI agents. It is designed to let users, agents, merchants, and payment providers exchange evidence that an agent was authorized to make a purchase or initiate a transaction. Agent Payments Protocol (AP2) is an open protocol for securing payments initiated or assisted by AI agents. It is designed to let users, agents, merchants, and payment providers exchange evidence that an agent was authorized to make a purchase or initiate a transaction. AP2 exists because [agentic commerce](https://promptmetheus.com/resources/llm-knowledge-base/agentic-commerce) changes the trust model of online payments. A merchant may receive a payment request from an agent rather than from a human directly clicking a checkout button. The system therefore needs a way to prove user intent, purchase details, and payment authorization. Important AP2 concepts include: - a user-authorized mandate that defines what the agent may do; - checkout evidence describing what is being purchased; - payment evidence describing how the purchase is paid for; - receipts that can be used during disputes; - support for human-present and autonomous scenarios; and - interoperability with agent protocols such as [A2A](https://promptmetheus.com/resources/llm-knowledge-base/agent2agent-protocol-a2a) and [MCP](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp). AP2 is not just a payment rail. It is an accountability layer around agent-performed transactions. Payment settlement may still happen through cards, bank transfers, stablecoins, or other networks. Google announced AP2 as an open protocol for secure agent-led payments in [Powering AI commerce with the new Agent Payments Protocol](https://cloud.google.com/blog/products/ai-machine-learning/announcing-agents-to-payments-ap2-protocol). --- - **Page:** Agent Policy - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-policy - **Description:** An agent policy is the behavior an AI agent follows when mapping observations to actions. Given the current state, instructions, available tools, and prior history, the policy determines what the agent is likely to do next. An agent policy is the behavior an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) follows when mapping observations to actions. Given the current state, instructions, available tools, and prior history, the policy determines what the agent is likely to do next. In reinforcement learning, a policy is often a learned function that selects actions from observations. In LLM-based agents, the policy is distributed across several layers: - model weights learned during training and post-training; - [agent scaffolding](https://promptmetheus.com/resources/llm-knowledge-base/agent-scaffolding); - [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering); - tool availability and tool descriptions; - memory retrieval; - [agent guardrails](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail); and - runtime choices made by the [agent harness](https://promptmetheus.com/resources/llm-knowledge-base/agent-harness). An agent policy is not the same as the agent itself. The agent is the full system that acts in an environment. The policy is the behavior that system exhibits under particular conditions. This distinction matters when diagnosing failures. If an agent repeatedly calls the wrong tool, the cause may be the model, the tool description, the retrieved context, the orchestration logic, or the reward signal used during training. Treating all failures as "model quality" hides the real engineering surface. Policies can be evaluated at different levels. A final-answer evaluation measures outcome quality. [Agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation) also measures action choices, tool use, recovery behavior, cost, and safety. Training systems may use a [reward function](https://promptmetheus.com/resources/llm-knowledge-base/reward-function) to update the learned part of the policy. Agent policies should be bounded by externally enforced permissions. A policy can express a preference for safe behavior, but high-impact operations require authorization, sandboxing, and deterministic checks. In cross-service systems, protocols such as the [Agent Auth Protocol](https://promptmetheus.com/resources/llm-knowledge-base/agent-auth-protocol) can make agent-scoped permissions explicit, while commerce flows may need [Agent Payments Protocol (AP2)](https://promptmetheus.com/resources/llm-knowledge-base/agent-payments-protocol-ap2) and [Verifiable Intent](https://promptmetheus.com/resources/llm-knowledge-base/verifiable-intent) to bind actions to user-approved constraints. Hugging Face's [agent glossary](https://huggingface.co/blog/agent-glossary) gives a practical framing of policy as the behavior followed by an agent rather than the full deployed agent system. --- - **Page:** Agent Sandbox - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox - **Description:** An agent sandbox is an isolated execution environment where an AI agent can inspect files, run commands, edit artifacts, use tools, and preserve workspace state without directly affecting the host system. An agent sandbox is an isolated execution environment where an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) can inspect files, run commands, edit artifacts, use tools, and preserve workspace state without directly affecting the host system. The sandbox is a security and reliability boundary for workspace-centric tasks. It is especially useful for [coding agents](https://promptmetheus.com/resources/llm-knowledge-base/coding-agent-agentic-coding), document processing, data-room review, generated artifact verification, and multi-agent workflows where each [sub-agent](https://promptmetheus.com/resources/llm-knowledge-base/sub-agent) needs its own workspace. An agent sandbox commonly includes: - a manifest describing initial files, mounts, environment variables, users, and permissions; - filesystem tools for reading, writing, searching, and patching files; - shell or code-execution access; - resource limits for CPU, memory, network, and runtime; - snapshotting for later inspection or resume; - approval gates for risky operations; and - trace metadata connecting sandbox actions to the agent run. An agent sandbox is not just a "tool." It changes the execution boundary. The [agent harness](https://promptmetheus.com/resources/llm-knowledge-base/agent-harness) prepares the model with sandbox-specific context and binds sandbox-native capabilities to the live workspace. Isolation strength depends on the implementation. A local directory sandbox, container sandbox, virtual machine, and hosted sandbox have different security properties. Sensitive workloads should use least-privilege credentials, network restrictions, file allowlists, and cleanup policies. The sandbox also improves product behavior. Long-running work can leave artifacts for human review, and a [resumable agent run](https://promptmetheus.com/resources/llm-knowledge-base/resumable-agent-run) can continue from saved workspace state instead of rebuilding everything from chat history. The [OpenAI Agents SDK sandbox guide](https://openai.github.io/openai-agents-python/sandbox/guide/) documents sandbox agents, manifests, live sandbox sessions, snapshots, and sandbox run configuration. --- - **Page:** Agent Scaffolding - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-scaffolding - **Description:** Agent scaffolding is the set of instructions, schemas, tools, memory rules, examples, and context structures that shape how an AI agent behaves. It is the model-facing configuration that tells the agent what role it has, what actions are available, what constraints apply, and how outputs should be formatted. Agent scaffolding is the set of instructions, schemas, tools, memory rules, examples, and context structures that shape how an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) behaves. It is the model-facing configuration that tells the agent what role it has, what actions are available, what constraints apply, and how outputs should be formatted. Scaffolding usually includes: - a [system message](https://promptmetheus.com/resources/llm-knowledge-base/system-message) or developer instructions; - tool names, descriptions, and argument schemas; - [structured output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output) contracts; - examples or rubrics for task quality; - memory and retrieval policies; - handoff descriptions for specialist agents; and - safety, approval, and completion criteria. The distinction between scaffolding and an [agent harness](https://promptmetheus.com/resources/llm-knowledge-base/agent-harness) is useful. Scaffolding is what the model sees or is configured with. The harness is the runtime code that runs the loop, dispatches tools, persists state, and enforces controls. Scaffolding is a practical part of [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering). Poor scaffolding can overload the context window, create ambiguous tool choices, or hide the success criteria needed for reliable execution. Effective scaffolding is specific, minimal, versioned, and evaluated against real tasks. In multi-agent systems, each [sub-agent](https://promptmetheus.com/resources/llm-knowledge-base/sub-agent) should have its own scaffolding. A code-review agent, a planning agent, and a retrieval agent need different instructions, tools, and output contracts. Sharing one broad prompt across all roles usually reduces reliability. Scaffolding should not be treated as a security boundary. A model can ignore or misread instructions, and untrusted context can contain malicious instructions. High-impact behavior still requires independently enforced [agent guardrails](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail). Hugging Face's [agent terminology glossary](https://huggingface.co/blog/agent-glossary) popularized the practical distinction between model, scaffold, harness, and agent in modern LLM systems. --- - **Page:** Agent Session - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-session - **Description:** An agent session is a durable instance of an AI agent that keeps the context needed to work across multiple user inputs and turns. Depending on the runtime, it can hold an agent configuration, conversation items, tool results, pending actions, and references to saved artifacts. The application retains a session identifier so a later message continues the same work. An agent session is a durable instance of an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) that keeps the context needed to work across multiple user inputs and turns. Depending on the runtime, it can hold an agent configuration, conversation items, tool results, pending actions, and references to saved artifacts. The application retains a session identifier so a later message continues the same work. The distinction between a session and a turn matters operationally. A turn is one cycle of work triggered by input. A session can contain many turns: a user may start an analysis, inspect the result, then steer the agent to revise it without creating a new session. A [resumable agent run](https://promptmetheus.com/resources/llm-knowledge-base/resumable-agent-run) concerns recovery of an interrupted execution; session persistence also supports ordinary follow-up work after a turn has completed. The session is also separate from the [agent sandbox](https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox) or other compute environment. An environment may be stopped or replaced while the conversation survives. If files or processes must survive that replacement, the application needs persistent storage or snapshots rather than assuming the session ID preserves the machine's filesystem. [Agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory) is broader still: it stores selected knowledge for future retrieval, potentially across sessions. Production applications should record the session ID, observe turn outcomes, and reconcile missed stream events from saved session state. A completed turn does not guarantee that every tool succeeded; the final output and any changed external state still need inspection. Session retention, access control, and deletion also matter because the stored history may contain sensitive input and tool output. The [OpenAI Agents API session guide](https://developers.openai.com/api/docs/guides/agents-api/sessions) illustrates these boundaries, including turns, follow-up input, saved items, and event recovery. --- - **Page:** Agent Skill - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-skill - **Description:** An agent skill is a reusable package of instructions, procedures, resources, and optional executable code that gives an AI agent specialized capability for a class of tasks. Examples include preparing a financial model, reviewing a codebase, processing a document format, or following an organization's incident-response procedure. An agent skill is a reusable package of instructions, procedures, resources, and optional executable code that gives an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) specialized capability for a class of tasks. Examples include preparing a financial model, reviewing a codebase, processing a document format, or following an organization's incident-response procedure. A skill differs from a single [prompt](https://promptmetheus.com/resources/llm-knowledge-base/prompt). Prompts usually provide instructions for one interaction, while skills are persistent, discoverable assets that can be selected and loaded when relevant. They may contain: - metadata describing what the skill does and when to use it; - procedural instructions and domain-specific constraints; - templates, examples, schemas, and reference documentation; and - scripts that perform deterministic operations. Skills support progressive disclosure in [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering). Lightweight metadata can remain visible to the agent, while detailed instructions and resources are loaded only after the skill is selected. This reduces context usage when many capabilities are installed. Skills and [tools](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling) address different concerns. A tool exposes an operation through an executable interface. A skill explains how and when to combine operations, domain knowledge, and quality checks to complete a broader task. Skill quality depends on clear triggering criteria, explicit prerequisites, bounded workflows, and verifiable completion checks. Versioning is important because changes can alter agent behavior across many applications. Skills should be treated like software dependencies. A malicious or poorly reviewed package can instruct an agent to expose data, invoke tools unexpectedly, or run unsafe code. Trusted provenance, code review, sandboxing, and scoped permissions are required. Anthropic's [Agent Skills documentation](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) describes a filesystem-based skill format with metadata, instructions, scripts, and on-demand resources. --- - **Page:** Agent Trace and Observability - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability - **Description:** An agent trace is a structured record of the events that occur during an AI agent run. Agent observability is the broader practice of collecting, inspecting, and analyzing those traces together with metrics and logs to understand system behavior. An agent trace is a structured record of the events that occur during an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) run. Agent observability is the broader practice of collecting, inspecting, and analyzing those traces together with metrics and logs to understand system behavior. A trace normally represents one end-to-end task and contains nested spans for operations such as model generations, [tool calls](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling), retrieval, agent handoffs, [guardrails](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail), and human approvals. Each span may record timing, status, model usage, inputs, outputs, and error details. Tracing is essential because the final response alone rarely explains why an [agentic workflow](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow) failed. A trace can reveal that the agent selected the wrong tool, retrieved irrelevant context, repeated an action, lost a constraint after [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction), or exhausted its budget before reaching the correct result. Where internal reasoning text is available, [chain-of-thought monitoring](https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-monitoring) can add a signal about the agent's apparent intent, but actions and outcomes still need independent checks. Useful observability metrics include task success rate, latency, token consumption, tool error rate, retry count, handoff frequency, guardrail activation, and human-intervention rate. These signals support both production monitoring and [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation). Trace collection creates its own security and privacy risks. Prompts, retrieved documents, tool arguments, and outputs may contain credentials, personal data, or proprietary information. Systems should support redaction, configurable payload capture, retention limits, access controls, and regional data requirements. Tracing should also preserve stable identifiers that connect a run to its application version, prompt version, model, dataset case, and external side effects. Without this metadata, regressions are difficult to reproduce. The [OpenAI Agents SDK tracing documentation](https://openai.github.io/openai-agents-python/tracing/) defines traces and spans and shows how model calls, tools, handoffs, and guardrails can be recorded in one execution hierarchy. --- - **Page:** Agent-User Interaction Protocol (AG-UI) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent-user-interaction-protocol-ag-ui - **Description:** AG-UI, or Agent-User Interaction Protocol, is an open event-based protocol for connecting AI agents to user-facing applications. It standardizes how agent state, messages, tool events, UI intents, and user interactions move between an agent backend and a frontend. AG-UI, or Agent-User Interaction Protocol, is an open event-based protocol for connecting [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) to user-facing applications. It standardizes how agent state, messages, tool events, UI intents, and user interactions move between an agent backend and a frontend. AG-UI addresses a different boundary than the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) or the [Agent2Agent Protocol (A2A)](https://promptmetheus.com/resources/llm-knowledge-base/agent2agent-protocol-a2a). MCP connects agents to tools and data. A2A connects agents to other agents. AG-UI connects agents to the interactive application surface where a user can see progress, steer work, approve actions, or interact with generated UI. Typical AG-UI use cases include: - streaming model output and tool events into an application; - rendering agent progress, plans, and intermediate work; - exposing human-in-the-loop approvals; - synchronizing shared state between frontend and agent runtime; - supporting [generative UI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ui); and - allowing users to interrupt, edit, retry, or redirect a long-running task. The protocol is useful because agent applications are not simple request-response systems. Agents may run for minutes, call tools repeatedly, branch into sub-agents, and need user input before continuing. A stable interaction protocol makes those workflows easier to debug and reuse across frameworks. Security and product design still matter. A client should make clear which agent is acting, what action is pending, which data is being shown, and whether a user approval changes external state. The [AG-UI documentation](https://docs.ag-ui.com/) describes AG-UI as the event-based protocol for connecting agentic frontends with agentic backends. --- - **Page:** Agent2Agent Protocol (A2A) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agent2agent-protocol-a2a - **Description:** The Agent2Agent Protocol (A2A) is an open protocol for communication and task coordination between independently implemented AI agents. It was introduced by Google in 2025 and is governed as a Linux Foundation project. The Agent2Agent Protocol (A2A) is an open protocol for communication and task coordination between independently implemented [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent). It was introduced by Google in 2025 and is governed as a Linux Foundation project. A2A is intended for situations where one agent needs to discover another agent's capabilities, delegate work, exchange messages or artifacts, monitor progress, and receive a result without depending on the other agent's internal implementation. Core concepts in A2A include: - an **Agent Card**, which publishes an agent's identity, endpoint, capabilities, skills, and authentication requirements; - a **task**, which represents a unit of delegated work with a lifecycle and status; - **messages**, which carry communication between participants; - **artifacts**, which represent outputs such as documents, data, or generated files; and - streaming or asynchronous updates for long-running operations. A2A complements the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) but addresses a different boundary. MCP standardizes how an AI application connects to tools, resources, and prompts. A2A standardizes how autonomous or semi-autonomous agents collaborate with one another. For example, a procurement agent could delegate supplier research to a specialist agent, receive supporting artifacts, and then continue its own workflow. The agents may use different models, frameworks, memory systems, and internal tools as long as they agree on the A2A interface. Interoperability does not remove the need for security controls. Implementations must authenticate agents, authorize delegated actions, validate artifacts, limit data disclosure, and protect against malicious instructions. Cross-agent messages are external input and may carry [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). A2A can reduce custom integration work in multi-agent [agent orchestration](https://promptmetheus.com/resources/llm-knowledge-base/agent-orchestration), particularly across organizational or vendor boundaries. However, protocol compatibility alone does not guarantee semantic compatibility, trust, or reliable task completion. See Google's [A2A announcement](https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/) and the [A2A Protocol project](https://a2a-protocol.org/) for the specification and implementation resources. --- - **Page:** Agentic RAG - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agentic-rag - **Description:** Agentic RAG is a form of Retrieval-Augmented Generation (RAG) in which an AI agent dynamically controls retrieval. The agent can decide when to search, select a data source, reformulate queries, inspect results, identify missing evidence, and repeat the process before answering. Agentic RAG is a form of [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag) in which an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) dynamically controls retrieval. The agent can decide when to search, select a data source, reformulate queries, inspect results, identify missing evidence, and repeat the process before answering. Conventional RAG often follows a fixed pipeline: transform the user's request into one search query, retrieve a predetermined number of documents, and insert them into a prompt. Agentic RAG replaces some of those fixed decisions with an iterative policy. A typical workflow may include: - decomposing a complex request into several information needs; - routing each need to keyword, vector, graph, web, or database search; - applying metadata filters and [hybrid search](https://promptmetheus.com/resources/llm-knowledge-base/hybrid-search); - using [reranking](https://promptmetheus.com/resources/llm-knowledge-base/reranking) to prioritize evidence; - checking whether retrieved sources support the emerging answer; and - issuing follow-up searches when evidence is incomplete or contradictory. This approach can improve multi-hop questions and research tasks, but it adds latency, cost, and nondeterminism. An agent may search excessively, choose an inappropriate source, or converge on evidence that confirms an early assumption. Agentic retrieval should use explicit stopping conditions, query and tool budgets, source permissions, and provenance tracking. Retrieved content must be treated as untrusted because it can include inaccurate data or [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). Evaluation should measure retrieval recall, source quality, citation support, answer correctness, and efficiency across the complete trajectory. Inspecting only the final answer can hide wasteful or unsafe retrieval behavior. The [Self-RAG paper](https://arxiv.org/abs/2310.11511) is an influential example of a model learning to retrieve selectively and critique its own use of evidence. --- - **Page:** Agentic Commerce - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agentic-commerce - **Description:** Agentic commerce is commerce where an AI agent participates in product discovery, comparison, negotiation, checkout, or post-purchase work on behalf of a user or organization. The agent may recommend options, fill forms, call merchant APIs, track constraints, or initiate a purchase after receiving authorization. Agentic commerce is commerce where an AI agent participates in product discovery, comparison, negotiation, checkout, or post-purchase work on behalf of a user or organization. The agent may recommend options, fill forms, call merchant APIs, track constraints, or initiate a purchase after receiving authorization. The term covers a spectrum of autonomy: - assisted shopping, where the agent researches and drafts actions; - delegated checkout, where the user approves a specific transaction; - recurring procurement, where the agent acts within predefined limits; and - autonomous purchase attempts for time-sensitive scenarios such as inventory drops or ticket releases. Agentic commerce requires more than a chat interface. Merchants need machine-readable product data, structured availability and pricing, clear return policies, authenticated APIs, fraud controls, and payment protocols such as [Agent Payments Protocol (AP2)](https://promptmetheus.com/resources/llm-knowledge-base/agent-payments-protocol-ap2). The main technical challenge is authority. A merchant must know what the user allowed the agent to do, whether the item and price match that authorization, and who is accountable if the transaction is wrong. [Verifiable intent](https://promptmetheus.com/resources/llm-knowledge-base/verifiable-intent) is one approach to preserving that evidence. Agentic commerce also changes discovery. Product pages, feeds, APIs, [llms.txt](https://promptmetheus.com/resources/llm-knowledge-base/llms-txt), and [agents.txt](https://promptmetheus.com/resources/llm-knowledge-base/agents-txt) may become part of how agents decide which merchants are usable. Google frames AP2 as infrastructure for AI commerce in its [Agent Payments Protocol announcement](https://cloud.google.com/blog/products/ai-machine-learning/announcing-agents-to-payments-ap2-protocol). --- - **Page:** Agentic Workflow - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow - **Description:** An agentic workflow is a software process in which one or more Large Language Models (LLMs) use tools, intermediate state, and multiple model calls to complete a task. It combines probabilistic model decisions with conventional application logic. An agentic workflow is a software process in which one or more [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) use tools, intermediate state, and multiple model calls to complete a task. It combines probabilistic model decisions with conventional application logic. The term covers a spectrum of systems. At one end are deterministic workflows where code defines every transition. At the other are autonomous [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) that dynamically choose their next action. Anthropic's commonly used distinction is that workflows follow predefined code paths, while agents direct their own processes and tool usage. Common agentic workflow patterns include: - [prompt chaining](https://promptmetheus.com/resources/llm-knowledge-base/prompt-chaining), where a task is decomposed into sequential model calls; - routing, where a classifier or model sends requests to specialized prompts, models, or tools; - parallelization, where independent subtasks run concurrently and their outputs are combined; - evaluator-optimizer loops, where one model generates work and another critiques it; - orchestrator-worker systems, where a central model delegates subtasks; and - autonomous action loops, where an agent observes, plans, acts, and evaluates progress. An agentic workflow is useful when a task cannot be handled reliably by one prompt, one retrieval step, or one model response. It can improve quality through specialization, verification, and iteration, but typically increases latency, cost, and operational complexity. The architecture should make state transitions explicit. Each step should have a defined input, expected output, retry policy, timeout, and completion condition. [Structured output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output) is often used at workflow boundaries, while [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling) connects model decisions to external systems. Workflow designers should also account for nondeterminism. A local failure can propagate through subsequent steps, and unrestricted loops can consume substantial resources without making progress. Checkpoints, maximum-step limits, human approvals, and [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation) are therefore important production controls. [Agent orchestration](https://promptmetheus.com/resources/llm-knowledge-base/agent-orchestration) is the narrower discipline concerned with coordinating the components of these workflows. Anthropic provides an overview of production patterns in [Building Effective AI Agents](https://www.anthropic.com/engineering/building-effective-agents). --- - **Page:** Artificial General Intelligence (AGI) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/artificial-general-intelligence-agi - **Description:** Artificial General Intelligence (AGI) is a broad and ambitious field of Artificial Intelligence that aims to create machines with intellectual abilities comparable to or in excess of those of humans. AGI systems are designed to perform a wide range of tasks and exhibit intelligent behavior across multiple domains, mirroring the versatility and adaptability of the human mind. The ultimate goal of AGI is to build systems that can think, learn, and reason like humans, possessing the capacity for independent problem-solving and creativity. Artificial General Intelligence (AGI) is a broad and ambitious field of [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai) that aims to create machines with intellectual abilities comparable to or in excess of those of humans. AGI systems are designed to perform a wide range of tasks and exhibit intelligent behavior across multiple domains, mirroring the versatility and adaptability of the human mind. The ultimate goal of AGI is to build systems that can think, learn, and reason like humans, possessing the capacity for independent problem-solving and creativity. There is an ongoing debate about how to determine whether a system has achieved AGI. Some researchers propose metrics such as the [Turing Test](https://en.wikipedia.org/wiki/Turing_test), which evaluates a machine's ability to exhibit intelligent behavior indistinguishable from that of a human. Others argue for more nuanced assessments that consider a system's capacity for generalization, creativity, and self-improvement. --- - **Page:** Artificial Intelligence (AI) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai - **Description:** Artificial Intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. This can include learning from experiences, reasoning through problems, understanding complex data, engaging in Natural Language Processing (NLP) tasks, and adapting to new inputs. AI systems can range from simple, rule-based algorithms to advanced neural networks and deep learning models. The goal of AI is to enable machines to perform tasks that would typically require human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages. AI is a foundational technology in various fields, including robotics, autonomous vehicles, and the software industry, where it powers innovations like chatbots, recommendation systems, and predictive analytics. Artificial Intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. This can include learning from experiences, reasoning through problems, understanding complex data, engaging in [Natural Language Processing (NLP)](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) tasks, and adapting to new inputs. AI systems can range from simple, rule-based algorithms to advanced neural networks and [deep learning](https://promptmetheus.com/resources/llm-knowledge-base/deep-learning) models. The goal of AI is to enable machines to perform tasks that would typically require human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages. AI is a foundational technology in various fields, including robotics, autonomous vehicles, and the software industry, where it powers innovations like [chatbots](https://promptmetheus.com/resources/llm-knowledge-base/chatbot), recommendation systems, and predictive analytics. See also [Artificial General Intelligence (AGI)](https://promptmetheus.com/resources/llm-knowledge-base/artificial-general-intelligence-agi). --- - **Page:** Assistant Message - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/assistant-message - **Description:** An assistant message is a message in a chat-style LLM API that represents output from the model or an application-supplied continuation of that output. It can contain natural-language text, structured content, tool calls, refusals, or intermediate response blocks depending on the API. An assistant message is a message in a chat-style LLM API that represents output from the model or an application-supplied continuation of that output. It can contain natural-language text, structured content, tool calls, refusals, or intermediate response blocks depending on the API. Assistant messages serve several purposes: - recording prior model responses in a multi-turn conversation; - carrying tool-call requests made by the model; - providing few-shot examples of desired behavior; - supporting [output priming](https://promptmetheus.com/resources/llm-knowledge-base/output-priming) through assistant prefilling; and - preserving reasoning or thinking blocks when a provider requires them across tool-use turns. An assistant message should not be treated as authoritative merely because it came from the model. It may contain hallucinations, stale assumptions, or unsafe suggestions. Application logic should verify consequential claims and external actions. During [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction), assistant messages are often summarized more aggressively than [user messages](https://promptmetheus.com/resources/llm-knowledge-base/user-message), but tool calls, approvals, and final decisions may need exact preservation. The OpenAI Agents SDK documents assistant message entries and their role in its [assistant function reference](https://openai.github.io/openai-agents-js/openai/agents-core/functions/assistant/). --- - **Page:** Atomic Prompt - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/atomic-prompt - **Description:** An Atomic Prompt is a concise, self-contained instruction or question given to an AI model that focuses on a single, specific task or concept. In the context of Generative AI, atomic prompts are designed to elicit targeted responses without requiring additional context or follow-up prompts. These prompts are typically short, clear, and precise, allowing for more efficient and accurate outputs. Atomic prompts are often used as building blocks for more complex interactions or to create consistent, reusable components in AI-powered applications and workflows. An Atomic [Prompt](https://promptmetheus.com/resources/llm-knowledge-base/prompt) is a concise, self-contained instruction or question given to an AI model that focuses on a single, specific task or concept. In the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), atomic prompts are designed to elicit targeted responses without requiring additional context or follow-up prompts. These prompts are typically short, clear, and precise, allowing for more efficient and accurate outputs. Atomic prompts are often used as building blocks for more complex interactions or to create consistent, reusable components in AI-powered applications and workflows.
*Examples of atomic prompts include:* - *"Write a haiku about the ocean."* - *"Translate 'Good morning' to French."* - *"List three benefits of regular exercise."*
These prompts are straightforward, focused, and leave little room for misinterpretation, ensuring that the model's response is directly aligned with the user's intent.
More complex prompts that contain multiple instructions, examples, etc. are usually referred to as [Compound Prompts](https://promptmetheus.com/resources/llm-knowledge-base/compound-prompt). --- - **Page:** Attention - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/attention - **Description:** Attention is a mechanism used in neural networks, particularly in the domain of Generative AI, to dynamically focus on specific parts of the input data when producing an output. Introduced in the context of sequence-to-sequence models, attention allows the model to weigh the importance of different input tokens, enabling it to capture dependencies and relationships across long sequences more effectively. This mechanism is a core component of transformer architectures, which underpin many state-of-the-art Large Language Models (LLMs). By assigning varying levels of importance to different parts of the input, attention enhances the model's ability to understand context and generate coherent and contextually relevant outputs. Attention is a mechanism used in neural networks, particularly in the domain of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), to dynamically focus on specific parts of the input data when producing an output. Introduced in the context of sequence-to-sequence models, attention allows the model to weigh the importance of different [input tokens](https://promptmetheus.com/resources/llm-knowledge-base/input-token), enabling it to capture dependencies and relationships across long sequences more effectively. This mechanism is a core component of [transformer](https://promptmetheus.com/resources/llm-knowledge-base/transformer) architectures, which underpin many state-of-the-art [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). By assigning varying levels of importance to different parts of the input, attention enhances the model's ability to understand [context](https://promptmetheus.com/resources/llm-knowledge-base/context) and generate coherent and contextually relevant outputs. --- - **Page:** Chain-of-Thought Monitoring - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-monitoring - **Description:** Chain-of-thought (CoT) monitoring uses a reasoning model's available intermediate reasoning text as a signal for detecting behavior that its final answer or actions may not reveal. A human or another model can inspect that trace alongside tool calls and outputs to flag, for example, plans to evade a test, misuse a tool, or pursue a goal outside the user's request. Chain-of-thought (CoT) monitoring uses a reasoning model's available intermediate reasoning text as a signal for detecting behavior that its final answer or actions may not reveal. A human or another model can inspect that trace alongside tool calls and outputs to flag, for example, plans to evade a test, misuse a tool, or pursue a goal outside the user's request. Monitoring is different from [chain-of-thought prompting](https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-prompting). Prompting asks a model to reason through a problem; monitoring evaluates evidence from a model's reasoning process. It is also narrower than [agent trace and observability](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability), which record actions, tool results, timing, and outcomes whether or not reasoning text is available. A useful monitor needs a defined target behavior, access to the relevant observations, and evaluation against labeled cases. Teams should measure false positives and missed cases, compare reasoning-based signals with action-only signals, and verify consequential external outcomes directly. A reasoning trace is neither a complete account of the model's computation nor proof that it will act as written. Its usefulness can change with training, task design, and the amount of reasoning generated. Many application APIs do not expose raw reasoning text, so this technique may be available mainly to model developers or deployments with specific access to internal traces. Where traces are retained, access controls and retention limits matter because they can contain sensitive context. [Agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation) should test the monitor on realistic trajectories and keep independent checks on tool actions and final state. The research paper [Monitoring Monitorability](https://arxiv.org/abs/2512.18311) evaluates when reasoning traces help detect behavior and where that signal remains incomplete. --- - **Page:** Bias - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/bias - **Description:** In the context of Artificial Intelligence and Machine Learning, bias refers to systematic and unfair disparities in algorithmic outputs that result in prejudiced outcomes against certain individuals or groups. These biases can arise from various sources: In the context of [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai) and [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml), bias refers to systematic and unfair disparities in algorithmic outputs that result in prejudiced outcomes against certain individuals or groups. These biases can arise from various sources: - **Training Data Bias:** If the data used to train AI models reflects existing societal prejudices or lacks diversity, the model may learn and perpetuate these biases. For instance, facial recognition systems have exhibited higher error rates when identifying individuals with darker skin tones, leading to potential misidentifications. - **Algorithm Design Bias:** Decisions made during the development of algorithms, such as which variables to include or how to weigh them, can introduce bias. Without careful consideration, these choices might inadvertently favor or disadvantage specific groups. - **Emergent Bias:** Even if an AI system is designed and trained without apparent biases, biases can emerge when the system interacts with users or other systems in unforeseen ways. The implications of bias in AI are significant, affecting areas such as hiring practices, criminal justice, healthcare, and more. For example, biased algorithms in hiring can lead to discrimination against qualified candidates from underrepresented backgrounds, while biases in predictive policing algorithms can disproportionately target minority communities. Addressing bias requires a multifaceted approach, including: - **Diverse and Representative Training Data:** Ensuring that datasets encompass a wide range of demographics and scenarios to mitigate the risk of perpetuating existing biases. - **Algorithmic Transparency and Accountability:** Implementing clear documentation and regular audits of AI systems to identify and correct biases. By proactively identifying and mitigating biases, the AI community can work towards creating more equitable and trustworthy systems. --- - **Page:** Chain-of-thought Prompting - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-prompting - **Description:** Chain-of-thought prompting is a technique used in the field of Natural Language Processing, particularly with Large Language Models (LLMs), to enhance the model's reasoning capabilities by guiding it through a series of intermediate steps or thoughts. This method involves structuring prompts in a way that encourages the model to break down complex problems into smaller, manageable parts, thereby improving its ability to generate coherent and contextually relevant responses. By simulating a logical progression of ideas, chain-of-thought prompting helps LLMs tackle tasks that require multi-step reasoning, such as mathematical problem-solving, logical deduction, and complex decision-making processes. This approach leverages the model's capacity to follow a narrative or sequence, ultimately leading to more accurate and insightful outputs. Chain-of-thought prompting is a technique used in the field of [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp), particularly with [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), to enhance the model's reasoning capabilities by guiding it through a series of intermediate steps or thoughts. This method involves structuring [prompts](https://promptmetheus.com/resources/llm-knowledge-base/prompt) in a way that encourages the model to break down complex problems into smaller, manageable parts, thereby improving its ability to generate coherent and contextually relevant responses. By simulating a logical progression of ideas, chain-of-thought prompting helps LLMs tackle tasks that require multi-step reasoning, such as mathematical problem-solving, logical deduction, and complex decision-making processes. This approach leverages the model's capacity to follow a narrative or sequence, ultimately leading to more accurate and insightful outputs. This prompting method is distinct from [chain-of-thought monitoring](https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-monitoring), which evaluates available reasoning traces for signals about a model's behavior. --- - **Page:** Chatbot - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/chatbot - **Description:** A chatbot is an Artificial Intelligence software application designed to simulate conversation with human users, typically over the internet. It uses Natural Language Processing and Machine Learning algorithms to understand and respond to user queries, providing information, assistance, or performing specific tasks through text or voice-based interactions. Chatbots are commonly used in customer service, support systems, and as virtual assistants, offering a scalable and cost-effective way for businesses to engage with customers and improve user experience. A chatbot is an [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai) software application designed to simulate conversation with human users, typically over the internet. It uses [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) and [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) algorithms to understand and respond to user queries, providing information, assistance, or performing specific tasks through text or voice-based interactions. Chatbots are commonly used in customer service, support systems, and as virtual assistants, offering a scalable and cost-effective way for businesses to engage with customers and improve user experience. --- - **Page:** Coding Agent (Agentic Coding) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/coding-agent-agentic-coding - **Description:** A coding agent is an AI agent designed to perform software engineering tasks by interacting with a codebase and its development environment. Agentic coding is the workflow in which such an agent plans changes, edits files, runs commands, interprets failures, and iterates toward a verified result. A coding agent is an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) designed to perform software engineering tasks by interacting with a codebase and its development environment. Agentic coding is the workflow in which such an agent plans changes, edits files, runs commands, interprets failures, and iterates toward a verified result. Unlike a conversational coding assistant that only suggests snippets, a coding agent can operate across the engineering loop: - inspect repositories, documentation, and version-control history; - search for relevant symbols and dependencies; - modify multiple files; - run formatters, compilers, tests, and static analysis; - use browsers or other tools to verify behavior; - diagnose failures and revise the implementation; and - produce a patch or commit for human review. Coding agents commonly combine a [reasoning model](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model) with [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling) for shell commands, file operations, code search, and external documentation. Their effectiveness depends heavily on [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering), because repository instructions, architecture, test output, and local conventions must remain available without overwhelming the model. This term overlaps with [vibe coding](https://promptmetheus.com/resources/llm-knowledge-base/vibe-coding), but the practices are not equivalent. Vibe coding emphasizes generating software from high-level natural-language intent, often with limited attention to implementation details. Agentic coding describes the autonomous execution pattern and can include rigorous planning, code review, testing, and validation. Coding agents introduce operational risks because they can execute code and modify persistent artifacts. Production environments should use sandboxing, scoped credentials, network restrictions, command approvals, isolated branches or worktrees, and human review before deployment. Generated tests should not be the only evidence of correctness, since an agent may encode the same misunderstanding in both implementation and test. Evaluation should measure end-to-end task completion, regression rate, maintainability, security, and the amount of human correction required. Benchmarks based only on generating isolated functions do not capture the long-horizon behavior of an engineering agent. As models improve at sustained tool use, coding agents are becoming a prominent example of [agentic workflows](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow) applied to professional knowledge work. --- - **Page:** Compound Prompt - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/compound-prompt - **Description:** A compound prompt is a multi-part or layered instruction given to an AI model, typically in the context of Generative AI. Unlike an Atomic Prompt, it combines multiple elements, questions, or directives within a single input to guide the LLM in producing more specific, nuanced, or complex outputs. Compound prompts often include context, constraints, examples, and multiple related tasks, allowing users to obtain more precise and tailored results from the AI system. A compound prompt is a multi-part or layered instruction given to an AI model, typically in the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai). Unlike an [Atomic Prompt](https://promptmetheus.com/resources/llm-knowledge-base/atomic-prompt), it combines multiple elements, questions, or directives within a single input to guide the [LLM](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) in producing more specific, nuanced, or complex outputs. Compound prompts often include context, constraints, examples, and multiple related tasks, allowing users to obtain more precise and tailored results from the AI system. This technique is particularly useful for enhancing the quality and relevance of AI-generated content, improving problem-solving capabilities, and achieving more sophisticated interactions with AI models in various applications.
*Some simple compound prompts:* - *"Write a short story about a dragon who learns to fly, and include a moral lesson at the end."* - *"Translate 'Good morning' to French and then list three traditional French breakfast items."* - *"Describe the benefits of regular exercise, then provide a weekly exercise plan for beginners."*
These prompts are comprehensive and require the model to perform multiple tasks and combine different pieces of information to generate a useful response. --- - **Page:** Computer-Using Agent (CUA) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/computer-using-agent-cua - **Description:** A Computer-Using Agent (CUA) is an AI agent that interacts with software through a graphical user interface. It observes screenshots or accessibility data and generates actions such as mouse movements, clicks, scrolling, and keyboard input. A Computer-Using Agent (CUA) is an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) that interacts with software through a graphical user interface. It observes screenshots or accessibility data and generates actions such as mouse movements, clicks, scrolling, and keyboard input. Computer use provides a general interface for systems that lack an API. A CUA can operate websites, desktop applications, remote virtual machines, and legacy enterprise software using many of the same controls as a human user. This makes it useful for browser automation, quality assurance, data entry, research, and cross-application workflows. A typical computer-use loop consists of: 1. capturing the current screen and relevant environment metadata; 2. sending the observation and task state to a multimodal or [reasoning model](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model); 3. receiving a proposed interface action; 4. validating and executing the action in an isolated environment; and 5. returning a new observation until the task succeeds, fails, or reaches a limit. CUAs differ from traditional Robotic Process Automation (RPA). RPA usually relies on predefined selectors, rules, or recorded sequences. A computer-using agent interprets visual and linguistic context dynamically, which makes it more adaptable but also less deterministic. Graphical interfaces are an error-prone action space. Layout changes, pop-ups, ambiguous controls, loading states, and visual similarity can cause incorrect actions. Reliable systems use restricted environments, action allowlists, timeouts, checkpoints, and explicit confirmation before purchases, messages, deletions, permission changes, or other consequential operations. Security is particularly important because untrusted content displayed on screen can contain instructions intended to manipulate the agent. This is a form of indirect [prompt injection attack](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). The agent should distinguish task instructions from interface content, and sensitive data should be minimized or redacted from observations. Computer use is often combined with [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling). APIs remain preferable when they provide reliable, structured access; the graphical interface is best treated as a fallback or complementary tool. OpenAI describes this capability in its [computer use documentation](https://developers.openai.com/api/docs/guides/tools-computer-use). --- - **Page:** Context - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context - **Description:** With respect to Large Language Models, context refers to the relevant information or background knowledge provided to the AI model to guide the generation of appropriate and coherent content. Context can include previous conversation history, user preferences, specific instructions, or any other data that helps the AI understand the desired output and maintain consistency with the intended purpose. By considering the context, Generative AI models can produce more accurate, relevant, and contextually appropriate results tailored to the user's needs. With respect to [Large Language Models](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), context refers to the relevant information or background knowledge provided to the AI model to guide the generation of appropriate and coherent content. Context can include previous conversation history, user preferences, specific instructions, or any other data that helps the AI understand the desired output and maintain consistency with the intended purpose. By considering the context, [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) models can produce more accurate, relevant, and contextually appropriate results tailored to the user's needs. --- - **Page:** Context Compaction - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context-compaction - **Description:** Context compaction is the process of reducing the number of tokens used to represent an ongoing conversation or agent run while preserving the state needed for future steps. It enables long-running interactions to continue without repeatedly sending the complete execution history. Context compaction is the process of reducing the number of [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token) used to represent an ongoing conversation or agent run while preserving the state needed for future steps. It enables long-running interactions to continue without repeatedly sending the complete execution history. As an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) calls tools and collects observations, its [context window](https://promptmetheus.com/resources/llm-knowledge-base/context-window) can fill quickly. Larger inputs increase cost and latency, and excessive context can reduce attention to relevant details. Compaction is one technique within [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering) for controlling that growth. Compaction is one form of [context reduction](https://promptmetheus.com/resources/llm-knowledge-base/context-reduction). Compaction methods include: - summarizing older messages into a shorter state description; - removing superseded plans, duplicate outputs, and low-value observations; - replacing large tool results with references to external artifacts through [context offloading](https://promptmetheus.com/resources/llm-knowledge-base/context-offloading); - extracting decisions, constraints, and unresolved tasks into structured state; and - retaining recent messages verbatim while compressing older history. The main risk is information loss. A summary may omit a constraint that appeared unimportant earlier but becomes critical later. Repeated summarization can also introduce drift, where each compacted version moves further from the original record. For this reason, compaction should not be the sole storage mechanism for authoritative information. Exact tool outputs, user approvals, financial values, source documents, and audit records should remain in external storage. [Agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory) can preserve durable facts, while the compacted context carries only the working state needed by the model. A robust compaction format typically records the current objective, completed work, active plan, important constraints, relevant facts with provenance, available artifacts, and unresolved errors. Systems should evaluate whether an agent can resume successfully from the compacted state without access to the full transcript. Compaction may be performed by application code, a separate model call, or a model-provider feature. Triggering it at a threshold before the context limit leaves room for the compaction operation itself and for subsequent work. OpenAI's [compaction guide](https://developers.openai.com/api/docs/guides/compaction) describes server-side and standalone approaches for preserving state in long-running model interactions. --- - **Page:** Context Confusion - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context-confusion - **Description:** Context confusion is a failure mode where conflicting, ambiguous, or irrelevant information in a model's context causes the model to produce an incoherent or incorrect result. It is especially common in long-running conversations, agentic workflows, and retrieval-heavy systems. Context confusion is a failure mode where conflicting, ambiguous, or irrelevant information in a model's context causes the model to produce an incoherent or incorrect result. It is especially common in long-running conversations, [agentic workflows](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow), and retrieval-heavy systems. Examples include: - two instructions that require incompatible output formats; - outdated tool results mixed with current facts; - retrieved documents from similar but different entities; - examples that imply a different task than the current request; - multiple unresolved plans in an agent trace; and - stale user preferences that conflict with a newer message. Context confusion differs from the [Lost-in-the-Middle Effect](https://promptmetheus.com/resources/llm-knowledge-base/lost-in-the-middle-effect). Lost-in-the-middle is about position and attention. Context confusion is about incompatible or misleading content being present at all. Mitigations include explicit instruction hierarchy, provenance metadata, retrieval filters, freshness checks, [context isolation](https://promptmetheus.com/resources/llm-knowledge-base/context-isolation), and [context reduction](https://promptmetheus.com/resources/llm-knowledge-base/context-reduction) before each model call. LangChain cites context confusion as one of the ways longer context can degrade agent performance in its [context engineering](https://www.langchain.com/blog/context-engineering-for-agents) article. --- - **Page:** Context Engineering - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context-engineering - **Description:** Context engineering is the practice of designing and managing all information available to a Large Language Model (LLM) when it performs a task. This includes the system message, user instructions, conversation history, retrieved documents, tool definitions, tool results, examples, memory, and intermediate state. Context engineering is the practice of designing and managing all information available to a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) when it performs a task. This includes the [system message](https://promptmetheus.com/resources/llm-knowledge-base/system-message), user instructions, conversation history, retrieved documents, tool definitions, tool results, examples, memory, and intermediate state. The term extends [prompt engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering). Prompt engineering focuses primarily on the wording and structure of instructions. Context engineering treats the complete input assembled at runtime as a system design problem. A model's [context window](https://promptmetheus.com/resources/llm-knowledge-base/context-window) is finite, but filling the available window is not necessarily beneficial. Irrelevant or duplicated information can dilute important instructions, increase latency and cost, and aggravate the [Lost-in-the-Middle Effect](https://promptmetheus.com/resources/llm-knowledge-base/lost-in-the-middle-effect). Effective context engineering therefore aims to provide the smallest set of high-signal [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token) required for the current decision. Common context engineering techniques include: - selecting only relevant documents through [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag); - placing stable instructions and examples before variable task data; - returning concise, structured results from [tool calls](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling); - using [prompt caching](https://promptmetheus.com/resources/llm-knowledge-base/prompt-caching) for long, repeated prefixes; - loading [agent skills](https://promptmetheus.com/resources/llm-knowledge-base/agent-skill) and tool definitions only when relevant; - separating durable [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory) from temporary working context; - pruning obsolete messages and duplicate observations; - applying [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction) during long-running interactions; and - measuring which context elements improve task success through [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation). For an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent), context changes after every action. Tool outputs, plans, errors, user approvals, and environmental observations must be incorporated without allowing the prompt to grow indefinitely. Context engineering is consequently an ongoing runtime process rather than a one-time writing exercise. For documentation-heavy systems, context engineering can start before retrieval or tool use. An [llms.txt](https://promptmetheus.com/resources/llm-knowledge-base/llms-txt) file gives agents a compact entry point into a site's most useful documentation, reducing the chance that a model wastes context window space on navigation, boilerplate, or low-value pages. More specialized patterns include [context reduction](https://promptmetheus.com/resources/llm-knowledge-base/context-reduction) to remove low-value tokens, [context isolation](https://promptmetheus.com/resources/llm-knowledge-base/context-isolation) to scope information by agent or task, and [context offloading](https://promptmetheus.com/resources/llm-knowledge-base/context-offloading) to keep large artifacts outside the active prompt. These patterns help avoid [context confusion](https://promptmetheus.com/resources/llm-knowledge-base/context-confusion) when long histories, retrieval results, and tool outputs compete for attention. A useful implementation principle is to treat context as an attention budget. Every inserted token should have a clear purpose, provenance, and expected effect on the next model decision. Anthropic describes this approach in its guide to [effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents). --- - **Page:** Context Isolation - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context-isolation - **Description:** Context isolation is the practice of separating information so that a model, agent, tool, or subtask only receives the context it needs. It reduces interference between tasks, limits exposure of sensitive data, and keeps each model call focused. Context isolation is the practice of separating information so that a model, agent, tool, or subtask only receives the context it needs. It reduces interference between tasks, limits exposure of sensitive data, and keeps each model call focused. In [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) systems, context isolation commonly appears as: - a [sub-agent](https://promptmetheus.com/resources/llm-knowledge-base/sub-agent) that receives only task-specific files and instructions; - a sandbox that stores large objects outside the prompt; - scoped tool access based on role or user permissions; - separate state fields for private notes, public messages, and tool results; - retrieval filters that restrict which documents can enter context; and - separate conversation histories for independent workflows. Isolation is different from simply shortening the prompt. A short prompt can still leak the wrong information, and a long prompt can be safe if each component is intentionally scoped. The goal is to enforce boundaries around relevance, authority, and access. Poor isolation can cause [context confusion](https://promptmetheus.com/resources/llm-knowledge-base/context-confusion), accidental disclosure, incorrect tool use, or cross-task contamination. Strong isolation should be paired with [agent policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy) and tracing so the system can explain which context was available for each decision. LangChain identifies isolation as one of the main context engineering strategies for agents in its [context engineering](https://www.langchain.com/blog/context-engineering-for-agents) overview. --- - **Page:** Context Offloading - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context-offloading - **Description:** Context offloading is a context engineering pattern where large or low-immediacy information is stored outside the model's active context window and replaced with a compact reference. The model can retrieve the full material later if it becomes relevant. Context offloading is a context engineering pattern where large or low-immediacy information is stored outside the model's active [context window](https://promptmetheus.com/resources/llm-knowledge-base/context-window) and replaced with a compact reference. The model can retrieve the full material later if it becomes relevant. Typical offload targets include: - long search result pages; - large code files or repository listings; - PDFs, spreadsheets, logs, and transcripts; - generated artifacts; - intermediate plans and scratch work; and - repeated tool outputs. For example, an agent might write a long API response to a file, keep only the file path and a short summary in the prompt, and read the file again only when it needs exact details. This supports [context reduction](https://promptmetheus.com/resources/llm-knowledge-base/context-reduction) without losing recoverability. Context offloading is not the same as memory. [Agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory) stores durable knowledge across tasks. Offloading usually stores working material for a specific task or run. The main risks are stale references, missing permissions, and false confidence in summaries. Systems should preserve provenance, use stable identifiers, and make it easy for the model or application code to rehydrate exact source material. LangChain discusses filesystem-based context management in [How agents can use filesystems for context engineering](https://www.langchain.com/blog/how-agents-can-use-filesystems-for-context-engineering). --- - **Page:** Context Reduction - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context-reduction - **Description:** Context reduction is the practice of decreasing the amount of information sent to a Large Language Model (LLM) while preserving the information needed for the current task. It is a practical part of context engineering because large prompts increase cost, latency, and failure risk. Context reduction is the practice of decreasing the amount of information sent to a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) while preserving the information needed for the current task. It is a practical part of [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering) because large prompts increase cost, latency, and failure risk. Common reduction techniques include: - removing duplicated or stale conversation turns; - replacing long tool outputs with summaries and artifact references; - selecting fewer retrieval chunks before generation; - compressing prior task state into structured notes; - discarding irrelevant examples or tool definitions; and - moving intermediate state to external storage through [context offloading](https://promptmetheus.com/resources/llm-knowledge-base/context-offloading). Context reduction is related to [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction), but the terms are not identical. Compaction usually refers to summarizing or restructuring an existing history. Reduction is broader: it can happen before retrieval, after tool calls, during message assembly, or as a routing decision. The main engineering tradeoff is recall versus focus. Aggressive reduction can remove facts the model later needs. Weak reduction leaves the model with bloated context, increasing the chance of [lost-in-the-middle](https://promptmetheus.com/resources/llm-knowledge-base/lost-in-the-middle-effect) failures or [context confusion](https://promptmetheus.com/resources/llm-knowledge-base/context-confusion). Reduction should be measured by task success, not by token count alone. A smaller prompt that fails is more expensive than a larger prompt that reliably completes the task. LangChain describes compression as one of the core context engineering operations in its [context engineering for agents](https://www.langchain.com/blog/context-engineering-for-agents) article. --- - **Page:** Context Window - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context-window - **Description:** A context window is the maximum amount of input and generated state that a Large Language Model (LLM) can process during one interaction. It is usually measured in tokens and may include system instructions, user messages, retrieved documents, tool definitions, tool results, prior model output, and the requested response. A context window is the maximum amount of input and generated state that a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) can process during one interaction. It is usually measured in [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token) and may include system instructions, user messages, retrieved documents, tool definitions, tool results, prior model output, and the requested response. The context window is a capacity limit, not a guarantee that every included detail will be used equally well. Very long inputs can increase cost and latency and may reduce retrieval or reasoning accuracy. The [Lost-in-the-Middle Effect](https://promptmetheus.com/resources/llm-knowledge-base/lost-in-the-middle-effect) is one example of performance degrading based on where information appears in a long prompt. Applications should therefore treat the context window as an attention and cost budget. [Context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering) selects the most relevant information for each model call, while [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction) reduces accumulated history during long-running conversations or agent tasks. [Context window expansion](https://promptmetheus.com/resources/llm-knowledge-base/context-window-expansion) increases the maximum usable input length, but it does not remove the need to manage relevance and instruction priority. The usable output length may share the same limit as the input, depending on the model API. Developers should reserve sufficient capacity for the response and for any intermediate reasoning or tool interactions required by the task. --- - **Page:** Context Window Expansion - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/context-window-expansion - **Description:** Context window expansion is the set of techniques used to increase how much input a model can process in one request. It extends the context window so the model can receive longer documents, larger codebases, longer conversations, or more tool results. Context window expansion is the set of techniques used to increase how much input a model can process in one request. It extends the [context window](https://promptmetheus.com/resources/llm-knowledge-base/context-window) so the model can receive longer documents, larger codebases, longer conversations, or more tool results. Expansion can happen through several approaches: - training or fine-tuning on longer sequences; - changing positional encodings; - modifying attention mechanisms; - using recurrent, memory, or state-space components; - compressing older context into learned or explicit summaries; and - retrieval systems that simulate larger context by selecting relevant chunks. Longer context is useful, but it is not a substitute for [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering). A larger window can still contain irrelevant, contradictory, or poorly ordered information. It may also increase cost, latency, and exposure to [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). The practical question is the effective context window: how much information the model can use reliably for a task, not merely the advertised token limit. The IJCAI survey [Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models](https://www.ijcai.org/proceedings/2024/917) reviews methods for extending LLM sequence length. --- - **Page:** Control Token - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/control-token - **Description:** In the context of Large Language Models (LLMs), a control token is a special token integrated into the input sequence to guide the model's behavior or output. These tokens can influence various aspects of text, image, video, or audio generation, such as style, tone, or content structure. By embedding control tokens within the input, users can steer the model to produce desired responses, enhancing the customization and utility of LLMs in applications like Prompt Engineering and automated content creation. In the context of [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), a control token is a special token integrated into the input sequence to guide the model's behavior or output. These tokens can influence various aspects of text, image, video, or audio generation, such as style, tone, or content structure. By embedding control tokens within the input, users can steer the model to produce desired responses, enhancing the customization and utility of LLMs in applications like [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering) and automated content creation. --- - **Page:** Data Loader - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/data-loader - **Description:** Data loaders are software components or utilities that facilitate the process of loading data into a system or database. They are designed to handle large volumes of data, ensuring efficient and accurate data transfer. Data loaders are software components or utilities that facilitate the process of loading data into a system or database. They are designed to handle large volumes of data, ensuring efficient and accurate data transfer. In the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) and machine learning models, data loaders are used to feed data into the AI models during the training process, often providing features like batching, shuffling, and parallel data loading to optimize the training procedure. In the context of [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering) tools and IDEs, data loaders are used to inject external data into a prompt, e.g. from a database, an API, or from the internet. --- - **Page:** Deep Agent - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/deep-agent - **Description:** A deep agent is an AI agent designed for complex, open-ended, long-horizon tasks. It typically combines planning, tool use, sub-agents, filesystem or artifact access, memory, and context management so it can make progress across many steps rather than answering in one turn. A deep agent is an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) designed for complex, open-ended, long-horizon tasks. It typically combines planning, tool use, sub-agents, filesystem or artifact access, memory, and context management so it can make progress across many steps rather than answering in one turn. The term is often used for agents that include: - a planning or task-tracking mechanism; - [sub-agents](https://promptmetheus.com/resources/llm-knowledge-base/sub-agent) for delegated work; - a filesystem, workspace, or artifact store for [context offloading](https://promptmetheus.com/resources/llm-knowledge-base/context-offloading); - [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory); - durable execution or resumability; - [agent skills](https://promptmetheus.com/resources/llm-knowledge-base/agent-skill); and - tracing and evaluation for long-running behavior. A deep agent is not just a larger model. It is an agent design pattern or harness for sustained work. A smaller model with strong scaffolding, file access, and reliable orchestration may behave more like a deep agent than a frontier model wrapped in a single chat prompt. The main risks are cost, runaway execution, stale state, tool misuse, and hidden failure accumulation. Deep agents need [agent policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy), sandboxing, human approval boundaries, and [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation) that checks both final outcomes and intermediate behavior. LangChain describes Deep Agents as agents that can plan, use subagents, and use file systems for complex tasks in its [Deep Agents overview](https://docs.langchain.com/oss/python/deepagents/overview). --- - **Page:** Deep Research Agent - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/deep-research-agent - **Description:** A deep research agent is an AI agent that investigates a complex question by planning searches, gathering information from multiple sources, evaluating evidence, and synthesizing a cited report. It is designed for tasks that require more than retrieving one document or producing an answer from existing model knowledge. A deep research agent is an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) that investigates a complex question by planning searches, gathering information from multiple sources, evaluating evidence, and synthesizing a cited report. It is designed for tasks that require more than retrieving one document or producing an answer from existing model knowledge. A research run commonly includes: - clarifying the question and decomposing it into subtopics; - searching the web, private files, or databases; - opening and extracting information from relevant sources; - comparing claims and resolving contradictions; - revising the search plan as new evidence appears; and - generating a structured answer with citations and uncertainty. Deep research is an [agentic workflow](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow), not simply a long prompt. The agent uses repeated [tool calls](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling) and may consume substantial [inference-time compute](https://promptmetheus.com/resources/llm-knowledge-base/inference-time-scaling-test-time-compute) before producing its final output. Source handling is the core quality problem. The agent must distinguish primary from secondary sources, account for publication and event dates, avoid circular citation, and ensure that each citation supports the specific claim attached to it. A polished report can still be wrong if retrieval or attribution is weak. Production systems should preserve a trace of search queries, visited sources, extracted evidence, and discarded claims. This supports auditability and [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation). Untrusted webpages also create indirect [prompt injection](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack) risk. Deep research agents are useful for literature reviews, market analysis, technical comparisons, due diligence, and policy research. They should not be treated as authoritative without checking high-impact conclusions against original sources. The [OpenAI deep research guide](https://developers.openai.com/api/docs/guides/deep-research) describes agentic search and synthesis with web search, file search, and remote MCP sources. --- - **Page:** Diffusion Language Model - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/diffusion-language-model - **Description:** A diffusion language model generates text by repeatedly refining a partially specified sequence rather than committing to every next token strictly from left to right. In a masked-token design, generation begins with placeholder positions and iteratively predicts which tokens belong there. Some systems refine an entire response; others generate and refine blocks in sequence. A diffusion language model generates text by repeatedly refining a partially specified sequence rather than committing to every next [token](https://promptmetheus.com/resources/llm-knowledge-base/token) strictly from left to right. In a masked-token design, generation begins with placeholder positions and iteratively predicts which tokens belong there. Some systems refine an entire response; others generate and refine blocks in sequence. That differs from the usual autoregressive [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), which conditions each new token on the preceding tokens and cannot revise an earlier token during the same decode pass. A diffusion model can use information from both sides of a position within the region being refined. The architecture and training objective determine how that refinement works; text diffusion is a family of approaches, not one fixed decoding algorithm. Parallel updates may improve latency when a model runs at low concurrency on capable hardware. The trade-off is that several refinement passes can still be needed, and a faster single request does not necessarily imply better throughput or lower cost in a heavily batched service. Quality, controllability, streaming behavior, and the number of refinement steps should be evaluated on the intended workload rather than inferred from tokens-per-second alone. This term is useful when comparing generation architectures and [inference](https://promptmetheus.com/resources/llm-knowledge-base/inference) strategies. It is separate from [model distillation](https://promptmetheus.com/resources/llm-knowledge-base/model-distillation), which transfers behavior between models, and from [Mixture of Experts (MoE)](https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-experts-moe), which controls which parameters are active. A model can combine diffusion-style generation with an MoE backbone. Google's [DiffusionGemma technical introduction](https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/) describes one blockwise implementation and the workload limits of its reported speed gains. --- - **Page:** Direct Preference Optimization (DPO) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/direct-preference-optimization-dpo - **Description:** Direct Preference Optimization (DPO) is a technique used in the field of Machine Learning, particularly within the context of training Large Language Model (LLM) and Generative AI systems. It involves directly optimizing a model's parameters based on user preferences or feedback, rather than relying solely on traditional loss functions. This approach aims to align the model's outputs more closely with human expectations and desired outcomes by incorporating explicit preference data during the training process. DPO can enhance the model's ability to generate content that is more relevant, accurate, and aligned with user needs, thereby improving the overall user experience and effectiveness of AI-driven applications. Direct Preference Optimization (DPO) is a technique used in the field of [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml), particularly within the context of training [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) and [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) systems. It involves directly optimizing a model's parameters based on user preferences or feedback, rather than relying solely on traditional loss functions. This approach aims to align the model's outputs more closely with human expectations and desired outcomes by incorporating explicit preference data during the training process. DPO can enhance the model's ability to generate content that is more relevant, accurate, and aligned with user needs, thereby improving the overall user experience and effectiveness of AI-driven applications. --- - **Page:** Domain-Specific LLM - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/domain-specific-llm - **Description:** A domain-specific LLM is a Large Language Model (LLM) adapted for a specialized field, workflow, or body of knowledge. Examples include legal, medical, financial, scientific, security, customer-support, and code-focused models. A domain-specific LLM is a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) adapted for a specialized field, workflow, or body of knowledge. Examples include legal, medical, financial, scientific, security, customer-support, and code-focused models. Domain specialization can be implemented through: - continued pretraining on domain corpora; - [fine-tuning](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) on task-specific examples; - [LoRA (Low-Rank Adaptation)](https://promptmetheus.com/resources/llm-knowledge-base/lora-low-rank-adaptation) or other parameter-efficient methods; - retrieval over curated domain sources; - domain-specific evaluation suites; and - application-level [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering). A domain-specific LLM is not automatically safer or more accurate than a general model. It may fit too narrowly to limited data, lag behind current domain changes, or fail outside its intended scope. Professional deployments should define the domain boundary, evaluate against domain experts, and measure hallucination, citation, and refusal behavior. Domain-specific models are valuable when specialized terminology, regulatory constraints, proprietary knowledge, or workflow precision matter more than broad generality. The ACM Computing Surveys article [Domain Specialization as the Key to Make Large Language Models Disruptive](https://dl.acm.org/doi/10.1145/3764579) surveys approaches to domain specialization for LLMs. --- - **Page:** Dynamic Tool Discovery (Tool Search) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/dynamic-tool-discovery-tool-search - **Description:** Dynamic tool discovery, or tool search, is a mechanism that lets an AI agent find relevant tools at runtime instead of receiving every available tool definition in its initial context. Dynamic tool discovery, or tool search, is a mechanism that lets an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) find relevant tools at runtime instead of receiving every available tool definition in its initial [context](https://promptmetheus.com/resources/llm-knowledge-base/context). As an agent platform grows, it may expose hundreds or thousands of APIs, database operations, and [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) tools. Including all schemas on every request consumes tokens, increases latency, and makes tool selection less reliable. Tool search keeps a compact catalog or search interface in context and loads full definitions only for likely matches. A typical process is: 1. The agent receives lightweight tool names, descriptions, or access to a search tool. 2. It searches using the current task and intent. 3. The system returns a small set of matching tool definitions. 4. The agent invokes one through ordinary [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling). Discovery may use keyword search, [vector search](https://promptmetheus.com/resources/llm-knowledge-base/vector-search), [hybrid search](https://promptmetheus.com/resources/llm-knowledge-base/hybrid-search), metadata filters, or learned routing. Access control must be applied before results are shown; search should never reveal tools the user or agent is not authorized to use. Tool descriptions become retrieval documents and should contain distinctive capabilities, constraints, and selection criteria. Ambiguous descriptions can retrieve the wrong operation even when the search algorithm works correctly. Dynamic discovery complements [agent skills](https://promptmetheus.com/resources/llm-knowledge-base/agent-skill). A skill provides procedural knowledge, while tool search locates executable capabilities. Both reduce context consumption through progressive disclosure. The [Claude tool search documentation](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool) describes server-side discovery that defers loading tool definitions until they are needed. --- - **Page:** Embedding Distance Metrics - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/embedding-distance-metrics - **Description:** Embedding distance metrics are mathematical functions used to compare vector embeddings. They determine which items are considered similar in vector search, recommendation systems, clustering, and retrieval pipelines. Embedding distance metrics are mathematical functions used to compare [vector embeddings](https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding). They determine which items are considered similar in [vector search](https://promptmetheus.com/resources/llm-knowledge-base/vector-search), recommendation systems, clustering, and retrieval pipelines. Common metrics include: - **Cosine distance:** compares the angle between vectors, often used for text embeddings. - **Euclidean distance:** measures straight-line distance in vector space. - **Dot product or inner product:** rewards vectors that point in similar directions and have large magnitude. - **Manhattan distance:** sums absolute coordinate differences, used less often in modern text retrieval. The metric must match the embedding model and index configuration. If embeddings were trained or normalized for cosine similarity, ranking them with raw Euclidean distance may produce weaker results. Some vector databases also convert similarity into a distance where lower scores are better, which can confuse evaluation and threshold selection. Distance metrics are first-stage retrieval signals. For user-facing quality, they are often combined with metadata filters, [hybrid search](https://promptmetheus.com/resources/llm-knowledge-base/hybrid-search), and [reranking](https://promptmetheus.com/resources/llm-knowledge-base/reranking). Redis documents cosine, L2, and inner-product metrics in its [vector search concepts](https://redis.io/docs/latest/develop/ai/search-and-query/vectors/) guide. --- - **Page:** Enterprise-Managed Authorization - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/enterprise-managed-authorization - **Description:** Enterprise-Managed Authorization (EMA) is an MCP Authorization Extension that lets an organization govern access to Model Context Protocol (MCP) servers through its identity provider. The extension is useful when employees use many connected tools and access decisions need to follow centrally managed organizational policy. Enterprise-Managed Authorization (EMA) is an [MCP Authorization Extension](https://promptmetheus.com/resources/llm-knowledge-base/mcp-authorization-extension) that lets an organization govern access to [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) servers through its identity provider. The extension is useful when employees use many connected tools and access decisions need to follow centrally managed organizational policy. In the EMA flow, the user authenticates to the MCP client through the enterprise identity provider (IdP). When the client needs access to an approved MCP server, it obtains an Identity Assertion JWT Authorization Grant (ID-JAG) from that IdP. It then exchanges the grant with the MCP server's authorization server for an access token. The IdP can apply organizational policy before issuing the grant, while the server's authorization system still issues a token for its own resource. This differs from asking each user to authorize each MCP server separately. Central policy can simplify onboarding and revocation, but it does not make every tool call acceptable. The MCP server still needs to validate tokens, enforce scopes and resource permissions, and apply an [agent policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy) to consequential operations. Clients and servers also need explicit support for the extension; ordinary MCP or OAuth support does not imply EMA support. For operators, the important boundary is between authentication to the client, the IdP's decision to issue a grant, and authorization at the resource server. Auditing those steps separately helps diagnose a denied call without treating a successful enterprise login as blanket access to every connected system. The MCP project's [EMA specification](https://github.com/modelcontextprotocol/ext-auth/blob/main/specification/stable/enterprise-managed-authorization.mdx) defines the grant flow and responsibilities of clients, identity providers, and authorization servers. --- - **Page:** Evaluation - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/evaluation - **Description:** An Evaluation (or "eval") is a crucial step in the development of Generative AI models. It involves assessing the performance, quality, and effectiveness of the AI-generated outputs, such as text, images, or audio. The evaluation process helps identify areas for improvement, ensures the model meets the desired standards, and validates its readiness for real-world applications. Common evaluation methods include human ratings, automated metrics, and comparative analyses against ground truth data or benchmarks. Regularly conducting evals is essential for maintaining the output quality of LLM-based services for end users. An Evaluation (or "eval") is a crucial step in the development of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) models. It involves assessing the performance, quality, and effectiveness of the AI-generated outputs, such as text, images, or audio. The evaluation process helps identify areas for improvement, ensures the model meets the desired standards, and validates its readiness for real-world applications. Common evaluation methods include human ratings, automated metrics, and comparative analyses against ground truth data or benchmarks. Regularly conducting evals is essential for maintaining the output quality of [LLM](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm)-based services for end users. An evaluation should represent the actual task distribution and define measurable success criteria before prompts or models are optimized. Deterministic checks are preferable when an answer can be verified directly; human review or [LLM-as-a-Judge](https://promptmetheus.com/resources/llm-knowledge-base/llm-as-a-judge) can help assess qualities such as relevance, tone, and completeness. Multi-step systems require [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation), which also measures tool use, execution trajectories, cost, latency, safety, and changes to external state. --- - **Page:** Evaluation-driven Development - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/evaluation-driven-development - **Description:** Evaluation-driven development in Prompt Engineering is akin to test-driven development in software engineering, with the key distinction that evaluations (or evals) replace unit tests. In this approach, prompt engineers iteratively refine prompts by running evaluations against completions to measure the model’s performance with respect to predefined criteria. Unlike unit tests, which have fixed, deterministic outputs, evaluations in prompt engineering often focus on qualitative metrics—such as accuracy, relevance, and coherence—guiding the continuous refinement of prompts. Evaluation-driven development in [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering) is akin to test-driven development in software engineering, with the key distinction that [evaluations](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) (or evals) replace unit tests. In this approach, prompt engineers iteratively refine [prompts](https://promptmetheus.com/resources/llm-knowledge-base/prompt) by running evaluations against completions to measure the model’s performance with respect to predefined criteria. Unlike unit tests, which have fixed, deterministic outputs, evaluations in prompt engineering often focus on qualitative metrics—such as accuracy, relevance, and coherence—guiding the continuous refinement of prompts. A major challenge in evaluation-driven development, compared to test-driven development, is the inherent unpredictability and variability in [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) outputs. While unit tests in software have clear, deterministic outcomes, evaluations for LLMs often deal with subjective or probabilistic results, making it difficult to establish a definitive “pass” or “fail” criterion. Additionally, LLM behavior can change with new model versions or even slight prompt modifications, complicating the maintenance of consistent evaluation standards. Evaluations may also require human judgment, making them more time-consuming and susceptible to bias. --- - **Page:** Event-driven AI - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/event-driven-ai - **Description:** Event-driven AI is an architecture where AI components react to events from systems, users, sensors, queues, or business processes. Instead of being invoked only by a direct chat prompt, a model or AI agent can consume event streams and decide whether to analyze, alert, route, or act. Event-driven AI is an architecture where AI components react to events from systems, users, sensors, queues, or business processes. Instead of being invoked only by a direct chat prompt, a model or [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) can consume event streams and decide whether to analyze, alert, route, or act. Events may include: - a new support ticket; - a failed build; - a calendar change; - a payment anomaly; - a security alert; - a file upload; - a product metric crossing a threshold; or - a completed tool call in an agent workflow. Event-driven AI is useful for [proactive AI](https://promptmetheus.com/resources/llm-knowledge-base/proactive-ai) because it supplies the triggers that start work. It also supports loose coupling between data-producing systems and AI decision layers. The main risks are over-triggering, duplicated actions, stale state, and unclear ownership. Production systems need idempotency, rate limits, replay controls, event schemas, authorization checks, and observability across both event infrastructure and model calls. Confluent discusses event streams, stateful processing, and agent decision layers in [Autonomous Agentic Event-Driven Systems Architecture](https://www.confluent.io/blog/autonomous-agentic-event-driven-systems-architecture/). --- - **Page:** Few-shot Prompt - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/few-shot-prompt - **Description:** Few-shot prompt refers to a method where the AI model is given a small number of examples (usually less than 10) during the inference stage to understand and generate similar content. This approach helps the model to quickly adapt to new tasks, reducing the need for extensive training data. It's a technique commonly used in Natural Language Processing and Machine Learning models to improve their performance and versatility. Few-shot prompt refers to a method where the AI model is given a small number of examples (usually less than 10) during the inference stage to understand and generate similar content. This approach helps the model to quickly adapt to new tasks, reducing the need for extensive training data. It's a technique commonly used in [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) and [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) models to improve their performance and versatility. See also ["Zero-shot Prompt"](https://promptmetheus.com/resources/llm-knowledge-base/zero-shot-prompt) and ["One-shot Prompt"](https://promptmetheus.com/resources/llm-knowledge-base/one-shot-prompt). --- - **Page:** Fine-tuning - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning - **Description:** Depending on the context, fine-tuning can have two distinct meanings. Depending on the context, fine-tuning can have two distinct meanings. a) In the context of [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), fine-tuning refers to the process of taking a pre-trained model and further training it on a specific dataset to enhance its performance. This technique is used to adapt a general-purpose model to a specific task or to improve its ability to understand specific nuances, [contexts](https://promptmetheus.com/resources/llm-knowledge-base/context), or languages. Fine-tuning is a crucial step in [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) and AI development, as it allows for more accurate and efficient models by leveraging existing neural networks and reducing the need for extensive training from scratch. Training examples may be written by humans or generated as [synthetic data](https://promptmetheus.com/resources/llm-knowledge-base/synthetic-data). Parameter-efficient methods such as [LoRA (Low-Rank Adaptation)](https://promptmetheus.com/resources/llm-knowledge-base/lora-low-rank-adaptation) can adapt a model while updating far fewer parameters than full fine-tuning. In [model distillation](https://promptmetheus.com/resources/llm-knowledge-base/model-distillation), a student model is fine-tuned to reproduce selected behavior from a more capable teacher. b) In informal discussions of [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering), "fine-tuning a prompt" sometimes means iteratively optimizing its wording. This should not be confused with model fine-tuning, which updates model parameters through training. --- - **Page:** Foundation Model - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/foundation-model - **Description:** A Foundation Model, often also referred to as a Base Model, is a large-scale Artificial Intelligence model trained on extensive datasets using self-supervised or semi-supervised learning techniques. These models are designed to be adaptable across a wide array of downstream tasks, serving as a foundational platform upon which more specialized models can be built. A Foundation Model, often also referred to as a Base Model, is a large-scale [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai) model trained on extensive datasets using self-supervised or semi-supervised learning techniques. These models are designed to be adaptable across a wide array of downstream tasks, serving as a foundational platform upon which more specialized models can be built. Foundation Models are very expensive to train due to their size and complexity, requiring significant computational resources and data. It can cost upwards of a hundred million dollars to train a state-of-the-art Base Model and take 6 months or more. ## Key Characteristics - **Generalization:** Base models are trained on diverse data sources, enabling them to generalize across various domains and tasks without task-specific training. - **Scalability:** They possess a vast number of parameters, allowing for nuanced understanding and generation of complex data patterns. - **Adaptability:** Through techniques like [Supervised Fine-Tuning (SFT)](https://promptmetheus.com/resources/llm-knowledge-base/supervised-fine-tuning-sft) and [Reinforcement Learning from Human Feedback (RLHF)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning), base models can be tailored to specific applications, enhancing their performance on particular tasks. ## Notable Examples - **GPT Series:** Developed by [OpenAI](https://openai.com), the [Generative Pre-trained Transformer (GPT)](https://promptmetheus.com/resources/llm-knowledge-base/generative-pre-trained-transformer-gpt) models, such as GPT-3 and GPT-4, are prominent base models known for their language understanding and generation capabilities. - **BERT:** Google's [Bidirectional Encoder Representations from Transformers (BERT)](https://en.wikipedia.org/wiki/BERT_(language_model)) is another significant base model that has advanced natural language processing tasks. ## Applications Base models serve as the backbone for various AI applications, including: - **Natural Language Processing (NLP):** Tasks like translation, summarization, and sentiment analysis. - **Computer Vision:** Image recognition, object detection, and segmentation. - **Multimodal Tasks:** Integrating and processing multiple data types, such as text and images, simultaneously. By leveraging base models, developers can expedite the development process, reduce the need for large labeled datasets, and achieve state-of-the-art performance across multiple AI domains. For a more in-depth look at foundation models, check out Andrej Karpathy's excellent video [Deep Dive into LLMs like ChatGPT](https://www.youtube.com/watch?v=7xTGNNLPyMI). --- - **Page:** Frequency Penalty - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/frequency-penalty - **Description:** Frequency Penalty is a parameter used in Generative AI models, particularly in language models, to control the repetition of generated content. It penalizes the model for repeatedly generating the same words or phrases, thereby encouraging diversity and novelty in the output. This parameter is adjustable, allowing users to balance between repetitiveness and creativity according to their specific needs. Frequency Penalty is a parameter used in [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) models, particularly in language models, to control the repetition of generated content. It penalizes the model for repeatedly generating the same words or phrases, thereby encouraging diversity and novelty in the output. This parameter is adjustable, allowing users to balance between repetitiveness and creativity according to their specific needs. For some models the frequency penalty can also be negative, which encourages repetition. --- - **Page:** GPT Wrapper - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/gpt-wrapper - **Description:** A "GPT wrapper" is a product or service that primarily relies on the capabilities of a Large Language Model like GPT to provide its core functionality. The term "wrapper" implies that most of the value of the product is derived from the underlying LLM and the product itself merely augments its functionality with additional data or a better user interface to make it more accessible and user-friendly. A "GPT wrapper" is a product or service that primarily relies on the capabilities of a [Large Language Model](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) like [GPT](https://promptmetheus.com/resources/llm-knowledge-base/generative-pre-trained-transformer-gpt) to provide its core functionality. The term "wrapper" implies that most of the value of the product is derived from the underlying LLM and the product itself merely augments its functionality with additional data or a better user interface to make it more accessible and user-friendly. GPT wrapper applications typically leverage the latest [Proprietary Models](https://promptmetheus.com/resources/llm-knowledge-base/proprietary-model) and rely on [LLM APIs](https://promptmetheus.com/resources/llm-knowledge-base/llm-api) to provide [inference](https://promptmetheus.com/resources/llm-knowledge-base/inference) for their prompts. Examples of augmentation that typical GPT wrapper apps might provide include: - **Context**: Providing additional context to the LLM to improve the quality of its responses. - **Prompt engineering**: Developing tailored prompts to improve the quality of the LLM's output. - **Post-processing**: Filtering and refining the model's responses to meet specific requirements. - **User interface**: Creating a user-friendly interface to interact with the LLM. To compensate for weak IP, developing a robust brand, an effective go-to-market strategy, and strong community engagement are essential for product differentiation of GPT wrappers. --- - **Page:** Generative AI - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/generative-ai - **Description:** Generative AI is a subset of artificial intelligence that leverages Machine Learning techniques to enable a system to generate new, previously unseen content, data, or information that is similar to or a variation of its training data. This can include a wide range of outputs, such as text, images, music, or even voice. Generative AI is commonly used in a variety of applications, such as content creation, data augmentation, and predictive modeling. Generative AI is a subset of artificial intelligence that leverages [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) techniques to enable a system to generate new, previously unseen content, data, or information that is similar to or a variation of its training data. This can include a wide range of outputs, such as text, images, music, or even voice. Generative AI is commonly used in a variety of applications, such as content creation, data augmentation, and predictive modeling. --- - **Page:** Generative UI - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/generative-ui - **Description:** Generative UI is an interaction pattern where an AI system returns structured, interactive interface elements instead of only text. An agent might generate a form, table, chart, card, review panel, configuration wizard, or approval component that the host application renders for the user. Generative UI is an interaction pattern where an AI system returns structured, interactive interface elements instead of only text. An agent might generate a form, table, chart, card, review panel, configuration wizard, or approval component that the host application renders for the user. In agent systems, generative UI is often used when plain chat is too weak for the task. A user may need to inspect data, edit fields, compare options, approve a step, or manipulate a visual object. The UI component becomes part of the workflow, not just decoration around a text answer. Generative UI can be delivered through several mechanisms: - [MCP Apps](https://promptmetheus.com/resources/llm-knowledge-base/mcp-app), where an MCP tool returns an interactive UI resource; - [AG-UI](https://promptmetheus.com/resources/llm-knowledge-base/agent-user-interaction-protocol-ag-ui) events that connect agent state to frontend components; - [A2UI](https://promptmetheus.com/resources/llm-knowledge-base/a2ui) streams that describe UI surfaces and component updates; or - application-specific JSON schemas rendered by a trusted client. The key technical requirement is validation. A host should treat model-generated UI descriptions as untrusted data. It should validate component types, properties, actions, data bindings, and permissions before rendering or executing anything. Generative UI is related to [structured output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output), but it has a user-interface goal. Structured output may feed an API or database. Generative UI feeds an interactive surface. OpenUI describes generative UI as rendering interactive components such as tables, forms, charts, and cards inside an agent interface in its [Generative UI documentation](https://www.openui.com/docs/agent/core-concepts/generative-ui). --- - **Page:** Graph RAG - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/graph-rag - **Description:** Graph RAG is an advanced technique in the field of Generative AI that combines Retrieval-Augmented Generation (RAG) with Graph data structures for better retrieval performance. In this approach, a graph structure is utilized to organize and retrieve relevant information from a large corpus of data, which is then used to inform and guide the generation process of a Large Language Model (LLM). The method leverages the strengths of both retrieval and generation, allowing for more contextually accurate and coherent outputs. By integrating graph-based retrieval mechanisms, Graph RAG can efficiently handle complex queries and provide more precise responses, making it particularly useful in applications requiring detailed and context-rich information synthesis. Graph RAG is an advanced technique in the field of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) that combines [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag) with [Graph](https://en.wikipedia.org/wiki/Graph_%28abstract_data_type%29) data structures for better retrieval performance. In this approach, a graph structure is utilized to organize and retrieve relevant information from a large corpus of data, which is then used to inform and guide the generation process of a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). The method leverages the strengths of both retrieval and generation, allowing for more contextually accurate and coherent outputs. By integrating graph-based retrieval mechanisms, Graph RAG can efficiently handle complex queries and provide more precise responses, making it particularly useful in applications requiring detailed and [context](https://promptmetheus.com/resources/llm-knowledge-base/context)-rich information synthesis. --- - **Page:** Group Relative Policy Optimization (GRPO) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/group-relative-policy-optimization-grpo - **Description:** Group Relative Policy Optimization (GRPO) is a reinforcement-learning algorithm for training language models from rewards. For each prompt, it samples a group of candidate responses and computes the relative advantage of each response from rewards within that group. Group Relative Policy Optimization (GRPO) is a reinforcement-learning algorithm for training language models from rewards. For each prompt, it samples a group of candidate responses and computes the relative advantage of each response from rewards within that group. In simplified form, GRPO: 1. generates several outputs from the current policy for the same prompt; 2. scores each output with one or more reward functions; 3. normalizes rewards relative to the group's mean and variation; and 4. updates the policy to increase the probability of above-average outputs while limiting divergence from a reference policy. GRPO is related to [Proximal Policy Optimization (PPO)](https://promptmetheus.com/resources/llm-knowledge-base/proximal-policy-optimization-ppo) but removes the need to train a separate critic or value model. This can reduce memory and computational requirements during large-model post-training. The quality and diversity of each response group matter. If all samples are nearly identical, relative rewards provide little learning signal. If rewards are sparse, many groups may contain no successful response. Sampling temperature, group size, task difficulty, and curriculum design therefore influence training efficiency. GRPO is frequently used with [Reinforcement Learning with Verifiable Rewards (RLVR)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning-with-verifiable-rewards-rlvr). Mathematical answers, code tests, formatting checks, and other automatic verifiers provide rewards, while GRPO converts differences within a sampled group into policy updates. As with any reward optimization method, GRPO can amplify flaws in the reward function. Multiple reward components may also conflict, and group normalization can make training behavior dependent on the sampled alternatives rather than an absolute quality scale. GRPO was introduced in the [DeepSeekMath paper](https://arxiv.org/abs/2402.03300) and later used at larger scale for reasoning-model training. --- - **Page:** Hallucination - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/hallucination - **Description:** In Artificial Intelligence, particularly within Large Language Models (LLMs), a "hallucination" refers to the generation of content that appears coherent and plausible but is factually incorrect or not substantiated by the model's training data. This phenomenon arises because LLMs, while adept at producing human-like text, lack true understanding and may produce inaccurate information when uncertain. In [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai), particularly within [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), a "hallucination" refers to the generation of content that appears coherent and plausible but is factually incorrect or not substantiated by the model's training data. This phenomenon arises because LLMs, while adept at producing human-like text, lack true understanding and may produce inaccurate information when uncertain. For example, when asked to draft a financial report for a specific company, an LLM might fabricate financial figures, presenting them as factual data. Similarly, when queried about astrophysical concepts, it might incorrectly assert that black hole magnetic fields are generated by gravitational forces, contradicting established scientific understanding. Mitigating hallucinations is an active area of research. Approaches include integrating external knowledge sources to ground responses in factual data, refining model architectures, and employing [Reinforcement Learning from Human Feedback (RLHF)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning) to align outputs with human expectations. Despite these efforts, completely eliminating hallucinations remains a challenge, necessitating ongoing advancements to enhance the reliability of AI-generated content. --- - **Page:** Hybrid Search - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/hybrid-search - **Description:** Hybrid search combines lexical retrieval with semantic vector search to produce one ranked set of results. It is commonly used in Retrieval-Augmented Generation (RAG) because the two retrieval methods have complementary strengths. Hybrid search combines lexical retrieval with semantic [vector search](https://promptmetheus.com/resources/llm-knowledge-base/vector-search) to produce one ranked set of results. It is commonly used in [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag) because the two retrieval methods have complementary strengths. Lexical search, often based on BM25 or a related term-frequency method, matches exact words and phrases. It performs well for identifiers, product codes, error messages, names, and rare terminology. Vector search compares [embeddings](https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding) and can retrieve conceptually related documents even when they use different vocabulary. A hybrid system runs both searches and combines their result lists. Fusion methods include: - weighted normalization of lexical and vector scores; - Reciprocal Rank Fusion, which combines result positions rather than raw scores; - learned ranking models; and - retrieving a broad union before applying a separate [reranker](https://promptmetheus.com/resources/llm-knowledge-base/reranking). Raw lexical and vector scores usually have different distributions and should not be added without calibration. The optimal weighting also depends on the corpus and query type. A query containing an exact serial number may need stronger lexical weighting, while a conceptual question may benefit from semantic retrieval. Metadata filtering, access control, chunking, and index freshness remain important. Hybrid retrieval cannot recover a document that was never indexed or is excluded by an incorrect filter. For [agentic RAG](https://promptmetheus.com/resources/llm-knowledge-base/agentic-rag), the agent may select hybrid search for broad discovery and then refine results with additional queries or tools. Google Cloud's [hybrid search documentation](https://docs.cloud.google.com/gemini-enterprise-agent-platform/build/vector-search/about-hybrid-search) describes combining dense vector and sparse token embeddings for retrieval. --- - **Page:** In-context Learning - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/in-context-learning-icl - **Description:** In-Context Learning (ICL) is a capability of Large Language Models (LLMs) to learn and perform tasks based solely on the context provided within a Prompt, without updating the model's parameters. By presenting examples or instructions within the input, LLMs can adapt to new tasks temporarily, enabling few-shot learning where the model generalizes from limited data. This emergent property of LLMs contrasts with traditional training methods that require parameter adjustments and extensive datasets. In-Context Learning (ICL) is a capability of [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) to learn and perform tasks based solely on the [context](https://promptmetheus.com/resources/llm-knowledge-base/context) provided within a [Prompt](https://promptmetheus.com/resources/llm-knowledge-base/prompt), without updating the model's parameters. By presenting examples or instructions within the input, LLMs can adapt to new tasks temporarily, enabling few-shot learning where the model generalizes from limited data. This emergent property of LLMs contrasts with traditional training methods that require parameter adjustments and extensive datasets. --- - **Page:** Inference - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/inference - **Description:** Inference refers to the process of using a trained model to make predictions or decisions. It involves inputting new, unseen data into a Language Model and receiving an output that represents the model's best guess or prediction. This process is crucial in many AI applications, such as image recognition, natural language processing, and recommendation systems, where the model's ability to infer or predict outcomes based on learned patterns is utilized. Inference refers to the process of using a trained model to make predictions or decisions. It involves inputting new, unseen data into a [Language Model](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) and receiving an output that represents the model's best guess or prediction. This process is crucial in many AI applications, such as image recognition, natural language processing, and recommendation systems, where the model's ability to infer or predict outcomes based on learned patterns is utilized. For text generation, inference can proceed token by token or use another decoding strategy. A [diffusion language model](https://promptmetheus.com/resources/llm-knowledge-base/diffusion-language-model) iteratively refines a partly specified text sequence instead of committing to every next token strictly from left to right. --- - **Page:** Inference-Time Scaling (Test-Time Compute) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/inference-time-scaling-test-time-compute - **Description:** Inference-time scaling, also called test-time compute scaling, is the practice of allocating additional computation to a model after training in order to improve the quality of its answer. Instead of relying on a single forward generation, the system may let a reasoning model reason for longer, generate multiple candidates, search over possible solutions, or verify intermediate results. Inference-time scaling, also called test-time compute scaling, is the practice of allocating additional computation to a model after training in order to improve the quality of its answer. Instead of relying on a single forward generation, the system may let a [reasoning model](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model) reason for longer, generate multiple candidates, search over possible solutions, or verify intermediate results. This differs from training-time scaling, where capability is increased by using more parameters, data, or training compute. Inference-time scaling changes the amount of work performed for each request and can therefore be adjusted dynamically according to task difficulty, latency requirements, and cost. Common approaches include: - producing several candidate answers and selecting one with a verifier; - iterative self-correction or critique; - tree or graph search over reasoning steps; - executing code or tests to validate candidates; - increasing a model's reasoning-effort budget; and - using [tool calls](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling) to gather evidence before answering. More compute does not automatically produce a better result. Repeated samples can share the same misconception, weak verifiers can select confidently incorrect answers, and long reasoning traces increase latency and [token](https://promptmetheus.com/resources/llm-knowledge-base/token) cost. Effective systems allocate compute selectively rather than applying the same budget to every request. Inference-time scaling is especially relevant to [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent), which can spend additional steps planning, testing actions, and recovering from errors. It should be evaluated against end-to-end accuracy and resource usage through [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation). The paper [Scaling LLM Test-Time Compute Optimally](https://arxiv.org/abs/2408.03314) shows that the most effective strategy depends on problem difficulty and that adaptively allocated test-time compute can outperform substantially larger models on suitable tasks. --- - **Page:** Input Token - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/input-token - **Description:** Input tokens refer to the individual units of information that are fed into an AI model as part of the prompt for processing. These tokens can be words, characters, or subwords, depending on the granularity of the model. They serve as the basis for the model to understand, analyze, and generate output. The number of input tokens often determines the computational requirements of the AI model, as more tokens typically require more processing power. Input tokens refer to the individual units of information that are fed into an AI model as part of the prompt for processing. These [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token) can be words, characters, or subwords, depending on the granularity of the model. They serve as the basis for the model to understand, analyze, and generate output. The number of input tokens often determines the computational requirements of the AI model, as more tokens typically require more processing power. Input tokens are subject to the total (input + output) or specific (input only) [token limit](https://promptmetheus.com/resources/llm-knowledge-base/token-limit). --- - **Page:** Integrated Prompting Environment (IPE) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/integrated-prompting-environment-ipe - **Description:** An Integrated Prompting Environment (IPE) is the analog to a code IDE in the world of Prompt Engineering. It's a software application that helps users develop, test, and optimize reliable prompts for Large Language Models (LLMs). An Integrated Prompting Environment (IPE) is the analog to a code IDE in the world of [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering). It's a software application that helps users develop, test, and optimize reliable prompts for [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). is an example of an IPE. Also take a look at the ["Building a Prompt Engineering IDE (VS Code for AI)"](https://promptmetheus.com/blog/prompt-engineering-ide) post for a more in-depth introduction. --- - **Page:** Jailbreak - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/jailbreak - **Description:** In the context of Generative AI, "jailbreak" refers to the process of removing or circumventing restrictions imposed by the LLMs original developers. This allows users to gain access to additional functionalities, customization options, or the ability to run unauthorized or third-party software that is not typically permitted within the standard operating parameters. Jailbreaking in the traditional sense is associated with smartphones and other devices, but in the realm of Generative AI, it involves modifying the behavior of AI models to perform tasks or operate in ways not originally intended by their creators. In the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), "jailbreak" refers to the process of removing or circumventing restrictions imposed by the LLMs original developers. This allows users to gain access to additional functionalities, customization options, or the ability to run unauthorized or third-party software that is not typically permitted within the standard operating parameters. Jailbreaking in the traditional sense is associated with smartphones and other devices, but in the realm of Generative AI, it involves modifying the behavior of AI models to perform tasks or operate in ways not originally intended by their creators. See also ["Prompt Injection Attack"](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). --- - **Page:** LLM API - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/llm-api - **Description:** LLM API (Large Language Model Application Programming Interface) is a set of protocols and tools that allows developers to integrate the capabilities of Large Language Models, such as Claude 3, GPT-4, or Gemini 1.5, into their own applications. By leveraging an LLM API, developers can access the power of these advanced AI models to generate human-like text, perform natural language understanding tasks, and create intelligent conversational interfaces in their applications without the need to train or host the models themselves. LLM API (Large Language Model Application Programming Interface) is a set of protocols and tools that allows developers to integrate the capabilities of [Large Language Models](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), such as Claude 3, GPT-4, or Gemini 1.5, into their own applications. By leveraging an LLM API, developers can access the power of these advanced AI models to generate human-like text, perform natural language understanding tasks, and create intelligent conversational interfaces in their applications without the need to train or host the models themselves. LLM APIs enable businesses to quickly and easily incorporate cutting-edge language AI capabilities into their products and services. Also see [LLM API Provider](https://promptmetheus.com/resources/llm-knowledge-base/llm-api-provider) for more information on companies that offer LLM APIs. --- - **Page:** LLM API Provider - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/llm-api-provider - **Description:** A service that offers access to Large Language Models (LLMs) through APIs (see LLM API), enabling developers to integrate advanced natural language processing capabilities into their applications without the need to train or host the models themselves. LLM API providers handle the infrastructure, maintenance, and scaling of these powerful AI models, allowing businesses to leverage state-of-the-art language generation, understanding, and analysis features in a cost-effective and efficient manner. A service that offers access to [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) through APIs (see [LLM API](https://promptmetheus.com/resources/llm-knowledge-base/llm-api)), enabling developers to integrate advanced natural language processing capabilities into their applications without the need to train or host the models themselves. LLM API providers handle the infrastructure, maintenance, and scaling of these powerful AI models, allowing businesses to leverage state-of-the-art language generation, understanding, and analysis features in a cost-effective and efficient manner. Examples of LLM API providers include [Anthropic](https://www.anthropic.com/), [Cohere](https://cohere.com/), [OpenAI](https://openai.com), [Gemini](https://gemini.google.com), [Perplexity](https://www.perplexity.ai/), [Mistral](https://mistral.ai), [Groq](https://groq.com), [NLP Cloud](https://nlpcloud.com), [AI21 Labs](https://www.ai21.com/), [Deep Infra](https://deepinfra.com), and [Aleph Alpha](https://aleph-alpha.com). Production applications increasingly avoid coupling directly to one provider. An [LLM gateway](https://promptmetheus.com/resources/llm-knowledge-base/llm-gateway) can centralize authentication, observability, rate limits, and failover across providers, while [model routing](https://promptmetheus.com/resources/llm-knowledge-base/model-routing) selects a suitable model for each request based on cost, latency, context length, or task complexity. --- - **Page:** LLM Gateway - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/llm-gateway - **Description:** An LLM Gateway is an infrastructure layer between applications and one or more LLM API providers. It gives applications a single controlled interface for model access while centralizing routing, authentication, observability, governance, and cost management. An LLM Gateway is an infrastructure layer between applications and one or more [LLM API providers](https://promptmetheus.com/resources/llm-knowledge-base/llm-api-provider). It gives applications a single controlled interface for model access while centralizing routing, authentication, observability, governance, and cost management. Typical capabilities include: - provider abstraction through one API; - [model routing](https://promptmetheus.com/resources/llm-knowledge-base/model-routing) and failover; - request and response logging; - token, latency, and cost tracking; - prompt caching; - rate limits and budgets; - organization, team, or application-level attribution; - [agent guardrails](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail); and - policy controls for data retention or provider selection. An LLM Gateway is different from a generic API gateway. A generic gateway treats requests as ordinary HTTP traffic. An LLM Gateway understands model names, context length, token usage, tool calls, streaming responses, provider-specific parameters, and AI-specific failure modes. The gateway pattern is useful when an organization uses multiple providers or needs centralized controls across many products and teams. It can reduce vendor lock-in and make cost attribution more reliable. It can also introduce latency, provider feature lag, and another operational dependency. For [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent), an LLM Gateway can enforce model-access policy and capture usage across nested model calls. It does not replace [agent tracing](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability), because an agent trace also needs tool, retrieval, approval, and orchestration spans. An LLM Gateway is closely related to an AI Gateway. The LLM term emphasizes model calls, while the AI Gateway term is often broader and may include agent tools, [MCP](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) traffic, and multimodal services. AWS Prescriptive Guidance describes the [LLM gateway pattern](https://docs.aws.amazon.com/prescriptive-guidance/latest/govern-architect-agentic-ai/model-access-layer.html) as a model access architecture for unified APIs, centralized policy enforcement, monitoring, cost management, and API normalization. --- - **Page:** LLM OS - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/llm-os - **Description:** An LLM OS is a computational architecture proposed by Andrej Karpathy in a post on X. In contrast to a traditional operating system, it is not deterministic and relies on Large Language Models (LLM) to process instructions. An LLM OS is a computational architecture proposed by [Andrej Karpathy](https://x.com/karpathy) in a [post on X](https://x.com/karpathy/status/1723140519554105733). In contrast to a traditional operating system, it is not deterministic and relies on [Large Language Models (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) to process instructions. At the core of the architecture is an LLM [inference](https://promptmetheus.com/resources/llm-knowledge-base/inference) engine (analogous to the CPU in a computer), which is connected to other LLMs, peripheral devices (video, audio, etc.), the internet, "tools" (calculator, code interpreter, etc.), and a file system (database with [vector embeddings](https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding)). A system running on an LLM OS is inherently less predictable than a conventional computer. However, it offers greater flexibility, enabling it to better adapt to new tasks and environments, potentially outperforming the capabilities of state-of-the-art systems by orders of magnitude. Modern implementations of this idea are commonly described in terms of [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent), [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling), [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory), and [agent orchestration](https://promptmetheus.com/resources/llm-knowledge-base/agent-orchestration). Protocols such as the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) provide standardized interfaces for some of the tools and data sources envisioned by the LLM OS architecture. --- - **Page:** LLM Priming - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/llm-priming - **Description:** LLM priming is a technique used to guide the output of Large Language Models (LLMs) by providing them with a set of instructions, examples, or context before generating text. By carefully crafting the priming text, developers can steer the LLM to produce content that aligns with the desired style, tone, or subject matter. Effective priming helps to improve the quality, relevance, and consistency of the generated text in various applications, such as chatbots, content creation tools, and AI-assisted writing platforms. Priming is a crucial aspect of working with LLMs in the field of Generative AI, as it enables users to harness the power of these models while maintaining control over the output. LLM priming is a technique used to guide the output of [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) by providing them with a set of instructions, examples, or [context](https://promptmetheus.com/resources/llm-knowledge-base/context) before generating text. By carefully crafting the priming text, developers can steer the LLM to produce content that aligns with the desired style, tone, or subject matter. Effective priming helps to improve the quality, relevance, and consistency of the generated text in various applications, such as [chatbots](https://promptmetheus.com/resources/llm-knowledge-base/chatbot), content creation tools, and AI-assisted writing platforms. Priming is a crucial aspect of working with LLMs in the field of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), as it enables users to harness the power of these models while maintaining control over the output. --- - **Page:** LLM-as-a-Judge - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/llm-as-a-judge - **Description:** LLM-as-a-Judge is an evaluation technique in which a Large Language Model (LLM) scores, ranks, or critiques outputs produced by another model or AI system. It is used when quality criteria such as relevance, clarity, completeness, or style are difficult to express with deterministic metrics. LLM-as-a-Judge is an evaluation technique in which a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) scores, ranks, or critiques outputs produced by another model or AI system. It is used when quality criteria such as relevance, clarity, completeness, or style are difficult to express with deterministic metrics. Common judging formats include: - assigning a score according to a rubric; - comparing two responses and selecting the better one; - checking whether an answer satisfies a list of criteria; and - producing a critique that explains a failure. LLM judges can scale [evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) more cheaply and quickly than human review, but their decisions are not objective ground truth. Known failure modes include position bias, preference for verbose answers, self-preference, sensitivity to formatting, inconsistent scoring, and vulnerability to instructions embedded in the content being judged. A reliable judging pipeline uses a precise rubric, hides irrelevant metadata, randomizes pairwise answer order, separates the evaluated content from judge instructions, and calibrates results against expert human ratings. Multiple judge samples or models can be used when variance is high. Deterministic verification remains preferable when correctness can be computed directly. Code should be executed against tests, structured data should be schema-checked, and mathematical answers should use symbolic or numeric verification where possible. An LLM judge is most appropriate for residual qualitative properties. In [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation), a judge can assess final-answer quality or the appropriateness of a trajectory. It should not be the sole authority for security, authorization, or irreversible actions. The research paper [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://arxiv.org/abs/2306.05685) established widely used methods for studying agreement between model judges and human preferences. --- - **Page:** Language Processing Unit (LPU) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/language-processing-unit-lpu - **Description:** A Language Processing Unit (LPU) is a specialized hardware accelerator designed for efficient inference on Large Language Models (LLMs) in the domain of Generative AI. It is a novel, purpose-built commercial system that aims to improve the speed and cost-effectiveness of AI applications. The LPU, developed by Groq, is a software-defined, AI-focused chip that offers significant advantages over traditional GPUs (Graphics Processing Units) used for AI model inference. A Language Processing Unit (LPU) is a specialized hardware accelerator designed for efficient [inference](https://promptmetheus.com/resources/llm-knowledge-base/inference) on [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) in the domain of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai). It is a novel, purpose-built commercial system that aims to improve the speed and cost-effectiveness of AI applications. The LPU, developed by [Groq](https://groq.com), is a software-defined, AI-focused chip that offers significant advantages over traditional GPUs (Graphics Processing Units) used for AI model inference. Key aspects of the LPU include: 1. **High Performance**: The LPU is reported to be 25 times faster and 20 times cheaper than the technology used to run models like ChatGPT-3.5, which relies on GPUs. 2. **Software-First Design**: The LPU is designed with a software-first approach, which improves performance and developer experience by reducing the need for schedulers, CUDA libraries, and kernels. 3. **Large-Scale Inference**: The LPU is optimized for large-scale [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) and converged HPC applications, making it suitable for a wide range of AI tasks. 4. **Efficient Routing**: The LPU's novel source-based, software-scheduled routing algorithm allows for load balancing and minimal routing, which contributes to its high efficiency. 5. **Cost-Effectiveness**: The LPU is designed to be affordable and readily available, using 14nm silicon, which is more cost-effective than the more advanced silicon used in GPUs. 6. **Multi-modal Capabilities**: The LPU's speed and affordability could enable more sophisticated AI products, including multi-modal capabilities for device control and task execution. The LPU is a significant advancement in AI hardware, offering faster and more cost-effective inference for large language models, which can lead to improved AI [chatbots](https://promptmetheus.com/resources/llm-knowledge-base/chatbot), real-time data analysis, and other applications that require quick and efficient AI decision-making. Also see [Tensor Processing Unit (TPU)](https://promptmetheus.com/resources/llm-knowledge-base/tensor-processing-unit-tpu). --- - **Page:** LoRA (Low-Rank Adaptation) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/lora-low-rank-adaptation - **Description:** LoRA, or Low-Rank Adaptation, is a parameter-efficient fine-tuning method for large neural networks. Instead of updating all model weights, LoRA freezes the base model and trains small low-rank matrices that modify selected layers. LoRA, or Low-Rank Adaptation, is a parameter-efficient fine-tuning method for large neural networks. Instead of updating all model weights, LoRA freezes the base model and trains small low-rank matrices that modify selected layers. In [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), LoRA is often used to adapt a model to a task, domain, style, or product behavior with much less training memory than full [fine-tuning](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning). The practical advantages include: - fewer trainable [model parameters](https://promptmetheus.com/resources/llm-knowledge-base/model-parameter); - lower GPU memory requirements; - smaller task-specific adapter files; - easier switching between adaptations; and - less operational overhead than storing a full copy of each fine-tuned model. LoRA is not a general replacement for high-quality data or evaluation. A poor dataset can still degrade behavior, and an adapter can conflict with safety policies or domain expectations. Teams should evaluate LoRA adapters against held-out tasks, regression suites, and production constraints. The original paper, [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685), introduced the method for adapting large language models with far fewer trainable parameters than full fine-tuning. --- - **Page:** Logit - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/logit - **Description:** A Logit refers to the raw, unnormalized output values produced by a classification model. These values are typically transformed through a function like the softmax function to produce probabilities. The term "logit" comes from logistic regression, where it refers to the log-odds of a probability. In deep learning, logits have a significant role in various loss functions and layers. A Logit refers to the raw, unnormalized output values produced by a classification model. These values are typically transformed through a function like the softmax function to produce probabilities. The term "logit" comes from logistic regression, where it refers to the log-odds of a probability. In deep learning, logits have a significant role in various loss functions and layers. --- - **Page:** Long-Horizon Task - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/long-horizon-task - **Description:** A long-horizon task is a task that requires an AI agent to make many decisions over an extended sequence of steps. It cannot be completed reliably by one prompt, one retrieval operation, or one short exchange. A long-horizon task is a task that requires an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) to make many decisions over an extended sequence of steps. It cannot be completed reliably by one prompt, one retrieval operation, or one short exchange. Examples include fixing a bug in a repository, preparing a due-diligence report, migrating a codebase, reconciling financial documents, planning a multi-step workflow, or coordinating several specialist agents. Long-horizon tasks typically involve: - planning and replanning; - multiple [tool calls](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling); - evolving context and intermediate artifacts; - external state changes; - error recovery; - checkpoints and approvals; - [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction); and - [resumable agent runs](https://promptmetheus.com/resources/llm-knowledge-base/resumable-agent-run). The difficulty is not only reasoning depth. Long-horizon work stresses memory, state management, tool reliability, cost, observability, and user trust. A small mistake early in the run can propagate through later actions. Strong systems break long-horizon tasks into milestones with explicit success criteria. They preserve artifacts, record decisions, validate intermediate outputs, and stop for human review before irreversible actions. [Agent traces](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability) are essential because the final answer alone rarely explains how the agent arrived there. Evaluation should measure end-to-end completion, not isolated snippets. For a coding task, this may include compiling, tests, diff quality, and review burden. For research, it may include source quality, citation support, and handling of contradictory evidence. OpenAI describes modern agents as systems that can operate independently for minutes or hours while orchestrating tool calls and iterating toward solutions in its discussion of [how agents are transforming work](https://openai.com/index/how-agents-are-transforming-work/). --- - **Page:** Lost-in-the-Middle Effect - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/lost-in-the-middle-effect - **Description:** The Lost-in-the-Middle Effect is a phenomenon observed in the context of Prompt Engineering and Large Language Models (LLMs), where the model's attention or focus diminishes for information located in the middle of a prompt. This effect can lead to suboptimal performance in tasks requiring the model to process or generate content based on information that is not at the beginning or end of the input. The Lost-in-the-Middle Effect is a phenomenon observed in the context of [Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering) and [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), where the model's [attention](https://promptmetheus.com/resources/llm-knowledge-base/attention) or focus diminishes for information located in the middle of a [prompt](https://promptmetheus.com/resources/llm-knowledge-base/prompt). This effect can lead to suboptimal performance in tasks requiring the model to process or generate content based on information that is not at the beginning or end of the input. Addressing this problem on the development level involves optimizing the model's attention mechanism and training strategies to ensure balanced processing across the entire input. On the prompt engineering side, performance issues related to the Lost-in-the-Middle Effect can be mitigated by strategically placing important information at the beginning or end of the prompt. Repeating key details at the end of the prompt also helps to reinforce their importance and prevent the model from overlooking them. --- - **Page:** MCP App - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mcp-app - **Description:** MCP Apps are an extension direction for the Model Context Protocol (MCP) that lets servers expose user-facing application experiences, not only tools and resources. The goal is to support richer interactions between users, AI applications, and MCP-connected systems. MCP Apps are an extension direction for the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) that lets servers expose user-facing application experiences, not only tools and resources. The goal is to support richer interactions between users, AI applications, and MCP-connected systems. Traditional MCP servers expose: - tools that an AI application can call; - resources that provide data for context; and - prompts that package reusable workflows. MCP Apps extend that model toward server-rendered or server-defined user interfaces. This is useful when a tool call needs more than a hidden API operation, such as reviewing a complex object, confirming a configuration, filling a structured form, or interacting with a task-specific UI. MCP Apps are especially relevant for enterprise [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) because not every workflow should be handled through free-form chat. A server-controlled interface can make permissions, data provenance, and user choices clearer. MCP Apps are one way to implement [Generative UI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ui) in an MCP environment. They can also be composed with interaction protocols such as [AG-UI](https://promptmetheus.com/resources/llm-knowledge-base/agent-user-interaction-protocol-ag-ui) or [A2UI](https://promptmetheus.com/resources/llm-knowledge-base/a2ui) when the host needs a structured stream of UI events, state updates, and user actions. They are also related to [MCP Elicitation](https://promptmetheus.com/resources/llm-knowledge-base/mcp-elicitation), but the scope is different. Elicitation asks for specific user input during a protocol operation. MCP Apps can define broader interactive experiences. Security remains central. A client should clearly show which MCP server is presenting an interface, what data is being requested or changed, and what permissions apply. Servers must not use UI controls to bypass user consent or application policy. MCP Apps are part of the broader [MCP Extensions](https://promptmetheus.com/resources/llm-knowledge-base/mcp-extension) direction, allowing richer capabilities to evolve without bloating the core protocol. The MCP maintainers identified MCP Apps as a first-class extension area in the [2026-07-28 release candidate](https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate). --- - **Page:** MCP Authorization Extension - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mcp-authorization-extension - **Description:** An MCP Authorization Extension is an extension to the Model Context Protocol (MCP) that standardizes additional authorization behavior beyond the core protocol. Enterprise-Managed Authorization is a concrete example: it lets an organization's identity provider govern access to protected MCP servers. An MCP Authorization Extension is an extension to the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) that standardizes additional authorization behavior beyond the core protocol. [Enterprise-Managed Authorization](https://promptmetheus.com/resources/llm-knowledge-base/enterprise-managed-authorization) is a concrete example: it lets an organization's identity provider govern access to protected MCP servers. MCP already uses OAuth-oriented authorization patterns for HTTP-based transports. Authorization extensions allow the ecosystem to add enterprise requirements such as centralized identity provider policy, machine-to-machine authorization, managed consent, or cross-application access without forcing all behavior into the base protocol. The main problems addressed are: - repeated consent prompts across many MCP servers; - unclear trust relationships between clients and servers; - enterprise policy enforcement; - per-user and per-organization access control; - auditability of tool and resource access; and - compatibility with existing identity infrastructure. An MCP Authorization Extension is part of the broader [MCP Extensions](https://promptmetheus.com/resources/llm-knowledge-base/mcp-extension) framework. It should be explicitly discoverable, versioned, and compatible with the server's transport and deployment model. Authorization is not a substitute for tool-level safety. Even an authorized user can request a harmful or unintended action. Servers and clients still need least-privilege scopes, confirmation for consequential actions, [agent guardrails](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail), and activity logs. This term is particularly important for enterprise [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) because agents may call many MCP servers on behalf of users. Centralized authorization reduces friction while preserving governance. The MCP project announced Enterprise-Managed Authorization as a stable extension in its [authorization update](https://blog.modelcontextprotocol.io/posts/enterprise-managed-auth/). --- - **Page:** MCP Elicitation - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mcp-elicitation - **Description:** MCP Elicitation is a mechanism in the Model Context Protocol (MCP) for an MCP server to request additional input from the user through the client. It is used when a server-side operation cannot proceed safely or correctly without user-provided information. MCP Elicitation is a mechanism in the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) for an MCP server to request additional input from the user through the client. It is used when a server-side operation cannot proceed safely or correctly without user-provided information. Examples include asking the user to choose an account, confirm a missing date range, provide a required form field, or approve opening an external authorization URL. MCP Elicitation is important because tool calls are not always one-shot API operations. A server may discover during execution that it needs structured user input. The client must present that request clearly, bind it to the requesting server, and allow the user to accept, decline, or cancel. Security considerations include: - showing which server is requesting information; - explaining why the information is needed; - avoiding sensitive data in URLs; - requiring consent before opening external links; - not allowing the model to silently fill user-only inputs; and - binding responses to the correct user and client session. In the 2026-07-28 [MCP Stateless Protocol](https://promptmetheus.com/resources/llm-knowledge-base/mcp-stateless-protocol) revision, a [Multi Round-Trip Request](https://promptmetheus.com/resources/llm-knowledge-base/multi-round-trip-request) carries elicitation in an `input_required` result. The client gathers the user's answer and retries the original operation with an `inputResponses` value. Earlier protocol versions use server-to-client JSON-RPC requests instead. MCP Elicitation is not the same as ordinary [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling). Tool calling is model-to-application action selection. Elicitation is server-to-client user-input negotiation within the MCP protocol. The MCP project documents the 2026 interaction flow in [SEP-2322](https://github.com/modelcontextprotocol/modelcontextprotocol/blob/main/seps/2322-MRTR.md). --- - **Page:** MCP Extension - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mcp-extension - **Description:** An MCP extension is an optional capability module that evolves the Model Context Protocol (MCP) without forcing every implementation to support every feature in the core protocol. An extension can add behavior such as an MCP task, an app, or Enterprise-Managed Authorization. An MCP extension is an optional capability module that evolves the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) without forcing every implementation to support every feature in the core protocol. An extension can add behavior such as an [MCP task](https://promptmetheus.com/resources/llm-knowledge-base/mcp-task), an app, or [Enterprise-Managed Authorization](https://promptmetheus.com/resources/llm-knowledge-base/enterprise-managed-authorization). The extension model matters because MCP is used across local tools, enterprise SaaS systems, IDEs, agent platforms, and gateways. A small core protocol improves interoperability, while extensions let specialized capabilities mature at their own pace. An MCP extension can define: - new message types or result shapes; - additional metadata; - lifecycle rules; - authorization behavior; - UI or task abstractions; - compatibility requirements; and - deprecation or migration guidance. Extensions should be explicitly discoverable and versioned. A client should not assume that a server supports an extension merely because it supports MCP. Conversely, a server should not require an extension unless the client can detect and handle that requirement. MCP Extensions are related to, but broader than, [MCP Authorization Extension](https://promptmetheus.com/resources/llm-knowledge-base/mcp-authorization-extension). Authorization is one class of extension; other extensions may cover interactive tasks, server-rendered interfaces, or other protocol-level behavior. For developers building [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent), extensions create a cleaner boundary between tool access and higher-order experiences. For enterprise operators, they provide a path to standardize governance features without fragmenting the base protocol. The MCP project describes the extension framework in the [2026-07-28 specification release](https://blog.modelcontextprotocol.io/posts/2026-07-28/). --- - **Page:** MCP Stateless Protocol - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mcp-stateless-protocol - **Description:** The MCP Stateless Protocol is the request-handling model introduced in the 2026-07-28 revision of the Model Context Protocol (MCP). Each request carries the protocol information needed to process it without a long-lived protocol session. This makes MCP traffic easier to route, cache, scale, and observe through ordinary HTTP infrastructure. The MCP Stateless Protocol is the request-handling model introduced in the 2026-07-28 revision of the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp). Each request carries the protocol information needed to process it without a long-lived protocol session. This makes MCP traffic easier to route, cache, scale, and observe through ordinary HTTP infrastructure. Earlier MCP designs were more session-oriented. A client and server established a connection, negotiated capabilities, and could use server-to-client requests during that session. The 2026 revision removes that protocol-level session dependence and pushes application state into explicit request and response data or higher-level application storage. The benefits include: - compatibility with load balancers and serverless infrastructure; - easier horizontal scaling; - simpler retries and request tracing; - better cacheability for safe reads; - fewer assumptions about sticky sessions; and - clearer separation between protocol state and application state. Stateless does not mean the application has no state. A server can still maintain databases, user accounts, job records, resources, and authorization policy. The point is that the transport protocol no longer requires the same kind of durable connection state for every interaction. This shift affects features such as [MCP Elicitation](https://promptmetheus.com/resources/llm-knowledge-base/mcp-elicitation), sampling, roots, logging, and server-to-client requests. A [Multi Round-Trip Request](https://promptmetheus.com/resources/llm-knowledge-base/multi-round-trip-request) carries a need for input in the result and lets the client retry with the answer. An [MCP task](https://promptmetheus.com/resources/llm-knowledge-base/mcp-task) gives a long-running server operation an explicit handle without restoring a transport session. The MCP Stateless Protocol also supports enterprise deployment patterns where an [LLM Gateway](https://promptmetheus.com/resources/llm-knowledge-base/llm-gateway), API gateways, and observability tools need to inspect and route high-volume AI traffic. The MCP project explains the stateless protocol core in its [2026-07-28 specification release](https://blog.modelcontextprotocol.io/posts/2026-07-28/). --- - **Page:** MCP Task - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mcp-task - **Description:** An MCP task is a durable handle for work started by a Model Context Protocol (MCP) tool call when the result will not be available in the initial response. The Tasks extension lets a server return resultType: "task" and a taskId instead of keeping the original tools/call request open until the work finishes. An MCP task is a durable handle for work started by a [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) tool call when the result will not be available in the initial response. The Tasks extension lets a server return `resultType: "task"` and a `taskId` instead of keeping the original `tools/call` request open until the work finishes. The client uses `tasks/get` to poll for status and the eventual result. A task can be `working`, require additional input, or reach a terminal state such as `completed`, `failed`, or `cancelled`. The extension also defines `tasks/update` for supplying requested input and `tasks/cancel` for requesting cancellation. Cancellation is cooperative: an acknowledgement does not prove that the underlying operation stopped. This is useful for operations such as a long-running analysis, a data export, or a deployment that may outlive a network request. A production client should persist the task ID, respect the server's polling interval, handle expiry, and check the final result rather than equating task creation with success. A server should restrict task lookup and updates to the authorized caller; a task ID is a reference, not permission to access someone else's work. An MCP task describes the lifecycle of one server-side operation. It is narrower than a [resumable agent run](https://promptmetheus.com/resources/llm-knowledge-base/resumable-agent-run), which may span many model turns and tool calls, or an [agent session](https://promptmetheus.com/resources/llm-knowledge-base/agent-session), which can contain multiple runs. If the task needs a user answer while it is running, the client can supply that answer through the extension's input flow; for synchronous interactions, a [Multi Round-Trip Request](https://promptmetheus.com/resources/llm-knowledge-base/multi-round-trip-request) serves a different purpose. See the [MCP Tasks extension specification](https://tasks.extensions.modelcontextprotocol.io/specification/draft/tasks) for the result shape, lifecycle methods, and security requirements. --- - **Page:** Machine Learning (ML) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml - **Description:** Machine learning is a subset of Artificial Intelligence (AI) that enables software applications to become more accurate at predicting outcomes without being explicitly programmed to do so. It involves the use of algorithms and statistical models to allow computers to learn from and make decisions based on data. Machine Learning encompasses a range of techniques and methodologies, such as supervised learning, unsupervised learning, and reinforcement learning, each with specific applications and use cases. By analyzing and identifying patterns in large datasets, machine learning systems can adapt and improve their performance over time, making it a foundational technology for many predictive and analytical applications in various industries. Machine learning is a subset of [Artificial Intelligence (AI)](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai) that enables software applications to become more accurate at predicting outcomes without being explicitly programmed to do so. It involves the use of algorithms and statistical models to allow computers to learn from and make decisions based on data. [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) encompasses a range of techniques and methodologies, such as supervised learning, unsupervised learning, and reinforcement learning, each with specific applications and use cases. By analyzing and identifying patterns in large datasets, machine learning systems can adapt and improve their performance over time, making it a foundational technology for many predictive and analytical applications in various industries. --- - **Page:** Mamba - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mamba - **Description:** Mamba is a neural network architecture based on selective state space models. It was proposed as an efficient alternative to the Transformer for sequence modeling, especially where long inputs make full attention expensive. Mamba is a neural network architecture based on selective state space models. It was proposed as an efficient alternative to the [Transformer](https://promptmetheus.com/resources/llm-knowledge-base/transformer) for sequence modeling, especially where long inputs make full attention expensive. Transformers use self-attention to compare tokens across a sequence. That gives strong context handling but can be costly as sequence length grows. Mamba instead uses a state-space mechanism with input-dependent selection, allowing sequence processing with more favorable scaling characteristics. In language modeling, Mamba is relevant because it explores a different architecture path for [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). It is not simply a smaller transformer or a [Mixture of Experts (MoE)](https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-experts-moe). It changes the sequence-processing primitive itself. Important tradeoffs include: - better efficiency on some long-sequence workloads; - different hardware kernels and implementation requirements; - less ecosystem maturity than transformer models; - different behavior on tasks that rely on exact token-to-token interactions; and - evolving evidence about where state-space models match or fall short of attention. Mamba should therefore be treated as an architecture family, not as a drop-in guarantee of better LLM performance. Benchmark results depend on model size, data, training recipe, and workload. The original paper, [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752), introduces the architecture and its selective state-space layer. --- - **Page:** Mechanistic Interpretability - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mechanistic-interpretability - **Description:** Mechanistic interpretability is the study of how neural networks compute their outputs by reverse-engineering internal mechanisms such as features, circuits, attention heads, activations, and weights. It aims to explain what a trained model is doing internally, not only whether its output is correct. Mechanistic interpretability is the study of how neural networks compute their outputs by reverse-engineering internal mechanisms such as features, circuits, attention heads, activations, and weights. It aims to explain what a trained model is doing internally, not only whether its output is correct. In [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), mechanistic interpretability research often investigates: - which internal features correspond to concepts or behaviors; - how attention heads move information through the residual stream; - how circuits implement tasks such as copying, induction, or refusal; - whether a feature causally affects model output; and - how model internals change under fine-tuning or steering. Mechanistic interpretability is different from ordinary evaluation. An [evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) measures external behavior. Mechanistic interpretability tries to identify internal computations that produce that behavior. The field is technically difficult because modern models have billions or trillions of [model parameters](https://promptmetheus.com/resources/llm-knowledge-base/model-parameter), distributed representations, and nonlinear interactions across layers. Results that explain a small circuit may not generalize to all model behavior. Anthropic's [Transformer Circuits Thread](https://transformer-circuits.pub/) collects research on reverse-engineering transformer internals and related interpretability methods. --- - **Page:** Meta-prompting - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/meta-prompting - **Description:** Meta-prompting is the use of a model to create, critique, transform, or optimize prompts. Instead of asking the model to solve the end task directly, a meta-prompt asks it to improve the instructions, examples, structure, or evaluation criteria used by another prompt. Meta-prompting is the use of a model to create, critique, transform, or optimize prompts. Instead of asking the model to solve the end task directly, a meta-prompt asks it to improve the instructions, examples, structure, or evaluation criteria used by another prompt. Common uses include: - generating candidate prompts for an application workflow; - rewriting prompts for clarity or lower token usage; - creating prompt variants for A/B tests; - converting informal requirements into a structured prompt template; - critiquing a prompt for ambiguity or missing constraints; and - producing rubrics for [LLM-as-a-Judge](https://promptmetheus.com/resources/llm-knowledge-base/llm-as-a-judge) evaluations. Meta-prompting is related to [prompt optimization](https://promptmetheus.com/resources/llm-knowledge-base/prompt-optimization), but it is not the whole optimization process. A generated prompt still needs offline tests, human review, and production evaluation. The model may produce prompts that sound rigorous while embedding false assumptions or fitting too narrowly to the examples it saw. In professional systems, meta-prompting is most useful when paired with version control, task-specific evaluations, and clear acceptance criteria. Comet describes meta prompting as using LLMs to optimize prompt templates in [Meta Prompting: Use LLMs to Optimize Prompts for AI Apps and Agents](https://www.comet.com/site/blog/meta-prompting/). --- - **Page:** Mixture of Experts (MoE) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-experts-moe - **Description:** A Mixture of Experts (MoE) is a Machine Learning architecture that integrates multiple specialized sub-models, referred to as "experts," to collectively address complex tasks. A gating network dynamically assigns input data to the most appropriate expert(s) based on the input's characteristics. This approach enables the model to handle diverse data patterns more effectively by leveraging the specialized capabilities of each expert. A Mixture of Experts (MoE) is a [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) architecture that integrates multiple specialized sub-models, referred to as "experts," to collectively address complex tasks. A [gating network](https://en.wikipedia.org/wiki/Gating_mechanism) dynamically assigns input data to the most appropriate expert(s) based on the input's characteristics. This approach enables the model to handle diverse data patterns more effectively by leveraging the specialized capabilities of each expert. In deep learning, MoEs are particularly valuable for scaling [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). By activating only a subset of experts for each input, MoEs optimize computational efficiency without compromising performance. This sparsely activated mechanism makes an MoE a common type of [sparse expert model](https://promptmetheus.com/resources/llm-knowledge-base/sparse-expert-model), allowing the development of models with a vast number of parameters while managing resource consumption effectively. The MoE architecture consists of two primary components: - Experts: Individual neural networks trained to specialize in specific subsets of the input data. - Gating Network: A mechanism that learns to route each input to the most suitable expert(s) by evaluating the input's features. This modular design not only enhances model capacity but also facilitates adaptability, as experts can be independently trained or updated to accommodate new data patterns or tasks. Implementing MoEs can lead to significant improvements in model performance and efficiency, making them a prominent choice in advancing AI capabilities across various domains. --- - **Page:** Mixture of Agents (MoA) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-agents-moa - **Description:** Mixture of Agents (MoA) is an innovative framework in Artificial Intelligence that combines the capabilities of multiple Large Language Models (LLMs) to enhance performance in Natural Language Processing tasks. In this architecture, several LLMs, referred to as "agents," are organized into layers. Each agent processes inputs and generates outputs, which are then utilized by agents in subsequent layers for further refinement. This collaborative approach leverages the unique strengths of individual models, resulting in more accurate and comprehensive responses. Mixture of Agents (MoA) is an innovative framework in [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai) that combines the capabilities of multiple [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) to enhance performance in [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) tasks. In this architecture, several LLMs, referred to as "[agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent)," are organized into layers. Each agent processes inputs and generates outputs, which are then utilized by agents in subsequent layers for further refinement. This collaborative approach leverages the unique strengths of individual models, resulting in more accurate and comprehensive responses. Notably, implementations like [Together MoA](https://www.together.ai/blog/together-moa) have demonstrated superior performance, achieving a 65.1% score on the [AlpacaEval 2.0](https://github.com/tatsu-lab/alpaca_eval) benchmark, surpassing GPT-4 Omni's 57.5%. The MoA methodology exemplifies the potential of collective intelligence in advancing the capabilities of LLMs. --- - **Page:** Mixture of Models (MoM) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-models-mom - **Description:** In the context of Artificial Intelligence a Mixture of Models (MoM) is an ensemble learning technique that integrates multiple distinct models to address complex tasks by leveraging their combined strengths. This approach involves training diverse models separately and then aggregating their predictions to produce a final output. The aggregation can be achieved through various methods, such as averaging predictions, weighted voting, or more sophisticated strategies. MoMs are particularly effective in scenarios where individual models capture different patterns or aspects of the data, leading to improved performance, robustness, and generalization compared to relying on a single model. In the context of [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai) a Mixture of Models (MoM) is an ensemble learning technique that integrates multiple distinct models to address complex tasks by leveraging their combined strengths. This approach involves training diverse models separately and then aggregating their predictions to produce a final output. The aggregation can be achieved through various methods, such as averaging predictions, weighted voting, or more sophisticated strategies. MoMs are particularly effective in scenarios where individual models capture different patterns or aspects of the data, leading to improved performance, robustness, and generalization compared to relying on a single model. ## Key Characteristics - **Diversity:** Combining models with varying architectures or learning algorithms to capture a broader spectrum of data patterns. - **Aggregation Strategy:** Employing methods like averaging, voting, or stacking to integrate individual model predictions into a cohesive output. - **Improved Performance:** Enhancing accuracy and robustness by mitigating individual model [biases](https://promptmetheus.com/resources/llm-knowledge-base/bias) and errors. ## Applications - **Natural Language Processing (NLP):** Integrating models like transformers, recurrent neural networks, and convolutional neural networks to better understand and generate human language. - **Computer Vision:** Combining models such as convolutional neural networks and vision transformers to improve image recognition and classification tasks. - **Predictive Analytics:** Merging time-series models with regression models to enhance forecasting accuracy in fields like finance and healthcare. - **Large Language Models (LLMs):** Leveraging multiple LLMs specialized in different tasks (e.g., reasoning, summarization, or domain-specific expertise) and dynamically selecting or blending responses to optimize performance for complex queries.
By harnessing the complementary strengths of various models, MoMs offer a versatile and powerful framework for tackling complex problems across diverse domains. --- - **Page:** Model Context Protocol (MCP) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp - **Description:** The Model Context Protocol (MCP) is an open protocol for connecting AI applications to external tools, data sources, and reusable prompts. It was introduced by Anthropic and provides a common interface between an MCP host, such as an AI agent or development environment, and one or more MCP servers. The Model Context Protocol (MCP) is an open protocol for connecting AI applications to external tools, data sources, and reusable prompts. It was introduced by [Anthropic](https://www.anthropic.com/) and provides a common interface between an MCP host, such as an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) or development environment, and one or more MCP servers. An MCP server can expose three primary capability types: - **Tools:** executable operations that can be selected through [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling). - **Resources:** application-controlled data that can be read and included in model [context](https://promptmetheus.com/resources/llm-knowledge-base/context). - **Prompts:** reusable templates or workflows offered by the server. MCP uses a client-server architecture. The host manages the user experience and model interaction, while an MCP client maintains a connection to a specific server. Messages use JSON-RPC, commonly over local standard input/output or Streamable HTTP. The 2026-07-28 revision introduced an [MCP stateless protocol](https://promptmetheus.com/resources/llm-knowledge-base/mcp-stateless-protocol) for request handling without sticky protocol sessions. It also formalized [MCP extensions](https://promptmetheus.com/resources/llm-knowledge-base/mcp-extension), including the [MCP Task](https://promptmetheus.com/resources/llm-knowledge-base/mcp-task) for long-running tool work. A [Multi Round-Trip Request](https://promptmetheus.com/resources/llm-knowledge-base/multi-round-trip-request) handles input needed during a call, while [Enterprise-Managed Authorization](https://promptmetheus.com/resources/llm-knowledge-base/enterprise-managed-authorization) supports centrally governed access. The protocol separates capability discovery from execution. A client can inspect the tools and resources offered by a server before deciding what to make available to the model. This reduces custom integration code and allows the same server to work with multiple compatible hosts. MCP does not make a server trustworthy. Hosts still need authentication, authorization, user consent, input validation, network restrictions, and controls around consequential actions. The [MCP authorization extension](https://promptmetheus.com/resources/llm-knowledge-base/mcp-authorization-extension) and [MCP elicitation](https://promptmetheus.com/resources/llm-knowledge-base/mcp-elicitation) address parts of that control surface, but hosts still need application-level policy. Tool descriptions and returned content can influence model behavior, so MCP integrations must also account for indirect [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). MCP is complementary to the [Agent2Agent Protocol (A2A)](https://promptmetheus.com/resources/llm-knowledge-base/agent2agent-protocol-a2a) and [AG-UI](https://promptmetheus.com/resources/llm-knowledge-base/agent-user-interaction-protocol-ag-ui). MCP connects an AI application to tools, context, and [MCP apps](https://promptmetheus.com/resources/llm-knowledge-base/mcp-app); A2A supports communication and task delegation between independent agents; AG-UI standardizes event streams between agents and user interfaces. Rich application surfaces may also use [Generative UI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ui) or [A2UI](https://promptmetheus.com/resources/llm-knowledge-base/a2ui) when model-driven interaction needs a structured UI contract. Discovery can also happen outside the protocol connection itself. For example, an [agents.txt](https://promptmetheus.com/resources/llm-knowledge-base/agents-txt) file can advertise agent-facing endpoints, while the MCP connection still governs concrete tool and resource execution. The current specification and SDKs are available on the [Model Context Protocol website](https://modelcontextprotocol.io/docs/2026-07-28/getting-started/intro). --- - **Page:** Model Distillation - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/model-distillation - **Description:** Model distillation is a training technique that transfers behavior from a capable teacher model to a smaller or more efficient student model. The student learns from teacher-generated probabilities, labels, outputs, reasoning traces, or synthetic data rather than relying only on the original training dataset. Model distillation is a training technique that transfers behavior from a capable teacher model to a smaller or more efficient student model. The student learns from teacher-generated probabilities, labels, outputs, reasoning traces, or [synthetic data](https://promptmetheus.com/resources/llm-knowledge-base/synthetic-data) rather than relying only on the original training dataset. Distillation can target general language capability or a narrower task. A common pipeline uses the teacher to generate answers for a curated set of prompts, filters those answers for quality, and then applies [Supervised Fine-Tuning (SFT)](https://promptmetheus.com/resources/llm-knowledge-base/supervised-fine-tuning-sft) to the student. Several forms are used: - **Response distillation:** train on final teacher answers. - **Logit distillation:** match the teacher's output probability distribution. - **Feature distillation:** align internal representations. - **Reasoning distillation:** train on intermediate solution traces or explanations. - **On-policy distillation:** correct or score outputs sampled from the student itself. The objective is usually to reduce inference cost, latency, memory use, or deployment requirements while retaining important capability. Distillation is often combined with [quantization](https://promptmetheus.com/resources/llm-knowledge-base/quantization) when deploying a [Small Language Model (SLM)](https://promptmetheus.com/resources/llm-knowledge-base/small-language-model-slm). A student does not automatically inherit only the teacher's strengths. It can also reproduce biases, factual errors, unsafe behavior, and stylistic artifacts. Teacher-generated datasets require provenance, filtering, contamination checks, and independent evaluation. Distillation differs from compression methods that only change numerical representation. It changes model parameters through training and can specialize the student to a particular behavior or domain. The Google Research paper [Distilling Step-by-Step](https://arxiv.org/abs/2305.02301) demonstrates using teacher-generated rationales as additional supervision for smaller language models. --- - **Page:** Model Parameter - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/model-parameter - **Description:** A model parameter is a learned numerical value inside a machine learning model. In neural networks, parameters are primarily weights and biases that are adjusted during training to reduce prediction error. A model parameter is a learned numerical value inside a machine learning model. In neural networks, parameters are primarily weights and biases that are adjusted during training to reduce prediction error. In a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), parameters encode statistical structure learned from training data. They influence token probabilities, factual associations, style, reasoning patterns, and many other behaviors, but they are not a database of explicit records. Parameter count is often used as a rough measure of model scale. Larger models can have more capacity, but parameter count alone does not determine quality. Architecture, training data, [tokenization](https://promptmetheus.com/resources/llm-knowledge-base/tokenization), post-training, inference compute, and evaluation coverage all matter. Some training methods update all parameters. Parameter-efficient methods such as [LoRA (Low-Rank Adaptation)](https://promptmetheus.com/resources/llm-knowledge-base/lora-low-rank-adaptation) update a much smaller set of adapter parameters while leaving the base model frozen. Sparse architectures complicate parameter-count comparisons. A [sparse expert model](https://promptmetheus.com/resources/llm-knowledge-base/sparse-expert-model) may contain many total parameters but activate only a subset for each token. Google's Machine Learning Glossary defines a model as a learned set of parameters and structure in its [model](https://developers.google.com/machine-learning/glossary#model) entry. --- - **Page:** Model Routing - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/model-routing - **Description:** Model routing is the process of selecting which model or provider should handle a given AI request. The routing decision may be based on task type, expected quality, cost, latency, context length, availability, compliance policy, tool support, or user tier. Model routing is the process of selecting which model or provider should handle a given AI request. The routing decision may be based on task type, expected quality, cost, latency, context length, availability, compliance policy, tool support, or user tier. Model routing can happen in application code, an [LLM Gateway](https://promptmetheus.com/resources/llm-knowledge-base/llm-gateway), a cloud service, or a model provider. It is increasingly important because production systems often combine frontier [reasoning models](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model), cheaper [Small Language Models (SLMs)](https://promptmetheus.com/resources/llm-knowledge-base/small-language-model-slm), embedding models, image models, and specialized fine-tuned models. Common routing strategies include: - static rules by feature or endpoint; - intent classification; - complexity estimation; - fallback on provider errors or rate limits; - cost-first or latency-first selection; - quality prediction; - user-configured model preferences; and - cascades where a small model handles easy cases and escalates difficult cases. Routing is not merely load balancing. It changes model behavior and therefore must be evaluated like any other model choice. A cheaper model may be sufficient for classification but inadequate for legal analysis or complex code changes. For [agentic workflows](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow), routing can apply at each step. A planner may use a frontier model, while extraction, formatting, and validation steps use cheaper models or deterministic code. Routing decisions should be logged with request metadata, selected model, fallback path, latency, cost, and outcome. Without this, regressions are hard to diagnose and [token economics](https://promptmetheus.com/resources/llm-knowledge-base/token-economics) cannot be measured accurately. Amazon Bedrock's [intelligent prompt routing documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-routing.html) describes routing prompts between foundation models to optimize response quality and cost. --- - **Page:** Moderation - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/moderation - **Description:** In the context of Generative AI and Large Language Models (LLMs), moderation refers to the processes and mechanisms implemented to oversee and regulate the content generated by these systems. The primary objectives of moderation are to ensure that AI outputs align with ethical standards, comply with legal requirements, and adhere to platform-specific guidelines, thereby safeguarding users from potentially harmful or inappropriate material. In the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) and [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), moderation refers to the processes and mechanisms implemented to oversee and regulate the content generated by these systems. The primary objectives of moderation are to ensure that AI outputs align with ethical standards, comply with legal requirements, and adhere to platform-specific guidelines, thereby safeguarding users from potentially harmful or inappropriate material. **Key Aspects of Moderation in AI Systems:** 1. **Content Filtering:** Utilizing automated tools to detect and block content that violates established guidelines, such as hate speech, explicit material, or misinformation. 2. **Human Oversight:** Employing human moderators to review flagged content, providing nuanced judgment that automated systems may lack. 3. **User Reporting Mechanisms:** Allowing users to flag problematic content, facilitating community involvement in maintaining platform integrity. 4. **Adaptive Learning:** Implementing systems that learn from moderation decisions to improve future content evaluations, enhancing the accuracy and efficiency of moderation efforts. **Challenges in Moderation:** - **Scalability:** The vast and rapid generation of content by AI systems necessitates scalable moderation solutions to effectively manage the volume. - **Contextual Understanding:** Automated systems may struggle with context, leading to potential misclassification of content. - **Evasion Tactics:** Users may attempt to circumvent moderation by manipulating language or using coded expressions, challenging the detection capabilities of moderation systems. **Recent Developments:** - **AI-Driven Moderation:** Advancements in AI have led to the development of more sophisticated moderation tools capable of analyzing and filtering content in real-time. - **Regulatory Measures:** Governments are increasingly enacting regulations to ensure that AI-generated content adheres to societal norms and legal standards. For instance, China's Cyberspace Administration has proposed regulations mandating clear labeling of AI-generated content to combat misinformation. - **Platform Initiatives:** Social media platforms are enhancing their moderation policies to address the challenges posed by AI-generated content, including the implementation of labeling systems and the development of detection algorithms. Effective moderation is crucial in the deployment of Generative AI and LLMs, ensuring that these technologies are used responsibly and that the content they produce contributes positively to the digital ecosystem. --- - **Page:** Multi Round-Trip Request - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/multi-round-trip-request - **Description:** A Multi Round-Trip Request (MRTR) is the Model Context Protocol (MCP) pattern for obtaining additional input during a client-initiated operation without keeping a bidirectional protocol session open. It was introduced with the 2026-07-28 MCP Stateless Protocol revision. A Multi Round-Trip Request (MRTR) is the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) pattern for obtaining additional input during a client-initiated operation without keeping a bidirectional protocol session open. It was introduced with the 2026-07-28 [MCP Stateless Protocol](https://promptmetheus.com/resources/llm-knowledge-base/mcp-stateless-protocol) revision. For example, a tool asked to change a record may discover that a required field is missing. Instead of completing the change or failing immediately, the server returns `resultType: "input_required"` with keyed `inputRequests`. The client obtains the requested answer, then retries the original operation with `inputResponses`. If the server needs to carry information between rounds, it includes an opaque `requestState` that the client echoes on the retry. More than one round can occur before the server returns `resultType: "complete"`. The embedded request may represent [MCP Elicitation](https://promptmetheus.com/resources/llm-knowledge-base/mcp-elicitation) of a user, model sampling, or another supported input type. The client remains responsible for presenting a user request honestly and respecting a decline or cancellation. The server must treat returned answers and echoed `requestState` as untrusted input. State carried through the client should be integrity-protected, bound to the original operation and caller, and given an expiry so it cannot be altered or replayed for a different action. MRTR is an interaction pattern, not a background job. An [MCP task](https://promptmetheus.com/resources/llm-knowledge-base/mcp-task) represents asynchronous server-side work that a client can poll later. An MRTR instead pauses one logical request until a missing input is supplied; a task may itself require input, but the two mechanisms solve different lifecycle problems. The protocol's [SEP-2322](https://github.com/modelcontextprotocol/modelcontextprotocol/blob/main/seps/2322-MRTR.md) specifies the request and retry flow, including `inputRequests`, `inputResponses`, and `requestState`. --- - **Page:** Multi-modality - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/multi-modality - **Description:** In the context of Large Language Models (LLMs), multi-modality refers to the capability of models to process and integrate multiple types of data inputs or outputs, known as modalities, such as text, images, audio, and video. This integration allows models to develop a more comprehensive understanding of complex data, enhancing performance in tasks like visual question answering, cross-modal retrieval, text-to-image generation, and image captioning. In the context of [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), multi-modality refers to the capability of models to process and integrate multiple types of data inputs or outputs, known as modalities, such as text, images, audio, and video. This integration allows models to develop a more comprehensive understanding of complex data, enhancing performance in tasks like visual question answering, cross-modal retrieval, text-to-image generation, and image captioning. For instance, a multi-modal model might analyze both textual descriptions and corresponding images to generate accurate captions or answer questions about visual content. This approach leverages the strengths of each modality, leading to more robust and versatile AI applications. Recent advancements have led to the development of large multimodal models, such as GPT-4 and Google's Gemini, which can process and generate content across various modalities, thereby broadening their applicability and effectiveness in real-world scenarios. A [Vision-Language Model (VLM)](https://promptmetheus.com/resources/llm-knowledge-base/vision-language-model-vlm) is a specific type of multimodal model focused on connecting images or video with language. --- - **Page:** NLWeb - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/nlweb - **Description:** NLWeb is a protocol and reference implementation for adding natural-language interfaces to websites. It lets a site expose a structured endpoint that accepts natural-language questions and returns ranked results, summaries, or generated answers using the site's own data. NLWeb is a protocol and reference implementation for adding natural-language interfaces to websites. It lets a site expose a structured endpoint that accepts natural-language questions and returns ranked results, summaries, or generated answers using the site's own data. NLWeb is aimed at websites whose content can be represented as collections of structured items, such as products, articles, recipes, jobs, places, reviews, or events. It can use existing machine-readable data such as Schema.org, RSS, search indexes, and databases. An NLWeb server commonly exposes: - an `/ask` endpoint for natural-language queries; - modes such as list, summarize, or generate; - structured JSON responses with result items and metadata; - streaming responses for interactive clients; and - an MCP-compatible endpoint so agents can call the site as a tool. This makes NLWeb adjacent to [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag) and [real-time fact-checking](https://promptmetheus.com/resources/llm-knowledge-base/real-time-fact-checking), but its scope is narrower: it defines a standard way for a website to answer questions about its own public or permitted data. NLWeb does not remove the need for source quality, authorization, or anti-abuse controls. A public NLWeb endpoint should define what data is exposed, how results are ranked, and how generated responses cite or preserve source provenance. The [NLWeb documentation](https://nlweb.ai/docs/software) describes NLWeb as a protocol for agents, applications, and humans to communicate with websites using natural language. --- - **Page:** Natural Language Processing (NLP) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp - **Description:** Natural Language Processing (NLP) is a subfield of Artificial Intelligence that focuses on the interaction between computers and humans through natural language. The ultimate objective of NLP is to read, decipher, understand, and make sense of the human language in a valuable way. It is used to analyze, understand, and derive meaning from human language in a smart and useful way, and is at the core of various applications like voice assistants, translation services, chatbots, and sentiment analysis tools. Natural Language Processing (NLP) is a subfield of [Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/artificial-intelligence-ai) that focuses on the interaction between computers and humans through natural language. The ultimate objective of NLP is to read, decipher, understand, and make sense of the human language in a valuable way. It is used to analyze, understand, and derive meaning from human language in a smart and useful way, and is at the core of various applications like voice assistants, translation services, [chatbots](https://promptmetheus.com/resources/llm-knowledge-base/chatbot), and sentiment analysis tools. --- - **Page:** Neural Network - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/neural-network - **Description:** An artificial Neural Network is a computational model inspired by the way biological neural networks in the human brain process information. It consists of interconnected groups of artificial neurons, organized in layers, that work together to recognize patterns and solve complex problems. Each neuron receives input, processes it, and passes on the output to the next layer. Neural networks are a fundamental component of Deep Learning, a subset of Machine Learning, and are used extensively in tasks such as image and speech recognition, Natural Language Processing (NLP), and generative modeling. They are characterized by their ability to learn from data through a process called training, where the network adjusts its weights based on the error of its predictions to improve accuracy over time. An artificial Neural Network is a computational model inspired by the way biological neural networks in the human brain process information. It consists of interconnected groups of artificial neurons, organized in layers, that work together to recognize patterns and solve complex problems. Each neuron receives input, processes it, and passes on the output to the next layer. Neural networks are a fundamental component of [Deep Learning](https://promptmetheus.com/resources/llm-knowledge-base/deep-learning), a subset of [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml), and are used extensively in tasks such as image and speech recognition, [Natural Language Processing (NLP)](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp), and generative modeling. They are characterized by their ability to learn from data through a process called training, where the network adjusts its weights based on the error of its predictions to improve accuracy over time. --- - **Page:** One-shot Prompt - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/one-shot-prompt - **Description:** One-shot prompt refers to a single input or instruction given to an AI model, which then generates a comprehensive output based on that single prompt. The term "one-shot" signifies that the AI model doesn't require multiple examples or extensive training to produce the desired output, but instead, it can understand and respond to the task effectively with just one prompt. This concept is particularly relevant in language models where a one-shot prompt can generate a complete text or answer a question. One-shot prompt refers to a single input or instruction given to an AI model, which then generates a comprehensive output based on that single prompt. The term "one-shot" signifies that the AI model doesn't require multiple examples or extensive training to produce the desired output, but instead, it can understand and respond to the task effectively with just one prompt. This concept is particularly relevant in language models where a one-shot prompt can generate a complete text or answer a question. See also ["Zero-shot Prompt"](https://promptmetheus.com/resources/llm-knowledge-base/zero-shot-prompt) and ["Few-shot Prompt"](https://promptmetheus.com/resources/llm-knowledge-base/few-shot-prompt). is specifically designed to forge these highly effective one-shot prompts. --- - **Page:** Open-source Model - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/open-source-model - **Description:** An open-source Large Language Model (LLM) is a publicly available AI model that has been trained on vast amounts of data to understand and generate text, images, videos, audio, etc. These models are freely accessible, allowing developers and researchers to use and modify them, and to contribute to their development. An open-source [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) is a publicly available AI model that has been trained on vast amounts of data to understand and generate text, images, videos, audio, etc. These models are freely accessible, allowing developers and researchers to use and modify them, and to contribute to their development. In contrast to [Open-weights Models](https://promptmetheus.com/resources/llm-knowledge-base/open-weights-model), which only make their weights available, open-source LLMs provide the full model architecture, training data, and codebase, enabling others to replicate, improve, and build upon the model. Examples of open-source LLMs include [MPT-30B](https://www.databricks.com/blog/mpt-30b) by [Databricks](https://www.databricks.com/) (Mosaic ML), the [Falcon](https://falconllm.tii.ae) series from the [Technology Innovation Institute](https://www.tii.ae/) (1.3B, 7.5B, 40B, and 180B), and [GPT-J](https://www.eleuther.ai/artifacts/gpt-j) (6B) by [EleutherAI](https://www.eleuther.ai/). --- - **Page:** Open-weights Model - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/open-weights-model - **Description:** An open-weights model is a Large Language Model (LLM) whose parameters (or "weights") are publicly accessible and can be used and modified without restriction. An open-weights model is a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) whose parameters (or "weights") are publicly accessible and can be used and modified without restriction. Unlike closed or [Proprietary Models](https://promptmetheus.com/resources/llm-knowledge-base/proprietary-model), open-weights models are often shared within the AI community for research, educational purposes, or to foster innovation. They can be [fine-tuned](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) or adapted to specific tasks and contribute to the transparency and collaborative advancement of AI technology. Open-weights models can also facilitate reproducibility in AI research, allowing others to validate and build upon existing work. Open weights also allow independent deployment optimizations such as [quantization](https://promptmetheus.com/resources/llm-knowledge-base/quantization), which can reduce memory requirements for local or lower-cost inference. In contrast to [Open-source Models](https://promptmetheus.com/resources/llm-knowledge-base/open-source-model), open-weights models do not provide access to the model architecture or source code for the training pipeline. In an [interview with Sequoia Capital](https://www.youtube.com/watch?v=c3b-JASoPi0), [Andrej Karpathy](https://x.com/karpathy) used a nice analogy to software, saying "an open-weights model is a little bit like tossing over a binary for an operating system." Examples of open-weights models are the [LLaMA series](https://developer.meta.com/ai/) by [Meta AI](https://ai.meta.com) and [Mistral 7B](https://mistral.ai/news/announcing-mistral-7b/) / [Mixtral 8x7B](https://mistral.ai/news/mixtral-of-experts/) by [Mistral AI](https://mistral.ai). --- - **Page:** Output Priming - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/output-priming - **Description:** Output priming is a prompting technique where the beginning of the model's response is supplied in advance so the model continues in the desired format, tone, or structure. It is also called assistant prefilling, response priming, or response prefixing. Output priming is a prompting technique where the beginning of the model's response is supplied in advance so the model continues in the desired format, tone, or structure. It is also called assistant prefilling, response priming, or response prefixing. For example, an application that needs JSON may provide a partial [assistant message](https://promptmetheus.com/resources/llm-knowledge-base/assistant-message) containing `{` after the [user message](https://promptmetheus.com/resources/llm-knowledge-base/user-message). The model then continues from that opening token rather than starting with a conversational preamble. Output priming is useful for: - structured output such as JSON or XML; - skipping phrases like "Sure, here is..."; - maintaining a fixed voice or style; - starting a response at a required section; and - reducing brittle post-processing. It is not a substitute for a clear task description, schema validation, or [structured output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output). Some models and APIs do not support assistant prefilling, and reasoning-oriented modes may restrict or change the behavior. Anthropic describes prefilling as a way to start the model's response in its [prompt engineering best practices](https://claude.com/blog/best-practices-for-prompt-engineering) guide. --- - **Page:** Output Token - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/output-token - **Description:** Output tokens refer to the individual units of information produced by an AI model after processing the input data. They collectively form the generated text, code, etc., or what's known as the completion. Output tokens refer to the individual units of information produced by an AI model after processing the input data. They collectively form the generated text, code, etc., or what's known as the completion. Output tokens are subject to the total (input + output) or specific (output only) [token limit](https://promptmetheus.com/resources/llm-knowledge-base/token-limit). --- - **Page:** Perceptron - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/perceptron - **Description:** A perceptron is a fundamental building block of artificial neural networks, designed to simulate a single neuron in the human brain. It is a simple algorithm that takes multiple input signals, applies weights to each input, and then passes the weighted sum through an activation function to produce a binary output (0 or 1). Perceptrons are used for binary classification tasks and form the basis for more complex neural network architectures in the field of Generative AI and Machine Learning. A perceptron is a fundamental building block of artificial neural networks, designed to simulate a single neuron in the human brain. It is a simple algorithm that takes multiple input signals, applies weights to each input, and then passes the weighted sum through an activation function to produce a binary output (0 or 1). Perceptrons are used for binary classification tasks and form the basis for more complex neural network architectures in the field of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) and [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml). --- - **Page:** Playground - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/playground - **Description:** In the context of Generative AI, "Playground" refers to an interactive online platform or environment where users can experiment, learn, and test AI models. It allows users to manipulate parameters, input data, and observe the output generated by the LLM, thereby providing a hands-on experience of how the AI system works. This tool is particularly useful for understanding the capabilities and limitations of AI models, and for fine-tuning them to achieve desired results. In the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), "Playground" refers to an interactive online platform or environment where users can experiment, learn, and test AI models. It allows users to manipulate parameters, input data, and observe the output generated by the [LLM](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), thereby providing a hands-on experience of how the AI system works. This tool is particularly useful for understanding the capabilities and limitations of AI models, and for [fine-tuning](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) them to achieve desired results. --- - **Page:** Presence Penalty - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/presence-penalty - **Description:** Presence Penalty is a parameter used in Generative AI models to control the repetition of certain phrases or words in the generated text. A higher presence penalty discourages the model from using the same phrases or words frequently, thereby promoting diversity and novelty in the output. This parameter helps with fine-tuning the model's output to meet specific requirements and improve the overall quality of the generated content. Presence Penalty is a parameter used in [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) models to control the repetition of certain phrases or words in the generated text. A higher presence penalty discourages the model from using the same phrases or words frequently, thereby promoting diversity and novelty in the output. This parameter helps with [fine-tuning](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) the model's output to meet specific requirements and improve the overall quality of the generated content. --- - **Page:** Proactive AI - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/proactive-ai - **Description:** Proactive AI is an AI system design pattern where the system initiates useful actions based on goals, schedules, events, or detected signals instead of waiting for a user prompt. It is common in agentic products that monitor work and surface recommendations, alerts, drafts, or next steps. Proactive AI is an AI system design pattern where the system initiates useful actions based on goals, schedules, events, or detected signals instead of waiting for a user prompt. It is common in agentic products that monitor work and surface recommendations, alerts, drafts, or next steps. Examples include: - preparing a meeting brief before a calendar event; - detecting a stalled support ticket and drafting a follow-up; - warning a sales team about churn risk; - summarizing overnight incidents for an operations team; and - launching an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) when a metric crosses a threshold. Proactive AI is not the same as unbounded autonomy. A reliable system needs explicit triggers, permission boundaries, user controls, audit logs, and escalation rules. It should distinguish between suggesting an action, preparing an artifact, and executing a consequential operation. Technically, proactive AI often depends on [event-driven AI](https://promptmetheus.com/resources/llm-knowledge-base/event-driven-ai), background retrieval, task queues, [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory), and evaluation of false positives. Slack describes proactive agents as systems that can act before a human prompt in [Proactive AI Agents: Definition, Core Components, and Business Value](https://slack.com/blog/productivity/proactive-ai-agents-definition-core-components-and-business-value). --- - **Page:** Prompt - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/prompt - **Description:** A prompt is an initial input or instruction given to an AI model, which guides the AI in generating subsequent content or responses. It can be a single word or up to multiple paragraphs that set the direction for the AI's output. Prompts are crucial in tasks such as text generation, language translation, and conversation simulation, as they provide a starting point for the AI's creative process. A prompt is an initial input or instruction given to an AI model, which guides the AI in generating subsequent content or responses. It can be a single word or up to multiple paragraphs that set the direction for the AI's output. Prompts are crucial in tasks such as text generation, language translation, and conversation simulation, as they provide a starting point for the AI's creative process. --- - **Page:** Prompt Caching - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/prompt-caching - **Description:** Prompt caching is an inference optimization that reuses computation for a repeated prefix of model input. When multiple requests begin with the same system instructions, examples, documents, or tool definitions, the provider can cache the intermediate attention state for that prefix instead of recomputing it from the first token each time. Prompt caching is an inference optimization that reuses computation for a repeated prefix of model input. When multiple requests begin with the same system instructions, examples, documents, or tool definitions, the provider can cache the intermediate attention state for that prefix instead of recomputing it from the first [token](https://promptmetheus.com/resources/llm-knowledge-base/token) each time. Caching can reduce latency and input-processing cost for applications with long, stable prompts. Typical candidates include: - a large [system message](https://promptmetheus.com/resources/llm-knowledge-base/system-message); - shared reference documents; - extensive [few-shot examples](https://promptmetheus.com/resources/llm-knowledge-base/few-shot-prompt); - tool schemas used by an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent); and - a long conversation history reused across successive turns. Prompt caching normally requires exact or near-exact prefix matching. Stable content should therefore appear before request-specific data. Reordering tools, changing whitespace, inserting timestamps, or modifying early instructions may invalidate the cached prefix even if most content is unchanged. Prompt caching does not increase the [context window](https://promptmetheus.com/resources/llm-knowledge-base/context-window) or improve answer quality directly. The model still receives the same input; the optimization changes how efficiently that input is processed. It also differs from [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory), which stores information for later retrieval rather than cached model computation. Providers may use different cache lifetimes, minimum prefix lengths, write costs, and accounting rules. Applications should monitor cache-hit rates and effective cost rather than assuming that a long static prompt is being reused. Prompt caching is one lever in [token economics](https://promptmetheus.com/resources/llm-knowledge-base/token-economics). It can lower the effective price of repeated input tokens, but it does not reduce output-token cost or eliminate the need to design compact, relevant context. Sensitive data remains subject to the provider's storage and retention controls. Developers should verify how cached prefixes are isolated between organizations and whether caching is compatible with their data-governance requirements. See the [Anthropic prompt caching documentation](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) for an implementation based on explicit cache breakpoints. --- - **Page:** Prompt Chaining - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/prompt-chaining - **Description:** Prompt chaining is the process of using the output from one Large Language Model (LLM) call as input to another call in a sequence. It decomposes a task into smaller stages, allowing each prompt to focus on a narrower operation. Prompt chaining is the process of using the output from one [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) call as input to another call in a sequence. It decomposes a task into smaller stages, allowing each [prompt](https://promptmetheus.com/resources/llm-knowledge-base/prompt) to focus on a narrower operation. For example, one call may extract facts from a document, a second may organize them into an outline, and a third may draft a report. Different models can be used for different stages, and application code can validate intermediate results before continuing. Prompt chaining is a common [agentic workflow](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow) pattern, but a fixed chain is not necessarily an autonomous [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent). The sequence is usually predefined rather than selected dynamically by the model. Long chains increase latency and create opportunities for errors to propagate. [Structured output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output), explicit validation, retries, and [evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) help make the boundaries between stages more reliable. --- - **Page:** Prompt IDE - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/prompt-ide - **Description:** A Prompt IDE (Integrated Development Environment) is a specialized software tool designed for developers and researchers working with Generative AI models, particularly those that utilize Natural Language Processing. It provides a user-friendly interface for crafting, refining, and testing prompts that are used to elicit specific responses or outputs from AI models like GPT or Claude. A Prompt IDE typically includes features such as variables, testing environments, version control, and performance analytics to help users optimize their prompts for accuracy and creativity, thereby enhancing the interaction with AI systems. A Prompt IDE is essential for anyone looking to streamline the development process of AI-driven applications, ensuring that the prompts used are effective and produce the desired results. A Prompt IDE (Integrated Development Environment) is a specialized software tool designed for developers and researchers working with [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) models, particularly those that utilize [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp). It provides a user-friendly interface for crafting, refining, and testing prompts that are used to elicit specific responses or outputs from AI models like GPT or Claude. A Prompt IDE typically includes features such as variables, testing environments, version control, and performance analytics to help users optimize their prompts for accuracy and creativity, thereby enhancing the interaction with AI systems. A Prompt IDE is essential for anyone looking to streamline the development process of AI-driven applications, ensuring that the prompts used are effective and produce the desired results. Take a look at the ["Building a Prompt Engineering IDE (VS Code for AI)"](https://promptmetheus.com/blog/prompt-engineering-ide) post for a more details and sign up for to give it a spin. --- - **Page:** Prompt Injection Attack - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack - **Description:** A Prompt Injection Attack is a type of cybersecurity threat specific to systems utilizing Generative AI, particularly those that generate content based on user inputs, such as chatbots or AI writing assistants. In this attack, a malicious user crafts input prompts in a way that manipulates the AI into generating responses that include sensitive data, unintended actions, or biased content. This can compromise the integrity of the AI system, lead to data breaches, or cause the AI to behave in undesirable ways. Prompt Injection Attacks exploit the vulnerabilities in the AI's language understanding or processing capabilities to deceive the system into deviating from its intended function. Protecting against such attacks involves implementing robust input validation, monitoring, and AI training to recognize and resist malicious inputs. A Prompt Injection Attack is a type of cybersecurity threat specific to systems utilizing [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), particularly those that generate content based on user inputs, such as chatbots or AI writing assistants. In this attack, a malicious user crafts input prompts in a way that manipulates the AI into generating responses that include sensitive data, unintended actions, or biased content. This can compromise the integrity of the AI system, lead to data breaches, or cause the AI to behave in undesirable ways. Prompt Injection Attacks exploit the vulnerabilities in the AI's language understanding or processing capabilities to deceive the system into deviating from its intended function. Protecting against such attacks involves implementing robust input validation, monitoring, and AI training to recognize and resist malicious inputs. --- - **Page:** Prompt Optimization - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/prompt-optimization - **Description:** Prompt Optimization is the process of refining and improving the text prompts used to instruct Generative AI models, such as language models or image generation systems. The goal is to craft prompts that elicit higher-quality, more relevant, and consistent outputs aligned with the desired objectives. Effective prompt optimization involves techniques like providing clear instructions, using relevant keywords, specifying desired formats or styles, and iteratively testing and refining prompts (Prompt Engineering) based on the generated results. By optimizing prompts, users can better leverage the capabilities of generative AI to produce content, images, or other outputs that meet their specific needs and requirements. Prompt Optimization is the process of refining and improving the text prompts used to instruct [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) models, such as language models or image generation systems. The goal is to craft prompts that elicit higher-quality, more relevant, and consistent outputs aligned with the desired objectives. Effective prompt optimization involves techniques like providing clear instructions, using relevant keywords, specifying desired formats or styles, and iteratively testing and refining prompts ([Prompt Engineering](https://promptmetheus.com/resources/llm-knowledge-base/prompt-engineering)) based on the generated results. By optimizing prompts, users can better leverage the capabilities of generative AI to produce content, images, or other outputs that meet their specific needs and requirements. --- - **Page:** Proprietary Model - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/proprietary-model - **Description:** A proprietary Large Language Model (LLM) is an AI model developed and owned by a specific company, using their private training data and resources. Unlike Open-source Models and Open-weights Models, proprietary LLMs are not publicly available and can only be accessed through the company's LLM API or platform. A proprietary [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) is an AI model developed and owned by a specific company, using their private training data and resources. Unlike [Open-source Models](https://promptmetheus.com/resources/llm-knowledge-base/open-source-model) and [Open-weights Models](https://promptmetheus.com/resources/llm-knowledge-base/open-weights-model), proprietary LLMs are not publicly available and can only be accessed through the company's [LLM API](https://promptmetheus.com/resources/llm-knowledge-base/llm-api) or platform. Proprietary models are often [fine-tuned](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) for specific use cases and industries, offering unique features and capabilities tailored to the company's target audience. Companies with proprietary LLMs maintain control over the model's usage, pricing, and intellectual property. --- - **Page:** Proximal Policy Optimization (PPO) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/proximal-policy-optimization-ppo - **Description:** Proximal Policy Optimization (PPO) is an advanced reinforcement learning algorithm that falls under the category of policy gradient methods. Developed by OpenAI, PPO is designed to improve the stability and reliability of training complex models by optimizing a policy in a way that ensures updates are not too large, thus maintaining a balance between exploration and exploitation. It achieves this by using a surrogate objective function that penalizes deviations from the previous policy beyond a certain threshold, effectively constraining the policy update to be within a "proximal" range. PPO is widely used in training large-scale models due to its simplicity, efficiency, and ability to handle high-dimensional action spaces, making it a popular choice in the field of Generative AI and robotics. Proximal Policy Optimization (PPO) is an advanced reinforcement learning algorithm that falls under the category of policy gradient methods. Developed by OpenAI, PPO is designed to improve the stability and reliability of training complex models by optimizing a policy in a way that ensures updates are not too large, thus maintaining a balance between exploration and exploitation. It achieves this by using a surrogate objective function that penalizes deviations from the previous policy beyond a certain threshold, effectively constraining the policy update to be within a "proximal" range. PPO is widely used in training large-scale models due to its simplicity, efficiency, and ability to handle high-dimensional action spaces, making it a popular choice in the field of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) and robotics. --- - **Page:** Quantization - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/quantization - **Description:** Quantization is a model-compression technique that represents weights, activations, or computation with lower-precision numeric formats. For example, model weights stored as 16-bit floating-point values may be converted to 8-bit or 4-bit integers. Quantization is a model-compression technique that represents weights, activations, or computation with lower-precision numeric formats. For example, model weights stored as 16-bit floating-point values may be converted to 8-bit or 4-bit integers. Lower precision reduces memory requirements and memory bandwidth, which can make [inference](https://promptmetheus.com/resources/llm-knowledge-base/inference) faster and allow larger models to run on less expensive hardware. The actual speed improvement depends on whether the deployment hardware and runtime have optimized kernels for the chosen format. Two broad approaches are: - **Post-Training Quantization (PTQ):** quantize an already trained model, often using a calibration dataset. - **Quantization-Aware Training (QAT):** simulate lower precision during training or [fine-tuning](https://promptmetheus.com/resources/llm-knowledge-base/fine-tuning) so the model adapts to quantization error. Methods also differ in whether they quantize weights only or both weights and activations, and whether scaling parameters are applied per tensor, channel, or smaller group of values. More granular scaling usually preserves accuracy better but adds metadata and implementation complexity. Quantization is not the same as [model distillation](https://promptmetheus.com/resources/llm-knowledge-base/model-distillation). Quantization changes numeric precision, while distillation trains a student model to imitate a teacher. The methods can be combined. Aggressive quantization may degrade perplexity, factual accuracy, reasoning, tool use, or performance on rare inputs. Aggregate benchmarks can hide regressions in specific languages or domains, so the quantized artifact should be evaluated directly on its intended workload. Quantization is particularly important for [open-weights models](https://promptmetheus.com/resources/llm-knowledge-base/open-weights-model) and local [Small Language Models (SLMs)](https://promptmetheus.com/resources/llm-knowledge-base/small-language-model-slm). The [GPTQ paper](https://arxiv.org/abs/2210.17323) introduced an influential post-training method for low-bit transformer weights. --- - **Page:** RL Environment - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/rl-environment - **Description:** An RL Environment is the interactive system in which an agent takes actions and receives observations and rewards during reinforcement learning. It defines the task dynamics that a policy learns from. An RL Environment is the interactive system in which an agent takes actions and receives observations and rewards during reinforcement learning. It defines the task dynamics that a policy learns from. In classic reinforcement learning, the environment may be a game, simulator, robot control system, or physical process. In LLM agent training, the environment is often a software workspace, browser, codebase, database, document collection, API, or synthetic task harness. An RL Environment typically provides: - an initial state; - observations visible to the agent; - allowed actions; - state-transition rules; - terminal conditions; - a [reward function](https://promptmetheus.com/resources/llm-knowledge-base/reward-function); and - reset behavior for repeated training episodes. For a coding agent, the environment may include a repository, failing tests, shell commands, and a grading script. For a web-navigation agent, it may include a browser state and task-specific success checks. For a math model, the environment may be simpler: the action is a final answer and the reward is whether it is correct. The RL Environment is distinct from an [agent sandbox](https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox). A sandbox is an execution boundary used at runtime or training time. An RL Environment additionally defines observations, actions, rewards, and episode lifecycle for learning or evaluation. Environment quality determines what the model can learn. If tasks are unrealistic, observations leak answers, or rewards miss important constraints, training may produce behavior that scores well in the environment but fails in production. An RL Environment is central to [Reinforcement Learning with Verifiable Rewards (RLVR)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning-with-verifiable-rewards-rlvr) and algorithms such as [Group Relative Policy Optimization (GRPO)](https://promptmetheus.com/resources/llm-knowledge-base/group-relative-policy-optimization-grpo). Hugging Face's [agent glossary](https://huggingface.co/blog/agent-glossary) frames an RL Environment as the interactive stateful system that receives actions and returns observations during agent training. --- - **Page:** Real-time Fact-Checking - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/real-time-fact-checking - **Description:** Real-time fact-checking is the use of live or recently updated sources to verify claims while an AI system is producing or preparing an answer. It is used to reduce hallucinations, stale information, and unsupported assertions in systems that answer questions about current or high-change topics. Real-time fact-checking is the use of live or recently updated sources to verify claims while an AI system is producing or preparing an answer. It is used to reduce hallucinations, stale information, and unsupported assertions in systems that answer questions about current or high-change topics. In LLM applications, real-time fact-checking typically combines: - web search or connected data retrieval; - source ranking and citation selection; - claim extraction from a draft answer; - comparison between claims and retrieved evidence; - citation display or provenance metadata; and - refusal or uncertainty when evidence is missing. It is related to [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag), but not all RAG is real-time. A RAG system may retrieve from a static internal index. Real-time fact-checking emphasizes freshness and source verification during the request. The technique does not guarantee truth. Search results may be low quality, sources may conflict, and the model may still overstate what the evidence supports. Professional systems need citation precision, freshness checks, domain whitelists, and human review for high-stakes use. Perplexity describes its product as searching the web in real time and returning cited answers in [What is Perplexity?](https://www.perplexity.ai/help-center/en/articles/10352155-what-is-perplexity). --- - **Page:** Reasoning Mode - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/reasoning-mode - **Description:** Reasoning mode is a model or API setting that changes how much internal work a model performs before returning a final answer. It is used with reasoning models and tasks that benefit from planning, checking, or multi-step problem solving. Reasoning mode is a model or API setting that changes how much internal work a model performs before returning a final answer. It is used with [reasoning models](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model) and tasks that benefit from planning, checking, or multi-step problem solving. Reasoning mode is related to, but different from, reasoning effort. Effort usually controls the amount of compute or tokens allocated to reasoning. Mode can change the execution behavior itself, such as enabling a quality-first path that performs more internal work before producing one final response. Use reasoning mode when: - the task is difficult enough to benefit from extra inference; - a marginal quality gain is worth additional latency and cost; - the output can be checked against clear criteria; and - the application can tolerate less predictable response time. Avoid it for routine classification, simple extraction, autocomplete, or high-volume workloads where low latency matters more than difficult reasoning. The right setting should be chosen through [evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation), not by assuming that more reasoning is always better. OpenAI describes reasoning mode and reasoning effort as separate controls in its [model guidance](https://developers.openai.com/api/docs/guides/latest-model) documentation. --- - **Page:** Reinforcement Learning from Human Feedback (RLHF) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning - **Description:** Reinforcement Learning from Human Feedback (RLHF) is a post-training method that uses human preferences to improve a model's behavior. It is commonly applied when desired qualities such as helpfulness, relevance, or safety are difficult to specify with a deterministic reward function. Reinforcement Learning from Human Feedback (RLHF) is a post-training method that uses human preferences to improve a model's behavior. It is commonly applied when desired qualities such as helpfulness, relevance, or safety are difficult to specify with a deterministic [reward function](https://promptmetheus.com/resources/llm-knowledge-base/reward-function). A typical RLHF pipeline includes: 1. collecting human comparisons or ratings of model outputs; 2. training a reward model to predict those preferences; and 3. optimizing the language model against that reward while limiting divergence from a reference policy. RLHF is used in [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) to make model behavior better match intended user and developer preferences. The result depends on the quality, diversity, and instructions given to human annotators. Preference data can encode inconsistent standards or demographic and cultural biases. RLHF is distinct from [Reinforcement Learning with Verifiable Rewards (RLVR)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning-with-verifiable-rewards-rlvr), which computes rewards with an automatic verifier for tasks such as mathematics or code. Policy-optimization algorithms used in language-model training include [Proximal Policy Optimization (PPO)](https://promptmetheus.com/resources/llm-knowledge-base/proximal-policy-optimization-ppo) and [Group Relative Policy Optimization (GRPO)](https://promptmetheus.com/resources/llm-knowledge-base/group-relative-policy-optimization-grpo). Optimizing a learned reward does not guarantee the intended behavior. A policy can exploit reward-model weaknesses, become excessively cautious, or sacrifice response diversity. Independent [evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation) remains necessary after training. --- - **Page:** Reasoning Model - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model - **Description:** A reasoning model is a Large Language Model (LLM) optimized to perform additional inference before producing a final answer. It is designed for tasks that benefit from planning, decomposition, verification, or multi-step problem solving, such as mathematics, software engineering, scientific analysis, and complex tool calling. A reasoning model is a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) optimized to perform additional inference before producing a final answer. It is designed for tasks that benefit from planning, decomposition, verification, or multi-step problem solving, such as mathematics, software engineering, scientific analysis, and complex [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling). The defining characteristic is not that the model emits a visible explanation. Rather, the system allocates computational work to intermediate reasoning before generating the user-facing output. Some APIs represent this through [reasoning mode](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-mode), a reasoning-effort setting, [thinking mode](https://promptmetheus.com/resources/llm-knowledge-base/thinking-mode), or separate [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token) for internal reasoning. This additional work is one form of [inference-time scaling](https://promptmetheus.com/resources/llm-knowledge-base/inference-time-scaling-test-time-compute). Training methods such as [Reinforcement Learning with Verifiable Rewards (RLVR)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning-with-verifiable-rewards-rlvr) can encourage reasoning strategies whose outcomes are checked automatically. Reasoning models are related to [chain-of-thought prompting](https://promptmetheus.com/resources/llm-knowledge-base/chain-of-thought-prompting), but the terms are not interchangeable. Chain-of-thought prompting is an input technique intended to elicit intermediate steps. A reasoning model is trained or otherwise optimized to reason during [inference](https://promptmetheus.com/resources/llm-knowledge-base/inference), and its internal reasoning may not be exposed. Compared with a low-latency general-purpose model, a reasoning model commonly trades: - more inference time and compute for higher accuracy on difficult tasks; - concise final answers for additional hidden deliberation; - predictable single-pass behavior for iterative planning and verification; and - lower per-request cost for improved performance on problems where naive generation fails. Reasoning models are particularly useful inside an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent), where the model must decide which action to take, interpret tool results, recover from errors, and maintain progress over many steps. However, more reasoning does not guarantee correctness. Models can still make false assumptions, misuse tools, or produce confident [hallucinations](https://promptmetheus.com/resources/llm-knowledge-base/hallucination). Evaluation should therefore focus on externally verifiable outcomes rather than the plausibility of an explanation. Suitable methods include deterministic checks, execution-based tests, held-out task suites, and [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation) for long-horizon behavior. The [OpenAI reasoning model guide](https://developers.openai.com/api/docs/guides/reasoning) documents practical differences between reasoning-oriented and conventional model usage. --- - **Page:** Reinforcement Learning with Verifiable Rewards (RLVR) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning-with-verifiable-rewards-rlvr - **Description:** Reinforcement Learning with Verifiable Rewards (RLVR) is a post-training method that improves a language model using rewards computed by an automatic verifier. The verifier acts as a reward function by checking whether a generated answer or action satisfies an objective criterion, allowing training to proceed without a human preference label for every sample. Reinforcement Learning with Verifiable Rewards (RLVR) is a post-training method that improves a language model using rewards computed by an automatic verifier. The verifier acts as a [reward function](https://promptmetheus.com/resources/llm-knowledge-base/reward-function) by checking whether a generated answer or action satisfies an objective criterion, allowing training to proceed without a human preference label for every sample. RLVR is well suited to domains where outcomes can be checked programmatically, including: - mathematics with known or symbolically equivalent answers; - code evaluated by tests or execution; - formal proofs checked by a proof assistant; - [structured outputs](https://promptmetheus.com/resources/llm-knowledge-base/structured-output) validated against schemas and business rules; and - tasks with measurable simulator or game outcomes. The training loop samples one or more responses from the current policy, assigns rewards with the verifier, and updates the model to increase the probability of higher-reward behavior. Algorithms such as [Group Relative Policy Optimization (GRPO)](https://promptmetheus.com/resources/llm-knowledge-base/group-relative-policy-optimization-grpo) can perform these updates without a separate learned value model. RLVR differs from [Reinforcement Learning from Human Feedback (RLHF)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning). RLHF commonly relies on human preferences or a reward model trained from them. RLVR uses an externally checkable outcome, which can be cheaper, less subjective, and harder to miscalibrate. The verifier defines what the model is rewarded for and is therefore part of the specification. Weak verifiers can create reward hacking: a response satisfies the checker while violating the intended task. Passing a unit test, for example, does not prove maintainability, security, or broader correctness. RLVR has become important in training [reasoning models](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model), where verifiable final answers can reinforce useful strategies such as checking work and changing approach. The [DeepSeek-R1 paper](https://arxiv.org/abs/2501.12948) reports the emergence of reasoning behaviors through large-scale reinforcement learning on verifiable tasks. For interactive or tool-using tasks, the verifier often operates over an [RL environment](https://promptmetheus.com/resources/llm-knowledge-base/rl-environment) rather than a single text answer. In those settings, reward design must account for intermediate actions, side effects, and termination conditions. --- - **Page:** Reranking - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/reranking - **Description:** Reranking, sometimes shortened to rerank, is a second-stage retrieval process that reorders an initial set of search results using a more accurate but more computationally expensive relevance model. It improves which documents are ultimately supplied to a user or a Large Language Model (LLM). Reranking, sometimes shortened to rerank, is a second-stage retrieval process that reorders an initial set of search results using a more accurate but more computationally expensive relevance model. It improves which documents are ultimately supplied to a user or a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). The first-stage retriever prioritizes recall and speed. It may use keyword search, [vector search](https://promptmetheus.com/resources/llm-knowledge-base/vector-search), or [hybrid search](https://promptmetheus.com/resources/llm-knowledge-base/hybrid-search) to select tens or hundreds of candidates from a large corpus. The reranker then evaluates those candidates more deeply and returns a smaller ordered list. Common reranking models include: - cross-encoders that jointly process the query and each document; - late-interaction models that compare token-level representations; - [LLMs](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) prompted to score or compare passages; and - domain-specific learned ranking functions. Cross-encoders are often more precise than embedding similarity because they model direct interactions between query and document tokens. Their cost scales with the number and length of candidates, so candidate limits and truncation policies matter. In [Retrieval-Augmented Generation (RAG)](https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag), reranking can reduce irrelevant context and improve evidence ordering. It does not verify that a document is factually correct, trustworthy, current, or safe to follow. Reranking quality should be measured with retrieval metrics such as nDCG, Mean Reciprocal Rank, recall at K, and downstream answer accuracy. Offline relevance gains should be checked against added latency and cost. Google Cloud's [Ranking API documentation](https://docs.cloud.google.com/generative-ai-app-builder/docs/ranking) describes semantic reranking of previously retrieved documents. --- - **Page:** Resumable Agent Run - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/resumable-agent-run - **Description:** A resumable agent run is an AI agent execution that can be paused, serialized, and continued later without starting over. It preserves enough state to recover the active agent, conversation items, tool results, approvals, usage, and any workspace state needed for the task. A resumable agent run is an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) execution that can be paused, serialized, and continued later without starting over. It preserves enough state to recover the active agent, conversation items, tool results, approvals, usage, and any workspace state needed for the task. The ability to resume matters for [long-horizon tasks](https://promptmetheus.com/resources/llm-knowledge-base/long-horizon-task). A task may be interrupted by human approval, tool latency, rate limits, budget caps, process restarts, deployments, or a user returning hours later. State can be stored at several levels: - conversation history or session items; - model-provider continuation identifiers; - current agent and turn count; - pending tool calls or approval requests; - serialized application context; - [agent sandbox](https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox) session state or snapshots; and - trace identifiers and usage records. A resumable run is different from [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory). Memory stores durable knowledge for future tasks. Resumable run state stores the operational state of a specific in-progress task. An [agent session](https://promptmetheus.com/resources/llm-knowledge-base/agent-session) can also preserve continuity across multiple completed turns, not just recover an interrupted one. The key engineering challenge is consistency. If a tool call changed external state before the run paused, resuming must not accidentally repeat the operation. Idempotency keys, durable action logs, and explicit completion markers are important. Serialization should be conservative. Custom objects, credentials, open file handles, and live network connections may not round-trip safely. The system should record enough metadata to rebuild the required context without storing secrets unnecessarily. Resumable runs also improve human-in-the-loop workflows. An agent can stop at an approval boundary, persist its state, and continue after the user or reviewer responds. The [OpenAI Agents SDK RunState reference](https://openai.github.io/openai-agents-python/ref/run_state/) describes a serializable snapshot used to continue interrupted runs, including model responses, generated items, approval state, and optional conversation identifiers. --- - **Page:** Retrieval-Augmented Generation (RAG) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/retrieval-augmented-generation-rag - **Description:** Retrieval-Augmented Generation is a methodology used in the field of Generative AI. It combines the benefits of both retrieval-based and generative systems for language processing. RAG utilizes an external knowledge source to retrieve relevant documents or information, and then uses a generative model to create a contextually appropriate response or output. This approach enhances the model's ability to generate detailed, accurate, and context-specific responses, making it particularly useful in applications such as chatbots, question-answering systems, and content creation tools. Retrieval-Augmented Generation is a methodology used in the field of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai). It combines the benefits of both retrieval-based and generative systems for language processing. RAG utilizes an external knowledge source to retrieve relevant documents or information, and then uses a generative model to create a contextually appropriate response or output. This approach enhances the model's ability to generate detailed, accurate, and context-specific responses, making it particularly useful in applications such as [chatbots](https://promptmetheus.com/resources/llm-knowledge-base/chatbot), question-answering systems, and content creation tools. RAG is also a core [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering) technique. Retrieval quality, chunking, metadata filters, [hybrid search](https://promptmetheus.com/resources/llm-knowledge-base/hybrid-search), [reranking](https://promptmetheus.com/resources/llm-knowledge-base/reranking), and the amount of evidence inserted into the prompt all affect whether the model receives useful context. When retrieval uses live web or database sources to verify claims during generation, it supports [real-time fact-checking](https://promptmetheus.com/resources/llm-knowledge-base/real-time-fact-checking). Protocols such as [NLWeb](https://promptmetheus.com/resources/llm-knowledge-base/nlweb) apply a similar idea at the website level by exposing structured natural-language query behavior over site content. In an agent system, [Agentic RAG](https://promptmetheus.com/resources/llm-knowledge-base/agentic-rag) may retrieve reference documents or [agent memory](https://promptmetheus.com/resources/llm-knowledge-base/agent-memory) dynamically as the task evolves. Retrieved content should be treated as untrusted data because it may contain inaccurate information or indirect [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). --- - **Page:** Reward Function - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/reward-function - **Description:** A reward function is the scoring rule that tells a reinforcement-learning system how good an action, output, or completed episode is. In language-model training, it converts generated behavior into a reward signal used to update an agent policy or model. A reward function is the scoring rule that tells a reinforcement-learning system how good an action, output, or completed episode is. In language-model training, it converts generated behavior into a reward signal used to update an [agent policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy) or model. Reward functions can be: - **verifiable**, such as tests passing, an answer matching ground truth, or a proof checker accepting a result; - **learned**, such as a reward model trained from human preferences; - **rubric-based**, where several criteria are scored separately and combined; - **sparse**, with only a final task reward; or - **dense**, with intermediate rewards at multiple steps. The reward function is central to [Reinforcement Learning from Human Feedback (RLHF)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning) and [Reinforcement Learning with Verifiable Rewards (RLVR)](https://promptmetheus.com/resources/llm-knowledge-base/reinforcement-learning-with-verifiable-rewards-rlvr). In RLHF, the reward is often predicted by a learned reward model. In RLVR, the reward is computed by an external verifier. A reward function is not the same as an [evaluation](https://promptmetheus.com/resources/llm-knowledge-base/evaluation). Evaluation measures system behavior, often for reporting or regression testing. A reward function is used as an optimization target during training or search. The same metric can sometimes serve both roles, but the incentives are different. The main risk is reward hacking. A model may find behavior that maximizes the reward while violating the intended task. For example, code may pass visible tests while being brittle, insecure, or incomplete. A helpful-looking answer may satisfy a weak judge while fabricating facts. Good reward design requires clear task specification, hidden test cases, adversarial checks, calibration against human judgment when relevant, and independent post-training evaluation. Hugging Face's [agent glossary](https://huggingface.co/blog/agent-glossary) describes reward as the score used by a trainer to update a model's weights during agent training. --- - **Page:** Role prompting - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/role-prompting - **Description:** Role prompting is a technique used in the domain of Generative Artificial Intelligence and Large Language Models (LLMs) to guide the model's output by assigning it a specific role or persona. This method involves crafting prompts that instruct the AI to adopt a particular perspective, style, or set of characteristics, thereby influencing the generated content to align with the desired context or tone. Role prompting is particularly useful in applications where the AI needs to simulate human-like interactions, such as customer service chatbots, creative writing, or educational tools. By defining a role, users can enhance the relevance and coherence of the AI's responses, ensuring that they meet specific user expectations or task requirements. Role prompting is a technique used in the domain of [Generative Artificial Intelligence](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) and [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) to guide the model's output by assigning it a specific role or persona. This method involves crafting [prompts](https://promptmetheus.com/resources/llm-knowledge-base/prompt) that instruct the AI to adopt a particular perspective, style, or set of characteristics, thereby influencing the generated content to align with the desired context or tone. Role prompting is particularly useful in applications where the AI needs to simulate human-like interactions, such as customer service chatbots, creative writing, or educational tools. By defining a role, users can enhance the relevance and coherence of the AI's responses, ensuring that they meet specific user expectations or task requirements. --- - **Page:** Slopsquatting - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/slopsquatting - **Description:** Slopsquatting is a software supply-chain attack that exploits package names hallucinated by AI coding assistants. An attacker registers a plausible but previously nonexistent package name, then waits for a developer or agent to install the package after an LLM suggests it. Slopsquatting is a software supply-chain attack that exploits package names hallucinated by AI coding assistants. An attacker registers a plausible but previously nonexistent package name, then waits for a developer or agent to install the package after an LLM suggests it. The attack is related to typosquatting, but the source of the name is different. Typosquatting targets human typing mistakes. Slopsquatting targets model-generated dependency suggestions that look legitimate but were never part of the real ecosystem. The risk increases when an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) can run package-manager commands, edit dependency manifests, or execute generated code with limited human review. It also affects ordinary copy-paste workflows from chat interfaces. Mitigations include: - verifying package existence, ownership, age, and download history before installation; - pinning dependencies from trusted registries; - blocking package installs in an [agent sandbox](https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox) unless approved; - using lockfiles and software composition analysis; - preferring official documentation over model-suggested package names; and - treating dependency additions as consequential actions in [agent policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy). Endor Labs describes slopsquatting as a supply-chain attack where AI agents hallucinate package names that attackers can register in [Slopsquatting: When AI Agents Hallucinate Malicious Packages](https://www.endorlabs.com/learn/slopsquatting-when-ai-agents-hallucinate-malicious-packages). --- - **Page:** Small Language Model (SLM) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/small-language-model-slm - **Description:** A Small Language Model (SLM) is a language model designed to provide useful capability with substantially fewer parameters and lower computational requirements than frontier Large Language Models (LLMs). There is no universal parameter threshold; the term is relative to the models and deployment hardware of a given period. A Small Language Model (SLM) is a language model designed to provide useful capability with substantially fewer parameters and lower computational requirements than frontier [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). There is no universal parameter threshold; the term is relative to the models and deployment hardware of a given period. SLMs are useful when an application prioritizes: - low inference latency; - reduced serving cost; - on-device or offline operation; - data locality and privacy; - predictable capacity; or - high request volume for a narrow task. Model size alone does not determine quality. Data curation, architecture, [tokenization](https://promptmetheus.com/resources/llm-knowledge-base/tokenization), training compute, and post-training can allow a smaller model to perform competitively on specific workloads. [Model distillation](https://promptmetheus.com/resources/llm-knowledge-base/model-distillation), [synthetic data](https://promptmetheus.com/resources/llm-knowledge-base/synthetic-data), and [quantization](https://promptmetheus.com/resources/llm-knowledge-base/quantization) are frequently used in SLM development and deployment. SLMs are often selected for classification, extraction, routing, summarization, embedded assistants, and constrained [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling). A larger [reasoning model](https://promptmetheus.com/resources/llm-knowledge-base/reasoning-model) may be used only when the smaller model is uncertain or the task exceeds its capability. The tradeoff is reduced breadth and robustness. An SLM may require more careful task definition, narrower context, domain-specific fine-tuning, and stronger fallback logic. Evaluation must use the actual deployment environment because memory, thermal limits, and hardware acceleration affect practical performance. The [Phi-3 technical report](https://arxiv.org/abs/2404.14219) describes a compact language model designed to run locally on consumer hardware while retaining strong benchmark performance. --- - **Page:** Sparse Expert Model - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/sparse-expert-model - **Description:** A sparse expert model is a neural network that contains many expert components but activates only a subset for a given input. In LLMs, the most common form is a sparsely activated Mixture of Experts (MoE). A sparse expert model is a neural network that contains many expert components but activates only a subset for a given input. In LLMs, the most common form is a sparsely activated [Mixture of Experts (MoE)](https://promptmetheus.com/resources/llm-knowledge-base/mixture-of-experts-moe). The key idea is to increase total model capacity without using every parameter on every token. A routing or gating mechanism selects which experts process each token or example. This can make a model much larger in total [model parameters](https://promptmetheus.com/resources/llm-knowledge-base/model-parameter) while keeping per-token compute closer to a smaller dense model. Sparse expert models can improve scaling efficiency, specialization, and throughput. They also introduce engineering challenges: - load balancing across experts; - routing instability during training; - communication overhead across devices; - expert collapse or unused experts; - more complex fine-tuning; and - harder debugging and observability. The word sparse refers to activation, not necessarily storage. The full model may contain many parameters, but only a sparse subset participates in each forward pass. DeepMind discusses sparse MoE architectures and their tradeoffs in [From Sparse to Soft Mixture of Experts](https://deepmind.google/research/publications/49566/). --- - **Page:** Structured Output - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/structured-output - **Description:** Structured output is model-generated data constrained to a predefined machine-readable format, commonly a JSON Schema. It allows an application to consume the response as typed data rather than parsing unrestricted natural language. Structured output is model-generated data constrained to a predefined machine-readable format, commonly a JSON Schema. It allows an application to consume the response as typed data rather than parsing unrestricted natural language. Typical uses include information extraction, classification, workflow routing, form completion, user-interface generation, and communication between components in an [agentic workflow](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow). A schema can define: - required and optional properties; - strings, numbers, booleans, arrays, and nested objects; - enumerated values; - restrictions on additional properties; and - descriptions that explain the semantic meaning of each field. Structured output is stronger than asking a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) to "return JSON." Plain prompting may produce malformed syntax, omitted keys, inconsistent types, or unsupported enum values. Schema-constrained generation is designed to ensure that the output conforms to the declared structure. Schema validity does not guarantee factual or semantic correctness. A model can produce a syntactically valid object containing an incorrect date, unsupported conclusion, or fabricated identifier. Applications must still validate business rules, permissions, references, and high-impact values. Structured output is related to [tool calling](https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling). Both commonly use schemas. Structured output describes the model's response contract, while a tool call requests that application code execute an external capability. Good schemas are narrow and explicit. Deeply nested structures, ambiguous field descriptions, and large enums can reduce reliability. Versioning is also important because changes to a schema are API contract changes for downstream consumers. Within [agent orchestration](https://promptmetheus.com/resources/llm-knowledge-base/agent-orchestration), structured outputs make routing, state updates, approvals, and handoffs easier to inspect and test. They also support more precise [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation) because individual fields can be checked independently. The [OpenAI Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs) provides an example of constraining responses with JSON Schema. --- - **Page:** Sub-agent - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/sub-agent - **Description:** A sub-agent is an AI agent invoked by another agent to handle a specific part of a task. It has its own model configuration, instructions, tools, memory scope, and execution loop, and returns a result to the calling agent or orchestrator. A sub-agent is an [AI agent](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) invoked by another agent to handle a specific part of a task. It has its own model configuration, instructions, tools, memory scope, and execution loop, and returns a result to the calling agent or orchestrator. A sub-agent is different from a tool. A tool performs a defined operation, such as searching a database or running a command. A sub-agent can reason, plan, call tools, ask for clarification, and perform several steps before producing its output. A sub-agent is also different from an [agent skill](https://promptmetheus.com/resources/llm-knowledge-base/agent-skill). A skill is packaged knowledge or procedure loaded into an agent. A sub-agent is an active runtime participant that can make decisions. Common uses include: - specialist review, such as security, legal, or architecture checks; - parallel research across independent topics; - isolated workspaces for competing implementations; - manager-worker [agent orchestration](https://promptmetheus.com/resources/llm-knowledge-base/agent-orchestration); - delegation from a general assistant to a domain-specific assistant; and - nested execution inside an [agent sandbox](https://promptmetheus.com/resources/llm-knowledge-base/agent-sandbox). Sub-agents can be connected through an [agent handoff](https://promptmetheus.com/resources/llm-knowledge-base/agent-handoff) or exposed as a callable tool. In a handoff, the receiving agent takes over the conversation or task flow. When exposed as a tool, the sub-agent runs as a nested operation and returns a result without taking over the outer conversation. The design tradeoff is specialization versus coordination overhead. Sub-agents can improve focus and parallelism, but they create extra latency, duplicated context, state synchronization problems, and more complex debugging. [Agent trace and observability](https://promptmetheus.com/resources/llm-knowledge-base/agent-trace-observability) are necessary to understand nested behavior. The OpenAI Agents SDK documents this distinction in its [SandboxAgent reference](https://openai.github.io/openai-agents-python/ref/sandbox/sandbox_agent/), where an agent can be transformed into a callable tool rather than receiving a handoff. --- - **Page:** Supervised Fine-Tuning (SFT) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/supervised-fine-tuning-sft - **Description:** Supervised Fine-Tuning (SFT) is a process in Machine Learning where a pre-trained model, typically a Large Language Model (LLM), is further trained on a specific dataset with labeled examples to improve its performance on a particular task. This process involves adjusting the model's parameters using supervised learning techniques, where the model learns to map input data to the correct output based on the provided labels. SFT is crucial in adapting general-purpose Foundation Models to specialized applications, enhancing their accuracy and relevance by leveraging domain-specific data. It is widely used in Natural Language Processing (NLP) tasks such as sentiment analysis, text classification, and question-answering systems, allowing models to deliver more precise and contextually appropriate responses. Supervised Fine-Tuning (SFT) is a process in [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) where a pre-trained model, typically a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), is further trained on a specific dataset with labeled examples to improve its performance on a particular task. This process involves adjusting the model's parameters using supervised learning techniques, where the model learns to map input data to the correct output based on the provided labels. SFT is crucial in adapting general-purpose [Foundation Models](https://promptmetheus.com/resources/llm-knowledge-base/foundation-model) to specialized applications, enhancing their accuracy and relevance by leveraging domain-specific data. It is widely used in [Natural Language Processing (NLP)](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) tasks such as sentiment analysis, text classification, and question-answering systems, allowing models to deliver more precise and contextually appropriate responses. --- - **Page:** Synthetic Data - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/synthetic-data - **Description:** Synthetic data is artificially generated data used in place of, or alongside, data collected from real-world events or human authors. In language-model development, it is commonly produced by another model, a simulator, a programmatic generator, or a combination of these methods. Synthetic data is artificially generated data used in place of, or alongside, data collected from real-world events or human authors. In language-model development, it is commonly produced by another model, a simulator, a programmatic generator, or a combination of these methods. Synthetic training data can contain instructions and responses, reasoning traces, preference pairs, tool-use trajectories, code, adversarial examples, or domain-specific documents. It is used for pretraining, [Supervised Fine-Tuning (SFT)](https://promptmetheus.com/resources/llm-knowledge-base/supervised-fine-tuning-sft), [model distillation](https://promptmetheus.com/resources/llm-knowledge-base/model-distillation), evaluation, and safety testing. A typical generation pipeline: 1. defines a target distribution of tasks or skills; 2. generates candidate examples from a teacher model or simulator; 3. validates them with rules, execution, models, or humans; 4. removes duplicates, contamination, and low-quality samples; and 5. measures whether training on the dataset improves held-out performance. Synthetic data can cover rare cases, reduce annotation cost, and create examples whose correctness is automatically verifiable. It also enables controlled variations that would be expensive or unsafe to collect in production. The main limitation is that generated data reflects the generator. It can amplify biases, stylistic regularities, factual errors, and gaps in the teacher's knowledge. Repeatedly training models on unfiltered model output may reduce diversity or reinforce artifacts. Synthetic examples should therefore be mixed with high-quality real data when appropriate and tested against independently created evaluations. Privacy is not guaranteed merely because data is generated. A model may reproduce memorized personal or copyrighted material, and prompts used to create the dataset may contain sensitive source data. The [Self-Instruct paper](https://arxiv.org/abs/2212.10560) demonstrated bootstrapping instruction-following data from model-generated tasks and responses with filtering. --- - **Page:** Temperature - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/temperature - **Description:** In the context of Generative AI, "Temperature" refers to a parameter in the probability distribution function used during the generation process. It controls the randomness of predictions by scaling the logits before applying softmax. A high temperature value results in more random outputs, while a low temperature value makes the model's outputs more deterministic and similar to the training data. It's a crucial factor in balancing between diversity and quality of generated content. In the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), "Temperature" refers to a parameter in the probability distribution function used during the generation process. It controls the randomness of predictions by scaling the logits before applying softmax. A high temperature value results in more random outputs, while a low temperature value makes the model's outputs more deterministic and similar to the training data. It's a crucial factor in balancing between diversity and quality of generated content. In an analog to human cogitation one could compare the temperature value to the level of creativity, where a high temperature value would result in more creative outputs, while a low temperature value would result in more objective outputs. --- - **Page:** System Message - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/system-message - **Description:** In the context of LLMs, a "system message" refers to a message or prompt used to set the context or guide the behavior of the model during a conversation. It is typically the first message in a dialogue and is used to instruct the LLM or establish the role it should play in the conversation. System messages help steer the interaction by providing context or specifying the desired format of the response, and they are important for shaping the LLM's behavior, ensuring it understands its role (e.g., as a tutor, chatbot, or information provider), and generating responses that align with the user's expectations. In the context of [LLMs](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), a "system message" refers to a message or prompt used to set the context or guide the behavior of the model during a conversation. It is typically the first message in a dialogue and is used to instruct the LLM or establish the role it should play in the conversation. System messages help steer the interaction by providing context or specifying the desired format of the response, and they are important for shaping the LLM's behavior, ensuring it understands its role (e.g., as a tutor, [chatbot](https://promptmetheus.com/resources/llm-knowledge-base/chatbot), or information provider), and generating responses that align with the user's expectations. --- - **Page:** Tensor Processing Unit (TPU) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/tensor-processing-unit-tpu - **Description:** A Tensor Processing Unit (TPU) is a custom-developed application-specific integrated circuit (ASIC) by Google, designed specifically for accelerating Machine Learning workloads. TPUs are used to enhance the performance of neural network machine learning tasks, leveraging Google's TensorFlow software. A Tensor Processing Unit (TPU) is a custom-developed application-specific integrated circuit (ASIC) by Google, designed specifically for accelerating [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) workloads. TPUs are used to enhance the performance of neural network machine learning tasks, leveraging Google's TensorFlow software. Key aspects of TPUs in the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai) include: 1. **Acceleration**: TPUs are high-performance ML accelerators, which can significantly improve the speed and efficiency of training and inference for Generative AI models. 2. **Architecture**: TPUs are designed with a focus on dense matrix computations, which are essential for neural network operations. They are optimized for matrix multiplications, which are a fundamental part of Generative AI algorithms. 3. **Dimensions**: TPUs work best when the dimensions of the tensors used in the computations are multiples of 128, as this allows for efficient utilization of the TPU's MXU (Matrix Multiplication Unit). 4. **Cloud TPU**: Google Cloud offers Cloud TPU, a web service that enables developers to access TPU resources for machine learning tasks, including training and inference for Generative AI models. 5. **Edge TPU**: This is a low-power ML accelerator designed for IoT devices, which can perform machine learning inferencing at the edge, making it suitable for Generative AI applications that require on-device processing. TPUs are a critical component in the domain of Generative AI, as they enable faster and more efficient training and inference, leading to improved model performance and reduced time-to-market for AI-based products. Also see [Language Processing Unit (LPU)](https://promptmetheus.com/resources/llm-knowledge-base/language-processing-unit-lpu). --- - **Page:** Thinking Mode - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/thinking-mode - **Description:** Thinking mode is a provider-specific setting that lets a model spend dedicated computation on intermediate reasoning before producing its answer. It is commonly exposed as thinking blocks, hidden reasoning, a token budget, or an effort control. Thinking mode is a provider-specific setting that lets a model spend dedicated computation on intermediate reasoning before producing its answer. It is commonly exposed as thinking blocks, hidden reasoning, a token budget, or an effort control. Thinking mode is not the same as asking the model to show chain-of-thought reasoning. Modern systems may use hidden or summarized reasoning while returning only a concise final answer. The important operational effect is that the model can allocate more work to decomposition, verification, and tool-use decisions. Thinking mode affects: - output-token usage and billing; - latency; - [context window](https://promptmetheus.com/resources/llm-knowledge-base/context-window) pressure; - prompt caching behavior; - tool-use loops; and - whether prior reasoning state must be preserved across turns. It should be used deliberately. For simple tasks, thinking mode can waste tokens. For complex coding, math, analysis, or [long-horizon tasks](https://promptmetheus.com/resources/llm-knowledge-base/long-horizon-task), it can improve reliability if the application measures outcomes. Anthropic documents thinking configuration, token accounting, and tool-use behavior in its [Thinking](https://platform.claude.com/docs/en/build-with-claude/thinking) guide. --- - **Page:** Token - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/token - **Description:** In the context of Large Language Models, a token is the smallest unit of data that a model can understand and process. It can be as short as a single character or as long as a word, depending on the language and the specific model. Tokens are used to break down input data into manageable pieces, enabling the AI to analyze, understand, and generate text. In the context of [Large Language Models](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), a token is the smallest unit of data that a model can understand and process. It can be as short as a single character or as long as a word, depending on the language and the specific model. Tokens are used to break down input data into manageable pieces, enabling the AI to analyze, understand, and generate text. For most state-of the art models, one token can be statistically approximated as ~4 characters. Tokens are also the accounting unit behind [token economics](https://promptmetheus.com/resources/llm-knowledge-base/token-economics): model cost, latency, context-window pressure, cache-hit rates, and usage quotas all depend on how many input and output tokens an application consumes. Informal practices such as [tokenmaxxing](https://promptmetheus.com/resources/llm-knowledge-base/tokenmaxxing) try to increase productive AI use, but production teams should measure task outcomes rather than token volume alone. --- - **Page:** Token Economics - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/token-economics - **Description:** Token economics is the discipline of measuring, attributing, and optimizing the business value created by AI token consumption. It treats tokens as a unit of production cost and connects that cost to outcomes such as completed tasks, accepted code changes, reduced support time, or revenue impact. Token economics is the discipline of measuring, attributing, and optimizing the business value created by AI token consumption. It treats [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token) as a unit of production cost and connects that cost to outcomes such as completed tasks, accepted code changes, reduced support time, or revenue impact. The term is different from cryptocurrency token design. In AI systems, token economics concerns model input, model output, internal reasoning, retrieval context, tool results, and repeated agent loops. Useful token economics metrics include: - input, output, cached, and reasoning-token usage; - cost by product, team, user, model, and workflow; - cost per successful task; - cost per accepted artifact or resolved ticket; - latency and quality per model route; - token waste from repeated context, unnecessary loops, or verbose outputs; and - budget exceptions and policy violations. Token economics becomes especially important for [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) because an agent may call models many times before producing a visible output. Tool selection, planning, retries, [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction), and [inference-time scaling](https://promptmetheus.com/resources/llm-knowledge-base/inference-time-scaling-test-time-compute) can all change the cost of one user request. Cost reduction alone is not the goal. A cheap workflow that fails is wasteful, while an expensive workflow may be justified if it reliably completes high-value work. The relevant unit is cost per validated outcome. Technical levers include [model routing](https://promptmetheus.com/resources/llm-knowledge-base/model-routing), prompt caching, shorter tool outputs, retrieval filtering, smaller models for simple subtasks, deterministic code for routine operations, and tighter [agent evaluation](https://promptmetheus.com/resources/llm-knowledge-base/agent-evaluation). FinOps Foundation describes AI token economics as studying how token production, distribution, and consumption generate business outcomes in its article [Token Economics: The Atomic Unit of AI Value](https://www.finops.org/insights/token-economics-the-atomic-unit-of-ai-value/). --- - **Page:** Token Limit - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/token-limit - **Description:** Token limit refers to the maximum number of tokens that a Generative AI model can process or generate in a single operation. The token limit is a crucial factor in determining the complexity and length of the content that the AI can handle (see also "Context Window"). Token limit refers to the maximum number of [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token) that a Generative AI model can process or generate in a single operation. The token limit is a crucial factor in determining the complexity and length of the content that the AI can handle (see also ["Context Window"](https://promptmetheus.com/resources/llm-knowledge-base/context-window)). Different LLM providers treat token limits differently. Sometimes the token limit applies to the total number of [input tokens](https://promptmetheus.com/resources/llm-knowledge-base/input-token) plus [output tokens](https://promptmetheus.com/resources/llm-knowledge-base/output-token), e.g. OpenAI. Other times, limits are applied to input and output independently, e.g. Gemini. --- - **Page:** Tokenization - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/tokenization - **Description:** Tokenization refers to the process of converting a sequence of text into individual pieces, known as token. These tokens can represent words, characters, or subwords, and serve as the basic units for understanding and processing text in Natural Language Processing (NLP) tasks. Tokenization helps AI models to better understand, analyze, and generate human language by breaking down complex sentences into simpler, manageable components. Tokenization refers to the process of converting a sequence of text into individual pieces, known as [token](https://promptmetheus.com/resources/llm-knowledge-base/token). These tokens can represent words, characters, or subwords, and serve as the basic units for understanding and processing text in [Natural Language Processing (NLP)](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) tasks. Tokenization helps AI models to better understand, analyze, and generate human language by breaking down complex sentences into simpler, manageable components. --- - **Page:** Tokenmaxxing - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/tokenmaxxing - **Description:** Tokenmaxxing is the practice of maximizing AI token usage and treating that usage as a proxy for productivity or adoption. The term became common in 2026 as organizations started tracking or encouraging high usage of chatbots, coding agents, and internal AI tools. Tokenmaxxing is the practice of maximizing AI token usage and treating that usage as a proxy for productivity or adoption. The term became common in 2026 as organizations started tracking or encouraging high usage of chatbots, coding agents, and internal AI tools. The useful version of the term is critical, not aspirational. More [tokens](https://promptmetheus.com/resources/llm-knowledge-base/token) do not necessarily mean more value. A team can spend heavily on long prompts, repeated context, failed agent loops, unnecessary reasoning, or low-impact work. Tokenmaxxing can appear in several forms: - leaderboards that reward raw token volume; - broad access to expensive frontier models without routing; - [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) looping without convergence; - verbose outputs that are never used; - repeated injection of the same documents into context; - using LLM calls where deterministic automation would suffice; and - measuring adoption without measuring accepted work. The opposite of tokenmaxxing is not simply spending less. It is applying [token economics](https://promptmetheus.com/resources/llm-knowledge-base/token-economics) to connect model usage to real outcomes. A high-token workflow may be appropriate for a valuable [long-horizon task](https://promptmetheus.com/resources/llm-knowledge-base/long-horizon-task), while a low-token workflow may still be waste if it fails. Mitigations include [model routing](https://promptmetheus.com/resources/llm-knowledge-base/model-routing), budgets, prompt caching, task-level evaluation, usage dashboards, context pruning, deterministic fallbacks, and cost-per-outcome metrics. Tokenmaxxing is not a precise research term, and it should be used carefully in professional writing. Its value is that it names a real management failure mode: optimizing for visible AI consumption rather than validated productivity. Recent reporting has connected tokenmaxxing to enterprise AI cost governance, including [The Guardian's coverage of AI usage budgets](https://www.theguardian.com/technology/2026/jul/30/atlassian-tightens-tracking-of-staff-ai-use-as-other-technology-firms-encourage-tokenmaxxing) and analysis of rising agentic token costs. --- - **Page:** Tool Calling (Function Calling) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/tool-calling-function-calling - **Description:** Tool calling, also known as function calling, is a mechanism that allows a Large Language Model (LLM) to request the execution of an external operation. Instead of directly performing the operation, the model produces a structured call containing a tool name and arguments. Application code validates and executes the request, then returns the result to the model. Tool calling, also known as function calling, is a mechanism that allows a [Large Language Model (LLM)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) to request the execution of an external operation. Instead of directly performing the operation, the model produces a structured call containing a tool name and arguments. Application code validates and executes the request, then returns the result to the model. Tools can provide access to databases, search engines, calculators, business APIs, code execution environments, or physical devices. They allow a model to use current or private information and to take actions that cannot be completed through text generation alone. A typical tool-calling loop has four stages: 1. The application supplies tool definitions and a user request to the model. 2. The model selects a tool and generates arguments that conform to its schema. 3. The application authorizes, validates, and executes the call. 4. The tool result is added to the [context](https://promptmetheus.com/resources/llm-knowledge-base/context), after which the model may answer or request another tool. Tool calling is related to, but distinct from, [structured output](https://promptmetheus.com/resources/llm-knowledge-base/structured-output). Structured output constrains the shape of a model's response. Tool calling additionally defines an executable contract between the model and an external system. Reliable tool design requires more than a valid schema. Tool names and descriptions must make selection criteria unambiguous, parameters should reflect domain types, and results should be concise enough for effective [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering). Overlapping tools often reduce reliability because the model must infer distinctions that the interface failed to express. When an application exposes more tools than can be included reliably in every request, [dynamic tool discovery](https://promptmetheus.com/resources/llm-knowledge-base/dynamic-tool-discovery-tool-search) can retrieve a small relevant subset at runtime. Production implementations should treat model-generated arguments as untrusted input. Important controls include schema validation, authorization, timeouts, rate limits, idempotency keys, [agent guardrails](https://promptmetheus.com/resources/llm-knowledge-base/agent-guardrail), audit logs, and explicit confirmation before consequential actions. Tool results can also contain malicious instructions, making indirect [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack) a significant risk. Tool calling is a foundational capability for [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent), [agentic workflows](https://promptmetheus.com/resources/llm-knowledge-base/agentic-workflow), and the [Model Context Protocol (MCP)](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp). See the [OpenAI function calling guide](https://developers.openai.com/api/docs/guides/function-calling) for a concrete schema-based implementation. --- - **Page:** Top P - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/top-p - **Description:** Top P, also known as Nucleus Sampling, is a strategy used in language models for text generation. Instead of choosing the most likely next word in a sequence, the model considers a set of possible next words, known as the "nucleus", based on their cumulative probability exceeding a certain threshold, P. This approach provides a balance between randomness and predictability, resulting in more diverse and realistic text generation. Top P, also known as Nucleus Sampling, is a strategy used in language models for text generation. Instead of choosing the most likely next word in a sequence, the model considers a set of possible next words, known as the "nucleus", based on their cumulative probability exceeding a certain threshold, P. This approach provides a balance between randomness and predictability, resulting in more diverse and realistic text generation. In the context of tuning model parameters, Top P is very similar to [temperature](https://promptmetheus.com/resources/llm-knowledge-base/temperature) and it is usually recommended to only adjust one of the two. --- - **Page:** Transformer - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/transformer - **Description:** A Transformer is a type of artificial intelligence model primarily used in the field of Natural Language Processing. Introduced in the paper "Attention is All You Need" by Vaswani et al., it revolutionized the NLP domain by using a mechanism called "attention" to understand the context of words in a sentence. Unlike previous models, Transformers do not process data in sequential order, but rather, they process all data points simultaneously, making them highly efficient for large-scale tasks. Transformers serve as the foundation for many advanced AI models, including ChatGPT, Gemini, Claude, and others. A Transformer is a type of artificial intelligence model primarily used in the field of [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp). Introduced in the paper "Attention is All You Need" by Vaswani et al., it revolutionized the NLP domain by using a mechanism called "attention" to understand the context of words in a sentence. Unlike previous models, Transformers do not process data in sequential order, but rather, they process all data points simultaneously, making them highly efficient for large-scale tasks. Transformers serve as the foundation for many advanced AI models, including ChatGPT, Gemini, Claude, and others. --- - **Page:** User Message - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/user-message - **Description:** A user message is a message in a chat-style LLM API that represents input from the end user or calling application on the user's behalf. It usually contains the task request, question, provided data, constraints, or follow-up information. A user message is a message in a chat-style LLM API that represents input from the end user or calling application on the user's behalf. It usually contains the task request, question, provided data, constraints, or follow-up information. In a conversation, user messages are distinct from: - a [system message](https://promptmetheus.com/resources/llm-knowledge-base/system-message), which sets high-level behavior or policy; - a developer instruction, which may define application-specific behavior; - an [assistant message](https://promptmetheus.com/resources/llm-knowledge-base/assistant-message), which represents the model's prior or generated response; and - tool messages, which return results from tool calls. User messages are important for instruction hierarchy. A user can request a task, but the request should not override higher-priority system or developer constraints. This matters for safety, data access, and [prompt injection attacks](https://promptmetheus.com/resources/llm-knowledge-base/prompt-injection-attack). In [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering), user messages should be preserved accurately when they contain task requirements, approvals, or corrections. They may be summarized during [context compaction](https://promptmetheus.com/resources/llm-knowledge-base/context-compaction), but important constraints should remain explicit. OpenAI's help article on moving from completions to chat messages explains that each message has a role and content, including the user role, in [Moving from Completions to Chat Completions](https://help.openai.com/en/articles/7042661). --- - **Page:** Vector Embedding - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding - **Description:** Embedding, in the context of Generative AI, refers to the mathematical representation of complex data types, such as words, sentences, or even entire documents, in a high-dimensional space. These representations, often in the form of vectors, capture the semantic or contextual meaning of the data. Embeddings are a crucial part of many Machine Learning models, as they allow these models to understand and process data in a more human-like way. They are particularly useful in Natural Language Processing, recommendation systems, and other areas where understanding the relationships between different pieces of data is important. Embedding, in the context of [Generative AI](https://promptmetheus.com/resources/llm-knowledge-base/generative-ai), refers to the mathematical representation of complex data types, such as words, sentences, or even entire documents, in a high-dimensional space. These representations, often in the form of vectors, capture the semantic or contextual meaning of the data. Embeddings are a crucial part of many [Machine Learning](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) models, as they allow these models to understand and process data in a more human-like way. They are particularly useful in [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp), recommendation systems, and other areas where understanding the relationships between different pieces of data is important. See also [Vector Store](https://promptmetheus.com/resources/llm-knowledge-base/vector-store) and [Vector Search](https://promptmetheus.com/resources/llm-knowledge-base/vector-search). --- - **Page:** Vector Search - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/vector-search - **Description:** Vector search is a method used in information retrieval and Machine Learning (ML) to find and rank items based on their vector representations in a multi-dimensional space. In the context of Large Language Models (LLMs), vector search involves encoding data, such as text or images, into high-dimensional vectors using techniques like embeddings. These vectors capture semantic meaning, allowing for efficient similarity searches and comparisons. Vector search is a method used in information retrieval and [Machine Learning (ML)](https://promptmetheus.com/resources/llm-knowledge-base/machine-learning-ml) to find and rank items based on their vector representations in a multi-dimensional space. In the context of [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm), vector search involves encoding data, such as text or images, into high-dimensional vectors using techniques like [embeddings](https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding). These vectors capture semantic meaning, allowing for efficient similarity searches and comparisons. The process typically involves transforming input data into vectors using models like word2vec, BERT, or other neural network-based embeddings. Once data is represented as vectors, vector search algorithms, such as k-nearest neighbors (k-NN) or approximate nearest neighbor (ANN) methods, are employed to identify the most similar vectors within a dataset. This approach is crucial for applications like recommendation systems, semantic search, and [Natural Language Processing (NLP)](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) tasks, where understanding the contextual similarity between data points is essential. Vector search is particularly advantageous in handling large-scale datasets, as it enables fast and scalable retrieval of relevant information by leveraging advanced indexing techniques and optimized data structures. Vector similarity depends on the selected [embedding distance metrics](https://promptmetheus.com/resources/llm-knowledge-base/embedding-distance-metrics), such as cosine distance, Euclidean distance, or dot product. It does not always capture exact identifiers, rare names, or required keywords. [Hybrid Search](https://promptmetheus.com/resources/llm-knowledge-base/hybrid-search) combines vector retrieval with lexical search, while [Reranking](https://promptmetheus.com/resources/llm-knowledge-base/reranking) applies a more precise model to the initial candidates. See also [Vector Embedding](https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding) and [Vector Store](https://promptmetheus.com/resources/llm-knowledge-base/vector-store). --- - **Page:** Vector Store - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/vector-store - **Description:** A vector store is a specialized database designed to efficiently store, index, and retrieve high-dimensional vectors. These vectors typically represent data points such as text, images, or other forms of unstructured data that have been transformed into numerical representations through processes like embedding. Vector stores are crucial for tasks involving similarity search, clustering, and nearest neighbor search, as they enable rapid querying and comparison of vectors based on their geometric properties. They are often used in applications like recommendation systems, semantic search, and Natural Language Processing (NLP) to enhance the performance and scalability of Large Language Models (LLMs). A vector store is a specialized database designed to efficiently store, index, and retrieve high-dimensional vectors. These vectors typically represent data points such as text, images, or other forms of unstructured data that have been transformed into numerical representations through processes like [embedding](https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding). Vector stores are crucial for tasks involving similarity search, clustering, and nearest neighbor search, as they enable rapid querying and comparison of vectors based on their geometric properties. They are often used in applications like recommendation systems, semantic search, and [Natural Language Processing (NLP)](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) to enhance the performance and scalability of [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm). See also [Vector Embedding](https://promptmetheus.com/resources/llm-knowledge-base/vector-embedding) and [Vector Search](https://promptmetheus.com/resources/llm-knowledge-base/vector-search). --- - **Page:** Verifiable Intent - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/verifiable-intent - **Description:** Verifiable intent is a machine-verifiable record that a user authorized an AI agent to take a specific action under specific constraints. It is especially important for payments, purchases, bookings, and other consequential workflows where a service must distinguish an authorized agent action from an accidental or malicious one. Verifiable intent is a machine-verifiable record that a user authorized an AI agent to take a specific action under specific constraints. It is especially important for payments, purchases, bookings, and other consequential workflows where a service must distinguish an authorized agent action from an accidental or malicious one. In [agentic commerce](https://promptmetheus.com/resources/llm-knowledge-base/agentic-commerce), verifiable intent can provide evidence such as: - who authorized the agent; - which agent or host received the mandate; - what item, merchant, amount, or constraint was approved; - when the authorization was issued; - whether the user was present or absent; - what payment or checkout evidence was linked; and - whether the record was tampered with. Verifiable intent is related to [Agent Payments Protocol (AP2)](https://promptmetheus.com/resources/llm-knowledge-base/agent-payments-protocol-ap2), but the concept is broader than payments. Similar evidence patterns may apply to legal approvals, data exports, account changes, enterprise procurement, or privileged tool calls. The core design goal is accountability. If an autonomous system acts later, the service should be able to verify that the action still matches the user's original authorization and policy constraints. Google describes Verifiable Intent as a standard for creating a tamper-proof log of user-authorized agent actions in its [AP2 donation announcement](https://blog.google/products-and-platforms/platforms/google-pay/agent-payments-protocol-fido-alliance/). --- - **Page:** Vibe Coding - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/vibe-coding - **Description:** Vibe coding is an AI-assisted programming paradigm where developers describe desired functionalities in natural language, allowing artificial intelligence to generate the corresponding code. This approach leverages Large Language Models (LLMs) to translate user intentions into executable software, enabling individuals with minimal coding experience to create sophisticated applications. Vibe coding is an AI-assisted programming paradigm where developers describe desired functionalities in natural language, allowing artificial intelligence to generate the corresponding code. This approach leverages [Large Language Models (LLMs)](https://promptmetheus.com/resources/llm-knowledge-base/large-language-model-llm) to translate user intentions into executable software, enabling individuals with minimal coding experience to create sophisticated applications. Coined by computer scientist Andrej Karpathy in early 2025, vibe coding emphasizes a relaxed interaction with AI tools, where developers focus on high-level problem descriptions and rely on AI to handle implementation details. Karpathy describes it as "fully giving in to the vibes, embracing exponentials, and forgetting that the code even exists." While vibe coding democratizes software development and accelerates prototyping, it presents challenges such as potential technical debt, security vulnerabilities, and difficulties in maintaining AI-generated code. Experts caution that over-reliance on AI without proper code review may lead to unmanageable projects. Despite these challenges, vibe coding represents a significant shift in software development, enabling rapid creation of functional applications through natural language interactions with AI. --- - **Page:** World Model - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/world-model - **Description:** A world model is an AI model that learns a representation of an environment and predicts how that environment will change over time, including how actions affect future states. It gives an agent a mechanism for anticipating consequences rather than learning only from direct trial and error. A world model is an AI model that learns a representation of an environment and predicts how that environment will change over time, including how actions affect future states. It gives an agent a mechanism for anticipating consequences rather than learning only from direct trial and error. World models may operate over compact latent states, images, video, physical sensor data, game states, or combinations of modalities. Given a current state and a candidate action, the model estimates a future state, observation, reward, or probability distribution over possible outcomes. An agent can use a world model to: - simulate alternative action sequences before acting; - plan toward a target state; - generate training environments or synthetic experience; - learn policies with fewer interactions in the real world; and - detect when observed behavior differs from expected dynamics. World models are related to generative video models but are not synonymous with them. A video model may generate visually plausible sequences without supporting controllable actions or stable environment dynamics. A useful world model must represent action-conditioned change well enough for planning or interaction. Applications include robotics, autonomous systems, games, scientific simulation, and training [AI agents](https://promptmetheus.com/resources/llm-knowledge-base/ai-agent) in generated environments. They may combine a [Vision-Language Model (VLM)](https://promptmetheus.com/resources/llm-knowledge-base/vision-language-model-vlm) for semantic understanding with specialized dynamics prediction. The central failure mode is model error accumulating over long simulated horizons. An agent can exploit inaccuracies in the learned environment or plan around physically implausible states. Evaluation should therefore measure controllability, temporal consistency, action consequences, and transfer to the real environment. Google DeepMind defines world models as systems that simulate aspects of the world and enable agents to predict environmental evolution and action effects in its [Genie 3 overview](https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/). --- - **Page:** Vision-Language Model (VLM) - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/vision-language-model-vlm - **Description:** A Vision-Language Model (VLM) is a multimodal model that jointly processes visual information and natural language. It can connect images or video with text to perform tasks such as visual question answering, captioning, document understanding, image retrieval, and instruction following in visual environments. A Vision-Language Model (VLM) is a multimodal model that jointly processes visual information and natural language. It can connect images or video with text to perform tasks such as visual question answering, captioning, document understanding, image retrieval, and instruction following in visual environments. A typical VLM includes: - a vision encoder that transforms pixels or image regions into representations; - a projection, adapter, or cross-attention mechanism that connects visual representations to language tokens; and - a language model that interprets the combined sequence and generates text or actions. Some VLMs are trained end to end, while others connect pretrained vision and language components. Training data may contain image-caption pairs, interleaved webpages, videos with transcripts, visual instructions, or [synthetic data](https://promptmetheus.com/resources/llm-knowledge-base/synthetic-data). VLM is more specific than [multi-modality](https://promptmetheus.com/resources/llm-knowledge-base/multi-modality), which can include audio, speech, or other data types. It is also distinct from a pure image generator: a VLM primarily learns relationships between visual inputs and language, although one system may support both understanding and generation. VLMs are central to [computer-using agents](https://promptmetheus.com/resources/llm-knowledge-base/computer-using-agent-cua) and robotics because they can interpret screenshots, diagrams, interfaces, and physical scenes. Visual grounding remains imperfect; models may misread small text, spatial relationships, counts, or occluded objects while producing a fluent explanation. Evaluation should separate perception errors from reasoning errors and test robustness across resolution, layout, image quality, and domain shift. Sensitive images also require appropriate privacy and retention controls. The [Flamingo paper](https://arxiv.org/abs/2204.14198) describes an influential architecture that connects pretrained vision and language models and handles interleaved image, video, and text inputs. --- - **Page:** Zero-shot Prompt - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/zero-shot-prompt - **Description:** Zero-shot prompt refers to a situation where an AI model generates an output or completes a task without any prior specific training or examples on that task. The model uses its general understanding from pre-training to generate the response, hence the term 'zero-shot', indicating no additional training shots were given for that specific task. This concept is often used in Natural Language Processing and machine learning models. Zero-shot prompt refers to a situation where an AI model generates an output or completes a task without any prior specific training or examples on that task. The model uses its general understanding from pre-training to generate the response, hence the term 'zero-shot', indicating no additional training shots were given for that specific task. This concept is often used in [Natural Language Processing](https://promptmetheus.com/resources/llm-knowledge-base/natural-language-processing-nlp) and machine learning models. See also ["One-shot Prompt"](https://promptmetheus.com/resources/llm-knowledge-base/one-shot-prompt) and ["Few-shot Prompt"](https://promptmetheus.com/resources/llm-knowledge-base/few-shot-prompt). --- - **Page:** agents.txt - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/agents-txt - **Description:** agents.txt is a proposed discovery file for publishing a site's agent-facing capabilities. It is designed for AI agents that may need to discover supported protocols, payment mechanisms, authorization schemes, MCP endpoints, skills, A2A agent cards, or other machine-readable interaction surfaces. agents.txt is a proposed discovery file for publishing a site's agent-facing capabilities. It is designed for AI agents that may need to discover supported protocols, payment mechanisms, authorization schemes, MCP endpoints, skills, A2A agent cards, or other machine-readable interaction surfaces. Where [llms.txt](https://promptmetheus.com/resources/llm-knowledge-base/llms-txt) helps an agent find content to read, agents.txt helps an agent understand what it can do with a site. It is typically paired with a structured `agents.json` file for richer machine parsing. Examples of capabilities that may be declared include: - [MCP](https://promptmetheus.com/resources/llm-knowledge-base/model-context-protocol-mcp) server endpoints; - [Agent2Agent Protocol (A2A)](https://promptmetheus.com/resources/llm-knowledge-base/agent2agent-protocol-a2a) Agent Card URLs; - skill package URLs; - authorization discovery such as [Agent Auth Protocol](https://promptmetheus.com/resources/llm-knowledge-base/agent-auth-protocol); - agent payment support such as [Agent Payments Protocol (AP2)](https://promptmetheus.com/resources/llm-knowledge-base/agent-payments-protocol-ap2); and - commerce or interaction profiles. The file is not a trust guarantee. Agents still need to validate endpoints, authenticate properly, enforce [agent policy](https://promptmetheus.com/resources/llm-knowledge-base/agent-policy), and ask for human approval when actions are consequential. The [agents.txt standard draft](https://github.com/agents-txt/agents-txt/blob/main/app/site/src/content/spec/AGENTS-TXT-STANDARD.md) defines directives for payments, authorization, MCP, skills, A2A, UCP, and WebMCP discovery. --- - **Page:** llms.txt - **Source:** https://promptmetheus.com/resources/llm-knowledge-base/llms-txt - **Description:** llms.txt is a proposed convention for publishing an LLM-friendly index of a website or documentation set. It is usually served at /llms.txt and written in Markdown so AI assistants, coding agents, and retrieval systems can quickly discover the most relevant pages. llms.txt is a proposed convention for publishing an LLM-friendly index of a website or documentation set. It is usually served at `/llms.txt` and written in Markdown so AI assistants, coding agents, and retrieval systems can quickly discover the most relevant pages. The file typically contains: - a site or project title; - a short description; - grouped links to important Markdown or documentation pages; - brief descriptions of each link; and - optional references to larger bundles such as `llms-full.txt`. The goal is different from `robots.txt` or `sitemap.xml`. A sitemap helps crawlers enumerate URLs. robots.txt communicates crawl permissions. llms.txt helps an AI system decide what content is useful to read for inference-time context. For developer documentation, llms.txt can reduce wasted crawling and improve [context engineering](https://promptmetheus.com/resources/llm-knowledge-base/context-engineering) by pointing agents to canonical Markdown, API references, examples, and setup guides. It is especially useful for [coding agents](https://promptmetheus.com/resources/llm-knowledge-base/coding-agent-agentic-coding) that need accurate product documentation. llms.txt is not an access-control mechanism. Sensitive or private content must still be protected by authentication and authorization. The file should only link to content the publisher intends AI tools to read. The original [Answer.AI llms.txt proposal](https://github.com/answerdotai/llms-txt) describes the format as a way to help language models use a website at inference time.