● Beta — KubeEra is in active development. Profiles are live; AI-perception measurement is rolling out.
KubeEra
G

Guidance

AI Agents
Open source · MIT

Guidance is a Python library for controlling the output of large language models at the token level — constrained generation, structured output (JSON, regex, context-free grammars), and interleaved control flow between your code and model g

updated 2026-07-01

What it is

Guidance is a Python library for controlling the output of large language models at the token level — constrained generation, structured output (JSON, regex, context-free grammars), and interleaved control flow between your code and model generation. It's not an agent framework despite how it gets categorized; it's closer to a templating and decoding-control layer that sits between your application and the model. You write a program that mixes text, logic, and generation calls, and Guidance enforces the output shape by manipulating the token sampling process directly (when running against a supported backend) rather than by re-prompting or regex-parsing after the fact.

Who builds it and why

Originating from Microsoft Research, the project lives under the guidance-ai GitHub org. It's maintained by a mix of Microsoft-affiliated engineers and external contributors — 84 contributors total, which is a healthy number for a library this specialized. The primary language classification as Jupyter Notebook is a signal in itself: a large share of the repo is examples and interactive demos rather than pure library code, which tracks with how the project is used — a lot of practitioners find it through notebooks, not package docs.

The motivation is straightforward: LLM APIs give you weak guarantees about output structure, and naive prompting-plus-parsing is fragile. Guidance's bet is that controlling generation at the token/grammar level is more reliable than hoping the model follows instructions.

Production readiness signal

21,521 stars and continued commits through 2026 indicate the project is active and has real mindshare, not abandoned. But a few things temper the "production-ready" read:

  • Versioning is still pre-1.0 (0.3.2). API stability across minor versions has historically been rocky — breaking changes between 0.x releases are common.
  • Backend support is uneven. The strongest guarantees (true constrained decoding) require specific backends (e.g., llama.cpp, transformers with local models). Against hosted APIs like OpenAI's, you lose the core value proposition and fall back to weaker techniques.
  • CNCF maturity: not publicly available — this isn't a CNCF project and shouldn't be evaluated against that governance bar.
  • No founding date publicly available, so tenure/stability trends are hard to verify beyond what the commit history shows.

Treat it as a solid library for teams willing to pin versions and test upgrades carefully, not a "install and forget" dependency.

Who should use this

  • Teams running local or self-hosted open-weight models who need guaranteed JSON/schema-conforming output and want to avoid brittle regex-based output parsing.
  • Engineers building structured extraction, form-filling, or classification pipelines where malformed output is a hard failure, not a retry-and-hope situation.
  • Anyone already comfortable working close to the token/sampling layer and willing to read source when docs fall short.

Who should NOT use this

  • Teams building multi-step autonomous agents, tool-use orchestration, or long-running workflows — Guidance doesn't provide agent loops, memory, or planning; if that's what you need, you're in the wrong category entirely despite how this tool gets labeled.
  • Teams exclusively using hosted APIs (OpenAI, Anthropic) where token-level control isn't exposed — you'll get a fraction of the benefit and added complexity for little gain.
  • Anyone needing a stable, versioned API contract for a regulated production system — the pre-1.0 status and history of breaking changes make this a liability if you can't tolerate mid-cycle churn.
  • Teams wanting strong vendor support/SLAs — this is a community/Microsoft-research-adjacent project, not a commercially supported product.

Alternatives

  • Outlines — similar constrained-generation approach, arguably more focused purely on structured output and grammar-based sampling, with broader backend support for local models.
  • LMQL — a query language for LLMs with similar goals (structured, controllable generation), different syntax philosophy (SQL-like).
  • Instructor — much lighter-weight, works via function-calling/schema validation against hosted APIs rather than token-level constraint, better fit if you're API-only.

Pricing

Fully open source, MIT licensed. No paid tier, no hosted service, no vendor lock-in. Cost is entirely your own compute (for local model backends) or your existing API spend if used against hosted models.

Frequently asked

What is Guidance?+
Guidance is a Python library for controlling the output of large language models at the token level — constrained generation, structured output (JSON, regex, context-free grammars), and interleaved control flow between your code and model generation.
Who builds Guidance?+
Originating from Microsoft Research, the project lives under the guidance-ai GitHub org. It's maintained by a mix of Microsoft-affiliated engineers and external contributors — 84 contributors total, which is a healthy number for a library this specialized.
Is Guidance production ready?+
21,521 stars and continued commits through 2026 indicate the project is active and has real mindshare, not abandoned. But a few things temper the "production-ready" read: Versioning is still pre-1.0 (0.3.2). API stability across minor versions has historically been rocky — breaking changes between 0.
Who should use Guidance?+
Teams running local or self-hosted open-weight models who need guaranteed JSON/schema-conforming output and want to avoid brittle regex-based output parsing. Engineers building structured extraction, form-filling, or classification pipelines where malformed output is a hard failure, not a retry-and-hope situation.
Who should not use Guidance?+
Teams building multi-step autonomous agents, tool-use orchestration, or long-running workflows — Guidance doesn't provide agent loops, memory, or planning; if that's what you need, you're in the wrong category entirely despite how this tool gets labeled.
What are the alternatives to Guidance?+
Outlines — similar constrained-generation approach, arguably more focused purely on structured output and grammar-based sampling, with broader backend support for local models. LMQL — a query language for LLMs with similar goals (structured, controllable generation), different syntax philosophy (SQL-like).
How much does Guidance cost?+
Fully open source, MIT licensed. No paid tier, no hosted service, no vendor lock-in. Cost is entirely your own compute (for local model backends) or your existing API spend if used against hosted models.