Changelog

[v0.40.0] - 2026-07-19

Added

• Durable runs now persist token usage: the delivery executor writes a tokens_usages.json result artifact ({"tokens_usages": [...], "usage_assembly_error": null}) alongside working_memory.json, the main_stuff.* renders, and the graph outputs. The records use the client-facing TokensUsageRecord wire shape — the same shape the /execute response carries on pipe_output.tokens_usages — so a client polling a durable run's result files reads costs the same way a sync /execute caller does. The artifact is written unconditionally: a run with usage assembly off yields explicit nulls, distinguishable from a pre-artifact run (file absent).
• TokensUsageRecord — a deliberate client wire shape for token usage (pipelex/reporting/usage_records.py). One flat record per inference call: model_type, inference_model_name, inference_model_id, pipe_code, job_category, unit_job_id, nb_tokens_by_category (raw provider counts), computed USD cost (null when the model has no rate table), and ISO started_at/completed_at. Enum-ish fields are open string sets on the wire, so runtime enum churn is non-breaking for clients. compute_tokens_usage_cost reuses the existing cost engine (model_cost_per_token + the cached/non-cached split), so the wire cost equals the CLI cost table's total for the same call. Documented in TokensUsage Wire Records.

Changed

• Breaking: tokens_usages records on client surfaces are now TokensUsageRecord wire records, not dumps of the internal usage models. Both the /execute response (pipe_output.tokens_usages, via the new apply_tokens_usage_wire_shape helper API servers apply to the response dump) and the durable tokens_usages.json artifact stop leaking runtime internals: job_metadata (with its user_id, session_id, request_id, pipe_run_id, otel_context, trace_context, content_generation_job_id, pipeline_run_id) and the unit_costs rate table no longer cross the wire — the kept JobMetadata fields (pipe_code, job-kind enums, timing) are flattened onto the record and cost is computed server-side. Null semantics unchanged (null = assembly off, [] = no inference). Internal crossings (runtime-bridge payloads, Temporal transport, the usage-event telemetry stream) keep full-fidelity records, untouched.

Fixed

• Validation errors stay domain-qualified: every raise site that names a pipe_code now also carries domain_code. The presentation chain (VS Code extension + mthds-ui) identifies pipes by full pipe ref (domain_code.pipe_code); a handful of PipeValidationError raise sites (PipeLLM static input checks, PipeStructure input-mismatch, the pipe sorter's circular-dependency error) omitted domain_code, degrading node decorations and click-to-navigate for those errors in multi-domain bundles. The sorter gains a domain_code parameter threaded from the bundle spec's domain.
• Hosted PipeFunc dry runs now honor output multiplicity. In sandbox-hosted mode a PipeFunc with a multiplicity output (e.g. Foo[]) mocked a single scalar item because the function annotation is unavailable in-process; downstream pipes expecting a list (e.g. batch_over) then failed the dry-run//validate of a perfectly valid method. The mock now takes its shape from the declared output multiplicity.
• get_optional_config() honors its non-raising contract before any hub exists. It used to raise RuntimeError: PipelexHub is not initialized when no hub had been created at all, which broke the documented safe pre-boot path — e.g. constructing a PipeFunc in an isolated unit test crashed instead of defaulting to the direct execution mode.
• PipeFunc transport hardening: the wire contract now rejects a non-finite timeout_seconds (float("inf") used to disable the runaway-code guard entirely), and the transported executor removes its materialized source workdir on every path (success, timeout, failure) instead of leaking a temp directory per invocation, pruning the sys.path entries it inserted for that workdir in the same cleanup so dead entries can't accumulate (or be hijacked by a recreated path) in a reused process.
• Error reference pages generated for the PipeFunc transport errors (PipeFuncTransportError, PipeFuncExecutionError, DuplicatePipeFuncExecutorError, UnknownPipeFuncExecutionModeError) — their type_uris previously dereferenced to 404s.

[v0.39.2] - 2026-07-17

Added

• LibraryManagerAbstract.is_crate_loaded(*, library_id, fingerprint) — public query over the per-library crate-fingerprint bookkeeping that already backs load_from_crate's idempotency. A True answer means the library's ClassRegistry holds the crate's dynamic classes, so callers can hydrate within an existing scope instead of opening a fresh one — preserving dynamic-class identity with instances that scope already produced. This is the core half of the fix for the PipeParallel same-concept combine failing over Temporal with expected X, got X pydantic model-type errors.

Fixed

• PipeCompose construct: whole-stuff copies into native fields now convert in every promised case. { from = "..." } referencing a whole native stuff used to hand the content wrapper (TextContent, ListContent, ...) to the composed field in several cases, failing the dry-run runnable gate on correct methods. Three gaps fixed: optional (non-required) fields never converted (the Optional[X] annotation defeated the type detection), list-of-text targets dumped each item as a {"text": ...} dict instead of extracting the string, and scalar wrappers other than Text were not handled at all. The conversion matrix now covers Text → text, Number → number, YesNo → boolean, and Date → date, for both required and optional target fields, scalar and list-item positions — including nullable list items (list[str | None]), which normalize like their non-nullable counterparts. Unconvertible list items now raise a clear StructuredContentComposerTypeError instead of surfacing as a cryptic pydantic error downstream. One fidelity guard: copying a Date that carries a time of day into a bare date field raises instead of silently dropping the time and its UTC offset (breaking: this case previously truncated silently).

[v0.39.1] - 2026-07-15

Changed

• Bumped the pipelex-tools-py runtime dependency to >=0.1.3 and the pipelex-tools dev dependency to >=0.7.2, picking up the latest MTHDS schema updates for plxt formatting and linting.

[v0.39.0] - 2026-07-14

Highlights

• From resolved library to typed clients: pipelex resolve and pipelex codegen — pipelex resolve distills a bundle's whole closure into the normalized library crate — a flat, fully-qualified, fingerprinted snapshot — and pipelex codegen types projects it into TypeScript (ts-zod) or Python (python-pydantic, python-structures) artifacts. A trust chain of stamped headers and a codegen.lock makes drift detectable offline: pipelex codegen check verifies artifacts by pure hashing — no engine, no network, no API key.
• Validation now fixes what it finds — When an error has a deterministic safe fix, every surface says so: a 💡 suggested-fix line in the CLI output, a structured suggested_fix object in agent and API payloads. The new pipelex fix bundle / pipelex-agent fix bundle commands apply those fixes in place through a shared fix loop, with --diff to preview the changes without writing anything.
• Smart Inputs: the signature decides the shape — Inputs are now interpreted top-down against the pipe's declared input signature instead of bottom-up from their shape alone: a bare string becomes the declared concept (never silently degrading to generic Text), lists shape element-wise, bare URLs/paths resolve for Image/Document inputs, and generated input templates default to the light bare-value form.
• New native concepts: Date, Time, and YesNo — Built-in concepts for calendar dates (preserving source precision — no fabricated times of day), times of day, and boolean judgments, plus a time structure-field type completing the temporal triple. Every native now materializes from its pinned normative definition, so crate fingerprints byte-agree across implementations.

Added

• pipelex resolve and the pipelex codegen family: pipelex resolve assembles a bundle closure and emits the normalized library crate — a flat, fully-qualified, natives-expanded, self-contained, fingerprinted snapshot of a resolved library — as JSON or TOML. On top of the crate:

  • pipelex codegen types --target ts-zod|python-pydantic|python-structures projects the crate's concept set into typed artifacts. The ts-zod target emits a pure types.ts (imports only zod, with wire-native snake_case keys so a schema validates a wire payload directly) plus a binder.ts exposing a typed parse/serialize pair per concept; python-pydantic emits self-contained BaseModels; python-structures emits runtime StructuredContent classes.
  • pipelex codegen inputs --pipe <ref> renders a runnable inputs template for a pipe (defaulting to the closure's main_pipe).
  • Generated files are never edited — each carries an AUTOGENERATED header pointing at the sibling extension-file mechanism.
  • Codegen trust chain (stamps · codegen.lock · offline codegen check): Every file pipelex codegen types writes now carries a self-describing stamp header (source crate fingerprint, engine version, projection, options, and a content hash of the body below it), and the artifact set is recorded in a sibling codegen.lock. pipelex codegen check [DIR] verifies generated artifacts are current offline — pure hashing, no engine, network, or API key — reporting drift by category (missing, modified, hand-edited, orphan) with the verdict on the exit code (0 current · 1 drift · 2 no lock). Emission is idempotent (write-if-changed — unchanged output leaves files untouched) and prunes previously-generated files that drop out of the set, so a deleted concept never lingers as a stale generated file. Input templates (codegen inputs) are deliberately not stamped or locked — they are user-editable scaffolds, not tracked generated code.
  • Agent-CLI codegen mirror (pipelex-agent codegen types|check): The codegen family is now available to machine consumers through the agent CLI's two-stream envelopes (--format markdown|json success stream, --error-format error stream). codegen types runs the same engine and write-if-changed emission as the bare CLI; codegen check presents the offline drift verdict structurally — drift is a CodegenDriftError envelope enumerating drifts[] by category (exit 1), a missing or unreadable codegen.lock is a no-verdict error (exit 2). With the pipelex runner, mthds-agent codegen … forwards to these commands.
  • Host-facing resolve/codegen engine cores (what the HTTP routes ride): pipelex.pipeline.resolve_bundle.resolve_crate_from_contents resolves in-memory MTHDS contents into the normalized crate — same engine, same ValidateBundleError verdict vocabulary, and the same loaded-on-success library contract as validate_bundle — so a host (the pipelex-api /v1/resolve + /v1/codegen routes) serves resolution without touching the filesystem. pipelex.codegen.emission.build_stamped_projection is the new pure stamping core (stamped files + codegen.lock content, no disk writes) that write_stamped_projection now rides, so artifacts served over HTTP are byte-identical to locally written ones and pass the offline codegen check verbatim. The shared bundle-loading error cascade translate_to_validate_bundle_error is now public (renamed from its underscore-private form) since it gained a third entry point.
  • Agent bundle fixer: Added pipelex-agent fix bundle, a deterministic in-place fixer over the shared validation fix loop. The command supports --format/--error-format, repeatable --select/--ignore fix-rule filters, --allow-signatures, --max-iterations, and the same bundle-file or directory resolution as pipelex-agent validate bundle. Successful fix results report pending_signatures and is_runnable; by default the command exits non-zero when a fixed bundle remains valid-but-not-runnable.
  • Human bundle fixer: Added pipelex fix bundle, the human counterpart of the agent fixer over the same fix loop, with Rich-rendered output naming every change made (fix descriptions, files written, iteration count), the same --select/--ignore/--allow-signatures/--max-iterations options, and the same bundle-file or directory resolution as pipelex validate bundle. A new --diff flag previews the changes as a unified diff without writing anything — the real fix loop runs against a temp-copy sandbox, and exit codes keep the verdict semantics so --diff answers "would it converge?".
  • Suggested fixes surfaced in pipelex validate: When a validation error has a deterministic safe fix, the human error output now shows a per-error 💡 Suggested fix: line and ends with an actionable footer naming the exact pipelex fix bundle command (echoing the invocation's -L dirs) in place of the generic tip.
  • suggested_fix field in /validate payloads: Each validation error that has a deterministic safe fix now carries an additive structured suggested_fix object — fix_code (the kebab-case rule id), description (the author-facing 💡 text), safety (safe/unsafe), optional source file, and ops (the semantic patch operations) — on every validation_errors[] item, both in the pipelex-agent validate/fix JSON envelopes and in the API /validate response body. This is the machine-actionable counterpart of the 💡 line, letting a software consumer apply the fix directly from the structured payload.

  • Smart Inputs (signature-driven input shaping): Inputs are now interpreted top-down against the pipe's declared input signature rather than bottom-up from their shape alone.

  • Bare values (strings, numbers, booleans, dicts, lists) are automatically shaped into the declared concept (e.g., a bare string becomes a legal.Question, not a generic native.Text).

  • Lists shape element-wise into the declared item concept; single values auto-wrap into lists where required, and empty lists are now legal.
  • Bare URLs/paths for Image or Document inputs are resolved relative to the inputs file's directory.
  • Declared structured lists (e.g., Person[]) accept a direct path to a .csv file by signature, without requiring an envelope.
  • Comprehensive input error handling: A suite of typed errors for input shaping failures (WrongScalarKindError, ListWhereSingularError, MultiplicityCountMismatchError, StructureValidationError, UnknownInputNameError) that provide clear hints and render the expected JSON shape.
  • CLI --explicit flag: Added --explicit to pipelex build inputs and pipelex-agent inputs to generate the legacy {"concept", "content"} envelope templates.
  • Native YesNo concept: Built-in concept (backed by YesNoContent) for boolean judgments. Requires a strict boolean, renders as yes/no in downstream prompts, and can be read via pipe_output.main_stuff_as_yes_no.yes_no.
  • Native Date concept: Built-in concept (backed by DateContent) for calendar dates. Requires a datetime.date and accepts an optional datetime.time. Preserves source precision (never forcing the LLM to fabricate a time of day) and retains UTC offsets verbatim.
  • Native Time concept: Built-in concept (backed by TimeContent) for a time of day, optionally with a UTC offset — never attached to an invented date. A top-level TOML time-of-day literal in an inputs file now maps to Time (Breaking: Time joins the reserved native concept codes, and the former bare-time rejection — InputsTimeOnlyNotSupportedError — is gone).
  • Structure-field type = "time": The concept structure language gains a time field type (a time of day, optionally with a UTC offset), completing the temporal triple beside date and datetime. Generates a datetime.time field; emitted by every codegen target.
  • Pinned normative native definitions: Native concepts now materialize into the normalized crate by lookup into the standard's pinned per-version definitions (pipelex/core/concepts/native/pinned_blueprints.py, mirroring the mthds spec's new native-concepts.md), never by reflection over the runtime content classes — so crate fingerprints over materialized natives byte-agree across implementations. The old reflection is retained solely as a consistency test proving each runtime content class still matches its pinned blueprint.
  • Subject-grants registry — the keyword-only convention hardened: A positional first parameter ("subject") in pipelex/ source is now legal only under an explicit grant recorded in subject_grants.toml at the repo root, carrying the subject's param name and an honest, def-specific review rationale — every grant in the registry has been individually reviewed against the grant rubric. pipelex-dev subject-grant is the sole registry writer; the check-keyword-only AST guard enforces the registry symmetrically (an ungranted subject fails, and so does a stale grant whose def was renamed, moved, demoted, or newly carved out) in make check, CI, and the edit-time hook. Subjects typed bool/int/float are banned outright — no grant can cover them. The registry schema is strict: exactly param + rationale, unknown keys fail the check. See docs/contribute/keyword-only-arguments.md.

  • Drift contracts (pipelex-dev drift plan|check|ack): Deterministic review obligations between code and docs. A root drift.toml declares contracts ("when these trigger files change, these review targets must be re-examined and the examination recorded"); committed ack files under .drift/acks/ record each fulfilled review with a digest, reviewer, and rationale. A contract is fulfilled iff its stored ack digest equals the digest recomputed from the current tree (staged blob OIDs from the git index — no base ref, no timestamps), so trigger edits, matched-file adds/deletes, and contract-definition edits all reopen it.

  • make drift-check gates the make check aggregate and runs as an advisory CI job; make drift-plan prints per-contract Markdown packets with exact per-file changes since the last ack.

  • make drift-ack CONTRACT=… RATIONALE="…" runs the contract's verify commands, then records the review and stages the ack file it writes, so the local gate checks the same index the commit is built from. If staging that ack fails, the ack file is removed rather than left unstaged, so a later check reports a missing ack instead of a false green.
  • For a contract with verify commands, a trigger file left unstaged-modified or untracked is a hard ack error (the verify run would certify content the digest does not cover) — checked both before and after the verify commands, so a trigger the commands themselves format or generate is caught too. For other contracts it stays a warning.
  • Seeded with contracts for the config docs, the CLI docs, and the keyword-only convention spec; workflow documented in docs/contribute/drift-contracts.md.

Changed

• Keyword-only demotions across the public API (Breaking): The subject-grant review demoted every positional first parameter that failed the grant rubric to keyword-only, and external code calling these surfaces positionally (or subclassing them with positional signatures) breaks. Notable demoted families: the PipeAbstract run family (run_pipe, dry_run_pipe, validate_before_run, … — job_metadata/visited_pipes now keyword-only), StuffContent.rendered_for_prompt/rendered_for_template_async (text_format) and the TextFormatRenderable/ImageRenderable protocols with all their implementations, the LibraryManagerAbstract load family (library_id), ensure_pipelex_booted (config_overrides), pipeline_run_setup (execution_config), ConceptLibrary.is_compatible (tested_concept), GraphTracerProtocol.setup (graph_id), the PromptImageFactory/PromptDocumentFactory content factories (uri), the model-manager/backend-library setup+load (secrets_provider), the reporting protocol's set_event_log/clear_event_log (context_key), and assorted tools helpers (pluralize/count_with_noun, LogLevel.from_int, set_dry_run_forced, …).
• pipelex build structures is now an alias of codegen types --target python-structures (Breaking): The legacy always-qualified per-file generator is deleted. The alias resolves the target's whole directory closure and emits one stamped structures.py module plus a codegen.lock (bare-when-unique class names, declared imprecision instead of the old silent TextContent guess for structureless concepts). The --force flag is gone (emission is write-if-changed), a .mthds file target now resolves its parent directory's closure, and concepts-only loading of invalid-pipe bundles is no longer supported (the crate requires a valid library). The concepts-only loading API (load_concepts_only, load_concepts_only_from_directory, and the library-manager methods behind them) is removed with it.
• pipelex build runner emits its structures through the codegen engine (Breaking): The scaffolded structures/ directory now contains the stamped single-module projection (structures.py + codegen.lock) instead of per-concept files, and the generated runner script imports and instantiates the emitted class spellings (from structures.structures import Invoice) — fixing dropped imports for nested custom concepts along the way. A user class backing a structure = "<ClassName>" concept is imported from its real module.
• Human validation error rendering routed through the shared structured items: pipelex validate's error details now render from the same ValidationErrorItems the agent CLI and API emit, which surfaces information the old renderer silently dropped: pipe-factory errors (e.g. an unknown concept) get their own section, a parse-level failure (e.g. a TOML syntax error) now shows its message instead of no detail at all, and pipe validation errors name their source file. The dry-run message remains visible alongside categorized errors.
• Validation error messages now speak MTHDS author syntax: Pipe validation errors state the problem and the fix the way an author writes them — a sequence output-multiplicity mismatch now reads "declares its output as 'StoryIdea', but its last step yields 'StoryIdea[]'. Update the sequence's output to 'StoryIdea[]'", and an input-type mismatch reads "input 'dish' is declared as 'Number' but its step needs 'Text'. Update the input to 'Text'" — instead of leaking Python internals (multiplicity=None, Concept(...)/PresenceMarker reprs). The misleading (required: […], provided: …) suffix is suppressed for multiplicity errors (where both sides are identical by definition) and rendered as joined author-syntax refs — never Python list-repr brackets — elsewhere.
• Agent validate / fix output renders prose, not a JSON dump: pipelex-agent validate bundle and pipelex-agent fix bundle markdown output now renders validation errors as category-grouped prose (a humanized title, identity fields, the message, and a 💡 Suggested fix: line) followed by a fix-aware footer naming the exact pipelex-agent fix bundle command when a safe fix exists — mirroring the human panel. The machine-facing JSON stream (--format json) is unchanged, so software consumers keep branching on the same structured fields.
• Clean validation summary message on every surface: The top-level error message (agent JSON, the API /validate body) is now a clean, author-facing summary derived from the structured errors, instead of the raw pydantic dump (Validation error(s): … Value errors: '<field>': Value error, …). The API's opt-in invalid-verdict rendered_markdown now carries the same per-error 💡 Suggested fix prose as the CLI. A still-invalid fix that stops with no progress now names the re-proposed fix in author terms rather than dumping an internal fingerprint string.
• Input templates default to "light" shape (Breaking): pipelex build inputs and pipelex-agent inputs now generate light templates by default, outputting the bare values expected by the signature rather than the verbose {"concept", "content"} envelopes. TOML templates include the declared concept as a # concept: ... comment.
• Structure-field type = "date" behavior (Breaking): A date field now generates a datetime.date (JSON schema format: date), preventing LLMs from hallucinating times for calendar dates. A new type = "datetime" field handles full timestamps (format: date-time), and default-value validation now rejects a datetime default on a date field.
• Strict input validation (Breaking): Undeclared input names are now a hard error (previously silently ignored), and top-level null values are rejected — absence must be expressed by omitting the key. Known limitation: this also rejects the extra stuffs carried by a pipelex-agent run --with-memory envelope, so piping one run's full working memory into a downstream pipe that declares only a subset of it now fails with UnknownInputNameError. Until envelope-derived inputs get their own leniency, narrow the envelope to the declared inputs before piping, or pass the inputs explicitly with --inputs.
• Explicit envelope validation (Breaking): The {"concept", "content"} escape hatch is now compatibility-checked against the pipe's declared signature; incompatible concepts raise a typed error.
• TOML date/time handling (Breaking): TOML date and datetime literals are natively supported and map directly to the Date concept; a bare time-of-day literal maps to the new Time concept (it never silently becomes a Date — shaping a Time into a Date slot fails the compatibility check).
• ImageContent flattens its pixel dimensions (Breaking): size: ImageSize | None is replaced by paired optional width / height integer fields (both-or-neither, enforced by a validator) — a named nested shape in the crate is a concept, and the size grouping did not earn that citizenship. ImageSize remains as the internal img-gen parameter type. Image wire payloads change shape accordingly.
• native.Date declares real structure (Breaking): Materialized Date is no longer structureless — its pinned definition carries a required date field plus an optional time field (the new time field type), so generated clients get real Date types. DateContent's wire form is unchanged.

Fixed

• Library loaders now bind the target library for the whole load: load_from_blueprints / load_from_crate / load_libraries took an explicit library_id but resolved concepts and the class registry through the ambient current-library ContextVar one hop below — so loading into a library that wasn't current read (and registered generated structure classes into) whatever library the ambient pointer happened to name. Production hosts always set the target current before loading, so this only bit multi-library callers — exactly the isolation the loaders exist to provide. The loaders (and blueprint removal) now wrap their body in scoped_current_library(library_id), restoring the caller's binding on exit, so everything beneath a load resolves against the load's target by construction. A regression test loads into a deliberately non-current library.
• PipeExtract broke crate normalization (500s on every static-core route): PipeExtractBlueprint.validate_output only accepted the authoring spelling Page[], but normalization qualifies concept refs (Page[] → native.Page[]) and then rebuilds the blueprints — so the validator rejected its own normalized output, and any bundle containing a PipeExtract raised a raw pydantic ValidationError out of resolve_crate_from_contents, 500-ing /v1/build/inputs, /v1/build/output, /v1/resolve, and /v1/codegen. The validator now compares the parsed concept ref and multiplicity instead of the raw string, so it holds for both spellings. The output contract is unchanged and still exact — a variable-length list of native Page — since a refined output would be an unverifiable promise: PipeExtract mechanically produces native pages and cannot make them more specific.
• resolve / codegen rejected bundles that pin a model: The resolve and codegen commands booted without model specs, so library validation could not check pipe model pins (model = "gpt-4o-mini") against the deck and rejected any bundle carrying one. They now boot like validate — offline, but with model specs loaded.
• Native Image no longer materializes structureless: Every native now materializes from its pinned normative definition, so generated clients for image-producing pipes expose url, public_url, caption, width, height, etc. (The brief interim dict-with-imprecision mapping for the nested size model is superseded by the ImageContent flatten above.)
• Deterministic input-template placeholders: Mock URL placeholders in generated inputs templates (codegen inputs / build inputs) are now deterministic (https://mock.invalid/<field>), so committed templates no longer churn on every regeneration.
• codegen inputs is write-if-changed: Like codegen types, an already-current inputs template is left untouched (no mtime churn) and the console reports Unchanged instead of claiming Generated.
• Opaque generated classes no longer strip content: A structureless or Python-class-backed concept's generated Python class now carries model_config = ConfigDict(extra="allow"), so model_validate keeps the payload verbatim instead of pydantic's default extra="ignore" silently dropping every field. Opaque is pass-through, never lossy — matching the ts-zod z.unknown() behavior.
• Human CLI ~ expansion: pipelex validate bundle (and the new pipelex fix bundle) now expand ~ in the bundle path and -L/--library-dir values, matching the agent CLI.
• Validate telemetry label: The CLI_COMMAND telemetry tag for pipelex validate subcommands no longer double-suffixes (previously "validate bundle bundle").
• Silent mistyping of inputs: Providing a bare string for a refined text concept (e.g., legal.Question) no longer silently degrades the type to a generic native.Text; the declared concept type is now retained.
• TOML inline table formatting: Light templates now use inline tables for structured values, preventing trailing scalars from being swallowed into [sections].

Documentation

• Smart Inputs guides: Updated provide-inputs.md, executing-pipelines.md, and native-concepts.md for the Smart Inputs paradigm, leading with the bare-value approach and demoting the envelope format to an escape hatch.

[v0.38.0] - 2026-07-06

Highlights

• Optionality is a first-class language feature — Concept references in pipe inputs/outputs can be marked ? (optional) or ! (force), so absent values become tracked, first-class data with provenance instead of crashes or silent nulls. Absence flows through a runtime trichotomy — a plain input skips its pipe, a ? input absorbs it and runs, a ! input fails loudly — and a static taint pass proves at author time that every maybe-absent value reaches an explicit sink.
• Storage and secrets are now plugin seams — Both backends are selected at boot from a config-keyed registry, so third parties can ship pipelex-storage-<backend> / pipelex-secrets-<backend> packages via the pipelex.plugins entry point — the same discovery machinery the inference and orchestrator seams already use.

Added

• Optionals & presence markers (?, !): Concept references in pipe inputs/outputs can now be marked ? (optional) or ! (force), e.g. clause = "PenaltyClause?". Absent values are first-class data, tracked in working memory via an AbsenceRecord ledger (variable, producing pipe, kind = declared-absent / skipped / not-provided, reason, upstream chain) with provenance captured at the moment absence is produced:

  • A plain input fed an absence causes the pipe to be skipped (implicitly lifted); its own output is recorded absent, chaining back to the origin so the miss short-circuits like Swift's a?.b.c.
  • A ? input absorbs the absence and the pipe runs, handling both arms.
  • A ! input fed an absence fails loudly with a typed OptionalValueAbsentError naming the variable, the consuming pipe, the producing pipe, and the original reason.
  • Callers may omit ?-marked method inputs; the slot starts as a recorded not_provided absence rather than raising a missing-inputs error, and the missing-required-inputs message now names the optional inputs a caller may omit.
  • Post-run reads get a tri-state resolved accessor (WorkingMemory.resolve_stuff / resolve_main_stuff: a value or a recorded absence). A slot with neither a value nor a record is still a hard error — never-produced is a bug, not an absence. Markers round-trip through blueprints, builder specs, IO contracts, and bundle representations; grammar misuse (X[]?, ! on an output, markers on concept definitions) is rejected at parse with optional_marker_invalid.
  • Static optionality validation (the absence-taint pass): Validation proves the absence-safety theorem at author time — a taint pass over each controller's dataflow computes per-slot presence (guaranteed / maybe-absent) and rejects any maybe-absent slot that escapes without an explicit sink, with typed errors (optional_not_handled, optional_output_required, optional_branch_required_field) that each name the absence origin, the propagation path, and the fixes. The validation report also lists every liftable (skippable) pipe (liftable_pipes) — build-time visibility for "may be skipped when X is absent" — and gains a general warnings array for advisory lints that never flip is_valid (first occupant: optional_force_redundant, a ! whose slot is guaranteed present in every flow).
  • Template guard-lint (optional_input_unguarded): Every template reference to a declared-optional (?) input must be guarded — @?var, a {% if var %}…{% endif %} block, or an inline presence conditional — or validation fails with the precise fix. Applies to PipeLLM prompts and system prompts, PipeCompose templates, and PipeCondition expressions. In the same motion, @? finally means what it says: an optional variable is no longer presence-required by the controller miss-gates, so the pipe runs and its guarded templates take the absent arm.
  • Execution graph updates: GraphSpec nodes gain the skipped status with a skip_reason — "why did my workflow produce nothing?" is now answerable from the graph — and data edges fed by a declared-optional (?) output carry optional: true. Both flow through the in-process tracer and the distributed event-replay assembler (new pipe_end_skipped trace event).
  • Storage provider plugin seam: The storage backend (local / in_memory / s3 / gcp) is selected at boot from a StorageProviderRegistry keyed by the open storage_config.method token, populated by the always-on built-in StoragePlugin. Third parties can ship a pipelex-storage-<backend> package that advertises itself under the pipelex.plugins entry-point group — the same discovery/denylist machinery the inference and orchestrator seams use. storage_config.method is now an open str: an unknown method fails loudly at boot with UnknownStorageMethodError (listing the registered methods), not at config parse. See the new docs/under-the-hood/storage-provider-plugins.md.
  • Secrets provider plugin seam: The secrets backend is now a plugin seam selected via the new open secrets_config.method token (default "env") from a SecretsProviderRegistry, populated by the always-on built-in SecretsPlugin. Third parties can ship a pipelex-secrets-<backend> package (Vault, AWS Secrets Manager, …) under the pipelex.plugins entry-point group — the same mechanism the storage seam uses. Out-of-the-box behavior is unchanged: env stays the default. An unknown method fails loudly at boot with UnknownSecretsMethodError. See the new docs/under-the-hood/secrets-provider-plugins.md.
  • TOML pipeline inputs & templates: The --inputs file passed to pipelex run and pipelex-agent run (pipe/bundle/method) can now be TOML in addition to JSON, discriminated by file extension — TOML's multi-line strings make text-heavy inputs much easier to author. run bundle <dir> auto-detects inputs.toml alongside inputs.json when --inputs is omitted, erroring if both exist. Complementarily, pipelex build inputs and pipelex-agent inputs accept --format json|toml (default json) to emit the generated inputs template as TOML. Inline JSON ({…}) and agent-CLI stdin inputs stay JSON-only; a bare TOML datetime/date/time literal is rejected with an explicit "quote it as a string" error (native datetime concept support is a separate track).
  • Gateway models: Added support for nano-banana-2-lite through Pipelex Gateway.

Changed

• Dropped Python 3.10 support (Breaking): requires-python is now >=3.11,<3.15. The 3.10 compatibility bridges are gone — StrEnum, Self, and Traversable are imported directly from the stdlib at every call site, the pipelex.types re-export module that existed only to paper over 3.10 is deleted, and the backports.strenum conditional dependency (and its type stub) is dropped. CI test/lint matrices, the package-check requires-python floor gate, and the contributor docs now start at 3.11.
• Plugin API version bumped to 3 (Breaking): PLUGIN_API_VERSION is now 3 (was 2) to add the add_storage_provider and add_secrets_provider registrar menu methods. Plugin discovery version-checks with strict equality, so every external plugin must re-declare targets_api = 3. pipelex-mistralai-workflows registers no storage or secrets provider, so the targets_api bump is its only change. pipelex-temporal needs the bump and a code migration: its payload-codec factory imports the now-removed make_storage_provider_from_config (see Removed) and must switch to resolving the provider via the registry (get_storage_provider_registry().get_required(method=...)).
• PipeSignature is not a pipe type (Breaking): A signature is now declared by omitting type — a [pipe.x] section with no type and nothing but the contract (description, output, optional inputs, optional signature_for) is a PipeSignature. Writing type = "PipeSignature" explicitly is rejected with a migration error ("PipeSignature is no longer a pipe type — delete the type line"), and a typeless section declaring any non-contract field is a hard error naming the field. The pipelex-agent pipe authoring command mirrors this. MTHDS JSON Schema shape change: the signature arm no longer carries a type property (an explicit tag now fails schema validation), so the downstream schema copies (mthds, vscode-pipelex, mthds-ui) must be re-synced on the next release via the mthds-schema-sync skill.
• PipeCondition continue now resolves the output as absent (Breaking): The continue special outcome no longer passes the current main stuff through (and no longer errors when there is none). It records a declared-absent AbsenceRecord for the condition's declared output — with the evaluated expression as the reason — and returns success, memory otherwise unchanged. A continue-reachable PipeCondition must therefore declare its output optional (e.g. output = "Constraint?"), or validation rejects it (optional_output_required). Dry-run parity holds. Migration: the previous value stays in working memory under its own name, so downstream pipes consume it explicitly by that name (declared ? when it may be absent); a coalescing operator is the planned ergonomic replacement for pass-through.
• Controllers combine under absence: PipeParallel no longer crashes when a branch result resolves absent (lifted branch or continue): with a Composite output the absent component is omitted (ledger note kept), and with a structured output a non-required field absorbs the absence as its default (None unless the author declared another), while a required field fed an absent branch raises a typed error naming the branch, the field, and the fixes. PipeBatch compacts: absent branch results are dropped from the aggregated list, so batching a "keep or skip" condition over items yields only the kept results.
• Absent output delivery is now a first-class success: A run whose declared ? output resolves absent succeeds everywhere. main_stuff_name on PipelexPipeRunOutput and the /execute response names the declared output slot even when it resolved absent — consumers branch on the absence record in the serialized working memory (absences), never on transport, and the ledger round-trips cross-process (hydrate_working_memory reconstructs it). The main_stuff.json/md/html artifact files become an explicit absence document ({"absent": true} plus the provenance chain) on both the typed and raw delivery paths, pipelex run / --save-main-stuff and the agent CLI's run surface it instead of crashing, and OTel/Langfuse capture serializes it. A working memory with neither a value nor a recorded absence still fails delivery loudly.

Fixed

• Template truth-tests on singular values no longer crash: {% if var %} (and @?var, which expands to it) on a present singular value used to raise TypeError: '…' content does not support len(). — Jinja2's truth test fell through to the artefact's list-only __len__. StuffArtefact now defines __bool__: a present non-list artefact is truthy, a list artefact follows list emptiness. This makes the optionals guard idiom safe on both arms.
• PipeCompose escaped-sigil literals ($$, @@) no longer double-rewrite: PipeCompose alone stored its template already-preprocessed and then re-ran the sigil rewriter at guard-lint and render time. Because the escape collapse ($$name → $name) is not idempotent, the second pass resurrected escaped literals into interpolations — raising a spurious optional_input_unguarded on a $$name that only looks like an optional reference, and silently rendering the value of name instead of the literal $name. PipeCompose now stores authored source and rewrites exactly once, like every other pipe.
• --inputs ~/… now expands on run pipe / run bundle: a quoted or =-form tilde path (--inputs "~/inputs.json", which the shell leaves unexpanded) is now expanduser()-ed before loading, resolving to the home directory instead of failing on a literal ~ — matching the existing run method behavior, on both the human and agent CLIs.
• Invalid storage/secrets method errors stay actionable under STRICT: UnknownStorageMethodError / UnknownSecretsMethodError are marked caller-facing, so their "registered methods: … — check storage_config.method" guidance survives STRICT error disclosure instead of being redacted to a generic internal-error message.
• Non-string JSON pipe type gives an actionable error: a type that is a list/dict/number in a pipelex-agent pipe JSON spec now surfaces as an ArgumentError naming the valid pipe types, instead of a cryptic internal TypeError: unhashable type.
• GitHub Actions: Removed an invalid environment block from the manual-trigger-tests-check.yml workflow.

Documentation

• Optionality guide: New "Understanding Optionality" page (docs/building-methods/pipes/understanding-optionality.md) beside the multiplicity guide — presence markers, the runtime trichotomy (skip / run / fail), absence records and provenance, template guards, controllers under absence, and the static safety net. PipeBatch documents compaction under absence, PipeLLM documents the @? optional block sigil, and the run CLI page documents the absence artifact an absent main output produces.

Removed

• make_storage_provider_from_config (Breaking): The module-level storage-provider factory (pipelex/tools/storage/storage_provider_factory.py) is removed — storage is now resolved through the config-selected StorageProviderRegistry at boot. Downstream code that imported this helper (notably pipelex-temporal's payload-codec factory) must select through the registry instead.

[v0.37.0] - 2026-07-04

Highlights

• PipeParallel always combines — and every run has a main output — A parallel now always combines its branch outputs into its declared output concept (with the new native.Composite concept as the ready-made combination vehicle), the combined_output field is deleted from the MTHDS language, and the main-stuff invariant is enforced end to end: every completed pipe run delivers a main_stuff, so downstream surfaces (delivery, graph tracing, telemetry, wire models) can rely on it unconditionally.
• Portable, statically validated image generation — A new portable size parameter ("1k"/"2k"/"4k" tiers or exact pixel dimensions) carries the same size intent across providers, Gemini models gain image-to-image editing and extreme banner aspect ratios, and Google image models are now validated against declarative geometry rules at blueprint-load time — unsatisfiable requests fail fast instead of at the provider call.

Added

• Portable image size (size) for PipeImgGen: New size parameter accepting portable tiers ("1k", "2k", "4k") or exact pixel dimensions (e.g. "2048x1152"). A tier means "this pixel class at my chosen aspect_ratio", mapped to each provider's own grid; an exact size is deterministic (declaring aspect_ratio alongside it is a validation error). Unsatisfiable requests fail as hard validation errors at blueprint-load time, never warn-and-ignore. When size is unset, no size intent is sent and the provider default applies. The MTHDS JSON Schema exposes the field, and an optional size default is supported in [cogt.img_gen_config.img_gen_param_defaults].
• native.Composite concept: New native concept backed by CompositeContent — an untyped, named composition holding sub-contents as top-level fields, serving as the default combination vehicle for parallel branches. Supports the full content surface: smart_dump, kajson/transport round-trip, and markdown/HTML rendering.
• Gemini image-to-image (img2img) support: The Google img-gen worker now forwards input images to the Gemini API as inline parts, enabling image editing and multi-image composition for the nano-banana models (including the new nano-banana-2-lite). Every img-gen worker now validates input images against the model's declared capability at job start, rejecting img2img on unsupported models with a clean ImgGenParameterError before any provider call.
• Banner aspect ratios: Support for extreme banner formats (landscape_4_1, landscape_8_1, portrait_1_4, portrait_1_8) from Gemini 3.1 image models. All other image-gen backends reject them with a clean parameter error.
• Static validation for Google image models: Google image models now use rules blocks (geometry taxonomies like gemini_3_flash) in the backend deck, catching unsupported aspect-ratio and size combinations at blueprint-load time. The Google img-gen factory is keyed by taxonomy instead of hardcoded model names (Breaking: the GoogleImageGenModel name enum is removed — model handles are deck config, not code constants).

Changed

• PipeParallel always combines (Breaking): A PipeParallel controller now always combines its branch outputs into its declared output concept and stamps it as the main output. The declared output is strictly validated at author time and must be Composite or a structured concept whose fields and types match the branch result names. Combination replaces the removed combined_output field. Migration: pipes declaring both fields just drop the combined_output line; add_each_output-only pipes replace their placeholder output with Composite or a matching structured concept. add_each_output keeps its meaning and now defaults to false.
• Main-stuff invariant enforced (Breaking): Every pipe run now guarantees a main_stuff; defensive "maybe there is no main stuff" branches are removed, wire models make the field required (main_stuff_name: str on PipelexPipeRunOutput and the /execute response extension), and PipeOutput.optional_main_stuff is replaced by PipeOutput.main_stuff.
• Orchestrator SPI delivery split (Breaking): OrchestratorProtocol.run is now strictly the blocking arm (returning a completed PipelexPipeRunOutput). A new OrchestratorProtocol.start handles fire-and-forget, returning a PipelexPipeDispatchAck (IDs only). PipelexPipeRunOutput.is_completed is deleted — it is always a completed output now.
• LLMWorkerInternalAbstract folded (Breaking): Removed and folded into LLMWorkerAbstract, which now owns the whole job lifecycle (capability checks, constraints, telemetry). Subclasses extend LLMWorkerAbstract directly and implement _gen_text/_gen_object, nothing else.
• Type preservation across transport: Composite components (and their nested lists) now retain strict types across transport boundaries (dump_for_transport / hydration) via private class markers.
• PipeParallel honors final_stuff_code: A requested final stuff code (e.g. from a PipeBatch) is now correctly stamped on the parallel's combined output.
• Docs — inference plugins: Rewrote using-inference-plugins.md to demonstrate a genuine pipelex.plugins entry-point package instead of a legacy config workaround.
• Linting pipeline (repo-local): in the pipelex repo itself, plxt lint now validates .mthds files against the locally generated schema (derived/mthds_schema.json) rather than the released schema bundled with plxt. This is a .pipelex/plxt.toml override for this repo only — the plxt.toml template distributed by pipelex init config is unchanged.
• Dry-run mocks: Reduced the default dry-run mock list generation from 3 items to 2 to cut dry-run processing time.
• Bumped mthds dependency from >=0.6.0 to >=0.7.0.

Fixed

• PipeCondition pass-through failure: A continue outcome with no pre-existing main stuff now fails loudly and actionably at the pipe level instead of crashing downstream surfaces.
• Stale main stuff in parallels: A pipeline ending in an add_each_output-only PipeParallel no longer silently reports the previous step's output as its main result.
• add_each_output optional in the builder spec: PipeParallelSpec now defaults add_each_output to false like the blueprint does, so a generated always-combine spec with just branches and output no longer fails validation before to_blueprint().
• Google img-gen size handling: The native Google worker and gateway path now send the requested image_size to the Gemini API instead of hardcoding "1K", making 2K/4K generation reachable.
• Img2img support checks: The capability check now keys off the model's declared inputs rather than requiring an input_images rule, which had falsely reported Gemini models as unsupported.
• Routing profile optional routes: optional_routes declared in routing_profiles.toml are no longer silently dropped by the factory.
• Unknown boot orchestrator fails loud: Requesting a boot orchestrator no installed plugin provides — via --orchestrator <name> or Pipelex.make(boot_orchestrator=...) — now raises UnknownBootOrchestratorError at boot instead of silently falling back to in-process execution.
• Failed boot no longer leaks process-global state: A Pipelex.make that raises during setup now releases the process-global singletons a partial boot acquired (config, logging, the kajson class registry, template registries), so a failed boot no longer poisons a subsequent boot in the same process.
• HuggingFace streaming errors: The provider error-body reader now safely tolerates unread httpx streaming responses without crashing with httpx.ResponseNotRead.

Removed

• combined_output field (Breaking): Removed from the MTHDS language for PipeParallel; pipes now declare their combination target directly in the output field.
• Legacy external plugin setter (Breaking): Removed set_llm_worker_from_external_plugin from the inference manager, fully replaced by the pipelex.plugins entry point.

Security

• Transformers vulnerability: Bumped transformers past CVE-2026-4372 (GHSA-29pf-2h5f-8g72) to resolve a high-severity RCE. To unblock this, the huggingface extra now requires huggingface_hub>=1.5.0,<2.0.0 (Breaking).

[v0.36.0] - 2026-06-30

Highlights

• Orchestration plugin SPI — Pipelex opens its execution path to pluggable orchestrators: the orchestrator that runs a job is now chosen per call by an open orchestration_mode token, with the core shipping an in-process direct orchestrator and the contract ready for plugins to supply distributed backends.
• Orchestration plugin contract (orchestration_mode + DeliveryMode): The contract is built on two independent axes — an open orchestration_mode token (which orchestrator runs the job — direct in core, other modes supplied by plugins) and a closed DeliveryMode enum (the wait-semantics axis — BLOCKING / FIRE_AND_FORGET). An orchestrator plugin implements OrchestratorProtocol.run with a required delivery: DeliveryMode keyword and declares a supports_fire_and_forget: bool capability; the orchestrator registries are keyed by the orchestration_mode token. PipelexPipeRunInput carries orchestration_mode + delivery.
• Orchestrator-dispatched /validate: A new per-call bundle-validator seam (BundleValidatorProtocol / BundleValidatorRegistry) makes /validate dispatch by orchestration_mode the way /start runs a pipe — direct validates in-process, distributed modes dispatch the job to a worker — with a byte-identical verdict across backends. The core direct plugin registers an in-process DirectBundleValidator.
• Honest fire-and-forget delivery: A fire-and-forget /start request is honored only when the resolved orchestrator can do genuine async; an orchestrator that cannot (e.g. the in-process direct mode) returns a 4xx rather than running the job to completion and falsely acking.

Added

• Explicit graph targets for validation: Graph generation during validation and dry runs now accepts an explicit pipe_code target, so developers can generate graphs for a specific pipe in a bundle even when no main_pipe is declared. This is wired through the CLI (validate bundle --pipe <target> --graph/--view), dry_run_pipeline, generate_graph_for_bundle, generate_view_for_bundle, and PipelexMTHDSProtocol.validate(extra={"graph_pipe_code": "..."}).
• GraphSpec source path enrichment: GraphSpec artifacts now include the declaration source file path for pipes and concepts in pipe_registry and concept_registry, powered by LibraryCrate.source_map.
• CI/environment heartbeats: Added WAIT_WITH_HEARTBEAT and RUN_WITH_HEARTBEAT Makefile macros that emit a periodic heartbeat (default every 20s) to prevent CI runners and sandboxes from timing out during long tasks; applied to the agent-test, pyright, mypy, and pylint targets.
• Developer guidelines: Expanded python_standards.md with rules on filesystem paths (pathlib.Path over str/os.path), data-holder shapes (NamedTuple vs. pydantic.dataclasses vs. BaseModel), keyword-only arguments (enforced via make cko/make fko), and custom exception placement (exceptions.py or <topic>_exceptions.py).

Changed

• dry_run_pipeline return shape: dry_run_pipeline now returns the fully resolved, domain-qualified pipe reference alongside the GraphSpec.
• Graph viewer assets bumped: The CDN-pinned @pipelex/mthds-ui graph viewer used by the generated ReactFlow HTML is upgraded from 0.6.4 to 0.11.0, with refreshed Subresource Integrity hashes. 0.11.0 adds PipeStructure to the viewer's recognized pipe types, so graphs containing a PipeStructure node (e.g. from the preliminary_text structuring path) render instead of showing the GraphSpec validation error screen.

Fixed

• Graph generation on bundles without a main_pipe: Requesting a graph or view via the CLI no longer fails with a missing main_pipe error when a valid pipe is targeted with --pipe.
• Developer guidelines typo: Fixed a minor spacing typo in pytest_standards.md.

[v0.35.1] - 2026-06-22

Changed

• PipeImgGen documentation: Rewrote the docs to clarify how inputs are consumed. PipeImgGen has no dedicated "prompt concept"; it uses a prompt string template into which declared inputs are injected at runtime — Text variables are interpolated directly, while Image variables (single or lists) are injected as reference images to enable image-to-image and editing workflows. Added examples demonstrating this vision pattern.

Fixed

• Blueprint-stage validation error categorization: PipeValidationErrors raised during blueprint parsing (e.g. PipeBatch or SubPipe item name collisions) previously lost their error_type because Pydantic wrapped them in a generic value_error, degrading them to uncategorized residual errors. The categorizer now unwraps them, preserving their structured error_type, pipe_code, domain_code, and source locators.

Removed

• Native concept ImgGenPrompt (Breaking): Removed the built-in native.ImgGenPrompt concept. It was structurally identical to Text (mapped to TextContent) and added no unique semantics; PipeImgGen never depended on it. Migration: replace ImgGenPrompt (or refines = "ImgGenPrompt") with Text in your .mthds files. The internal ImgGenPrompt runtime model, the TemplateCategory.IMG_GEN_PROMPT category, and ImgGenPromptError are unchanged.
• Dead validation error type (Breaking): Removed the PipeValidationErrorType.img_gen_input_not_text_compatible enum value. It had no raise sites and contradicted the current design, where PipeImgGen accepts image inputs as a first-class feature.

[v0.35.0] - 2026-06-18

Added

• Structured validation errors: Bundle validation failures now emit categorized validation_errors[] items with identity locators, so machine consumers can read error_type and locators instead of parsing text. Categories: unresolved concept reference (pipe_validation/unresolved_concept with pipe_code, concept_code, field_name; or blueprint_validation/unresolved_concept with owning concept_code), undefined pipe dependency (pipe_validation/unresolved_pipe_dependency with pipe_code and the new missing_pipe_code locator), and unknown pipe type (blueprint_validation/unknown_pipe_type with pipe_code).
• New error locators and types: Added the missing_pipe_code locator field and the PipeValidationErrorType values unresolved_concept, unresolved_pipe_dependency, and unknown_pipe_type.
• String utilities: Added pluralize and count_with_noun to pipelex.tools.misc.string_utils for correct CLI output grammar (e.g. "1 pipe" instead of "1 pipe(s)").
• Documentation: Documented the 0/1/2 exit-code policy in docs/under-the-hood/error-model.md and added wip/structured-validation-errors-deferred-findings.md tracking deferred follow-ups.

Changed

• Validate exit-code policy (Breaking): Both pipelex validate and pipelex-agent validate now use a three-tier exit code mirroring the hosted /validate API: 0 = valid (including valid-but-not-runnable with --allow-signatures), 1 = negative verdict (invalid, or valid-but-not-runnable without --allow-signatures), 2 = no verdict (bad arguments, unresolvable target, or setup/internal errors). Consumers testing zero vs. non-zero are unaffected; machine consumers should rely on the structured is_valid JSON field.
• Relocated markdown renderer: Moved format_validate_markdown from CLI internals to the public pipelex.pipeline.validation_render module so other surfaces (e.g. the pipelex-api /validate route) can use it without Typer/CLI dependencies, and added render_invalid_validation_markdown for invalid verdicts.
• Error class hierarchy: ConceptLibraryError now extends LibraryLoadingError (instead of LibraryError) so it can carry per-reference structured items through the existing error cascade.
• Strict protocol arguments: PipelexMTHDSProtocol now rejects extra extension arguments, raising PipelineRequestError, since the local runtime defines no extension arguments.
• Bumped mthds dependency from >=0.4.1 to >=0.5.0.

Fixed

• Dropped pipe_code on blueprint errors: The blueprint validation-error categorizer matched the wrong Pydantic loc key (pipes instead of pipe), causing categorized items (e.g. missing_input_variable) to silently lose their pipe_code locator; it now populates correctly.
• CLI output grammar: Validation CLI output now uses the new string utilities, fixing awkward pluralizations (e.g. "Validated 1 pipe(s)" is now "Validated 1 pipe").

[v0.34.0] - 2026-06-17

Highlights

• Structured validation errors — validation failures now return a typed, per-error validation_errors[] on the error wire instead of a bare detail string, with the same shape across the HTTP API and the agent CLI.
• Structured validation_errors on the error wire: ErrorReport gains a typed validation_errors: list[ValidationErrorItem] | None field. ValidateBundleError.to_error_report() flattens its per-error data onto it via a single shared builder (pipelex/pipeline/validation_errors.py), so the structured error report an HTTP API surfaces carries machine-mappable per-error diagnostics — category, message, identity fields, and a source (declaring file) for cross-file mapping — instead of only a detail string. The closed ValidationErrorCategory set is blueprint_validation / pipe_factory / pipe_validation / dry_run; a residual dry-run failure (DryRunError / PipeRunError, no structured locator) is projected as one dry_run item only when no categorized error has data, so an invalid verdict always carries a non-empty validation_errors[] (the structured-info invariant) rather than a bare message. The same builder feeds the agent CLI's validation_errors JSON array, so the CLI and API structured shapes cannot drift. The list is surfaced under STRICT disclosure (_STRICT_KEPT_FIELDS) because it describes the caller's own submitted bundle, not server internals.
• Changed — agent CLI validation_errors shape: the validate commands' error envelope now omits null-valued keys per entry (exclude_none) and gains the previously-dropped source, field_name, and concept_code fields. Entries are guaranteed to carry category and message; all other keys are present only when populated, so consumers must treat them as optional rather than assume a fixed key set.
• Per-content source on the in-memory validate path: PipelexInterpreter.make_pipelex_bundle_blueprint(mthds_source=...) and validate_bundle(mthds_sources=...) let a caller attach a logical source to each in-memory bundle string, threaded into blueprint.source. A sourceless mthds_contents submission previously produced source=None, breaking cross-file diagnostics; a host (e.g. an HTTP API) can now pass per-item sources so the structured validation_errors carry a real owning file. The on-disk CLI path is unchanged (it already records real file paths).

• dry_run validation category: ValidationErrorCategory gains a dry_run value. A dry-run residual failure that previously produced a bare-message ValidateBundleError with an empty validation_errors[] now surfaces one structured dry_run item carrying the message (graph-level, so typically no source) — closing the structured-info gap that drove consumers to fabricate a category.

• Keyword-only argument enforcement — a mechanical guard enforces the keyword-only argument convention across pipelex/, end to end — local autofix through the CI gate.

• Keyword-only AST guard: A custom AST-based linter (pipelex-dev check-keyword-only / make cko) that mechanically enforces the keyword-only argument convention across pipelex/, wired into make check and make agent-check.
• Keyword-only auto-fix: A non-gating --fix mode (pipelex-dev check-keyword-only --fix / make fix-keyword-only / make fko) that rewrites every mechanically-fixable violation by inserting a bare * as far left as possible (after self/cls), re-parsing each rewrite before writing and reporting the shapes it can't fix mechanically. It runs early in make agent-check; the read-only check still runs last and owns the pass/fail gate.
• Claude Code hook: A PostToolUse bash hook (.claude/hooks/check-keyword-only.sh) that runs the guard on edited files for immediate blocking feedback to AI agents.
• CI integration: A dedicated lint-keyword-only job in the GitHub Actions linting workflow to block non-compliant signatures from merging.
• Convention documentation: New docs/contribute/keyword-only-arguments.md, plus updates to CLAUDE.md and AGENTS.md establishing the keyword-only convention as a standing rule.

Added

• is_valid on the canonical validation report: PipelexValidationReport gains is_valid: Literal[True] = True, the always-true discriminant of the valid arm of the hosted /validate response union (mirrored by pipelex-api's InvalidReport's Literal[False]). It sits beside is_runnable: a sound bundle may still be not-yet-runnable.
• Offline CLI unit tests: Coverage for the pipelex-internal logic behind the CLI commands — doctor diagnostics, run execution and its sync wrapper, the build codegen cores, the readiness gate, show/which, and the gateway/telemetry/signature error handlers (the spec'd CLI interface stays owned by the conformance suite).
• Offline inference & runtime unit tests: Coverage for layers that can break without a provider — structured-output↔instructor mode mapping, model-deck reference checks, the image-gen argument and worker-routing factories, gateway request-shaping and extract parsing, the Mistral factory, the local observer sink, the output renderer, builder spec validation / to_blueprint(), the pipeline runner's error paths, the TOML config-sync engine, and the storage config validators.

Changed

• [BREAKING] Keyword-only public API: Top-level public surfaces now require keyword arguments after the first parameter. A caller passing a second-or-later argument positionally must switch to keyword form — affected methods include Pipelex.make(integration_mode, *, ...), Pipelex.setup(integration_mode, *, ...), and PipelexHub.setup_config(config_cls, *, ...). Downstream consumers to check: pipelex-api, pipelex-worker, n8n-nodes-pipelex, cookbook, starter.
• Codebase-wide keyword-only refactor: The entire pipelex/ tree (leaf tools, domain core, inference layer, execution path, framework-sensitive packages) now places a bare * after the subject parameter, requiring all later arguments to be named at the call site (e.g. copy_file(src, target_path=dst, overwrite=False)).
• Signature subject corrections: Reordered parameters so the true semantic subject is the first positional argument, e.g. parse_pipe_spec(spec_data, *, pipe_type), hydrate_content(raw_content, *, concept), and write_manifest(manifest, *, deck_dir). parse_pipe_spec is consumed by the hosted runner API (pipelex-api), updated in lockstep.
• [BREAKING] Canonical protocol validation report: PipelexValidationReport is reworked into the canonical typed shape shared by every backend and moved to pipelex/pipeline/validation_report.py — a typed bundle_blueprint (replacing the untyped single-or-list blueprint dump), pipe_io_contracts keyed by namespaced pipe_ref (new pipelex/pipeline/pipe_io_contracts.py), validated_pipes, and the pending_signatures / is_runnable verdict. It is assembled in exactly one place (build_validation_report) with a single shared primary-blueprint rule (select_primary_blueprint).
• [BREAKING] validated_pipes key pipe_code → pipe_ref: the entry has always carried the namespaced pipe_ref (domain.code) under a key named for the wrong identity; the key now says what the value is.
• Protocol validate now produces a best-effort graph_spec: when the batch declares a main_pipe, the local runtime dry-runs it in-process and ships the resulting GraphSpec via one shared implementation (best_effort_graph_spec); a graph-arm domain failure degrades to graph_spec=None with validation still successful. (dry_run_pipe_in_process moved to pipelex/pipe_run/dry_run_in_process.py to break an import cycle.)
• [BREAKING] MTHDS_PROTOCOL_VERSION deleted: the hardcoded duplicate is gone from runner.py; the SDK's PROTOCOL_VERSION (mthds.protocol.protocol) is the single source of truth, imported directly by consumers including the hosted API.
• Additive multi-file library construction: Same-domain .mthds libraries can now be built as separate, additive files (forward-declared PipeSignature headers plus concrete definitions in sibling files), enabling parallel top-down construction. A PipeSignature and a same-code concrete pipe reconcile (contracts compared by normalized concept identity) instead of colliding; concept and qualified pipe references resolve against the merged library across sibling and separately-loaded files; domain description / system_prompt merge order-independently; and a successful validate reports library-wide pending_signatures plus an is_runnable verdict in both JSON and markdown.
• [BREAKING] PipeSignature evicted from the executable pipe taxonomy: a signature is a contract, not a way of running, so PipeType.PIPE_SIGNATURE and PipeCategory.PIPE_SIGNATURE are removed (any code matching on them must drop that arm). It stays a PipeAbstract subclass with pipe_category = None, and is_signature is now a class fact rather than an enum read.
• [BREAKING] Signatures are never a validation error (signatures-as-data): the validator no longer raises when a pipe reaches an unimplemented PipeSignature. Strict and lenient modes return the same report body — the assembled library's outstanding signatures via pending_signatures and is_runnable = not pending_signatures. allow_signatures narrows to a sweep-mechanics flag (whether signature pipes are mock-run and listed in validated_pipes); it no longer changes the verdict. The "is this a failure?" decision moves to the consumer: the whole-bundle / whole-library surfaces (pipelex validate, pipelex validate --all, pipelex-agent validate bundle / validate method / validate pipe --all) derive the exit code from the library-wide runnability verdict — strict by default, exiting non-zero on not is_runnable unless --allow-signatures, while still emitting the success envelope (carrying pending_signatures + is_runnable: false). Single-pipe surfaces make no library-wide runnability claim and never gate: bare validate pipe <code>, and validate bundle / validate method invoked with --pipe (the slice can be fully implemented even when unrelated placeholders remain elsewhere). The execute/run path is unchanged — running a stub still raises PipeSignatureNotExecutableError.
• [BREAKING] Host-wiring guards reclassified to PipelexUnexpectedError: validate_bundle / load_concepts_only's "provide exactly one of mthds_contents / mthds_file_path" guards (and the existing mthds_sources-length-mismatch guard) now raise PipelexUnexpectedError (→ 500), not ValidateBundleError — a caller wiring bug is a programmer error, not a content verdict to be reported as an invalid bundle. The empty-mthds_contents guard stays caller-facing.

Fixed

• Protocol model deck no longer loses aliases to cross-category collisions: PipelexModelDeck.aliases / waterfalls are now keyed by model category ({category: {alias: model}}) instead of being flattened with update(), which silently kept only the last category's entry for an alias name shared across categories. Breaking for the deck's extension shape.
• JSON-Schema rendering failures on the validate surfaces are now structured errors: a pydantic schema-generation failure while building pipe_io_contracts now raises the new PipeIOContractError (naming the offending pipe/input/concept) instead of a raw exception.
• Protocol validate teardown can no longer mask the real error: a raising library teardown is suppressed while a body error is propagating, and the validation library id is captured once so the graph arm and the teardown always target the same library.
• Empty mthds_contents is rejected with a structured error: validate_bundle and select_primary_blueprint now raise ValidateBundleError instead of crashing primary-blueprint selection with a raw IndexError; the hand-rolled first-declaring-main_pipe loops in execution_seams, dry_run_pipeline, and inputs_ops are folded into the single select_primary_blueprint.
• Structured-info invariant is now total — parse-level failures carry a structured item: a malformed .mthds file (a TOML-syntax error, an empty blueprint, or a bundle-elaborator failure) is raised with only a message and no categorized data, so an invalid verdict previously rode an empty validation_errors[] for the single most common failure mode. The shared builder (build_validation_error_items) gains a last-resort fallback_message residual: when no categorized error and no dry_run residual produced an item, it emits one blueprint_validation item carrying the message (no source, no error_type — the bundle could not become a blueprint at all). Both surfaces — ValidateBundleError.to_error_report() and the agent CLI's extract_validation_errors() — pass it, so every invalid verdict now carries a non-empty validation_errors[], never a bare message.
• Qualified same-domain pipe references resolve across files: a controller referencing a sibling-file pipe by qualified name (research.find_key_findings) is now deferred to the merged library like bare references; a reference no file declares is still rejected at load.
• Concepts-only loading validates concept references: the pipelex structures / load_concepts_only path now runs the cross-file concept-reference check (batching sibling files into one pass) instead of silently accepting a structure field pointing at an undeclared concept.
• Multi-file dependency packages reconcile signatures with their definitions: a dependency package split into a PipeSignature header plus its concrete sibling now goes through the same additive merge instead of colliding on the duplicate code and dropping one declaration by load order.
• OpenAI image moderation mapping was inverted: is_moderated=true sent the less restrictive moderation="low" and false sent "auto" — the mapping now matches the flag (enabled → "auto", disabled → "low"). ImgGenSetting.is_moderated also defaults to None, so workers omit the parameter and the provider's own default applies (which also stops force-disabling FAL's enable_safety_checker).
• Storage config validation now covers every provider and requires a real {hash} slot: the uri_format check is enforced uniformly across local / in-memory / S3 / GCP — every placeholder must be a plain supported {name}, a {hash} slot is required (GCP previously accepted the bare substring hash, silently overwriting every stored object), and buckets must set a positive signed_urls_lifespan_seconds. A misconfigured format now fails fast at Pipelex boot instead of at the first content store.
• Mistral chat requests now send the system message before the user message: make_simple_messages previously appended it after the user message, contradicting its own docstring and the OpenAI-typed sibling.
• String concept values in bundle specs are constructible again: a bare string in the ConceptSpec | str union now validates (no longer crashing the mode="before" validator) and passes through to the blueprint as the concept's description instead of into structure, where the loader rejected it.
• LocalObserver JSONL records always carry the true lifecycle event name: a payload's own event_type key can no longer overwrite the event name in the written record, which had broken event-type filtering for JSONL consumers.
• Tests under tests/**/build/ are no longer silently skipped: pytest's default norecursedirs includes build, so the whole tests/unit/pipelex/cli/commands/build/ tree was never collected; the config now overrides norecursedirs and sets testpaths = ["tests"].
• Anything-output option numbering is now deterministic: the output renderer sorts a PipeCondition's possible outputs by pipe code instead of iterating a set, so output_option_N / schema_option_N numbering is stable across runs.
• Documentation examples: Updated code snippets in docs/under-the-hood/ to match the new keyword-only signatures.
• Test mock assertions: Migrated mock assertions from .call_args.args to .call_args.kwargs to reflect keyword-only calls.

Removed

• [BREAKING] SignaturesNotAllowedError deleted: with signatures no longer a validation error, the strict-mode signature pre-pass (BundleValidator._signature_pre_pass), the signature_check_error field on ValidateBundleError, the handle_signatures_not_allowed_error CLI renderer, and the SignaturesNotAllowedError exception class itself are removed. Code that caught SignaturesNotAllowedError on the validate path should read the report's pending_signatures / is_runnable instead; the execute path's PipeSignatureNotExecutableError is unaffected.
• Keyword-only migration scaffolding: With the whole pipelex/ tree converted, the temporary baseline file and the pipelex-dev check-keyword-only --regen-baseline flag are removed; the guard now hard-blocks on any new violation rather than burning one down.

[v0.33.0] - 2026-06-11

Breaking Changes

• PipelexRunner → PipelexMTHDSProtocol — the runner class now implements the MTHDS Protocol (mthds.protocol.protocol.MTHDSProtocol, mthds 0.4.1). Method renames: execute_pipeline → execute, start_pipeline → start (still NotImplementedError locally). execute/start carry the protocol's basic args plus a generic extra passthrough — server-specific args (a client-supplied run id, callbacks, a stored method_id) ride extra, never named params. Response classes: PipelexPipelineExecuteResponse → PipelexRunResultExecute (subclasses RunResultExecute — pipeline_run_id + pipe_output), PipelexPipelineStartResponse → PipelexRunResultStart (subclasses RunResultStart — pipeline_run_id only). state / created_at / finished_at / main_stuff_name / workflow_id are pipelex extension fields on the protocol's base responses; the run identifier keeps the name pipeline_run_id everywhere.
• mthds pin → 0.4.1 — the restructured SDK (mthds.protocol + mthds.runners.api). The agent CLI's API run path uses MthdsAPIClient (mthds.runners.api.client) directly; domain shapes import from mthds.protocol.*, Dict wire models from mthds.runners.api.models.

Changed

• --dry-run now mocks at the cogt leaf instead of swapping out the operators. run_mode rides a new CogtRunParams carrier (derived from PipeRunParams.run_mode, stamped on every cogt assignment), so a DRY run mocks inside the inference leaf at zero AI cost, with no API keys and no storage IO (the img/extract DRY branches sit above the store step). ContentGeneratorDry is deleted: operators no longer swap generators (the base operator's dry path simply reuses the live path), and the in-process validation scopes pass an inline ContentGenerator. Object mocks are now schema-built; classes with exotic format constraints should declare examples/mock_format (a re-validation failure surfaces as a typed DryRunObjectFidelityError, and a deterministic mock-build failure as the new DryRunMockBuildError).
• Unified dry run — --mock-inference removed (breaking). There is now exactly one non-live run mode: --dry-run. The mock-inference mode is retired; its capability — non-zero synthetic usage so the cost report renders — survives as an internal is_mock_usage sub-flag of DRY on CogtRunParams (replacing is_mock_inference; setting it on a LIVE run is a validation error). It is exposed on the Python surface (PipelexMTHDSProtocol / execute / prepare_pipe_job via is_mock_usage=...) and as a hidden test-only CLI trigger, deliberately undocumented. Dry-run leaf coverage was already uniform across every operator, so MockInferenceUnsupportedError and its img-gen/extract/search guards are deleted; MockInferenceObjectFidelityError is renamed DryRunObjectFidelityError (mock-built objects are DRY-only now); the mock_inference usage sentinel is renamed mock_usage.
• Keyless boot (needs_inference=False) now forces every run to DRY instead of installing a mock generator. Generator selection is purely backend-keyed; the forced-DRY flag is consumed at PipeRunParamsFactory.make_run_params — the single writer of run_mode — so every execution entry point is covered, including the runtime bridge.

Removed

• dry_run_config.apply_to_jinja2_rendering config key: it was dead — PipeCompose renders templates directly and the jinja2 parse check survives in the templating leaf's DRY branch. Remove it from your .pipelex/pipelex.toml override if present (config is strict and will reject the unknown key).

Fixed

• Resubmitting a pipeline_run_id after its run finished no longer fails permanently. PipelineManager was the one per-run registry with no per-run removal: every run permanently stranded its key, so a second submission of the same pipeline_run_id against a long-lived server process raised PipelineManagerAlreadyExistsError — surfacing as an unrecoverable 500 from the hosted runner API until process restart. The registry entry is now freed on every exit path: execute's finally removes it after each run (success or failure), and pipeline_run_setup removes its own registration when setup fails after registering (the caller never learns the id on that path, so setup must self-clean). Serial resubmission of a completed/failed run id now succeeds; only genuinely concurrent same-id runs still collide — deliberately, because the collision raise fires before open_tracer and shields the live direct-mode tracer (keyed by the caller-suppliable pipeline_run_id) from open_tracer's stale-key pop-and-replace healing. The hosted API maps the remaining concurrent-duplicate case to 409 Conflict (pipelex-api change).

• dry_run_pipeline now owns its graph transport — graph generation no longer depends on the host's tracing_config. The function requests a graph explicitly (generate_graph=True) but relied on the host's configured tracing backend as the emit/assemble channel, so a host with tracing_config.is_enabled = false (e.g. pipelex-api's /validate in direct mode, where the web-app's Dry Run button lands) always got "Pipeline execution did not produce a graph spec" and a response without graph_spec. The run now traces through a scoped InMemoryEventLog — the graph comes back under any tracing config, and validation dry-runs stop writing NDJSON/DynamoDB trace events as a side effect when tracing IS enabled. The no-graph contract violation is now a typed DryRunGraphNotProducedError (was a bare PipelexError, ambiguous in host logs that record only the exception type) — raised by both dry_run_pipeline and dry_run_pipe_in_process.

• tz-aware datetime payloads no longer fail to decode on hosts without a timezone database — kajson bumped to 0.7.0. kajson ≤ 0.6.0 decoded timezone-aware datetimes via ZoneInfo(...), which needs the IANA tz database: system tz files or the tzdata package. On hosts with neither (slim containers, uv-managed standalone Pythons — including the CI test runner, where tzdata only arrived transitively via pandas on Python < 3.11), decoding any aware-datetime payload raised ZoneInfoNotFoundError. Fixed upstream in kajson 0.7.0: its self-sufficient timezone wire format carries the UTC offset (decoding degrades gracefully to a fixed offset instead of raising, and plain UTC decodes with zero tz-database dependence), and kajson now declares tzdata as its own runtime dependency, so a tz database is always available for named zones.

Added

• Protocol validate / models / version on PipelexMTHDSProtocol — validate wraps validate_bundle (blueprints + per-pipe structures into the protocol ValidationReport); models wraps the builder's list_models into a ModelDeck; version reports protocol_version 0.6.0 with the installed pipelex version.

[v0.32.1] - 2026-06-09

Added

• render_cost_report_for_output(pipe_output) submitter helper. A one-arg convenience over render_run_cost_report that unpacks pipeline_run_id and tokens_usages from a finished PipeOutput and derives the --costs gate from the output itself — pipe_output.tokens_usages is None is exactly the signal that cost reporting was off for the run, the decision the runner already resolved (with all --costs/--no-costs overrides applied) and recorded on the output. Embedders and the pipelex run CLI no longer re-derive the three primitive arguments by hand or re-read global config to reconstruct the gate. render_run_cost_report is unchanged as the low-level primitive for paths where the three values come from different places (the agent CLI's build_cost_summary JSON envelope, distributed reassembly).

Changed

• typer is now capped (>=0.16,<0.27). pipelex subclasses typer's TyperGroup and overrides make_context, so it is coupled to typer/click internals and a future typer minor can break the CLI. The cap is tested through 0.26; a new dependency-canary CI job resolves the latest dependencies on a schedule so the bound is bumped deliberately rather than discovered in a user's install.

Fixed

• CLI no longer crashes under newer typer/click. Every pipelex subcommand (build, run, validate, init, worker, …) raised RuntimeError: There is no active click context and exited 1 when run against typer >= 0.26 / click >= 8.4 — the versions a fresh pip install pipelex resolves. The root app_callback fetched the context via the global click.get_current_context() instead of the ctx Typer already injects; the global context stack isn't populated when a subcommand is dispatched under those versions. It now uses the injected ctx, which is version-robust. The sibling --traceback flag had the same flaw — is_traceback_requested() read it via click.get_current_context(), so under those versions it was silently ignored on handled errors — and is now recorded at parse time in a context-independent flag. A subprocess CLI smoke test (tests/unit/pipelex/cli/test_cli_entrypoint_smoke.py) guards every subcommand against regressing.

[v0.32.0] - 2026-06-09

This cycle reworks cost reporting so it stops leaking. Cost now rides on the run result instead of a side buffer: the success-path registry leak is gone by removal, and a new --costs switch decouples cost collection from --graph. It also extracts the framework-agnostic runtime bridge so any host runtime — not just Mistral Workflows — can embed Pipelex through one boundary.

Added

• --costs / --no-costs (default on). pipelex run pipe|method|bundle gains a dedicated cost switch that emits usage tracing events and renders the end-of-run cost report. It rides the shared trace-event transport independently of --graph, so --no-graph --costs reports cost without building a graph and --graph --no-costs builds a graph with no cost report. It replaces the removed --cost-report (see Changed).

• tokens_usages on PipeOutput. New tokens_usages: list[AnyTokensUsage] | None and usage_assembly_error: str | None fields, mirroring graph_spec / graph_assembly_error. Cost is now part of the run result and is exposed automatically wherever a PipeOutput is returned, including the Pipelex API response. Render it with CostRegistry.generate_report(tokens_usages=...).

• --mock-inference. A LIVE run that fakes AI calls at the inference leaf with reportable synthetic usage, so cost reporting can be validated cheaply and deterministically without billing tokens. Mutually exclusive with --dry-run. It covers the LLM leaf; image-generation / extract / search under --mock-inference fail loud with MockInferenceUnsupportedError (pointing at --dry-run) rather than silently calling the real provider.

• Cost report in the agent CLI JSON. pipelex-agent run ... --with-memory attaches a best-effort structured cost_report ({total_cost, by_model}, real USD) to its JSON envelope when the run did reportable work and summary aggregation succeeds — treat it as optional (absent for dry runs, --no-costs, the API-runner path, or an aggregation failure). The agent surface stays JSON-only — no Rich table on stderr; compact mode is unchanged.

Changed

• Mistral Workflows integration extracted into a dedicated package. The optional pipelex[mistralai-workflows] extra and the pipelex.plugins.mistralai_workflows.* modules have been removed from pipelex. The Mistral Workflows integration now ships separately as its own package, pipelex-mistralai-workflows. The framework-agnostic runtime-bridge core (boundary types, run_pipe_via_bridge, PipelexExecutionMode, ensure_pipelex_booted) has been promoted from pipelex.plugins.mistralai_workflows.* to pipelex.runtime_bridge.* so any host runtime — not just Mistral Workflows — can embed Pipelex. No behavior changes; activities, boundary types, and execution modes are identical.

• --cost-report removed, folded into --costs. Breaking: --cost-report/--no-cost-report is gone from run pipe|method|bundle. Use --costs (default on) instead.

• Cost reporting is event-sourced; the submitter-side UsageRegistry is removed. The cost report is rendered from PipeOutput.tokens_usages, not from an in-process registry. ReportingProtocol (and ReportingNoOp) no longer expose open_registry / close_registry / generate_report / inject_tokens_usages, and the UsageRegistry model is gone. Embedders that rendered cost via get_report_delegate().generate_report() must render from the returned pipe_output.tokens_usages instead.

• is_log_costs_to_console now defaults true. With --costs on, the CLI prints the cost table at end of run by default (parity with --graph producing visible output). Per-inference-job console logging is removed — the report renders once at the end, not per call. Library embedders who don't want console output set it false.

• Dry runs never emit a cost report; free-model runs still do. Suppression keys on whether the run did reportable work (any tokens or any cost), not on total cost alone: a dry run (zero tokens, zero cost) is suppressed, while a real run on a free / zero-price model (e.g. Ollama) still reports its token usage with a zero total.

Fixed

• Docs deploy crashed on a fresh runner with InferenceSetupRequiredError. pipelex-dev generate-error-pages (a prerequisite of every docs-* make target) bootstrapped Pipelex with needs_inference=True, so on a CI runner with the gateway enabled but no on-disk service config it hit the first-run inference-setup gate and exited non-zero — breaking the Deploy docs workflow on main. The command only introspects PipelexError subclasses to write markdown and never calls inference, so it now bootstraps with needs_inference=False.

• UsageRegistry success-path leak. The per-run cost registry was opened during run setup but closed only on the failure path, so every successful run leaked its registry — and in a long-lived process (e.g. the Pipelex API) reusing a pipeline_run_id could collide on the orphaned registry. The registry is removed entirely, so the leak is structurally impossible.

[v0.31.0] - 2026-06-04

This release hardens Pipelex at its edges. The headliners: a full error-handling overhaul that gives every error a stable, RFC 7807-shaped identity and carries it all the way out to webhooks and external surfaces; lenient validation with pipe signatures, so you can sketch and dry-run a whole pipeline top-down before a single pipe is implemented; and a first, experimental cut of CSV tabular support for reading and writing typed lists straight from .csv.

Added

• Stable error identity (RFC 7807). Every PipelexError now exposes title() and a stable type_uri() — a real, dereferenceable https://docs.pipelex.com/latest/errors/<error>/ URL — and every ErrorReport carries both as populated fields, so consumers read report.title / report.type_uri directly instead of humanizing class names. ErrorReport.to_problem_document() renders an application/problem+json document with no web-framework dependency, and a new DisclosureMode (VERBOSE / STRICT) controls how much leaks onto external surfaces: STRICT drops provider/model attribution and redacts any message that wasn't authored to be caller-facing, keeping only the stable identifiers.

• Structured error payloads on failure webhooks. When a run fails, the delivery webhook now includes an error object — the full ErrorReport as a dict — so receivers can rehydrate it with ErrorReport.from_dict(...), render an RFC 7807 response, or route on error_domain / retryable. A WebhookTarget.payload that collides with a Pipelex-reserved key (pipeline_run_id, status, result_url, error) is now rejected at construction.

• Per-class error reference pages. Every PipelexError subclass now has a generated reference page under docs/errors/, surfaced in the docs site as "Error Reference" — so a type_uri dereferences straight to a populated page. Regenerate with the new pipelex-dev generate-error-pages command (make generate-error-pages, alias make gep); pages a maintainer claims with a <!-- pipelex:authored --> marker are preserved across runs.

• request_id on JobMetadata. An optional caller-supplied request id, threaded through the run and into the log context. Set it at dispatch with pipeline_run_setup(..., request_id="...") and read it back off job_metadata.request_id.

• PipeSignature — contract-only pipes for top-down design. A new pipe type (type = "PipeSignature") declares a pipe's inputs, output, and description with no implementation, so an author or agent can sketch a complete pipeline before committing to the operator that will eventually do the work. At dry-run time a signature mints a mock output matching its declared type and multiplicity; at runtime it raises PipeSignatureNotExecutableError. The optional signature_for field hints which pipe type the stub stands in for. See Signature Pipes.

• Lenient validation with --allow-signatures. pipelex validate pipe|bundle and every pipelex-agent validate subcommand take --allow-signatures (default off) to dry-run a pipeline that still contains PipeSignature stubs. Strict by default: without the flag, validation refuses any pipeline whose dependency graph reaches a signature and raises SignaturesNotAllowedError, reporting every reachable stub plus the controller chain that leads to it — so you know exactly which pipes are still placeholders.

• CSV tabular support (experimental). Read a .csv straight into a typed ListContent[YourConcept]: point an inputs.json reference at a .csv whose column headers match the concept's field names, and each row becomes an instance, with cells coerced via Pydantic (birth_year → int, ISO dates → date; an empty cell becomes None, so the field must be optional). Write a flat-list output back out with the new --save-csv <path> flag on pipelex run pipe|bundle|method. v1 is deliberately narrow — local file paths only (remote URLs with a tabular suffix are rejected), the row concept must be flat (scalar fields only; a nested/list/concept-typed field is rejected with a clear error naming it), and .xlsx is recognized but routed to a "needs pipelex[tabular]" message (the Excel backend isn't built yet). See the CSV Input & Output guide.

• --traceback CLI flag for full stack traces. By default CLI commands print a friendly one-line error; --traceback also prints the Rich-rendered stack trace before it. It's position-agnostic — pipelex run --traceback pipe ... and pipelex run pipe ... --traceback both work.

• Test tooling for hanging runs. New make agent-test-debug (alias make atd) runs the suite with upfront stale-process cleanup, an outer wall-clock timeout, and live per-test logging — for when make agent-test hangs or fails opaquely. Paired with a debugging playbook at docs/agents/debugging-hanging-pytest-runs.md.

Changed

• ErrorReport is now a frozen Pydantic BaseModel (was a frozen Pydantic dataclass) — still immutable and still round-trips through to_dict() / from_dict(), but an attempted mutation now raises pydantic.ValidationError instead of dataclasses.FrozenInstanceError.

• Dry-run and validation consolidated into BundleValidator. The standalone dry_run_pipe / dry_run_pipes functions and the modules pipelex/pipe_run/dry_run.py, dry_run_with_graph.py, and dry_pipe_router.py are gone; their work now lives in BundleValidator, and validate_bundle / BundleValidator gained the allow_signatures: bool = False flag (strict by default). Validation ordering and outputs are unchanged — only the import surface moved. Callers importing from pipelex.pipe_run.dry_run must switch to BundleValidator.

• validated_pipes now identifies every pipe by its qualified pipe_ref (domain.code). Previously validate pipe reported the bare code while validate bundle / validate all reported the namespaced ref, so the same pipe could appear under two identities depending on which command produced it. Every validate surface now emits the qualified ref, which can't collide across domains. Consumers that parsed pipe_code expecting the bare form (e.g. the MTHDS skills) must match on the namespaced ref.

• Gemini deck refresh. Google shut down gemini-3-pro-preview, so the gemini-3.0-pro handle is removed across the google, portkey, and openrouter backends (and the test-profile collections) — use gemini-3.1-pro (→ gemini-3.1-pro-preview) instead, now registered on portkey too for parity with google. The gemini-3.0-flash-preview handle is renamed to gemini-3.0-flash. The Pipelex Gateway lists models from its own remote config, so drop gemini-3.0-pro there separately.

• Filesystem path helpers moved to pathlib.Path. Helpers in pipelex.tools.misc.file_utils and pipelex.tools.misc.json_utils (save_text_to_path, load_json_from_path, load_binary, copy_file, ensure_directory_exists, get_incremental_file_path, and the rest) now take and return Path instead of str; path handling is Path-based throughout pipelex/, converting to/from str only at boundaries. Callers passing bare strings must wrap them with Path(...).

• teardown_current_library() renamed to clear_current_library() in pipelex.hub, pairing it cleanly with set_current_library().

• Dev tooling: pyright bumped 1.1.408 → 1.1.410 (drove two behavior-neutral internal adjustments).

Fixed

• The generated MTHDS schema now requires type on every pipe. In the schema consumed by plxt lint and the VS Code Taplo LSP, each pipe blueprint variant declared type with a literal default — so it was optional, and because the Draft-4 export drops the union discriminator, a pipe table written without type matched several oneOf branches at once and was rejected with an ambiguous multi-match (worse, a type-less table carrying fields unique to one variant validated silently). type is now required on every variant, so a type-less pipe table fails with a clear "missing type" and a typed table resolves to exactly one variant. The patched set derives from PipeBlueprintUnion, so new pipe types are covered automatically. Regenerate with pipelex-dev generate-mthds-schema.

• InputStuffSpecsFactoryError was shadowed by a duplicate class definition. input_stuff_specs_factory.py declared a local class with the same name as the canonical one in exceptions.py, leaving two distinct class objects in play so an except on one would miss the other. Consolidated to the single canonical class.

Security

• urllib3 floor >=2.7.0 to patch two high-severity issues: CVE-2026-44431 (GHSA-qccp-gfcp-xxvc) forwards sensitive headers across origins on proxied low-level redirects, and CVE-2026-44432 (GHSA-mf9v-mfxr-j63j) bypasses the decompression-bomb safeguards in parts of the streaming API. Floor added to the runtime dependencies.
• pymdown-extensions floor >=10.21.3 to patch CVE-2026-46338 (GHSA-62q4-447f-wv8h, medium): a regression in pymdownx.snippets reintroduced the sibling-prefix path-traversal bypass despite restrict_base_path. Docs-only; floor added to the docs extra.
• idna floor >=3.15 (lockfile resolves to 3.18) to patch CVE-2026-45409 (GHSA-65pc-fj4g-8rjx, medium): specially crafted inputs to idna.encode() could bypass the CVE-2024-3651 fix. Floor added to the runtime dependencies.
• Proactive floor bumps in the same hardening pass, past known-vulnerable releases: pillow >=12.1.1, protobuf >=6.33.5, python-dotenv >=1.2.2, requests >=2.33.0 (runtime); Pygments >=2.20.0 (docs extra).

[v0.30.3] - 2026-05-28

Added

• claude-4.8-opus model added to the anthropic and bedrock backends. Adaptive thinking mode, supports text / images / pdf in and structured outputs, max_prompt_images = 100, max_tokens = 128000, costs { input = 5.0, output = 25.0 } per million tokens. Carries the temperature_unsupported constraint like claude-4.7-opus. Also available via the Pipelex Gateway.

[v0.30.2] - 2026-05-26

Changed

• pipelex-agent markdown error envelope no longer includes the ## Error source stack-frame section. Markdown is the agent / human-facing channel; internal frames like LibraryError @ pipelex/libraries/library.py:140 are noise for an LLM trying to fix a .mthds file and forced every consumer (e.g. the mthds-plugins validate hook) to strip them. The error_source field remains in the JSON envelope (--error-format json) untouched for programmatic consumers — same shape, same cause-chain ordering — so any tooling that parses JSON keeps the full diagnostic surface. The ## Details section (which carries error_domain and other structured fields) is unchanged. No change to error categorization, error_domain values, message wording, or stdout/stderr routing.

[v0.30.1] - 2026-05-26

Fixed

• pipelex-agent now silences every Python logger on stderr regardless of user TOML. The agent CLI is machine-consumed: stdout is reserved for the structured success envelope (JSON / markdown) and stderr for the structured error envelope. Free-floating log.* calls — a log.debug from telemetry_factory.py, a log.warning from validation_error_categorizer.py, or an INFO/WARNING line from any third-party dep (anthropic, httpx, botocore, openai, anything a transitive dep configures) — would corrupt the stderr channel for downstream parsers (mthds-js's PipelexRunner doing JSON.parse(stderr) on the validate hook). Two layers, defense-in-depth:
• Layer 1 — pin pipelex's own logs off via config. make_pipelex_for_agent_cli injects config_overrides into Pipelex.make() that pin default_log_level = OFF and package_log_levels.pipelex = OFF from the very first log.configure call. A user setting [pipelex.log_config.package_log_levels] pipelex = "DEBUG" in ~/.pipelex/pipelex.toml can no longer leak its own DEBUG/INFO/WARNING lines.
• Layer 2 — process-global cutoff that covers every logger. New silence_logging_for_agent_cli() calls logging.disable(sys.maxsize) — a process-global threshold checked inside Logger.isEnabledFor BEFORE any per-logger level. No record gets created for any logger at any level (including custom levels above CRITICAL), regardless of which package emits or what level the user configured. Wired into the Typer app_callback so every subcommand — including init and accept-gateway-terms, which bypass make_pipelex_for_agent_cli — silences logging before any command body runs; idempotent per-call invocations remain inside make_pipelex_for_agent_cli and agent_doctor_cmd as belt-and-braces for direct library callers.

A new e2e regression test pins the contract by setting anthropic, httpx, botocore, openai to DEBUG in the user TOML and asserting both stdout and stderr stay clean.

Changed

• pipelex-agent no longer accepts --log-level. Log suppression is unconditional by design — there is no verbosity setting on the agent CLI. The log_level parameter is removed from make_pipelex_for_agent_cli() and apply_agent_cli_output_discipline(), and the corresponding ctx.obj["log_level"] plumbing is gone from every caller. For verbose debugging, use the human pipelex CLI, which honors the user's TOML log config.

• pipelex/cli/commands/doctor_cmd.py::setup_doctor_runtime now deep_updates log_config_overrides into the loaded config (was a flat-merge that silently replaced nested dicts like package_log_levels). With the flat merge, pinning package_log_levels.pipelex = OFF for the agent doctor path would have wiped out all the third-party package levels (anthropic, asyncio, botocore, ...) shipped in the default config. The deep merge preserves them.

• Removed ctx: typer.Context from 8 agent CLI commands that no longer used it. validate/{pipe,bundle,method}_cmd.py, inputs/{pipe,bundle,method}_cmd.py, models_cmd.py, check_model_cmd.py had ctx.obj["log_level"] as their only ctx usage; with --log-level gone, the parameter became dead weight. The run/ commands keep ctx because they still read ctx.obj["runner"]. Test callers (and the now-unused agent_ctx conftest fixture) updated to match.

[v0.30.0] - 2026-05-25

Fixed

• console_log_target package default is now stderr (was stdout). Logs now stay off the data channel by default, matching the intent of PR #452 ("default to stderr for outputs happening before initialization"). Downstream tooling that parses pipelex / pipelex-agent stdout as JSON (e.g. mthds-js's PipelexRunner) is no longer at risk of stdout pollution from package-level logs — the bug was latent for stock installs because the agent-CLI JSON paths happen not to log at INFO+, but surfaced for anyone who raised package_log_levels.pipelex to DEBUG or added a setup-time log on the command path. The same flip is applied to the kit template (pipelex/kit/configs/pipelex.toml) that pipelex init copies to ~/.pipelex/. Note: console_print_target is intentionally left at stdout — the main pipelex CLI emits human-facing tables (show backends, show models, which, doctor) via that channel, and downstream piping (pipelex show backends > out.txt) must keep working.

• pipelex-agent now pins both console targets to stderr regardless of user config. make_pipelex_for_agent_cli injects config_overrides into Pipelex.make() that force console_log_target = "stderr" and console_print_target = "stderr" from the very first log/print fired during init, so a user override of either knob in ~/.pipelex/pipelex.toml can no longer leak diagnostics onto the agent CLI's JSON data channel. Defense-in-depth post-init calls to log.redirect_to_stderr() and get_pipelex_hub().set_console_print_target(STDERR) remain. A new adversarial E2E test (tests/e2e/agent_cli/test_stdout_is_clean_json.py::test_models_json_stdout_resists_user_targets_override_to_stdout) pins the contract by overriding both targets to stdout and package_log_levels.pipelex to DEBUG and asserting json.loads(stdout) still parses.

[v0.29.1] - 2026-05-21

Fixed

• pipelex run now prints the aggregated cost table when [pipelex.reporting_config].is_log_costs_to_console = true, with --cost-report/--no-cost-report to override per invocation. The cost-tracking.md and reporting-config.md docs both promised a summary cost table at the end of a CLI run, but pipelex/cli/commands/run/_run_core.py never called get_report_delegate().generate_report() — the flag only triggered log.verbose(...) lines per inference job, which the default INFO log level swallows, so the table never appeared. The CLI now calls generate_report() after a successful run when either is_log_costs_to_console or is_generate_cost_report_file_enabled is true, so the Rich table (one row per model, plus a totals row) prints right before the "✓ Pipeline execution completed successfully" recap — and the CSV export branch finally fires too. The new --cost-report/--no-cost-report tri-state flag (default unset → use config) lets you force the cost table on for a single invocation (--cost-report) or skip reporting entirely — no Rich table and no CSV file (--no-cost-report) — without touching .pipelex/pipelex.toml. Applies to pipelex run bundle, pipelex run pipe, and pipelex run method; works in dry-run mode as well (synthetic usage rows).

[v0.29.0] - 2026-05-20

Added

• gemini-3.5-flash model added to the google backend. Adaptive thinking mode, supports text / images / pdf in and structured outputs, max_prompt_images = 3000, costs { input = 1.5, output = 9.0 } per million tokens.

• New PipeStructure operator that turns text into a structured concept via a single LLM call. Takes one Text-compatible input (or a domain concept that refines = "Text"), produces any structured output with the usual multiplicity options (Foo, Foo[], Foo[N]). Useful whenever the source text comes from a PDF extraction, a search result, an upstream pipe, or any non-LLM origin. Documented at building-methods/pipes/pipe-operators/PipeStructure.md.

• Documented render_js and include_raw_html on PipeExtract. Both fields were already shipping on PipeExtractBlueprint; only docs were missing. render_js = true asks the extraction backend to render JavaScript before fetching web-page content; include_raw_html = true populates each extracted Page's raw_html field with the fetched HTML. Added to building-methods/pipes/pipe-operators/PipeExtract.md.

• Documented XHIGH value on ReasoningEffort. The enum already shipped seven levels; the under-the-hood/reasoning-controls.md table listed only six. XHIGH sits between HIGH and MAX and maps to provider-specific xhigh values where supported.

• Bounded fan-out concurrency for PipeBatch. A PipeBatch over many items no longer spawns every branch — every coroutine, every deep-copied working memory, every inference call — at once. Branches now run in bounded chunks driven by the new [pipelex.pipeline_execution_config] setting max_concurrency (default 8; set to the literal "unbounded" for unbounded fan-out, the previous behavior). It keeps a large workload (one pipe over thousands of documents) from overwhelming asyncio, memory, and provider rate limits. Results still preserve input order, and a branch failure still propagates (first error by input index wins). PipeParallel, which fans over a fixed pipe-defined branch set, is unchanged.

• Authoritative error_domain → HTTP-status mapping. pipelex/base_exceptions.py now owns the mapping that downstream HTTP APIs (pipelex-relay, pipelex-back-office) need to render an ErrorReport as an HTTP response: error_domain_to_http_status() (the pure domain table — INPUT → 422, CONFIG/RUNTIME/unknown → 500) and the ErrorReport.http_status property, which adds a provider-429 passthrough so the API can emit a Retry-After header from provider_metadata.retry_after_seconds. The library stays HTTP-agnostic — no web-framework dependency, just the mapping table; downstream FastAPI handlers call the helper instead of reinventing the contract.

• AMBIGUOUS inference-error category for outcome-uncertain failures. New InferenceErrorCategory.AMBIGUOUS for failures where the error type is known but the outcome is not — the operation may or may not have committed (e.g. a connection dropped mid-request). It is non-retryable, like CONFIGURATION / CONTENT / CAPACITY, but semantically distinct from UNKNOWN, which means the error could not be classified at all. The azure_rest image-generation worker now raises AMBIGUOUS for mid-request transport failures — ReadError / WriteError / RemoteProtocolError and ReadTimeout / WriteTimeout — so an automatic-retry layer won't re-fire a non-idempotent, billable image submit whose outcome is unknown; pre-request failures (ConnectError / ConnectTimeout / PoolTimeout) stay TRANSIENT and retryable.

• Explicit, uniform Tier 1 transport retry across every inference worker. A new [cogt] setting transport_max_retries (default 2) is now wired explicitly into every inference SDK client factory — Anthropic, OpenAI / Azure OpenAI, the Portkey-backed gateway clients, Mistral, and Google — instead of each factory silently inheriting whatever retry posture its SDK happens to default to. The two SDK families that default to no transport retry are brought up to the same floor: the Mistral client gets a bounded-backoff RetryConfig (retry_connection_errors=True) and the Google GenAI client gets HttpOptions(retry_options=...). The genuinely SDK-less path — the azure_rest image-generation worker, which talks to Azure over raw httpx — gets a tenacity-based transport-retry wrapper (new module pipelex/cogt/inference/transport_retry.py) that retries connection failures and transient HTTP statuses (408/409/429/5xx) and honors Retry-After. When no Retry-After header is present, the fallback backoff uses full jitter (wait_random_exponential) so a burst of failures retried together does not re-fire in lockstep. On a non-idempotent submit-style POST it narrows the retry to failures that prove the server did no billable work — a request that was never delivered, or a 408/429 rejection — withholding an ambiguous 5xx, a 409 conflict, and a post-delivery timeout such as a ReadTimeout. Transport retry is now a deliberate, configured, uniform policy rather than a per-provider accident. This is the transport-level floor of the retry model — pipeline execution makes a single attempt on top of it. (The portkey-ai SDK does not expose a retry knob — it carries its own internal retry — so the gateway's AsyncPortkey client is left as-is; only its underlying OpenAI clients are wired.)

• Offline mode for Pipelex Gateway setup and dry-run. When the gateway is enabled but the remote config service is temporarily unreachable, Pipelex now falls back to a previously primed on-disk cache (~/.pipelex/cache/remote_config.json, schema-versioned) instead of failing setup outright. Dry-run, validation, and pipelex-agent run bundle --dry-run complete normally; only the actual inference call still needs the network at runtime. The cache is primed on every successful fetch and on pipelex init while online. When the gateway is disabled (BYOK), no remote fetch is attempted at all — setup is fully offline. A new RemoteConfigStaleWarning (UserWarning) is emitted whenever stale cache is in use; the agent CLI surfaces it on the JSON envelope as warnings: [{"type": "RemoteConfigStale", ...}]. Telemetry is suppressed (no-op) when running on a cached config so stale model identities don't pollute metrics. The doc/fixture generators (pipelex-dev update-gateway-models, preprocess_test_models_cmd) refuse the cache fallback via a new require_fresh=True flag, so committed reference docs and test fixtures never bake in stale data.

• GatewayUnknownModelError (pipelex.cogt.exceptions). Raised at setup time when the active model deck references a gateway model handle that isn't present in the (fresh or cached) gateway specs. Carries the model name and the config source (RemoteConfigSource.FRESH | CACHED); the message branches on source so a cached-source failure suggests pipelex init while online and a fresh-source failure points at deck/typo fixes. Wired through both the Rich CLI (handle_gateway_unknown_model_error in error_handlers.py) and the agent CLI (AGENT_ERROR_HINTS / AGENT_ERROR_DOMAINS).

• RemoteConfigUnavailableError (pipelex.system.pipelex_service.exceptions). User-facing offline-mode error: raised only when the network fetch fails AND no usable cached fallback exists. The message names the cache file path and the two remediation paths (run pipelex init while online to prime the cache; or disable pipelex_gateway in backends.toml for permanent BYOK operation). Distinct from the internal RemoteConfigFetchError, which is kept as the retry-layer exception.

• PIPELEX_REMOTE_CONFIG_URL environment variable. Overrides the default remote-config URL. Useful for staging/testing environments; defaults to the production URL when unset.

Changed

• Inference error handling refactored to Extract / Classify / Render (internal). Every inference worker's SDK-exception block now collapses to metadata = extract_*_metadata(exc); classification = classify_inference_error(metadata); raise render_*_error(...) from exc. New modules: pipelex/cogt/inference/error_classify.py (single shared classify_inference_error() returning ClassificationResult(category, user_action_kind, is_model_not_found)), pipelex/cogt/inference/error_render.py (single shared render_llm_error / render_img_gen_error / render_extract_error / render_search_error, picking the CogtError subclass from an InferenceErrorFamily tag plus the is_model_not_found flag), and pipelex/cogt/inference/provider_name.py (the ProviderName enum keying the extract-fn registry). ProviderErrorMetadata gains a message field plus is_quota_exhaustion / is_content_policy_violation / is_network_error @property accessors; the per-provider extract_*_metadata functions are now the only plugin-local piece — Classify and Render live once. Mistral and the gateway-search worker specialize HTTP 404 to ExtractModelNotFoundError / SearchModelNotFoundError; Azure img-gen keeps two worker-specific AMBIGUOUS branches for mid-request transport failures. Removed: the per-provider *_error_classification.py modules and their tests, AnthropicCredentialsError, GatewayFactory.classify_error_category / make_user_action_from_portkey_error / make_error_summary_from_portkey_error, and every inline _classify_*_error / _raise_categorized_* worker method. New unit-test tests/unit/pipelex/cogt/inference/test_provider_classification_parity.py walks every ProviderName against the extract-fn registry + worker-family map, so an unwired new provider fails fast. No user-facing API change — the structured ErrorReport contract is unchanged. Documented at under-the-hood/error-model.md.

• instructor's structured-output retry no longer re-runs completions on transport errors. Passed a bare int, instructor's max_retries builds a retry loop whose predicate retries any exception — so the structured-output path (PipeLLM / PipeStructure) was re-running the whole completion on transport / API errors, a second retry loop nested on top of the SDK client's own transport retry. Each instructor call site now passes a tenacity.AsyncRetrying (built by the new pipelex/cogt/llm/instructor_retry.py helper) whose retry predicate matches only validation failures (pydantic.ValidationError, json.JSONDecodeError, and instructor's own validation-error types). A transport error now propagates immediately as the raw SDK exception for the worker's except clause to classify — transport retry is the SDK client floor (Tier 1) alone, and instructor's retry is confined to genuine schema re-ask. As part of this the Google and Mistral structured-generation workers gained an httpx.TransportError except clause: those SDKs let raw connection / timeout errors propagate outside their own exception hierarchies, so with instructor no longer wrapping them they must be caught and classified directly. The Mistral structured path also now gets schema re-ask at all — it previously passed no max_retries.

• Inference schema-retry setting moved and renamed: [cogt.llm_config.llm_job_config] max_retries → [cogt.llm_config] schema_reask_max_attempts (breaking). The setting is instructor's schema re-ask budget for structured-output validation failures. The old name max_retries gave no hint of that scope and collided conceptually with the new top-level cogt.transport_max_retries — which counts retries beyond the initial attempt, whereas this one is a total attempt count (stop_after_attempt). The new name makes both the scope (schema re-ask) and the unit (attempts, not retries) explicit. The single-key [cogt.llm_config.llm_job_config] sub-table is also dropped — the setting now sits directly under [cogt.llm_config]. Projects that override this key in their own pipelex.toml must move and rename it.

• Agent CLI run / validate / init default to markdown output, with independent success/error format options (breaking). These commands previously always emitted JSON; they now accept --format markdown|json and default to markdown, matching models / check-model / doctor. A second flag --error-format markdown|json controls error reporting on stderr independently from success output — it defaults to the value of --format, so --format json still flips both, but the two can now be set separately (e.g. --format markdown --error-format json for human-readable success with machine-parseable errors). Internally, only the error format is carried in a ContextVar; the success format is threaded explicitly to agent_success_formatted(). The inputs / concept / pipe / fmt / lint / accept-gateway-terms commands are unaffected (always JSON / raw passthrough).

• pipelex-agent validate bundle graph-format option renamed --format → --graph-format (breaking). The --format/-f flag that selected the graph renderer (mermaidflow / reactflow / both) is now --graph-format/-f, freeing --format to be the uniform markdown/json output-format flag across every agent-CLI command.

• structuring_method = "preliminary_text" on PipeLLM works again, via build-time elaboration. When PipelexInterpreter parses a .mthds file with a PipeLLM carrying structuring_method = "preliminary_text", the new BundleElaborator rewrites it before any pipe runs into a PipeSequence of two synthetic pipes: a PipeLLM producing Text (step 1, inheriting the original prompt + inputs + model) and a PipeStructure producing the original output (step 2, optionally using model_to_structure). The synthetic codes are tracked in a PipelexBundleBlueprint.elaboration_metadata side-table (excluded from serialization). Output multiplicity is preserved: step 1 always emits a single Text; step 2 emits Foo, Foo[], or Foo[N] according to the original output. Two LLM calls are issued per invocation. The user-facing pipe code is unchanged, so callers, main_pipe, and the run API keep working as before. Mechanism documented at under-the-hood/build-time-elaboration.md.

• PipeLLM runtime no longer knows about structuring_method. The field is now a pure build-time directive. The runtime PipeLLM class no longer carries structuring_method, the validator that rejected mismatched output concepts has moved to PipeLLMBlueprint (where it fires at parse time), and the NotImplementedError previously raised at runtime when preliminary_text was selected is gone — the elaborator handles it. PipeLLMSpec exposes structuring_method so AI agents authoring via specs can still opt in.

• StructuringMethod enum import path moved. The enum is now defined in pipelex.pipe_operators.llm.pipe_llm_blueprint (it lives next to the blueprint that consumes it) instead of pipelex.pipe_operators.llm.pipe_llm. Direct importers must update their from pipelex.pipe_operators.llm.pipe_llm import StructuringMethod line to from pipelex.pipe_operators.llm.pipe_llm_blueprint import StructuringMethod.

• Retry removed from the gateway workers; [cogt.tenacity_config] removed (breaking). The tenacity-based retry inside GatewayExtractWorker and GatewaySearchWorker is gone, along with the [cogt.tenacity_config] config block and its TenacityConfig model. Because config models forbid unknown keys, an existing ~/.pipelex/pipelex.toml (or any layered override) that still carries [cogt.tenacity_config] will fail to load — remove that block from your config. Transient transport blips are left to the provider SDK clients' own retry; the tenacity library itself is still used elsewhere (FAL job polling, remote-config fetch) and remains a dependency.

• PipeRouter.run() now reports CogtError failures to the observer. A CogtError raised out of pipe execution previously propagated past run() without an observe_after_failing_run notification (only PipeRunError was caught). It is now observed on the failing path, then re-raised as-is — the cause chain is preserved and it is not wrapped into PipeRouterError (only the PipeRunError path still wraps).

• Provider HTTP 404s now raise dedicated *ModelNotFoundError across LLM, image-gen, extract, and search. A model-or-deployment-not-found 404 from any LLM or image-gen provider, or from the Pipelex Gateway extract / search workers, now raises a dedicated *ModelNotFoundError (LLMModelNotFoundError, ImgGenModelNotFoundError, ExtractModelNotFoundError, SearchModelNotFoundError — all ModelNotFoundError subclasses, CONFIGURATION category) instead of a generic LLMCompletionError / ImgGenGenerationError / ExtractJobFailureError / GatewaySearchResponseError. Because these are siblings of the generic errors — not subclasses — a 404 propagates past the operator's generic except to except ModelNotFoundError in PipeOperator._live_run_pipe, which re-raises PipeOperatorModelAvailabilityError carrying the unavailable model_handle.

• RemoteConfigFetcher.fetch_remote_config() now returns a RemoteConfigResult carrying config, source (fresh | cached), and cached_at, instead of a bare RemoteConfig. Callers unwrap .config for the payload and may branch on .source to know whether the config is fresh or restored from cache. The fetcher accepts a new keyword-only require_fresh: bool = False — when True, a cached fallback raises RemoteConfigUnavailableError instead. RemoteConfigValidationError is never satisfied by the cache (server-side schema breaks must surface loudly).

• ModelManager.setup() and BackendLibrary._load_gateway_model_specs() accept a new gateway_config_source: RemoteConfigSource | None parameter. Passed through from Pipelex.setup() so the deck-level gateway membership check can branch its error message on FRESH vs CACHED. GatewayConfig itself stays extra="forbid" and source-free — provenance is plumbed alongside, not baked in.

• RemoteConfigUnavailableError message branches on whether the cache was refused vs absent. When require_fresh=True (dev-CLI generators) refuses to fall back to an existing cache, the message now reads "the local cache at <path> was refused because a fresh fetch is required" instead of the previously misleading "no local cache is available at <path>". The cache-truly-missing path keeps its original wording.

Fixed

• Inference-failure ErrorReports now carry model and provider in production. A real LLM / image-gen / extract / search failure used to surface an ErrorReport with model = None and provider = None: the leaf errors (LLMCompletionError, ImgGenGenerationError, ExtractJobFailureError, SearchJobFailureError, ExtractOutputError) carry no model or provider of their own, and CogtError.to_error_report() only duck-typed whatever attributes happened to be set on the exception. Each inference worker family now fills model_handle / backend_name from the worker — where both are unambiguously known — at its public-method chokepoint (gen_text / gen_object, gen_image / gen_image_list, extract_pages, search_sourced_answer / search_structured), via the new CogtError.fill_model_and_provider(). The fill never overwrites a value an inner error already set and skips the "unknown" placeholder external plugins report. model_handle / backend_name are now declared on CogtError (so to_error_report() reads them directly instead of getattr), making them uniformly str | None across the exception hierarchy.

• Two worker error-classification miscategorizations corrected. LinkupNoResultError (a search or fetch that returned nothing) had no explicit branch in either Linkup worker's _classify_linkup_error and fell through to the TRANSIENT catch-all — marking a query that cannot succeed by retrying as retryable. It is now classified CONTENT + CHANGE_INPUT in both the search and the extract worker. Behavior change: a no-result search is now non-retryable (CONTENT.is_retryable is False), so an automatic-retry layer no longer retries a query that returns nothing on every attempt. Separately, the FileNotFoundError branch in the Docling and pypdfium2 extract workers was classified CONFIGURATION — inconsistent with its sibling branches (CONTENT + CHANGE_INPUT) and with the rest of the codebase, where CONFIGURATION is reserved for setup problems. A missing input file is a content problem; it is now CONTENT. This second change does not alter retry behavior — CONFIGURATION and CONTENT are both non-retryable.

• Wrapped exceptions now surface the underlying inference error's classification. PipelexError.to_error_report() enriches its report from the __cause__ chain, so a PipelineExecutionError — or any wrapper around a transient CogtError — now reports error_category, retryable, model, and provider instead of dropping them. Previously the agent-CLI JSON / markdown error output for a failed pipeline run lost the worker's category and retryability once the error had been wrapped by the pipe operator → router → runner layers, leaving an agent unable to tell a transient failure from a fatal one.

• MTHDS JSON schema no longer leaks the elaboration_metadata side-table. PipelexBundleBlueprint.elaboration_metadata carried Field(exclude=True) so it stayed out of model_dump() / model_validate() round-trips, but Pydantic v2's exclude=True does not affect model_json_schema() — so derived/mthds_schema.json ended up declaring a top-level elaboration_metadata property plus $defs/ElaborationMetadata and $defs/StepRole entries, even though the side-table is process-local in the reference runtime. Wrapped the field type with SkipJsonSchema[...] so it is hidden from the schema. After regenerating, the three entries are gone from the public schema, matching the spec's silence on them.

• A directory containing a .pipelex/ config dir is now recognized as a project root. .pipelex was added to PROJECT_ROOT_MARKERS, so project-level config (e.g. .pipelex/inference/backends.toml) is honored even when the directory has no .git, pyproject.toml, or other source-project marker. Previously such a directory fell through to the global ~/.pipelex/ config, silently ignoring the project's own overrides — so a backend disabled in the project's backends.toml could still demand credentials because the global config (where it was enabled) was loaded instead. The home directory remains excluded from project-root detection, so the global ~/.pipelex/ is unaffected.

• PipeLLM outputting a concept that refines the native JSON concept no longer crashes with NameError: name 'Any' is not defined. On a LIVE run, such a concept resolves to a structured-output model carrying a dict[str, Any] field inherited from JSONContent. SchemaToModelFactory generates that model as source with from __future__ import annotations (every annotation becomes a string) and then rebuilds each class to resolve the string annotations. The rebuild namespace was assembled from the exec'd user types plus a hand-listed Literal, but typing.Any is a special form, not a type, so it was filtered out — model_rebuild then raised PydanticUndefinedAnnotation evaluating "dict[str, Any]". The rebuild namespace is now the exec namespace itself (minus __builtins__), so it carries exactly the names the generated source was written against and cannot drift as codegen emits other typing constructs. Fixed in pipelex/cogt/content_generation/schema_to_model_factory.py; covers both the sender path (make_from_json_schema) and the cross-process receiver path (make_types_from_source).

[v0.28.0] - 2026-05-13

Changed

• Breaking (rendering): graph viewer assets are no longer bundled into generated HTML. The ReactFlow HTML output now references @pipelex/mthds-ui (the standalone IIFE bundle + its CSS) and elkjs from cdn.jsdelivr.net with pinned versions and sha384 Subresource Integrity hashes. Three external requests now load from jsDelivr at view time; the HTML itself is still generated offline by Pipelex — only viewing the rendered graph requires network access. The previous make sync-graph-ui / make check-graph-ui-sync workflow and the committed pipelex/graph/reactflow/assets/graph-viewer.{js,css} files have been removed; the manifest-style package.json and the Graph UI sync check GitHub Actions workflow are gone. A single Python module (pipelex/graph/reactflow/standalone_assets.py) is the source of truth for pinned versions and integrity hashes — the template reads them via Jinja2, so a bump touches one file plus the regenerated SRI constants. The new pipelex-dev refresh-graph-ui-sri command re-fetches the URLs, recomputes sha384, and rewrites the constants module; the update-graph-ui skill drives that flow. Previously, the "self-contained HTML" was already not offline (elkjs loaded from unpkg.com without integrity), and bumping the vendored bundle meant committing ~2 MB of regenerated JS+CSS on every mthds-ui release — SRI hashes give the same tamper-evidence with none of the diff churn.
• Breaking: @var / @?var sigils must be alone on their own line, and inline @<ident> raises at load time only when <ident> collides with a declared input of the surrounding pipe. The @/@? rewriters wrap the value in a tag-shaped block envelope (<var>...</var> via the tag() filter), so the inline shapes never produced sensible output and are now rejected at load time with a new TemplateSigilSyntaxError — surfaced through pydantic validation with the line number, the offending span, and a migration hint. To stay friendly to coding agents authoring HTML/CSS templates (<style>@media (...) { ... }</style>, @font-face { ... }, Python/Java decorators like @deprecated / @Override), the validator is gated on the surrounding pipe's declared inputs: an inline @<ident> raises only when the candidate's root identifier matches a declared input (a real typo). Every inline candidate on a line is inspected, so a later collision can't hide behind an earlier CSS at-rule. Other inline @ candidates pass through silently. The line-bounded rewriter (alone-on-line @var → {{ var|tag("var") }}, leading/trailing whitespace preserved) is unchanged and remains inputs-agnostic. The strict rule replaces the heuristic regex tightening from the prior CSS-collision fix: there are no residual rewrites for @font-face / @counter-style / @view-transition / @property --x, no special-case lookbehind/lookahead arms. The $var inline sigil is unchanged. API: preprocess_template(template, *, declared_inputs=...) gained a keyword-only declared_inputs: set[str] | None parameter (default None = lenient, no validation); a new pair of helpers validate_template_sigils(template, declared_inputs) and rewrite_template_sigils(template) exposes the validate and rewrite steps separately. render_template (runtime) calls rewrite_template_sigils directly — templates are already validated at load time, so the render path doesn't need the declared_inputs context. All pipe blueprints (PipeLLMBlueprint, PipeComposeBlueprint, PipeSearchBlueprint, PipeImgGenBlueprint, PipeComposeFactory) and template analyzers (TemplateDocumentAnalyzer, TemplateImageAnalyzer) thread their declared inputs into the validator; the base TemplateBlueprint and ConstructBlueprint-discovery validators remain lenient (no inputs context at that layer — the surrounding pipe's validator catches typos). Migration: for inline values, switch from @var to $var; for block-shaped content, move the sigil onto its own line; for literal @ or $ characters, use the @@ / $$ escapes documented below.

Added

• @@ and $$ template escapes. Authors can opt out of sigil interpolation per occurrence: @@var renders as the literal string @var (no {{ ... }}), and $$var renders as the literal $var. The escapes are non-overlapping left-to-right (@@@@var → @@var, two literal @s). Use these for any literal @/$ the author needs to keep in the rendered output — most commonly @@font-face, @@namespace, @@media inside CSS <style> blocks, or $$10 for a literal dollar amount. Cross-link: wip/template-preprocessor-line-bounded-at.md.
• TemplateSigilSyntaxError (pipelex/cogt/templating/template_errors.py): raised by the preprocessor when a candidate @/@? sigil is not alone on its line. Pipe blueprints (PipeLLMBlueprint, PipeComposeBlueprint, PipeImgGenBlueprint, PipeSearchBlueprint, TemplateBlueprint, ConstructBlueprint, pipe_compose_factory) and the TemplateBlueprint validator catch and re-raise as pydantic ValueError with pipe-specific context, so the diagnostic surfaces through normal MTHDS validation errors and IDE/plxt check diagnostics.

Fixed

• Template preprocessor: $<var> no longer substitutes when $ is adjacent to a word character on the left (e.g. micro$oft, user$host.com, P@ssw$rd123, a$b$c now pass through unchanged instead of producing mid-word {{ ... }} substitutions), and no longer emits invalid Jinja for shapes like $name.. (now renders {{ name|format() }}.. with both trailing dots preserved as literal punctuation). The $ arm now mirrors the @ candidate pattern's word-boundary lookbehind ((?<!\w)) and uses a strict segmented identifier [a-zA-Z_][a-zA-Z0-9_]*(?:\.[a-zA-Z_][a-zA-Z0-9_]*)* that rules out a leading digit and consecutive dots structurally. The trailing-dot kludge in the $ replacement function is gone (unreachable under the strict identifier shape). $var keeps its silent pass-through posture for non-candidate shapes — it does not raise like the @ arm does.
• Project .pipelex/telemetry.toml created by pipelex init no longer silently disables globally-enabled telemetry. The kit previously shipped one telemetry.toml template carrying explicit enabled = false / mode = "off" defaults, and pipelex init telemetry copied it into both ~/.pipelex/ and the project's .pipelex/. Because the layered loader appends the project base file after ~/.pipelex/telemetry_override.toml, a user who enabled Langfuse/PostHog globally in their override still had it turned off in every initialized project. The kit now ships a separate telemetry.project.toml template with every setting commented out, used only by project-level init. The global init still copies the active telemetry.toml. As a safety net, telemetry_allowed_modes defaults (ci=False, cli=True, docker=True, fastapi=True, mcp=True, n8n=True, pytest=False, python=False) now live in the TelemetryConfig Pydantic model rather than the TOML template, so a project file that omits the section can't disable custom telemetry for everyone.
• Global pipelex.toml and telemetry.toml overrides no longer silently disappear when a project .pipelex/ exists. ConfigLoader.load_config() previously sourced all override files (pipelex_local.toml, pipelex_{env}.toml, pipelex_{run_mode}.toml, pipelex_override.toml, pipelex_temporary_override.toml) from a single "effective config dir" — project .pipelex/ if present, else ~/.pipelex/. As a result, the user's machine-wide overrides in ~/.pipelex/ were ignored entirely the moment a project shipped its own .pipelex/. load_telemetry_config had the same shadowing behavior for telemetry.toml and telemetry_override.toml, which was particularly painful because telemetry holds Langfuse/PostHog/OTLP secrets that users typically set once globally. Both loaders now layer global → project: package defaults → global base → global overrides → project base → project overrides, deep-merged so machine-wide personal preferences keep applying unless explicitly overridden by the project. The unit-testing runmode behavior is preserved: ./tests/pipelex*{run_mode}.tomlis still the only run_mode file loaded under unit testing, replacing both layers to keep test runs hermetic. Migration: if a project previously relied on its.pipelex/to fully replace* the user's global config rather than merging over it, the project must now redeclare any keys it wants to set explicitly — silent inheritance from~/.pipelex/ is the new default.
• pipelex build structures now emits domain-qualified class names and cross-references. ConceptFactory registers each concept's structure class under make_qualified_structure_class_name(domain, concept_code) (e.g. expense_validator__SpendingLimitCheck), but the CLI generator was passing the bare concept_code to StructureGenerator. Generated files therefore contained class ConceptCode(StructuredContent) and unqualified cross-imports/types, while the registry expected the qualified name. PipeFunc validation against concept.structure_class_name then rejected the mismatch with output concept expects structure class 'domain__X', but the function return type is 'X', breaking library load for any project that had regenerated structures. Fixed in pipelex/cli/commands/build/structures_cmd.py: the class-definition call sites now call make_qualified_structure_class_name(blueprint.domain, concept_code); _build_concept_ref_to_class_info qualifies cross-reference class names while keeping the file stem unqualified so output filenames stay domain__concept_code.py; the refines: branch resolves the refined concept's domain via QualifiedRef.parse_stripping_cross_package and qualifies the base-class name for non-native refines, mirroring ConceptFactory._handle_refines.
• pipelex build structures cross-package refines now emit the base-class import. A concept that refines a class defined in another already-installed package (e.g. RefiningConcept with refines: some_alias->other_domain.OtherConcept) was generating a file that inherited from other_domain__OtherConcept without importing it. StructureGenerator._validate_execution injects the base class into exec_globals from the class registry, so codegen-time validation passed; the failure surfaced only when the generated structures module was later loaded via the normal Python import system, raising NameError: name 'other_domain__OtherConcept' is not defined. The generator now falls back to the class registry's __module__ when concept_ref_to_class_info has no entry for the base class and emits the corresponding import (guarded on cls.__name__ == base_class so we never emit a name the target module won't expose). Regression test in tests/unit/pipelex/cli/commands/build/test_structures_cmd_cross_package_refines.py was tightened to assert the import statement is present.

[v0.27.0] - 2026-05-07

Changed

• RunEnvironment env var renamed from ENVIRONMENT to PIPELEX_ENV. The variable that selects the environment-specific pipelex_<env>.toml overlay (and is stamped on OTel spans as deployment.environment) is now namespaced to avoid collisions with unrelated ENVIRONMENT variables already set in deployment environments. Update any shell, CI, or container config that exported ENVIRONMENT=... to export PIPELEX_ENV=... instead.

[v0.26.4] - 2026-05-06

Fixed

• clean_json_content() no longer crashes on nested Pydantic instances in working_memory_raw. When working_memory_raw (typed dict[str, Any]) held rehydrated BaseModel instances (e.g. PageContent from Page[] outputs), clean_json_content() walked only dicts/lists/scalars and let BaseModel instances reach json.dumps, which raised TypeError: Object of type PageContent is not JSON serializable. clean_json_content now reduces any BaseModel it encounters via model_dump(serialize_as_any=True) (the canonical smart_dump path) before continuing the recursive walk.

[v0.26.3] - 2026-05-06

Fixed

• OpenAI SDK 2.34.0 compatibility for Portkey/Gateway clients. The new SDK adds both a constructor-time check (_enforce_credentials) and a request-time _validate_headers guard that reject an empty api_key. Pipelex's gateway and Portkey factories were passing api_key="" because auth is delivered via x-portkey-api-key and x-portkey-config headers, not the OpenAI Authorization header. Replaced the empty string with a non-empty placeholder ("unused-auth-via-portkey-headers") in gateway_completions_factory, gateway_responses_factory, portkey_completions_factory, and portkey_responses_factory. Same fix applied to openai_client_factory for the no-auth case (e.g. local Ollama backends), where backend.api_key is intentionally unset. Bumps the openai requirement to >=2.0.0.

[v0.26.2] - 2026-05-06

Fixed

• choices fields no longer fail validation with 'EnumName.MEMBER_NAME' errors. A concept declared with choices = [...] produces a Literal[...] field on the dynamic Pydantic class. That schema is round-tripped through SchemaToModelFactory.make_from_json_schema (used to feed structured-output schemas to LLM providers). Previously the round-trip silently re-emitted the field as a plain Python Enum class — e.g. Literal["Strong Match", "Good Match", "Partial Match", "Poor Match"] became class Recommendation(Enum): Poor_Match = "Poor Match"; .... LLMs filling that schema then returned the enum's Python repr ("Recommendation.Poor_Match") instead of the literal string ("Poor Match"), which failed Pydantic validation against the original choice set with errors like Invalid choice errors: 'recommendation': got 'Recommendation.Poor_Match', expected one of 'Strong Match', 'Good Match', 'Partial Match' or 'Poor Match'. _generate_source_from_schema now passes enum_field_as_literal=LiteralType.All to datamodel-code-generator, so enum: [strings] schema nodes round-trip as Literal[...] instead of being regenerated as Enum classes. _exec_source_to_types now also exposes Literal in the rebuild namespace so model_rebuild resolves the deferred annotations.

[v0.26.1] - 2026-05-05

Changed

• Live-run graph output: pipelex-agent run now emits the assembled GraphSpec as live_run_graph.json alongside live_run.json and the ReactFlow HTML.

[v0.26.0] - 2026-05-04

Highlights

• Distributed tracing — pluggable event-log backends (NDJSON, DynamoDB, in-memory, buffering) emit TraceEvents that reassemble into a complete GraphSpec. Enabled by default; works in single-process mode too.

Added

Pipeline execution & delivery

• PipeRun layer (pipelex/pipe_run/): new PipeRunProtocol with the in-process PipeRun implementation. Wraps pipe execution + delivery in a single orchestration unit.
• Delivery framework: DeliveryAssignment model with webhook (HTTP POST) and storage (S3 / local) variants, run inline after a pipe completes.
• LibraryCrate IR (pipelex/libraries/library_crate.py, library_crate_factory.py): serializable intermediate representation capturing the exact set of libraries (concepts, pipes, structures) required to execute a job — no shared filesystem assumed.
• Schema-based dynamic model construction (pipelex/cogt/content_generation/schema_to_model_factory.py): rebuilds dynamic Pydantic models from the JSON schema embedded in ObjectAssignment, instead of relying on class identity. Cache bounded by an LRU (_SCHEMA_CACHE_MAX_SIZE = 1024); codegen serialized through a file lock to kill thundering-herd duplication when many distinct schemas arrive. Adds datamodel-code-generator>=0.55.0 as a runtime dependency.
• Pre-generated pipeline_run_id support: pipeline_run_setup(), PipelineFactory, and PipelineManager accept an optional pipeline_run_id parameter for external run-record creation.

Distributed tracing (pipelex/tracing/)

• EventLogProtocol with pluggable backends: NdjsonEventLog, DynamoDBEventLog, InMemoryEventLog, BufferingEventLog. Emits TraceEvents as pipes start/finish across processes. Configured via [pipelex.tracing_config] (backend = "ndjson" | "dynamodb"). Tracing is enabled by default — single-process runs also produce a trace.
• pipelex[dynamodb] extra: pip install "pipelex[dynamodb]" for the DynamoDB event log backend.
• writer_id on TraceEvent (default "primary"): every event records which writer (worker / process) produced it, propagated through every backend. Separate writer processes use a distinct writer_id so multiple writers can emit into the same backend partition without colliding on (workflow_id, sequence). Stamped at construction time (no copy-on-emit). Enables cross-writer dedup and per-writer aggregation.
• NDJSON multi-writer key scheme: dedup key is (workflow_id, writer_id, type, sequence); sort key is (workflow_id, sequence, writer_id) — sequence is primary so the runner's UsageReportEvent does not sort before the router's PipeStartEvent. The file-handle cache key includes writer_id, so two writers for the same (pipeline_run_id, workflow_id) cannot share a stale handle.
• DynamoDB sort key format: EVENT#{workflow_id}#{writer_id}#{sequence:010d}.
• ContentGeneratorDry emits UsageReportEvents: invokes report_inference_job with a synthetic LLMTokensUsage so dry-run mode produces real events observable through the same backend path as live runs.
• GraphSpecAssembler: rebuilds a complete GraphSpec offline from a stream of TraceEvents — equivalence with the live GraphTracer is pinned by tests.
• UsageAggregator: aggregates per-pipeline LLM/extract/img-gen usage across workers from the event log.
• Hub DI: PipelexHub.set_event_log() / get_event_log() for dependency injection of custom event log backends. make_event_log() factory selects the backend from config or hub injection; the 3 prior call sites that hardcoded NdjsonEventLog now go through it.

Inference error classification

• InferenceErrorCategory enum (transient, configuration, content, capacity) with per-provider helpers that distinguish quota exhaustion from rate limits and detect content-policy violations — covers OpenAI, Anthropic, Google, Mistral, AWS Bedrock, and Pipelex Gateway.
• All inference workers attach error category + user action: every LLM, image-gen, extract, and search worker across all providers raises exceptions with an InferenceErrorCategory and actionable user_action hint.
• Structured ErrorReport: PipelexError.to_error_report() returns a dataclass with error type, category, retryable flag, user-action hint, model, and provider. CLI error handlers consume to_error_report() for consistent, structured display.
• SecurityError(PipelexError): new base class in pipelex.base_exceptions for security-policy violations. UnsafeSchemaError now extends it (was ContentGenerationError) so security signals are not silently swallowed by domain except handlers.

Documentation

• New "Under the Hood" pages: pipe-routing-and-execution.md and distributed-content-generation.md. Cover routing/dispatch and the dynamic-model serialization story.
• New AGENTS.md at repo root, generated from the same source as CLAUDE.md via make rules.

Other

• PipeOutput.graph_assembly_error: optional str field set when graph assembly raises, so consumers can distinguish "graph never produced" from "graph assembly failed". Previously the failure was logged as a warning and graph_spec silently stayed None.
• pipe extract web URL support with full test coverage in tests/integration/pipelex/pipes/pipe_operators/pipe_extract/.
• needs_inference flag on CLI commands: non-inference subcommands (e.g. doctor) skip inference setup, shaving cold-start time.
• Environment-specific config: RunEnvironment is now loaded from the ENVIRONMENT env var (was ENV). Accepted values: local, dev, staging, prod.

Changed

• Tightened inference quota classification patterns to reduce false positives on auth/setup errors:
• Anthropic: bare "credit" token replaced with "credit balance", "out of credits", "insufficient credit". No longer misclassifies "your credit card was declined" or "we will credit your account" as quota exhaustion.
• Google: bare "billing" token replaced with "billing limit", "billing quota", "billing exceeded", and "billing account". Continues to match real 429-class messages ("Billing account is not active", "billing limit reached"); stops matching unrelated billing-mentioning text.
• Mistral / Gateway: bare "billing" likewise narrowed to "billing limit" / "billing quota". Mistral additionally distinguishes "out of credits" vs "insufficient credits".
• Removed duplicate ContentGenerationError class: the unused pipelex.cogt.content_generation.exceptions.ContentGenerationError was deleted (had zero external imports). Its subclasses re-parented: NeitherUrlNorDataError → PipelexError, UnsafeSchemaError → new SecurityError.
• Storage providers unified: StorageConfig flattened to inheritance-based config so providers (local, s3) share a single shape across delivery and trace storage.
• make rules now generates both CLAUDE.md and AGENTS.md by default: the pipelex.kit_config.preferred_agent_target setting has been renamed to preferred_agent_targets and is now a list. The default is ["claude", "agents"]. Cursor remains exclusive (["cursor"]) and cannot be combined with single-file targets. Downstream projects overriding this setting must rename the key and wrap the value in a list.
• Cold start trimmed: heavy SDK imports (boto3, huggingface_hub, etc.) deferred behind TYPE_CHECKING or function-local guards across multiple plugins.
• Graph viewer updated to mthds-ui v0.3.4: bumped from v0.3.0 — additional polish atop the resizable detail panel, escape-to-close, sticky header, prompt expand/collapse with copy button, concept refinement display.
• Default models: small-vision and creative defaults now point to gemini-3.0-flash-preview. Added claude-4.6-sonnet and Bedrock token-auth support.
• kajson upgraded from 0.3.1 to 0.5.0 — required for the dynamic-class source registry that backs schema-to-model reconstruction.
• README install instructions: replaced step-by-step Claude Code setup with single copy-paste messages for Claude Code and Codex, added manual install section.

Fixed

• Anthropic structured generation no longer flattens all errors to CONTENT. AnthropicLLMWorker._gen_object runs requests through instructor, which wraps SDK exceptions in InstructorRetryException. The previous handler categorized every wrapped error as CONTENT, so genuine RateLimitError, APITimeoutError, APIConnectionError, PermissionDeniedError, and AuthenticationError cases never reached the typed branches — they were reported as content failures (non-retryable) instead of TRANSIENT / CAPACITY / CONFIGURATION. The handler now unwraps InstructorRetryException.failed_attempts[-1].exception (with a __cause__/RetryError.last_attempt fallback) and routes recognized SDK exceptions through a shared categorization helper; truly unrecognized errors keep the CONTENT fallback.
• Managed-deck detection no longer misclassifies user TOMLs. _is_managed_deck_filename previously treated any non-x_custom_*.toml as kit-managed, so a user file like my_overrides.toml dropped into the deck dir would have been reported (and on pipelex update overwritten) as if it were a kit file. Now requires the numbered ^\d+_.*\.toml$ prefix that the kit actually ships.
• PipeLLM output structure prompt used the unresolved concept ref. When is_structure_prompt_enabled, get_output_structure_prompt was called with pipe_run_params.dynamic_output_concept_ref or output_stuff_spec.concept.concept_ref. The first form can be a bare code (no domain), which broke downstream concept resolution. Now passes the already-resolved output_stuff_spec.concept.concept_ref directly.
• PIPELEX_NO_DECK_NOTICE suppression was too lax. Any non-empty value silenced the deck-staleness notice — including PIPELEX_NO_DECK_NOTICE=0, which users might reasonably expect to enable the notice. Now requires the documented PIPELEX_NO_DECK_NOTICE=1.
• Shipped .kit_manifest.json carried kit_version: "0.25.0" for the 0.25.1 release, which would have triggered false stale-detection on fresh installs. Bumped to 0.25.1 and re-synced into pipelex/kit/configs/.
• Stored-XSS guard: HTML escaped in fallback main_stuff.html.
• GatewayExtractWorker teardown: guarded done-callbacks against cancelled tasks to stop noisy teardown errors.
• URL checker: 401/403 are now treated as OK for auth-walled URLs.
• Pipeline duplicate guard: registering the same pipeline twice raises explicitly instead of silently shadowing.

Documentation

• docs/tools/cli/update.md no longer claims the deck advisory fires on every CLI invocation. Clarified that it is suppressed for login, init, doctor, update, and which.

Security

• schema_to_model rejects x-python-* codegen extensions and restricts __import__ to an allowlist. datamodel-code-generator honors the x-python-import JSON Schema extension by emitting arbitrary from <module> import <name> statements with no sanitization. Combined with the prior gap that _make_restricted_builtins() did not block __import__, an attacker able to plant a crafted object_class_schema (where ObjectAssignment.object_class_schema: dict[str, Any] round-trips untouched) could cause arbitrary modules to be imported during exec() of generated code. Two-layer defense added: (1) _reject_unsafe_schema_extensions raises UnsafeSchemaError if any x-python-* key is present anywhere in the schema; (2) __import__ in the exec namespace is now wrapped to allow only pydantic, typing, typing_extensions, enum, datetime, decimal, uuid, __future__, collections, and re.
• Restricted exec builtins, path sanitization, and atomic fingerprint caching in the schema-to-model pipeline.
• No leaked exception strings on error paths; prefix guards on storage URIs.

[v0.25.2] - 2026-04-30

Fixed

• pipelex update no longer flags user-added deck files for removal. The "managed file" filter accepted any .toml not prefixed with x_custom_, so project-local additions like pipelex-cookbook's cookbook.toml were reported as "removed upstream" with action "back up + remove". The filter now matches the documented numbered convention only (<digits>_*.toml); any other filename — cookbook.toml, x_custom_*.toml, etc. — is invisible to the update planner.

[v0.25.1] - 2026-04-29

Added

• pipelex update command + model-deck staleness detection. New top-level command refreshes the installed model deck (~/.pipelex/inference/deck/) to match the kit shipped with the running pipelex version. Tracks per-file SHA-256 hashes plus the kit version in a .kit_manifest.json written next to the deck files. Supports --dry-run, --yes (non-interactive), --no-backup (skip .bak files for locally-modified deck files), and --local (project-local deck instead of global).
• Boot-time deck staleness warn. Every pipelex CLI invocation prints a one-line yellow advisory when the installed deck's recorded kit version is older than the running pipelex (or when no manifest exists yet). Cost is one file read + one string compare. Suppress with PIPELEX_NO_DECK_NOTICE=1. Skipped automatically for pipelex login, init, doctor, update, and which.
• pipelex doctor deck section. Reports per-file deck status (up_to_date, kit_added, kit_removed, clean_behind, locally_modified) and offers pipelex update as an auto-fix under --fix.
• Manifest written by pipelex init. Fresh installs land with a current .kit_manifest.json so future updates can detect drift cleanly.
• --dynamic-output-concept / -O flag on pipelex run. All three subcommands (run bundle, run pipe, run method) accept a concept ref (e.g. document_qa.ReferenceCount) used to resolve a pipe whose output is declared as Dynamic. Threaded through _run_core.execute_run to PipelexRunner.execute_pipeline(dynamic_output_concept_ref=...). Until now, Dynamic-output pipes were only callable from the Python runner.
• Line-length-safe wrapping in the structures generator. pipelex build structures now wraps long descriptions so every emitted line stays under the 150-char ruff limit. Long class docstrings become a multi-line triple-quoted block; long Field descriptions become a parenthesized implicit-string-concatenation block (description=("first chunk " "second chunk")). Short descriptions still emit the compact single-line form. New unit tests in tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_wrapping.py cover both lengths and the combined long-everything case. Previously, descriptions above ~140 chars produced files that failed ruff check with E501.

Changed

• Numbered deck files (N_*_deck.toml) are pipelex-managed. Each file now carries a header banner explaining that customizations belong in x_custom_*.toml (which pipelex never tracks or overwrites). Local edits to numbered files are preserved with a timestamped .bak.<UTC-timestamp> backup on pipelex update but will not survive future updates.

Fixed

• PipeLLM Dynamic-output detection compared the wrong fields. pipe_llm.py checked self.output.concept.code == "native.Dynamic", but concept.code is the bare code ("Dynamic") not the qualified ref. The check never matched, so when a caller passed dynamic_output_concept_ref for a Dynamic-output pipe, the resolver branch was skipped silently: the output structure stayed DynamicContent (an empty StuffContent subclass), the LLM produced JSON shaped like the requested concept, and the result deserialized to {}. Detection now uses concept.code == NativeConceptCode.DYNAMIC and concept.domain_code == SpecialDomain.NATIVE.

• Dynamic-output concept resolution rejected qualified refs. When the runtime override was supplied (e.g. "document_qa.ReferenceCount"), the previous code called make_concept_ref_with_domain(domain_code=self.domain_code, concept_code=output_concept_ref), producing "document_qa.document_qa.ReferenceCount" and a missing-concept lookup. Now uses make_concept_ref_with_domain_from_concept_ref_or_code, which extracts the domain from the input when it's already qualified and falls back to the pipe's domain only for bare codes. Callers can pass either form.

[v0.25.0] - 2026-04-28

Added

• GPT-5.5 model support. New gpt-5.5 entry on the openai, azure_openai and gateway backends.

• gpt-image-2 image generation (OpenAI). New model entry with img-gen routing and constraints on openai, azure_openai.

Fixed

• PDF input declared on Azure OpenAI and gateway for the GPT-5.4 series. gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-5.4-pro (and gpt-5.5) now advertise pdf in their inputs on azure_openai and the gateway, matching the existing OpenAI direct entries. Verified end-to-end with live document tests against both backends.

Changed

• Img-gen taxonomies aligned for gpt-image-2. Convention: gpt_image for taxonomies/values shared across all OpenAI GPT Image models (legacy + gpt-image-2); gpt_image_legacy when the value applies only to gpt-image-1/-1-mini/-1.5 (gpt-image-2 uses unavailable where a legacy-only option does not apply). Renames:
• AspectRatioTaxonomy.OPENAI_GPT_IMAGE_LEGACY → GPT_IMAGE_LEGACY (value "openai_gpt_image_legacy" → "gpt_image_legacy")
• AspectRatioTaxonomy.OPENAI_GPT_IMAGE_2 → GPT_IMAGE_2 (value "openai_gpt_image_2" → "gpt_image_2")
• OutputFormatTaxonomy.GPT → GPT_IMAGE_LEGACY (value "gpt" → "gpt_image_legacy")
• OutputCompressionTaxonomy.GPT_IMAGE → GPT_IMAGE_LEGACY (value "gpt_image" → "gpt_image_legacy")
• InputFidelityTaxonomy.OPENAI_IMAGE → GPT_IMAGE_LEGACY (value "openai_image" → "gpt_image_legacy")
• NumImagesTaxonomy.GPT → GPT_IMAGE (value "gpt" → "gpt_image")
• InferenceTaxonomy.GPT → GPT_IMAGE (value "gpt" → "gpt_image")
• Removed dead AspectRatioTaxonomy.GPT.

• InputImagesTaxonomy.GPT_IMAGE unchanged (already correct — shared across legacy and gpt-image-2).

• HuggingFace and Fal img-gen workers honor model_choice rule. huggingface_img_gen and fal workers now pop "model" from args_dict instead of hardcoding inference_model.model_id, mirroring how prompt is already extracted. The model_choice = "model_id" rule is now required on every model under these backends; missing it raises ImgGenParameterError at call time. Rules added to qwen-image (HF) and to all Fal models (flux-pro, flux-pro/v1.1, flux-pro/v1.1-ultra, flux-2, fast-lightning-sdxl).

[v0.24.1] - 2026-04-22

Security

• lxml floor >=6.1.0 to patch CVE-2026-41066 (GHSA-vfmq-68hx-4jfw): default configuration of iterparse() and ETCompatXMLParser() allowed XXE to local files (resolve_entities=True). lxml 6.1.0 changes the default to resolve_entities='internal'. Transitive via docling; floor added to the docling extra in pyproject.toml so downstream installs of pipelex[docling] cannot resolve a vulnerable version.
• cryptography floor >=46.0.7 to patch CVE-2026-39892 (GHSA-p423-j2cm-9vmq): non-contiguous Python buffers passed to hashing APIs (e.g. Hash.update()) could read past the end of the buffer on Python >3.11. Transitive via google-auth (pulled by google, gcp-storage, google-genai extras) and moto (dev). Floor added to each affected extra in pyproject.toml — previous bump was lockfile-only, which did not protect downstream users resolving fresh from PyPI metadata.
• pytest bumped to 9.0.3 to patch CVE-2025-71176 (GHSA-6w46-j5rx-g56g): vulnerable /tmp/pytest-of-{user} directory handling on UNIX could let a local user cause DoS or gain privileges. Dev-only dependency; pyproject.toml minimum bumped from >=9.0.2 to >=9.0.3.
• transformers CVE-2026-1839 (GHSA-69w3-r845-3855) risk-accepted, alert dismissed. The vulnerability requires calling transformers.Trainer._load_rng_state() with an attacker-controlled checkpoint file. Pipelex only pulls transformers transitively through docling-ibm-models for PDF layout inference; the Trainer class is never imported or executed. Upgrade path is blocked upstream: docling-ibm-models 3.13.0 pins transformers!=5.0.*,!=5.1.*,!=5.2.*,!=5.3.*,<6.0.0,>=4.42.0, explicitly excluding the patched 5.0.0rc3 release. Revisit when docling-ibm-models adds support for transformers>=5.4.
• Release-publishing GitHub Actions pinned to SHAs: pypa/gh-action-pypi-publish and sigstore/gh-action-sigstore-python in publish-pypi.yml are now pinned to full commit SHAs (version kept as a trailing comment) so a compromised tag on a third-party action cannot silently alter a PyPI release. Dependabot keeps them fresh.
• .github/dependabot.yml added: declares pip and github-actions ecosystems, weekly cadence, with dev and runtime deps grouped to reduce PR noise. Security updates fire immediately regardless of schedule.
• dependency-review.yml workflow added: runs GitHub's dependency-review-action on PRs to main, dev, and release branches. Fails the PR if it introduces a dependency with a moderate-or-higher CVE. Respects the existing transformers (GHSA-69w3-r845-3855) risk-acceptance via allow-ghsas. Enable as a required status check in branch protection for main to block vulnerable merges.

[v0.24.0] - 2026-04-16

Added

• claude-opus-4-7 model: Registered on anthropic, bedrock, and gateway backends with 128k max output tokens, $5/$25 per MTok pricing, adaptive thinking, and PDF/vision support
• XHIGH reasoning effort level: New effort tier across all providers, mapped to Anthropic's xhigh (recommended for coding/agentic work), OpenAI's xhigh, and best-available equivalents for Google and Mistral
• TEMPERATURE_UNSUPPORTED constraint: New listed constraint for models that reject sampling parameters entirely, checked in both Anthropic and OpenAI completions workers
• claude-4.6-sonnet model: Registered on anthropic, bedrock, and gateway backends
• LLM deck cheap presets: Added cheap variants for writing-factual, retrieval, and engineering-code presets, with retrieval tiers from gemini-2.5-flash-lite to claude-4.7-opus
• Bedrock bearer token authentication: New bedrock_access_variant config option supports "bedrock_token" auth using AWS_BEARER_TOKEN_BEDROCK env var, alongside the existing "aws_access" method (default)

Changed

• Anthropic adaptive thinking rejects reasoning_budget: _build_thinking_params_for_budget now raises LLMCapabilityError for adaptive thinking models, guiding users to reasoning_effort instead — extended thinking (type: "enabled") is removed on Opus 4.7+
• LLM deck: Updated best-claude alias to claude-4.7-opus, switched img-gen-prompting-cheap to @default-small-creative, removed deprecated builder presets, standardized cheap preset descriptions
• Google MINIMAL reasoning level: ReasoningEffort.MINIMAL now maps to Google's ThinkingLevel.MINIMAL (was previously collapsed to LOW); GoogleThinkingLevel enum gains MINIMAL and the default google_config.effort_to_level_map updates accordingly

[v0.23.9] - 2026-04-14

Added

• Graph UI asset sync workflow: package.json pins @pipelex/mthds-ui to a git tag, make sync-graph-ui clones and builds standalone assets, make check-graph-ui-sync verifies version alignment
• CI check: graph-ui-check.yml workflow validates graph viewer assets match the pinned version on PRs to main
• /update-graph-ui skill: automates bumping the mthds-ui version, syncing assets, and running tests

Changed

• Graph viewer updated to mthds-ui v0.3.0: resizable detail panel, escape-to-close, sticky header, prompt expand/collapse with copy button, concept refinement display
• README install instructions: Replaced step-by-step Claude Code setup with single copy-paste messages for Claude Code and Codex, added manual install section

[v0.23.8] - 2026-04-07

Changed

• Standalone ReactFlow graph rendering: Replaced Jinja2 template-based HTML generation with a single standalone HTML asset, simplifying the graph rendering pipeline and removing the Jinja2 template dependency for ReactFlow output.

[v0.23.7] - 2026-04-06

Added

• Graph tracing for pipe run data: Pipe run data and concept are now included inside the flowchart graph spec, enabling richer visualization of pipe execution results across all pipe types (LLM, extract, compose, search, image gen, sequence, condition, batch, parallel).
• Assignment pipe: New pipe_assignment pattern for direct value assignment within pipe execution.

[v0.23.6] - 2026-04-06

Changed

• Sub-pipe input normalization: Silently drop unsupported inputs field from step/branch dicts instead of logging a warning.

[v0.23.5] - 2026-04-04

Added

• Gateway config: Introduced GatewayConfig to bundle gateway model specs with AWS region, propagating it through the backend library so bedrock backends use the correct region.
• Config coverage tests: Integration tests that validate one model per Portkey config for each model type (LLM, image gen, extract, search), with all_configs_gw test profile and make ticc target.
• nano-banana-2 model: Added gemini-3.1-flash-image-preview as nano-banana-2 with updated Google image gen costs.
• DeepSeek models on bedrock: Added DeepSeek models to the bedrock backend configuration.

Changed

• Image gen deck aliases: Updated aliases to nano-banana model variants and removed flux-2-pro.
• Remote config: Bumped to v08.
• Gateway model docs: Regenerated, removing retired models (claude-3.7-sonnet, deepseek-v3.1, deepseek-v3.2-speciale, flux-2-pro).

Fixed

• deepseek-v3.1 structured output: Removed unsupported structured output capability from the bedrock deepseek-v3.1 model spec — the bedrock_aioboto3 worker does not implement object generation, so structured calls would fail at runtime.

[v0.23.4] - 2026-04-02

Changed

• Pipe spec output alias: Removed output_type alias from parse_pipe_spec, keeping only output_concept as the single alias for the output field. Simplified the alias resolution logic accordingly.

[v0.23.3] - 2026-04-02

Changed

• Pipe spec output aliases: parse_pipe_spec now accepts output_concept and output_type as aliases for the output field, with smart fallback when both alias and canonical field are present.

Fixed

• Gateway terms check: Terms acceptance is now only required for inference operations, not for read-only operations like model spec fetching during validation.

[v0.23.2] - 2026-03-30

Changed

• Claude Code plugin install command: Fixed as → /plugin install mthds@mthds-plugins across README and docs.
• Claude Code plugin reload instructions: Added /reload-plugins as the primary method to activate the plugin, with exit/reopen as fallback.

[v0.23.1] - 2026-03-30

Changed

• Concept spec: concept_code replaces the_concept_code as the canonical field name in concept specs and working memory factory.
• Shared spec parsing: concept_cmd and pipe_cmd now delegate to the shared parse_concept_spec and parse_pipe_spec helpers, removing stale duplicate parsing logic while preserving compatibility and fixing alias/dict-mutation edge cases.
• concept_ref / pipe_ref aliases: parse_concept_spec and parse_pipe_spec now accept concept_ref and pipe_ref as input aliases for better AI-agent compatibility.
• Replace pip with uv in install commands across config files and error messages.
• Docs links: Updated mthds.ai links to include /latest/ path.

Fixed

• Concept alias bug: Concept alias handling previously listed concept_code as an alias instead of the_concept_code, causing valid input to be silently dropped. Fixed by the new shared parse_concept_spec helper.

[v0.23.0] - 2026-03-29

Added

• Builder Operations module (pipelex/builder/operations/): New standalone operation functions for the build agent — concept_ops, inputs_ops, models_ops, output_ops, pipe_ops, runner_code_ops, validate_ops. These decouple build logic from CLI commands so it can be reused by agents and the API.
• dry_run_pipeline (pipelex/pipe_run/dry_run_pipeline.py): New shared entrypoint to dry-run a pipeline from MTHDS contents and produce a GraphSpec. Used by both CLI graph commands and the API.

Changed

• mthds_content → mthds_contents: Unified singular mthds_content: str | None into mthds_contents: list[str] | None across PipelexRunner, pipeline_run_setup, validate_bundle, CLI commands, and builder operations. Callers now pass a list of bundle content strings.
• bundle_uri → bundle_uris: Renamed bundle_uri: str | None to bundle_uris: list[str] | None across PipelexRunner.__init__, pipeline_run_setup, dry_run_pipeline, and all CLI call sites. Multiple bundles can now be loaded simultaneously.
• Bump mthds dependency to >=0.2.0 (from >=0.1.1) to match the updated RunnerProtocol interface.
• Graph rendering refactor: Extracted dry-run logic from graph_rendering._dry_run_bundle into the new shared dry_run_pipeline function.
• Agent CLI inputs: Refactored _inputs_core to delegate to builder.operations.inputs_ops.build_inputs_for_pipe instead of duplicating bundle validation and input rendering logic.
• Agent CLI run: Added graph generation support with ReactFlow HTML output and side-effect metadata tracking in _run_core.
• Deduplicate models_cmd → models_ops: models_cmd.py is now a thin CLI wrapper delegating to list_models() and format_models_markdown() in models_ops.py, consistent with other agent CLI commands.

Removed

• BuilderLoop and its iterative build-validate-fix cycle (builder_loop.py, builder.py, builder_errors.py). The build-agent CLI now drives spec construction directly.
• pipelex build pipe CLI command and associated MTHDS workflow files (builder.mthds, agentic_builder.mthds, pipe_design.mthds, concept_fixer.mthds, synthesize_image.mthds).

• pipelex-tools runtime dependency: Moved to dev-only dependency; plxt is now invoked via subprocess passthrough.

• Talent system: Removed talent enums, config mappings (talent_preset_mappings), and talent preset tests. Pipe specs accept model presets directly via the model field, making the talent indirection unnecessary.

Fixed

• Per-bundle dedup in pipeline run setup: Fixed duplicate bundle loading when the same bundle appears in multiple sources.
• HTTP utils: Use HEAD-first strategy for URL validation to avoid downloading large payloads unnecessarily.
• library_dirs passthrough: Fixed library_dirs not being forwarded in builder operations (inputs_ops, validate_ops).
• Dry-run status: Fixed validation status reporting for dry-run results.

[v0.22.0] - 2026-03-25

Added

• MiniMax Backend Support: Added integration for MiniMax models (M2, M2.1, M2.5, M2.7 series including high-speed variants) with an all_minimax routing profile.
• check-model Agent CLI Command: Introduced pipelex-agent check-model to validate model references. Provides fuzzy matching suggestions, cross-collection suggestions, and wrong-sigil hints when an invalid model is provided.
• Domain-Qualified Pipe References (pipe_ref): Added a pipe_ref property (domain.code) as the primary pipe index, allowing pipes with the same code to coexist across different domains. Bare code lookups remain supported as a fallback when unambiguous.
• Duplicate Concept Detection: Added validation to warn about duplicate concept declarations across different bundles within the same library.
• Shared TOML Formatter: Extracted a shared format_toml_string utility for consistent multi-line string formatting across the MTHDS factory and Agent CLI.

Changed

• Agent CLI Output Formats: Overhauled pipelex-agent CLI output for LLM agents: concept and pipe commands now output raw TOML to stdout; models and doctor commands output Markdown by default (JSON still available via --format json).
• Unified model Field in Pipe Specs: Replaced type-specific talent fields (llm_talent, extract_talent, img_gen_talent, search_talent) with a single model field accepting model presets directly. Talent mappings removed from pipelex-agent models output accordingly.
• Initialization Checks: check_is_initialized now additionally verifies the presence of plxt.toml.
• Documentation & Handoffs: Updated Claude Code skills plugin docs, CLI references.

Fixed

• Domain Metadata Conflicts: Loading multiple bundles declaring the same domain with differing descriptions or system prompts now emits a warning and keeps the first declaration instead of throwing an error.
• Cross-Domain Execution Tracking: Pipe controllers and the dry-run engine now track visited pipes using pipe_ref instead of bare codes, preventing false-positive loop detections when crossing domains.

Removed

• assemble Agent CLI Command: Removed the pipelex-agent assemble command.

[v0.21.0] - 2026-03-19

Added

• Web Page Extraction: PipeExtract now supports extracting content from web pages via the new linkup-fetch model, with render_js and include_raw_html parameters. Added @default-extract-web-page alias to the extraction model deck.
• New AI Models: OpenAI / Azure: gpt-5.4, gpt-5.4-pro, gpt-5.4-mini, gpt-5.4-nano, gpt-5.2-codex; Mistral: mistral-small-4
• DocumentContent Enhancements: Added public_url, filename, title, and snippet fields to support web pages and search citations.
• AI Agent Documentation: Integrated mkdocs-llmstxt-md plugin to generate /llms.txt and /llms-full.txt for AI agents.
• Claude Code Skills: Added internal .claude/skills (add-model, test-model) for registering and testing new inference models.

Changed

• Web Fetching → Extract Domain: Moved web fetching from the Search domain into the Extract domain. Replaced FetchJob, FetchWorkerAbstract, and related classes with native handling in PipeExtract and a new GatewayExtractWorker. Split LinkupWorker into dedicated LinkupSearchWorker and LinkupExtractWorker.
• Search Results Unification: SearchResultContent now uses DocumentContent instead of the removed SearchSourceContent.
• Search Job Refactoring: Encapsulated SearchJob parameters into a new SearchJobParams object; updated SearchWorkerAbstract signatures accordingly.
• Documentation Build: Replaced inline HTML generation with a docs/root-index.html template. Updated robots.txt to allow AI crawler access to llms.txt.

Fixed

• Remote Image Fetching: Added error handling (httpx.HTTPStatusError, httpx.RequestError) in GeneratedContentFactory when downloading remote images, with graceful fallback to using the original URL as public_url.

Removed

• page_image_captions parameter: Removed from PipeExtractSpec and ExtractJobParams.

[v0.20.13] - 2026-03-16

Removed

• Remove unused ViewSpec class and related code from graph/reactflow — dead code that was no longer referenced.

[v0.20.12] - 2026-03-16

Fixed

• Fix Google "Sitemap could not be read" error by adding Allow: /sitemap.xml to ROOT_ROBOTS_TXT — Disallow: / was blocking Googlebot from fetching the root sitemap even though Sitemap: pointed to it.

[v0.20.11] - 2026-03-16

Fixed

• Fix docs-deploy-root silently failing due to shell comments breaking the \ continuation chain in the Makefile recipe — root sitemap.xml and updated robots.txt were never deployed in v0.20.10.

[v0.20.10] - 2026-03-16

Fixed

• Fix sitemap double-path bug (latest/0.20.9/page/) caused by site_url including /latest/ while mike also inserts the version prefix during deployment.
• Override canonical URLs, og:url, and JSON-LD url to always point to /latest/ via template override.
• Add root-level sitemap.xml generation with /latest/ URLs to docs-deploy-root.

[v0.20.9] - 2026-03-16

Changed

• Reorganize documentation site architecture and navigation.
• Add SEO meta descriptions to all doc pages.
• Redesign README: lead with value, collapse details.

[v0.20.8] - 2026-03-13

• Add needs_model_specs=True to agent CLI run and models commands.

[v0.20.7] - 2026-03-12

• Bump mthds dependency to >=0.1.1.

[v0.20.6] - 2026-03-12

Added

• needs_model_specs parameter: New option on Pipelex.make() and CLI factories to load real gateway model specs without enabling full inference. Validation commands (pipelex validate, pipelex-agent validate) now fetch live model specs, improving accuracy of pipe/method/bundle validation.

Removed

• Deprecated pipelex_inference backend: Removed all traces of the legacy backend — config files, enum values, migration logic, deprecation warnings, and documentation admonitions. The transition to pipelex_gateway is complete.

Fixed

• pipelex init inference --local: Fixed file filter to also copy .md files (e.g. Gateway model lists) to the backends directory, not just .toml files.
• Gateway terms persistence: Hardened failure handling — when terms acceptance can't be persisted (e.g. unwritable global directory), gateway is now properly disabled in backends.toml as a fallback, preventing GatewayTermsNotAcceptedError at runtime.

[v0.20.5] - 2026-03-09

Changed

• Config paths use pathlib.Path: All ConfigLoader properties and methods now return Path instead of str. Consumer code across the config system (doctor, init, backends, routing, credentials, telemetry, agent CLI) updated accordingly.
• Layered config resolution for pipelex doctor and pipelex init: Config files are now resolved with project-first, global-fallback layering per file. Fixed replace_backend_file using CWD-relative path that broke when run from another directory.
• Gateway terms are explicitly global: update_service_terms_acceptance now targets ~/.pipelex/ by default, since gateway terms are a user-level agreement, not project-level.
• Tweaks to the pipelex-agent CLI to play well with the Claude Code plugin and skills.

[v0.20.4] - 2026-03-04

• Improve README.md instructions for Claude Code and MTHDS skills.

[v0.20.3] - 2026-03-04

Fixed

• Rewrote README with updated CV Batch Screening example, corrected CLI command, inputs schema, Python snippet, and pipe descriptions.

[v0.20.2] - 2026-03-04

Fixed

• Fixed removal of the --reset flag from the init command docs.
• Fixed the name of the PipeSequence results within a batch.

[v0.20.1] - 2026-03-03

Changed

• Cost Report Consistency: Renamed platform_llm_id field to platform_model_id in LLMTokenCostReport, aligning with all other report types (ImgGen, Extract, Search, Fetch) that already use platform_model_id.
• Test Coverage: Added linkup backend to the coverage test profile.

[v0.20.0] - 2026-03-03

Added

• CLI Authentication: pipelex login command that initiates a browser-based OAuth flow (GitHub/Google) to authenticate with the Pipelex Gateway and save the API key locally.
• Search Filtering: from_date, to_date, include_domains, and exclude_domains options on the PipeSearch operator, routed through a new GatewaySearchWorker.
• Expanded Cost Reporting: Token usage and cost tracking now covers Search, Fetch, Extraction, and Image Generation jobs, in addition to LLMs.
• New Models: Added gpt-5.3-codex (text, images, pdf inputs) and nano-banana-2.

Changed

• Search Configuration: Renamed Linkup model IDs from linkup/standard to linkup-standard (and deep variant), simplified presets to $standard and $deep.
• Default Pipeline Execution: Changed from in_memory to local.
• Default Extraction Model: Changed from mistral-document-ai-2505 to azure-document-intelligence; removed mistral-document-ai-2505 from the supported Gateway models list.
• Content Handling: TextAndImagesContent now supports an optional raw_html field.
• Dev Experience: Makefile commands (generate-mthds-schema, update-gateway-models) run in quiet mode by default.

[v0.19.0] - 2026-03-02

Added

• Web Search Integration: Introduced a new PipeSearch operator, native support for Linkup as a search backend provider, SearchResult and SearchResultContent concepts for handling answers with citations, and Model Deck support for search models.
• Graph View Generation: Added a --view option to pipelex validate bundle that generates a ViewSpec JSON (compatible with ReactFlow) for client-side graph rendering without writing files to disk.

Changed

• Test Configuration: Added a search pytest marker, excluded from default test runs.
• mthds bumped to >=0.1.0.
• pipelex-tools bumped to >=0.2.3.
• linkup-sdk>=0.12.0 added as optional dependency for the Linkup search provider.

[v0.18.6] - 2026-03-01

Added

• GitHub URL support for CLI method targets — All CLI commands accepting a method target (pipelex validate method, pipelex run method, pipelex build inputs method, etc.) now accept a public GitHub URL (e.g., https://github.com/org/repo/tree/main/methods/my-method). The repository is cloned automatically, and the method package is discovered and validated. Subdirectory URLs are supported.
• Local path support for CLI method targets — Method targets can now also be local filesystem paths pointing to a directory containing a METHODS.toml. This works for both pipelex and pipelex-agent CLIs.
• Added clone_default_branch() in mthds package for shallow-cloning a git repository's default branch.

[v0.18.5] - 2026-03-01

Changed

• pipelex-agent assemble command now outputs JSON to stdout by default.

[v0.18.4] - 2026-03-01

Added

• Introduced a lenient loading mode for InferenceBackendLibrary and RoutingProfileLoader: when enabled, logs warnings instead of raising errors for missing credentials, variable fallback failures, or disabled backends — skipping individual backends gracefully rather than crashing the entire load process.
• Added needs_inference parameter to Pipelex.make, make_pipelex_for_cli, and make_pipelex_for_agent_cli to control whether full inference setup (credentials, gateway checks, telemetry) is required.
• Added needs_inference_in_pipelex helper to shared_pytest_plugins to replace the inverted logic of the previous helper.

Changed

• CLI Robustness: pipelex build, validate, show, graph, inputs, and which commands now run with needs_inference=False, allowing them to succeed even when backend credentials are missing or incomplete.
• Refactored Pipelex.make: renamed disable_inference to needs_inference (default True). When False, enables lenient backend loading, uses a dummy remote config, disables telemetry, and skips validate_model_deck().

Deprecated

• is_inference_disabled_in_pipelex in shared_pytest_plugins — migrate to needs_inference_in_pipelex.

[v0.18.3] - 2026-02-27

• Updated pipelex-tools dependency to 0.2.1

[v0.18.2] - 2026-02-27

Added

• Inputs path resolution — Relative file paths (e.g., "url": "data/invoice.pdf") inside inputs.json are now resolved relative to the inputs file's parent directory, making bundle directories self-contained and portable. Applies to both user and agent CLIs. Handles HTTP, data:, pipelex-storage://, and absolute paths by leaving them unchanged.

Changed

• pipelex-agent init defaults to local project — The init command now targets the project-level .pipelex/ directory by default (at detected project root) instead of auto-detecting. The --local flag has been removed. Use --global/-g to target ~/.pipelex/. Errors out if no project root is found without -g.
• pipelex-agent doctor supports --global/-g — The doctor command now accepts --global/-g to check the global ~/.pipelex/ directory. Without the flag, it auto-detects project .pipelex/ if present, else falls back to ~/.pipelex/.
• pipelex-agent init --config improved help — The --config option help text now shows the JSON schema with field types upfront for better discoverability.

Fixed

• pipelex-agent init -g service agreement — The --global flag now correctly writes the gateway service terms acceptance (pipelex_service.toml) to the target directory instead of always writing to the auto-detected config dir.
• Deterministic file discovery order — find_files_in_dir now sorts rglob/glob results for consistent ordering across platforms and Python versions. On Linux with Python < 3.13, rglob returned filesystem-order results, which could cause test_validate_all to fail by picking up a test package's METHODS.toml before pipelex internal bundles.

[v0.18.1] - 2026-02-25

Fixed

• Fix docs deployment failure caused by shell metacharacters (parentheses) in Makefile PRINT_TITLE macro argument for docs-deploy-root target
• Fix mike unable to find mkdocs binary by adding venv bin/ to PATH in all mike-based Makefile targets

[v0.18.0] - 2026-02-25

Highlights

• Pipelex Gateway — The deprecated pipelex_inference backend is now replaced by pipelex_gateway, featuring remote model configuration fetching so you always have access to the latest models without updating Pipelex.

Getting your API key: 1. Get your API key at app.pipelex.com 2. Add it to your .env file: PIPELEX_GATEWAY_API_KEY=your-key-here

Gateway Supported models — included with the Free API Key - Language Models (LLM): - OpenAI: all models up to GPT-5.2 and Codex - Anthropic: Claude 3.7 Sonnet through Claude 4.5 Haiku/Sonnet/Opus - Google: Gemini 2.0/2.5 Flash, Gemini 2.5 Pro, Gemini 3.0 Flash/Pro - xAI: Grok 3/mini, Grok 4 (+ fast reasoning variants) - Open-source: Mistral Large 3, DeepSeek v3.1/3.2/Speciale, GPT-OSS 20B/120B1, Kimi K2, Phi-4, Qwen3-VL 235B - Document Extraction: Mistral Document AI, Azure Document Intelligence, DeepSeek OCR - Image Generation: GPT-Image 1/1.5, Flux 2 Pro, Nano Banana/Pro

Accepting the Terms of Service:

When you run pipelex init, you'll be prompted to accept the Gateway terms of service. By using Pipelex Gateway, telemetry is automatically enabled and identified by your API key (hashed for security) to monitor service quality and enforce fair usage. We only collect technical data (model names, token counts, latency, error rates)—never your prompts, completions, or business data. See our Privacy Policy.

⚠️ Migration deadline: If you were using pipelex_inference, please migrate soon—the legacy service will be shut down within few days. Get your new Gateway key at app.pipelex.com.

• Execution Graph Visualization System (preview feature) — Comprehensive tracing and visualization for pipeline executions.

CLI: --graph flag on pipelex run generates execution graphs. New pipelex graph render <graph.json> command for post-run rendering.

Viewers: Interactive ReactFlow viewer (reactflow.html) with pan/zoom and node inspector. Mermaid diagrams (mermaidflow.html) with subgraphs and clickable nodes.

Makefile: make view-graph (vg) and make serve-graph (sg) to start a local graph viewer.

• Pydantic Structure Generation — Two new CLI commands bridge Pipelex's declarative concepts with your Python code:

pipelex build structures /library_dir/ — Generates Pydantic models from all concept definitions found in the specified library directory. Now you have your structures as Python code: you can iterate on them, add custom validation functions, or use them as type hints in your code.

pipelex build runner — Now automatically generates both the Python runner file AND the required Pydantic structures. When you run this command, it creates a complete, ready-to-execute Python script that imports the generated structures, so you can immediately use typed objects in your pipeline code.

See the Build Commands documentation for usage examples.

• New Backends & Models:
• Hugging Face Inference — Support for Hugging Face Inference API, including qwen-image text-to-image model.
• Google gemini-3.0-flash-preview
• Mistral OCR latest model mistral-ocr-2512
• Scaleway inference provider support for open-source models

• Portkey AI backend integration for unified access to multiple models through a single API key

• Document Support in PipeLLM — Include Document objects (like PDFs) directly in prompts using @variable or $variable syntax. Supports single documents, multiple documents, and lists, combinable with text and image inputs.

• PipeCompose Construct Mode — New mode for deterministically building StructuredContent objects without an LLM. Compose fields from working memory variables, fixed values, templates, and nested structures.

• Content Storage System — Configurable storage for generated artifacts (images, extracted pages). Supports local filesystem (.pipelex/storage/), in_memory, AWS S3 (pip install pipelex[s3]), and Google Cloud Storage (pip install pipelex[gcp-storage]). Cloud providers support both public URLs and time-limited signed URLs. Content referenced via stable pipelex-storage:// URIs.

• Langfuse & OpenTelemetry Observability — New OpenTelemetry-based observability system enables powerful tracing and Evals through Langfuse integration. Also supports OTLP-compatible backends (Datadog, Honeycomb, etc.). Configured via .pipelex/telemetry.toml.

• Python 3.14 Support — Officially tested and supported.

• Agent CLI (pipelex-agent): New machine-first CLI for AI agents with structured JSON output for all commands (build, run, validate, inputs, concept, pipe, assemble, graph, models, doctor).

• LLM Reasoning Controls: Unified support for "Thinking" models (Chain of Thought) with reasoning_effort, reasoning_budget, and thinking_mode parameters. Supports Anthropic Extended Thinking, Google Gemini Thinking, OpenAI Reasoning (o1/o3), and Mistral/Magistral models. Includes new presets: $deep-analysis and $quick-reasoning.
• Image-to-Image Generation: PipeImgGen now supports input images via input_images field, with InputImagesTaxonomy for provider-specific handling and variable reference detection ({{ var }}, $var, @var) in prompts.
• PipeCompose "Construct" Mode: New construct mode for building structured objects (dictionaries/Pydantic models) directly from variables.
• Nested concepts in inline structures: You can now define nested structures for your concepts in your .plx files. Learn more here: Nested Concepts in Inline Structures.

Breaking Changes

• CLI restructure: method / pipe subcommands — pipelex run, pipelex validate, and all pipelex build subcommands (runner, inputs, output) now require an explicit method or pipe keyword. For example: pipelex run method my-method or pipelex run pipe scoring.compute. The old pipelex run <target> form is no longer supported. The agent CLI (pipelex-agent) follows the same structure.

Added

• Method resolution (cli/method_resolver.py) — resolves installed method names to pipe codes and library directories. Integrates with mthds discovery module to find methods in ~/.mthds/methods/ and ./.mthds/methods/.
• run method command — run an installed method by name, optionally overriding the pipe with --pipe.
• validate method command — validate all bundles in an installed method.
• build runner/inputs/output method commands — generate build artifacts for installed methods.
• Persistent Credential Storage (~/.pipelex/.env): pipelex init now prompts for missing API keys after backend selection and saves them to ~/.pipelex/.env with 0600 permissions. Credentials are loaded automatically at startup (global defaults, overridden by project .env).
• pipelex init credentials Command: New standalone focus to re-enter API keys without resetting configuration. Scans enabled backends, detects missing env vars, and prompts only for those.
• Package Management System: Introduced a full package manager for MTHDS with METHODS.toml manifests, methods.lock lockfile generation, local path and remote Git-based dependency resolution (MVS), a local package cache (~/.mthds/packages), and a new pipelex pkg CLI command group (init, list, add, lock, install, update, publish, search, inspect, graph).
• Hierarchical Domains: Support for dot-separated domain namespaces (e.g., legal.contracts.shareholder) and cross-package references via alias->domain.pipe syntax.
• PipelexRunner: Introduced the PipelexRunner class as the primary entry point for executing pipelines, replacing PipelexClient and standalone execute_pipeline/start_pipeline functions.
• Linting & Formatting: Integrated plxt (Pipelex Tools) for formatting and linting .mthds and .toml files, including CI/CD checks alongside ruff.
• JSON Schema: Automatic generation of mthds_schema.json for standard definition, IDE validation, and auto-completion.
• Dev CLI: Added pipelex-dev CLI for internal development tasks (e.g., schema generation).
• CSP Nonce Support: Added Content Security Policy nonce support for generated HTML graphs (Mermaid/ReactFlow) to enable secure rendering in VS Code webviews.
• OpenRouter Backend: Added OpenRouter as an inference backend with 337 chat model definitions and 14 image generation models.
• Builder Auto-Repair: Self-healing capabilities including auto-generation of undeclared concepts, multiplicity mismatch fixes, and pruning of unreachable pipes/unused concepts.
• Concept Field Features: Added choices support (compiles to Literal types/Enums) and explicit list type with item_type/item_concept_ref.
• PipeFunc Execution Error Context: When a @pipe_func function fails during execution, the error message now includes detailed context: the function name, actual input values from working memory, expected output type, and the original error with its type. This makes debugging PipeFunc errors much easier.
• pipelex build output CLI Command: New command to generate example output JSON for a pipe, complementing the existing pipelex build inputs command. Shows the expected output structure based on the pipe's output concept type, with multiplicity support. For pipes with native.Anything output (e.g., PipeCondition with different mapped pipe outputs), displays all possible outputs from mapped pipes.
• Telemetry System: Introduced anonymous usage tracking and exception capture for CLI commands (graph render), reporting to both user-configured and Pipelex analytics endpoints.
• PipeExtract Operator Validation: Added strict input validation that raises configuration errors for incompatible input types or when document-specific parameters are used with image inputs.
• PipeCondition Output Auto-Fix in Builder Loop: The pipe builder now automatically fixes PipeCondition output concept errors during validation. If all mapped pipes have the same output, the PipeCondition output is set to that concept; otherwise it's set to native.Anything.
• PipeFunc Return Type Validation: Added validation to ensure that a PipeFunc function's return type matches the output concept's structure class.
• URL Validation on ImageContent and DocumentContent: Both models validate external resources (HTTP/HTTPS URLs via HEAD request, local file paths via existence check) through a validate_resources() method called during pipeline execution (validate_before_run) rather than at model instantiation. Internal URIs (data:, pipelex-storage://) skip validation entirely. Validation is skipped in dry-run mode where inputs use mock URLs.
• Literal Type Support in Dry Run Mocks: DryRunFactory now detects Literal type annotations (including Optional[Literal[...]]) and generates valid mock values by randomly picking from the allowed choices instead of producing invalid random strings.
• Literal Type Support in pipelex build inputs/pipelex build output: The concept representation generator now handles Literal fields by picking a random value from the allowed choices, and generates mock URL patterns for url fields.
• Literal Error Handling in Validation Messages: Pydantic validation error formatting now recognizes literal_error types and displays them as "Invalid choice errors" with the actual value and expected options.
• PipeRunError Catching in Bundle Validation: validate_bundle now catches PipeRunError during dry run and wraps it in ValidateBundleError with a clear message.
• ValidationError Catching in Pipeline Execution: execute_pipeline now catches Pydantic ValidationError from input construction, formats the errors, and raises a PipeExecutionError with a clear message.
• Broader Error Handling in CLI pipelex run: The CLI run command now catches PipelexError in addition to PipelineExecutionError, providing better error messages for failures that occur outside the pipeline execution itself.
• Content Filenames: Added filename field to ImageContent and DocumentContent, with auto-population from local file paths via new extract_filename_from_uri helper.
• Batch Validation Error Type: Introduced PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION for naming conflicts in batch operations.
• Documentation: Added naming convention rules to builder.plx and pipe_design.plx (batch input lists should be plural, item names singular).
• --library-dir CLI Option: --library-dir / -L option for pipelex run, pipelex validate, and pipelex build subcommands (one-shot-pipe, partial-pipe) to specify additional directories for searching pipe definitions. Can be specified multiple times.
• Automatic File Loading: The core pipeline execution functions (pipelex.execute_pipeline, pipelex.start_pipeline) can now directly load a pipeline from a file path via a new bundle_uri parameter.
• Dry Run Mode: pipelex run --dry-run executes pipeline logic without API calls, useful for validating structure and generating orchestration graphs. Combine with --mock-inputs to generate mock data for missing required inputs.
• | with_images Jinja2 Filter: Explicitly extract and include all nested images from complex data structures (e.g., Page objects or custom concepts with Image fields). Renders the object's text representation while making associated images available to the LLM.
• System Prompt Media Support: Reference images and documents in system_prompt using the same $variable and @variable syntax as the user prompt.
• Pipelex Gateway Service: Terms of service management via .pipelex/pipelex_service.toml and interactive acceptance flow in pipelex init and pipelex init agreement.
• Gateway Available Models Documentation: Auto-generated reference of all LLM, Document Extraction, and Image Generation models available through the Gateway.
• Configurable Retry Logic: Exponential backoff for inference API calls, configurable in pipelex.toml under [cogt.tenacity_config]
• Context Manager Support: The Pipelex class now supports with Pipelex.make(): ... for graceful shutdown
• Validation Improvements:
• Pipelex Bundle concept keys: prevent bundles from re-creating a native concept
• PipeSequence: output multiplicity must match the last step's output multiplicity
• PipeFunc: output multiplicity must match the function return type (ListContent subclass for multiplicity=true)
• Rendering Protocols: Three new @runtime_checkable protocols (ImageRenderable, TagRenderable, TextFormatRenderable) to formalize the interaction between data types and Jinja2 filters.
• Unified URI Handling System: New pipelex.tools.uri module providing type-safe parsing for HTTP/HTTPS URLs, local file paths, file URIs, pipelex-storage:// URIs, and base64 data URLs.
• Automatic Input Data Storage: Pipeline pre-processing step that converts large data: URLs in an ImageContent or DocumentContent into pipelex-storage:// URIs for improved performance, by storing the data with the configured storage provider. Configurable via is_normalize_data_urls_to_storage in pipelex.toml.
• pipelex build inputs Command: New CLI command to generate example input JSON files for pipes. Supports --library-dir (-L) to specify library directories.
• Pipe Code Syntax Validation: Bundle validation now checks that pipe codes and main_pipe values use valid snake_case syntax, with proper error categorization (INVALID_PIPE_CODE_SYNTAX).

Changed

• CI Runner Optimization: Split self-hosted runners into dedicated test (D32) and lint (D4) pools with pre-baked Docker image (Python 3.10-3.14, UV, dependency cache) for faster job startup.
• MTHDS Light Client Extraction: The light client protocol (runner, pipeline models, pipe output abstractions) has been extracted from pipelex into the new mthds package on PyPI. Pipelex now depends on mthds>=0.0.1 and implements its RunnerProtocol.
• Global/Local Config Split: pipelex init now creates configuration in ~/.pipelex/ (global) by default. Use pipelex init --local to create project-level overrides in {project_root}/.pipelex/. Config loading merges: package defaults → global → project → overrides.
• IDE Extension Detection: Extension check now uses code --list-extensions / cursor --list-extensions for reliable detection instead of folder scanning. Shows separate marketplace links for VS Code (Microsoft Marketplace) and Cursor (Open VSX Registry).
• Quieter pipelex init: Removed verbose file listing and reset messages from config initialization output.
• pipelex init Help Text: Detailed focus option descriptions now shown in pipelex init --help explaining each focus (all, config, credentials, inference, routing, telemetry, agreement).
• File Extension: Pipeline definitions now use .mthds instead of .plx; the project refers to "Methods" (MTHDS) rather than "Pipelex workflows".
• Syntax: In PipeParallel definitions, parallels field renamed to branches; plx_config in pipelex.toml renamed to mthds_config.
• Documentation: Comprehensive update to reflect .mthds file format, package management, and new project structure.
• Model Deck Updates: Default premium model now claude-4.6-opus; added Mistral models (mistral-small-3.2, mistral-large, magistral series) and gpt-5 placeholders.
• Jinja2 Rendering: Image objects now replaced with placeholders (e.g., [Image 1]) during text generation; async filters only registered in async environments.
• Backend Configurations: Added effort_to_level_map and effort_to_budget_maps for reasoning translation; disabled Google Vertex AI by default.
• Dependencies: Updated pypdfium2, anthropic, and mistralai version constraints.
• pipelex run --dry-run: No longer pretty prints the main_stuff output, matching the expected behavior for dry runs where no actual inference occurs.
• pipelex build structures Command: Now uses a lightweight loading mechanism that only processes domains and concepts, skipping pipe loading and validation entirely. This fixes the chicken-and-egg problem where structure generation would fail due to pipe validation errors before the structures were even created. Added --force / -f flag to regenerate all structures without checking if classes already exist.
• Test Profile System: Refactored integration tests to use a new configuration system (.pipelex/test_profiles.toml) with dev, ci, and full profiles for controlling which AI models are used in parametrized tests, replacing runtime filtering and hardcoded model lists.
• pipelex run --graph Flag: Now acts as an override for pipelex.toml settings instead of defaulting to true.
• Default Image Generation Models: Updated in base_deck.toml: base-img-gen: flux-2-pro, best-img-gen: nano-banana-pro, fast-img-gen: gpt-image-1-mini
• Remote Configuration: Updated service URL to version 3.
• GatewayExtractWorker: Now checks model capabilities before attempting image captioning.
• Change the output validation of PipeCondition: If all mapped pipes have the same output concept, PipeCondition's output MUST be that same concept. If mapped pipes have different output concepts, PipeCondition's output MUST be the native concept Anything.
• CLI: Changed pipelex validate all to pipelex validate --all (or -a).
• StructuredContent.rendered_html(): Now recursively calls rendered_html() on nested StuffContent fields instead of using json2html conversion. Also skips None values and uses HTML table format.
• Batch Pipe Validation: Enforced stricter naming rules for batch specs—input_item_name must differ from input_list_name and not shadow existing input keys, with clearer error messages suggesting plural/singular conventions.
• Jinja2 Integration Refactored to Protocol-Based Approach: Replaced the Jinja2Registry singleton and handler functions with a decoupled protocol-based system, eliminating circular dependencies between the template layer and core domain logic.
• StuffArtefact Redesigned as Delegation Adapter: Now a lightweight, immutable adapter that delegates attribute access directly to underlying Stuff and StuffContent objects, improving performance and providing more intuitive field access.
• Image Extraction Moved to Content Types: StructuredContent, ListContent, ImageContent, and TextAndImagesContent now implement the ImageRenderable protocol, replacing centralized handler logic.
• ⚠️ Breaking: User override config renamed from pipelex_super.toml to pipelex_override.toml.
• Image Generation Architecture: Refactored to taxonomy-based approach. Standardizes parameter translation (aspect_ratio, quality, output_format) to provider-specific APIs.
• Document Extraction Improvements: pypdfium2 extractor now extracts embedded images from PDFs. Response parsing uses dedicated Pydantic schemas for validation.
• Default Model Change: extract_text_from_visuals deck now defaults to azure-document-intelligence
• pipelex_inference replaced by pipelex_gateway: See Highlights for migration details. New PIPELEX_GATEWAY_API_KEY environment variable; default routing profiles updated to pipelex_gateway_first.

• Telemetry System Split: Now two separate streams: 1. Pipelex Gateway telemetry for service monitoring (never collects prompts/completions/business data) 2. Custom telemetry to user-configured backends 3. Config updated accordingly (telemetry.toml):

  • Renamed [posthog] to [custom_posthog] to distinguish user's PostHog from Pipelex Gateway telemetry
  • Added new [custom_portkey] section with force_debug_enabled and force_tracing_enabled settings

• Main Configuration Overrides Updated (.pipelex/pipelex.toml):

• pipelex_override.toml (final override) renamed from pipelex_super.toml to pipelex_override.toml and moved from repo root to .pipelex/ directory
• telemetry_override.toml (personal telemetry settings)

• is_generate_cost_report_file_enabled default changed from true to false

• Documentation:

• Clarified Setup (first run) vs Configuration (TOML reference), added a Setup overview page, and added contributor docs for configuration defaults/overrides.
• Added the "Under the Hood" page documenting the execution graph tracing system.
• pipelex init: Now creates a documented telemetry.toml template instead of prompting for preferences
• Model Catalog Updated: Latest models (gpt-5.1, claude-4.5-opus, gemini-3.0-pro, etc.) and updated waterfalls in base_deck.toml
• Model Constraints Refactored: From simple lists to structured valued_constraints dictionaries (e.g., valued_constraints = { fixed_temperature = 1 })
• OpenAI Responses API: New implementation now differentiates between openai_completions and openai_responses
• CLI Initialization: Commands refactored to use centralized Pipelex initialization factory for improved error handling
• pipelex doctor: Enhanced to detect outdated telemetry.toml formats and suggest fixes
• --output-dir Option: The runner, structures, and inputs CLI commands now accept this option
• Cost Report: Now displays a note clarifying that it only includes LLM costs
• description Field Now Required: In PipeAbstract, PipeBlueprint, and PipeSpec classes.
• Configuration: New [pipelex.pipeline_execution_config.graph_config] section in pipelex.toml for fine-grained control over graph generation, data embedding, and rendering options.
• CLI: All pipelex commands now accept --no-logo to suppress the Pipelex banner in the terminal — useful to reduce tokens.

Fixed

• find_project_root Home Directory Bug: The project root walker no longer considers the home directory (~) as a project root, even if it contains stray marker files like package.json.
• Python 3.10 Compatibility: Fixed datetime.UTC import (Python 3.11+) to use datetime.timezone.utc.
• Graph Rendering: Fixed dashed edge rendering for PipeBatch and PipeParallel relationships.
• Image Generation Response Parsing: Hardened image response parsing to handle varied provider response formats more robustly.
• Helpful Error for get_stuff_as(ListContent[T]): When users incorrectly call get_stuff_as("name", ListContent[Something]) instead of get_stuff_as_list("name", Something), the error message now explicitly suggests using get_stuff_as_list().
• PipeFunc ListContent[T] Validation: Fixed validation rejecting valid ListContent[T] return types for array outputs (T[]). Previously, a function returning ListContent[Expense] would fail validation for output = "Expense[]" with a misleading error. The validation now correctly extracts and validates the generic type parameter from Pydantic's metadata.
• PipeFunc Class Name Matching: Fixed validation failing when the return type class and concept structure class are logically the same but loaded from different contexts. The validation now uses class name matching as a fallback, allowing ListContent[Expense] to match Expense[] even if the Expense class objects differ.
• Fixed PipeImgGen not properly converting ImageContent to custom subclasses (e.g., Receipt(ImageContent)). The pipe now uses smart_dump() before model_validate() to correctly instantiate the output concept's structure class.
• Corrected output directory creation logic in pipelex run to properly respect the --no-graph flag and configuration settings.
• Fixed a bug when trying to print HTML content in a TextContent object.
• Fixed the Pipelex CLI for generating structures, inputs, runner files.
• Fixed @pipe_func decorated functions showing "function not found" instead of explaining why the function is ineligible (e.g., missing return type annotation).
• Fixed PipeLLM with list output (e.g., output = "Item[]") not producing ListContent when run inside a nested PipeSequence with batch_over.
• Duplicate Pipe Error Message: When a pipe code is declared in multiple .plx files (or twice in the same file), the error message now shows which bundle file(s) contain the conflicting declarations instead of a misleading message about "running the same pipe twice in the same pipeline".
• Fixed pipelex build runner and pipelex build inputs generating string placeholders (e.g., "number_int | float") instead of numeric values for Number concepts with int | float union type fields.
• Fixed structure generation failing with PydanticUserError when a concept structure references native concepts (e.g., native.Html). The generator now properly resolves native concept refs to their content classes (e.g., HtmlContent) with correct imports.
• Nested Image Handling: Images nested within structured data are now properly replaced with [Image N] tokens. The | with_images filter and ImageRegistry system correctly extract images from complex nested structures.
• PipeCondition Validation: Output multiplicity validation now works correctly.
• Error Reporting: PipeCompose validation errors now include formatted details and failing field values.
• Duplicate Pipeline Registration: Running a pipeline from a file that was also part of a pre-loaded library (via PIPELEXPATH) no longer causes a duplicate domain registration error. The system now tracks absolute paths of loaded library files and skips files already loaded.
• pipelex build structures: Corrected output file naming and resolved import path generation.
• ConceptFactory.make_from_blueprint: Now correctly handles native concepts.

Removed

• Legacy Client & Execution Modules: Removed PipelexClient, protocol files, and standalone pipelex.pipeline.execute/pipelex.pipeline.start modules (replaced by PipelexRunner and mthds package).
• Legacy Config: Removed plx_config.py and .plx-specific configuration references.
• pipelex kit Command: The kit commands have been removed from the main CLI. They are now internal tools for Pipelex contributors only, available via pipelex-dev kit rules.
• pipelex kit migrations Command: Removed entirely.
• pipelex kit remove-rules Command: Removed entirely.
• PLX Syntax Agent Rules: Removed write_pipelex.md and run_pipelex.md agent rules. These PLX syntax guides are no longer installed in client projects.
• [pipelex.kit_config] Configuration: Removed from client project configuration (.pipelex/pipelex.toml).
• openai_utils Module: Removed pipelex.plugins.openai.openai_utils; logic now in centralized image preparation utilities.
• Pipeline Tracking feature: Removed entirely, including the pipelex/pipeline/track module, PipelineTracker components, related configuration, tracker calls in pipe controllers, and associated documentation.
• Flow Generator: The old flow generator has been removed.

Security

• CSP Nonce Support: Added Content Security Policy nonce for generated HTML graph outputs.

Deprecated

• pipelex_inference backend in favor of pipelex_gateway (marked as "🛑 Legacy" in configuration template)

For Contributors

• Technical Documentation: Added a new "Under the Hood" page documenting the StuffArtefact delegation pattern and image rendering architecture.
• Enhanced Testing: Added extensive unit and integration tests for the protocol-based rendering system, including nested image extraction and filter error conditions.
• Agent Rules: Added pipelex_standards.md outlining standards for the Pipelex configuration system, also included as rules for AI development agents.
• Agent Rules: New target make agent-check for faster linting.
• ⚠️ Breaking — Content Handling Overhaul: GeneratedImage replaced by internal GeneratedImageRawDetails; ImageContent is now standard (without base_64 field). PipeExtract outputs PageContent list directly. Content persistence now handled automatically by storage system.
• ⚠️ Breaking — Image Prompt Representation: Redesigned PromptImage models—consolidated PromptImagePath and PromptImageUrl into PromptImageUri; now uses Pydantic discriminated union for URI, base64, and raw bytes sources.
• Centralized Image Preparation: Moved image fetching and base64 conversion logic to pipelex.cogt.image.prompt_image_utils, simplifying LLM provider plugins (Anthropic, Google, Mistral, OpenAI).
• Unified Resource Loading: Updated all file/URL reading components (PDF renderers, document extractors) to use the new URI handling system, replacing the legacy pipelex.tools.misc.path_utils module.
• Async HTTP Fetching: Renamed fetch_file_from_url_httpx_async to fetch_file_from_url_httpx; removed redundant synchronous version.
• PreparedImage Abstraction: New models (PreparedImageHttpUrl, PreparedImageBase64) representing images ready for LLM provider APIs.
• Pipelex Gateway Model Management: New CLI commands (pipelex-dev update-gateway-models, pipelex-dev check-gateway-models) and corresponding make targets (ugm, cgm) to generate and verify gateway model catalog. CI now validates this documentation is up-to-date.
• Test Suite Pre-flight Check: Verifies Gateway terms acceptance before running tests, providing clear error messages.
• Content Rendering: StuffContent.rendered_* methods now provide both synchronous and asynchronous variants.

Migration Notes

• Telemetry configuration migration: If you have an existing telemetry.toml, rename:
• [posthog] → [custom_posthog]
• [posthog.tracing] → [custom_posthog.tracing]
• [posthog.tracing.capture] → [custom_posthog.tracing.capture]
• Or run pipelex init telemetry --reset to regenerate the file with the new structure

Refactored

• Anthropic Backend: Internal streaming for standard completions to prevent SDK timeouts, configurable structured output timeout (structured_output_timeout_seconds), improved error mapping, and increased Bedrock Claude max_tokens (8K → 64K) with removal of max_output_tokens_limit constraint.
• ⚠️ Breaking — Pipe I/O Specification: The output (and inputs) of a pipe is now a StuffSpec object that holds the concept and the multiplicity.
• Naming Convention: Renamed domain to domain_code where relevant.
• Dry Run Methods: Refactored the dry run methods of the PipeAbstract class.

[v0.18.0b4] - 2026-02-23

Added

• Persistent Credential Storage (~/.pipelex/.env): pipelex init now prompts for missing API keys after backend selection and saves them to ~/.pipelex/.env with 0600 permissions. Credentials are loaded automatically at startup (global defaults, overridden by project .env).
• pipelex init credentials Command: New standalone focus to re-enter API keys without resetting configuration. Scans enabled backends, detects missing env vars, and prompts only for those.
• Package Management System: Introduced a full package manager for MTHDS with METHODS.toml manifests, methods.lock lockfile generation, local path and remote Git-based dependency resolution (MVS), a local package cache (~/.mthds/packages), and a new pipelex pkg CLI command group (init, list, add, lock, install, update, publish, search, inspect, graph).
• Hierarchical Domains: Support for dot-separated domain namespaces (e.g., legal.contracts.shareholder) and cross-package references via alias->domain.pipe syntax.
• PipelexRunner: Introduced the PipelexRunner class as the primary entry point for executing pipelines, replacing PipelexClient and standalone execute_pipeline/start_pipeline functions.
• Linting & Formatting: Integrated plxt (Pipelex Tools) for formatting and linting .mthds and .toml files, including CI/CD checks alongside ruff.
• JSON Schema: Automatic generation of mthds_schema.json for standard definition, IDE validation, and auto-completion.
• Dev CLI: Added pipelex-dev CLI for internal development tasks (e.g., schema generation).
• CSP Nonce Support: Added Content Security Policy nonce support for generated HTML graphs (Mermaid/ReactFlow) to enable secure rendering in VS Code webviews.
• OpenRouter Backend: Added OpenRouter as an inference backend with 337 chat model definitions and 14 image generation models.

Changed

• CI Runner Optimization: Split self-hosted runners into dedicated test (D32) and lint (D4) pools with pre-baked Docker image (Python 3.10-3.14, UV, dependency cache) for faster job startup.
• MTHDS Light Client Extraction: The light client protocol (runner, pipeline models, pipe output abstractions) has been extracted from pipelex into the new mthds package on PyPI. Pipelex now depends on mthds>=0.0.1 and implements its RunnerProtocol.
• Global/Local Config Split: pipelex init now creates configuration in ~/.pipelex/ (global) by default. Use pipelex init --local to create project-level overrides in {project_root}/.pipelex/. Config loading merges: package defaults → global → project → overrides.
• IDE Extension Detection: Extension check now uses code --list-extensions / cursor --list-extensions for reliable detection instead of folder scanning. Shows separate marketplace links for VS Code (Microsoft Marketplace) and Cursor (Open VSX Registry).
• Quieter pipelex init: Removed verbose file listing and reset messages from config initialization output.
• pipelex init Help Text: Detailed focus option descriptions now shown in pipelex init --help explaining each focus (all, config, credentials, inference, routing, telemetry, agreement).
• File Extension: Pipeline definitions now use .mthds instead of .plx; the project refers to "Methods" (MTHDS) rather than "Pipelex workflows".
• Syntax: In PipeParallel definitions, parallels field renamed to branches; plx_config in pipelex.toml renamed to mthds_config.
• Documentation: Comprehensive update to reflect .mthds file format, package management, and new project structure.

Fixed

• find_project_root Home Directory Bug: The project root walker no longer considers the home directory (~) as a project root, even if it contains stray marker files like package.json.
• Python 3.10 Compatibility: Fixed datetime.UTC import (Python 3.11+) to use datetime.timezone.utc.
• Graph Rendering: Fixed dashed edge rendering for PipeBatch and PipeParallel relationships.
• Image Generation Response Parsing: Hardened image response parsing to handle varied provider response formats more robustly.

Removed

• Legacy Client & Execution Modules: Removed PipelexClient, protocol files, and standalone pipelex.pipeline.execute/pipelex.pipeline.start modules (replaced by PipelexRunner and mthds package).
• Legacy Config: Removed plx_config.py and .plx-specific configuration references.

Security

• CSP Nonce Support: Added Content Security Policy nonce for generated HTML graph outputs.

[v0.18.0b3] - 2026-02-11

Highlights

• Agent CLI (pipelex-agent): New machine-first CLI for AI agents with structured JSON output for all commands (build, run, validate, inputs, concept, pipe, assemble, graph, models, doctor).
• LLM Reasoning Controls: Unified support for "Thinking" models (Chain of Thought) with reasoning_effort, reasoning_budget, and thinking_mode parameters. Supports Anthropic Extended Thinking, Google Gemini Thinking, OpenAI Reasoning (o1/o3), and Mistral/Magistral models. Includes new presets: $deep-analysis and $quick-reasoning.
• Image-to-Image Generation: PipeImgGen now supports input images via input_images field, with InputImagesTaxonomy for provider-specific handling and variable reference detection ({{ var }}, $var, @var) in prompts.
• PipeCompose "Construct" Mode: New construct mode for building structured objects (dictionaries/Pydantic models) directly from variables.
• Nested concepts in inline structures: You can now define nested structures for your concepts in your .plx files. Learn more here: Nested Concepts in Inline Structures.

Added

• Builder Auto-Repair: Self-healing capabilities including auto-generation of undeclared concepts, multiplicity mismatch fixes, and pruning of unreachable pipes/unused concepts.
• Concept Field Features: Added choices support (compiles to Literal types/Enums) and explicit list type with item_type/item_concept_ref.
• PipeFunc Execution Error Context: When a @pipe_func function fails during execution, the error message now includes detailed context: the function name, actual input values from working memory, expected output type, and the original error with its type. This makes debugging PipeFunc errors much easier.
• pipelex build output CLI Command: New command to generate example output JSON for a pipe, complementing the existing pipelex build inputs command. Shows the expected output structure based on the pipe's output concept type, with multiplicity support. For pipes with native.Anything output (e.g., PipeCondition with different mapped pipe outputs), displays all possible outputs from mapped pipes.
• Telemetry System: Introduced anonymous usage tracking and exception capture for CLI commands (graph render), reporting to both user-configured and Pipelex analytics endpoints.
• PipeExtract Operator Validation: Added strict input validation that raises configuration errors for incompatible input types or when document-specific parameters are used with image inputs.
• PipeCondition Output Auto-Fix in Builder Loop: The pipe builder now automatically fixes PipeCondition output concept errors during validation. If all mapped pipes have the same output, the PipeCondition output is set to that concept; otherwise it's set to native.Anything.
• PipeFunc Return Type Validation: Added validation to ensure that a PipeFunc function's return type matches the output concept's structure class.
• URL Validation on ImageContent and DocumentContent: Both models validate external resources (HTTP/HTTPS URLs via HEAD request, local file paths via existence check) through a validate_resources() method called during pipeline execution (validate_before_run) rather than at model instantiation. Internal URIs (data:, pipelex-storage://) skip validation entirely. Validation is skipped in dry-run mode where inputs use mock URLs.
• Literal Type Support in Dry Run Mocks: DryRunFactory now detects Literal type annotations (including Optional[Literal[...]]) and generates valid mock values by randomly picking from the allowed choices instead of producing invalid random strings.
• Literal Type Support in pipelex build inputs/pipelex build output: The concept representation generator now handles Literal fields by picking a random value from the allowed choices, and generates mock URL patterns for url fields.
• Literal Error Handling in Validation Messages: Pydantic validation error formatting now recognizes literal_error types and displays them as "Invalid choice errors" with the actual value and expected options.
• PipeRunError Catching in Bundle Validation: validate_bundle now catches PipeRunError during dry run and wraps it in ValidateBundleError with a clear message.
• ValidationError Catching in Pipeline Execution: execute_pipeline now catches Pydantic ValidationError from input construction, formats the errors, and raises a PipeExecutionError with a clear message.
• Broader Error Handling in CLI pipelex run: The CLI run command now catches PipelexError in addition to PipelineExecutionError, providing better error messages for failures that occur outside the pipeline execution itself.
• Content Filenames: Added filename field to ImageContent and DocumentContent, with auto-population from local file paths via new extract_filename_from_uri helper.
• Batch Validation Error Type: Introduced PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION for naming conflicts in batch operations.
• Documentation: Added naming convention rules to builder.plx and pipe_design.plx (batch input lists should be plural, item names singular).

Changed

• Model Deck Updates: Default premium model now claude-4.6-opus; added Mistral models (mistral-small-3.2, mistral-large, magistral series) and gpt-5 placeholders.
• Jinja2 Rendering: Image objects now replaced with placeholders (e.g., [Image 1]) during text generation; async filters only registered in async environments.
• Backend Configurations: Added effort_to_level_map and effort_to_budget_maps for reasoning translation; disabled Google Vertex AI by default.
• Dependencies: Updated pypdfium2, anthropic, and mistralai version constraints.
• pipelex run --dry-run: No longer pretty prints the main_stuff output, matching the expected behavior for dry runs where no actual inference occurs.
• pipelex build structures Command: Now uses a lightweight loading mechanism that only processes domains and concepts, skipping pipe loading and validation entirely. This fixes the chicken-and-egg problem where structure generation would fail due to pipe validation errors before the structures were even created. Added --force / -f flag to regenerate all structures without checking if classes already exist.
• Test Profile System: Refactored integration tests to use a new configuration system (.pipelex/test_profiles.toml) with dev, ci, and full profiles for controlling which AI models are used in parametrized tests, replacing runtime filtering and hardcoded model lists.
• pipelex run --graph Flag: Now acts as an override for pipelex.toml settings instead of defaulting to true.
• Default Image Generation Models: Updated in base_deck.toml: base-img-gen: flux-2-pro, best-img-gen: nano-banana-pro, fast-img-gen: gpt-image-1-mini
• Remote Configuration: Updated service URL to version 3.
• GatewayExtractWorker: Now checks model capabilities before attempting image captioning.
• Change the output validation of PipeCondition: If all mapped pipes have the same output concept, PipeCondition's output MUST be that same concept. If mapped pipes have different output concepts, PipeCondition's output MUST be the native concept Anything.
• CLI: Changed pipelex validate all to pipelex validate --all (or -a).
• StructuredContent.rendered_html(): Now recursively calls rendered_html() on nested StuffContent fields instead of using json2html conversion. Also skips None values and uses HTML table format.
• Batch Pipe Validation: Enforced stricter naming rules for batch specs—input_item_name must differ from input_list_name and not shadow existing input keys, with clearer error messages suggesting plural/singular conventions.

Fixed

• Helpful Error for get_stuff_as(ListContent[T]): When users incorrectly call get_stuff_as("name", ListContent[Something]) instead of get_stuff_as_list("name", Something), the error message now explicitly suggests using get_stuff_as_list().
• PipeFunc ListContent[T] Validation: Fixed validation rejecting valid ListContent[T] return types for array outputs (T[]). Previously, a function returning ListContent[Expense] would fail validation for output = "Expense[]" with a misleading error. The validation now correctly extracts and validates the generic type parameter from Pydantic's metadata.
• PipeFunc Class Name Matching: Fixed validation failing when the return type class and concept structure class are logically the same but loaded from different contexts. The validation now uses class name matching as a fallback, allowing ListContent[Expense] to match Expense[] even if the Expense class objects differ.
• Fixed PipeImgGen not properly converting ImageContent to custom subclasses (e.g., Receipt(ImageContent)). The pipe now uses smart_dump() before model_validate() to correctly instantiate the output concept's structure class.
• Corrected output directory creation logic in pipelex run to properly respect the --no-graph flag and configuration settings.
• Fixed a bug when trying to print HTML content in a TextContent object.
• Fixed the Pipelex CLI for generating structures, inputs, runner files.
• Fixed @pipe_func decorated functions showing "function not found" instead of explaining why the function is ineligible (e.g., missing return type annotation).
• Fixed PipeLLM with list output (e.g., output = "Item[]") not producing ListContent when run inside a nested PipeSequence with batch_over.
• Duplicate Pipe Error Message: When a pipe code is declared in multiple .plx files (or twice in the same file), the error message now shows which bundle file(s) contain the conflicting declarations instead of a misleading message about "running the same pipe twice in the same pipeline".
• Fixed pipelex build runner and pipelex build inputs generating string placeholders (e.g., "number_int | float") instead of numeric values for Number concepts with int | float union type fields.
• Fixed structure generation failing with PydanticUserError when a concept structure references native concepts (e.g., native.Html). The generator now properly resolves native concept refs to their content classes (e.g., HtmlContent) with correct imports.

Removed

• pipelex kit Command: The kit commands have been removed from the main CLI. They are now internal tools for Pipelex contributors only, available via pipelex-dev kit rules.
• pipelex kit migrations Command: Removed entirely.
• pipelex kit remove-rules Command: Removed entirely.
• PLX Syntax Agent Rules: Removed write_pipelex.md and run_pipelex.md agent rules. These PLX syntax guides are no longer installed in client projects.
• [pipelex.kit_config] Configuration: Removed from client project configuration (.pipelex/pipelex.toml).

[v0.18.0b2] - 2026-01-20

Highlights

• Pipelex Gateway — The deprecated pipelex_inference backend is now replaced by pipelex_gateway, featuring remote model configuration fetching so you always have access to the latest models without updating Pipelex.

Getting your API key: 1. Get your API key at app.pipelex.com 2. Add it to your .env file: PIPELEX_GATEWAY_API_KEY=your-key-here

Gateway Supported models — included with the Free API Key - Language Models (LLM): - OpenAI: all models up to GPT-5.2 and Codex - Anthropic: Claude 3.7 Sonnet through Claude 4.5 Haiku/Sonnet/Opus - Google: Gemini 2.0/2.5 Flash, Gemini 2.5 Pro, Gemini 3.0 Flash/Pro - xAI: Grok 3/mini, Grok 4 (+ fast reasoning variants) - Open-source: Mistral Large 3, DeepSeek v3.1/3.2/Speciale, GPT-OSS 20B/120B1, Kimi K2, Phi-4, Qwen3-VL 235B - Document Extraction: Mistral Document AI, Azure Document Intelligence, DeepSeek OCR - Image Generation: GPT-Image 1/1.5, Flux 2 Pro, Nano Banana/Pro

Accepting the Terms of Service:

When you run pipelex init, you'll be prompted to accept the Gateway terms of service. By using Pipelex Gateway, telemetry is automatically enabled and identified by your API key (hashed for security) to monitor service quality and enforce fair usage. We only collect technical data (model names, token counts, latency, error rates)—never your prompts, completions, or business data. See our Privacy Policy.

⚠️ Migration deadline: If you were using pipelex_inference, please migrate soon—the legacy service will be shut down within few days. Get your new Gateway key at app.pipelex.com.

• Execution Graph Visualization System (preview feature) — Comprehensive tracing and visualization for pipeline executions.

CLI: --graph flag on pipelex run generates execution graphs. New pipelex graph render <graph.json> command for post-run rendering.

Viewers: Interactive ReactFlow viewer (reactflow.html) with pan/zoom and node inspector. Mermaid diagrams (mermaidflow.html) with subgraphs and clickable nodes.

Makefile: make view-graph (vg) and make serve-graph (sg) to start a local graph viewer.

• Pydantic Structure Generation — Two new CLI commands bridge Pipelex's declarative concepts with your Python code:

pipelex build structures /library_dir/ — Generates Pydantic models from all concept definitions found in the specified library directory. Now you have your structures as Python code: you can iterate on them, add custom validation functions, or use them as type hints in your code.

pipelex build runner — Now automatically generates both the Python runner file AND the required Pydantic structures. When you run this command, it creates a complete, ready-to-execute Python script that imports the generated structures, so you can immediately use typed objects in your pipeline code.

See the Build Commands documentation for usage examples.

• New Backends & Models:
• Hugging Face Inference — Support for Hugging Face Inference API, including qwen-image text-to-image model.
• Google gemini-3.0-flash-preview
• Mistral OCR latest model mistral-ocr-2512
• Scaleway inference provider support for open-source models

• Portkey AI backend integration for unified access to multiple models through a single API key

• Document Support in PipeLLM — Include Document objects (like PDFs) directly in prompts using @variable or $variable syntax. Supports single documents, multiple documents, and lists, combinable with text and image inputs.

• PipeCompose Construct Mode — New mode for deterministically building StructuredContent objects without an LLM. Compose fields from working memory variables, fixed values, templates, and nested structures.

• Content Storage System — Configurable storage for generated artifacts (images, extracted pages). Supports local filesystem (.pipelex/storage/), in_memory, AWS S3 (pip install pipelex[s3]), and Google Cloud Storage (pip install pipelex[gcp-storage]). Cloud providers support both public URLs and time-limited signed URLs. Content referenced via stable pipelex-storage:// URIs.

• Langfuse & OpenTelemetry Observability — New OpenTelemetry-based observability system enables powerful tracing and Evals through Langfuse integration. Also supports OTLP-compatible backends (Datadog, Honeycomb, etc.). Configured via .pipelex/telemetry.toml.

• Python 3.14 Support — Officially tested and supported.

Added

• --library-dir CLI Option: --library-dir / -L option for pipelex run, pipelex validate, and pipelex build subcommands (one-shot-pipe, partial-pipe) to specify additional directories for searching pipe definitions. Can be specified multiple times.
• Automatic File Loading: The core pipeline execution functions (pipelex.execute_pipeline, pipelex.start_pipeline) can now directly load a pipeline from a file path via a new bundle_uri parameter.
• Dry Run Mode: pipelex run --dry-run executes pipeline logic without API calls, useful for validating structure and generating orchestration graphs. Combine with --mock-inputs to generate mock data for missing required inputs.
• | with_images Jinja2 Filter: Explicitly extract and include all nested images from complex data structures (e.g., Page objects or custom concepts with Image fields). Renders the object's text representation while making associated images available to the LLM.
• System Prompt Media Support: Reference images and documents in system_prompt using the same $variable and @variable syntax as the user prompt.
• Pipelex Gateway Service: Terms of service management via .pipelex/pipelex_service.toml and interactive acceptance flow in pipelex init and pipelex init agreement.
• Gateway Available Models Documentation: Auto-generated reference of all LLM, Document Extraction, and Image Generation models available through the Gateway.
• Configurable Retry Logic: Exponential backoff for inference API calls, configurable in pipelex.toml under [cogt.tenacity_config]
• Context Manager Support: The Pipelex class now supports with Pipelex.make(): ... for graceful shutdown
• Validation Improvements:
• Pipelex Bundle concept keys: prevent bundles from re-creating a native concept
• PipeSequence: output multiplicity must match the last step's output multiplicity
• PipeFunc: output multiplicity must match the function return type (ListContent subclass for multiplicity=true)
• Rendering Protocols: Three new @runtime_checkable protocols (ImageRenderable, TagRenderable, TextFormatRenderable) to formalize the interaction between data types and Jinja2 filters.
• Unified URI Handling System: New pipelex.tools.uri module providing type-safe parsing for HTTP/HTTPS URLs, local file paths, file URIs, pipelex-storage:// URIs, and base64 data URLs.
• Automatic Input Data Storage: Pipeline pre-processing step that converts large data: URLs in an ImageContent or DocumentContent into pipelex-storage:// URIs for improved performance, by storing the data with the configured storage provider. Configurable via is_normalize_data_urls_to_storage in pipelex.toml.
• pipelex build inputs Command: New CLI command to generate example input JSON files for pipes. Supports --library-dir (-L) to specify library directories.
• Pipe Code Syntax Validation: Bundle validation now checks that pipe codes and main_pipe values use valid snake_case syntax, with proper error categorization (INVALID_PIPE_CODE_SYNTAX).

Fixed

• Nested Image Handling: Images nested within structured data are now properly replaced with [Image N] tokens. The | with_images filter and ImageRegistry system correctly extract images from complex nested structures.
• PipeCondition Validation: Output multiplicity validation now works correctly.
• Error Reporting: PipeCompose validation errors now include formatted details and failing field values.
• Duplicate Pipeline Registration: Running a pipeline from a file that was also part of a pre-loaded library (via PIPELEXPATH) no longer causes a duplicate domain registration error. The system now tracks absolute paths of loaded library files and skips files already loaded.
• pipelex build structures: Corrected output file naming and resolved import path generation.
• ConceptFactory.make_from_blueprint: Now correctly handles native concepts.

Changed

• Jinja2 Integration Refactored to Protocol-Based Approach: Replaced the Jinja2Registry singleton and handler functions with a decoupled protocol-based system, eliminating circular dependencies between the template layer and core domain logic.
• StuffArtefact Redesigned as Delegation Adapter: Now a lightweight, immutable adapter that delegates attribute access directly to underlying Stuff and StuffContent objects, improving performance and providing more intuitive field access.
• Image Extraction Moved to Content Types: StructuredContent, ListContent, ImageContent, and TextAndImagesContent now implement the ImageRenderable protocol, replacing centralized handler logic.
• ⚠️ Breaking: User override config renamed from pipelex_super.toml to pipelex_override.toml.
• Image Generation Architecture: Refactored to taxonomy-based approach. Standardizes parameter translation (aspect_ratio, quality, output_format) to provider-specific APIs.
• Document Extraction Improvements: pypdfium2 extractor now extracts embedded images from PDFs. Response parsing uses dedicated Pydantic schemas for validation.
• Default Model Change: extract_text_from_visuals deck now defaults to azure-document-intelligence
• pipelex_inference replaced by pipelex_gateway: See Highlights for migration details. New PIPELEX_GATEWAY_API_KEY environment variable; default routing profiles updated to pipelex_gateway_first.

• Telemetry System Split: Now two separate streams: 1. Pipelex Gateway telemetry for service monitoring (never collects prompts/completions/business data) 2. Custom telemetry to user-configured backends 3. Config updated accordingly (telemetry.toml):

  • Renamed [posthog] to [custom_posthog] to distinguish user's PostHog from Pipelex Gateway telemetry
  • Added new [custom_portkey] section with force_debug_enabled and force_tracing_enabled settings

• Main Configuration Overrides Updated (.pipelex/pipelex.toml):

• pipelex_override.toml (final override) renamed from pipelex_super.toml to pipelex_override.toml and moved from repo root to .pipelex/ directory
• telemetry_override.toml (personal telemetry settings)

• is_generate_cost_report_file_enabled default changed from true to false

• Documentation:

• Clarified Setup (first run) vs Configuration (TOML reference), added a Setup overview page, and added contributor docs for configuration defaults/overrides.
• Added the "Under the Hood" page documenting the execution graph tracing system.
• pipelex init: Now creates a documented telemetry.toml template instead of prompting for preferences
• Model Catalog Updated: Latest models (gpt-5.1, claude-4.5-opus, gemini-3.0-pro, etc.) and updated waterfalls in base_deck.toml
• Model Constraints Refactored: From simple lists to structured valued_constraints dictionaries (e.g., valued_constraints = { fixed_temperature = 1 })
• OpenAI Responses API: New implementation now differentiates between openai_completions and openai_responses
• CLI Initialization: Commands refactored to use centralized Pipelex initialization factory for improved error handling
• pipelex doctor: Enhanced to detect outdated telemetry.toml formats and suggest fixes
• --output-dir Option: The runner, structures, and inputs CLI commands now accept this option
• Cost Report: Now displays a note clarifying that it only includes LLM costs
• description Field Now Required: In PipeAbstract, PipeBlueprint, and PipeSpec classes.
• Configuration: New [pipelex.pipeline_execution_config.graph_config] section in pipelex.toml for fine-grained control over graph generation, data embedding, and rendering options.
• CLI: All pipelex commands now accept --no-logo to suppress the Pipelex banner in the terminal — useful to reduce tokens.

Removed

• openai_utils Module: Removed pipelex.plugins.openai.openai_utils; logic now in centralized image preparation utilities.
• Pipeline Tracking feature: Removed entirely, including the pipelex/pipeline/track module, PipelineTracker components, related configuration, tracker calls in pipe controllers, and associated documentation.
• Flow Generator: The old flow generator has been removed.

Deprecated

• pipelex_inference backend in favor of pipelex_gateway (marked as "🛑 Legacy" in configuration template)

For Contributors

• Technical Documentation: Added a new "Under the Hood" page documenting the StuffArtefact delegation pattern and image rendering architecture.
• Enhanced Testing: Added extensive unit and integration tests for the protocol-based rendering system, including nested image extraction and filter error conditions.
• Agent Rules: Added pipelex_standards.md outlining standards for the Pipelex configuration system, also included as rules for AI development agents.
• Agent Rules: New target make agent-check for faster linting.
• ⚠️ Breaking — Content Handling Overhaul: GeneratedImage replaced by internal GeneratedImageRawDetails; ImageContent is now standard (without base_64 field). PipeExtract outputs PageContent list directly. Content persistence now handled automatically by storage system.
• ⚠️ Breaking — Image Prompt Representation: Redesigned PromptImage models—consolidated PromptImagePath and PromptImageUrl into PromptImageUri; now uses Pydantic discriminated union for URI, base64, and raw bytes sources.
• Centralized Image Preparation: Moved image fetching and base64 conversion logic to pipelex.cogt.image.prompt_image_utils, simplifying LLM provider plugins (Anthropic, Google, Mistral, OpenAI).
• Unified Resource Loading: Updated all file/URL reading components (PDF renderers, document extractors) to use the new URI handling system, replacing the legacy pipelex.tools.misc.path_utils module.
• Async HTTP Fetching: Renamed fetch_file_from_url_httpx_async to fetch_file_from_url_httpx; removed redundant synchronous version.
• PreparedImage Abstraction: New models (PreparedImageHttpUrl, PreparedImageBase64) representing images ready for LLM provider APIs.
• Pipelex Gateway Model Management: New CLI commands (pipelex-dev update-gateway-models, pipelex-dev check-gateway-models) and corresponding make targets (ugm, cgm) to generate and verify gateway model catalog. CI now validates this documentation is up-to-date.
• Test Suite Pre-flight Check: Verifies Gateway terms acceptance before running tests, providing clear error messages.
• Content Rendering: StuffContent.rendered_* methods now provide both synchronous and asynchronous variants.

Migration Notes

• Telemetry configuration migration: If you have an existing telemetry.toml, rename:
• [posthog] → [custom_posthog]
• [posthog.tracing] → [custom_posthog.tracing]
• [posthog.tracing.capture] → [custom_posthog.tracing.capture]
• Or run pipelex init telemetry --reset to regenerate the file with the new structure

Refactored

• Anthropic Backend: Internal streaming for standard completions to prevent SDK timeouts, configurable structured output timeout (structured_output_timeout_seconds), improved error mapping, and increased Bedrock Claude max_tokens (8K → 64K) with removal of max_output_tokens_limit constraint.
• ⚠️ Breaking — Pipe I/O Specification: The output (and inputs) of a pipe is now a StuffSpec object that holds the concept and the multiplicity.
• Naming Convention: Renamed domain to domain_code where relevant.
• Dry Run Methods: Refactored the dry run methods of the PipeAbstract class.

[v0.17.6] - 2026-02-14

Added

• Claude Code GitHub Actions: Added claude.yml workflow for interactive Claude Code assistance on issues and PR comments, and claude-code-review.yml workflow for automated code review on pull requests.

[v0.17.5] - 2026-01-16

• Added target make docs-deploy-404 to deploy the 404.html file to the gh-pages root for versionless URL redirects.

[v0.17.4] - 2026-01-16

Added

• Added the mike dependency to support mutiple docs versions. version plugin added to the MkDocs configuration, make targets and CI scripts.

[v0.17.3] - 2025-12-01

Fixed

• Fixed the issue with the find_files_in_dir force including virtual environment directories: Now it force includes the pipelex.builder directory.
• Fixed a bug with the comparison of Concept structures.

[v0.17.2] - 2025-12-01

Added

• New AI models support: Added GPT-5.1, Claude 4.5 Opus, and Gemini 3 Preview to the available models.
• Codex Cloud support: Added support for running Pipelex in Codex Cloud environments with appropriate configuration and testing capabilities.
• Enhanced file discovery: Added force_include_dirs parameter to the find_files_in_dir function. This allows specific directories to be force included in the search even when they are nested within excluded directories. For example, you can now exclude .venv while still including .venv/lib/python3.11/site-packages/pipelex for loading Pipelex libraries from installed packages.
• Added validation of the PipeLLM inputs at the blueprint level.
• Added a xfailed test for PipeCondition: if one of the outcome of the pipe is continue, it does nothing, but the main stuff still points towards the last step. Therefore when trying to get the main stuff out of the working memory as a specific type, it fails.

Changed

• Backend fallback now only activates when explicitly opted-in, giving users more control over model selection.
• Renamed and improved the Azure Image Generation SDK implementation.
• Enhanced language spec examples, operator details, and added Viewpoint documentation.

Fixed

• Fixed kit rules to be idempotent and work correctly across multiple executions.

[v0.17.1] - 2025-11-27

Fixed

• Fixed a bug in the find_files_in_dir function.

[v0.17.0] - 2025-11-27

Highlights

• Previously, in the pipelex config files (.toml files in the .pipelex/ directory, such as .pipelex/pipelex.toml, but also the routing profiles files, backends, etc.), when an array was overridden, the new array was concatenated to the old array. Now, the new array overrides the old array.

Fixed

• Relaxed concept structure field naming restrictions: Users can now use field names like content, stuff_code, stuff_name, and concept in their concept structures without conflicts. Internal metadata fields in stuff artefacts now use underscore prefixes (_stuff_name, _content_class, _concept_code, _stuff_code, _content) to avoid collisions with user-defined fields. Reserved field names (Pydantic BaseModel attributes like model_config, model_fields, etc.) and field names starting with underscore remain forbidden with improved error messages that clearly specify which fields are problematic.

Changed

• Modified the GHA version-check.yml so that the check of the version is only applying to release branches.
• Removed the pyproject.toml file from the build.
• No more implicit concepts. A concept reference has a domain and a code. If there is no domain, it should be a native concept, or it is declared in the same bundle.

Refactored

• The find_files_in_dir function was coded in 3 different places, now it's in pipelex/tools/misc/file_utils.py, and accepts excluded_dirs.
• Refactored the Pipe factories: Centralized everything in the PipeFactory class.

[v0.16.0] - 2025-11-25

Highlights

• Library manager now supports multiple libraries. You can now have multiple libraries in your project, each with its own set of concepts, pipes, and stuffs. You can run the same pipe at the same times as much as you want, with different inputs. Side effets: Unit tests now run in 30s.

Fixed

• Fixed some issues with inputs of pipes: The validation methods was not detecting misconceptions with implicit concepts.

Changed

• Improved pipe builder by auto-fixing errors, forcing consistency in the inputs and outputs of the pipes.

Refactor

• PipeCondition: Moved the expression/expression_template choosing to the factory.
• Moved a lot of validation to blueprints instead of pipe instances.
• Refactored the Blueprint validation errors, and validation functions.
• Refactored the PipelexInterpreter validation errors.
• Refactored the pipe builder validation loop.
• Reorganized the unit tests, and added new ones.
• Reorganized the config files.
• Refactored methods execute_pipeline and start_pipeline.
• Moved dev_cli to cli.dev_cli.

[v0.15.7] - 2025-11-18

Fixed

• Fixed issue with get_console() function returning None if Pipelex is not initialized. Now always defaults to stderr if not set.

[v0.15.6] - 2025-11-18

Contributors

• Welcome to our new contributor @0x090909 (yup, that's his github username) for his work on Groq support in PR #445! 🎉

Added

• Improved configuration repair: New --fix option for pipelex doctor command that interactively detects and repairs outdated or invalid backend configuration files using latest templates from the Pipelex kit.
• Developer CLI & tooling: New internal pipelex-dev CLI for project maintenance with check-config-sync command to verify user-facing configuration templates (.pipelex/) are synchronized with package's internal kit configs. Includes make check-config-sync command and CI check (lint-check.yml) to enforce synchronization.
• Enhanced test infrastructure: Integration tests now automatically parameterized to run against all supported backend routing profiles. Tests are intelligently skipped at collection time if a model is not supported by the active backend profile, with a summary of skipped tests provided at session end.
• New routing profiles: Added all_groq and all_pipelex_inference routing profiles.
• Vision support flag: Added is_vision_supported property to LLMWorkerAbstract class for explicit checks of model vision capabilities.
• New type of StuffContent: JSONContent to support an arbitrary JSON object as input or output of a pipe.
• Azure image generation: Support for image generation models via Azure OpenAI backend using gpt-image-1.

Changed

• Unified structured output: Complete overhaul of structured generation settings. Replaced global configuration setting with new structure_method parameter in backend .toml files (configurable at backend level in [defaults] or per individual model). Expanded StructureMethod enum to include dozens of modes supported by instructor, enabling fine-grained control over provider-specific features like OpenAI Structured Outputs or Anthropic Tools, and various JSON-based modes.
• Groq integration: Updated to use standard openai SDK, simplifying integration.
• Default configurations: All official backend providers now enabled by default after pipelex init. Default prompting style changed from ticks to xml.
• Code & test organization: Unit test suite reorganized from tests/unit/core and other directories into unified tests/unit/pipelex/ structure. Integration test fixtures modularized from conftest.py into separate files within tests/integration/pipelex/fixtures/.
• Console output settings: Added console_print_target and console_log_target settings in pipelex.toml for redirecting output to stdout or stderr, with CLI and logging refactored to use centralized console instance. This makes it easier to support MCP communication based on stdio.

Fixed

• Improved Pydantic validation error messages when loading backend configurations to clearly indicate the specific file and model containing the error.

Removed

• Perplexity backend: Default configuration for Perplexity AI backend (perplexity.toml) removed from kit (it was obsolete, it will come back).
• Groq plugin: Dedicated pipelex/plugins/groq plugin removed (now uses standard openai SDK).
• Global instructor config: Global is_openai_structured_output_enabled setting, replaced by per-model structure_method approach.

[v0.15.4] - 2025-11-12

Added

• Enhanced pipelex build Command: Now generates a self-contained directory (e.g., results/pipeline_01/) containing bundle.plx, inputs.json, run_{pipe_code}.py, bundle_view.html, and bundle_view.svg. New CLI options: --output-name (-o) for custom base name, --output-dir for custom directory, and --no-extras to generate only the .plx file.
• CLI Readiness Check: Verifies that a virtual environment is active for development installations.
• Model Deck Presets: Added llm_for_creativity and cheap_llm_for_creativity model waterfalls, plus [cogt.model_deck_config] section in pipelex.toml for configuring model fallback behavior.
• WIP: Groq Inference Backend Support: Integrated full support for the Groq API with configuration file (.pipelex/inference/backends/groq.toml), model specifications, costs, capabilities, new model aliases (base-groq, fast-groq, vision-groq), and routing profile (all_groq).

Changed

• CLI Output and Visualization: Overhauled command-line output with rich, table-based layouts for pipeline components. Final output of pipelex run is now pretty-printed and adapts to content type.
• Documentation: Updated "Get Started" and "Build Reliable AI Workflows" to reflect new directory-based build output and CLI options.
• Internal Code Refactoring: Reorganized exception hierarchy into dedicated exceptions.py files per module, centralized validation logic into validation.py modules, added ValueError to blueprints, and removed unused exceptions for improved maintainability.
• Updated pytest to >=9.0.1 to support their new pyproject.toml config format.

Fixed

• Adjusted default temperature for llm_for_testing_gen_object preset from 0.5 to 0.1 for more deterministic structured data generation.
• Corrected LLM_FOR_VISUAL_DESIGN skill in pipe_llm_spec to point to cheap_llm_for_creativity preset.
• Standardized input variable names in pipe_llm_vision.plx from imageA/imageB to image_a/image_b.

Removed

• Deleted pipelex/core/validation_errors.py file as part of exception hierarchy refactoring.

[v0.15.3] - 2025-11-07

Fixed

• Fixed weird import issues with posthog and StrEnum

[v0.15.2] - 2025-11-07

Fixed

• Fixed resetting routing profile when calling with --reset flag in pipelex init

[v0.15.1] - 2025-11-07

Fixed

• Bumped OpenAI dependency to >=1.108.1 to support their breaking change: "change optional parameter type from NotGiven to Omit"
• get_selected_backend_keys() now correctly considers backends enabled by default (like before v0.15.0)

[v0.15.0] - 2025-11-07

Highlights

This release dramatically simplifies onboarding with interactive CLI setup, comprehensive documentation relaunch, and intelligent model fallbacks, making Pipelex more accessible and resilient than ever.

Added

• Model Waterfalls: Define prioritized model lists in base_deck.toml (e.g., smart_llm = ["gpt-4o", "claude-4.5-sonnet", "grok-3"]). Pipelex automatically falls back to the next model if the preferred one is unavailable.
• Advanced Routing Profiles: New capabilities in routing_profiles.toml: fallback_order (Global fallback sequence specifying which backends to try if a model isn't found) and optional_routes (Routes that activate only when their target backend is enabled)
• New Models: Anthropic claude-4.5-haiku (Pipelex Inference, Anthropic, and Bedrock backends) and Azure OpenAI o3
• Comprehensive Documentation Relaunch: Complete restructure under /home/ with new "Get Started" guides for pipelex build and manual workflows, plus in-depth sections on Domains, Bundles, Concepts, and Pipe lifecycle.
• Enhanced CLI: pipelex init now interactively guides backend selection and automatically configures routing profiles, including primary backend and fallback order. Added pipelex init routing focus.
• Enhanced CLI: Improved error reporting across all commands (build, validate, run, show) with clear, actionable feedback for configuration errors, missing models, and invalid presets.
• Enhanced CLI: pipelex doctor now validates model deck configuration.
• New Routing Profiles: Full suite of all_* profiles (e.g., all_openai, all_anthropic, all_google) to route all requests to a single provider.

Changed

• BREAKING: for inline concept structures, the fields are now optional by default: the required property defaults to false. Explicitly set required = true to make fields mandatory, which we discourage as it increases risks of hallucinations.
• LLM Presets Overhaul: Rationalized and renamed default presets in base_deck.toml. Single-model aliases replaced with waterfall aliases. Key renames: llm_for_complex_reasoning → engineering-structured, llm_to_answer_hard_questions → llm_to_answer_questions, llm_to_write_questions → llm_for_writing_cheap. Removed redundant older presets.
• Stricter Configuration Validation: Pipelex validates model deck on startup and raises errors if presets reference unavailable models.

Fixed

• Local OpenAI-Compatible Endpoints: OpenAI plugin now handles empty API keys, enabling seamless integration with local servers like Ollama.

Removed

• Old Documentation Structure: Previous /pages/ directory documentation removed in favor of new structure.

[v0.14.3] - 2025-10-29

Added

• Image generation models via BlackBoxAI backend: flux-pro, flux-pro/v1.1, flux-pro/v1.1-ultra (Black Forest Labs), fast-lightning-sdxl (ByteDance), and nano-banana (Google). Implemented using new openai_alt_img_gen SDK worker with chat completion-style API.
• Language model: claude-4.5-sonnet (Anthropic) via BlackBoxAI backend.
• Routing profile: all_blackboxai profile routes all supported model requests to BlackBoxAI backend.

Changed

• Model aliases in base_deck.toml: base-img-gen → flux-pro/v1.1-ultra, best-img-gen → nano-banana, llm_for_large_codebase now includes claude-4.5-sonnet.
• Configuration file: BLACKBOX_RULES.md renamed to .blackboxrules.

Fixed

• Image generation schema: ImgGenJobParams.seed field now explicitly defined with default=None.
• CLI bundle validation: pipelex validate command now accepts bundle path (.plx file) which are in the package and already loaded and performs dry run on all pipes in the bundle.

[v0.14.2] - 2025-10-29

Chaged

• Improved pipe builder.

Added

• CLI to generate inputs JSON.

[v0.14.1] - 2025-10-27

Added

• Tutorial GIF on the README.md file.

[v0.14.0] - 2025-10-27

Added

• pipelex doctor command: Diagnoses and fixes common configuration issues including missing files, invalid telemetry settings, and unset environment variables for enabled backends.
• Interactive backend selection in pipelex init: Multi-select menu for enabling/disabling inference backends (OpenAI, Anthropic, Amazon Bedrock, etc.).
• JSON input support: pipelex run --inputs flag accepts a JSON file path for passing structured data to pipelines.
• pretty_print methods: Added to PipeSpec, ConceptSpec, and Stuff objects for readable debugging output.
• VS Code debug configuration: "Debug run pipe" launch configuration for debugging pipeline executions.
• display_name attribute: Added to all inference backends in backends.toml for better UI presentation.
• Documentation headers: All default .toml configuration files now include headers with links to documentation and support channels.

Changed

• pipelex init redesign: Transformed into a unified, interactive setup wizard with rich terminal UI for configuration files, backend selection, and telemetry preferences. Telemetry is now configured here instead of via first-run prompt.
• README.md rewrite: Complete overhaul featuring a simplified 5-step quick-start guide highlighting the pipelex build command.
• Documentation updates: "Quick Start" guide renamed to "Writing Workflows" with simplified content. Python examples updated to use JSON input method, removing manual Stuff and WorkingMemory object creation boilerplate. Developer guides and AI assistant rules now recommend pipelex validate over make validate. Added instructions emphasizing .venv activation before running commands.
• Error handling improvements: Pipelines now validate required inputs upfront and fail early with PipeRunInputsError. pipelex run prints full rich-formatted exception tracebacks on error.
• Default enabled backends: Amazon Bedrock, Google AI, and Google Vertex AI are now enabled by default.
• Naming consistency: "AWS Bedrock" renamed to "Amazon Bedrock" throughout codebase, configuration, and documentation.

Fixed

• Some documentation links were broken.

[v0.13.2] - 2025-10-25

Added

• Added the n8n documentation page for the n8n-nodes-pipelex package.
• Added optional telemetry system with first-run interactive prompt offering three modes: off (no data collected), anonymous (usage data without identification), and identified (usage data with user identification). Automatically respects DO_NOT_TRACK environment variable and redacts sensitive data (prompts, responses, file paths, URLs). Configuration stored in .pipelex/telemetry.toml.
• Added telemetry documentation: user-friendly setup guide and comprehensive configuration reference.

Changed

• Updated the PipelexClient and changed the route of the API calls to v1/pipeline/execute and v1/pipeline/start.
• Changed the parameter input_memory to inputs in the documentaton.

[v0.13.1] - 2025-10-22

Changed

• Changed the pydanticdependency from ==2.10.6 to >=2.10.6,<3.0.0 to avoid compatibility issues.

[v0.13.0] - 2025-10-21

Highlights

This release focuses on making Pipelex more accessible and easier to use, with major improvements to the CLI, simplified syntax for multiplicity, and a complete documentation overhaul:

• New CLI commands: Run pipelines directly with pipelex run, generate Python runners with pipelex build runner, and inspect your AI backend configuration with pipelex show backends
• Simplified pipeline inputs: The new inputs parameter replaces input_memory and accepts strings, lists, or content objects directly - no more complex dictionary structures
• Getting started faster: Completely rewritten quick-start guide and new documentation sections help you go from installation to your first pipeline in minutes

Added

• CLI command pipelex run: Top-level command to execute pipelines directly from the CLI. Can run pipes from the package or from any .plx bundle file, with options to provide inputs from a JSON file and save the output
• CLI command pipelex build runner: Generates Python script with imports and example input structures for any pipe
• CLI command pipelex show backends: Displays configured AI providers, their status, and active routing rules
• Model presets: Added task-oriented presets including llm_to_write_questions, llm_to_code, llm_for_basic_vision, llm_for_visual_analysis
• Documentation: Complete quick-start guide rewrite, new guides for "Understanding Multiplicity", "API Guide", "Executing Pipelines with Inputs", and updated README with video demo
• Migration guide: Updated guide at pipelex/kit/migrations/migrate_0.11.0_0.12.x.md

Changed

• Unified bracket notation for multiplicity: Single items use "Concept", variable lists use "Concept[]", fixed-count lists use "Concept[3]". Applies to both inputs and output fields in .plx files
• Pipeline input format: input_memory parameter renamed to inputs; now accepts strings, lists of strings, StuffContent objects, or explicit concept dictionaries instead of CompactMemory
• Bundle main_pipe attribute: Pipelex bundles (.plx files) now support a main_pipe attribute to designate the primary entry point of the bundle. Used by pipelex run and pipelex build runner commands to simplify execution
• Model preset names: llm_to_reason → llm_for_complex_reasoning, base_ocr_mistral → extract_text_from_visuals, base_extract_pypdfium2 → extract_text_from_pdf, base_img_gen → gen_image_basic, fast_img_gen → gen_image_fast, high_quality_img_gen → gen_image_high_quality
• Unified model parameter: PipeExtract and PipeImgGen now use model parameter for consistency across all operator pipes
• PipeExtract operator: Output is now consistently validated to be the Page concept, simplifying its usage for document processing
• CLI improvements: pipelex run and pipelex validate now auto-detect pipe code vs .plx bundle files; pipelex validate promoted to top-level command with improved error reporting and syntax-highlighted code snippets
• CLI reorganization: Main command-line interface restructured for better usability with improved help texts and more logical command order
• Python API: Pipelex.make() now accepts dependency injection arguments directly
• Python coding standards: Updated internal coding standards to recommend declaring variables with a type but no default value to better leverage linters for bug detection
• Default configuration: Azure and AWS inference backends now disabled by default in template configuration

Fixed

• Structure generation: Special characters (double quotes, backslashes) in concept field descriptions or default values no longer produce invalid Python code

Removed

• Legacy multiplicity syntax: nb_output, multiple_output parameters, and complex input dictionary syntax with multiplicity field
• Pipe-specific model parameters: ocr parameter from PipeExtract and img_gen parameter from PipeImgGen
• prompt_template_to_structure and system_prompt_to_structure configurations at the pipe and domain level
• Project Name discovery from Configuration
• Temporary design document for the new inference backend system (feature now fully implemented and documented)

[v0.12.0] - 2025-10-15

Highlights

Moving fast and breaking things:

• Added the new builder pipeline system for auto-generating Pipelex bundles from user briefs
• it's a pipeline to generate pipelines, and it works!
• the pipeline definitions are in pipelex_libraries/pipelines/base_library/builder/
• removed the previous draft which was named meta_pipeline.plx

Breaking changes... for good!

We tried to group all the renamings we wanted to do which impact our language, so that you get one migration to apply and then we will be way more stable in the future releases.

This is all in the spirit of making Pipelex a declarative language, where you express what you want to do, and the system will figure out how to do it. So our focus inwas to make the Pipelex language easier to understand and use for non-technical users, and at the same time use more consistent and obvious words that developers are used to.

💡 Pro tip: To make migration easier, pass the migration guide to your favorite SWE agent (Cursor, Claude Code, github copilot, etc.) and let it handle the bulk of the changes!

• Removed centralized pipelex_libraries folder system
• Pipelines are now auto-discovered from anywhere in your project—no special directory required
• No config path parameters needed in Pipelex.make() or CLI commands (just call Pipelex.make())
• Custom functions require @pipe_func() decorator for auto-discovery
• Structure classes auto-discovered (must inherit from StructuredContent)
• Configuration stays at repository root in .pipelex/ directory

• See migration guide for details on reorganizing your project structure

• General changes

• renamed definition fields to description across all cases

• Renamed PipeJinja2 to PipeCompose

• the fact that our templating engine is Jinja2 is a technnical detail, not fundamental to the language, especially since we included a pre-processor enabling insertion of variables in prompts using @variable or $variable, in addition to the jinja2 syntax {{ variable }}
• renamed jinja2 field to template for the same reason

• for more control, instead of providing a string for the template field, you can also use a nested template section with template, category and templating_style fields

• Renamed PipeOCR to PipeExtract

• this is to account for various text extraction techniques from images and docs, including but not only OCR; e.g. we now have integrated the pypdfium2 package which can extract text and images from PDF, when it's actually real text (not an image), and soon we'll add support for other document extraction models solutions
• removed obligation to name your document input ocr_input, it can now be named whatever you want as long as it's a single input and it's either an Image or a PDF or some concept refining PDF or Image
• renamed ocr_page_contents_from_pdf to extract_page_contents_from_pdf
• renamed ocr_page_contents_and_views_from_pdf to extract_page_contents_and_views_from_pdf
• introduced model settings and presets for extract models like we had for LLMs

• renamed ocr_model to model for choice of model, preset, or explicit setting and introduced base_ocr_mistral as an alias to mistral-ocr

• PipeLLM field renames

• image inputs must now be tagged in the prompt like all other inputs; you can just drop their names at the beginning or end of the prompt, or you can reference them in meaningful sentences to guide the Visual LLM, e.g. "Analyze the colors in $some_photo and the shapes in $some_painting."
• renamed prompt_template field to prompt
• renamed llm field to model

• renamed llm_to_structure field to model_to_structure

• PipeImgGen field renames

• renamed img_gen field to model for choice of model, preset, or explicit setting
• removed some technical settings such as nb_steps from the pipe attributes, instead you can set these as model settings or model presets

• introduced model settings and presets for image generation models like we had for LLMs

• PipeCondition field renames

• renamed pipe_map to outcomes

• renamed default_pipe_code to default_outcome and it's now a required field, because we need to know what to do if the expression doesn't match any key in the outcomes map; if you don't know what to do in that case, then it's a failure and you can use the fail value

• Configuration file changes (.pipelex/ directory)

• Renamed parameter llm_handle to model across all LLM presets in deck files
• Renamed parameter img_gen_handle to model across all image generation presets in deck files
• Renamed parameter ocr_handle to model in extraction presets
• Renamed ocr section to extract throughout configuration files
• Renamed ocr_config to extract_config in pipelex.toml
• Renamed base_ocr_pypdfium2 to base_extract_pypdfium2
• Renamed is_auto_setup_preset_ocr to is_auto_setup_preset_extract
• Renamed nb_ocr_pages to nb_extract_pages
• Updated pytest marker from 'ocr' to 'extract'

Added

• Added cheap-gpt model alias for gpt-4o-mini
• Added cheap_llm_for_vision preset using gemini-2.5-flash-lite
• Added llm_for_testing_vision and llm_for_testing_vision_structured presets for vision testing
• Added is_dump_text_prompts_enabled and is_dump_response_text_enabled configuration flags to have the console display everything that goes in and out of the LLMs
• Added generic_templates section in llm_config with structure extraction prompts
• Added useful error messages with migration configuration maps pin-pointing the fields to rename for config and plx files
• Added improved error message for PipeFunc when function not found in registry, mentioning @pipe_func() decorator requirement since v0.12.0
• Added pytest filterwarnings to ignore deprecated class-based config warnings
• Added Flow class that represents the flow of pipe signatures
• Added pipe-builder command flow to generate flow view from pipeline brief
• Added FlowFactory class to create Flow from PipelexBundleSpec or PLX files
• Added sort_pipes_by_dependencies() function for topological sorting of pipes
• Added pipe_sorter.py module for pipe dependency sorting utilities
• Added search_for_nested_image_fields_in_structure_class() method to Concept class
• Added image_field_search.py module with utilities to search for image fields in structure classes
• Added pipe_dependencies property to PipeBlueprint and controller blueprints
• Added ordered_pipe_dependencies property to PipeBlueprint for ordered dependencies
• Added get_native_concept() function to hub
• Added get_pipes() function to hub
• Added remove_concepts_by_codes() method to ConceptLibraryAbstract
• Added remove_pipes_by_codes() method to PipeLibraryAbstract
• Added template preprocessing with preprocess_template() function
• Added better dependency checking for optional SDK packages (anthropic, mistralai, boto3, aioboto3)
• Added MissingDependencyError exception for missing optional dependencies
• Added library_utils.py module with utility functions for PLX file discovery using importlib.resources
• Added class_utils.py module with are_classes_equivalent() and has_compatible_field() functions
• Added comprehensive unit tests for CostRegistry, WorkingMemory, and ModuleInspector
• Added ScanConfig class with configurable excluded directories for library scanning
• Added CSV export capabilities to CostRegistry with save_to_csv() and to_records() methods
• Added default configuration template in pipelex/kit/configs/pipelex.toml

Changed

• Replaced package toml by tomli which is more modern and faster
• Updated Gemini 2.0 model from gemini-2.0-flash-exp to gemini-2.0-flash with new pricing (input: $0.10, output: $0.40 per million tokens)
• Updated Gemini 2.5 Series comment from '(when available)' to stable release
• Updated best-claude from claude-4-sonnet to claude-4.5-sonnet across all presets
• Updated kajson dependency from version 0.3.0 to 0.3.1
• Updated httpx dependency to >=0.23.0,<1.0.0 for broader compatibility
• Cleanup env example and better explain how to set up keys in README and docs
• Changed Gemini routing from google backend to pipelex_inference backend
• BREAKING: Major module reorganization - moved tools/config/, tools/exceptions.py, tools/environment.py, tools/runtime_manager.py to system/ package structure (system/configuration/, system/exceptions.py, system/environment.py, system/runtime.py)
• BREAKING: Reorganized registry modules from tools/ to system/registries/ (affects class_registry_utils, func_registry, func_registry_utils, registry_models)
• BREAKING: Split pipelex.core.stuffs.stuff_content module into individual files per content type (affects imports: StructuredContent, TextContent, ImageContent, ListContent, PDFContent, PageContent, NumberContent, HtmlContent, MermaidContent, TextAndImagesContent)
• BREAKING: Renamed package pipelex.pipe_works to pipelex.pipe_run and moved PipeRunParams classes into it
• BREAKING: Cost reporting changed from Excel (xlsx) to CSV format using native Python csv module instead of pandas
• Renamed ConfigManager to ConfigLoader
• Renamed PipelexRegistryModels to CoreRegistryModels
• Renamed PipelexTestModels to TestRegistryModels
• Renamed generate_jinja2_context() to generate_context() in WorkingMemory and ContextProviderAbstract
• Renamed ConceptProviderAbstract to ConceptLibraryAbstract
• Renamed DomainProviderAbstract to DomainLibraryAbstract
• Renamed PipeProviderAbstract to PipeLibraryAbstract
• Renamed PipeInputSpec to InputRequirements
• Renamed PipeInputSpecFactory to InputRequirementsFactory
• Renamed pipe_input.py to input_requirements.py
• Renamed pipe_input_factory.py to input_requirements_factory.py
• Renamed pipe_input_blueprint.py to input_requirement_blueprint.py
• Changed hub methods from get_*_provider() to get_*_library() pattern
• Changed hub methods from set_*_provider() to set_*_library() pattern
• Changed PipeLLM validation to check all inputs are in required variables
• Updated LLMPromptSpec to handle image collections (lists/tuples) in addition to single images
• Changed Mermaid diagram URL generation from /img/ to /svg/ endpoint
• Changed PipeLLMPromptTemplate.make_llm_prompt() to private method _make_llm_prompt()
• Updated pipe-builder prompts to include concept specs for better context
• Updated PipelexBundleSpec.to_blueprint() to sort pipes by dependencies before creating bundle
• Changed exception base class from PipelexError to PipelexError throughout codebase
• Updated Makefile pyright target to use --pythonpath flag correctly
• Enhanced LibraryManager to use importlib.resources for reliable PLX file discovery across all installation modes (wheel, source, relative path)
• Simplified FuncRegistryUtils to exclusively register functions with @pipe_func decorator (removed decorator_names and require_decorator parameters)
• Updated ReportingManager to get config directly instead of via constructor parameter
• Updated PipeFunc documentation to reflect @pipe_func() decorator requirement and auto-discovery from anywhere in project
• Added warnings about module-level code execution during auto-discovery to PipeFunc and StructuredContent documentation

Fixed

• Fixed Makefile target pyright to use correct pythonpath flag
• Fixed bug with inputs of the PipeLLM where image inputs couldn't be used and tagged in prompts
• Fixed image input handling in LLMPromptSpec to support both single images and image collections
• Fixed template preprocessing to handle jinja2 templates correctly
• Fixed hard dependencies by moving imports to function scope in model_lists.py
• Updated README badge URL to point to main branch instead of feature/pipe-builder branch

Removed

• Removed centralized pipelex_libraries folder system and pipelex init libraries command
• Removed config path parameters from Pipelex.make() (relative_config_folder_path, config_folder_path, from_file)
• Removed Gemini 1.5 series models: gemini-1.5-pro, gemini-1.5-flash, and gemini-1.5-flash-8b
• Removed base_templates.toml file (generic prompts moved to pipelex.toml)
• Removed gpt-5-mini from possible models in pipe-builder
• Removed useless functions in LLMJobFactory: make_llm_job_from_prompt_factory(), make_llm_job_from_prompt_template(), make_llm_job_from_prompt_contents()
• Removed add_or_update_pipe() method from PipeLibrary
• Removed get_optional_library_manager() method from PipelexHub
• Removed get_optional_domain_provider() and get_optional_concept_provider() methods from hub
• Removed unused test fixtures (apple, cherry, blueberry, concept_provider, pretty) from conftest.py
• Removed some Vision/Image description pipes from the base library, because we doubt they were useful as they were
• Removed pandas and openpyxl dependencies (including stubs: pandas-stubs, types-openpyxl)
• Removed Excel file generation for cost reports and to_dataframe() method from CostRegistry
• Removed should_warn_if_already_registered parameter from func_registry.register_function()
• Removed decorator_names and require_decorator parameters from FuncRegistryUtils methods
• Removed _find_plx_files_in_dir() and _get_pipelex_plx_files_from_dirs() methods from LibraryManager (refactored to library_utils module)
• Removed hardcoded excluded directories from ClassRegistryUtils and FuncRegistryUtils (now use ScanConfig)
• Removed are_classes_equivalent() and has_compatible_field() methods from ClassRegistryUtils (moved to class_utils module)

[v0.11.0] - 2025-10-01

Highlights

• New pipe builder pipeline to generate Pipes based on a brief in natural language: use the cli pipelex build pipe "Your task" to build the pipe.
• New observer system: inject your own class to observe and trace all details before and after each pipe run. We also provide a local observer that dumps the payloads to local JSONL files = new-line delilmited json, i.e. one json object per line.
• Full refactoring of OCR and Image Generation to use the same patterns as LLM workers and pipes.

Added

• Added claude-4.5-sonnet to the model deck.
• Added a badge on the README.md to display the number of tests.
• Added new test cases for environment variable functions
• Added new documentation for PipeFunc on how to register functions.
• Added pipelex show models [BACKEND_NAME] command to list available models from a specific backend.

Changed

• Renamed llm_deck terminology to model_deck throughout codebase and documentation, now that it's also used for OCR and Image Generation models
• Renamed is_gha_testing property to is_ci_testing in RuntimeManager
• Refactored all_env_vars_are_set() function to only accept a list of keys, single string support now uses is_env_var_set()
• Modified any_env_var_is_placeholder() to use new placeholder detection logic
• Updated test environment setup to use dynamic placeholder generation instead of hardcoded values

Fixed

• Fixed logic error in any_env_var_is_placeholder() function - now correctly returns False when no placeholders are found

Removed

• Removed get_rooted_path() and get_env_rooted_path() utility functions which were not used
• Removed hardcoded placeholder dictionary and ENV_DUMMY_PLACEHOLDER_VALUE constant in test setup
• Removed function run_pipe_code in pipe router because it was not relevant (used mostly in tests)
• Remove the use of PipeCompose in PipeCondition, to only use jinja2 directly, through the ContentGenerator
• Remove the template libraries from the pipelex libraries.
• Removed claude-3.5-sonnet and claude-3.5-sonnet-v2 from the model deck.

[v0.10.2] - 2025-09-18

Added

• Unified OCR system using model handles instead of separate OcrHandle enum
• ModelType enum supporting LLM and TEXT_EXTRACTOR types
• Enhanced error handling in library loading with better validation messages
• Config template management with config-template and cft Makefile targets to update templates from the .pipelex/ directory

Changed

• ⚠️ Breaking changes:
• Renamed ocr_handle to ocr_model in PipeExtract blueprint, so you'll need to update your PLX code accordingly
• Updated .env.example file with slightly modified key names (more standard).
• OCR system now uses InferenceModelSpec with unified model handles
• Renamed get_llm_deck() to get_model_deck() and updated parameter names from llm_handle to model_handle
• Simplified OCR worker factory using plugin SDK matching
• Enhanced plugin system compatibility with InferenceModelSpec
• Improved error messages throughout system
• Improved management of placeholder environment variables for unit tests

Removed

• Legacy OCR classes: OcrHandle, OcrPlatform, OcrEngine, OcrEngineFactory
• Obsolete configuration fields and setup methods
• PipelexFileError exception class

[v0.10.1] - 2025-09-17

Changed

• Enabled all backends, still required to pass all unit tests.
• A few tweaks to the base model deck.

[v0.10.0] - 2025-09-17

Highlights

New Inference Backend Configuration System — We've completely redesigned how LLMs are configured and accessed in Pipelex, making it more flexible and easier to get started:

• Get started in seconds with Pipelex Inference: Use a single API key to access all major LLM providers (OpenAI, Anthropic, Google, Mistral, and more)
• Flexible backend configuration: Configure multiple inference backends (Azure OpenAI, Amazon Bedrock, Vertex AI, etc.) through simple TOML files in .pipelex/inference/
• Smart model routing: Automatically route models to the right backend using routing profiles with pattern matching
• User-friendly aliases: Define shortcuts like best-claude → claude-4.1-opus with optional fallback chains
• Cost-aware model specs: Each model includes detailed pricing, capabilities, and constraints for better cost management

For complete details, see the Inference Backend Configuration documentation.

Added

• New inference backend configuration system in .pipelex/inference/ directory
• Support for 10+ inference backends: OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Mistral, Vertex AI, XAI, BlackboxAI, Perplexity, Ollama, and Pipelex Inference
• Model routing profiles with pattern matching (*model*, model*, *model)
• Model aliases with waterfall fallback chains
• Environment variable and secret substitution in TOML configs (${VAR} and ${secret:KEY})
• Comprehensive model specifications with detailed cost categories
• Unified plugin SDK registry for all backends
• CI environment detection with automatic placeholder API keys for testing
• Improved pipelex init config command to copy entire configuration template directory structure to .pipelex/ with smart file handling (skips existing files, shows clear progress messages)
• Added FuncRegistryUtils to register functions in a pipelex folder that have a specific signature.
• Added mistral-medium and mistral-medium-2508 to the Mistral backend configuration.
• Added gemini-2.5-flash to the VertexAI backend configuration.

Changed

• LLM configuration moved from pipelex_libraries/llm_deck/ to .pipelex/inference/deck/
• LLM handles simplified to direct model names or user-defined aliases
• Model deck completely redesigned with inference models, aliases, and presets
• Plugin system refactored to use backend-specific TOML configuration
• Token categories renamed to cost categories with expanded types

Fixed

• Improved error messages for missing environment variables
• Enhanced TOML configuration validation
• More robust model routing and backend selection

Removed

• Legacy LLM model library system (llm_integrations/ directory)
• Platform-specific configuration classes (AnthropicConfig, OpenAIConfig, etc.)
• Deprecated LLM engine blueprint and factory classes
• Old LLM platform and family enumerations

Security

• Enhanced secret management with secure fallback patterns
• Improved API key handling through centralized backend configuration

[v0.9.5] - 2025-09-12

Highlights

• Pinned instructor to version <1.10.0 to avoid errors with mypy

Added

• Added PIPELEX_INFERENCE LLM family enum value
• Added support for PIPELEX_INFERENCE in OpenAI LLM worker
• Added Azure OpenAI platform support for Grok models (grok-3 and grok-3-mini)
• Added debug logging for PipeParallel output contents
• Added TOML file filtering in LLM model library loading
• Added error handling for Unicode decode errors in LLM model library
• Added new test model configurations for pipelex and vertex_ai platforms

Changed

• Improved error messages in StuffFactory to include concept code and stuff name
• Disabled is_gen_object_supported for all Grok models (grok-3, grok-3-mini, grok-3-fast)
• Updated test configurations to use different LLM models and platforms
• Modified Jinja2 filter to use default TagStyle.TICKS instead of raising error
• Added proper error handling for Unicode decode errors when loading model libraries
• Improved error handling in Anthropic plugin tests with specific AuthenticationError handling
• Image handling in AnthropicFactory now converts image URLs to base64 data URLs with proper MIME type prefix
• Put back Discord link in README.md

Fixed

• Pinned instructor to version <1.10.0 to avoid errors with mypy

[v0.9.4] - 2025-09-06

Added

• Added support for BlackboxAI models

[v0.9.3] - 2025-09-06

Added

• Better support for BlackboxAI IDE
• VS Code extensions recommendations file with Pipelex, Ruff, and MyPy extensions
• File association for .plx files in VS Code settings

[v0.9.2] - 2025-09-05

Fixed

• Fix the rules of all agents.

Added

• Added agent rule for copilot
• Added a rule to forbidden structuring basic text concepts

[v0.9.1] - 2025-09-05

Fixed

• Fixed many inconsistencies in the documentation.

[v0.9.0] - 2025-09-02

Refacto

• Changed the pipeline file extension from .toml to .plx: Updated the LibraryManager in consequence.

Fixed

• Fixed the structuring_method behavior in the PipeLLM pipe: Putting it to preliminary_text, the PipeLLM will always generate text before generating the structure -> Reliability increased by a lot.

Fixed

• Fixed a bug in the needed_inputs method of the PipeSequence pipe.

Changed

• dry_run_pipe now returns a DryRunOutput object instead of a str with additional information.
• Updated cocode dependency from version v0.0.10 to v0.0.15.

Added

• Added the FuncRegistryUtils class to register functions in the library.

[v0.8.1] - 2025-08-27

Bugfix

• Bugfix: Fixed the PipeFunc output concept code and structure class name in the dry run.

[v0.8.0] - 2025-08-27

Refactor

• Refactored the concepts: Blueprints are now more explicit, and hold only concept strings or code. Pipes hold concept instances.
• Organized code: Created subfolders for controller and operator pipes.
• Say goodbye to PipeLLMPrompt.
• Removed the PipeCompose and PipeLLMPrompt from the PipeLLM.

Added

• Added a lot of unit tests.
• Loading the library can now be done from toml file or from PipelexBundleBlueprint.

Fixed

• Backported backports.strenum to >=1.3.0 to support Python 3.10 now in dependencies and not in optional dependencies.

[v0.7.0] - 2025-08-20

Refactor

• Refactored the Blueprints. Introduces the PipelexInterpreter that interprets the Pipelex language and creates the Pipelex Blueprints (and vice versa)
• Modified the way we declare pipes. Use the field type = "PipeLLM" instead of field PipeLLM. (Same for all pipes)
• Refactored the LibraryManager.
• Refactored CLI commands and added new ones. Modified CLI command structure:
• pipelex init - Initialization commands
  • pipelex init libraries [DIRECTORY] - Initialize pipelex libraries (creates pipelex_libraries folder)
  • pipelex init config - Initialize pipelex configuration (creates pipelex.toml)
• pipelex validate - Validation and dry-run commands
  • pipelex validate all -c pipelex/libraries - Validate all libraries and dry-run all pipes
  • pipelex validate pipe PIPE_CODE - Dry run a single pipe by its code
• pipelex show - Show and list commands
  • pipelex show config - Show the pipelex configuration
  • pipelex show pipes - List all available pipes with descriptions
  • pipelex show pipe PIPE_CODE - Show a single pipe definition
• pipelex migrate - Migration commands
  • pipelex migrate run - Migrate TOML files to new syntax (with --dry-run and --backups options)
• pipelex build - Build artifacts like pipeline blueprints
  • pipelex build draft PIPELINE_NAME - Generate a draft pipeline
  • pipelex build blueprint PIPELINE_NAME - Generate a pipeline blueprint
• Organized concept, pipe, working_memory, stuff files into folders.

Changed

• Allow aiofiles version >=23.2.1
• GHA Cla assistant fixed with Github App

Added

• New LLM families LLMFamily.GPT_5, LLMFamily.GPT_5_CHAT and LLMFamily.CLAUDE_4_1
• Added support for Claude 4.1 and GPT 5 models (inc. mini, nano, chat)
• New Pipe that generates pipe. Pipe code: build_blueprint
• New tests. Especially for the PipelexInterpreter.
• Migration files and cli commands to migrate Pipelex language to new syntax.
• Introduces PipelexBundle, which correspond to the python paradigm of the Pipelex TOML syntax.

[v0.6.10] - 2025-08-02

Added

• New test file for source code manipulation functions (tests/cases/source_code.py)
• New integration test for PipeFunc functionality (tests/integration/pipelex/pipes/pipe_operator/pipe_func/test_pipe_func.py)
• New package structure file for pipe_func tests (init.py)
• Simplified input memory creation for native concepts (Text, Image, PDF) in pipeline execution
• Added Pipeline requests link to GitHub issue template config

Changed

• Updated pipeline execution documentation and examples to use input_memory instead of working_memory
• Renamed pipeline from 'extract_page_contents_from_pdf' to 'ocr_page_contents_from_pdf'
• Renamed pipeline from 'extract_page_contents_and_views_from_pdf' to 'ocr_page_contents_and_views_from_pdf'
• Updated cocode dependency from version 0.0.6 to 0.0.9

Fixed

• Fixed typo in pipeline description ('aspage views' to 'as full page views')

Removed

• Removed WorkingMemoryFactory and StuffFactory imports from pipeline execution examples
• Removed working memory creation code from pipeline examples

[v0.6.9] - 2025-07-26

Changed

Simplified input memory:

• The concept code can now be provided with arg named concept in addition to concept_code
• You can pass a simple string to create a Text stuff

[v0.6.8] - 2025-07-25

Added

• New method make_stuff_using_concept_name_and_search_domains in StuffFactory for creating stuff using concept names and search domains.
• New method make_stuff_from_stuff_content_using_search_domains in StuffFactory for creating stuff from stuff content using search domains.
• New method make_from_implicit_memory in WorkingMemoryFactory for creating working memory from implicit memory.
• New method create_mock_content in WorkingMemoryFactory for creating mock content for requirements.

Changed

• Refactored PipeInput to use InputRequirement and TypedNamedInputRequirement classes instead of plain strings for input specifications.
• Updated WorkingMemoryFactory to handle PipelineInputs instead of CompactMemory.
• Replaced ExecutePipelineException with PipelineInputError in execute_pipeline function.
• Updated PipeBatch, PipeCondition, PipeParallel, PipeSequence, PipeFunc, PipeImgGen, PipeCompose, PipeLLM, and PipeExtract classes to use InputRequirement for input handling.
• Updated PipeInput creation in various test files to use make_from_dict method.
• Updated pyproject.toml to exclude pypdfium2 version 4.30.1.
• Updated Jinja2TemplateCategory to handle HTML and Markdown templates differently.

Fixed

• Corrected error messages in StuffFactory and StuffContentFactory to provide more detailed information about exceptions.

[v0.6.7] - 2025-07-24

Removed

• Removed the structure_classes parameter from the Pipelex class.

[v0.6.6] - 2025-07-24

Added

• Added a new method verify_content_type in the Stuff class to verify and convert content to the expected type.
• Added cocode==0.0.6 to the development dependencies in pyproject.toml.

Changed

• Updated Stuff class methods to use the new verify_content_type method for content verification.
• Updated vertexai.toml to change LLM IDs from preview models to released models: gemini-2.5-pro and gemini-2.5-flash.

Removed

• Removed reinitlibraries, rl, v, and init targets from the Makefile.

[v0.6.5] - 2025-07-21

Fixed

• In the documentation, fixed the use of execute_pipeline.

[v0.6.4] - 2025-07-19

• Fixed the README.md link to the documentation

[v0.6.3] - 2025-07-18

Changed

• Enhanced Stuff.content_as() method with improved type validation logic - now attempts model validation when isinstance check fails

[v0.6.2] - 2025-07-18

Added

• New dry-run-pipe cli command to dry run a single pipe by its code
• New show-pipe cli command to display pipe definitions from the pipe library
• New dry_run_single_pipe() function for running individual pipe dry runs

Changed

• Updated init-libraries command to accept a directory argument and create pipelex_libraries folder in specified location
• Updated validate command to use -c flag for the config folder path

[v0.6.1] - 2025-07-16

• Can execute pipelines with input_memory: It is a CompactMemory: Dict[str, Dict[str, Any]]

[v0.6.0] - 2025-07-15

Changed

• Enhanced Pipelex.make() method: Complete overhaul of the initialization method with new path configuration options and robust validation:
• Added relative_config_folder_path and absolute_config_folder_path parameters for flexible config folder specification
• The from_file parameter controls path resolution: if True (default), relative paths are resolved relative to the caller's file location; if False, relative to the current working directory (useful for CLI scenarios)
• Renamed Makefile targets like make doc to make docs for consistency

Added

• Added github action for inference tests
• load_json_list_from_path function in pipelex.tools.misc.file_utils: Loads a JSON file and ensures it contains a list.
• Added issue templates
• Updated Azure/OpenAI integrations, using dated deployment names systematically

[v0.5.2] - 2025-07-11

• log a warning when dry running a PipeFunc
• Update Readme.md

[v0.5.1] - 2025-07-09

Fixed

• Fixed the ConceptFactory.make_from_blueprint method: Concepts defined in single-line format no longer automatically refine TextContent when a structure class with the same name exists
• ConceptFactory.make_concept_from_definition is now ConceptFactory.make_concept_from_definition_str

Added

• Bumped kajson to v0.3.0: Introducing MetaSingleton for better singleton management
• Unit tests for ConceptLibrary.is_compatible_by_concept_code

[v0.5.0] - 2025-07-01

Highlights

Vibe Coding an AI workflow becomes a reality — Create AI workflows from natural language without writing code: the combination of Pipelex's declarative language, comprehensive Cursor rules, and robust validation tools enables AI assistants to autonomously iterate on pipelines until all errors are resolved and workflows are ready to run.

Added

• Complete Dry Run & Static Validation System - A comprehensive validation framework that catches configuration and pipeline errors before any expensive inference operations.
• WorkingMemoryFactory Enhancement: New make_for_dry_run() method creates working memory with realistic mock objects for zero-cost pipeline testing
• Enhanced Dry Run System: Complete dry run support for all pipe controllers (PipeCondition, PipeParallel, PipeBatch) with mock data generation using polyfactory
• Comprehensive Static Validation: Enhanced static validation with configurable error handling for missing/extraneous input variables and domain validation
• TOML File Validation: Automatic detection and prevention of trailing whitespaces, formatting issues, and compilation blockers in pipeline files
• Pipeline Testing Framework: New dry_run_all_pipes() method enables comprehensive testing of entire pipeline libraries
• Enhanced Library Loading: Improved error handling and validation during TOML file loading with proper exception propagation

Configuration

• Dry Run Configuration: New allowed_to_fail_pipes setting allows specific pipes (like infinite loop examples that fail on purpose) to be excluded from dry run validation
• Static Validation Control: Configurable error reactions (raise, log, ignore) for different validation error types

Documentation & Development Experience

• Cursor Rules Enhancement: Comprehensive pipe controller documentation covering PipeSequence, PipeCondition, PipeBatch, and PipeParallel, improved PipeOperator documentation for PipeLLM, PipeOCR
• Pipeline Validation CLI: Enhanced pipelex validate all -c pipelex/libraries command with better error reporting and validation coverage
• Improved Error Messages: Better formatting and context for pipeline configuration errors

Changed

• Error Message Improvements: Updated PipeCondition error messages to reference expression_template instead of deprecated expression_jinja2

[v0.4.11] - 2025-06-30

• LLM Settings Simplification: Streamlined LLM choice system by removing complex for_object_direct, for_object_list, and for_object_list_direct options. LLM selection now uses a simpler fallback pattern: specific choice → text choice → overrides → defaults.
• Image Model Updates: Renamed image_bytes field to base_64 in PromptImageTypedBytes for better consistency. Updated to use CustomBaseModel base class to benefit from bytes truncation when printing.

[v0.4.10] - 2025-06-30

• Fixed a bad import statement

[v0.4.9] - 2025-06-30

Highlights

Plugin System Refactoring - Complete overhaul of the plugin architecture to support external LLM providers.

Added

• External Plugin Support: New LLMWorkerAbstract base class for integrating custom LLM providers, and we don't mean only an OpenAI-SDK-based LLM with a custom endpoint, now the implementation can be anything, as long as it implements the LLMWorkerAbstract interface.
• Plugin SDK Registry: Better management of SDK instances with proper teardown handling
• Enhanced Error Formatting: Improved Pydantic validation error messages for enums

Changed

• Plugin Architecture: Moved plugin system to dedicated pipelex.plugins package
• LLM Workers: Split into LLMWorkerInternalAbstract (for built-in providers) and LLMWorkerAbstract (for external plugins)
• Configuration: Plugin configs moved from main pipelex.toml to separate pipelex_libraries/plugins/plugin_config.toml (⚠️ breaking change)
• Error Handling: Standardized credential errors with new CredentialsError base class

[v0.4.8] - 2025-06-26

• Added StorageProviderAbstract
• Updated the changelog of v0.4.7: Moved Added StorageProviderAbstract to v0.4.8

[v0.4.7] - 2025-06-26

• Added an API serializer: introducing the compact_memory, a new way to encode/decode the working memory as json, for the API.
• When creating a Concept with no structure specified and no explicit refines, set it to refine native.Text
• JobMetadata: added job_name. Removed top_job_id and wfid
• PipeOutput: added pipeline_run_id

[v0.4.6] - 2025-06-24

• Changed the link to the doc in the README.md: https://docs.pipelex.com

[v0.4.5] - 2025-06-23

Changed

• Test structure overhaul: Reorganized test directory structure for better organization:
• Tests now separated into unit/, integration/, and e2e/ directories
• Created tests/cases/ package for pure test data and constants
• Created tests/helpers/ package for test utilities
• Cleaned up test imports and removed empty __init__.py files
• Class registry refactoring: Updated kajson from 0.1.6 to 0.2.0, adapted to changes in Kajson's class registry with new ClassRegistryUtils (better separation of concerns)
• Dependency updates:
• Added pytest-mock to dev dependencies for improved unit testing

Added

• Coverage commands: New Makefile targets for test coverage analysis:
• make cov: Run tests with coverage report
• make cov-missing (or make cm): Show coverage with missing lines
• Test configuration: Set xfail_strict = true in pytest config for stricter test failure handling
• Pydantic validation errors: Enhanced error formatting to properly handle model_type errors

Fixed

• External links: Removed broken Markdown target="_blank" syntax from MANIFESTO.md links
• Variable naming consistency: Fixed redundant naming in OpenAI config (openai_openai_config → openai_config)
• Makefile optimization: Removed parallel test execution (-n auto) from codex-tests, works better now

Tests

• Unit tests added: New comprehensive unit tests for:
• ClassRegistryUtils
• FuncRegistry
• ModuleInspector
• File finding utilities

[v0.4.4] - 2025-06-20

Fixed

• Changed the allowed base branch names in the GHA guard-branches.yml: doc -> docs
• Fixed kajson dependency (see kajson v0.1.6 changelog)

Cursor rules

• Added Cursor rules for coding best practices and standards (including linting methods). Added TDD (Test Driven Development) rule on demand.
• Various changes

Documentation

• Added documentation for referencing images in PipeLLM.
• Fixed typos

Refactor

• Removed the images field from PipeLLM - images can now be referenced directly in the inputs
• Moved the list-pipes CLI function to the PipeLibrary class.

[v0.4.3] - 2025-06-19

Fixed

• Removed deprecated Gemini 1.5 models: Removed gemini-1.5-flash and gemini-1.5-pro from the VertexAI integration as they are no longer supported
• Fixed multiple import statements across the codebase

Documentation

• Enhanced MkDocs search: Added search functionality to the documentation site
• Proofreading improvements: Fixed various typos and improved clarity across documentation

Refactor

• Mini refactor: changed kajson dependency to kajson==0.1.5 (instead of >=) to tolerate temporary breaking changes from kajson

[v0.4.2] - 2025-06-17

• Fixed the inheritance config manager method (Undocumented feature, soon to be removed)
• Fixed the deploy-doc.yml GitHub Action
• Grouped the mkdocs dependencies in a single group docs in the pyproject.toml file

[v0.4.1] - 2025-06-16

• Changed discord link to the new one: https://go.pipelex.com/discord
• Added hello-world example in the cookbook-examples of the documentation.

[v0.4.0] - 2025-06-16

Highlights

Complete documentation overhaul:

• MkDocs setup for static web docs generation
• Material for MkDocs theme, custom styling and navigation
• Other plugins: meta-manager, glightbox
• GitHub Pages deployment, mapped to docs.pipelex.com
• Added GHA workflows for documentation deployment and validation
• Added to docs:
• Manifesto explaining the Pipelex viewpoint
• The Pipelex Paradigm explaining the fundamentals of Pipelex's solution
• **Cookbook examples** presented and explained, commented code, some event with mermaid flow charts
• And plenty of details about using Pipelex and developing for Pipelex, from structured generation to PipeOperators (LLM, Image generation, OCR…) to PipeControllers (Sequence, Parallel, Batch, Condition…), workflow optimization, workflow static validation and dry run… there's still work to do, but we move fast!
• Also a major update of Cursor rules

Tooling Improvements

• Pipeline tracking: restored visual flowchart generation using Mermaid
• Enhanced dry run configuration: added more granular control with nb_list_items, nb_extract_pages, and image_urls
• New feature flags: better control over pipeline tracking, activity tracking, and reporting
• Improved OCR configuration: handle image file type for Mistral-OCR, added default_page_views_dpi setting
• Enhanced LLM configuration: better prompting for structured generation with automatic schema insertion for two-step structuring: generate plain text and then structure via Json
• Better logging: Enhanced log truncation and display for large objects like image bytes (there are still cases to deal with)

Refactor

Concept system refactoring

• Improved concept code factory with better domain handling, so you no longer need the native domain prefix for native domains, you can just call them by their names: Text, Image, PDF, Page, Number…
• Concept refines attribute can now be a string for single refined concepts (the most common case)

Breaking Changes

• File structure changes: documentation moved from doc/ to docs/
• Configuration changes: some configuration keys have been renamed or restructured
• StuffFactory.make_stuff() argument concept_code renamed to concept_str to explicitly support concepts without fully qualified domains (e.g., Text or PDF implicitly native )
• Some method signatures have been updated

Tests

• Added Concept refinement validation: TestConceptRefinesValidationFunction and TestConceptPydanticFieldValidation ensure proper concept inheritance and field validation