Changelog
[v0.95.1] - 2026-10-30
Fixed
- A caller webhook's
attemptcan continue when its delivery step runs again:DeliveryExecutor.execute_caller_deliveryanddeliver_caller_webhookstake a keyword-onlyhost_attempt, the host's own attempt at the caller step counted from 1, and number the step's calls after every call its earlier attempts could have made,ATTEMPTS_PER_STEPper URL each, so a durable orchestrator that passes its attempt number when it retries the step hasattemptkeep increasing under the samewebhook-id, possibly skipping numbers, where it restarted at 1. The default of 1 numbers each run of the step from 1 as before, soattemptstays informational and a receiver still deduplicates onwebhook-id; a value below 1 raisesValueErrorbefore anything is sent. The delivery's log lines also carryevent_attempt, theattemptthe call carried.
[v0.95.0] - 2026-10-29
Added
callback_urlsandpipeline_run_idon/v1/execute: the API server's blocking run route takes the caller's webhooks, whose events are sent before it answers, and a caller-chosenpipeline_run_id, honored as/v1/starthonors it, a run still in flight under the same id answering409pipelex.run.already_exists.PipelexMTHDSProtocol.executetakespipeline_run_idtoo.COMPLETION_CALLBACK_URL, the deployment's completion target: the API server notifies this URL, which may name the run's id with one{pipeline_run_id}placeholder in its path or its query, when each/v1/startrun ends, with the completion notice in the operator's view and itsX-Completion-Signature. A malformed value, or one set withoutCOMPLETION_CALLBACK_SECRET, stops the server at boot, and a/v1/startrequest'scallback_urlsentry equal to the run's completion URL is notified once, as the completion target, while on/v1/execute, which notifies no completion target, it is a caller's webhook.pipelex.webhook.secret_not_configured: a422registry code naming insettingthe runner setting that holds no secret to sign a caller's webhooks with.- The library delivers caller webhooks:
DeliveryAssignment.caller_webhookslistsCallerWebhookTargets, each with thedelivery_idits calls carry, andpipelex.tools.network.standard_webhookssigns a call in the Standard Webhooks format.DeliveryExecutordelivers a run in two steps a durable orchestrator may run apart:execute_host_deliverystores the results and notifies the host's webhooks, thenexecute_caller_deliverysends the caller's events, last and once, with how the run ended, whichrun_ending_after_host_failuregives when the host step failed.DeliveryExecutor.executeruns both, and takes the run'sfinished_atandextras.
Changed
callback_urlsreceive the signedrun.finishedevent (Breaking): each URL a request to the API server registers receives the caller-webhook contract's event,type,timestamp,attemptanddata, the run as the protocol's status read serves it with a failed run's problem in the caller's view, signed with the Standard Webhooks headerswebhook-id,webhook-timestampandwebhook-signatureunder thewhsec_secret or secrets inCALLER_WEBHOOK_SECRET. It replaces the completion notice and itsX-Completion-Signature, which no caller webhook receives any more, so a self-hosted receiver now verifies the Standard Webhooks signature, and a server withoutCALLER_WEBHOOK_SECRETrefuses a request carryingcallback_urlswith a422pipelex.webhook.secret_not_configured. Each URL is delivered on its own, three attempts within 15 seconds each and no redirect followed, and a delivery that fails is logged and never changes the run.- The completion notice goes to
COMPLETION_CALLBACK_URLalone (Breaking): a host that relied oncallback_urlsto receive the completion notice and itsX-Completion-Signature, as a hosting platform does, setsCOMPLETION_CALLBACK_URLto its completion URL before taking this release; otherwise its URL is a caller webhook and, on a server withoutCALLER_WEBHOOK_SECRET, every/v1/startcarrying it is refused. The notice now always carries the operator's view of a failed run's problem, whoever started the run.
Fixed
- A
/v1/startrun whose dispatch fails gives its id back: the API server releases what it set up for a run its orchestrator could not enqueue, its registration, its graph tracer and its library, so the samepipeline_run_idcan be started again;pipeline_run_setup's new companionrelease_run_setupdoes it for any host. - A cancelled run is never reported completed: a run cancelled before it ended, which
PipeRunre-raises the cancellation of, now tells its webhooks, the host's completion notice included, that it failed, with the problemrun.cancelled, where the notice saidCOMPLETED; a delivery that fails afterwards never replaces the cancellation. - A failed notice to a host's webhook names its reason: the
pipelex.webhook.delivery_failedproblem a run fails with when one of its host's webhooks cannot be notified carries thereasonits code requires,http_statuswith the receiver'sreceiver_status, orunreachable. - A refused webhook no longer replaces a failed run's own failure: a delivery destination the SSRF guard refuses after the run failed logs its security event and the delivery's failure, and the run fails with its own error, where the refusal used to be raised in its place.
[v0.94.0] - 2026-10-29
Changed
mthdsandpipelex-sdkversions (Breaking): The pins move tomthds==0.26.0, the release following the MTHDS standard'sv8.0.1, andpipelex-sdk==0.57.0, the release built on it, whose own breaking changes since0.53.0, such aslist_runsanditerate_runstakingmethod_idas a keyword-only filter and method-version fields no longer coerced, reach anyone importingpipelex_sdkbeside pipelex. Everyone downstream inherits those exact versions.- MTHDS standard
8.0.1and MTHDS Protocol2.0.1(Breaking): every crate pipelex normalizes, on the command line and on the API server'sPOST /v1/resolve, is stampedmthds_version: "8.0.1", aMETHODS.tomlwhosemthds_versionis exactly8.0.0now warns where>=8.0.1passes, andpipelex --version,pipelex-agent --versionandGET /v1/versionreport the MTHDS Protocol2.0.1. Crate fingerprints do not move. - The problems a run raises carry titles written for whoever follows it: every problem pipelex emits under a core code a run can raise, in a run's failure, an API server answer or the CLI's error output, carries the title the core error registry
8.0.1gives it for someone following a run of a method they did not write:pipe.output_multiplicityis titled "A step produced a list instead of one result, or the reverse" where it read "A pipe's output does not have the declared multiplicity", andpipe.output_invalid"A step's result does not have the structure the method declares". No code, domain, retry verdict, step kind or status changes.
Fixed
- The API server checks a
filesmap's blobs last: as the MTHDS Protocol2.0.1orders a map's refusals, a map holding a key the tree's rule refuses, colliding keys or a malformed entry is refused for that before any blob it names is read, where a hash held at another size than its entry states answered422request.invalid, and a damaged stored blob500pipelex.storage.damaged, ahead of the key's own refusal.
[v0.93.1] - 2026-10-28
Added
pipelex.billing.credit_exhaustednames the credit ledger that ran out: the code declares an optionalledgerparameter,inferenceorusage, so the hosted platform's refusal of a run says whether the organization's inference credit or its usage credit is spent, andBillingCreditExhaustedParamsgains theledgerfield. A model call refused once the credit has run out names no ledger, as before.
[v0.93.0] - 2026-10-27
Fixed
- A direct provider's spent quota is read from its structure, not its wording: a call with your own keys to OpenAI or a backend on the OpenAI SDK, Anthropic, Google Gemini, Mistral or Amazon Bedrock is a spent quota only on the status, error code or error detail that provider documents for one, and is otherwise retried as
provider.rate_limited. A 429 whose message mentioned a quota, as Gemini's per-minute throttles do, used to fail at once asprovider.quota_exhausted. Anthropic's credit-balance and self-set usage-limit refusals, which carry no code, are still read, by their opening words. - Throttles sent on another status than 429 are retried: OpenRouter's 402 for its in-flight budget, Groq's 498 for its flex tier at capacity, and Bedrock's
ServiceQuotaExceededException, from Bedrock's SDK or from Claude on Bedrock through the Anthropic SDK, are nowprovider.rate_limitedwhere they failed at once. - Gemini's retry delay is carried: the delay Gemini gives in its error's
RetryInfodetail, not in aRetry-Afterheader, is now the failure'sretry_afterand is named in its advice. - A delay in milliseconds is read: a
retry-after-msheader above zero, which Azure OpenAI sends besideretry-after, is now the delay a rate limit carries, read first as the OpenAI and Anthropic SDKs read it, on a call through either SDK, Azure's REST API or a gateway. The transport retry of a call to Azure's REST API now waits that delay too, where it readRetry-Afteralone. - More of Bedrock's wordings of an input over the window are recognized: a
ValidationExceptionsaying "Input Tokens Exceeded" (Amazon Nova), "Too many input tokens" (Amazon Titan) or "prompt tokens (N) exceed model maximum (M)" is nowmodel.context_exceededwhere it wasruntime.unexpected.
Removed
ProviderErrorMetadata.is_quota_exhaustion(Breaking): a plugin readsProviderErrorMetadata.limit_verdictinstead, aProviderLimitVerdictofquota_exhausted,rate_limitedorneither, the one entry point the classifier reads.
[v0.92.3] - 2026-10-27
Fixed
- A gateway's rate limit is no longer read as a spent quota: a 429 relayed by a Portkey-dialect gateway is now
provider.rate_limited, retried with the provider'sRetry-After, unless its error code is one the provider documents for a spent quota: OpenAI'sinsufficient_quota,credit_balance_exhausted,organization_spend_limit_exceeded,project_spend_limit_exceededororganization_usage_limit_exceeded. A 429 whose message mentioned a quota, as Azure OpenAI's throttles do, used to fail at once asprovider.quota_exhaustedwith advice to add credit. A 402 is still a spent quota.
[v0.92.2] - 2026-10-26
Added
- A fetched method's provenance carries its declared name:
MethodProvenancegainsdisplay_name, thedisplay_namethe package'sMETHODS.tomldeclares at the fetched commit, andnullwhen it declares none, so themethod_provenanceof amethod_refrun on/v1/executeand/v1/startreports it beside the address, tag, commit and tree digest, and a host can name the run's method after it. A fetched copy's.provenance.jsonsidecar records it too, and a sidecar or a record written without it still reads, itsdisplay_namenull.
[v0.92.1] - 2026-10-26
Fixed
- The S3 storage provider reuses one client per event loop: each event loop opens one aiobotocore client on its first S3 operation, and every later operation on that loop shares it and its pool of
S3_MAX_POOL_CONNECTIONSconnections, where every operation, a presigned URL included, built a client and opened a TLS connection of its own; the client is closed as its loop shuts down or at the runtime's teardown, which now callsStorageProviderAbstract.teardown(), a no-op that a storage plugin keeping a client open between operations overrides to close it. Measured from a laptop against a bucket in another region, the API server's read of a method tree of 20 blobs went from 4,261 ms of wall time and 425 ms of CPU to 1,582 ms and 32 ms with this change alone. - The API server reads each blob a request names in one request: a run, a validation and the tooling routes read each hash entry of
filesanddependencieswith one read of at most one byte past the size the entry states, 16 at a time, where they looked each blob's size up first and then read the blobs 4 at a time; a hash held at another size still answers422request.invalid, its size looked up only for that refusal's message, and damaged bytes still answer500pipelex.storage.damaged. Measured as above with both changes, reading a tree of 1, 20 and 50 blobs takes 194, 419 and 838 ms of wall time and 1.3, 14 and 31 ms of CPU, where it took 1,142, 4,261 and 10,253 ms and 26, 425 and 1,000 ms. The local storage provider now reads such a leading range of a file without loading the whole file.
[v0.92.0] - 2026-10-25
Changed
- The API server refuses a method's unmet requirements as an invalid method (Breaking):
/v1/executeand/v1/startrefuse a method whoserequirements.txtthe PipeFunc sandbox does not meet, or holds a line the standard refuses, with a422method.invalidwhoseerrorshold onemethod.dependency_missingormethod.package_invaliditem per refused line, located atrequirements.txtand the line, the problem a local run of the method raises and the items/v1/validategives, where they answered a baremethod.dependency_missingormethod.package_invalidwith no item and no location; a client reads the refused requirement from the item'sparamsandlocation.
Fixed
- A failed step's trace records the failure's own code: a failed node of the execution graph, and the
pipe_end_errortrace event, now carry the problem the run fails with when the step's error wraps a carrier of a problem recovered across a transport boundary, such aspipe.function_raisedfor aPipeFuncwhose function raised in a sandbox, located at the step, where they carried the carrier's catch-allruntime.unexpected. The step's own message names that recovered code too,Error: <carrier class> (pipe.function_raised), andreported_failure_code(pipelex.pipe_run.located_failure) reads the code a failure reports without building its problem.
[v0.91.0] - 2026-10-25
Added
- OpenRouter model research:
pipelex-dev openrouter list-modelslooks up the public catalog across modalities, filters and sorts models, and emits Markdown or clean JSON.--savewrites disabled draft backend TOML with output-token limits and known token prices for review, independently of remote-config tooling. - The validation report names the files each step reads:
PipelexValidationReport, and so the valid verdict of/v1/validate, carriestree_files_read, mapping each validated pipe'spipe_refto the files of the method's tree it reads, each aTreeFileReadwith itstree_pathand itsuse(templatefor aPipeDocGenstep'stemplate_file,template_assetfor a file its HTML template names by a literal relative URL,python_modulefor the module holding aPipeFuncstep's function), an empty list for a pipe reading none, andtree_file_readers, mapping each of those files to the pipes that read it.pipelex-agent validate bundleandvalidate methodcarry both in their envelope, and the Markdown of a valid verdict lists them in a "Files the steps read" section. - A sandboxed PipeFunc reports its function's own failure: when the function a
PipeFunccall runs raises inside a sandbox,python -m pipelex.pipe_operators.func.pipe_func_transported_entrypointwrites aPipeFuncRaisedReportto its result file before it exits non-zero, holding the operator's view of thepipe.function_raisedproblem, which names the function and the step and the class of what the function raised, never its text, while a sandbox failing any other way, such as a timeout inside it, still writes nothing there. An out-of-process executor reads the result file of a call that exited non-zero withread_pipe_func_raised_report(pipelex.pipe_operators.func.pipe_func_execution_transport), which answers the problem, rendered again from the registry and with only the diagnostics a sandbox writes, orNonefor no report or a report of any other code, and raises a Pipelex error carrying it, so the step readspipe.function_raisedas it does in process. - A fetched package is checked against the calling method's
methods.lock: when a method carries amethods.lockat the root of its tree, a package it calls by address that the runtime fetched itself, for this load or by an earlier fetch whose installed copy records its provenance, must have the tree digest the lock records for its address, whatever version the lock names, else the load is refused with the core codesecurity.integrity_mismatch(domainmethod, a422with its security event on the API server), whose message names the locked and fetched versions and digests and says to pin the reference to the locked version or to lock the method again, or, when an installed copy answered a reference pinning another tag, says so and to remove that copy. A lock not naming the address, or no lock, checks nothing, and packages shipped with the method or installed by another tool are not checked. The lock is read only when a dependency resolves to such a copy, and one the runtime cannot read, one recording ahashfrom before tree digests included, then refuses the load withmethod.package_invalidand the new reasonlock_invalid.
Changed
- Every S3 write carries its SHA-256 (Breaking): the S3 storage provider sends the SHA-256 of every object it writes, so S3 refuses a body that arrived damaged and keeps the checksum beside the object, where a
HeadObjectwithChecksumMode=ENABLEDreturns it, and a generated file's write carries the whole digest of which its key carries the first 16 digits; its client no longer signs the payload, which the signed checksum protects.StorageProviderAbstract.storetakes the digest aschecksum_sha256_hexfrom a caller that hashed the bytes already, and hands it to_store, so a storage plugin adds the keywordchecksum_sha256_hexto its_store, sending it when its storage can check a write's checksum and ignoring it otherwise. pipelex-agent validate bundle --graphwrites into an output folder (Breaking): the flowchart it draws,dry_run.html, andmermaidflow.htmlwith--graph-format mermaidfloworboth, go into a new numbered folder,<pipe_code>_output_<NN>/, under theresults/directory arun bundleof the same target writes to (beside the bundle file named, beside the bundle defining a folder's entry pipe, or in the folder's ownresults/when--pipenames the pipe), holding the.gitignorea run's output folder holds, instead of beside the bundle drawn, so validating a method no longer adds files to its tree or changes its digest. Read each file's path fromgraph_filesin the envelope, the Markdown listing them under## Graph files, and the folder fromgraph_output_dir.- The API server's examples send a method in
files: the server's README and the API Server pages send a method to/v1/executeand/v1/validateinfiles, a map from each file's path in the method's folder to its content, where they sentmthds_contents. The reference pages and the OpenAPI artifact now say thatmthds_contents, withbundle_b64,resourcesand/v1/validate'smthds_sources, is being retired and will be refused by a later release, so a client sendsfilesnow. - A PipeFunc package-offer factory is never called on a thread running an event loop (Breaking): the factory a plugin registers with
PluginRegistrar.add_pipe_func_package_offermay block for as long as its work takes and must not use an event loop of its caller's, and pipelex enforces it: a load that would call the factory on a thread running an event loop fails withPipelexUnexpectedErrorinstead, so a host that calls a load itself from asynchronous code, such as a worker's activity, moves it off the loop withrun_method_load(pipelex.libraries.method_load_threads) or withacquire_library_async(pipelex.pipeline.execution_seams), the twin ofacquire_librarythat loads off the loop. The factory's signature is unchanged, and so is the plugin API version.run_method_loadruns a blocking call in the method-load pool, a named pool of at mostMETHOD_LOAD_MAX_THREADSthreads (four) apart from asyncio's default executor, in a copy of the caller's context, and when its caller is cancelled, the shutdown ofasyncio.runincluded, withdraws a call no thread has taken yet or lets the cancellation through only once the running call has ended; the general mechanism isWorkerThreads(pipelex.tools.misc.async_utils). - The refusal of a package the request did not send: the message of the
unresolved_package_dependencyrefusal a static resolution gives, such as the one behind/v1/resolve,/v1/codegenand/v1/pipe-io, tells its reader to send the package's files independencies, keyed by its address, beside the method's own files infiles, where it spoke of an in-memory resolve and the host filesystem and advised directory-based resolution; it still carries the wordsaddress-based dependency '<address>'. - A run lists each method tree once (Breaking): the method trees a run reads, the method's own, its library directories' and its dependency packages', are each listed once, and the run loads the trees its caller listed, so the
method_tree_digesta run records is the digest of the tree it loaded:PipelexMTHDSProtocol.execute,pipeline_run_setup,acquire_library,acquire_library_async,validate_bundle,validate_bundles_in_process,fix_bundle_file,fix_method_folder, andLibraryManagerAbstract.load_librariesandload_from_blueprintstaketree_listing, aMethodTreeListing(pipelex.libraries.method_tree_listing) holding the trees already listed, whichrun bundle,validate bundleandfix bundleon both CLIs hand down from the listing that resolved their target. The listing reads a path as the standard's lister does before it takes anything from what it holds: a folder and a symbolic link naming it are one listing, since the lister follows such a link, a bundle file named alone inside a library directory shares the listing of the tree that directory roots, and a symbolic link to a bundle file, named alone, is refused as the lister refuses it, even once the run listed the file it points to. Loads of one run running at the same time list each tree once and list different trees side by side. The API server's run routes and/v1/validateload the trees their check of the method's requirements listed, whichensure_served_method_requirements_metnow lists throughtree_listingin place of itslimits, and which reaches the in-process validator throughscoped_tree_listing, so the bundle-validator plugin contract is unchanged.list_library_treesandlist_library_dir_trees(pipelex.libraries.method_trees) take the lister of a directory's own tree or of its trees (own_tree_of,trees_of), through which the listing lists them, andtree_bundle_of_lone_fileis removed: a bundle file named alone is placed in its tree withMethodTreeListing.lone_file_bundle. - A run asks the PipeFunc package-offer factory once per tree: the run's
MethodTreeListingkeeps each method tree and each dependency package whoserequirements.txta check found met, with the execution mode and the factory it was judged against (PipeFuncEnvironmentSource,current_pipe_func_environment_sourceinpipelex.libraries.package_environments), so a run judges each tree once against them however many of its steps and loads read it, two loads judging one tree at the same moment ask once (MethodTreeListing.judge_requirements_once), and on the API server the load serving a request takes the verdict of the runner's check of the served method instead of asking the factory again. A check against another mode or factory, a refused tree and every new run ask again: pipelex keeps a verdict for one run alone, so a factory whose answer takes time keeps its own cache across runs, by the requirements' texts.ensure_method_requirements_metandensure_package_requirements_mettake the listing astree_listing. mthds0.25.0 andpipelex-sdk0.53.0 (Breaking): pipelex pinsmthds==0.25.0andpipelex-sdk==0.53.0, the release whose errors carry the hosted API's problem document, so it implements the MTHDS standard 8.0.0 and the protocol 2.0.0, which the version handshake reports, and a method tree never holds a run's inputs file,inputs.jsonorinputs.tomlat any depth, so a method folder holding one is listed, and digested, without it.PipelexMTHDSProtocoltakes the protocol'sfiles(Breaking):execute,startandvalidatetake the MTHDS protocol'sfilesmap in the position the protocol gives it, and the local runtime refuses a map holding a file withrequest.unsupportedand the reasonmethod_files_on_local_runtime,METHOD_FILES_UNSUPPORTED_REASONinpipelex.errors.unsupported_reasons;mthds_contentsis a keyword-only argument, whichvalidaterequires. The API server'sApiRunnertakes the same arguments, and its routes, which write a method sent infilesout as a tree before they run it, are unchanged.- A hosted run sends its method as a tree (Breaking):
pipelex run --hostedandpipelex-agent run --runner hostedsend a local method to/v1/pipe-io,/v1/startand/v1/executeas the MTHDS protocol'sfilesmap, each.mthdsbundle under its path in the method's tree and each library directory beside the method under a top-level folder named after it, inline up to 256 KiB and by hash beyond, where the run took the bundles' bare contents and pipe-io a list the runner now refuses with a422, so a hosted run whose inputs name a local file reads the signature again, and a method that does not load is reported by the tree path of each faulty file.HostedRunRequest(pipelex.hosted.hosted_run) takescontents, each bundle's bytes by tree path, in place ofmthds_files. A file of the tree that cannot be uploaded stops the run before anything starts, as pipelex-sdk'sMethodFileTransferError, which both CLIs report with the SDK's verdict and a next step: run again, or report a file the storage refused. A bundle file given with-Lthat the membership rule keeps out of its tree, a hidden one under a dot segment or one.gitignoreexcludes, is refused before anything is sent, as a local run refuses it, where it was sent as it was. - A hosted run's lost connection follows pipelex-sdk's reading:
pipelex run --hostedandpipelex-agent run --runner hostedreport a connection lost on the start or the execute request asHostedRunOutcomeUnknownError(client.outcome_unknown) whenever pipelex-sdk reads it as one that may have started the run (its problem's retry verdictoutcome_unknown), a proxy's failure and a TLS handshake that was dropped included, which read as an unreachable API, while a connection lost on a request sent before the run's, such as the blob check of a tree sent by hash, stays pipelex-sdk'sApiUnreachableError. - Both CLIs report a hosted failure as its problem (Breaking):
pipelex run --hostedprints the problempipelex-agent run --runner hostedreports, its title, detail, next step, code with its domain and retry verdict, request id, run and items, and a failed run's own problem below it, both read from pipelex-sdk'sexc.problemthroughdescribe_hosted_error, which now takescredential_sentand returns the problem, the error's own sentence and pipelex's advice. The readers of the hosted API's answers in the shape that predates the problem document are removed: an answer that is no problem document keeps pipelex-sdk's reading by its status, except a bare401or403, which reads asauth.missing_credentialwhen no key was sent andauth.invalid_credentialwhen one was, and a bare404,request.not_found, each with the next step that names the CLI's settings.HostedMethodInvalidErrorcarries the items of the pipe-io verdict aserrors, the standard's problem items, where it carriedvalidation_errors, and a run past the wait reports pipelex-sdk'sclient.follow_failed. pipelex-agent validate bundleandfix bundleexit 2 on a configuration refused before the boot: a configuration that does not validate, which these commands read before the boot for the limits a method's folder is listed under, exits 2, no verdict, where it exited 1, aspipelex validate bundleandpipelex fix bundleexit on it; the problem is unchanged, andrun bundleandinputs bundlestill exit 1.- The development Python is pinned in
.python-version: the library's.venvand the API server'sapi/.venvare built on the version the root.python-versionnames, and the Makefiles hand that version to every uv command, soPYTHON_VERSION=<version> make installnow moves an existing environment to the version asked for, where it used to keep the environment's own Python without a word. The CI jobs that run another version set it for the whole job, and the matrix jobs check that each cell ran the version it names.
Fixed
- A hosted run refusing a local file names its reason: the
request.unsupportedproblempipelex run --hostedandpipelex-agent run --runner hostedreport when the inputs name a local file and the method calls another by its address carries the parameters its code requires,field(inputs) andreason(local_file_with_address_dependency,LOCAL_FILE_WITH_ADDRESS_DEPENDENCY_UNSUPPORTED_REASONinpipelex.errors.unsupported_reasons), so a client can tell this refusal apart. - A file name's type is the same on every Python:
mime_type_of_file_nameanswerstext/markdownfor.mdand.markdown,application/yamlfor.yamland.yml, and the OpenDocument types for.odt,.odsand.odpon every supported Python, where Python 3.11's table knew no Markdown and only 3.14's knew the others, so a file of these types has the format keymd,yaml,odt,odsorodpwhatever the Python, and a bare zip named.odt,.odsor.odpis identified as that OpenDocument type. - A method edited between its listing and its load is refused: a run read its method's files again after listing them, so a file edited in between was loaded under a digest that named other bytes. Every file a load reads from a tree listed on this host, a bundle it parses, the
requirements.txtit judges and the Python it scans, is now read against the hash the listing recorded, a load reads every file of a tree the run listed before it again, once per run, as it takes the tree, and the bundle text a CLI read for its run is checked against the listed file, so a bundle, a template or any other file of the tree edited, replaced or removed since the listing is refused before anything of the method is parsed, imported or run, withMethodTreeChangedError(pipelex.libraries.exceptions), codedmethod.package_invalidwith the new reasontree_changedand the file'spathin the tree;run,validateandfixreport it as an invalid method whose item names that file. Arequirements.txtthat can no longer be read is this refusal too, where it wasrequirements_invalid. The checked reads areread_tree_file_as_listed,read_tree_bundle_as_listed,ensure_tree_files_as_listedandensure_bundle_text_as_listedinpipelex.libraries.method_trees,MethodTreeListing.loading()marks a load reading a listing andensure_unchanged()checks a tree a caller placed a bundle in, andMthdsParser.make_pipelex_bundle_blueprinttakes the bytes it parses asbundle_data, beside thebundle_paththat names them. - Method loads leave the event loop: every method load pipelex reaches from asynchronous code, a run's, a validation's and those of the commands that run a loop, runs in the method-load pool, where it ran on the event loop, the reading and parsing of the bundles it loads included, so listing and hashing a method's files, a sandbox's judgment of its
requirements.txtand a fetch of a package it calls by address no longer hold up the other requests or activities the loop serves; the API server's run routes and/v1/validatealso write a request's method tree to disk and remove it there, and judge its requirements there, so a client that disconnects never has the tree removed while the check still reads it, the crate routes resolve and project there, and the in-process validator reads a tree's entry pipe for its graph there too. A module imported by file while a method loads, such as one defining a structure class, is published and executed under a lock that the load's class discovery also holds, so two loads in parallel never take a module half executed, and a module that fails to execute leavessys.modulesrather than stay there half imported./v1/validateof a method tree calling a package nothing shipped still resolves it as a run does, from the methods installed on the server and by a fetch on a miss, so its verdict matches the run's, and the fetch now runs off the event loop. - Where a method's Python travels, in the docs: the PipeFunc Requirements, PipeFunc and run pages say that the API server receives a method as a tree holding its bundles and its Python together and judges its
requirements.txtfrom the tree's listing, that the runner never imports a method's Python, that a run's crate names each step's method tree by its digest and carries none of its files, that aPipeFunccall's files are read through the method file store the worker's host installs, and that a worker a method tree's validation is dispatched to checks the method's requirements again, where they said the server received a method's Python apart from its bundles, the Python was captured as text, the crate carried the method's Python sources and the worker never checked anything.
Removed
resolve_crate_from_contents(Breaking):pipelex.pipeline.resolve_bundleno longer resolves a crate from bundle contents held as strings. A host resolves a method as/v1/resolve,/v1/codegenand/v1/pipe-iodo, by writing its tree out and callingresolve_crate_from_method_dir, with the packages it ships inmethods_dirs; a call into a package nothing shipped is then anunresolved_package_dependencyitem of the invalid verdict, never therequest.unsupportedrefusal with the reasonaddress_based_dependency, which no resolution of this release gives: its constantADDRESS_DEPENDENCY_UNSUPPORTED_REASON(pipelex.errors.unsupported_reasons) is removed withADDRESS_DEPENDENCY_REFUSAL_MARKER(pipelex.methods.address_dependencies), and a hosted run no longer reads that refusal from/v1/pipe-io, only the invalid verdict naming the package nothing shipped.
Security
aiohttp3.14.5 in the API server image: the lock thepipelex/pipelex-apiimage installs from movesaiohttppast GHSA-494j-54xm-rwp3, GHSA-5wr8-gcqf-gqf3 and GHSA-2g87-pp6c-29x5, withyarl1.25.1 alongside; the publishedpipelexwheel's own requirements are unchanged.
[v0.90.0] - 2026-10-19
Added
- The Pipelex error registry: the codes Pipelex emits beyond the MTHDS core registry, under the
pipelexextension prefix the core registry registers to it, are authored in one public file in the MTHDS error registry format,pipelex/errors/registry/pipelex-error-registry.yaml, each declaring its title, domain, retry verdict, steps, typed parameters, templates and status. The package ships its JSON snapshot,pipelex-error-registry.json, published under Pipelex's version, a pre-release's written in the SemVer form the registry format requires (0.88.0rc1as0.88.0-rc.1), whichload_pipelex_error_registry()reads into a typed model, and every release attaches the snapshot to its GitHub release. pipelex.errors.registry.pipelex_codes:PipelexErrorCode, a member per code of the Pipelex error registry, and for each code a closed, frozen and strict parameters model typed as the registry declares, with the code and its operator-only parameters as class attributes,PIPELEX_ERROR_PARAMSmapping each code to its model.- A documentation page per error code: docs.pipelex.com documents each code of the Pipelex error registry on its own page, under the Error Codes index at
errors/, and its type URI,https://docs.pipelex.com/errors/<code>, leads to that page once a release documenting the code is the latest. pipelex-dev generate-error-registryandpipelex-dev check-error-registry(make ger,make cer): generate the registry's snapshot, its typed Python and its pages from the source, refusing a source the format's JSON Schema, the typed model or the registry's rules refuse, or check that every derived file is what the generator writes;pipelex-dev write-error-type-redirectswrites the redirects that make a code's type URI resolve, which every documentation deploy runs.- Every error carries its registry code: every
PipelexErrorsubclass declares inerror_codethe code it raises, a code of the MTHDS core registry or of the Pipelex error registry, or else anErrorCodeSetits raise sites choose from withwith_error_code(), its cause's code (InheritsCause, with a fallback), orABSTRACT_ERRORfor a base never raised itself.resolve_error_code()gives the code an error reports, the first one named along its cause chain, andruntime.unexpectedwhen none is;pipelex.errors.error_codesreads any code's title, domain, retry verdict, status and steps (error_code_spec,parse_error_code). PipelexError.with_items(): a raise site that finds several faults of one kind gives one problem item per fault, so aPipeParallelwhose branches give several results of a multiplicity their fields do not declare names each one in itspipe.output_multiplicityproblem'serrors, beside the first in its sentence.PluginRegistrar.declare_error_codes: a plugin, a closed one included, declares the public codes it raises while it registers, and a code neither the MTHDS core nor the Pipelex registry defines refuses the boot withPluginErrorCodeNotRegisteredError(pipelex.plugin.broken), naming the plugin and the code.--error-formatonpipelex-agentitself and on itsinputscommands: the error format can be given before the command,pipelex-agent --error-format json validate bundle …, where a command's own--error-formatstill wins and its--formatstill serves when neither is given, so a caller forwards one flag whatever the command; theinputscommands take--error-formattoo, falling back to the program's and then to JSON, never to their--format, which is the template's serialization.InputFreeValueErrorand the input-free rendering of a refusal: a pydantic validator raisingInputFreeValueError(pipelex.tools.typing.validation_exceptions) declares that its message names nothing of the value it refused, which keeps its sentence wherever a refusal of a run's data is written, andformat_validation_error_without_input()anddescribe_exception_without_input()(pipelex.tools.typing.pydantic_utils) write aValidationErroras its locations and reasons, never the values it refused.PipelexRequestError: the runtime's own refusal of a request it cannot serve is aPipelineRequestErrorof the mthds package carrying the core coderequest.invalid(orrequest.unsupported) and its problem document, so a client reads the same refusal whether the runtime or the standard's client raised it.- Validation warnings as problem items:
warning_problem_items()inpipelex.pipeline.validation_errorsgives a valid report's warnings as problem items, each under the code its type names, a hint lint'smethod.hint_invalidnaming its hint from the newValidationErrorItem.hint_keyand located at the declaration the hint is on, a concept atconcept.<Concept>, a field of its structure atconcept.<Concept>.structure.<field>, a pipe's input at the pipe and the input, androot_escape_of(), whoseRootEscapegives both the reason a path leaving its root is refused and itsrule, andpath_refusal_error()inpipelex.core.bundle_resources.bundle_resource, which builds the refusal of a submitted file's path from that escape, with its code and its parameters, theruleofsecurity.input_path_refusedor the fieldresourcesofrequest.invalid. - The security event: a refusal coded in the
security.orpipelex.security.family fails its run under that code and logs one record on its own logger,pipelex.errors.security_event, at warning level, with the messageA security rule refused a requestandevent.nameset topipelex.security.refusal, naming the code, its domain, the kind of target refused, the class of destination when the rule knows it (loopback,private,link_local,metadata,scheme,foreign_storage_key, which the caller's problem states as the registry'sprivate, since its sentence reads the class as a word), the run, the request and each of the run's extras asrun.extras.<key>, which the API server reads off a request'sanalytics_groupsas soon as its JSON body is parsed, on the diagnostic routes as on the run routes, and never the URL, path, template or any input value. The process that raises a refusal logs it, once: the pipe run for a refusal raised while it executes, its delivery included, the validation sweep for one a dry run meets before its pipe run starts, such as while its mock inputs are built, the runner for one before the run exists and for one raised in its own process that no pipe run logged, and the API server for one answered before any run, the refusals it authors itself included (a reserved object key, a callback URL the SSRF rule refuses, a resource path or a method file path leaving the method: absolute, naming a drive or a network share, or climbing with.., read with either separator, while a path of the wrong shape, empty or holding a backslash or a colon, logs none and isrequest.invalidas a key ofresourcesor a file submitted beside a method, andmethod.package_invalidwith the reasoninvalid_tree_pathas a key offilesor a member ofbundle_b64), which keep their response, while a failure rebuilt from another process's report is never logged again, since the orchestrator that rebuilds it marks it, and no event names a run id a request body chose;pipelex.errors.security_eventgives a hostemit_security_event(),mark_security_event_emitted()andlog_security_event(), and a raise site states the class of destination withPipelexError.with_security_target_class(). pipelex.test_extras.required_params_sweep:spy_on_problem_warnings, set before a test, andassert_problems_carried_required_params, run after it, fail a test that builds a problem or an item lacking a parameter its code requires, unless the test carries thebuilds_problems_without_required_paramsmarker with its reason, and the library's unit tests and the API server's run under them.pipelex.request.dependency_key_mismatch: a dependency a run or validation request keys by another address than the one its ownMETHODS.tomldeclares, such as its repository's address alone, is refused with this 422 code, whosekeyandaddressparameters give the key sent and the address to key it by, where it was a genericrequest.invalid.
Changed
- The problem document replaces
ErrorReport(Breaking): everyPipelexErroranswers a problem document,PipelexError.to_problem(view=...)building aPipelexProblem(pipelex.errors.problem), where it built anErrorReport: itstype,title,status,domain,retryandnext_stepare its code's, itstypethe code's page (https://mthds.ai/errors/<code>for a core code,https://docs.pipelex.com/errors/<code>for a Pipelex code), itsdetailand its step are rendered from the code's templates and the parameters the error gives, the code's title standing in for a sentence a missing parameter leaves unrendered, a refusal of several faults lists them inerrors, and the Pipelex membersrequest_id,run_erroranddiagnosticsjoin the standard's. A problem never writes a member a model declares asnull, while anullan extension member or a free-form object holds is written as it was read,to_wire()writes back every problem a reader accepts however deeply an extension member nests, a parameter of a type its code does not declare, an infinite orNaNnumber and a value that is neither a JSON scalar nor a list of them included, is dropped and logged rather than raise in place of the error, and an item never carries a parameter its code keeps for the operator, whoever built it, the caller reading an item whose code this process does not know with no parameter and a generic sentence.ErrorReport,TracedFailureand the graph'sErrorSpecare gone, anderror_typeis no member of a problem: the operator reads the error's class asdiagnostics.error_class. - The reader's identity picks the view of a failure (Breaking): a problem is read in the caller's view, its sentences rendered from the parameters a caller may read, or in the operator's, which adds
diagnostics(the error's class and message, its causes, the model, the provider's answer, a migration, the runtime version and the trace) and the operator's own next step;DisclosureMode, theERROR_DISCLOSUREsetting andcaller_facing_messageare gone, andcaller_view_of()projects an operator's problem to the caller's, giving a code this process's registries do not hold the generic step of its kind,reportfor an operator's code. The runner answers the operator's view to a request carrying its static API key (AUTH_MODE=api_key) or reaching it through a trusted proxy (AUTH_MODE=nonewithTRUST_FORWARDED_IDENTITY_HEADERS=true), which owes its own callers their caller view, and the caller's view to every other request, and a webhook receives the view its target'sproblem_viewnames, the caller's by default, the problem built only for a run that has a webhook and never replacing the run's own error when it cannot be built. - The runner answers each code's HTTP status (Breaking): an error response's status is its code's, from the error registry, where it followed the error's domain and a table of per-class overrides, so
provider.unavailableanswers 502,provider.timeout504,model.unavailable501, a method that cannot be fetched (method.fetch_failed) 502 where it answered 422, and a fetched method declaring Python structures (pipelex.deployment.python_structures_refused) 422 where it answered 403;Retry-Afterrides a 429 alone, and an exception that is not a Pipelex error answersruntime.unexpected, a 500. - An input refusal names the input and what is wrong with it: a refusal of a run's inputs carries the input in
params.inputandlocation.input, with the facts its code declares, the expected and provided counts ofinput.wrong_multiplicity, the declared and provided concepts ofinput.wrong_conceptand the field at fault ofinput.invalidwhen the concept's structure names one, so itsdetailnames the input and its fault; a refused file, whether its url is blank or malformed, its storage reference names nothing, its data URL or local path cannot be read, the run may not read it, or its format is one its input or its model cannot take, names the run's input asinputand the path down to the file aslocation.field, such asdocuments[1], ordocuments[1].urlfor its url, whichinput.invalidalso gives as itsfield. A CSV table input refused before it is read says what happened: a url that does not parse isinput.file_unreadable, a url that is not a local pathrequest.unsupportedwith the reasonremote_table_url, and a concept with no structure classmethod.concept_invalid, where each was aninput.invalidabout the concept's structure, and every CSV refusal, before or while the table is read, is located on the input inlocation.input. - A run failure's problem names its pipe and path: the problem of a run that failed inside a pipe carries the failing pipe and the pipes leading to it as
location.pipeandlocation.path, and fills thepipeparameter of a code that declares one, while a failure of the entry pipe before any step carries no location. The location also keeps what the errors between the run and its root fault know, such as the field and the file a pipe met an unknown model at, and the diagnostics keep the model and the backend a worker attributed the failure to on an error above the root fault, an inner router's stand-in for a foreign exception included. A parameter a wrapper above the error whose code the problem reports gave, and its code declares, fills the one that error gave none of, and the items a wrapper gave follow that error's own. - A graph node's error is its problem (Breaking): a failed node of a
GraphSpec, and the trace event that ended its pipe, carry the problem in the caller's view inerror, located at the node's pipe and path, and the stack trace, when the graph's data inclusion asks for it, inerror_stack, capped at the same length on both, whereerrorhelderror_type,message,stackandcaller_facing. - Plugin API v9 (Breaking): an HTTP-error mapper registered with
add_http_error_mapper(exc_type_provider=..., to_pipelex_error=...)returns thePipelexErrora transport exception stands for, whereto_error_reportreturned a report, and a bundle validator'svalidate_bundlesreturns aBundleValidationFailure, the refusal's problem in the operator's view with its validation items, as its invalid verdict, and a host plugin's carrier of a problem recovered across a transport boundary overridesPipelexError.recovered_problem()to return it rather than overridingto_problem(), which a located run failure no longer reads as a carrier;PLUGIN_API_VERSIONmoves to 9, so a plugin declaringtargets_api = 8is refused at discovery withPluginApiVersionMismatchError. pipelex.test_extras.problem_parity(Breaking): the parity fixtures a distributed executor's tests share with the local runner's moved frompipelex.test_extras.error_report_parity, and their classes areProblemParityTestData,SearchProblemParityTestDataandJudgeProblemParityTestData.- A run failure reports its root fault's code (Breaking): the root fault a run failure's problem reports is the first error down its cause chain that declares a code of its own, and its code, title, type, domain and retry verdict are the problem's, with no
runtimefloor on the domain; a consumer that branched on the wrapper'serror_typebranches oncode. APipeFuncwhose function raised readspipe.function_raisedwhatever it raised, a coded error of a helper the function called included, which rides as the cause, while a failure of the executor running it, such as its timeout, keeps its own code, and an exception an executor lets through that is not a Pipelex error, such as an out-of-process executor's transport failure, readsruntime.unexpected. A failure below an exception that is not a Pipelex error lends the error above it no code, fact, location or advice. - The runner's own refusals are coded problem documents (Breaking): every error response of the API server, a 404 for a path no route serves, a 405 and a 413 included, is a problem document carrying a registry
code, and the API'sErrorTypeis gone with theerror_type,error_domain,retryableandinstancemembers, thetree_refusal,missing_blobsandmismatched_blobsmembers becoming the code's parameters or the operator's diagnostics. A field the runner refuses isrequest.invalidnaming it asparams.field, with one item inerrorsper refused field up to a bound, which the operator'sdiagnostics.messagekeeps too, saying how many more faults there were, a rule over several fields naming the one to change, such as the second of two selectors sent ormthds_sourcesbeside amethod_ref, a body or a part of one it cannot parserequest.malformedwith areason, a size limitrequest.too_largewith itslimitand thesizein bytes, an encoded value measured by the bytes it carries, and a method tree it cannot readmethod.package_invalidwith areason, the standard's tree refusal code or one of the runner's own, one per rule and one per limit, so an archive past its member count readsbundle_too_many_files, while a body refused beyond its schema on/v1/executeor/v1/start(a bundle sent withmthds_contents,method_refbeside a bundle,resourcesordependencieswithout their files, no run source, a hosted-only field such asmethod_id) answers aPipelexRequestErrorcarryingrequest.invalidorrequest.unsupported; the API'sProblemDocumentschema documentscode, andrequest_id, taken fromX-Request-IDor minted, is what correlates a response with the server's log. - The runner's refusals answer their code's status (Breaking): an orchestration-mode override the deployment forbids is a 422 where it was a 403, custom Python on a deployment that is not sandbox-hosted a 501 where it was a 403,
/v1/starton a blocking-only deployment a 501runtime.unsupportedwhere it was a 400, a registry-formmethod_refa 422request.unsupportedwhere it was a 501, invalid base64 and a client gone mid-upload a 422 where they were a 400, an archive past its file-count or decompressed-size ceiling and too manyresourcesa 422 where they were a 413, a blob past the per-file limit a 413 where it was a 422, a forwardedX-User-Idthat is not path-safe a 422request.invalidnaming the header where it was a 400, a body FastAPI could not parse, holding bytes that are not UTF-8 or an integer past Python's digit limit, a 422request.malformedwith the reasonnot_utf8orinteger_too_longwhere it was a 400, while the framework's own refusals, plain{"detail": …}bodies until now, keep their status under a code whose required parameters the runner can fill, a 400 asrequest.malformedwith the reasonunreadable, a 401 asauth.missing_credentialkeeping itsWWW-Authenticatechallenge, a 403 asauth.invalid_credential, a 413 asrequest.malformedwith the reasontoo_large, a 429 asruntime.unavailablekeeping itsRetry-After, and any other, such as a 415, asrequest.malformedwith the reasonunreadable, aread_scopeajwtcaller may not read a 403auth.forbiddenwhoserequirementisread_scope_under_callerwhere it was a 403ReadScopeForbidden, and a token whose claims the runner cannot use a 500 where it was a 401; a 401 carriesWWW-Authenticate: Beareronly where the deployment reads a bearer token (AUTH_MODE=api_keyorAUTH_MODE=jwt), so behind a trusted proxy a missingX-User-Idis a 401 asking for no token. - Damaged stored bytes are a damaged stored file, never retried: bytes the API server's blob store holds under a hash that do not hash to it, or are not the size it holds them at, answer a 500
pipelex.storage.damaged, a new code of the Pipelex error registry, naming the kind of storage,pluginfor a storage plugin's own, whose verdict is not to retry, since a retry reads the same damaged object, and which/v1/execute,/v1/start,/v1/validate,/v1/resolve,/v1/codegen,/v1/pipe-ioandPUT /v1/blobs/{sha256}declare in their OpenAPI responses; the server's line naming the object to remove,Stored blob bytes do not match their hash, carriesblob_sha256,blob_size_bytes,read_size_bytesandstorage_scheme. - The
/validateMarkdown presents the verdict's items (Breaking): the opt-inrendered_markdownof an invalid verdict is rendered from the problem items itserrorsholds, in the same view, each under its code's title with its code, location, sentence, suggestions and suggested fix, so a caller reads no wording its view of the items leaves out, where it rendered the validation items' own messages;pipelex.pipeline.validation_render.render_invalid_verdict_markdown(items=..., message=...)replacesrender_invalid_validation_markdown(report). - The diagnostic routes' verdicts carry problem items (Breaking): the invalid arm of
/v1/validate,/v1/resolve,/v1/codegenand/v1/pipe-iocarrieserrors, one problem item per finding with itscode,detail,paramsandlocationand a suggested fix naming its file asfile, where it carriedvalidation_errors, the valid arm'swarningsare problem items too, and a top-level member a verdict has no value for, such asdefault_pipe_reforgraph_spec, is absent rather thannull. The OpenAPI artifact no longer publishesValidationErrorItemor pipelex'sSuggestedFix, and names every schema reachable from the standard's problem item with aProblemprefix, such asProblemSuggestedFixandProblemSetKeyOp. - Pipe selection refusals carry the code of their case: an entry pipe the selector names but no closure declares reads
request.pipe_not_found, offering as itssuggestionsonly pipes the caller may select, the method's own for a bare or domain-qualified code, never a pipe of the server's own library directories, and for analias->…code the pipes that dependency exports, a bare selector matching pipes in several domainsrequest.pipe_ambiguous, a request with no selector over a closure declaring nomain_pipe, or several,request.pipe_unspecified, naming the declared ones as itscandidates, a manifestmain_pipethe closure lacksmethod.pipe_unresolved, naming themain_pipedeclaration as itspipeand the nearest declared pipes as itscandidates, and one it declares in several domainsmethod.ref_ambiguous, all still a 422, a dependency'salias->codematching pipes in several of its domains naming them ascandidatestoo; a run of a method on disk with no pipe named refuses several declaredmain_pipes and a manifestmain_pipeseveral bundles define the same way. PipeIOContractErroris the method's (Breaking): a structure that cannot be projected into its IO contract answers a 422 carryingmethod.concept_invalid, naming the concept and the finding, where it answered a 500.- Storage and remote-file failures say whose they are: an S3 or GCS refusal of the deployment's credentials (
AccessDenied, an invalid or expired key,Forbidden) carriespipelex.config.credential_refused, a bucket that does not existpipelex.config.invalidand missing credentialspipelex.config.credential_missing, apart from an outage,pipelex.storage.unavailable, naming the kind of storage; a remote file that times out, refuses the connection or answers a 5xx, 408, 425 or 429 isinput.url_unreachable, which a retry may cure, and any other failed fetchinput.file_unreadable. - Builtins raised where a reader sees them are coded classes: refusals that raised a bare
ValueError,RuntimeError,ImportErrororNotImplementedErrorraise a codedPipelexError, such asRunParameterErrorfor a run's metadata, extras or storage scope (still aValueError),ExportedPipeNotFoundErrorandExportedPipeAmbiguousErrorfor a CLI pipe selector,BundleClosureEmptyError,StorageNotConfiguredError,UrlSyntaxError,LocalFileMissingErrorandUnsupportedImageMimeTypeError, and the classes that were rooted outsidePipelexError(PipeValidationError,ConceptValueError,QualifiedRefError,SemVerError,MethodNotFoundErrorand the like) join it. - Inference failures carry the code their classification picks (Breaking): the classification of a provider's failure picks a code with its category, a
ServiceErrorCodea plugin contributes names itserror_code, and a category that repeats the call within the run must pick a code that invites a retry; an error built with a category alone reports the category's code (provider.unavailableforTRANSIENT,runtime.misconfiguredforCONFIGURATION,provider.quota_exhaustedforCAPACITY,provider.outcome_unknownforAMBIGUOUS). A provider 400 that carries no context-window or content-policy signal isruntime.unexpected, answered 500, where it was the caller's 422, and a local extractor'sOSErrorisruntime.unexpected, no longer repeated within the run. - One rule for a request id (Breaking): a run's
request_idholds ASCII letters, digits,_and-alone, at most 128 of them, the rule the API server applies to an inboundX-Request-ID, now one function,pipelex.system.job_metadata.is_valid_request_id(), where a run accepted any printable ASCII, a space, a quote or an=that could forge a field of a key=value log line or a parameter of a header included;RunParameterErrorno longer quotes the refused value. - A truncated completion says which limit cut it:
LLMCompletionTruncatedErrorcarriesmodel.output_truncatedwhen the method'smax_tokenscut the answer andmodel.context_exceededwhen the context window did, both the method's to change. ModelDeckPresetValidationErroris spelt right (Breaking): the error a model deck preset that fails validation raises wasModelDeckPresetValidatonError, and its reports carried thaterror_type; a consumer that branched on the old name branches on the new one, or on its code,pipelex.config.invalid.- Usage statistics carry the error's code (Breaking): a failed
command_completedorrun_completedcarries the code of its error inerror_codeand that code's domain inerror_domain, now one ofinput,method,account,operatorandruntimewhere it readinput,config,runtimeorother, and amodel_usedcounts its failed calls by the domain of their code, infailures_input,failures_method,failures_account,failures_operatorandfailures_runtime, where it counted them by the runtime's inference category, which no event carries any more. A failed hosted run'scommand_completedcarries the code and the domain the hosted API sent, and none when it sent none, and every surface reads a refusal a server answered by that rule, a run'srun_completedreading the catch-all where nothing was sent rather than the code the client infers from the status. Every event'sschemais 2, so a receiver tells these events from an earlier release's. pipelex-agentreports a failure as its problem (Breaking): the agent CLI's error stream is the failure's problem in the operator's view, Markdown by default and JSON with--error-format json: itscode,domain,retryverdict andnext_step, itsparams,locationanderrors, itsrequest_idandpipeline_run_id, anddiagnosticswith the error's class, the model and the provider of a failed call and a pending configuration migration, which ridesdiagnostics.migrationwhere it rodemigration, beside the command's ownmessageand members. The envelope'serror,error_type,error_domain,error_code,error_category,retryable,hint,validation_errors,error_source, and a failed run'spipe_code,pipe_stack,cause_typeandcause_message, are gone, and so areAGENT_ERROR_HINTS,AGENT_ERROR_DOMAINS,AGENT_INIT_FAILURE_HINT,run_failure_fields()andextract_validation_errors(): an agent branches oncode, an invalid verdict's items are the problem'serrors, a failed run is located bylocation.pipeandlocation.path, and the advice an error's chain gives is a Markdown tip, a boot that fails with no advice of its own tipping the recovery path throughdoctor,migrateandinit. A refusal the CLI makes itself carries its own code,client.argument_invalidfor an argument,client.file_unreadablefor a file it cannot read, the bundle file of every bundle command included,client.tool_missingwhenplxtis not installed, andpipelex.config.file_missingwheninitfinds a configuration file it should have written missing, and an exception that is no Pipelex error readsruntime.unexpected, an error of the system that names no file, such as a refused connection or a timeout, included. A bundle file that is not UTF-8 text does not parse, sorun bundleandfix bundlereport it asvalidate bundledoes, the invalid verdict with amethod.syntaxitem, whererun bundleread it as a file it could not read;fix bundlecodes the errors left asvalidate bundlecodes them, where an item no stage classified readmethod.schema_violation, and a fix it cannot write isclient.write_failed. The Markdown lists the problem's items one per line, each with its code, sentence, location, suggestions and suggested fix, an invalid verdict's included, where it rendered a verdict's items asvalidate's category-grouped prose. A run on a runner (--runner hosted) that fails reports the problem the runner answered or the client decided, as it came, by the members the standard and Pipelex declare alone, in the caller's view: the runner's problem document with itserrors,client.unreachablenaming the base URL for a runner that cannot be reached,client.outcome_unknownfor a server fault on the run request, andrun.failednaming the run for a run that started and failed, the client's own sentence asmessage, nesting the run's own problem asrun_errorin the caller's view, read off its stored report whatever its shape; an answer in the shape that predates the standard's reads as the core code its status and the hosted plane's own code name,auth.invalid_credential,auth.missing_credentialwhen no key was sent,auth.forbiddenfor an operation the platform refuses a key it accepted, under one of its authorization codes,request.not_found,run.not_foundormethod.not_found, oraccount.rate_limitedwith itsretry_after, a runner's refusal of a method that does not validate lists its validation items undermethod.invalid, and any other such answer keeps the client'sclient.response_unreadablewith the reason it gave; each takes the next steppipelex run --hostedgives, of the kind the answer's advice names when it gave one, and so does a failure the client meets on its own side; what the CLI refuses before sending it is a problem in the operator's view, and so is a failure of its own, under its own code, except a methodrun bundlerefuses while reading it, a bundle that does not parse or validate or a bundle file or folder with no pipe to start at, which reads in the caller's view as the runner's refusal of the same method would, with its code, items, locations and next step and neitherdiagnosticsnormessage.- The
pipelexerror panels print the problem: a panel names the problem's title, its detail, its next step, its code with its domain and retry verdict, its location and its request id beside the fields it printed, and a run that fails, or any failure a command root catches, prints its problem the same way on stderr, where it printed one line with the message. Every panel, an invalid bundle's, an unavailable model's or a telemetry file's included, and the stack trace--tracebackadds to it, now goes to stderr, where the panels and their stack traces went to stdout, so a command's piped output holds only what it produced. - A configuration refusal names its setting and its file:
pipelex.config.invalidraised for a configuration that does not validate, a telemetry configuration, a tracing sink or what a former release left in the inference configuration names the setting, its file and the configuration it belongs to in the operator's next step, a file the refusal can single out, a former release's or one that does not parse (pipelex.config.syntax, wherepipelex show confignamedpipelex.tomlwhichever layer failed), named relative to the working directory or as~/…under the home directory, never with the home directory's account, and its advice, the tip a CLI prints, follows the pending migration,pipelex migratewhen it would rewrite the files and a correction by hand when it would not, and else the refusal's code:pipelex initfor a missing file, setting the variable for one that is not set, andpipelex doctorfor a configuration that does not validate or parse. The refusal of what a former release left carries indiagnostics.migrationwhatpipelex migratewould do about it, from the cleanup's dry run, so a JSON reader, who gets no tip, reads the remedy too. - A pipe's failure line carries its code, not its text:
The execution of a pipe failedcarries the problem'serror.codebesideerror.typeand no longererror.message, so a log query selects failed pipes by code, and the line holds neither the caller's own text, a function's message or a template's source, nor a refused host. Jinja2TemplateSecurityErroris aSecurityError(Breaking): the template sandbox's refusal at render time now derives fromSecurityError, likeUriReadRefusedError,SsrfBlockedErrorandUnsafeSchemaError, instead ofToolError, so a host catchingSecurityErrorcatches it too and anexcept ToolErrorno longer does; it stays a type of its own, apart fromJinja2TemplateRenderError, and its class name, which the operator reads asdiagnostics.error_class, its code,security.template_refusedon a method's template, its title and its classification are unchanged.- A security refusal is never a validation verdict (Breaking): a refusal a raise site coded in the
security.family is let through by the bundle-validation cascade whatever its class, as aSecurityErroralways was, so aPipeDocGenstep whosetemplate_fileclimbs above its method tree's root failsvalidateand a run withsecurity.method_path_refusedand logs its security event, where it was amethod.invalidverdict naming the pipe; a refusal a validation's dry run meets, such as a template the sandbox refuses at render, fails it the same way under its own code, every failure riding its items and the event logged once, where it was an item of an invalid verdict. mthds0.22.1 andpipelex-sdk0.46.0 (Breaking): the exact pins move frommthds==0.22.0andpipelex-sdk==0.45.0tomthds==0.22.1, the release whose errors carry the MTHDS problem document read by its registry code, andpipelex-sdk==0.46.0, the release built on it; everyone downstream inherits those exact versions.- A problem carries every parameter its code requires: every problem and item the runtime and the API server build gives each parameter its code's registry entry requires, so its sentences state the occurrence rather than the code's generic wording: a model no catalog knows offers its
suggestions, an empty list when none is near, an invalid identifier names itsidentifier_kindandname, a name declared twice, or a native concept declared again, names thenameand theconflictin one item per bundle, each located on its own file and on the colliding declaration's path, such asconcept.Rating, in a validation verdict and a run's refusal alike, while a batch item named as its list stays on its pipe and the field it collides on,input_item_nameor a step'ssteps.0.batch_as, an unresolved concept and a selected pipe the method lacks offer the nearest declared ones, a pipe reference naming nothing the pipe whose body writes it and the nearest pipes it may name, of a dependency only those it exports unless the reference is written in that dependency, and beside a server's own library directories only the pipes of the bundles the request sent, a template the sandbox refuses names what it reached for asnamewhen its own source spells it, the format string a.formatcall formats included but no other string literal, and is located on its pipe and the field the template is written in, while a name the run's data chose is withheld and its sentences say the template looked a name up from the run's data, a validation item gives the facts its check knows (the concept a pipe declares and those that would fit, the multiplicity it declares and the one its use expects, the site where an optional value escapes, the path a binding cannot follow), andmethod.package_invalidalways gives itsreason,tree_unreadablefor a tree that cannot be listed among the new ones. A failure met while running names its facts too: a stored file gone and a storage outage name the kind of storage asscheme, which a storage provider's newstorage_kindgives,pluginunless a provider says otherwise, a pipe added twice to a library, a duplicate branch output and a module the tree shadows name thenameand theconflict, a file a model cannot read itsformatandmodel, a remote file that cannot be fetched itshostor itsreason, a completion the context window cut, and a prompt a provider refuses as over the context window, itslimit, a Gemini image a filter refused itsmodel, a model a provider does not know or refuses and a waterfall with nothing served themodel_typeandreasonofmodel.unavailable, aPipeFuncpast its deadline itsfunctionandtimeout, aPipeJudgequestion that reads a file the step'sinputand the concept itdeclared, a requirement whose extra needs a missing or mismatched distribution that distribution withnot_foundorno_matching_version, a call to a pipe another domain of the package does not export thepipeand thepackage, one item per call located on the bundle, the pipe and the field that write it, and bundles declaring a reserved domain, which a package'sMETHODS.tomlrefuses asmethod.name_conflicttoo, or two declarations of one name whose bundles are unknown, one item each, on its own file.ValidationErrorItemandPipesAndConceptValidationErrorDatacarry those facts inparams, and the problem builders logA problem lacks a parameter its code requireswhen a raise site gives fewer, except for a parameter withheld by rule, the refused name a template looked up from the run's data. - A refusal of one fault carries no
errorslist (Breaking): a problem whose only item restated its own code, such as/v1/models/check's refusal of a missingreferenceor a template the sandbox refused while validating, carries that item's parameters and location itself and noerrors; a refusal whose single item has another code, such asmethod.invalidover one fault, keeps its list, and one whose single item carries suggestions or a suggested fix takes that item's parameters and location and keeps the item listed, so its fix reaches the caller. - A model no enabled backend serves is
model.unavailable(Breaking): a method naming a model a catalog of the deployment knows but no backend it enables serves, such asclaude-4.5-haikuwithanthropicswitched off, answers a 501model.unavailablewith the reasonnot_served, located on the pipe and itsmodelfield, when it is validated and when it runs, where it was the method'smodel.unknownverdict; a switched-off backend's model file that is missing, is not UTF-8 text or does not parse declares nothing and never stops the boot.
Fixed
- A model deck's own unresolved default or override is the deployment's fault: a deck default or override naming a model the deck does not define raises
ModelDeckChoiceUnresolvedError, naming the deck setting and carryingmodel.unavailable, with the reasonnot_served, for a bare handle no backend serves, answered 501, andpipelex.config.invalidfor a preset, an alias or a waterfall the deck does not define, answered 500, with the deck setting in the operator's diagnostics alone, where it read as the caller's unknown model, and aPipeJudgenaming no model, whose default is resolved as its method loads, raises it rather than anunknown_modelverdict item; a bad override is refused when the deck is built, and a handle the deck serves only as another type of model ismodel.capability_unsupported. - A host library that refuses to load is the deployment's fault: a method of a server's own library directories that refuses to load beside a caller's bundles raises
HostLibraryLoadRefusalError, carryingpipelex.config.invalid, which an API answers with a 500 whose message reaches the operator's diagnostics alone, where it answered the caller a 422 verdict carrying the host library's text; the same refusal in the caller's own content, or in library directories the caller owns, stays the caller's 422 verdict. - A template error found only at render is a located refusal: a template naming a filter or a test that does not exist, which Jinja2 only finds when the branch renders, fails the run at the pipe with
Jinja2TemplateInvalidError, naming the unknown filter or test and none of the template, codedmethod.template_invalid, a 422, when the template is the method's, in aPipeConditionexpression, aPipeLLMprompt or system prompt and aPipeDocGentemplate, andruntime.unexpectedwhen it is one of Pipelex's own, such as the deployment's structure prompt rendered in aPipeLLMstep, where it failed as an unclassified 500. A filter refusing its argument, which may be a value of the run's data, ispipe.template_render_failedrather than the template's fault. - A failure's message is bounded: the error's message a problem carries in
diagnostics.message, a provider's answer and every text parameter are cut at 2,000 characters, ending in[…], so an oversized exception cannot swell a response, a stored run or a log line. - A dry run failing on a fault of the runtime's is no verdict: validating a method whose dry run fails only on faults that are not the method's, such as a storage outage or an unexpected error, answers the first of those faults' codes, every failure listed in
errors, where it answeredmethod.invalidand blamed the method; a verdict needs one failure in the method's domain, and the first fault is chosen with anoretry verdict before a retryable one, then the operator's before the account's, the input's and the runtime's, the answer's sentence rendered from the failure whose code it takes. A mock value a routed pipe cannot build, a pydantic or polyfactory refusal, stays the method'smethod.mock_unsatisfiable. A refusal met while loading that is not the method's keeps its own class and code in the same way, sopipelex run bundleandpipelex-agent run bundleon a folder whose bundles declare no singlemain_pipeprint the run's refusal,request.pipe_unspecifiedorrequest.pipe_ambiguous, where they printed an invalid verdict. A hosted run whose inputs name a local file tellsPOST /v1/pipe-io's verdict on a method calling a package by its address that the request did not ship apart from a fault of the method by the verdict'smethod.dependency_missingitem of the kindpackage, naming the package itsparams.namegives, where it matched the words of the item's message. - A provider's 408 is retried: a provider answering HTTP 408, that it gave up waiting, is a transient failure carrying
provider.timeout, so the call is repeated within the run, a billed image call included, where it was read as the caller's input and the run failed on its first attempt. - A
PipeFuncreturning a list of anything but stuff contents fails its output: a function returning a list that holds a value other than a stuff content, such as a dict or a number, fails the step withFuncOutputTypeError(pipe.output_invalid), where a dict became an empty content and any other value failed as though the function had raised. - A CLI command's unexpected failure logs no run data: the root of
pipelex runand ofpipelex graph renderlogs an unexpected failure with the code of its problem, its class, the classes along its cause chain and the frames it went through, each by its module, line and function with no source line or local (error.frames), where it attached the exception, whose traceback printed the text of every chained exception, a foreignValidationErrorquoting the values it refused or a method function's exception included; the traceback rides the record only under--traceback, and the terminal prints the failure's message capped. - A prompt over the model's context window reads
model.context_exceeded: a provider 400, or another 4xx such as the 422 or 413 a gateway may answer, saying the inputs are over what the model can read, by OpenAI'scontext_length_exceededcode or by the wording OpenAI, Anthropic, Mistral, Bedrock and Gemini give it, in the error's message or its body, is the caller'smodel.context_exceeded, answered 422, where it readruntime.unexpected, orinput.too_largeon a 413. A language model's window holds the prompt and the output requested withmax_tokenstogether, and a 400 themax_tokenssent caused reads the same, so its next step is to shorten the input or lowermax_tokens, or choose a model with a larger context window; an image, extraction, search or judgment model's next step names what it reads instead. A 400 about a malformed request still readsruntime.unexpected. - A structured output cut at
max_tokensreadsmodel.output_truncated: the Anthropic, OpenAI Chat Completions and Mistral workers turn instructor'sIncompleteOutputException, raised for a structured answer cut at its output limit, intoLLMCompletionTruncatedError, naming a structured output, the stop value, the output tokens and themax_tokenssent, where it escaped unclassified asruntime.unexpected.LLMCompletionTruncatedErrortakes anoutput_kind, a text or a structured output. When the Anthropic structured-output timeout lowered the limit that cut it, the error says so and advises a lower reasoning effort or a smaller output, never a highermax_tokens. - A Gemini image a safety filter blocked reads
model.content_refused: the Google image worker reads the answer's finish reason, or the prompt's block reason when no candidate came back, and raisesImgGenGenerationErrorwithmodel.content_refused, answered 422 with a next step to revise the prompt or its images, where anIMAGE_SAFETYorPROHIBITED_CONTENTanswer read as an empty one,provider.response_invalid. The message names the reason Gemini gave and what it says, so an image stopped for recitation or for an unsupported language reads the same code without being called a safety block. An answer the model's output token limit stopped before it held an image (MAX_TOKENS) raises the newImgGenOutputTruncatedError,model.output_truncated, answered 422 with a next step to simplify the prompt or choose another image model, where it readprovider.response_invalid. - No error message carries a run's data: an error's message, which reaches the operator's diagnostics, the error tracker and the log lines, no longer quotes an input's value, a stuff's content, a model's answer or a value a pipe built: a refused input is described by its kind and size, a list item by its index and type, a URL by its scheme when it is one Pipelex reads, a date or a time by the form it expected, a pydantic refusal of a run's data, a provider's metadata included, by its locations and reasons, a stuff by its code and concept, and a file of unknown type by the number of bytes read. A
PipeFuncwhose function raised names the class of what it raised and never its text, as does any other failure of its executor, by its class and its code; a template that fails to render is described by the class Jinja2 raised and the template's line, never by Jinja2's text, which quotes a key of the data the template reached for, nor by its source, a name it reached for, the sandbox's refusals and the render budget's refusal of a call included, being named only when the parsed template spells it, a syntax error by its kind and line rather than the token Jinja2 quotes, and Pipelex's filters never quoting the argument they refuse; a structured generationinstructorgave up on is described by the refused answer's reasons, located by the class it asked for, by the class of any other failure to parse the answer, a model's refusal included, or by the class of what ended its loop; an exception that is not a Pipelex error stands in a problem's diagnostics as its class alone, a run's and the API server's catch-all's alike; and no message names a refused or unreachable host, whichinput.url_unreachabletakes as itshostparameter, alongside the input the URL came from, which an extract worker, a prompt's image or document and a document's page views name, while a provider's file that could not be fetched is given no parameter its code does not declare. A failure following a pipe's own, a delivery's or the graph tracer's, is logged by its class and code, a webhook's transport failure is named by its class, a trace event log's refusal and the tracing assembly's errors are written by their reasons, and no sink writesinstructor's log records, which quote the model's answers. A location never quotes a key of the value's own: a key of a dict, or one the model does not declare, is written<key>(entries.<key>.score), only what the validated type's schema names is written as it is, and a refusal whose model is not known is written by its reasons alone; every site that validates an input into its concept's class, an explicit envelope and a transported stuff included, locates its refusal by that class, an explicit envelope's naming the input and the field asinput.invalid's parameters, and a refusal of a method author's own document is located as pydantic gives it (is_authored_input). A CSV input's refusal names the file by the input it was given as, never by its path, and quotes no cell nor a column the concept does not declare; its problem takes the input as itsinputparameter. - Pre-release documentation is published: a push to a
pre-release/branch deploys its documentation under thepre-releasealias of docs.pipelex.com, where the deploy workflow called a make target with no recipe and published nothing.
Removed
- The per-class error pages (Breaking): the generated page every
PipelexErrorclass had underdocs/errors/,pipelex-dev generate-error-pages(make gep) andpipelex.errors.error_pages_generatorare gone; a problem'stypeis now its code's page, and docs.pipelex.com redirects each retired page to the page of its class's code, or to the Error Codes index when the class is gone or raises no single code. - Class-level error classification (Breaking): an error class no longer declares
error_domainorretryable,PipelexError.title()andPipelexError.type_uri()are gone, and so isInferenceErrorCategory.error_domain: the class declares itserror_code, which decides all four, and an inference failure's category, which stays inside the runtime, picks the code its worker raises.as_caller_fault()is gone too, since a caller reads the sentences a code renders and never an error's own message.
Security
- Callback URLs are held to the runtime's SSRF rule: the runner refuses a
/v1/startcallback URL with the rule the webhook delivery applies when it fires,pipelex.tools.network.host_rules, so a cloud metadata name spelled in capitals or with a trailing dot, and an address that is not publicly routable such as a carrier-grade NAT one, are refused with a 422security.input_url_refusedbefore the run, where the runner's own copy of the rule let them through to the delivery; a host of dots alone, such ashttp://./hook, names no host, and a host and port written with no scheme, such aslocalhost:3000/cb, opens no authority, so each is a 422request.invalidnamingcallback_urls, with no security event, while a scheme-relative URL whose host the rule refuses, such as//169.254.169.254/latest/meta-data, is the rule's refusal, classed by its host. - The API server's error line quotes no exception (Breaking): every error line carries
error.codeand names the classes of the failure's exception chain inerror.chain, and an error-level line addserror.frames, where the deepest exception was raised, in place of the traceback underexception, so no exception's text, which a foreign exception can fill with a run's inputs or a model's answer, reaches the log, and an exception the catch-all answered no longer reaches the server, whose log would print its traceback; thedetailfield is gone too. The error's message rides the operator view'sdiagnosticsalone, which only a request the operator authenticated reads (AUTH_MODE=api_key, or a trusted proxy's forwarded identity), so every line names the problem's operator-only parameters aserror.params.<name>, such as thecomponentof aruntime.misconfigured, whichever view the response took. A refusal the runner answers in place of a framework exception, FastAPI'sRequestValidationError, Starlette'sClientDisconnector anHTTPException, names that exception's class after its own inerror.chain, and a failure no handler recorded that ended a request after its response had started, the catch-all's own failure while a response streamed, is written asA request failed after its response had started, naming its path, the status already sent and its classes alone, where it was dropped unrecorded; so is a failure to send the answer the server built for a failure no handler answered, such as to a client already gone. A line's level says who has to act, from its status and its code's domain: a 4xx is a warning, a 5xx an error with frames only when its domain is the runtime's or the operator's, a 501, what the deployment does not offer, such as custom Python without a sandbox or/v1/starton a blocking-only orchestrator, a warning without frames, where every 5xx was an error, and a 404 or 405 for a request no route serves a debug line, where it was a warning.
[v0.89.0] - 2026-10-17
Added
resolve_crate_from_method_dir:pipelex.pipeline.resolve_bundleresolves a method's tree a host wrote out to a directory into the normalized crate, loading it as the caller's library directory the wayvalidate_bundleloads amethod_dir, withmethods_dirsholding the packages shipped with it, under the sameValidateBundleErrorverdict and loaded-on-success contract asresolve_crate_from_contents. The resolution is static: a package the method calls by its address is loaded frommethods_dirsalone, never from the methods installed on the host or by a fetch, and one they do not hold is anunresolved_package_dependencyitem of the verdict.address_dependency_sources:LibraryManager.load_librariesandload_from_blueprintstake anAddressDependencySources(pipelex.methods.address_dependencies),ANYWHEREby default, which looks for a package called by its address among the load'smethods_dirs, then the installed methods, then fetches it when fetch-on-miss is on, orSHIPPED, which looks only inmethods_dirsand reports a package they do not hold as anunresolved_package_dependencyitem raised from the newMethodDependencyNotShippedError.
Changed
- Every command taking a method's folder reads it as its method tree (Breaking):
validate bundleandfix bundleon both CLIs judge every bundle of the folder's tree, as the MTHDS protocol's validation does, instead of abundle.mthdsor the folder's only.mthdsfile, so a folder whose bundles declare severalmain_pipes, or none, is validated and fixed like any other, one holding no bundle exits with code2underEmptyMethodTreeError, one whose tree cannot be read is their invalid verdict naming the folder (FixBundleErroronpipelex-agent fix bundle), and a fix writes anywhere in the tree from its root, the--diffpreview proposing what the real run writes.pipelex-agent inputs bundle,pipelex build inputs bundleandpipelex build output bundleact, with no--pipe, on the piperun bundlestarts at, theMETHODS.toml'smain_pipeor, with no manifest, the onemain_pipeits bundles declare, loaded with the rest of the method's folder, and refuse a folder that decides none with theMethodEntryErrorrun bundlegives.pipelex-agent validate bundle <folder> --graphdraws that same pipe, or the one--pipenames, and when the folder decides none it says why ingraph_unavailableinstead of failing the validation; on every command reading a folder, a configuration that does not validate, read before the boot for the tree's limits, is reported as the failed boot it would be. - Validating a method's tree judges only its own pipes (Breaking):
validate_bundle(method_dir=...), and soPOST /v1/validateon a method's files andvalidate bundle <folder>, dry-runs and reports only the bundles of the method's own tree; a nested package below it with its ownMETHODS.toml, the packages it depends on and the-Ldirectories still load for its bundles to use, but their pipes are no longer judged with it. /v1/resolve,/v1/codegenand/v1/pipe-iotake the method tree (Breaking): the three routes take the method as/v1/validateand the run routes do,filesbeing the MTHDS protocol's map of file entries withdependencies,resourcesandread_scopebeside it, or amethod_refwhose package's whole tree is loaded with the packages it ships, so a call into another package by its address resolves when that package was shipped, while one nothing shipped is the invalid verdict, never looked up among the runner's installed methods or fetched, and aMETHODS.tomlat the root of a sent tree names the default pipe of/v1/pipe-io, which reads it only for a request naming no pipe. Afileslist of{content, source}items is refused with a422InvalidMethodFiles, the tree's refusals and the custom-Python403s are those of/v1/validate,include_filesechoes the method's.mthdsbundles as a map from tree path to text, and the request modelsMthdsFilesRequest,MthdsFileItemandMthdsPipeRequestare removed. The routes load the method in a worker thread, so a slow load never holds up the server's other requests.- A bundle validator receives the read scope, plugin API version 8 (Breaking):
BundleValidatorProtocol.validate_bundlestakesread_scope, the scope the host named for the validation,Nonewhen it named none, so a validator dispatching the validation to another process hands the method's trees to its worker through that scope's blob store and the worker reads them by digest under it, as a run's worker reads under the run's read scope. The API server'sPOST /v1/validatehands its validator the request'sread_scopeon every arm, never the caller's own id, andPLUGIN_API_VERSIONmoves to 8, so a plugin declaringtargets_api = 7is refused at discovery withPluginApiVersionMismatchError. - A PipeFunc package-offer factory receives the method's requirements, under plugin API version 8 (Breaking): a factory registered with
PluginRegistrar.add_pipe_func_package_offeris called with aPipeFuncPackageOfferRequest, thepipe_funcconfig and the requirements of the one method tree being judged, where it received the config alone, so a sandbox mode building an environment for each method states the environment it builds for them, or answers aPipeFuncPackageRefusalwhen none can hold them, which refuses the method with aMethodRequirementsNotMetErrorcarrying the plugin's reason. Each tree judged in one check gets the offer stated for its own requirements, and the factory stays synchronous, so a plugin whose answer takes time caches it by the requirements. The requirements arrive with their markers unevaluated, and the newrequirements_needed_in(pipelex.libraries.method_requirements) picks out those whose markers hold where the mode would run the method, the only ones a refusal may rest on;parse_requirements_text,RequirementLineandREQUIREMENTS_FILE_NAMEmove topipelex.core.method_trees.requirements_file.
Fixed
- A
main_pipethat is not a pipe code is an invalid bundle: a bundle declaring itsmain_pipeas another type, such as a list, is refused with a validation error located onmain_pipeinstead of crashing the parse with aTypeError, and a folder's entry rule refuses amain_pipethat is no pipe code, such asdomain.pipe, a camelCase name or an empty string, with the same located error instead of reading an empty one as declaring none. - Load errors name a written tree's bundles by their tree paths: an error raised once the bundles of a tree the server wrote out have parsed, such as a concept declared in two bundles, names them by their tree paths instead of the directory they were written to, so a verdict of
/v1/validate, a run or a crate route no longer discloses a path on the server.
Security
- An auto-detected inputs file must be a regular file (Breaking):
pipelex run bundle <folder>andpipelex-agent run bundle <folder>read the folder'sinputs.jsonorinputs.tomlonly when it is a regular file, opened without following a link and judged on the descriptor its bytes are read from, and refuse a symbolic link, a directory or any other entry of either name with the newAutoDetectedInputsFileError, an input error pointing to--inputs; they used to follow a link the method's tree keeps out, such as one its.gitignoreignores, so a method you checked out could point its inputs at data outside its folder. An inputs file named with--inputsis read as given, a link followed. - A
jwtcaller'sread_scopeis bound to its own id (Breaking): on a deployment withAUTH_MODE=jwt, every route taking aread_scope(the run routes,/v1/validate,/v1/resolve,/v1/codegen,/v1/pipe-ioandPOST /v1/blobs/missing) refuses a scope that does not lie under the caller's tokenuser_idwith a403 ReadScopeForbidden, so a caller holding its own token can no longer read another caller's blobs or run under another's scope. WithAUTH_MODE=noneorapi_key, the caller is a trusted host or a single tenant, and the scope is taken as before.
[v0.88.0] - 2026-10-16
Added
crate_fingerprinton the run answers: the API server'sPOST /v1/startacknowledgement andPOST /v1/executeresult,PipelexRunResultStartandPipelexRunResultExecute, carry the fingerprint of the library crate the run's method was loaded into, so a host records it on its run beside the digest of the method tree that ran; it isnullfor a run built from no crate.StorageProviderAbstract.store_if_absent: a storage provider creates an object only when its key holds none and answers whether it did, never replacing an object already there, and checks the bytes againstchecksum_sha256_hexbefore writing, raising the newStorageChecksumMismatchErroron a mismatch. S3 makes the condition withIf-None-Match: *, sends the SHA-256 asx-amz-checksum-sha256for S3 to check, and retries a409 ConditionalRequestConflicta bounded number of times, GCS uploads on generation 0, and the local provider hard-links a temporary file to the key; a provider that does not override_store_if_absent, such as an external storage plugin, looks the key up and then stores.
Changed
- The API server's blob store writes create-only (Breaking): every blob and tree listing the server writes is created only when its key is free, with its SHA-256 as its checksum, so a bucket refusing unconditional writes under
blobs/andtrees/refuses none of them. A blob held at another size than the bytes uploaded under its hash is no longer written over: the upload is a500BlobStoreCorruptedthat leaves the object for the deployment's operator to remove, and the local storage's folder must sit on a filesystem that supports hard links.
Fixed
POST /v1/validatewithout aread_scopestores nothing: a validation of afilesmap records its tree in the blob store only under theread_scopeits request carries, where it used to record the tree's blobs and listing under the caller's own id, a prefix no host lists or erases; hash entries still resolve in the caller's own scope.
[v0.87.0] - 2026-10-16
Highlights
A method is now its whole folder, its method tree, and every file of it travels with it. Pipelex lists each method as the MTHDS standard's method tree, so its templates, images, stylesheets, fonts, Python and requirements.txt reach its steps wherever it runs: locally, on the API server through the files map and the blob routes, or fetched by method_ref, and every run records the digest of the tree it read. A PipeFunc now names its function by import path, a method's requirements.txt is checked before any of its Python runs, and a relative URL in an HTML template names a file of the method.
Added
[interpreter.method_tree_limits]: the limits every method tree is listed under on this machine,max_files(10,000) andmax_total_bytes(1 GiB) by default withmax_file_bytesunset, so a folder handed over by mistake is refused before any file is hashed, with the advice to run the method's own folder or to raise the limits; these are not the hosted API's limits, which a deployment sets.- A run records its method tree:
pipelex run bundleprints the digest of the method tree it read with its recap and writesmethod_tree.json(the digest, the tree's root, its entry and each file's path, hash and size) in its output folder,pipelex-agent run bundleaddsmethod_tree_digestto its output file and to its--with-memoryoutput, and every run'sRunMetadatacarriesmethod_tree_digest, the digest of the tree its pipe belongs to, which changes with any byte of the method, besidecrate_fingerprint, which changes only with the meaning of its concepts and pipes. MethodTreeLoadError,MethodEntryErrorandHostLibraryTreeError: theerror_typeof a method tree that cannot be listed (a symbolic link, a tree past its limits), of a method folder with no single pipe to start at, and of a server's own library directory whose tree cannot be listed beside the bundles a caller submitted, a configuration fault whose message, naming the server's paths, never reaches the caller.PluginRegistrar.add_pipe_func_package_offer: the plugin providing an out-of-process PipeFunc execution mode states, keyed by the same mode, the Python packages and the marker environment its sandbox offers (PipeFuncPackageOffer), which a method'srequirements.txtis checked against; a mode stating none refuses every method declaring requirements withPipeFuncPackageSetUnstatedError, a configuration fault.MethodRequirementsFileError,MethodRequirementsNotMetError,HostLibraryRequirementsError,PipeFuncPackageSetUnstatedErrorandDuplicatePipeFuncPackageOfferErrorare newerror_types.pipelex fix function-namesandpipelex-agent fix function-names: rewrite every flatPipeFuncfunction_nameof a method folder as the function's import path and drop the argument of every@pipe_func(name=...)marker, reading the files as syntax trees without importing them or booting Pipelex, and keeping every comment and alignment; a flat name no module of the method's tree registers, or that two register, is left as it is and reported with the markers registering it, which keep their argument, and so is a marker whose name is not a string literal, the agent command'sis_completebeing false and both commands exiting1, with--dry-runto preview.pipe_func_function_refusedand the method-tree Python errors: aPipeFuncwhose function cannot be found or proven is refused at validation with thepipe_func_function_refusedvalidation error type, located on the step'sfunction_name, and the newPipeFuncFunctionError,TreePythonError,TreePythonHeldError,TreeModuleShadowingErrorandFunctionNamesMigrationErrorname the failures of a function, of a tree's Python and of the migration.- The method file store: a step reads a file of its method where reading is allowed, at the moment it uses it, through
MethodFileStoreProtocol, whoseread_tree_listing(tree_digest=..., read_scope=...)andread_blob(sha256=..., read_scope=...)return a tree's listing and a file's bytes under the run's read scope, each read checked against the tree's digest (pipelex.core.method_trees.method_files). The local store,LOCAL_METHOD_FILES, serves the trees this process's libraries hold, and a plugin whose activities run apart from the process that loaded the method claimsHubSlot.METHOD_FILE_STOREwithPluginRegistrar.claim_method_file_store; a file the store does not hold for the run fails its step with the newMethodFileUnavailableError. An orchestrator running a job out of process copies every tree the job's crate names into the store its workers read before itsexecuteorstartreturns, since the runner may drop the files as soon as it does, andread_stored_trees(tree_digests=..., read_scope=...)reads them for that copy, each as a store keeps it, its listing under its digest and its files under their hashes. The local store finds a file by its hash in one lookup and reads a file on disk off the event loop. - Relative URLs in HTML templates read the method's own files: a relative URL in a
PipeDocGenHTML template, inline or a template file, such as an image'ssrc, a stylesheet'shref, a font, or aurl()inside a stylesheet the template links, names a file of the template's method tree, resolved by the MTHDS standard's rules against the template file, or against the bundle file of an inline template, and read through the method file store under the run's read scope wherever the document prints, so a logo, a stylesheet and a font travel with their method. Only images, stylesheets and fonts are read, by suffix, so a URL a run's input supplies cannot make a document carry the method's other files. A URL the template writes as it is is checked when the method loads, and a URL built from the inputs that climbs above the tree's root, names a folder, a missing file, a file the tree's membership rule keeps out or a file that is no asset fails the print with the newTemplateUrlRefusedError, an input error naming the URL and the template; a URL with a scheme, such ashttps://,data:orpipelex-storage://, is read as before. RenderJob.base_urland the method-file arm ofRenderResources.load: an HTML job whose template belongs to a method tree carries the URL a document engine joins the document's relative URLs to, underMETHOD_FILE_URL_PREFIX(https://method-files.pipelex.invalid/, which never resolves), andRenderResources.loadreads a URL joined under it, a stylesheet's own URLs included, from the document's method tree, with no media type;DocumentComposition.url_basecarries the template's place in its tree to the print stage. They join plugin API version 7, and an engine that never readsbase_urlkeeps refusing a relative URL.StorageProviderAbstract.load_size: a storage provider reads an object's size without its bytes, from a file's metadata, an S3HEADor a GCS object's metadata, and a provider that does not override_load_sizemeasures the object it loads, so an external storage plugin keeps working.- The API server's blob routes,
POST /v1/blobs/missingandPUT /v1/blobs/{sha256}: the server keeps a blob store in its configured storage, under<scope>/blobs/<sha256>, declares it withblob_store: trueonGET /v1/version, and serves the MTHDS protocol's blob routes, so a sender asks which of its method's files the runner lacks, uploads each one to the signed target the answer names, and then sends the method by hash; the scope is the request'sread_scope, else the caller's id, the targets start withPUBLIC_BASE_URLwhen it is set, else with the URL the request arrived at, are signed withBLOB_UPLOAD_SIGNING_KEY(a key per process when unset) and expire afterBLOB_UPLOAD_URL_TTL_SECONDS, and once the server has read a tree in full it records its listing under<scope>/trees/<digest>, so asking with that digest answers in one lookup for blobs of that tree. A blob counts as held only at the size the sender states, read from the storage's metadata, so bytes held short are asked for again and written over by their upload.BlobMismatch,BlobUploadNotAuthorizedandBlobStoreCorruptedare newerror_types. filestakes every entry form, besidedependencies: on/v1/execute,/v1/startand/v1/validate, each value offilesis the file's text,{"base64": "…"}or{"sha256": "…", "size": n}naming bytes the blob store holds, so a method's templates and other binary files travel with it, anddependenciescarries the method packages it calls by address, each a map of its own keyed by its address;/v1/validatealso takesread_scope. A hash the server does not hold is a422MissingBlobslisting the hashes inmissing_blobs, a hash it holds at another size than the entry states a422BlobMismatchlisting each with its held size inmismatched_blobs, checked before any byte is read, and a tree overMAX_METHOD_TREE_FILES,MAX_METHOD_TREE_FILE_KIBorMAX_METHOD_TREE_TOTAL_KIB, the filesresourcesjoins to it included, or a method with more thanMAX_METHOD_DEPENDENCIESdependency trees or more thanMAX_METHOD_REQUEST_TOTAL_KIBacross its trees, a422MethodTreeTooLarge.method_provenance.tree_digest: a run bymethod_refreports the digest of the method tree it loaded beside the commit it was read from (MethodProvenance.tree_digest).- Validating a method directory:
validate_bundle,validate_bundles_in_processandBundleValidatorProtocol.validate_bundlestakemethod_dir, a method's tree validated whole as the caller's, withmethods_dirsholding the packages shipped with it, every.pyfile of the tree compiled for its syntax and refused by its tree path and line, and the graph drawn from the tree's entry pipe;write_method_treeandWRITTEN_METHOD_TREES(pipelex.libraries.written_trees) write a tree out and make every listing of its directory, while it is held, the tree as written.
Changed
- Library directories load through their method trees (Breaking): a library directory's
.mthdsfiles are the bundles of its method tree and of the trees of the nested packages and repositories under it, each loaded as a method of its own, so a hidden, gitignored or virtual-environment.mthdsfile is no longer loaded, a symbolic link in the tree refuses the load, and[interpreter.scan] excluded_dirsnow applies only to the scan for Python classes and functions; a hosted run sends the same bundles. A directory holding nothing but nested methods loads them, a nested package or repository holding no.mthdsfile is skipped before it is listed, and a dependency package whose tree cannot be listed is refused naming the package by its address, never the directory it was installed into. -L <subdir>inside a package loads the whole package (Breaking): a library directory is listed as its method tree, which is rooted at the nearestMETHODS.tomlat or above it, so naming a subfolder of a package with-L, or as a library directory of a runner, loads every bundle of the package.- A
.mthdsfile named alone belongs to its method tree (Breaking): a bundle file handed torun bundle,validate bundleor a runner is the member of the tree rooted at the nearestMETHODS.tomlat or above its folder, and with none its method is that one file, so a step naming any other file, such as atemplate_file, is refused with the advice to run the method's folder or to add aMETHODS.toml. template_fileresolves in the bundle's method tree (Breaking): a PipeDocGentemplate_fileread from disk is looked up among the files of its bundle's method tree, where a..climbing to another folder of the tree is now allowed, and a file the tree does not hold (above its root, gitignored, hidden or in a nested package) is refused, the message naming each file by its path in the tree; a URL is refused when the bundle parses, and a bundle loaded from a string still reads only the files sent with it.run bundle <folder>starts at the method's main pipe (Breaking): with no--pipe,pipelex run bundleandpipelex-agent run bundleon a folder, locally as on the hosted API, start at itsMETHODS.toml'smain_pipe, or with no manifest at the onemain_pipeits bundles declare, and refuse none or several, naming each bundle by its path in the tree; abundle.mthdsor a single.mthdsfile in the folder is no longer looked for. A hosted run names that pipe asdomain.pipe_code, so it starts where the local run would when two domains of the method define the same code.- Run output folders hold a
.gitignore: every folderpipelex runcreates for a run's outputs holds a.gitignoreignoring everything in it, so a run started inside a method's folder leaves the method's tree, and its digest, unchanged; the folder is named after the pipe's own code even when the pipe was named asdomain.pipe_code, on a hosted run too. pipelex-agent runwrites into an output folder (Breaking): its output file (live_run.jsonordry_run.json) and its graph files go into a new numbered folder,results/<pipe_code>_output_<NN>/beside the bundle it ran, else undermthds-wip/, holding the same.gitignore, instead of beside the bundle itself, so two runs of a method read the same tree, with the samemethod_tree_digest.mthdsandpipelex-sdkversions, MTHDS protocol 1.0.0 and standard 7.0.0 (Breaking): the pins move tomthds==0.22.0, the release that lists method trees, carries protocol 1.0.0 and implements MTHDS standard 7.0.0, andpipelex-sdk==0.45.0, the release built on it, so the runner andGET /v1/versionreportprotocol_version1.0.0, a crate pipelex normalizes is stampedmthds_version7.0.0, and aMETHODS.tomldeclaringmthds_version = "^5.0.0"now warns that the engine does not satisfy it. A dependency package fetched before is fetched again into the release's new cache layout, a package'sMETHODS.tomlis found only at its root and manifest discovery stops at any.gitentry; everyone downstream inherits those exact versions.- A method's
requirements.txtis checked when it loads (Breaking): a method tree holding Python and arequirements.txtat its root is refused before any of its Python is imported, naming each requirement, when the environment its PipeFunc steps run in does not meet it, or when the file holds a line the MTHDS standard refuses (a URL, an option such as-ror-e, a comment after a requirement, a continued line). A package a method calls by its address is checked the same way, named by its address. Locally the environment is the interpreter Pipelex runs in and the refusal gives theuv pip installcommand that fixes it; on a sandbox-hosted deployment it is the package set the deployment states for its sandbox, a run is refused with a422and a validation bymethod_refwith anis_valid: falseverdict located on the line, naming no path on the server. A method of a server's own library directories whose requirements are not met is the server's fault, a500HostLibraryRequirementsErrorwhose message never reaches the caller. Pipelex never installs anything. A project root loaded as a library directory is such a tree, so hand over the folder holding the methods instead when the root'srequirements.txtis an application's. - A
PipeFuncnames its function by import path (Breaking):function_nameis the function's import path from the root of the step's method tree, such asfuncs.totals.compute_totalsfor the functioncompute_totalsoffuncs/totals.py, and a flat name is refused as the bundle is read with the advice to runpipelex fix function-names. Validation locates the module among the tree's files and proves, without running any Python, that the function is a top-leveldeforasync defthe module defines carrying the@pipe_func()marker; the module is imported under its real name, the tree's root first on the import path, when the step first runs, where its signature and return type are checked. A bundle passed as content imports locally from the process's import path, and on a sandbox-hosted deployment from the Python submitted with it. The runner's OpenAPI document gives thePipeFuncblueprint'sfunction_nameits import path pattern andPipeValidationErrorTypeitspipe_func_function_refusedvalue. - One method tree's Python per process (Breaking): a run holds the Python of the tree it imports from until its library is torn down, and a step of another tree that would load its Python meanwhile is refused with
TreePythonHeldErrornaming both trees by a bundle each holds and the start of its digest, never by a path of the host, so a run whose own steps and a dependency package's steps both call Python is refused locally; once no run holds a tree, its root leavessys.pathand the next tree evicts every module imported from a file of it, whatever imported it. A top-level module of a tree named like a standard-library module or like a module importable outside the tree is refused at validation,structuresexcepted, and on a sandbox-hosted deployment, whose sandbox holds its own packages, only a standard-library name is, and while a tree is held, importing a file its membership rule keeps out, such as a.gitignored module, is refused naming the rule, and no bytecode is written into the tree. - The function registry is retired and
@pipe_func()takes no argument (Breaking):FuncRegistry,func_registry,FuncRegistryUtils,FuncRegistryError,RuntimeHub.get_func_registryand thename=argument of@pipe_funcare removed, no function is discovered or imported while a library loads, andis_marked_pipe_func(func=...)tells a function carrying the marker. - Kernel func ops take the function (Breaking):
pipelex.kernel.func_ops.call_registered_functionbecomescall_function(function=..., memory=...), andrun_functakes thefunctionitself in place offunction_name. - Plugin API version 7 (Breaking):
PLUGIN_API_VERSIONmoves to 7, so a plugin declaringtargets_api = 6is refused at discovery withPluginApiVersionMismatchError: a bundle validator'svalidate_bundlesreceivessubmitted_filesin place ofbundle_resourcesand takesmethod_dirandmethods_dirs, itsmthds_contentsbecoming optional, a PipeFunc executor's transported request names its tree bytree_digestand carries nofiles, the transported entrypoint takes the directory the call's files were unpacked into, an orchestrator running a job out of process copies the job's trees before it returns, andclaim_method_file_storejoins the registrar's menu. LibraryCratenames its method trees by digest (Breaking):python_sources,bundle_resourcesandhost_bundle_filesgive way totree_bundles, which names each pipe's bundle by its tree's digest and its tree path (TreeBundleRef), so a crate carries no file's bytes, no listing and no path of the host that assembled it, a pipe built from it resolves a file its bundle names lexically and reads nothing, a Temporal workflow loading it included, and its fingerprint ignores the trees. Bundles loaded from content strings sit at the root of the tree of the files submitted with them, which the files of a library directory whose own tree holds no bundle join; two directories submitting different files at one path refuse the load withBundleResourcePathError, naming the path in the tree alone and joining none of their files, and a structure class is refused in every tree the load lists, a package or repository nested in a library directory included. A load that validates a crate builds each pipe as a member of its tree when a library of this process holds the tree under the digest the crate names, so aPipeDocGenHTML template file is checked as it was where the crate was assembled, and refuses such a step with the newPipeDocGenTemplateFileUncheckedErrorwhen no library holds its tree, rather than passing its template unchecked; a load skipping validation still reads no file.LibraryCrate.concepts_only()gives the crate's concepts and domains without its pipes.- A
PipeFunccall names its method tree (Breaking):PipeFuncExecutionRequest.tree_digestnames the tree of the called step, and the process running the call reads its files, every one but the.mthdsfiles and a generatedstructuresmodule, through the method file store under the run's read scope (read_pipe_func_call_files). The direct executor's transported run writes them into a temporary directory, generatesstructuresunless the method shipped one, imports the function by its import path under the process's hold on that directory and unloads its modules before removing it; the transported entrypoint takes as a required third argument the directory a sandbox client unpacked the files into, since a sandbox reaches no method file store, and runsDirectPipeFuncExecutor.run_pipe_func_transported_aton it. Only the called step is built in the tree written for the call, which holds none of the tree's bundles:load_from_crateandrehydrate_library_and_memorytakewritten_pipe_trees, keyed by the step's pipe ref, and every other step, aPipeDocGenof the same tree included, is built as the crate names it, its template file named by the crate's digest. The transported library's id is public,TRANSPORTED_LIBRARY_ID. - A
PipeDocGentemplate file is read where the document prints (Breaking):DocumentComposition.template_file, aComposedTemplateFilenaming the file by its tree's digest and tree path (MethodFileRef), replacestemplate_pathandtemplate_resource, and the print stage reads it through the method file store under the run's read scope, renders an.htmlone with the step's data, carried as aTemplateRenderContext(the stuffs the step's inputs name, the concepts and domains of the run's crate with none of its pipes, the crate's fingerprint, and the run's runtime parameters), through the template file stage the interpreter installs at boot, a dry print included, in the run's own library where this process holds it and otherwise in a library loaded from the concepts, which the process keeps for later prints (RenderLibraryCache), and hands an office one's bytes to its engine, whose template check moves from the step's dry run into the dry print. An.htmltemplate file whose bytes changed after its tree was listed is refused at load, never read from the disk unchecked.make_render_jobis synchronous again and takes the renderedhtmlor the template's bytes; an inlinetemplateis still rendered where the step runs. - Files submitted with content strings make one method tree (Breaking):
PipelexMTHDSProtocol,validate_bundle,validate_bundles_in_process,acquire_library,pipeline_run_setup,resolve_crate_from_contentsandBundleValidatorProtocol.validate_bundlestakesubmitted_files, a mapping from the path a step names to the file's bytes, in place ofbundle_resources, andLibraryManagerAbstract.add_submitted_filesreplacesadd_bundle_resources. The run names their tree by its digest and carries none of the files, so nothing is stored under the run'sassets/bundle/before it leaves the process, the 512 KiB bound on.htmltemplate files is lifted, and files that cannot make one tree, such as two paths equal once case-folded, raiseBundleResourcePathError. A step names a submitted file by any spelling of its path, sotemplates//invoice.docxreads the file submitted astemplates/invoice.docx. - A relative URL in an HTML template names a file of the method (Breaking): a literal relative URL of an HTML template, or of a stylesheet it links, that names no image, stylesheet or font of the method's tree now fails the method's load,
pipelex validateand dry runs included, where it loaded and failed only at print. A relative path such asimages/logo.pngthat an engine printing HTML handsRenderResources.loadas written now resolves in the template's method tree, and is refused withTemplateUrlRefusedErrorwhen the template belongs to none, where an unscoped run read it from its working directory; a root-absolute local path, and every URL of a document printed from the auto-layout or an office template, read as before. - A method sent to the API server is a method tree (Breaking):
filesandbundle_b64are read as one map under the MTHDS standard's tree rule, so a key holding a dot file or directory (.mthds/included), an empty,.or..segment, a backslash or a colon, a key given twice, or two keys naming one file is a422InvalidMethodFilescarrying the standard'stree_refusal(codeandnames) in place ofInvalidBundle, and a package the method calls travels independenciesbesidefiles, while abundle_b64archive still carries it under.mthds/methods/<name>/. The whole tree is written out and loaded as the caller's library directory, so a template beside its bundle needs noresourcesentry and counts against the tree's limits rather than those ofresources, an image, stylesheet or font only a template's relative URL names travels with the method, a file sent inresourcesjoins the tree with no step needing to name it, a bundle that does not parse is a422naming it by its tree path, and with nopipe_codethe run starts at the tree's entry pipe by the protocol's rule. method_refis read from its commit and run as a tree (Breaking): a package fetched by address is read from the git objects of the commit it resolved to, never from the clone's working tree, and runs or validates as a tree sent infilesdoes, its template files and Python included, the packages it ships under.mthds/methods/<name>/becoming dependency trees that/v1/validatenow loads too; its entry pipe is its manifest'smain_pipe, so a manifest declaring none answersdefault_pipe_ref: null./v1/validatevalidates a method tree whole (Breaking): a method sent asfilesor fetched bymethod_refis validated as it would run, every.pyfile of it compiled for its syntax, a file that does not parse or is not UTF-8 making the verdictis_valid: falsewith an item at its tree path and line, and itsdefault_pipe_refand graph follow the tree's entry pipe, bothnullfor a tree with none;mthds_sourcesaccompaniesmthds_contentsalone.
Fixed
- The local storage provider publishes a write whole:
LocalStorageProviderwrites to a temporary file beside the key, flushes it, then renames it onto the key, so a full disk or a killed process never leaves a file cut short under a key, and two writers of one key each land whole. - A bundle file that is not UTF-8 text is a located diagnostic: a
.mthdsfile read from disk that does not decode is a parser error naming its source and the line and column of its first undecodable byte, so a method tree sent to the API server holding one is the caller's422on a run and an invalid verdict on/v1/validate, never a server error. - Python a method tree holds that cannot parse is its diagnostic on every interpreter: a
.pyfile holding a NUL byte, which Python 3.11 refuses with aValueErrorwhere later versions raise aSyntaxError, or one nested too deeply to parse, is an invalid verdict located at its tree path, and aPipeFuncwhose module is such a file is refused as one that does not parse. - A
template_fileno sent file could be keyed by is a blueprint item at its step: a bundle loaded from content strings, as the API's routes andvalidate_bundleload one, whosePipeDocGenstep'stemplate_fileholds a..segment or a NUL character, or names no file, is refused when it parses, whatever files the request sends, as oneblueprint_validationitem withfield_pathpipe.<code>whose message names the field and not the path, instead of a load refusal saying the file was not supplied. A bundle read from disk still resolves a..in its method tree. - A
method_refpackage's template files reach its steps: aPipeDocGenstep of a package run or validated bymethod_refreads the template file the package holds beside its bundle, where the run before had no way to read it. - A request's own method that does not load is the caller's
422: a method sent to the API server whose bundle does not parse, or whose tree cannot load, is refused as the caller's fault, naming each file by its tree path, instead of a500naming the server's temporary directory. - A TOML syntax error names the bundle's source: a bundle read from a file under a source the caller gives names that source in its TOML syntax error, never the file's path on the host.
Removed
- Bundle resources and their dispatch (Breaking):
BundleResource,BundleResources,make_bundle_resources,BundleResourceReadStage,MAX_LOAD_STAGE_RESOURCES_BYTES,prepare_job_for_remote_dispatch,relocate_run_resourcesanddoes_pipe_run_leave_processare removed,InterpreterHub.set_pipe_runtakes the pipe run alone, andBundleResourceErrorandBundleResourcesTooLargeErrorare no longererror_types: no template file rides in a run's payloads, so none is bounded by them, and a method file a run cannot read raisesMethodFileUnavailableError.
[v0.86.1] - 2026-10-16
Changed
- A scoped event log captures usage events: a usage event emitted while
runtime_hub.scoped_event_logis set now goes into the scoped event log and nowhere else, ahead of an event log registered withReportingManager.set_event_logand of the per-process runner fallback, including inside an isolated sub-execution such as a Temporal activity, where it always went to the fallback. Code running a leaf inside an activity can therefore scope a freshInMemoryEventLogaround it and return the leaf's usage events with its result, which is how a backend with no shared tracing store, such as Mistral Workflows, collects a run's usage; with no scoped event log set, usage events are routed as before.
[v0.86.0] - 2026-10-13
Highlights
A run's trace now records each decision as it is made and names every pipe by its place in the method. A condition's chosen outcome and a batch's width are written before anything runs under them, and every trace event carries its invocation path, its mode and the attempt of the execution that wrote it, so a graph built from a run's events names each pipe the same way in every run, live or dry, and shows a retried step once.
Added
condition_resolvedandbatch_fanned_outtrace events: a livePipeConditionwritescondition_resolvedthe moment its choice is final, before anything runs under it, with therendered_valueits expression rendered and theselected_outcomekey of the outcome map it matched,nullwhen it took the default outcome, and a chosen pipe whose inputs are missing fails the condition with no outcome recorded; both graph builders derive from that event and the start of the chosen pipe theselected_outcomeedge from the condition, labelled with that key, which no run drew before, whichever workflow the chosen pipe started in. APipeBatchwritesbatch_fanned_outonce it knows its width, before any item starts, withitem_count,branch_pipe_code, the bare code of the pipe each item runs,branch_pipe_ref, its qualified reference, andbranch_pipe_type. A dry run resolves no condition, so it writes nocondition_resolved, and both graph builders build the same graph from a run's events.step_countandbranch_countonpipe_start: aPipeSequence's start event states the number of steps its author wrote, the binding of a dottedbatch_overnot among them, and aPipeParallel's the number of its branches; both arenullon every other pipe.
Changed
- Every trace event states its
mode(Breaking): each trace event carriesmode,liveordry, the mode the run's pipes run in, which the run'sTraceContextcarries as the newrun_mode, so a usage event an activity writes states it too, and a live run a keyless boot (needs_inference=False) forces dry statesdryon its trace context, its events and its graph, and, like every dry run, opens no OpenTelemetry trace; an event stored withoutmodeby an earlier version is refused withEventLogSchemaMismatchErrorrather than read as live, and a consumer that builds trace events passes it. Callers follow new signatures:TraceContext.run_modeis required, with no default, and so isrun_modeonGraphTracerManager.open_tracerand on the tracers'setup(GraphTracerProtocol,GraphTracerandGraphTracerNoOp), where it replacesmode: GraphSpecMode, so a workflow opening a tracer of its own for part of a run passes therun_modeof the context it received, andPipelexKernel.makerefuses, withTraceContextRunModeError, atrace_contextstating another mode than the one its run takes, which the newruntime_hub.resolve_effective_run_modegives, dry on a keyless boot.GraphSpecAssembler.assembleno longer takesmodeand raises the newTraceEventModeErroron a set of events that mixes modes or holds none, andassemble_tracingandassemble_tracing_on_outputno longer takerun_mode: the graph states the mode its events state, so a dry run assembled from its events in a durable workflow gets a dry graph where it got a live one, andassemble_tracingreports aTraceEventModeErroras the run'sgraph_assembly_error. - Controllers register their execution data as they decide (Breaking): a
PipeSequenceregistersstep_countas it starts, now the number of steps its author wrote; aPipeParallelregisters itsbranch_count,add_each_outputandcombined_output_conceptbefore its branches run, aPipeBatchitsitem_countandbranch_pipe_codebefore its items run, and a livePipeConditionitsevaluated_expressionandselected_outcomeonce its choice is final, before the chosen pipe runs, where each used to register them once it had finished.GraphTracerProtocoladdsregister_batch_fan_outandregister_condition_resolution, andadd_selected_outcome_edgetakesselected_outcome: str | Nonein place ofoutcome_value. - Trace events and graph nodes name their invocation (Breaking): every trace event carries
path, the invocation path the hosted progress contract defines, which names a run of a pipe the same way in every run of the method, live or dry, on every engine: the entry pipe'spipe_ref, then one segment per controller level,step_<n>as the author numbered the steps, binding steps included,branch_<n>,item_<n>counting the items from 1, andoutcome_<key>ordefaultfor a condition's outcome group, the outcomes that run one pipe as the library resolves it, however they spell it, named by the first key that runs it, percent-encoded, sofilings.review_filings/step_3/item_2/step_1, and a dry run runs each group's pipe once; a binding step's events carrynull. EveryNodeSpeccarries the samepath,pipe_startcarriespipe_ref,pipe_end_success,pipe_end_errorandpipe_end_skippedcarrypipe_code,pipe_refandpipe_type, and a failed node'sErrorSpeccarriescaller_facing, true when its message was written for the caller, or the error's text withcaller_facingfalse when the error's report cannot be built, a fault the tracer logs while the run still fails with the pipe's own error. A controller hands each pipe its place throughJobMetadata.for_child_at; since tracing never fails a run, a pipe started without one is traced with anullpath and the tracer logs the fault, the newInvocationPathError, as an error, while an inactive tracer and the no-op tracer derive no path;GraphTracerManager.on_pipe_startand the tracers'on_pipe_startrequirepipe_ref,on_pipe_end_errortakescaller_facing, andTraceContext.copy_for_childrequiresinvocation_path. An event stored without these fields by an earlier version is refused withEventLogSchemaMismatchError, and a consumer that builds trace events passes them. - Trace events name the attempt of the execution that wrote them (Breaking): every trace event carries
attempt, the attempt of its execution counting from 1, andlineage, the{execution, attempt}entries it was dispatched under, empty for the run's own execution, besideworkflow_id, which names the execution,directin direct mode.GraphTracerManager.open_tracertakesattemptandreceived_context, the trace context a dispatched workflow received, which it continues: its parent node, its path and, when it names the workflow's place as a batch item's does, its chain as the workflow's lineage;workflow_idmust be the id the engine keeps across attempts, andTraceContextcarries the newparent_path,child_segmentandchain, the last required with no default, so a host building a context of its own states it,make_direct_chain()in a single process. Node and edge ids name the chain that minted them,<graph_id>:<workflow_id>:attempt_<n>:node_<k>, followed after the attempt by a digest of the lineage for a dispatched workflow, so two attempts of one workflow never mint the same id; direct mode keeps<graph_id>:node_<k>.ReportingManager.set_event_logno longer takesworkflow_id: a usage event takes its execution, attempt and lineage from the chain of its trace context, on the registered path as on the runner fallback. A trace event is frozen, and one whoseworkflow_idis empty is refused withEventLogSchemaMismatchError. The event logs keep every attempt's events, and every one of them, the DynamoDB and buffering logs included, returns a run's events deduplicated on the event's chain, writer, kind and sequence and ordered by chain, sequence and writer; the DynamoDB log's sort key becomesEVENT#<chain digest>#<writer_id>#<sequence>, the chain digest being the SHA-256 of the chain key, so the key stays within DynamoDB's limit however deep the lineage, and the NDJSON log writes one file per attempt of each execution,wf_<workflow_id>__c_<chain digest>.ndjson,__w_<writer_id>added for a writer other thanprimary.GraphSpecAssemblerbuilds the graph, and dates it, from current events only: a higher attempt of an execution supersedes the earlier one and everything it dispatched, so a retried invocation appears once at its path, a lost attempt leaves no failed node, and a superseded attempt's usage counts in the run's total as unattributed usage.
Fixed
- Trace events written to DynamoDB expire (Breaking): the DynamoDB event log now writes every event with a
ttlin epoch seconds, its write time plus the newruntime.tracing.dynamodb.retention_days(30 by default, at least 1), so a table that expires items onttldeletes a run's events, full output data included, where it used to keep them forever. Only events written from this version on carry attl, so the rows an earlier version wrote stay until they are deleted or given attlby hand.DynamoDBEventLogtakesretention_daysand now takes all its arguments by keyword;make_event_logreads the setting from the configuration.
[v0.85.1] - 2026-10-12
Fixed
- A
PipeDocGendocument's name is one the hosted API serves: a renderedfilenameholding a letter outside ASCII or a dot, such asZoë Ångström — Q3.2026, was stored under apipelex-storage://URI the hosted API refuses to serve, so the completed document could never be downloaded. The name is now spelled in ASCII where it can be, accents dropped and letters such asø,æ,ł,ß,đ,þ,œandıtransliterated, every other character, an inner dot included, becomes a hyphen, repeated separators collapse, and a name that leaves nothing, one written wholly in Greek, Cyrillic, Han or Arabic for one, falls back to the pipe's code, so that name isZoe-Angstrom-Q3-2026.pdf, on a local run as on a hosted one. - The files a run stores from what it was sent are named and typed for their readers: the copy of a
.docx,.xlsxor.pptxtemplate file a run stores underassets/bundle/before it is dispatched now carries its Office Open XML content type, named in code, where a host whosemimetypestable did not know the extension stored it asbinary/octet-stream, and its name follows the rule a document's name follows, soModèle trimestriel.v2.docxis stored asModele-trimestriel-v2.docx. A local file input stored underassets/that its bytes and type do not identify keeps its own suffix only when it is plainly letters and digits, and is stored as.binotherwise. - The refusals of the files sent with a bundle say a retry cannot succeed:
BundleResourceNotNamedError,BundleResourcesTooLargeErrorandBundleResourcePathErrornow carryretryable: falsein their error report, and so in the API server's problem document, asInvalidResourcePathalready did. APipelexErrorsubclass declares it with the new class attributeretryable, which its report carries and the generated error reference shows; a wrapper raised from such an error inherits it, as it doeserror_domain.
[v0.85.0] - 2026-10-12
Highlights
A method that prints with a template file now runs wherever it is sent. A PipeDocGen step's .xlsx, .docx, .pptx or .html template file travels with its bundle, as resources on the API server's routes, inside a bundle_b64 or files bundle, or as bundle_resources in Python, so a caller with no local pipelex can validate, dry-run and run it, and a bundle loaded from content strings never reads the runtime's own disk.
Added
resourceson the API server's routes:POST /v1/validate,/v1/execute,/v1/start,/v1/pipe-io,/v1/resolveand/v1/codegentake the files a bundle sent as content reads, such as aPipeDocGenstep'stemplate_file, as a map from the path the step names to the file's bytes in standard base64, so a caller with no local pipelex can validate, dry-run and run a method that prints with an.xlsx,.docx,.pptxor.htmltemplate file. A key that cannot name a file inside the bundle answers422 InvalidResourcePath, a file no step names422 BundleResourceNotNamedError, a value that is not base64400 InvalidBase64, and a request overMAX_RESOURCES_PER_REQUEST,MAX_RESOURCE_FILE_KIBorMAX_RESOURCES_TOTAL_KIB(32 files, 5 MiB each and 10 MiB in all by default)413 PayloadTooLarge;resourcesbeside amethod_refis a422. A method bundle sent asbundle_b64orfilesmay ship such a file instead, beside the.mthdsfile whose step names it, and it reaches the step the same way, under the same limits; two different files under one path, shipped twice or shipped and sent, answer422 InvalidResourcePath, as two spellings of one key do before any value is decoded. On an orchestrator other thandirect, such as Temporal, a run stores each office template file under itsstorage_scopeatassets/bundle/before it is dispatched, so the job carries apipelex-storage://reference rather than the bytes, while the.htmltemplate files, which travel inline with the method, may total 512 KiB at most, a fixed bound refused with413 BundleResourcesTooLargeErroronce the bundle is parsed.- Template files sent with a bundle loaded from content strings:
PipelexMTHDSProtocol,validate_bundle,validate_bundles_in_process,acquire_library,pipeline_run_setupandresolve_crate_from_contentstakebundle_resources, built withmake_bundle_resourcesfrom the bytes of each file keyed by the path its step names. The library holds them and itsLibraryCratecarries them, outside its fingerprint, so aPipeDocGenstep finds its template file wherever the bundle is loaded: an.htmlone is checked against the inputs at load, the engine's template check runs on an office one at validation and in the dry run, and the print stage reads it, from the run's storage when the crate carries a reference. Before a run, live or dry, leaves the process,prepare_job_for_remote_dispatchstores its office template files under the run's storage scope;PipelexMTHDSProtocol.executecalls it when the hub's pipe run is a boot orchestrator's, and a dry run's template check reads a stored file under the run's read scope, as the print stage does. A step of a bundle the runtime read from its own disk, such as the host's library, still reads only beside it, so a file sent under the same path never stands in for its template. A file no step names raisesBundleResourceNotNamedError, a path outside the bundleBundleResourcePathError, and.htmltemplate files totalling more thanMAX_LOAD_STAGE_RESOURCES_BYTES(512 KiB), which travel inline with the method wherever it runs,BundleResourcesTooLargeError. - Template-check warnings in the validation verdict: a warning the engine's template check finds in a
PipeDocGenstep's office template file, such as an Excel defined name or a PowerPoint shape name that matches no input field, now rides the verdict's advisorywarningsonpipelex validate,pipelex-agent validateand the API server'sPOST /v1/validate, where it was only logged. It is adry_runitem with the newerror_typetemplate_check_warning, the step'spipe_codeanddomain_code, and two new optional locators ofValidationErrorItem,template_fileandtemplate_location, the latter unset when the engine names no place, which the server's OpenAPI schema publishes, a step of the server's own library directories naming no file of theirs; an error finding still fails the validation,validate pipe <code>on both CLIs carries these warnings alone, and a dry run that returns no verdict, such aspipelex run --dry-run,pipelex fix bundleorpipelex build inputs, still logs them.
Changed
BundleValidatorProtocol.validate_bundlestakesbundle_resources(Breaking): a bundle validator registered for an orchestration mode receives the files sent with the contents, orNone, and hands them to the validation it runs, so a remote validator implements the parameter and carries the files to its worker.InterpreterHub.set_pipe_runtakesleaves_process(Breaking): whoever sets the hub's pipe run says whether it hands each job to an orchestrator outside the process, as a boot orchestrator's does, whichdoes_pipe_run_leave_processreads back so a run's submitted files are stored before it leaves.DocumentComposition.make_render_jobis asynchronous (Breaking): it takes the run'sstorage_provider, since a composition's template file may be aBundleResourceread back from storage, and aTEMPLATE_FILEcomposition carries eithertemplate_pathortemplate_resource. A stored template file's URI is among the composition'sreferenced_uris(), so the run's read scope authorizes it before the print stage reads it.- A
PipeDocGenstep'stemplate_filemust be a path inside its bundle (Breaking): an empty or absolute path, a Windows root, a drive, a UNC share, a..segment, a backslash or a colon is refused when the blueprint validates, by the rule the files sent with a bundle are keyed by, so every template file a step names can be sent with its bundle; an absolute path naming a file inside the bundle's directory, a path such astemplates/../invoice.docxand a file name holding a colon used to load. The refusal does not repeat the path, so a host library's step can no longer put the host's own path into a caller's verdict through its template file. Beside a server's own library directories, the verdict also withholds the template files their steps name, as their bundle files already were, both when such a step is refused while they load and when the engine's template check refuses its template in the dry run of a caller's pipe that runs it; a template file the caller's own content names is kept. - A string-loaded
PipeDocGenstep without its template file says it was not supplied: the load refusal now names the path the file is expected under and says to send it with the bundle, where it used to say the bundle was not loaded from a file on disk. build_advisory_warningsandcollect_advisory_warningstake the sweep'sdry_run_result(Breaking): the composition point of the advisory warnings reads what the dry runs found from the per-pipe status map, so a caller passes the map its validation sweep returned.DryRunOutputcarries those findings in its newwarningswhen the sweep is asked to collect them with the newcollect_dry_run_warningsofvalidate_bundle,BundleValidator.validate_pipesandBundleValidator.validate_current_library; a sweep that is not asked leaves them to the log, as before.
Fixed
- Contents named by
bundle_urisread their template files beside their file: a bundle passed asmthds_contentswith the path of the file it was read from inbundle_uris, toPipelexMTHDSProtocol,acquire_libraryorpipeline_run_setup, and not already loaded from the library directories, now loads as read from that file, so aPipeDocGenstep'stemplate_fileis found beside it, where it was refused as if the bundle had no directory, and an address-based reference is looked for in a.mthds/methods/directory above it.
Security
- A bundle loaded from content strings never reads the runtime's disk: a
PipeDocGenstep of such a bundle read itstemplate_filebeside whatever its source named, so a caller'smthds_sourceson/v1/validate, afiles[].sourceon/v1/resolve,/v1/codegenor/v1/pipe-io, or asourcekey written on the step in the bundle's text could make the runner read a file from its own disk, and an address-based reference searched.mthds/methods/directories above that label too. Both now start only from a bundle file the runtime read itself, which the parser records asPipelexBundleBlueprint.host_bundle_fileand the crate carries asLibraryCrate.host_bundle_files, outside its fingerprint and its normalized form; a content-string bundle reads only the files sent with it, and a step'ssourceis overwritten whenever the pipe is built.
[v0.84.0] - 2026-10-12
Highlights
pipelex now reports anonymous usage statistics, on by default. Commands, runs, validations and code generations are counted with closed-list properties only, never a method, a prompt, an input or an output, and DO_NOT_TRACK, PIPELEX_USAGE_STATS=0, pipelex usage-stats disable or [usage_stats] enabled = false in a project turn them off.
Added
- Anonymous usage statistics: pipelex now reports anonymous counts of its use to Pipelex, on by default: a
command_completedper command of thepipelexandpipelex-agentCLIs, arun_completedper run however it ended, with amodel_usedper model when the run's usage events were read, amethod_validatedper validation and acode_generatedper code generation, whatever started them. Every property is a count, a boolean or a value from a closed list: no method, pipe code, concept name, prompt, input, output, file name, error message or argument is ever sent, and a model outside the backend tables pipelex ships is sent asother. The first command with statistics on prints a notice on stderr, once per machine, except where stderr is a machine contract (pipelex-agentwith JSON errors), andpipelex initshows the same text on stderr whenever statistics are on, once per run. Statistics stay off underDO_NOT_TRACK,PIPELEX_USAGE_STATS=0,pipelex usage-stats disable, themthdsCLI's formerDISABLE_TELEMETRYopt-out,[usage_stats] enabled = falsein a project's.pipelex/pipelex.tomlor in the packaged configuration, a~/.mthds/usage-stats/pipelex cannot look into, and thepytestintegration mode; the usage statistics page lists what is collected. A dedicated client sends them to PostHog's EU region, from a detached sender after a CLI command and from a daemon thread in a Python program or the API server;telemetry.tomland the telemetry you configure are untouched. pipelex usage-stats:status(with--format json),disable,enableandreset-idshow and change the statistics of every Pipelex tool on the machine, through the state they share in~/.mthds/usage-stats/.PIPELEX_USAGE_STATS_DEBUGprints each event to stderr as a JSON line instead of sending it, andPIPELEX_USAGE_STATS_ENDPOINTsends to a loopback origin for tests. Each queued file names the endpoint its events were queued for, and a sender posts the files naming its own origin and those naming none, so events queued under the variable never reach PostHog.[usage_stats] enabled: a newpipelex.tomlsection,truein the packaged configuration. Set tofalsein a project's.pipelex/pipelex.tomlit turns statistics off for everyone working in the project; it is read from the project's own file, never from~/.pipelex/, and a file that cannot be read turns them off too.- Token usage names the backend that served the call:
LLMTokensUsage,ImgGenTokensUsage,ExtractTokensUsage,SearchTokensUsageandJudgmentTokensUsagegainbackend_name, set by the job from the backend whose worker ran it, so a run's usage events credit each call to its own backend even when two backends call the model by the same id, as after a fallback. A synthetic report no backend made, a dry run's, leaves itNone. @default-judgment-vision: the judgment deck names a second alias, for a judgment model that reads images, beside@default-judgment, which stays TypeSafe's text-onlyjev-1.13.0. It points at OpenAI'sgpt-6-luna, which judges through OpenAI's Decisions API; no inference backend of Pipelex calls that API yet, so on your own keys aPipeJudgenaming the alias finds no model when it runs. Runpipelex updateto bring the alias into an installed deck.
Changed
ConceptLibraryAbstract.list_own_concepts(Breaking): the concept library declares the concepts a loaded method declares itself, without the natives and the concepts of its dependency packages, so a custom implementation ofConceptLibraryAbstractimplements it.
Fixed
- A
{% filter %}block prints its result as a{{ }}tag does: a filter block whose filter returns something other than text, such as{% filter markdown %}…{% endfilter %}in a document template or{% filter length %}, now prints its result as a{{ }}tag prints a value, through the environment'sfinalizeand then as text, where it used to fail the render with a type error naming a Python type. A block whose filter returns text prints as before, and the printed result is charged to the render's budget like any printed value.
Security
uv.lockrefreshed past the open Dependabot alerts: the lock thepipelex/pipelex-apiDocker image installs from now holds patched releases ofanyio,datamodel-code-generator,multidict,oauthlibandurllib3, and a development checkout also gets patcheddocling,transformers,torch,accelerate,fsspecandsetuptoolsthrough thedoclingextra and a patchedWerkzeugthrough thedevextra. Nopyproject.tomlrange moved, so an application installing thepipelexwheel picks these releases up through its own resolution.
[v0.83.0] - 2026-10-11
Added
LLMCompletionTruncatedErrorandLLMCompletionRefusedError: two subclasses ofLLMCompletionError, the first for a text completion cut at its output limit or its context window, the second for one the model declined to write or a safety filter stopped. Both are content errors in theinputdomain, not retryable, answered with HTTP 422 and caller-facing under STRICT disclosure, and carrymodel_handle,pipe_codeand the provider'sstop_reason, a truncation also carrying the limit it hit astruncation_limit,max_tokensorcontext_window, theoutput_tokensused and themax_tokenssent; the message names the model, the pipe and those values, never the partial text. Each ends with achange_inputnext step fitted to what stopped the text: revise the prompt or the input after a refusal, raise the pipe'smax_tokensor shorten its input at the output limit, set the pipe'smax_tokensup to the model's limit when the request sent none, and shorten the input, lower the reasoning effort or choose a model with a larger window, never a highermax_tokens, at the context window.
Changed
- The Bedrock client's
chatreturns aBedrockChatResult(Breaking):BedrockClientProtocol.chatreturns a named result holding the text, the token usage and Converse'sstopReason, in place of a(text, usage)tuple, so a custom Bedrock client returnsread_converse_response(response=…)or builds the result itself. - A lowered Anthropic structured-output limit is said: when a structured generation on an Anthropic model lowers
max_tokensto fit its structured-output timeout, the call logs it, at warning level when the pipe set the limit and at debug level when it is the model's default, and an error the lowered limit can explain, a structured output that failed validation or came back incomplete, or a timeout, ends with a sentence saying the limit was lowered and to what, never a rate limit, an authentication failure, an overload, a server error or a lost connection.
Fixed
- A truncated, refused or filtered text no longer passes for a result (Breaking): every LLM worker, OpenAI chat completions whether direct or behind a gateway such as Portkey or OpenRouter, the OpenAI Responses API, Anthropic, Google, Bedrock and Mistral, now reads its response's stop value before it returns a text, so a text that stopped short fails its pipe with
LLMCompletionTruncatedErrororLLMCompletionRefusedErrorwhere it used to come back as a successful output, empty or partial, its billed usage still reported and recorded on its span. Each provider's vocabulary is read, including a gateway's pass-through of another provider's value such asmax_tokensorcontent_filtered, an unknown value being logged at warning level and taken as normal, except that anincompleteResponses API answer is a truncation whatever its reason; a Gemini prompt blocked before any candidate is a refusal, and structured generation keeps its configured re-asks. A Bedrock answer is read for every text block it holds, so one holding none fails with anLLMCompletionErrorwhere it used to raise anIndexErrorwhen its message was empty, or aKeyErrorwhen its first block was not text.
[v0.82.0] - 2026-10-10
Added
Ratingcarries the label of its level: theRatingnative has an optionallabel, which the MTHDS standard's native set pinned at5.0.0defines as the name of the selected level when the scale declares labels, held inRatingContent.labeland stated by the input-form descriptor, the I/O contracts and the generated Python and TypeScript types. APipeJudgewhose levels carry labels fills it with the label of the level it chose, while aPipeLLMwhose output isRating, or any other producer, may fill it like any optional member; either way aRatingrenders in a prompt as its label when it has one and as its level otherwise, so$severityreadsWorkaround availablefor a rating carrying that label and1for one carrying none, and its JSON rendering carries the label only when present. Since a crate materializes the pinned definition, a crate whose methods useRatinggets a new normalized fingerprint, and types generated before this release gain the field whenpipelex codegen typesregenerates them.- A log line's advice in
user_action: a log call puts what its reader should do in auser_actionfield, one imperative sentence, and theconsolesink prints it whole on a line of its own under the record, after→, in cyan, while every other sink writes it as a field like any other. Pipelex's own warnings carry their advice there, such asRun pipelex migrate to update iton a stale configuration file, so their message says only what happened. - A
PipeJudgeasks several questions in one request: aPipeJudgemay setquestionsinstead ofquestion, a table of questions keyed by name, each holding itsquestionand its ownoptions,levels,criteriaandthreshold, checked under the single form's rules with a refusal naming the question, and the judging model answers them all in one request over the same evidence; the verdicts fill the output's structure, whose fields must be exactly the question names, each holding its question'sYesNo,ChoiceorRatingor a concept refining it, which the load checks field by field (inadequate_output_concept). A question the model declines leaves its field holding nothing when the field may hold nothing, being neitherrequirednor given adefault_value, or, in a structure class, typed to admitNone, with a warning naming it underjudgment_questionand its refusal in the step's execution data, and fails the step withJudgmentRefusedErrornaming the question otherwise, since a default would read as a verdict the model never gave. The kernel'srun_multi_judgmentdoes the same for a programmatic caller, taking the questions asAskedQuestions and the output's content class, and returning aMultiJudgmentResultwith each question's outcome; a class whose fields are not the questions' names is refused withJudgmentOutputFieldsErrorbefore the model is called. package_prefixchooses which console lines name their package: a setting under[runtime.log.rich_log]decides which lines theconsolesink starts with their logger's top-level package name, dimmed.libraries, the default, keeps the rule that names another library's package and leaves a line frompipelexor apipelex_package bare;allnames every line's package, sopipelex:,pipelex_api:and a plugin's name tell apart which component spoke;nonenames none. A line logged on the root logger never carries one.
Changed
- MTHDS standard 5.0.0 (Breaking): pipelex now requires
mthds==0.21.0andpipelex-sdk==0.38.0, the releases that report MTHDS standard5.0.0, so everyone downstream inherits those exact versions and a project that pins either package itself has to move with them.pipelex --versionprintsmthds-standard 5.0.0, every crate pipelex normalizes is stamped withmthds_version5.0.0and materializes the native set the standard pinned at5.0.0, and a package whoseMETHODS.tomlconstrainsmthds_versionto^4.0.0,~4.0.0,4.*or exactly4.0.0now logs a warning when it loads, which>=4.0.0or^5.0.0does not.PipeJudgejudges the evidence of itspromptwith two-sided criteria and labelled levels, and asks severalquestionsin one request, as the standard's5.0.0PipeJudge defines. PipeJudgejudges the evidence of itsprompt(Breaking):promptis now required and holds the evidence, a template rendered exactly as aPipeLLM's prompt is with its images and documents presented to the judging model as files, beside a requiredquestion:promptis no longer a synonym ofquestion, a pipe writing its question there is refused with a message saying where each goes, and a pipe is rewritten asprompt = "@message"beside its question. The model sees an input only as the two templates render it: every declared input must be read by the prompt or the question (extraneous_input_variable), an image or a document read by the question is refused (input_stuff_spec_mismatch), the model's file capability is checked against what the prompt reads, an absent optional input renders as its guard says, such as@?note, and aDynamicinput renders as text.criteriatakes bothyesandno, a one-sided or empty table being refused; a level oflevelsmay be a{ label, description }table, labels all or none and distinct, and theRatingverdict then carries the label of the level chosen; a model that declines the question fails the step with the newJudgmentRefusedError, a content error naming the pipe and the model, where no verdict is ever read off a refusal.- The judgment worker contract carries a rendered prompt (Breaking):
JudgmentJobandJudgmentAssignmentcarry aJudgmentPrompt, the rendered text with its images and documents, in place of thestate,imagesanddocumentskeyed by input name, andJudgmentStateis gone;YesNoQuestioncarries one optionalcriteriaof typeYesNoCriteriawith both sides in place ofyes_criterionandno_criterion;RatingQuestion.levelsis a list ofRatingLevel, each with alabel, adescriptionor both; andJudgmentWorkerAbstract.judgereturns aJudgmentOutcomeper question, an answer or aJudgmentRefusal. The judgment contract a backend plugin implements, with the family's errors a worker raises, is published in the Inference SPI table, and the kernel'srun_judgmenttakes the evidence as aUserPromptContent,build_judgment_materialandmaterial_valuebeing gone. - Console lines without the emoji prefix or the path column (Breaking): a line logged under the
pipelexpackage or apipelex_one, the runtime's, the API server's or a plugin's named that way, and a line logged on the root logger start with their message, and a line another library logged starts with that library's top-level package name, dimmed, such ashttpx:, where every line used to start with an emoji or a bracketed logger name. The column naming the source file and line of each record is off by default; setis_show_path = trueunder[runtime.log.rich_log]to show it,is_link_path_enabledstill making it clickable. Thepipelex.tools.log.log_formattermodule is gone with itsEmojiLogFormatter, so code that formatted records with it uses the stdlib'slogging.Formatter. - Log messages are one short sentence (Breaking): Pipelex's log messages, and its API server's, say what happened in one sentence that starts with a capital and ends with no period, and at
INFOand above hold no identifier and no backtick in at most 80 characters, the advice they used to append now ridinguser_action. Warnings about one event became one: whichever kind of inference job carried no usage, the line isAn inference job reported no usagewithinference_job_type; a stale configuration file isA configuration file is out of date and was migrated in memory onlywhatever its remedy; and aPipeBatchover a large list isA PipeBatch fans out over a large list without durable execution, bounded or not, withmax_concurrencyabsent when the fan-out is unbounded. What breaks is a log query or an alert written against an old message. - The API server's refusals of a caller's credentials log at
DEBUG(Breaking): a request with no bearer token, an expired or invalid token or a wrong API key logsA caller's credentials were refusedatDEBUG, withauth_modeandauth_refusal_reason(missing_bearer_token,expired_token,invalid_tokenorapi_key_mismatch), since the error line the server writes atWARNINGfor every401already records the refusal. A token that verifies but whoseuser_idclaim is missing, not a path-safe segment or the reserved single-tenant id still logs atWARNING, itsuser_actiontelling the token issuer what to fix, and an unsetJWT_SECRET_KEYorAPI_KEYlogs atERRORwith the variable's name inenv_var. An alert on the old authentication warnings counts the401error lines instead. - Exceptions ride as fields at every level: a
DEBUGorVERBOSEline about a handled exception, such as a Google client that could not be closed, has a fixed message and carries the exception's class and text aserror.typeanderror.message, as the lines atINFOand above already did, rather than splicing them into its text. The retry line names the class of what the attempt raised undererror.typerather thanexception_type. make check-log-callsholds a message's wording and its exceptions: the log-call guard also refuses, at every level, a message that starts with a lowercase letter, ends with a period or an ellipsis, holds a backtick or splices a handled exception into its text, through an f-string, a%, a.format()or a task'sexception(), and, atINFOand above, a message that holds an identifier, a word with an underscore or a call such assetup(), or runs past 80 characters. The rules arelowercase-start,trailing-period,backtick,spliced-exception,identifierandlength, specified on the log-call guard's page of the contributor documentation, and every call in the tree passes them with an empty baseline.- A refused enum value names the valid ones: a validation error for an unknown enum value, in a configuration file or anywhere else Pipelex formats pydantic's errors, now lists the values the field accepts,
invalid enum value 'inptu', expected 'input', 'input_cached', …, where it used to name only the refused value. - A field that never holds nothing refuses
null(Breaking): a structure field with adefault_valueis generated asX = Field(default=…)rather thanX | None, and a required field namingnative.AnythingasNonNullAny(pipelex.core.stuffs.non_null_any), any value butNone, so a value setting either tonull, an input or a model's answer, is refused, and a binding step reading such a field never records an absence its static analysis ruled out. Leave the key out, or leave its CSV cell empty, to get a field's default. The codegen projections follow: a defaulted field isX = Field(default=…)inpython-pydanticandpython-structuresand a bare.default(…)ints-zod, typedTrather thanT | null. The JSON schema of a defaulted field, the one a pipe's IO contract publishes as an input'sjson_schemaand a model is asked to fill, drops itsnullarm:{"type": "string", "default": …}rather than{"anyOf": [{"type": "string"}, {"type": "null"}], "default": …}.
Fixed
- An absent optional image or document is left out of its prompt: a
PipeLLMorPipeImgGenwhose prompt read an optional image or document behind its guard, such as@?photoor{% if photo %}, failed with "Could not find image 'photo' in context" when the input was not given, dry runs included. The file is now left out, taking no number and reaching no model, while an absent required file is still refused. A template reading a name aliased to a slot later resolved absent no longer fails with aKeyErroreither: the alias holds nothing, as its target does. - A Google client's close at teardown is no longer garbage-collected: when a Google LLM or image-generation worker was torn down inside a running event loop, nothing but the loop referenced the background task closing its async client, and a loop holds its tasks weakly, so the task could be collected before the client was closed, with the
asyncioerrorTask was destroyed but it is pending!. Each close task is now held until it is done. - A condition whose outcome may be skipped declares it: a PipeCondition routing to an outcome pipe that takes plainly an input the condition declares optional must declare its own output optional, and is refused with
optional_not_handledotherwise, since a run without that input skips the outcome and resolves the condition's output absent; an outcome the condition's expression chooses only with that input present declares it forced (!) instead, since validation does not read the expression. Every name such an outcome always stores in its caller's memory counts as maybe-absent too, so a sequence binding one into a plain output is refused where it used to validate and then resolve absent; the outcome is listed inliftable_pipesunder its condition, and a!input on an outcome counts in theoptional_force_redundantlint. - A sequence step that may be skipped keeps what its run may store: a step an absent input may lift, whose pipe stores a name on some runs only, now leaves that name maybe-absent when a run may store an absence there, so a sequence binding it into a plain output is refused with
optional_not_handledrather than validating and then resolving absent. - A configuration value of the wrong type is a validation error (Breaking): a model's
costs,listed_constraints,valued_constraintsorrulesin a backend file or in a plugin's internal model, a backend's constraints inbackends.toml, andpackage_log_levelsinpipelex.tomlno longer crash the boot with a bareAttributeErrororTypeErrornaming nothing when written as a value of the wrong shape, such ascosts = "free"or a date; the boot fails with a message naming the field, the model, the backend and the file, or the plugin, and an emptyinference.dry_run.image_urlsis reported the same way. A plugin may set an internal model'scoststoNone. Some values that used to be accepted are now refused: a table aslisted_constraints, a string asinterpreter.scan.excluded_dirs, a price that isnanor infinite, and amax_tokenssetting that is neither an integer nor"auto", which used to fall back to the model's default. - A construct path holding nothing leaves a defaulted field to its default: a
PipeComposeconstruct field fed{ from = "doc.note" }wherenoteholds nothing is left unset when the target field refusesnullbut has a default, so it takes that default rather than failing the composition; a target field that admitsnullstill receives it. - A model's
nullon a defaulted field is refused where it can be asked again: the class a worker rebuilds from a structure's JSON schema keeps a defaulted field non-nullable, so a model answeringnullthere is asked again rather than failing when its answer reaches the caller's class. A model answeringnullon a requirednative.Anythingfield is asked again in process, but across a worker boundary its schema stays open, so the step fails there. - A field whose
LiterallistsNonemay hold nothing: a Python class field typedLiteral["a", None]orobjectis read as one that may hold nothing, so a binding step reading it declares a possible absence and aPipeComposepath holding nothing gives itNonerather than its default. - Binding and batch refusals name what the author wrote: a binding step carrying a pipe step's fields proposes the remedy that applies, a dotted
batch_overfor a dottedfrom, a plain batch for a bare name, or movingnb_outputandmultiple_outputto a pipe step, and a sequence step's refusal of its batch or count fields names the step rather than blaming the pipe the step runs. At run time, a batch over a dottedbatch_overnames that path, never the private_bound_name the sequence binds it under, in its message and in the error'svariable_name, and a value there that is not a list is refused withPipeRunInputsErrorrather than an uncaughtWorkingMemoryTypeError.
Removed
ConfigModelErrorandConfigModel's conversion helpers (Breaking):ConfigModel.transform_dict_str_to_enum,transform_dict_of_floats_str_to_enum,transform_dict_keys_str_to_enumandtransform_list_of_str_to_enumare gone, and with themConfigModelError, which only they raised. A configuration model holding enums in a container annotates them withLaxEnumfrompipelex.system.configuration.config_model,list[LaxEnum[MyEnum]], and pydantic converts the strings itself.[inference.gateway_test](Breaking): the maintainer-only section whoseconfig_id_substitutionsmap swapped the Pipelex Gateway's model configurations is gone, with itsGatewayTestConfigmodel and thegateway_testfield ofInferenceConfig, as nothing has read it since the Gateway backend left in 0.73.0. Migration: runpipelex migrate, which deletes the section from the file on disk (ledger entrypipelex-config@8).
Security
- The API server refuses to start on an unknown
AUTH_MODE(Breaking): a value that is not exactlynone,jwtorapi_key, such asjwttorJWT, now stops the server at startup withInvalidAuthModeError, whose message names the valid modes, where the server used to log a warning and serve every/v1route without authentication. An unset or emptyAUTH_MODEstill meansnone. A deployment that booted on a mistyped mode sets it to the mode it meant.
[v0.81.1] - 2026-10-10
Added
- Formatted Markdown for document engines, and a
markdownfilter for plain-data templates: the document engine contract gainsformat_markdowninpipelex.cogt.doc_gen.formatted_markdown, which reads a Markdown text by the rules the built-in PDF engine prints it by and returns aFormattedMarkdown, plain data of paragraphs, headings, list items with their marker and the paragraph or heading they open with, code blocks, rules and tables whose cells carry their column's alignment, every block with the list depth and the quote depth it sits at and holding text spans with their bold, italic, strikethrough, code and link, whosestr()is its plain text and which is false when the text formats into nothing, so an engine prints Markdown formatted in its own format.make_plain_data_template_environment()registers amarkdownfilter that returns one,{{ invoice.notes | markdown }}, which is false for an empty or a blank text, so{% set notes = invoice.notes | markdown %}{% if notes %}…{{ notes }}…{% endif %}converts the text once to test and print it, and takes an optionalfinalizethrough which an engine prints it, a value printing its plain text without one. Inside a template render the conversion is charged to the render's budget; called from an engine's own code it spends from a budget of its own, as large as one render's, and an overdraft raisesMarkdownFormattingBudgetError.
Fixed
- A PDF of deeply nested emphasis prints: the built-in PDF engine no longer fails with a
RecursionErroron a Markdown text whose emphasis nests hundreds deep, such as four hundred*abefore a word and four hundredc*after it; inline markup nested past fifty levels is dropped and its text printed.
[v0.81.0] - 2026-10-10
Added
- The console shows a log call's fields and renders named layouts: the
consolesink prints a record's fields after its message as akey=valuesuffix,Scanned the inputs files=7, each value on one line and cut short past 80 characters, a string or the JSON of a list or a mapping before it is quoted so a quoted value keeps both its quotes and a cut one leaves no unbalanced quote or bracket, a path or a list of paths (a field namedfile.pathor ending in_path,_paths,_dir,_dirs,_fileor_files) at its start so the last file's name stays, a handled exception'serror.messageand a PipeDocGen template finding's text only past 2000, leaving out the run identifiers, and colours a value by its field's name, a pipe code red and a concept bold green, with any other field dimmed. A call can name a console layout withlayout=, a Rich template over its fields in which every value is escaped, starting withLogLayout.PIPE_RUNfor the pipe-run tree; every other sink writes the plain message and the fields. - Log-call conventions and their guard: the logging documentation states how Pipelex's own log calls are written, a fixed message with its values in
fieldsnamed from a shared vocabulary, an exception carried by the record rather than spliced into the text, no run identifier, payload, markup or emoji, and what each level promises.make check-log-calls(pipelex-dev check-log-calls), part ofmake agent-checkand CI, refuses an interpolated or non-literal message atINFOand above and a Rich markup tag in a message at any level, against a committed baseline of the existing calls that only shrinks:--against <ref>and--against-merge-base <ref>refuse an entry the baseline at the base revision does not list, which CI runs against a pull request's base andmakeagainst the merge base withorigin/dev.pipelex.tools.log.error_fieldsgives a handled exception's class and text as theerror.typeanderror.messagefields of a line that carries no traceback, a pydanticValidationError's text as its locations and reasons without the values it refused or the name of a key the model forbids, and any message that may quote the input, a custom validator's or one of pydantic's own templates', such as an unknown discriminator tag, as its error type alone. - Summary events for inference calls and pipe runs: every inference call, an LLM call, an image generation, an extraction, a search, a judgment or a document print, a call its checks refuse included, ends with one
INFOevent,Inference call ends, carrying the operation undergen_ai.operation.name, the model undermodel_handle,backend_name,sdk,gen_ai.request.modelandgen_ai.response.model, the tokens undergen_ai.usage.input_tokensandgen_ai.usage.output_tokensand the price undercost_usdwhere the call recorded them,duration_ms, andoutcome,success,errorwitherror.type, orcancelledfor work stopped from outside, logged by the worker bases so every provider's and plugin's worker gets it. Every live pipe run ends withPipe run ends, logged as the pipe's own span closes and carrying its ids, with the announcement's fields,duration_ms,outcomeanderror.type, so a run'sjsonoutput alone gives latency and cost per operation and model; an event that cannot be built is replaced by the warningA summary event could not be loggedand never changes the work's outcome. The console draws each on one line throughLogLayout.INFERENCE_CALL_ENDandLogLayout.PIPE_RUN_END; a document engine overridesrenderalone, and the print stage that calls it logs the event. - Request- and page-priced usage: a usage says what its counts are in
pricing_unit(PricingUnitinpipelex.cogt.usage.pricing_unit),token,requestorpage, and a worker billed by the request or by the page records it withrecord_unit_priced_usage(pipelex.cogt.usage.usage_cost), as the Linkup workers and the extraction page fallback now do, so the summary event carries the call's cost and no token counts.
Changed
- Log lines carry a fixed message and their values as fields (Breaking): every
INFO,WARNING,ERRORandCRITICALline of Pipelex and of its API server has one fixed sentence for its message and carries what varies, a pipe code, a model handle, a file path, a handled error'serror.typeanderror.message, as fields named from the vocabulary in the logging documentation, so a log store groups a line by its message and filters it by its fields, and the baseline ofmake check-log-callsis empty, every call being held to the conventions. A line nobody can act on moved down toDEBUG, a lost inference usage record up toERROR, an exception logged and then re-raised, a cleanup's failure while another error propagates and a swallowedValidationErrorcarry no traceback, whose text could quote a run's inputs, so a failedpipelex runprints one, andpipelex runandpipelex graph renderlog an unexpected failure atERRORwith its traceback, which the console renders under the line without its local variables and a structured sink writes to the log store, while the command's own output names the exception's type and message whatever the sink and the level. What breaks is a log query: the API server's error record drops itsevent: api_errorfield and renamesroute,user_id,error_type,status,providerandmodeltourl.path,user.id,error.type,http.response.status_code,backend_nameandmodel_handle, the API server no longer writes a refuseduser_idclaim orX-User-Idheader underuser.id, the claim's type ridingclaim_typeinstead, and the warning that a configuration file is out of date is one line per stale file, the file infile.path, where it used to list them all in one. - Debug and verbose log output: the
DEBUGandVERBOSElines are fewer and shorter, each naming what it is about rather than narrating every step, and none carries a payload any more: no base64 data, prompt, rendered template, raw response or model dump. An LLM call's reasoning settings are logged once per call, atDEBUGwith each setting sent as a field, rather than twice atVERBOSE, and the error for undecodable base64 input gives the input's length instead of its first bytes. - The pipe announcement is a plain message with fields: every live pipe run logs
Pipe run startswithpipe_type,pipe_code,output_conceptandpipe_depthas fields, where thejson,otlpandgcpsinks used to receive Rich markup inside the message, and the console draws the same coloured pipe tree from those fields throughLogLayout.PIPE_RUN. The warning that a model's fixed temperature replaced the requested one carries its values as fields in the same way:model_handle,backend_name,sdk, the model's handle undergen_ai.request.modeland the provider's model id undergen_ai.response.modelas on the LLM call's span, the requested temperature undergen_ai.request.temperature, andfixed_temperature; the verbose line saying a model's settings were not checked, since its backend's SDK is not installed, carries the same model fields. - The default output logs every inference call and every live pipe run: at the default
INFOlevel an application embedding pipelex now gets anInference call endsline after each inference call and aPipe run endsline after each live pipe run. To quiet either, raise its logger's level in[runtime.log.package_log_levels]:pipelex-cogt-inference-inference_call_summary = "WARNING"for the inference event, andpipelex-pipe_machinery-pipe_abstract = "WARNING"for the pipe-run end, which quiets the pipe announcement too, since the two share that logger. log.set_level_by_nametakes only Pipelex level names (Breaking): it accepts aLogLevelname in any case and raisesValueErrorfor any other, so the stdlib spellingsWARN,FATALandNOTSETit used to accept are refused. PassWARNINGforWARNandCRITICALforFATAL;NOTSEThas no counterpart,VERBOSEbeing the lowest level.
Fixed
- Model descriptions and credential errors carry no backslashes: a model's description in an error message or a log line, such as
gpt-5 → SDK[openai]•Backend[openai]•Model[gpt-5-2025], no longer has a backslash before each bracket, and the error naming a backend's missing credentials tells you to addenabled = falseunder[openai]rather than\[openai].pipelex runprints a failed run's message exactly as written rather than reading it as markup. - No HTTP request line per inference call: at the default
INFOlevel an inference call made through openai 3.x no longer logs anHTTP Request: POST https://… "HTTP/1.1 200 OK"line, carrying the provider's endpoint URL and the HTTP status, on the console or in thejsonsink. The defaults under[runtime.log.package_log_levels]now givehttpx2andhttpcore2, the HTTP client forks openai 3.x sends its requests through, the levelshttpxandhttpcorealready had,WARNINGandINFO; sethttpx2 = "INFO"there to see the line again. - An empty list's output panel says so: the title of the panel showing an operator pipe's output ends with
[empty list]when the pipe produced an empty list, a label the console used to drop, as it already showed[1 item]or[3 items].
Removed
- The DEV log level (Breaking):
LogLevel.DEV,LOGGING_LEVEL_DEVandlog.dev()are gone; the levels areVERBOSE,DEBUG,INFO,WARNING,ERROR,CRITICALandOFF, and code callinglog.dev()callslog.debug()instead. A configuration file namingDEV, asdefault_log_levelor as a level under[runtime.log.package_log_levels], is migrated in memory at boot by ledger entrypipelex-config@6, which reads it asINFO, the level that lets through the same records now that nothing logs atDEV, and the boot warns that the file is out of date. Migration: runpipelex migrate, which rewritesDEVtoINFOin the file on disk. - Logging helpers nothing called (Breaking):
FuncRegistry.log()andFuncRegistry.set_logger(), with thefunc_registrylogger they wrote to,InferenceManager.print_workers()andLogLevel.from_int()are gone, with no replacement. Code that needs aLogLevelfrom a stdlib level number maps it itself. - Cost helpers outside
pipelex.cogt.usage.usage_cost(Breaking):compute_tokens_usage_costis imported frompipelex.cogt.usage.usage_costrather thanpipelex.cogt.usage.cost_registry,CostRegistry.compute_total_costis gone forcompute_total_costin the same module, and theTokensUsagealias ofpipelex.reporting.reporting_typesis gone: importTokensUsagefrompipelex.cogt.usage.usage_cost, where it is a plain union of the usage types without the old discriminator, or use the discriminatedAnyTokensUsage, which stays inpipelex.reporting.reporting_types. is_markup_enabled(Breaking): the console reads no log message as Rich markup, so the[runtime.log.rich_log]key that switched it is gone, along withVERBATIM_MARKinpipelex.tools.log.log_fields. A message now prints exactly as written, so alist[int], a bracketed path, the[myapp.jobs.nightly]:prefix of a logger named in lowercase and the[cycle]marker of a circular content no longer vanish, while a message of your own that relied on markup for its colour shows its tags and should put its values infields. Migration: runpipelex migrate, which deletes the key from the file on disk (ledger entrypipelex-config@7).
[v0.80.0] - 2026-10-09
Changed
- MTHDS standard 4.0.0 (Breaking): pipelex now requires
mthds==0.20.0andpipelex-sdk==0.37.0, the releases that report MTHDS standard4.0.0, the language this runtime has implemented since v0.75.0, so everyone downstream inherits those exact versions and a project that pins either package itself has to move with them.pipelex --versionprintsmthds-standard 4.0.0, every crate pipelex normalizes is stamped withmthds_version4.0.0while crate fingerprints stay the same, and a package whoseMETHODS.tomlconstrainsmthds_versionto^3.0.0,~3.0.0or exactly3.0.0now logs a warning when it loads, which>=3.0.0or^4.0.0does not. - Stored names are plain input names (Breaking): every name under which a step stores a value in working memory, a
PipeSequencestep's or aPipeParallelbranch'sresult, thebatch_asof either, and aPipeBatch'sinput_item_name, must now be a plain input name matching[a-z][a-z0-9_]*, as a binding step'sresultalready had to, and a name in any other form, such asPages,doc.pagesor_pages, is refused asinvalid_input_namewherever a bundle loads, and by the MTHDS JSON Schema, which now carries the input-name pattern on these fields. Rename such a name to a plain one, such aspages, along with every input, binding path andbatch_overthat reads it; a plainbatch_overis still refused only when it starts with the reserved_bound_. - Hosted-run guidance in messages and help: the next step of a refused Pipelex API key names
pipelex loginand the working directory's.envrather than a shell export, which a.envfile setting the key overrides, andpipelex loginnames[run] execution = "hosted"rather thanpipelex initto make hosted runs the default. A CLI run missing a provider key, and thepipelex doctorreport, offer the hosted Pipelex API as a way out; a failedpipelex-agentboot points atpipelex-agent migrateandpipelex-agent initrather than the interactivepipelex init config; andFormerReleaseConfigErrorcarries its remedy,pipelex migrate, as itsuser_action. Therunhelp of both CLIs names hosted runs, catalog ids, local method directories and TOML or inline inputs, andpipelex-agent --helpdescribes the program rather than printing an internal note.
Fixed
- The
jsonlog sink keeps each field's type: a field value JSON refuses outright, a mapping with a non-string key or a circular reference, is written as itsrepralone, where it used to turn every other field on the line into itsrepras well, so a number or a boolean beside it arrived as"7"or"True"and a query over that field missed the line. The other fields now keep their JSON types, as they already did under thegcpsink.
[v0.79.1] - 2026-10-09
Fixed
- The
wait_and_retryadvice says to run again: theuser_action.detailof a transient inference failure said "the system will retry automatically", which no longer held by the time anyone read the failed run's report, and now says to wait, at least the provider'sRetry-Afterdelay when it gave one, then run it again; the advice on an empty or malformed Mistral response says the same. - A malformed
Retry-Afteris no delay: a provider'sRetry-Afterthat is negative or not a finite number (inf,nan,1e999) is ignored rather than taken as a delay, so the advice says to wait a moment instead of naming an impossible wait.
[v0.79.0] - 2026-10-09
Highlights
Run any method on the hosted Pipelex API with nothing but a Pipelex API key. pipelex run --hosted sends your bundle, a published address or a catalog id (mt_…) to the hosted API and saves the outputs where a local run would, with no provider key and no inference configuration on your machine. pipelex login gets the key through your browser, pipelex init offers the hosted API as its default, and Python code runs there directly with pipelex-sdk's PipelexAPIClient, which now ships with pipelex.
Added
- Hosted runs with
--hosted,--localand[run] execution(Breaking):pipelex run method|pipe|bundleandpipelex-agent runrun on the hosted Pipelex API under--hostedand on this machine under--local, the new[run] executionsetting (defaultlocal) choosing when neither is given, and--base-urlorPIPELEX_BASE_URLnaming another server. A hosted run boots nothing locally, uploads the local files its inputs name, prints the run id as soon as the run is acknowledged and names it in any later failure, Ctrl-C included. The agent CLI's--runner pipelex|apibecomes--runner local|hosted, on pipelex-sdk'sPipelexAPIClient, whichpipelexnow pins exactly. pipelex login: gets a Pipelex API key through the browser, or from a hidden prompt with--paste, checks it against the hosted API and saves it to~/.pipelex/.envasPIPELEX_API_KEYwithout ever printing it. It warns when a working-directory.envwould override the saved key, andPIPELEX_APP_URLaims it at another Pipelex app.pipelex initasks where runs execute: Enter takes the hosted Pipelex API, which signs you in and needs no provider key, while the other answer sets up local backends and credentials as before; the answer is saved as[run] execution.pipelex doctorandpipelex-agent doctorjudge a hosted setup by its API key,pipelex doctor --fixtakes the hosted default only for a brand-new home, andpipelex-agent initaccepts"execution": "hosted"in--config.pipelex migrateremoves what a former release left for the Pipelex Gateway: a configuration written for the Pipelex Gateway is refused at boot with the newFormerReleaseConfigError, which points atpipelex migrate. That command,pipelex-agent migrate(underformer_release),pipelex initandpipelex doctor --fixremove the Gateway's backend, routing profiles, routes and files, keeping Pipelex Manifold's and a copy of every file they change.- Shared helpers in the document engine contract:
pipelex.cogt.doc_gen.layout_display(display_scalar,is_numeric_column,markdown_as_html) andpipelex.cogt.doc_gen.template_environment(make_plain_data_template_environment()) give a plugin's engines the value display, Markdown conversion and sandboxed, strict Jinja environment of the built-in PDF engine, without importing ReportLab or Pipelex's internals.
Changed
RenderResources.loadreturns the media type with the bytes (Breaking): a document engine'sloadreturns aLoadedResourceholdingdataand themime_typeits source declares (anhttps://response'sContent-Type, adata:URL's type, or the storage provider's type for apipelex-storage://key), elseNone.PLUGIN_API_VERSIONmoves to 6, so a plugin declaringtargets_api = 5is refused at discovery until it is rebuilt to read.data.
Fixed
- A partial home configuration directory gets the kit's missing files: every boot copies into
~/.pipelex/(orPIPELEX_HOME) each kit file it lacks, never overwriting a file or writing through a symbolic link, the inference files as one unit only wheninference/backends.tomlis absent. A directory holding anything at all, such as an earlier release'spipelex_service.toml, was left as it was, and every boot then failed with "Config files are missing for the inference backend library". - The internal backend's models are served whatever the routing profile: a profile routing a model's name to no enabled backend left the internal models (
reportlab-pdf,pypdfium2-extract-pdf,docling-extract-textand those plugins add) unserved, so local extraction and document steps were refused at load, as under the defaultall_enabled_backendsprofile with only the internal backend enabled. The internal backend now serves such a name, while a route sending it elsewhere still decides and disabling the backend still withholds its models. pipelex-agent init --helpprintslist[str]: the--confighelp showed the schema's"backends": list[str]aslist, because Rich read[str]as markup.
[v0.78.0] - 2026-10-08
Highlights
A client can ask the runner what a model reference means. GET /v1/models/check answers whether a reference resolves, as what kind, and to the model a run would call, from pipelex's own parser and deck, so a client keeps no grammar of model references of its own.
A handle names one model per model type. One handle can be an LLM and a judgment model at once, and a pipe naming a model served only as another type is refused when the method loads rather than at its first call.
Added
- Claude Haiku 5.5 on the
anthropicbackend:claude-5.5-haiku(model idclaude-haiku-5-5) takes text, images and PDF, produces text and structured output, and reasons with adaptive thinking, with up to 128K output tokens. It costs $0.10 / $0.50 per million tokens for a prompt up to 100,000 tokens and $0.50 / $2.50 above, of which the backend file records the first. It refusestemperaturebut, unlike Sonnet 5.5 and Opus 5.5, accepts a forcedtool_choice, so structured output keepsinstructor/anthropic_tools. - One handle, one model per model type: two models of different model types may share a handle, so
gpt-6-lunacan be an LLM and a judgment model at once, and each pipe reaches the one its family asks for. A backend file declares the second model of a handle on a table of its own with the newhandlekey, which may not appear in[defaults], and a file declaring one handle twice as the same type fails to load, naming both tables. A plugin may declare an internal model under a nameinternal.tomlalready declares as another type. GET /v1/models/check, the model reference check: the server answers whether one model reference (?reference=, optionally in one?type=category,doc_genincluded) resolves on this runner, as a200verdict discriminated onresolution(resolvedornot_found) that carries the reference'skindandname, one match per category where it resolves with the model a run would call (resolves_to,nullwhere its binding reaches no model the runner can call) and itstarget,description,fallbacksorvia, and, onnot_found, thesuggestions,other_kindsandother_categoriesa failing validation of the same reference offers, written as references. It answers from pipelex's own parser and the deck lookups a validation runs, so a client no longer keeps its own grammar of model references, and it is a Pipelex API extension, not taggedx-mthds-protocol. A missing or repeatedreferenceor a repeatedtypeis a422ValidationError, a reference that is blank, a sigil or a namespace alone, or longer than 199 characters once trimmed is a422of the newerror_typeInvalidModelReference, and an unknown or emptytypeis a422InvalidModelCategory. The library serves the same check ascheck_model_referenceinpipelex.cogt.models.model_reference_check, and the suggestion rule asfind_model_alternativesinpipelex.cogt.models.model_suggestion.
Changed
- A model served only as another type is refused when the method loads (Breaking): a pipe naming a handle the deck serves only as another model type, such as a
PipeJudgenaming an LLM, is refused bypipelex validate,POST /v1/validateand a run's load as anunknown_modelitem saying the deck serves the handle but not as the type the pipe needs, whose suggestions list only models of that type, where it passed validation and failed at its first call; a deck preset naming one is reported at boot in the same words (a warning, or a refusal undermissing_presets_reaction = "raise"), and an LLM override naming one refuses boot. A bare name resolves as a model of the pipe's type, then an alias of that type, then a waterfall of that type, an order the validation and the run's lookup share, so a handle served as another type no longer hides an alias or a waterfall of the same name, and the validation and the setting a pipe'smodelfield is read into accept the waterfall only while model fallback is on.ModelDeckgainsis_bare_handle_resolvable,is_reference_defined,get_presets_for_typeandpeek_inference_model, the run's model lookup without its logs. - Model specs are keyed by model type and handle (Breaking):
InferenceBackend.model_specsandModelDeck.inference_modelsare the newModelSpecIndex, read throughget(model_type=…, handle=…),handles_of_type,all_specs,all_handlesandtypes_serving, andInferenceBackend.get_model_spectakesmodel_typeandhandle.InferenceBackend.list_model_names,InferenceBackendLibrary.list_all_model_namesandInferenceBackendLibrary.get_all_models_and_possible_backendsare removed.PluginModelDeclarations.internal_modelsis a tuple ofPluginInternalModel, which carries itsname, andDuplicateInternalModelErrornames themodel_typetwo plugins both declared. - Usage is reported per model type and name (Breaking): a run calling both kinds of one handle reports two rows in the cost table and the agent CLI's cost summary, and two entries in the graph's usage attribution, where they merged into one row typed by whichever call came last.
AggregatedCosts.grouped_by_modelis keyed by the newModelUsageKey, and itsmodel_typesis removed. handle:reads no sigil (Breaking): ahandle:reference writes a bare name without reading a sigil in the validation, the run and the check alike, sohandle:@nameis the bare name@name, not the aliasname, and like any bare name it resolves as a model, then an alias, then a waterfall of that name, a validation refusing it where the deck has none.
Fixed
- A
~namemodel reference runs the waterfall: a pipe naming~nameorwaterfall:namecalled a model or an alias that shares the waterfall's name, because its setting kept the bare name, and every model type's setting now keeps the sigil, so a run calls the model the validation andGET /v1/models/checkname; theresolved_modelevery pipe reports, and aPipeLLM'sresolved_model_for_object, now write such a waterfall~name, and so does a refusal of a file input naming one. - A waterfall step that leads back to itself or runs out serves no model: an alias and a waterfall leading to each other, or a waterfall listing its own name, overflowed the stack, and a step naming another waterfall, directly or through an alias, none of whose models is served failed its waterfall with that inner waterfall's error. Such a step, whether it leads back to an alias or a waterfall being resolved or reaches a waterfall that runs out, now serves no model, so the waterfall goes on to its next step, and a run fails with the waterfall's own error only when none of its steps serves a model.
pipelex-agent models --backendlists past a binding no backend serves: an alias or a preset bound to a waterfall none of whose models the deck serves failed the listing withModelWaterfallError, and it is now left out like any binding of another backend, the listing reading the deck without a run's logs.pipelex-agent check-modelapplies the validation's rule: a bare alias name, or a bare waterfall name while model fallback is on, is nowvalid, aspipelex validateandGET /v1/models/checkanswer, and--typetakesdoc_genas the check does.
[v0.77.0] - 2026-10-07
Added
- Claude Opus 5.5 and Claude Sonnet 5.5 on the
anthropicbackend:claude-5.5-opus(model idclaude-opus-5-5, $4 / $20 per million tokens) andclaude-5.5-sonnet(model idclaude-sonnet-5-5, $2 / $10) take text, images and PDF, produce text and structured output, and reason with adaptive thinking, with up to 128K output tokens each. Both refusetemperatureand a forcedtool_choice, so structured output usesinstructor/anthropic_reasoning_tools, asclaude-5.1-fabledoes. - GPT-6.1 Sol and GPT-6 Luna on the
openaiandazure_openaibackends:gpt-6.1-sol($2 / $10 per million tokens) andgpt-6-luna($0.10 / $0.50) take text, images and PDF, produce text and structured output, reason with a manual effort and fix their temperature at 1, like the rest of the GPT-6 series. Onazure_openaitheir model ids are the deploymentsgpt-6.1-sol-2026-09-29andgpt-6-luna-2026-09-22. The shipped deck is unchanged.
Fixed
- A reasoning setting reaches a structured output: a pipe whose model setting carries
reasoning_effortorreasoning_budget, the deck's$deep-analysis,$quick-reasoningand$retrieval-premiumpresets included, now generates its structured output with that setting on every provider, where every language-model worker refused it withLLMCapabilityError"does not support reasoning parameters for structured generation". A model refuses a reasoning setting on a structured output only where it refuses it on text, or on Mistral where its structure method is notinstructor/mistral_tools, whose JSON parsing cannot read a reasoning reply; a structured call on an Anthropic tool method with thinking on leaves the tool choice to the model, since a forced tool choice cannot carry thinking. - A manual thinking budget leaves room for the answer and stays in the model's range (Breaking): an Anthropic or Gemini thinking budget, resolved from an effort or set explicitly, is cut to leave a quarter of
max_tokensfor the answer, where it could take all but one token of it, and held within the range the model declares with the newmin_thinking_budgetandmax_thinking_budgetvalued constraints. The kit's Anthropic models declare the API's minimum of 1,024 tokens, so the 512 thatminimalmaps to is raised to it, and its Gemini 2.5 models declare their ranges, somaxeffort's 65,536 tokens, which every Gemini 2.5 model refused, are cut to the model's maximum even without amax_tokens. Amax_tokenstoo small to hold the model's minimum beside the answer reserve is refused with anLLMCapabilityErrornaming the numbers, where the request was sent, and a reasoning effort ofnoneon a Gemini model that always thinks, which the kit marks with the newthinking_cannot_be_disabledlisted constraint, is refused rather than sent as a budget of 0 that Gemini answers with a 400; a project that copied the kit'santhropic.toml,bedrock.tomlorgoogle.tomlgets the bounds by declaring them in its copy. - Mistral reasoning is sent as
reasoning_effort: a reasoning effort on a Mistral model is sent asreasoning_effort = "high", the only reasoning value Mistral's reasoning models accept, where it was sent asprompt_mode = "reasoning", which every current Mistral model refuses, so any reasoning effort on Mistral failed. Thereasoninglevel of[inference.llm.mistral.effort_to_level_map]is unchanged. - A model setting its model refuses is refused when the method loads (Breaking): a
PipeLLMorPipeStructurewhose model setting the model it resolves to cannot take, such as a reasoning setting on a model declaringthinking_mode = "none", a reasoning budget on an OpenAI or Mistral model, a thinking budget itsmax_tokenscannot hold, or a structured output on a Bedrock native model, is refused bypipelex validate,POST /v1/validateand a run's load as an item of the newPipeValidationErrorTypevaluellm_setting_refused_by_modelon itsmodelormodel_to_structurefield, where it validated as runnable and failed at its first call. The check is the newcheck_requestclassmethod of the LLM workers, whichLLMWorkerAbstractnow runs before every provider call and which an LLM backend registers for validation through the new optionalcheck_llm_requestofPluginRegistrar.add_inference_backend, whoseCheckLLMRequestFn,LLMJobParamsandLLMCapabilityErrorthe Inference SPI table publishes; a backend registering none, an external plugin's included, is held to the rule every worker shares, that a model declaringthinking_mode = "none"takes no reasoning setting. The item names the model by its deck handle and is shown even under strict error disclosure, so an external plugin's refusal is shown as written only when the plugin raises it as caller-facing copy withas_caller_fault(), and by its title otherwise. - A structured output on Google without a system prompt no longer fails: a structured call on the
googlebackend whose prompt has no system text failed withTypeError"generate_content() got an unexpected keyword argument 'system'", and is now sent with no system instruction. - A model listing
temperature_unsupportedgets no temperature on any SDK: a model whose backend file lists thetemperature_unsupportedconstraint is now called without a temperature on the OpenAI Responses, Google, Mistral and Bedrock-native SDKs too, where those sent one anyway and only the Anthropic and OpenAI chat-completions SDKs honoured the constraint. The model spec answers it as the newaccepts_temperatureproperty, and a Bedrock client'schatnow takestemperature=Noneto send none. - A binding step survives a kajson round trip:
BindingStepBlueprintis encoded underfrom, the key it validates, where kajson wrote its field namefrom_path, which decoding refused as an extra field. A library crate holding a sequence with a binding step can now cross a process boundary, so a run of such a method no longer fails to start on a Temporal worker with "Could not instantiate pydantic BaseModel … BindingStepBlueprint".
[v0.76.0] - 2026-10-06
Added
- A run request carries the method packages its bundle calls (Breaking): a
filesorbundle_b64request to/v1/executeor/v1/startmay ship each package its bundle calls by address under.mthds/methods/<name>/at its root, with itsMETHODS.toml, and the server searches them before its installed copies and fetch-on-miss, for that run alone, so a shipped package wins over an installed copy and runs with fetching off; a package fetched bymethod_refthat ships its own.mthds/methods/is handled the same way. Those files no longer join the bundle's own.mthdscontent or its library directory, any other.mthdspath in a bundle is refused with a422 InvalidBundlenaming the entry, as are two shipped packages declaring the same address and name without regard to case, and on a sandbox-hosted deployment a shipped package declaring a Python structure class is refused with403 MethodStructuresRefusedError. The engine takes these directories as the new keyword-onlymethods_dirsofPipelexMTHDSProtocol,pipeline_run_setup,acquire_libraryandLibraryManagerAbstract.load_from_blueprints, which every implementation of the abstract must accept. - The MTHDS Test Corpus covers a sequence step's multiplicity: the invalid entry
invalid_input_stuff_spec_mismatch_step_multiplicityhas a step read as a singlePagethe pages an earlier extraction stored asPage[], and fails withinput_stuff_spec_mismatch; the valid offline entrycontroller_sequence_house_rulesdeclareshouse_rules = "Markdown"for a sequence whose first step reads it asMarkdownand whose second reads it asText, and runs it. - The MTHDS Test Corpus covers the
markdowntemplate filter: the new hand-maintainedfeature.markdown_filtertag names the filter of HTML templates that turns Markdown held in a plain text field into HTML, and the offline-tier entryfeature_markdown_filter_cake_order_slipcovers it with a PipeCompose run that pins its output, raw HTML in the source shown as text and only a URL with a scheme made a link.
Changed
- The documentation no longer describes the Pipelex Gateway: Configure AI Providers offers your own provider keys or a local model, says that the shipped defaults run as they are on OpenAI or Azure OpenAI and must be repointed for any other provider, OpenRouter included, and points to the Quick Start for running on the hosted API; the MTHDS language tutorial no longer runs
pipelex login. The telemetry, logging, feature and under-the-hood pages describe only the telemetry streams you configure yourself and the backends the runtime ships, without the Gateway stream, its backend or its plugin. - A call into a method package that cannot be resolved is a verdict item (Breaking): when a bundle names a pipe of another package by its address and the package is not installed and cannot be fetched (fetch-on-miss off, an address this runtime cannot fetch, a failed fetch), the load is refused with one item of the new
PipeValidationErrorTypevalueunresolved_package_dependencyper package, on the first pipe naming it, with the reference as written inmissing_pipe_code, where it raisedMethodFetchDisabledErrororMethodDependencyFetchErrorand the API answered a no-verdict500.POST /v1/validatenow answers200withis_valid: falseand the run routes a422, every package of the bundle is tried before the refusal, a failed clone's message no longer names the runner's temporary clone directory, and the remedies name no directory on the host and include shipping the package with the bundle under.mthds/methods/<name>/. The MTHDS Test Corpus covers it asinvalid_unresolved_package_dependency. - A package bundle that does not parse refuses the load (Breaking): when a bundle of a package the consumer calls by address does not parse, the load is refused with the parser's items, each naming the file as
<address>/<path inside the package>and never by its path on the host, whether the package was shipped with the bundle, installed or fetched. It used to be skipped with a warning, so the caller learned of it from a later refusal about a pipe.
Fixed
- A validation message no longer names one concept as both required and provided: the
(required: …, provided: …)suffix of a validation item is left off when the provided concept is among the required ones, since the disagreement then lies elsewhere, so a step reading onePagewhere an earlier step storedPage[]no longer ends its message with(required: native.Page, provided: native.Page). - A provenance sidecar that is not UTF-8 no longer breaks method discovery: an installed or shipped method package whose
.provenance.jsonis not valid UTF-8 is discovered with no recorded provenance, as one that does not parse already was, where discovery raisedUnicodeDecodeErrorand a run calling the package answered a500. - A
filesentry holding a lone surrogate is refused as an invalid bundle: a name or a text in a run request'sfilesmap holding an escaped lone surrogate such as"\ud800", which JSON carries but UTF-8 cannot encode, is refused with a422 InvalidBundlenaming the entry, where the server answered a500. - A bundle naming one path twice is refused: a run request whose bundle holds two entries for one file, spelled differently in a
filesmap (a/./bbesidea/b) or twice in a zip, or a file that is also the directory of another entry, is refused with a422 InvalidBundlenaming the path, where the engine silently took the last copy of a file named twice and the server answered a500for a file that is also a directory.
[v0.75.0] - 2026-10-06
Added
- The binding step: a PipeSequence step
{ from = "invoice.total", result = "total_amount" }binds a deep copy of the value at a path in working memory to a new name, as a fresh stuff whose concept is derived from the declared structures the path walks, a path crossing a list binding one flattenedX[], and a later step reads it under that plain name. The sequence checks a step reading a binding, or a binding ending it, against the derived concept and multiplicity, a path reaching nothing records an absence under the optionality model, and the execution graph draws each binding as a node of the newNodeKindbinding. A malformed step is refused as the newPipeValidationErrorTypevaluebinding_step_invalid, which the MTHDS JSON Schema refuses too, and a path the declared structures cannot walk asbinding_path_unresolved; the MTHDS Test Corpus covers both under thefeature.binding_steptag, and its offline-tier entries now run with theirinputs.jsonin the test suite. pipelex fix bundledeletes a dotted input key beside its root: the new ruledelete-redundant-dotted-inputremoves a key such as"page.page_view" = "Image"from a pipe'sinputswhen the same table declares its rootpage, which already supplies every field a template reads through it. The fix is safe, and applied, only when the root's declared concept holds the field the key names and the root keeps the presence marker and multiplicity the table gave it: a root declared with a concept that lacks the field, such asdata = "Number"beside"data.text" = "Text", or a key that set the markers otherwise, such as"data.text" = "Text!"declared afterdata = "Text?", gets an unsafe fix thatpipelex fix bundlenever applies, and both the fix and the refusal name what differs, with the edit that settles it on the root (here, replace?with!ondata). A lone dotted key gets no automatic fix, since nothing states its root's concept, and its refusal names the two remedies instead.
Changed
GraphAnalysis.stuff_producerslists every producer (Breaking): it maps a digest to a list of node ids,get_produceris replaced byget_producers, and the newshared_stuff_controllersnames the controller a stuff with several writers belongs to, a parallel or batch controller combining or aggregating into the stuff counting as one of them. The Mermaid graph draws such a stuff once, inside that controller, with an edge from each producer.InferenceBackendLibrary.loadtakescredentials(Breaking): itslenientflag is replaced bycredentials: CredentialResolution,REQUIREorSKIP. A backend aSKIPload built records what it left unresolved inInferenceBackend.unresolved_credentials, andModelManager.get_required_inference_backendrefuses it with anInferenceBackendCredentialsErrorof typenot_resolved_on_keyless_bootthat names the variables.- A routing profile naming a disabled backend is refused on every boot (Breaking): a boot without inference, such as
pipelex validate, now refuses a routing profile whose default or routes name a backend that is not enabled, as a boot with inference already did, andload_active_routing_profileloses itslenientargument. - A variable in a field that describes a model is refused on every boot (Breaking): a
${…}placeholder may stand only in a value a call sends, which meansapi_key,endpointand a backend's extra keys, or a model'smodel_id,endpoint_pathand request headers. A placeholder in a field such asmodel_type,sdk,thinking_modeor the constraints is now refused with an error naming the field, where a boot with inference substituted it and a boot without inference could not, so the two gave different verdicts. - An unknown variable prefix is refused wherever it stands in a fallback chain (Breaking):
${env:KEY|secrte:KEY}is refused for itssecrteprefix whether or notKEYis set, where a boot read the candidates in turn and accepted the chain whenever an earlier one resolved. A boot without inference, which resolves nothing, refuses it too, so a file gets the same verdict on its syntax on every boot and every machine. pipelex show backendslists every backend with its credential variables: a backend whose key is unset is listed instead of hidden, a new column names the variables each backend reads its credentials from, and an endpoint read from a variable is shown as not resolved.- An input name is a plain name (Breaking): every key of a pipe's
inputs, on every operator, controller and signature, and a PipeBatch'sinput_list_namemust match[a-z][a-z0-9_]*, so a dotted name such as"invoice.total" = "Number"or a malformed one such asInvoiceTotalis refused when the bundle is parsed, as the newPipeValidationErrorTypevalueinvalid_input_name, which also replaces PipeJudge's own refusal that reached callers with noerror_type. Its message names the remedies: declare the root with its whole concept (invoice = "Invoice") and read the field in the template ($invoice.total), which every operator handles by the concept the path reaches, so@page.page_viewwithpage = "Page"is still attached as an image, or have the calling sequence bind the field with a binding step ({ from = "invoice.total", result = "total_amount" }). The generated MTHDS JSON Schema refuses the same names, throughpatternPropertieson everyinputstable and apatternoninput_list_name, and the corpus vocabulary recordserror.invalid_input_namewithfails_at = "schema". The dotted-input fold goes with them:InputStuffSpecsno longer rewrites an input key to its root,NamedStuffSpec.requirement_expressionis gone,pipelex.pipe_machinery.validationdropsis_variable_satisfied_by_inputs,validate_input_nameand thedotted_input_supplies_its_pathargument ofcheck_inputs_match_variablesandcheck_variables_are_declared, in favour ofvalidate_input_namesandcheck_input_list_name, andbuild_judgment_materialno longer walks a dotted input name as a path. - A sequence's steps include binding steps (Breaking):
PipeSequenceBlueprint.stepsholdsBindingStepBlueprintbesideSubPipeBlueprint, andPipeSequence.sequential_sub_pipesholdsBindingStepbesideSubPipe, so code readingpipeorpipe_codeoff every step reads the newpipe_stepsproperty instead. A PipeParallel'sbrancheskeep the pipe-step shape. - Names a nested controller stores (Breaking): a name a nested sequence, a condition's outcomes or an
add_each_outputparallel always stores in the caller's memory counts as stored by the step calling it, so it is no input of the caller, and a sequence that declared such a name only to pass validation is now refused asextraneous_input_variable: remove the declaration. A nested sequence or condition skipped for an absent plain input now records an absence for every name it always stores, so a later step finds a value or an absence there, never nothing. - Every sequence step is checked against what it reads (Breaking): validation follows a PipeSequence step by step and refuses, as
input_stuff_spec_mismatch, a pipe step whose pipe reads a name as a concept or a multiplicity the value stored there does not have, whether the sequence declared it or an earlier step stored it, so a step readingcv_pages = "Page"where an extraction storedPage[]now fails before anything runs: declarePage[]. A declared input every step reading it accepts validates even when it differs from what the last one reads, so a sequence whose first step readsnoteasMarkdownand a later one asTextdeclaresnote = "Markdown", which the sequence's inputs check used to refuse for not being the last step'sText, and that declaration is what the sequence needs from its caller, so a step calling it with aTextundernoteis refused. A step calling a nested controller is checked against the inputs the controller declares. A value a pipe step stores asAnythingorDynamic, as a condition whose outcomes produce different concepts declares its output, is assumed to be the concept a later step reads, its multiplicity still checked, while a sequence's own input declaredAnythingand a binding's value are checked as any other, also where a condition whose outcomes do not all store the name may leave them in place. A batched last step yieldsX[]whatevernb_outputit asks for, so a sequence ending with one declares a variable list, where a fixed count or a single output used to pass.PipeLibraryAbstractandConceptLibraryAbstractgain the abstract propertystate_token, which every change to a library replaces, so a sequence builds its typed flow once per state of the libraries it resolves in. - A dotted
batch_overis a binding followed by a batch (Breaking): a sequence step'sbatch_over = "catalog.pages"binds the list at that path under a private name,_bound_catalog_pages, by every rule of a binding step, then batches over it, so a list root such ascatalogs.pagesbatches over the pages of every catalog, the sequence needs the root as its own concept, and the execution graph draws the binding feeding the batch. A path deriving a single value is refused before the run asinput_stuff_spec_mismatch, as a batch over a value that is not a list is, and a path outside the path grammar, or a dottedbatch_overon a PipeParallel branch, asbinding_step_invalid, which the MTHDS JSON Schema refuses too. The file-format check before the run follows each item of the bound list back to where it sits, as it does after an explicit binding step, so a document atcases[1].transcripts[0]that the batched pipe's model cannot read is refused before the batch starts. The_bound_prefix is reserved for these private names, since a nested sequence binds in its caller's working memory and would otherwise overwrite a caller's value stored under one: a pipe step's or a PipeParallel branch'sresult,batch_asor plainbatch_over, or a PipeBatch'sinput_item_name, starting with_bound_is refused asinvalid_input_name, which the MTHDS JSON Schema refuses too.
Fixed
- A batched step's other inputs: a sequence now needs every input a batched step's pipe reads besides its item even when an earlier step stored the list, where it needed them only for a list the sequence received, so declaring one was refused as
extraneous_input_variableand leaving it out failed the run inside the batch. - A PipeBatch branch gets its own copy of its item: each branch's item is taken from the branch's own copy of the batched list, so a branch that rewrites its item in place, as a
PipeFunccan, no longer changes the batch's input list. - A dry-run condition's output is one stuff every outcome produces: in the graph of a dry run, every outcome of a
PipeConditionnow carries the condition's output digest, so the step reading the condition's result is wired to all of them rather than to the outcome whose pipe code sorts last, and when the outcomes write or declare different outputs, the condition's declared output types that stuff on every node outside the outcomes: the condition, the controllers whose output it is, such as a sequence ending on the condition, and the steps reading it, while each outcome keeps the concept it writes. Each outcome but the last now runs on a copy of the memory the condition received, so an outcome reading the condition's slot reads the value from before the condition rather than a sibling's output, and the default outcome runs last, so the steps after the condition read its value. - Validation and dry runs no longer depend on which keys the machine holds: a boot without inference (
Pipelex.make(needs_inference=False), used bypipelex validate,pipelex run --dry-run,pipelex showand the agent CLI's validation) now loads every enabled backend with its models and constraints and resolves none of their credentials, where it dropped every backend whose key was missing. A method using a search preset or pinning a bare handle such asmodel = "gpt-4o-mini"now validates on a machine or CI runner with no key, and such a boot no longer reads the backends' keys or requests a Vertex AI token. - Linkup uses the key its backend names: the Linkup search and fetch workers take the key from the
[linkup]table'sapi_key, like every other provider, where they readLINKUP_API_KEYthemselves and ignored a configuration naming another variable. A Linkup backend with noapi_keyis refused withLinkupErrorwhen its worker is built.
[v0.74.1] - 2026-10-05
Added
PIPELEX_HOMErelocates the home configuration directory: when set and not empty, it names the directory Pipelex uses in place of~/.pipelex, for the configuration layers, the inference files and their overrides, the credentials.env, the first-boot copy of the kit,pipelex init,pipelex doctor,pipelex update,pipelex migrateand the agent CLI's--global. It must be in the process environment before Pipelex is imported, a relative value is resolved against the working directory at import, and aPIPELEX_HOMEline in a.envfile is ignored. The first boot now fills the home directory from the kit when it exists but is empty, as it already did when it was missing, so an empty volume mount or amktemp -dworks as it is.
[v0.74.0] - 2026-10-05
Added
- A reference to a package's private pipe is refused as unexported: a consumer naming a pipe that a loaded dependency package declares but does not export is refused at load with the new
PipeValidationErrorTypevalueunexported_pipe_dependency, whose item names the referring pipe inpipe_codeand the reference inmissing_pipe_code.
Changed
- The hosted-only inference provider plugin is no longer shipped (Breaking): its built-in plugin, its sdk tokens, its error classes and their reference pages, and the
PipelexBackendmember naming its backend are removed, leavingPipelexBackend.INTERNALalone. A project config that still declares the hosted backend disabled, or a routing profile that names it but is not active, keeps booting; a model spec naming one of its sdk tokens now fails when its worker is built, unless a plugin of your own registers that token.
Fixed
- A dependency package's pipes call their own sub-pipes: a sequence, parallel, condition or batch inside an installed package now reaches the package's own pipe, stored as
alias->domain.code, where it used to look the plaindomain.codeup in the consumer's library. A package whose controllers call its own pipes, such asgithub.com/Pipelex/methods/documentsfrom 0.1.3, no longer fails to load withPipeNotFoundError, and a consumer pipe of the same domain and code is no longer run in the package's place. - An exported pipe can call its package's private pipes (Breaking): a package with
[exports]loads every pipe its exported pipes andmain_pipes reach, not the exported ones alone, and a private pipe nothing public reaches is not built. Exports are read by domain, so exportingxfrom one domain no longer exports a pipe namedxin another, andPipeLibrary.add_dependency_pipetakesis_exported. - A missing pipe of a loaded package is refused at load: a reference to a pipe a loaded dependency package does not have is refused as
unresolved_pipe_dependency, naming the package, where it was skipped and reported as an extraneous input with a wrong suggested fix. A dependency's own reference to another package is not checked against the consumer's packages, since a dependency's dependencies are not loaded. - A pipe wrapping another package's pipe of the same domain and code reports its inputs: the recursion guards of the needed-inputs, signature and file-input walks tell two pipes of the same
domain.codeapart by object rather than bypipe_ref. A consumer pipe calling a dependency's pipe of its own domain and code, or one package's pipe calling another's, was taken for an already-visited pipe and reported no inputs, so a correct bundle was refused with an extraneous-input error. - A bare reference into a package ignores its private pipes when an exported one matches: a consumer's
alias->codethat matches an exported pipe and a private pipe of the same code in another domain resolves to the exported one, where it was refused as ambiguous. - A dependency pipe that fails to build is refused naming why: a reference to a pipe the package declares but could not build, such as a
PipeFuncnaming an unregistered function, is refused with that build error, where it was reported as a pipe the package does not have. A barealias->codematching such a pipe is refused the same way, rather than resolved to another pipe of that code.
[v0.73.0] - 2026-10-05
Added
- Service error codes, contributed by the plugin that speaks to the service: a plugin declares each error code its gateway or hosted service emits on its own as a
ServiceErrorCode, with its category, its user action, its advice and whether it means an unknown model, and hands them toPluginRegistrar.add_service_error_codes.classify_inference_errorconsults the booted plugins' codes ahead of the HTTP status ladder andrender_inference_errorrenders the matched entry's advice; a code two plugins contribute fails the boot withDuplicateServiceErrorCodeError, and one the runtime classifies itself withReservedServiceErrorCodeError. - Request extras on the Anthropic worker:
AnthropicLLMWorkertakes an optionalextras_factory, theBackendExtrasFactorythe OpenAI-substrate workers already take, whose headers and body additions join every text and structured call, andAnthropicFactory.make_anthropic_clienttakes ansdk_variant, so a plugin can register an sdk token of its own over the Anthropic protocol.
Changed
- The completions image worker's fixed output format comes from its registration:
OpenAICompletionsImgGenWorkertakesfixed_output_format, the one image format a service answers in, rather than inferring it from the backend's name; a request for another format is refused before the call, and a URL answer from a worker built without one is refused as unreadable. - The Inference SPI table lists every symbol a provider plugin builds on:
docs/under-the-hood/inference-backend-plugins.mdnow publishes, one module per row, the worker contracts with their jobs and outputs, the OpenAI- and Anthropic-protocol factories and workers, the image-generation and prompt-preparation helpers, the error pipeline and the runtime accessors, andparse_retry_after_secondsanderror_code_from_body_code_firstbecome public inpipelex.cogt.inference.error_classificationfor a plugin that distils a plain-HTTP failure itself. - The Anthropic worker no longer adds a backend's headers by backend name (Breaking): a worker built without an
extras_factorysends nothing extra, whatever its backend, so a backend that relied on the worker adding its request headers must register its own sdk token with an extras factory and name that token in thesdkof its model specs, local overrides included; a model left onanthropickeeps answering, without those headers. - The gateway refusal families leave the core classifier (Breaking):
GatewayRequestLimit,GatewayUnresolvedReferenceandGatewayRoutingRefusal, the matching properties ofProviderErrorMetadataand fields ofClassificationResult, andMANAGED_GATEWAY_BACKEND_NAMESare removed; a gateway's own codes now reachClassificationResult.service_error_codethrough the plugin that contributes them, and Portkey'smodel_not_allowed_erroris read byProviderErrorMetadata.is_model_not_allowedandClassificationResult.is_model_not_allowed. Errors raised by the hosted-only provider plugin now report thetypeandtitleof their generic family, such asextract-job-failure-error, rather than pages and titles of their own. - The default extractors run on your own keys (Breaking):
@default-extract-documenttakes the first of Mistral OCR andpypdfium2-extract-pdfthat an enabled backend serves, and@default-extract-imageand@default-premiumtake Mistral OCR, where all three named Azure Document Intelligence, which no shipped backend serves any more. APipeExtractstep naming no model therefore runs on Mistral OCR with your own key or, for a PDF, on its text layer read locally, which recovers no text from a scanned page; a project's managed3_extract_deck.tomltakes the new aliases withpipelex update.
Fixed
- A Manifold failure on the extract and search routes reports the service's own message:
extract_manifold_metadatatakes the message from the response body'serror.message, or else its top-levelmessage, so a refused key now readsPortkey Error: Invalid API Key. Error Code: 03rather than httpx'sClient error '401 Unauthorized' for url …; a body carrying neither, a body that is not JSON and a network failure keep the exception's own text. Because the quota check reads that message, a 429 on those routes whose body names a quota, such asMonthly quota exceeded, now classifies as quota exhaustion, to be fixed in billing, rather than as a transient rate limit to retry. - A kernel-only boot over a stale configuration file stays kernel-only (Breaking): the boot-tolerance replay no longer loads interpreter modules into a
RuntimeBootprocess, because the fix-op applier, the file transaction and their errors moved frompipelex.pipelineto the new kernel-layerpipelex.fix_opspackage; importFixWriteConflictErrorandFixTransactionErrorfrompipelex.fix_ops.exceptions, and the applier and transaction frompipelex.fix_ops.applierandpipelex.fix_ops.file_transaction.
Removed
- The builder:
pipelex-agent concept,pipelex-agent pipe,pipelex build runnerandpipelex.builder(Breaking): the two agent commands that converted a JSON concept or pipe spec into MTHDS TOML, andpipelex build runner bundleandpipelex build runner method, which wrote a Python script running a pipe, are gone with the spec layer behind them,ConceptSpec,PipeSpec,PipelexBundleSpecand theConceptSpecError,PipeSpecErrorandPipelexBundleSpecBlueprintErrorclasses; write a method in MTHDS directly and check it withpipelex validate, and take a runner's typed structures frompipelex build structuresorpipelex codegen types --target python-structures. Thepipelex.builderpackage no longer exists: the modules the runtime still uses moved,pipelex.builder.conventionstopipelex.pipeline.default_file_names,pipelex.builder.operations.models_opstopipelex.cogt.models.model_listingandpipelex.builder.operations.inputs_opstopipelex.pipeline.inputs_template, and the others are deleted with the helpers only they used,force_include_dirsonpipelex.tools.misc.file_utils.find_files_in_diramong them, which only forced the builder package into library loading. Thecircular_dependency_errorvalidation error type, which only the spec layer's pipe sorter raised, leaves theerror_typeregistry and the API server's OpenAPI schema. - The Pipelex Gateway backend is gone from the runtime (Breaking):
pipelex_gatewayleaves the shippedbackends.toml, itsbackends/pipelex_gateway.tomloverride file and the generatedpipelex_gateway_models*.mdreferences leave the kit, theall_pipelex_gatewayrouting profile is gone, and thepipelex-gatewayprovider plugin is deleted along with itsgateway_*SDKs, its error classes and their docs pages. Abackends.tomlthat still enables a[pipelex_gateway]table, as the previous release's kit did, is refused at boot, because an enabled backend whose file declares no model is now an error rather than a backend every routed model silently goes missing from: disable or remove the table, and move a routing profile naming it, the previous defaultall_pipelex_gatewayincluded, to another backend. The shipped default routing profile is nowall_enabled_backends, which needs no Pipelex account: it sends each model to the first enabled backend that serves it, in the order itsfallback_orderlists.pipelex initpre-selects OpenAI, whose one key serves every default language-model and image-generation tier of the shipped model deck, and no longer setsall_pipelex_gatewaywhen several backends are chosen;pipelex-agent initgiven no backend list keeps the template's routing profile rather than demanding aprimary_backend. Thepipelex/pipelex-apiimage enablesopenaiin its place and boots onOPENAI_API_KEY, which serves every default language-model and image-generation tier of its deck. Every bring-your-own-keys backend is untouched. - The Gateway terms of service and the first-run check are gone (Breaking):
GatewayTermsNotAcceptedError, the[agreement]section ofpipelex_service.toml,pipelex init agreementand thepipelex-agent accept-gateway-termscommand are removed, andpipelex-agent initno longer readsaccept_gateway_termsfrom its config. The check that stopped a live run on a Pipelex-managed backend until inference setup had been recorded is removed as well, withInferenceSetupRequiredError, its CLI handling andIntegrationMode.requires_terms_acceptance, so a server, a container or a test suite boots with no setup recorded at all. - The Pipelex-owned telemetry stream is gone (Breaking): the runtime no longer reports to Pipelex's PostHog on the Gateway's behalf, so
GatewayApiKeyMissingError,GatewayDoNotTrackConflictError,GatewayTelemetryManagerInjectedError,TelemetryManagerAbstract.is_pipelex_telemetry_enabled,IntegrationMode.is_test_harnessand the maintainer-only[pipelex_gateway]section oftelemetry.toml(with itsis_pipelex_gateway_portkey_*knobs) are removed;TelemetryFactory.make_telemetry_managerandTelemetryManagertake only the user'stelemetry_config. The user's own opt-in telemetry under[custom_posthog],[langfuse]and[[otlp]]is untouched. Migration: runpipelex migrate; ledger entrytelemetry-config@3deletes the[pipelex_gateway]table from everytelemetry.tomlandtelemetry_*.tomlfile in~/.pipelex/and in each project.pipelex/, keeping one timestamped backup per file. Nothing else needs deciding, since the section only ever configured the stream that no longer exists. pipelex loginis gone (Breaking): it wrote aPIPELEX_GATEWAY_API_KEYthat nothing reads any more, and it also leaves_CORE_COMMAND_ORDERand the setup-command set that skips the readiness check — an order list naming a command the CLI does not register makespipelex --helpitself exit non-zero. The credential flow for the hosted service is rebuilt separately.- The
check-gateway-modelsandupdate-gateway-modelsdev commands are gone, with theirmake ugm/make cgmaliases and theticcconfig-coverage target that ran the Gateway's Portkey configs. - The runtime no longer downloads a remote config, and the hosted-only inference backend is no longer shipped (Breaking): the boot fetches nothing, so
PIPELEX_REMOTE_CONFIG_URL, the~/.pipelex/cache/remote_config.jsoncache, the cache priming inpipelex initandpipelex-agent init, the remote fetch inpipelex doctorandmake regenerate-test-models, and theRemoteConfigFetchError,RemoteConfigValidationError,RemoteConfigUnavailableError,GatewayUnknownModelErrorandGatewayConfigMergeErrorclasses are gone, with thewarningsarray the agent CLI added to a success envelope for a stale cache. Every backend now reads its model specs from its own file underinference/backends/, somodel_specs_sectionis no longer a backend setting, and the kit no longer ships the hosted-only backend'sbackends.tomltable, its file underinference/backends/or the routing profile that sent every model to it; a project config that still declares them keeps booting while that backend is disabled, since a disabled backend is skipped and only the active routing profile loads. An enabled backend whosebackends.tomltable still carriesmodel_specs_sectionis refused at boot, naming the key: list its models in itsbackends/<name>.tomland remove the key, or disable it.pipelex_service.tomlis retired with itsPipelexServiceConfigandPipelexServiceErrorclasses:pipelex-agent initno longer writes it or reportsinference_setup_completed,cache_primedorcache_priming_error, and a copy an earlier release left behind is ignored, bypipelex migratetoo. Theneeds_model_specsargument ofPipelex.make, which only chose whether to fetch, is removed.
[v0.72.0] - 2026-10-04
Highlights
PipeJudge brings judgment into MTHDS methods: a step asks a model one closed question about its inputs and gets back a typed verdict with the probabilities behind it. A yes/no question returns a YesNo, a choice among declared options returns a Choice, and a position on a declared scale returns a Rating, so a PipeCondition can route on the verdict or gate a step on how sure the model was.
TypeSafe's Jev is the first judgment model Pipelex runs. Jev answers closed questions with calibrated probabilities rather than free text: install pipelex[typesafe], set TYPESAFE_API_KEY and name @default-judgment in the step.
Added
PipeJudge, an operator asking a judgment model one closed question about its inputs: a step declaring neitheroptionsnorlevelsasks a yes/no question, which may carrycriteriaand athreshold, and outputs aYesNo; one declaringoptionsoutputs aChoice; one declaringlevelsoutputs aRating, each verdict carrying the probabilities the model measured. Every declared input is sent to the model as material whether or not thequestionreads it, andpromptis read as a synonym ofquestion. A step is refused when the method loads if it names no model while the deck names no default for judgments, if its output disagrees with its kind of question (Dynamicincluded), if an input name reaches into a field with a dot, or if it has an image or a document input its model does not read, and a document a run is given in a format its model does not read is refused before the run starts. The builder andpipelex-agent pipewrite it withquestion, and the MTHDS Test Corpus gainsoperator_judge_urgent_messageand theoperator.judgetag.- A
judgmentinference family: a new family besidellm,img_gen,extract,searchanddoc_gen, for models that answer a closed-form question over some material — yes or no, one option out of a declared set, or a position on a declared scale — and report how sure they are. The worker contract is batch-shaped (one state, a map of questions, a map of answers), every measure of uncertainty is optional so a backend that measures none can still serve the family honestly, and usage is reported as real input and output tokens priced per million. AJudgmentJoband itsJudgmentAssignmentcarryimagesanddocumentskeyed by input name, which a scoped run's read authorization sees, and a judgment worker refuses files its model does not read withJudgmentCapabilityErrorbefore calling the provider. The deck gains a[judgment]section in a new managed file,6_judgment_deck.toml, whichpipelex updateinstalls, and a deck installed before it keeps loading with no judgment model; it ships with nochoice_default, because the family's models come with your own key and a default the Pipelex Gateway does not serve would fail every boot, so a judgment names its own model.pipelex-agent modelslists the new category, and so does the MTHDS protocol's model list,GET /v1/modelson the API server, where a judgment preset is an entry typedjudgmentand?type=judgmentfilters on it; ajudgmentpytest marker joins the inference markers. - TypeSafe, the judgment family's first backend: a built-in
typesafeplugin, installed with the newpipelex[typesafe]extra, serves the family from TypeSafe's System One API and is reachable as@default-judgment, pinned to the versioned modeljev-1.13.0. SetTYPESAFE_API_KEYto use it: the default routing profile sendsjev-*models to the backend whenever it is enabled, because the Pipelex Gateway does not serve them. A rating scale longer than ten levels is refused by this backend before any request is made, and a response answered by a model other than the pinned one is reported as a warning. - Log sink settings can name secrets: the values of
headersunder[runtime.log.otlp]andcredentials_file_pathunder[runtime.log.gcp]accept the${…}placeholders the inference backends use, resolved through the configured secrets provider, so a collector's token is written"Bearer ${OTLP_AUTH_TOKEN}"and the key path"${GCP_CREDENTIALS_FILE_PATH}". The configuration keeps the placeholders, and one that does not resolve stops the boot withLogSinkVariableErrornaming the section, the key and the variable. A header value that holds a line break once resolved, as a secret read from a file often does, stops the boot withLogSinkHeaderValueErrorrather than installing an exporter that refuses every batch.pipelex doctorgains a Secrets Provider row,checks.secrets_providerinpipelex-agent doctor.
Changed
mthdsversion, MTHDS protocol 0.7.0 and standard 3.0.0 (Breaking): the pin moves tomthds==0.19.0, which carries protocol 0.7.0, whose model list adds thejudgmentcategory besidellm,extract,img_genandsearch, so the runner andGET /v1/versionreportprotocol_version0.7.0, and which implements MTHDS standard 3.0.0. A crate pipelex normalizes is stampedmthds_version3.0.0, and aMETHODS.tomldeclaringmthds_version = "^2.0.0"now warns that the engine does not satisfy it.PipelexModelDeckinherits the release's openModelInfo.type, typedModelCategory | str | None, though pipelex itself only ever emits a protocol category; everyone downstream inherits that exactmthds.- The native concepts are pinned at MTHDS 3.0.0:
YesNogains a probability, andChoiceandRatingare added (Breaking):YesNocarries an optionalprobability, which aPipeLLMwhose output isYesNomay fill with the model's own estimate;Choice(an option key, with optionalconfidenceandprobabilities) andRating(a level index, with optionalconfidence,probabilitiesandposition) are new natives, so a bundle declaring a concept namedChoiceorRatingmust rename it. AYesNowritten to the wire carries"probability": nullwhen none was reported, the output-form descriptor states aYesNoresult as an object carryingyes_noandprobabilitywhile an input stays a bare boolean, and a crate's normalized fingerprint changes wherever it materializesYesNo. Python callers read the new verdicts withmain_stuff_as_choiceandmain_stuff_as_rating. Every measure is a strict number, so a boolean or a numeric string is refused rather than read as1.0or0.9; and aPipeDocGenoffice template or auto-layout sees a verdict as its verdict alone, while apdftemplate reads it whole. - The method-app template and the SDK references link
Pipelex/pipelex-sdk: the SDKs and the method-app templates now live in thePipelex/pipelex-sdkrepository, so the README and the documentation's Quick Start link the method-app template to itsmethod-apps/directory, the webapp's by-hand commands clone it and copymethod-apps/webapp-js/, and the@pipelex/sdkandpipelex-sdkreferences point at itsjs/docs/andpython/docs/directories. - Boot builds the secrets provider before the log sink, and a sink factory receives it (Breaking): a
LogSinkFactoryFnis now called asfactory(log_config, secrets_provider=…), and the plugin API version moves to 5, so a plugin declaringtargets_api = 4is refused at discovery and is rebuilt against this release. A secrets provider that fails to build now reports on stderr, redacted, rather than through the configured sink. pipelex-agent inputsnames the pipepipe_ref, qualified (Breaking): the JSON success envelope ofinputs bundle,inputs pipeandinputs methodis{success, pipe_ref, inputs}, asmthds-agentprints on the API runner, where it waspipe_code. The value is the qualifieddomain.pipe_codeof the pipe that was resolved, so a bare--pipe summarizeanswersdoc.summarizeand a dependency pipe keeps itsalias->prefix, and a pipe that declares no inputs names its pipe too, where it printednullwhen--pipewas omitted, and no longer adds amessagekey. That pipe's message, the TOML comment here and the linepipelex build inputsandpipelex codegen inputsprint, now readsPipe '<pipe_ref>' declares no inputs.
Fixed
- An image worker refusing a model's spec reports a configuration fault: a worker refused a model with no
rules, nomodel_choiceorendpoint_pathwhere it needs one, or on the Google Gemini paths anaspect_ratiotaxonomy that is missing, unknown to this release or of another family, withImgGenParameterErrorin thecontentcategory, so a server answered 422 and told the caller to change their input; and a rule value this release does not know escaped the OpenAI, Azure, fal, Hugging Face and gateway image workers as a bareValueError. Each now raisesImgGenParameterErrorin theconfigurationcategory, naming the model, and answers 500, while a ratio or size the model's grid refuses is still the caller's 422. - A bare value is refused where it reads as a concept the input does not accept (Breaking): an input declared as a container native (
Html,Page,TextAndImages,SearchResult,Composite), asChoiceorRating, or as a concept refining one, built a bare string as anative.Textand handed that text on, so the step reading the input met a text where it declared another concept. The value is now refused withStructureValidationErrornaming the concept it read as; send it in its{concept, content}envelope. The concept's own content sent without that envelope, such as{"choice": "billing"}, is refused with the same advice, where it was told to redeclare the input asJSON, and so is a bare number or boolean at aRatingorChoiceinput, such as a level2sent alone, where it was told to redeclare the input asAnything. ADynamicinput still keeps whatever its value reads as. - A name a template sets is read where Jinja binds it (Breaking): the input check of
PipeLLM,PipeCompose,PipeSearch,PipeImgGenandPipeDocGen,PipeDocGen's field check and the optional-input guard now treat a{% set %}name as the template's own inside the loop, macro or block that sets it and after an{% if %}every branch of which sets it, so a template setting a name in a loop body or in each branch of anifand reading it there is no longer refused as reading an undeclared input. A name only some branches set is still read from the input of that name. Newly refused: an input declared only so that such a set would load, which is now never read, a macro called before its{% macro %}, an optional input read unguarded in a macro argument's default, and a name a loop or a macro binds read inside a{% block %}withoutscoped, which Jinja reads from the input of that name. - File inputs are checked against the models that read them (Breaking): run setup now identifies every image and document input from its first bytes, refuses an
Imageinput holding something other than an image withPipelineInputNotAnImageError, and refuses a document certain to reach aPipeExtractorPipeLLMstep whose model does not read its format withPipelineInputFormatUnsupportedError, both 422 input errors naming the input, the step, the model and the formats it reads; a step reached only through a condition refuses the file itself withExtractInputFormatErrororPromptDocumentFormatError. APipeLLMdocument its model does not read now raisesPromptDocumentFormatErrorwhere it raisedLLMCapabilityError, so a consumer branching onerror_typemust follow.docling-extract-textnow declares the Word, PowerPoint, Excel, HTML, Markdown, CSV, plain-text, WebVTT and email files it already extracted, whichpipelex updateinstalls. - Markdown with unclosed emphasis is converted in memory proportional to its length: the Markdown parser behind the
markdownfilter, theMarkdownconcept's HTML and the built-in PDF engine held memory growing with the square of the text when emphasis or strikethrough markers were left unmatched, about 1.5 GB for 64,000 characters of**a; it now joins the text they leave behind in one pass, and renders the same HTML. - Routing profiles no longer rewrite their own routes: matching a model merged the enabled
optional_routesinto the profile's declaredroutesin place, so after the first match a route declared optional had become an unconditional one in the loaded profile. Only the enablement checks each match repeats kept that from changing a routing decision. - The logging documentation no longer promises that a server refuses the
consolesink: the log-sink, logging, logging-configuration and Rich-imports pages said a server installed without thecliextra stops at boot when it selects theconsolesink or therichpretty-print mode. The core dependenciestyperandinstructorinstall Rich anyway, so such a server boots on the Rich surface; the pages now say a server selectsjsonand a Rich-free mode in its own configuration, since nothing at boot makes that choice for it. pipelex doctorchecks the backends against the configured secrets provider: the backend-file probe and the Models row resolved credentials from environment variables whatever[runtime.secrets] methodselected, so a deployment on another secrets method was diagnosed against secrets it does not use. Both now use the provider the configuration selects, and say they could not run when it does not build; a provider that raises on a lookup, an unreachable vault or a missing SDK, is reported in the Models row rather than stopping the doctor.
Removed
- The API server's
/v1/build/*routes (Breaking):pipelex-apino longer servesPOST /v1/build/inputs,/v1/build/output,/v1/build/runner,/v1/build/conceptor/v1/build/pipe-spec, which now answer404, and theMAX_AGENT_SPEC_KIBsetting that bounded the last two is gone. A client reads a pipe's I/O fromPOST /v1/pipe-ioand projects an inputs template from itsinput_formwith themthdspackages' projection (renderInputsTemplate, which@pipelex/sdkre-exports, ormthds.protocol.inputs_templatein Python), asmthds-agent inputsdoes; the output representation, the runner script and the concept and pipe spec conversions have no HTTP replacement.
Security
- Every template render spends from a budget (Breaking): a render is charged for what it reads, allocates and prints, and an operation that would overdraw its budget, such as
'x' * 10 ** 9, a huge format width,lipsum(n=10 ** 9), nested empty loops or a runaway recursive macro, is refused before it allocates and raisesJinja2TemplateBudgetError. Converting Markdown to HTML, with themarkdownfilter or by printing a Markdown value in an HTML template, is charged for every character and table cell of its source, so one render converts at most about 65,000 characters of Markdown. Template arithmetic stops at 64-bit integers, and a template can no longer callstr.encodeorint.to_bytes. A filter or test registered on a template environment needs a cost in the render budget's tables, or the first compile raises aTypeErrornaming it, and Pipelex now requiresjinja2below 3.2. pipelex doctorredacts the failures its rows quote: the log-sink row printed a sink's failure as raised, so a credential an exporter or a client quoted in its error reached the terminal and the JSON ofpipelex-agent doctor. That row, the new Secrets Provider row and the Models row, which quotes a secrets provider's own text when a backend's credential does not resolve, now scrub the text with the patterns the log redaction uses, the configuredextra_patternsincluded.
[v0.71.1] - 2026-10-01
Fixed
- A symlink in a fetched package is a 422 on the API server:
MethodPackageSymlinkError, raised when a package fetched bymethod_refcarries a symlink, is now answered as a 422 the method's author can act on, where it was a 500 server fault.
Security
- Locating a fetched package never follows a symlinked
METHODS.toml: whenpipelex run <address>, fetch-on-miss or the API server'smethod_refscans a fetched repository for its packages, aMETHODS.tomlthat is a symlink, or is not a regular file, is now skipped and reported without being opened. A repository could commit one linking to a file on the fetching machine and get that file's values back in the error message, even under strict error disclosure, or link it to/dev/zeroand make the read run without end. The read now also stops at the manifest size ceiling whatever size the file reports, and a manifest that is not valid UTF-8, or a directory namedMETHODS.toml, is skipped and reported instead of failing the whole fetch.
[v0.71.0] - 2026-10-01
Highlights
The Pipelex API server and its pipelex/pipelex-api image now ship from the pipelex repository, so they take pipelex's version number. The image's last standalone release was 0.33.2, on the server's own release line, and its next tag is 0.71.0, the same version as the pipelex and pipelex-api packages published beside it.
Added
PipeDocGengenerates a document file: a new operator turns its inputs into a storedDocumentnamed by itsfilenameexpression, whichpipelex runcopies into the run's results folder. Apdfwithout a template is printed by a built-in engine on ReportLab from the auto-layout of the inputs, and apdfwith an HTML template or anyxlsx,docxorpptxstep is refused when the method loads, with an error naming the Pipelex document generation plugin. A template's field paths are checked against the inputs' concepts at load, and its templates and filename render strictly, so a missing value fails instead of printing as empty text.- Document engines are models: the engine that prints a
PipeDocGenstep is a model of a newdoc_genfamily, which a step names withmodel = "pipelex-pdf"or leaves to the default for its format and source in the new deck file5_doc_gen_deck.toml, and whose worker a plugin registers withadd_inference_backend(family=InferenceFamily.DOC_GEN, …), declaring its model withadd_internal_modeland its deck defaults withadd_doc_gen_default. A step whose engine is not in the deck, does not print its format from its source, or has no installed worker is refused when the method loads, so a runtime that disables the built-inreportlabplugin in[runtime.plugins] disabledrefuses everyPipeDocGenstep. An engine receives aRenderJobof plain data, reads the files its document names throughRenderResourcesunder the run's read scope, and may check a step's template file at the dry run.pipelex updatenow also refreshesbackends/internal.toml, backing up a locally edited copy first, so run it to get the deck file and the built-inreportlab-pdfengine. - The
markdowntemplate filter: an HTML template turns Markdown held in a text field into HTML with{{ order.notes | markdown }}. Raw HTML in the source is shown as text, and only URLs with a scheme become links. - The
Markdownnative concept:MarkdownrefinesTextand holds a text written in Markdown, which aPipeLLMwithoutput = "Markdown"writes as free text and aMarkdowninput takes as a string or as{"concept": "Markdown", "content": "…"}; Pipelex defines it ahead of the MTHDS standard. AMarkdownvalue is accepted wherever aTextis, while aText, or any concept refiningText, is refused where aMarkdownis required.pipelex run --save-main-stuffsaves it verbatim asmain_stuff.md, an HTML template prints{{ report }}and$reportas HTML with raw HTML escaped, andStuff.as_markdownandPipeOutput.main_stuff_as_markdownread it from Python. - The Pipelex API server ships with
pipelex: the FastAPI runner now lives in theapi/directory, as thepipelex-apimember of a uv workspace, and runs thepipelexof the same commit instead of a pinned release. Every release publishes it underpipelex's version, as the newpipelex-apipackage on PyPI, which pinspipelexto that same version, and as thepipelex/pipelex-apiimage on Docker Hub, whose tag therefore jumps from0.33.2, the server's last release on its own, to this version. The image keeps its name, its port and its/root/.pipelexconfiguration mount, andPipelex/pipelex-apikeeps the server's earlier history. x-pipelex-metadataheader on Pipelex Manifold calls: every inference request the Manifold backend makes (chat completions, responses, image generation, extraction, search, and Claude over the Anthropic SDK) carries one JSON header holding the run'sextrasas they are, plus the run'suser_idandpipeline_run_idand the step'spipe_run_id,pipe_codeandcontent_generation_job_idwhen set, so the service can attribute each call's spend and log it. On a key collision the run's own id wins over the extras entry, and no other backend receives the header.
Changed
Markdownis a reserved native concept code (Breaking): a bundle declaring its own concept namedMarkdownis refused withnative_concept_redeclaration; rename it, or use the native.- A string under a concept refining
Textkeeps the concept's class: a bare string or a{"concept": …, "content": "…"}envelope for such a concept now holds the concept's own content class where it held a plainTextContent, as theYesNo,DateandTimeenvelopes already did. markdown-it-pyreplacesmarkdownin the dependencies: themarkdownpackage, which pipelex no longer imported, is dropped, andmarkdown-it-pywith itslinkifyextra renders the Markdown ofPipeDocGen, themarkdownfilter and theMarkdownconcept.- The API server's import package is
pipelex_api(Breaking): the server's packageapiis renamedpipelex_api, so an application wrapping the server importspipelex_api.mainand servespipelex_api.main:app, and theloggerfield of the server's log records readspipelex_api.…. - The API server's documentation is part of docs.pipelex.com: its pages are the API Server section of the Pipelex documentation, versioned with each release, in place of the separate site at
pipelex.github.io/pipelex-api, and the committed OpenAPI artifact is published beside them asapi-server/openapi/pipelex-api.openapi.yaml. - The API image is built from the
pipelexrepository:docker build -f api/Dockerfile .at the repository root installspipelexandpipelex-apifrom that commit, with the dependency versions of the repository's lock, which moves some provider SDKs to the versionspipelexis tested with. The container startsuvicornfrom its virtual environment rather than throughuv run, and carries nouv.
Fixed
- The API server refuses a deeply nested body the same way on every Python: a JSON request body whose arrays and objects nest deeper than 128 levels, its own envelope included, is now refused before it is parsed, on every route that takes one, with a 422
InvalidJSONsaying it is nested too deeply. On Python 3.14 whether the run routes refused such a body depended on the size of the thread's stack, so it could reach validation and fail as aValidationErroror be accepted, and the other routes answered it with a bare 400 rather than a problem document. - A client leaving mid-upload is a 400 in the API server's logs: when a client disconnects before its request body arrived in full, the server answers a 400
BadRequestand logs it at warning level, where the run routes logged a 500 server fault with a traceback. - The API image runs methods by address: the image carries
git, which fetching a method named bymethod_refneeds, where the0.33.2image removed it with the build tools and failed every such run. - A workflow action the organization does not allow is caught on its pull request: GitHub rejects a whole workflow at startup when one of its actions is outside the organization's Actions policy, and pull-request CI never loads the release workflows, so such an action surfaced only as a release that published nothing.
make check-actions-allowlist(aliascaa), whichmake agent-checkand theLint (actions allowlist)CI job run, now refuses any workflow action outside the policy mirrored in.github/actions-allowlist.toml; see Actions Allowlist.
[v0.70.0] - 2026-09-29
Highlights
This release hardens pipelex for running methods it did not write. Every template now renders in Jinja's sandbox, a run with a read scope reads only under that scope and never the local disk, a URL fetch refuses private and metadata addresses, and a sandbox-hosted load no longer imports a method's Python into the loading process. Validation gets stricter too: PipeCompose, PipeSearch and PipeImgGen refuse an input their templates never read, and a concept whose field names a concept nobody declares is refused at load.
Changed
read_scopeis a required run parameter (Breaking):RunMetadata,pipeline_run_setup,prepare_pipe_job,PipelexKernel.make,shape_inputs,InputShaper.shape,WorkingMemoryFactory.make_from_pipeline_inputs,StuffFactory.make_stuff_from_stuff_content_or_data,normalize_data_urls_to_storageand the runtime bridge'sPipelexPipeRunInputtake aread_scopewith no default: a host serving several tenants passes the tenant's prefix, which must contain the run'sstorage_scope, andNonestates that the run is unscoped.PipelexMTHDSProtocoldefaults it toNone, which is the truth on a laptop.- Dry-run mocks of images and documents carry readable URLs: a mocked
ImageContentcarries one ofinference.dry_run.image_urlsand a mockedDocumentContenta blank one-page PDF as adata:URL, where both carried a random string that reads as a local path; and a model nested two levels or more inside a mocked structure now gets the same field providers as a shallower one. - A structure field naming a concept that cannot be found refuses the load (Breaking): a concept whose field names a concept that no bundle it can see declares now raises
LibraryLoadingErrorat load, naming the concept, its bundle and the missing structure class, where it loaded and failed with pydantic's "not fully defined" error at first use. A concept declared only by a bundle loaded afterwards cannot be found either.load_from_cratedoes not refuse, since a crate carries no dependency package, and neither does a method package's own load, since its own dependencies are never loaded; a concept holding a method package's concept left incomplete that way is not refused either. - PipeCompose, PipeSearch and PipeImgGen refuse an input they never read (Breaking): as PipeLLM already did, validation refuses a declared input that none of the pipe's templates reads (the template or the construct of a PipeCompose, the prompt of a PipeSearch, the prompt or negative prompt of a PipeImgGen) as
extraneous_input_variable, naming the input: reference it in a template, or remove it frominputs. A variable a PipeImgGen'snegative_promptreads without declaring it is refused when the bundle is parsed, andUnusedInputError, which nothing raised, is removed.
Fixed
- Concepts holding other concepts are complete: a method package's concept holding another of the package's concepts, a field typed by a method package's concept (
address->domain.Concept), a field typed by a concept that an earlier load put into the same library, a field typed by a concept backed by a Python class (structure = "ClassName"), and a field typednative.Anything, which holds any value, now resolve, where each loaded and failed with pydantic's "not fully defined" error at first use. A method package whose concepts form a cycle is refused as a bundle's own concepts are, naming the package's bundle. - The input errors of the template operators carry their type and names: a missing or unread input of a PipeLLM, PipeCompose, PipeSearch or PipeImgGen is reported with its
error_typeand withvariable_namesholding the plain names, sorted. A PipeCompose template reading an undeclared variable was an uncategorized item, the PipeSearch and PipeImgGen items carried no names, and PipeLLM's unread names arrived quoted ('platform'). - A template read through a subscript, a call or a
setcounts as a read: an input a template reads only as{{ items[0].text }},{{ (items|first).text }}or{{ record['meta'].title }}, or reassigns with{% set topic = topic|trim %}, is seen as read, where PipeLLM refused it as unread, and such a read of an undeclared variable is refused as missing. A read placed before a top-level{% set %}of the same name, and aforloop's iterable orelsebranch, read the input. The name a top-level{% set %}…{% endset %}block assigns, a call to one of Jinja's globals such asrange(count)ornamespace(), andcaller,varargsandkwargsinside a macro are no longer demanded as inputs, while an input named like a global, such asrange, is read wherever the template reads the name without calling it. The images and documents PipeLLM attaches, and the reference images PipeImgGen attaches, are found under the same scopes, so an image read after{% set image = image %}or as the iterable of a loop named after it is attached. - A generated object's storage key names its own format: the extension of a key under
generated/follows the whole MIME type the object is stored under, so a fetchedimage/svg+xmlis stored as.svg, and a type with no extension of its own gets.bin, where both used to get.jpg. The MIME type's parameters and case no longer change the extension. - An include in a method template is a render error:
{% include %},{% extends %}and{% import %}in a template rendered without a loader raiseJinja2TemplateRenderError, where they leaked Jinja's ownTemplateNotFound. - A template reads a
Composite's parts by name:{{ combo.summary }},{{ combo['summary'] }}andcombo.get('summary')resolve the named parts of aCompositeoutput, anditer_keyslists them; they used to render empty, and only the raw_content, now refused, reached them. Like any private name, a part whose name starts with an underscore is refused when a template reads it on an input. - A template's arithmetic error is a render error: a division by zero, or a
rangelonger than the sandbox's cap, raisesJinja2TemplateRenderErrorwhere the raw Python exception escaped, which validation reported as an internal error rather than as the method's. - A sandboxed PipeFunc's imports resolve to the same file in every process: the sandbox searches a method's root directory first and then its subdirectories in sorted order, where the order followed the hash of their paths, which changes from one process to the next. A module name used in two directories of a method, such as a
structures.pyshipped in a subdirectory beside thestructuresmodule the sandbox generates at the root, could resolve to either file. A temporary directory reached through a symlink, as on macOS, no longer lets a file's own directory jump ahead of that order or stay onsys.pathafter the run. - Docling extracts HTML, Markdown and CSV: a document Docling reads without pages (HTML, Markdown, CSV, AsciiDoc) comes back as one page holding the whole document, where it came back with no pages and no text, and one given as a storage key or a
data:URL reaches Docling, where its bytes were refused as an unknown file type.
Security
- Every template renders in a sandbox (Breaking): prompts, compose and construct templates, image and search prompts and condition expressions may read data and call methods of plain values (strings, numbers, dates, containers) only, so a template that reaches for a Python internal, calls a method of a Pipelex or pydantic object, or reads a name starting with an underscore other than an input's
_stuff_name,_content_class,_concept_codeand_stuff_coderaisesJinja2TemplateSecurityErrorinstead of rendering, and validation refuses the visible cases with the newtemplate_private_nameerror._contentand thestuffproperty are no longer readable on an input, and bracket access andgeton an input resolve only its content fields and metadata fields; a plain dict's keys stay readable with brackets,record['_id']included, and anamespace()holds data only, refusing a callable. Thejinja2floor rises to 3.1.6, the release carrying Jinja's own sandbox fix for theattrfilter. - Dotted paths refuse underscore segments: a PipeCompose construct
frompath such asnote.__class__.__init__.__globals__.sys.modules.os.environwalked into Python internals with plaingetattrand could copy the process environment into a method's output. Afrompath orlist_to_dict_keyed_byname with a segment starting with an underscore is now refused at validation, and every dotted path the runtime walks, batch lists and prompt image references included, refuses one. - A scoped run reads only under its read scope, and never the local disk: on a run with a read scope, a
pipelex-storage://key is read only when it lies under that scope, compared segment by segment, and a bare path or afile://URI is never read, whichever value carries it — an input, aPipeComposeconstruct that assembles a URL from text, a model's structured output. Every content-generation leaf checks the URLs it will read before the dry-run branch, and the input seam checks a CSV input and every file it would upload or sign; a refused read raisesUriReadRefusedError, answered as a 422, which names where the URL sat without quoting it. A run whose read scope isNonereads as before. - A sandbox-hosted load imports no method Python, and refuses structure classes (Breaking): on a deployment whose PipeFunc execution mode is not
direct, loading a library directory captures every.pyfile as source for the sandbox and imports none of them, where a file declaring aStructuredContentsubclass was imported into the loading process and its module-level code ran. A load whose Python declares such a class, however the method arrived, now raisesMethodStructuresRefusedErrornaming each file and class by its relative path, the error a fetched package already got: declare the types as MTHDS concepts with inline structures, which a PipeFunc imports in the sandbox from the generatedstructuresmodule. The modulepipelex build structureswrites is accepted while it is left as generated, by this refusal and by the fetched-package one, and is not sent to the sandbox, which generates its own from the same concepts. Both refusals, and the local warnings that predict them, now give that same route. Adirectload is unchanged. - Fetching a URL refuses private destinations (Breaking): every download of a URL a value carries — a prompt image or document for a provider that does not take URLs, a document to extract or render, a generated image a provider returned as a link — now refuses a host that is
localhost, a cloud metadata alias or resolves to any private, loopback, link-local or metadata address, on the first request and on every redirect hop, raisingSsrfBlockedErrorand failing the pipe, where it followed redirects anywhere and returned the body. The guarded fetch connects directly and ignoresHTTP_PROXY/HTTPS_PROXY; a self-hosted deployment that reads documents from an intranet host or reaches the internet through a proxy sets the new[runtime.network] is_fetch_ssrf_guard_enabled = false.make_base64_url_from_any_uriandmake_base64_url_from_http_urlmove topipelex.tools.uri.uri_base64.
[v0.69.0] - 2026-09-28
Highlights
Structured output moves to instructor 1.17, and the provider SDKs move with it. The mistralai extra takes mistralai 2.x, and the Bedrock and S3 extras take aiobotocore in place of aioboto3, which renames the bedrock_aioboto3 SDK handle to bedrock_aioboto. Structured output on the google backend now accepts enum fields and sends the system prompt, the temperature and the token limit.
Changed
- A bare string at an
Anythinginput is anAnythingvalue (Breaking): it becomes anative.Anythingstuff holding text, where it used to become anative.Textstuff. - A
JSONinput reads a JSON object literally (Breaking): a bare string at aJSONinput is refused instead of becomingnative.Text, the compact inputs template of aJSONinput is the bare object rather than the envelope{"concept": "native.JSON", "content": {"json_obj": …}}, and aJSONContentholding a value JSON cannot encode, such as a date or a set, raises a pydanticValidationErrorinstead of aTypeError. - An envelope-shaped item of a bare
Anything[]orJSON[]list is refused (Breaking): an item holdingconceptandcontentkeys is no longer read as data; wrap the whole list in one envelope to type it. - A value the fallback has no reading for is a
StructureValidationError(Breaking): at aDynamicor container-native input, a bare value the bottom-up reading cannot build, such as a list of plain objects, raisesStructureValidationErrornaming the input and its concept instead of an unnamedStuffFactoryError, and its next step names the declaration that reads the value (JSON,JSON[],AnythingorAnything[]) when one does. - An explicit single value at a fixed-count input is refused (Breaking): an envelope holding one value at a
Concept[N]input raisesMultiplicityCountMismatchErrorinstead of being stored as a single value. - NaN and the infinities are refused at
Number,JSONandAnythinginputs (Breaking): a TOML inputs file can spellnanandinf, which JSON cannot hold, so aNumberorAnythinginput given one raisesWrongScalarKindError, and an object holding one anywhere inside it raisesStructureValidationError, where each used to store a number that JSON serialization turns intonull. AJSONContentor aNumberContentholding one raises a pydanticValidationError, which is also what an explicit envelope building one reports. - Chaining an
Anythingvalue through--with-memorychanges it (Breaking): anAnythingstuff's content is dumped as its content class's fields, so a bare"hi"sent to anAnythinginput is dumped as{"text": "hi"}, and a method reading that memory at anAnythinginput receives that object rather than the string; a list is dumped as one object holding itsitems. It used to travel asnative.Text, since a bare string at anAnythinginput became anative.Textstuff. - The
Anythinginput schema excludes array and null: the JSON Schema published for anAnythinginput carries"not": {"type": ["array", "null"]}, and so does each item of anAnything[]input's schema. aiobotocorereplacesaioboto3, and thebedrock_aioboto3SDK handle is renamedbedrock_aioboto(Breaking): thebedrockands3extras depend onaiobotocore3.x instead ofaioboto3, whose latest release pinsboto3below 1.40.62, so both extras now takeboto3>=1.42.42. The new handle names the async Bedrock client rather than the library under it, so a later change of library will not rename it again. Migration: a backend file that still setssdk = "bedrock_aioboto3"keeps loading but fails on every call to its models, with an error that points atpipelex migrate, which names each such file underinference-backend@3without rewriting it, so replace the handle withbedrock_aiobotoby hand.instructor1.17, and themistralaiextra onmistralai2.x (Breaking): pipelex now requiresinstructor>=1.17.0,<2.0.0, and themistralaiextra takesmistralai>=2.10.0, whose client lives undermistralai.client, so a project that importsmistralai1.x beside pipelex must move to 2.x with it.instructor1.17 dropped its strict tool mode, so theinstructor/openai_structured_outputsstructure method now sends the same non-strict tool schema asinstructor/openai_tools, and OpenAI no longer enforces the schema on its side: pipelex still validates the response and re-asks on a mismatch.opentelemetry-apiandopentelemetry-sdk1.39 or later: pipelex now requires both at>=1.39.0instead of>=1.38.0.
Fixed
Anythinginputs take any JSON value: a string, a number, a boolean, an object or a date at anAnythinginput becomes its natural content under theAnythingconcept,Anything[]takes a list of them, a prebuilt list mixing contents of different kinds included, and a typed envelope keeps its own concept; they used to be refused, a bare string aside. A sequence declaring anAnythingoutput over a narrower last step now validates.JSONinputs take objects and lists of objects: a JSON object at aJSONinput, and a list of them at aJSON[]input, now build, where a list of plain objects used to be refused with an error naming neither the input nor its concept.- A single
Anythingvalue survives the transport boundary: a stuff whose concept isnative.Anythingkeeps its content's class throughdump_for_transportand hydration, which a distributed or sandboxed run relies on; hydration used to fail looking for anAnythingContentclass. - A composite nested in an
Anythingvalue or in another composite survives transport: its components, a nested composite and a list of composites included, come back through hydration as typed contents, where they came back as dicts carrying class markers. - The result files of an
Anythingoutput render its value: a delivery whose main output isAnythingrendersmain_stuff.json,.mdand.htmlfrom the hydrated content, where it fell back to a raw dump of the transport form. - A prebuilt list is refused where one value belongs: a
ListContentgiven as an item of a plural input, or as the content of anAnythingenvelope at a single input, is refused instead of being stored nested or in a singular slot. bedrock_anthropicmodels run withAWS_BEARER_TOKEN_BEDROCKset: underbedrock_access_variant = "aws_access", the default, Anthropic models on Bedrock sign their requests with the configured AWS access keys and ignore a Bedrock bearer token in the environment, where every one of them used to fail at client construction with a rawValueError. When the access keys are missing and the environment carries such a token, theAwsCredentialsErrornow says so and namesbedrock_access_variant = "bedrock_token"as the way to authenticate with a bearer token.- Structured output on the
googlebackend accepts enum fields: underinstructor/genai_tools,instructorvalidated Gemini's function-call arguments strictly and refused a string for any enum field, so every schema with an enum failed on every attempt; those arguments are now validated in lax mode, so agoogle.tomlan earlierpipelex initcopied keeps working unchanged. The backend file we ship now setsstructure_method = "instructor/genai_structured_outputs", Gemini's native JSON output. - Structured output on the
googlebackend sends the system prompt, temperature and token limit:instructorignored the Google config pipelex handed it, so every structured call ran with no system prompt, the default temperature and the model's default output token limit, whichever structure method the backend used; they now reach Gemini. - A transport error inside a structured-output call is classified again under
instructor1.15.3 and later: those releases wrap a provider's rate limit, timeout or refused connection in their own retry exception, which pipelex classified as an unknown error, or after a re-ask as the earlier parse failure; the provider's error is now read first, so it keeps its category, its retryability and its provider metadata. - A gateway that does not allow the model asks for another model: the inference gateway's
model_not_allowed_errorrefusal (HTTP 412) now reads as a configuration error whosechange_modelaction names the model handle and says who settles a model deck that lists a model the gateway refuses, where it said to review the prompt, parameters and inputs. The HTTP status stays 500. - A structured-output call is re-asked when the model returns no tool call or no JSON: under
instructor1.16 and later such a response raises a parse error that pipelex's re-ask loop did not retry, so it failed at once; the loop now retries exactly the parse errorsinstructoritself re-asks. - A model whose
structure_methodnames a provider-specificinstructormode sets up again:instructor1.17 refuses another provider's mode on an OpenAI-compatible client, which failed the setup of such a model, the gateway'smistral-largeamong them; each structure method now builds with the core modeinstructoritself resolves it to. - Mistral OCR on an image returns the figures it finds: an extract from an image, rather than a document, failed with
MistralExtractResponseErrorwhenever Mistral OCR detected a figure inside it, because the request never asked for the figures' pixels; it now sends the same image options as a document extract, somax_nb_imagesandimage_min_sizeapply to images too, and a request for captions, which Mistral OCR cannot write, is refused on an image as it already was on a document instead of returning none. - A no-op tracer no longer stamps a fake Pipelex span on log lines: from
opentelemetry-api1.40, the no-op tracerOTEL_SDK_DISABLEDhands out returns its parent instead of an invalid span, so every line of a run carriedpipelex.trace_idand apipelex.span_idof0000000000000001naming no span; a pipe or an LLM call now starts no span under such a tracer, as with no tracer at all.
Removed
pipelex.providers.openai.openai_func(Breaking): the module and itscreate_pydantic_model_from_function,create_openai_schema_from_functionandlist_openai_toolshelpers are removed; nothing in pipelex called them.
[v0.68.0] - 2026-09-27
Highlights
Every log line can be joined to its run and to your own traces. A run driven through the kernel, each kernel step and a run dispatched to a distributed orchestrator's workers now stamp request_id, pipeline_run_id and pipe_run_id on their lines as an interpreted run does, and a host passes the inbound request id it filters its logs on. A line's standard trace fields now name your process's current OpenTelemetry span, with Pipelex's own span beside them under pipelex.*, and the gcp sink refuses to boot on credentials Google rejects instead of losing every record in silence.
Added
PipelexKernel.log_context()andJobMetadata.log_context():JobMetadata.log_context()binds a job'srequest_id,pipeline_run_idandpipe_run_idonto the log context in one call, andPipelexKernel.log_context()binds a kernel run'srequest_idandpipeline_run_id, which a host wraps its run in so the lines it emits between kernel calls name the run.PipelexKernel.maketakesrequest_idandpipeline_run_id: a kernel-driven run can now carry the inbound request id a hosted deployment filters its logs on, and name itself with an id the host chooses. A host inside a replay-based executor passes apipeline_run_idor a trace context, since the defaultuuid4changes on every replay; a trace context and apipeline_run_idthat disagree, or an emptypipeline_run_id, raiseValueError.PipelexMTHDSProtocol.executetakesrequest_id: a host runningexecutepasses the inbound request id, which the run'sRunMetadatacarries, so the log lines of a run dispatched to a distributed orchestrator's workers carryrequest_idas a direct run's do. A value that is not one to 128 printable ASCII characters is refused with aValueErrorbefore the run is set up, where it used to surface as aPipeExecutionErrorblaming the caller's input; the constraint is stated once, asvalidate_request_idinpipelex.system.job_metadata, whichRunMetadataapplies too.
Changed
- The
gcplog sink stops the boot when Google refuses its credentials (Breaking): the sink refreshes its credentials once when it is built, within five seconds, and credentials Google refuses (a revoked or expired refresh token, a deleted service-account key, a Google Cloud machine with no service account) or that cannot be loaded (no Application Default Credentials, a key file that is missing or holds no valid key) raise the newGcpLogSinkCredentialsError, aconfig-domainCredentialsErrornaming the credentials, Google's answer and what renews them, whichpipelex doctorreports in its log-sink row. Such a process used to boot and lose every record in silence, since over gRPC the failed refresh is retried for a minute before anything reports it; a refresh that cannot reach the token endpoint or the metadata server, gets an answer that says to retry (a5xx,408or429), or does not answer in time, is said on stderr and the boot goes on. - A log line's standard trace fields name OpenTelemetry's current span, and the Pipelex span rides under
pipelex.*(Breaking): in thejson,otlpandgcpsinks,trace_id,span_idandtrace_flags(theotlprecord's own trace context, thegcpentry'strace) now always name your process's current span and are absent when there is none, inside a Pipelex run too, where they used to name the pipe's or the LLM call's span; that span is written beside them aspipelex.trace_idandpipelex.span_id, keys in thejsonline and thegcppayload and attributes on theotlprecord, reserved on every line like the sinks' other keys. Thegcpentry'straceis no longer derived frompipeline_run_idand is set only under a current span, with the newspanIdandtraceSampledbeside it, and thepipeline_run_idlabel still selects a run's lines. If you export Pipelex's spans to your own backend, join a line inside a run to them onpipelex.trace_idandpipelex.span_id. For a sink plugin,span_context_for_logs()andotel_context_for_logs()are replaced bycurrent_span_context_for_logs(),pipelex_span_context_for_logs()andpipelex_trace_fields_for_logs()inpipelex.system.telemetry.current_span.
Fixed
- The
gcplog sink says when its flush runs out of time, and why: a teardown whose flush reaches its five-second deadline with records still held now prints on stderr that they are lost unless the transport sends them while it closes, followed by what a fresh refresh of the credentials answers, where only the client library's "Failed to send N pending logs." used to appear. A handler that has closed no longer flushes again when the process exits, which cost a second five-second wait. - A
GcpLogSinkinstalled a second time raises instead of sending nothing: installing the same sink object again after a reset built a handler on the transport the first teardown had closed, which queued every record and sent none;make_handlernow raisesRuntimeError, as theotlpsink does, and the registeredgcpfactory still builds a new sink at every boot. - A kernel step's log lines name its run and its step:
run_llm_text,run_llm_object,generate_object_content,run_extract,run_searchandrun_img_gennow bind thejob_metadatathey are handed for the whole call, so a program driving the kernel directly gets log records carryingrequest_id,pipeline_run_idandpipe_run_id, as an interpreted run's do. An interpreted run's lines are unchanged.
[v0.67.1] - 2026-09-27
Changed
pipelex-agent --runner api runreports the runner's refusal, not its status line: when the API runner answersrun pipe,run bundleorrun methodwith a non-2xx status, the error envelope, in markdown and in JSON, now carries what its problem document says: the runner's error class aserror_type(ValidateBundleError,ModelNotFoundError,StuffFactoryError), amessagequoting its reason, which names the failing pipe and its path on a failed run, its next step ashint, itserror_domain,error_categoryandretryable, itsvalidation_errorsrendered asvalidaterenders them,pipe_codeandpipe_stackwhen it carries them, itscodeaserror_code, theRetry-Afterdelay asretry_after_seconds, andhttp_statusandrequest_id, where the envelope used to read onlyHTTPStatusErrorandClient error '422 Unprocessable Entity'. An answer that advises no next step gets one from its status: a 401 or 403 points at the API key, a 404 atMTHDS_BASE_URL, a 429 says to wait and retry and isretryable, a 503 warns that a retry can repeat a paid run, any other 4xx says to change the request, and any other status says to report therequest_id. It requires mthds 0.17.0, whose client raises the typedApiResponseError.
Fixed
- The API runner's credentials hint names the variables it reads: a
ClientAuthenticationErrornow says to setMTHDS_API_KEYandMTHDS_BASE_URLor to runmthds config set, where it pointed atpipelex-agent doctorandPIPELEX_API_KEY, which the API runner does not read.
[v0.67.0] - 2026-09-27
Added
- An unknown model is a located
unknown_modelvalidation item: a pipe whose model field names a handle, alias, preset or waterfall the model deck does not define now validates to an invalid verdict with onepipe_validationitem of the new closederror_typeunknown_model, carrying thepipe_code,domain_code,source,field_name, thefield_pathof the reference (pipe.<code>.model, orpipe.<code>.model_to_structure), and the new optional item fieldsmodel_reference(the reference as written),model_typeandsuggestions(the deck's close matches of the same kind, also kept in the message). Every pipe type that names a model gives the same item, wherePipeExtractandPipeSearchused to give anunknown_validation_errorwhose message was a Python repr, and with exactly one suggestion the item carries an unsaferename-modelfix, whichpipelex fix bundlenever applies on its own because a fuzzy match can name a different model. The MTHDS Test Corpus gains theinvalid_unknown_modelentry covering the newerror.unknown_modeltag. - An unsafe suggested fix says so wherever it is printed: the per-error line of
pipelex validate, ofpipelex fix's remaining errors and of everypipelex-agentMarkdown rendering, therendered_markdownshared with/validateincluded, reads💡 Suggested fix (unsafe, confirm before applying):for a fix whosesafetyisunsafe, while a safe fix keeps the plain💡 Suggested fix:line, so a person or an agent can tell which fixes to check before making them by hand. - A run graph marks a list-valued stuff as a list: every io item of a GraphSpec produced by a dry or live run now carries
multiplicity,truewhen the stuff is a list andnullotherwise, so a renderer shows an input declaredDocument[], aRecord[]output, a batch's aggregate, a step run withnb_outputor a lifted plural output as a list, on both the in-process graph and the one assembled from trace events. A trace event carrying the field cannot be read by an earlier pipelex, so the process that assembles a graph must run a pipelex at least as new as the workers that wrote its events. The Mermaid renderer labels such a stuffRecord[], and its labels now keep square brackets instead of turning them into parentheses. - Validation items carry the positions and declared concepts the parser knew: a TOML syntax error's item carries the 1-based
lineandcolumnthe parser stopped at, as new optional item fields, and anunresolved_conceptitem carriesdeclared_concepts, the bare codes of the concepts the validated bundle declares in its domain, so an editor can anchor the diagnostic and an author sees what the reference could have named.
Changed
- Every refusal raised while loading a bundle is a validation verdict (Breaking): validation turns any refusal of the caller's input that has no arm of its own into an invalid verdict with one item, keeping the refusal's message only when it is caller-facing and otherwise its title, while a configuration or runtime fault still propagates as no verdict. A refusal raised while a pipe is built leaves the library load as the new
PipeLoadRefusalError, naming the pipe, its domain and its file with the original as its cause, and gives apipe_validationitem located there; any other gives ablueprint_validationitem.pipelex validate bundleno longer prints a traceback for an unknown model,pipelex-agent validate bundleanswers it withis_valid: falseand exit 1 instead of the no-verdict envelope and exit 2, and the in-process validator behind/validatereturns it as an invalid verdict instead of raising. PipeOperatorModelChoiceErroris the located unknown-model refusal (Breaking): it is now raised whenever a pipe naming an unknown model is built, with a message naming the pipe and the field; it isinput-domained and caller-facing, carriesdomain_code,field_name,suggestionsandsource, and itsmodel_choiceis the reference as written. The agent CLI's hint for it points atcheck-modelrather thandoctor.- A
PipeParallelbranch whose multiplicity differs from its field's is refused with the fix to make: when the combine of the branch results refuses a list branch fed into a single field, or a single branch fed into a list field, the refusal now names the parallel, the branch, the result and the field, and says to declare the field with the branch's multiplicity or to make the branch produce the field's, instead of reporting a Python validation error aboutListContent. The dry run ofpipelex validateandpipelex-agent validatecarry it in theirdry_runitem, and a run reports it as aStuffFactoryErrorin theinputdomain, so a hosted runner under STRICT disclosure answers a 422 whose message andchange_inputnext step name the change, where it answeredAn internal error occurred.. - The invalid-bundle verdict's next step points at its items:
ValidateBundleError's user action, the💡 Tip:ofpipelex validateand thehintofpipelex-agent's JSON, now reads "Edit the bundle as each validation error says: apply its suggested fix where it has one, after confirming an unsafe one" instead of "Check the validation_errors array for specific issues". - A run refuses an invalid bundle with the validation verdict (Breaking): a run loads its bundle before any pipe runs, and a bundle that load refuses now raises
ValidateBundleErrorwith the same locatedvalidation_errorsvalidating it gives, which a hosted run route answers as an HTTP 422 that keeps them under STRICT disclosure, instead of the raw class of whichever check refused it (a 500 for a misspelled concept or a wiring mismatch, a 422 without items for a TOML fault, aruntimePipeExecutionErrorfor a check inside a pipe's validator, aPipeOperatorModelChoiceErrorfor an unknown model); an unknown entry pipe still raises itsPipeNotFoundError.pipelex runprints the grouped panelpipelex validateprints,pipelex-agent runand any other agent error envelope caused by such a verdict carry itsvalidation_errors, and the runner's newlibrary_dirs_are_callersoption, which both CLIs set, gives a refusal in the caller's own library directories the same verdict while a host's directories still load untranslated. - A pipe factory's refusal is a located validation item:
PipeLLMFactoryError,PipeComposeFactoryError,PipeExtractFactoryError,PipeConditionFactoryErrorandPipeParallelFactoryErrorare nowinput-domained and caller-facing, so aPipeExtractwhose input is neither an image nor a document validates to apipe_validationitem on that pipe and its file, whose message names the input and says to redeclare it, where it used to escape every validator as no verdict; aPipeLLMwhose prompt or inherited system prompt does not parse is refused with the line of that prompt it fails at and no longer quotes any of the prompt, which may be a host's; the builder'svalidate_ops.validate_pipeandvalidate_allalso answer a refusal raised while they load libraries or dry-run a pipe with theValidateBundleErrorverdict instead of the raw error.pipelex-agent validate pipe --allkeeps the file on an item raised while its libraries load. pipelex validate pipeandpipelex validate --allrender a refusal as an invalid bundle: a pipe whose dry run fails, and a refusal of the libraries while they load, now print the grouped invalid-bundle output and exit 1 instead of a traceback;pipelex-agent validate pipeand--allanswer a load-time refusal with the invalid-verdict envelope.- A failing dry run is one located item per failing pipe (Breaking): validation reports a dry-run failure as one
dry_runitem per failing pipe, with the error typeDryRunErrorand thepipe_code,domain_codeandsourceof the innermost pipe that failed and is not allowed to fail (when submitted content is validated beside a host's library directories, thesourceonly for that content's own pipes, never a file of the host's own library directories), where it used to report one item holding a Python repr of the sweep's records, once more for every controller the failure passed through. The item keeps the failure's own message only when that message is caller-facing and otherwise names the failure's title, so a configuration or storage failure met during a dry run no longer reaches a hosted caller through the verdict;pipelex validatelists the items underDry Run Errors:,pipelex-agent validate pipeandvalidate pipe --allanswer a failing dry run with the invalid-verdict envelope and its items instead of aDryRunErrorenvelope with none, and the agentinputscommands drop theirdry_run_errorkey. In Python,ValidateBundleErrortakesdry_run_failuresin place ofdry_run_error_message, andDryRunErrorcarriesfailures. - A run failure reports its root fault, located at the failing pipe (Breaking): the report of a
PipelineExecutionError, and of thePipeRouterErrorunder it, now carries theerror_type,title,type_uri, caller-facing flag and own message of the root fault, the innermost Pipelex error on the cause chain, with the message prefixed by the failing pipe and its path (Pipe 'summarize' failed (two_steps → summarize): …), and the exception'spipe_codeandpipe_stackname that pipe instead of the entry pipe. A bridge's sentence no longer reaches the report, a report recovered from a remote worker is taken as it is, so a run reports the same failure in process, through the in-process orchestrator or on a worker, and the fallback action names the failing pipe instead of sayingCheck pipe_stack to identify which pipe failed. A consumer branching onerror_type == "PipelineExecutionError"branches on the root fault's type instead, as the error model describes. - The pipe router locates every failure (Breaking):
PipeRouterProtocol.run()now re-raises any failure of its pipe as aPipeRouterErrorchained to it, where it used to let everything but aCogtErrorthrough, and a failure already located by an inner router rises untouched. An exception that is not a Pipelex error becomes aPipelexUnexpectedErrorwhose message names its class, and a host router overrides the new_as_pipelex_failure()hook to convert its transport's failures or let a control-flow exception through. Code that caught a pipe's own error class fromget_pipe_router().run()catchesPipeRouterErrorand reads its__cause__. ModelNotFoundError's message states the fact only: a handle missing from the model deck readsModel handle '<handle>' was not found in the model deck., without the local-deck advice it used to carry into every surface, hosted runs included.pipelex runnow renders the model panel for a run failure, naming the failing pipe, the model and the pipe stack, and the panel's tip carries that advice.- The agent CLI's run errors name the root fault:
pipelex-agent runreports a failed run'spipe_codeandpipe_stackat the failing pipe, and itscause_typeandcause_messageare the root fault's instead of the wrapper right under the run error.
Fixed
- Every validation error reaches the reader: an error the blueprint categorizer does not know, such as a misspelled field, is its own
blueprint_validationitem without anerror_type, carrying itssource,pipe_codeandfield_path, where it used to vanish whenever another item existed and surface only once that one was fixed; andformat_pydantic_validation_errorrenders every error it is given, listing the kinds without a heading of their own under "Other validation errors" whatever else is present. - A validation verdict names no path on the host: an item located inside a package the method depends on by address, and a duplicate declaration inside that package, now name the bundle by the package's address and its path inside the package (
github.com/acme/harbour-methods/tides/notices/tide_notices.mthds) instead of the directory the host installed it in, on every surface; and when submitted content is validated or run beside a host's own library directories, an item located in one of their files carries nosourceorfield_pathnaming it and a message that named such a file, or the module one of their Python files was imported as, reads<host library file>, so STRICT disclosure no longer hands a hosted caller a path on the server. A bundle validated from a file keeps the full paths of its own library directories, and so does content validated against directories that are the caller's:validate_bundleandvalidate_bundles_in_processtakelibrary_dirs_are_callers, whichPipelexMTHDSProtocol.validatenow passes as itsexecutedoes. - A run that fails on the caller's own method reads its reason under STRICT disclosure: a
PipeParallelwhose branch results do not fit its output, a pipe started without a required input, aPipeConditionwhose expression renders nothing, whose outcome isfailor whose chosen pipe misses inputs, and a step naming in an inline setting a model the deck neither defines nor names now report theinputdomain, so an HTTP surface answers 422 instead of 500, a message STRICT disclosure keeps instead ofAn internal error occurred., and a next step: for the model, achange_modelaction naming the model as the method wrote it and the type of model its pipe needs, which the model panel ofpipelex runfollows with its local-deck advice. APipeCondition's dry-run refusals, among them an expression rendering nothing and every outcome mapped tofail, now give adry_runitem carrying their reason, and name the missing pipes instead of printing the outcome map; afailoutcome's message names the pipe and no longer the value its expression rendered, and neither does the absence reason acontinueoutcome records, which the message of a step force-unwrapping the condition's output quotes under STRICT disclosure. A model that the deck itself names but does not serve stays a redactedconfigerror. The newPipelexError.as_caller_fault()is how a raise site classifies one error of a class whose other instances are not the caller's fault, as the error model describes. - A
PipeConditionwhose expression does not parse is an item of the verdict: validation refuses it as ablueprint_validationitem on the condition, naming itsexpressionorexpression_templateand the line of it that fails, and refuses the same way an expression that parses but that the run could not compile, such as one naming a filter or a test that does not exist outside any conditional (Jinja2 leaves a name inside an{% if %}or an inlineifto be resolved when the expression renders); a run of the bundle is refused with that verdict, which a hosted run answers with HTTP 422, where the parser's error used to escape the load with no verdict and a hosted run answered 500An internal error occurred.. The item quotes neither the expression nor the parser's diagnosis, since the condition may be a host library's.
[v0.66.1] - 2026-09-26
Added
load_from_cratetakesis_crate_prevalidated:LibraryManager.load_from_crate(library_id=…, crate=…, is_crate_prevalidated=True)skips the library validation that ends a load, for a crate built from a library that the same pipelex version already loaded and validated, such as the crate a distributed run hands its worker; everything else the load does still runs, and the default stays the validating load.
Changed
are_classes_equivalentanswers without a JSON schema when it can: a class is equivalent to itself, and two plain models with different field names are not equivalent, both now decided before any schema is generated, which removes most of the cost of validating a library whose pipes output structured concepts. A plain model is one built only from an allow-list of annotations (scalars,Any,Literal, unions, builtin containers and nested plain models), inert constraints and plain defaults, with no alias and no schema hook or schema-shaping setting, so the properties pydantic publishes for it provably equal its field names; the structure classes generated from a concept'sstructuretable and most native content classes are plain. Verdicts are unchanged: every other pair still goes to the full schema comparison, and a schema that cannot be generated still raises.
[v0.66.0] - 2026-09-25
Highlights
Logs are structured, scrubbed and sent where you choose. Every log call can carry named fields and the run's identifiers, one configuration key selects the sink (the Rich console, JSON lines for a log agent, OpenTelemetry, Google Cloud Logging or a plugin's own), secrets are redacted before any sink sees a record, and a line logged inside a span carries its trace context. Rich moves behind the cli extra, so the runtime no longer needs it on the way to running a method.
Added
- Named fields and a run-scoped context on the log calls: the seven
log.*methods take a keyword-onlyfields={...}, carried as attributes of the stdlibLogRecordand never rendered into the message, andwith log.context(request_id=..., pipeline_run_id=..., pipe_run_id=...)stamps the run's identifiers onto every record emitted in scope, nested bindings merging and an absent identifier staying absent rather than readingNone. A direct-mode run binds its request and pipeline identifiers from the job's metadata, and every pipe binds its ownpipe_run_idaround the whole of its run. Adictorlistcontent is also carried as the record'sdataattribute, a JSON-ready snapshot of the call, and a field named like an attribute the record already owns, like a name Pipelex sets on it,dataincluded, or like a key a sink writes itself rides under afield_prefix instead of raising or overwriting it, as Logging describes. - Log sinks are a plugin capability selected by
[runtime.log] sink(Breaking): where the records go is a registered sink token:console(the Rich handler, the default),json(one JSON object per line carryingtime,severity,logger,message, the fields, the context identifiers and the exception, with no ANSI ever, for a server behind a log agent),otlp(the OpenTelemetry logs signal, configured under[runtime.log.otlp]), or a token an installed plugin registers throughadd_log_sink. A token nobody provides stops the boot naming the registered ones, andpipelex doctorreports it in a row of its own.log_mode,poor_loggers,generic_poor_loggerandis_console_logging_enabledare gone with the rich-or-poor mode, andset_log_mode,set_poor_log_formatterandLevelAndEmojiLogFormatterwith them, so a server that wants plain output selectsjsonand a host that wants Pipelex silent setspipelex = "OFF"under[runtime.log.package_log_levels]. Migration: runpipelex migrate; ledger entrypipelex-config@5deletes the four retired keys from~/.pipelex/and every project.pipelex/file, after which the defaults supplysink = "console". A file that hadlog_mode = "poor"oris_console_logging_enabled = falsechose a behaviour that no migration may write back, so it then setssink = "json"orpipelex = "OFF"under[runtime.log.package_log_levels]by hand, as the entry's title says. Log Sink Plugins documents the seam. - A
gcplog sink writing to Google Cloud Logging:sink = "gcp"sends each record to Cloud Logging through thegoogle-cloud-loggingclient library, which the newgcp-loggingextra installs, and selecting it without the extra stops the boot naming the extra. Each record becomes one structured entry, with the level mapped onto Cloud Logging's severities, the fields in its JSON payload, the run's identifiers as its labels and the run's trace id in itstracefield, so a line is filed with the spans of its run; the entries leave through the library's background transport, and[runtime.log.gcp]carries the log name, the project and an optional service-account credentials path. Most processes on Google Cloud wantjsoninstead: Cloud Run, GKE and any platform whose logging agent reads the container's stdout ingest what that sink writes with no client at all, as Logging Configuration spells out. - Log lines carry the trace context of the span they were logged in: inside a valid span the
jsonsink writestrace_id,span_idandtrace_flagsin lowercase hex, the keys OpenTelemetry specifies for trace context in a JSON log that is not OTLP, and theotlpsink files each record under that span. The span is the Pipelex span active at the log call, a pipe's or an LLM call's, whenever the runtime traces a run, and otherwise OpenTelemetry's current span, only read, so a line your code logs under its own span joins your trace. Pipelex never makes its own spans current in the process's OpenTelemetry context, so a host's instrumentation is never re-parented under them, as Logging describes. - Secrets are scrubbed and field values made injection-safe before any sink sees a record: one processor, on by default and configured under
[runtime.log.redaction], replaces with[REDACTED]the credential of anAuthorizationheader, an api-key or signature header, an OAuthcodeparameter, every cookie crumb, aname=valuesecret entry and ansk_/plx_sk_/pk_/bl_key, in the message, in the rendered exception and in every string a field carries, and the whole value of any field or mapping entry, at any depth and in a structured content too, whose name is a secret's (authorization,password,api_key,access_tokenand the like). Control characters in a field's string values become printable escapes, so a caller-supplied string cannot forge a line or a terminal colour, andextra_patternsadds the secret shapes only a deployment knows, a pattern theremodule refuses or one matching the empty string being a configuration error at load. The processor runs ahead of every sink, and of a handler the host adds after it, and fails closed: a record it cannot redact is reduced to a[REDACTION FAILED: …]notice or dropped, and a sink processor that raises is reported on stderr by its type and name, never by its text, as Logging describes.
Changed
- Rich is the
cliextra (Breaking):richis no longer a core dependency, so installpipelex[cli]for thepipelexandpipelex-agentcommands, theconsolelog sink and therichpretty-print mode (uv tool install "pipelex[cli]"). The runtime imports Rich only inside the code that renders, so a method runs withsink = "json"andpretty_print_modeset to"poor"or"silent"without it, while a boot selecting theconsolesink or therichmode without Rich stops naming the extra, andget_console(),rendered_pretty(), the model and pipe listings and the directory diff raiseMissingDependencyErrornaming it too.PrettyPrintable,TextTypeandStyleTypeinpipelex/tools/misc/pretty.pynow exist only for type checking, so importing them at runtime raisesImportError. A plainpip install pipelexstill installs Rich andimport pipelexstill loads it, since the core dependenciestyperandinstructorboth require it, and apoor-mode panel title no longer substitutes emoji codes; Rich Imports documents the convention and itsmake check-rich-importsguard. - Loggers are named by module (Breaking): a record from
pipelex/pipe_operators/pipe_llm.pyis emitted on thepipelex.pipe_operators.pipe_llmlogger instead ofpipelex, so a handler keyed on the bare package name must key on the prefix, and a filter an integration attached to thepipelexlogger must move to its handler, since Python never consults an ancestor logger's filters for a record propagating up from a module logger. Apackage_log_levelskey now works at module depth too,pipelex-pipe_operators-pipe_llm = "DEBUG"for instance, and the caller-info templates that name the module now render instead of raising. include_exceptioncarries the traceback as the record'sexc_info: instead of splicingtraceback.format_exc()into the message, so the console renders one Rich traceback, thejsonsink anexceptionfield and theotlpsink theexception.typeandexception.stacktraceattributes, and a call made outside anexceptblock carries nothing where aNoneType: Noneline used to be appended.- Teardown removes only the handlers Pipelex installed:
log.reset()no longer strips every handler off the root logger nor callslogging.shutdown(), so a host's or a test's handler survives a Pipelex teardown; the sink's handler is flushed and closed, which is where a batching sink ships what it still holds. A sink installed again after the reset builds a fresh handler, except anOtlpLogSinkobject, whose close shuts its processor down, so installing the same one a second time raises; the registered factory builds a new one at every boot. - The pitch names who each way of running a method is for: the README, the documentation's front page and Quick Start, and the site description now say a method runs "as an MCP for chatbots, as a webapp for people, or via API for your software", instead of "from your agent or your chatbot via MCP, as a webapp, or via API in any software".
- The quick start says the plugin also puts a method in your software: the README and the documentation's Quick Start now say the plugin's skills build methods, run them and put them in your software, name
/pipelex-scaffoldfor a webapp and/pipelex-integratefor a typed call from TypeScript or Python code, and say that@pipelex/sdkis the TypeScript SDK andpipelex-sdkthe Python one. - The documentation's cookbook is one page that points to the cookbook: the cookbook now holds Pipelex's example methods as packages you run by their address on the hosted API, each with its own page showing every way to use it, so the documentation no longer keeps a page per example. The Pipelex Cookbook says what the cookbook is and links to it, the pages that linked an example now link its page in the cookbook or in the method library, and every old example address redirects to the new page. The two pages that documented the runtime itself move into the runtime's own sections: Writing an Inference Plugin, under Under the Hood, now carries the whole sample plugin and the configuration that routes a model to it, and CV Batch Screening, Step by Step joins Get Started. The README's cookbook line says what the cookbook now is.
Fixed
- S3 links name the bucket's own host (Breaking): the S3 storage provider now signs links, and reads and writes objects, on the bucket's regional host
<bucket>.s3.<region>.amazonaws.cominstead of the region's shared endpoints3.<region>.amazonaws.com, so a content security policy can allow one bucket's links without allowing every bucket in the region. A legacy bucket name that cannot be a hostname, one with uppercase letters or an underscore, keeps the path-style form on the regional endpoint, for signed and unsigned links alike, where an unsigned link used to name a virtual host that could not serve it; the configuration still refuses a bucket name containing a dot. A self-hosted deployment whose egress rules allow S3 by hostname must allow*.s3.<region>.amazonaws.com. - Image and document inputs carry a
public_urlwhatever form their url takes: an input given as apipelex-storage://reference now gets a link signed through the storage provider, replacing any link it carried, which may have expired, and an input given as anhttp(s)URL gets that URL as itspublic_urlunless it names another. This holds for an image or document passed alone, in a list or in a structured field, so a template writing{{ image.public_url }}no longer rendersNonefor one uploaded to storage or passed by URL, and a reference the storage provider refuses as a key is reported as an input error before the run starts. - A NUL byte in an input's path or storage reference is an input error: a local file path, or a
pipelex-storage://reference on local storage, holding a NUL byte used to escape as an internal error; it is now reported as an input error before the run starts. - Unsigned storage links percent-encode their key: the S3 and GCS providers' unsigned links now encode a key holding a space,
+,#or?as a signed link does, instead of naming a path the URL cuts short. - A cancelled pipe run ends its span: the pipe's span was closed only when the run raised an
Exception, so a run cancelled by a timeout, a disconnected client or a cancelled orchestration task never ended its span and no exporter received it. It now ends with an error status like any other failure. - A log call before
log.configureno longer raises: it goes to the stdlib's default handling at the stdlib's default level, so a library that logs before Pipelex boots, or a boot that logs while it configures, cannot crash on the log line itself. Betweenlog.configureand the sink's arrival, the boot's records are held and replayed through the selected sink in order, and a boot that dies before its sink arrives sends all of them to stderr. - A log call never raises on a content JSON refuses or on a bracket Rich reads as markup: a content
jsonrefuses, a circular reference or a mapping with a non-string key, is rendered as itsreprwith nodatainstead of raisingValueErrororTypeErrorout oflog.info. On theconsolesink, a message carrying a tag-shaped span,list[int]in a type complaint or a bracketed path, no longer raisesMarkupErrorout of the log call, and an error assembled from an exception is rendered verbatim rather than read as markup.
[v0.65.0] - 2026-09-25
Changed
BundleValidatorProtocol.validate_bundlesrequiresgraph_pipe_code(Breaking): a host names the pipe the best-effort graph arm dry-runs, resolved the way a run resolves its entry pipe, withNonekeeping the primary blueprint'smain_pipe. A host validating a fetched method package by address can now graph the pipe its manifest names, which a bundle declaring nomain_pipeof its own left without a graph. Every validator implementation must accept the argument and hand it to its graph arm, and a host must pass it;DirectBundleValidatorforwards it tovalidate_bundles_in_process.- The quick start shows both agents' installs: the README's Codex install now starts open beside the Claude Code one instead of folded, and the documentation site links its content tabs, so choosing Codex on one page selects it on every page.
- The documentation's front page takes the README's hero: it opens on "Turn your expertise into an AI-powered App, MCP or API" and the paragraph on building a method with your coding agent and where it runs. Get Started now follows the buttons, and What a Method Looks Like, which now introduces the MTHDS language, and Why Methods? come after it. The front page's entry in the navigation is now Overview.
[v0.64.2] - 2026-09-24
Fixed
- The quick start says the Pipelex plugin needs Node.js: the README's quick start and the documentation's Quick Start and Pipelex Plugin pages said, in the Claude Code install step, that the plugin needed nothing else to install. Both agents' install steps now say that the plugin's hook and the Pipelex tools run on Node.js, so you need Node.js on your
PATH; without it the hook passes silently and the tools never start. - An unknown model name keeps its message on a hosted surface:
ModelChoiceNotFoundErroris now caller-facing, so under STRICT disclosure its message — the model reference the method named and the "Did you mean" suggestions — reaches the caller instead of "An internal error occurred.". A hosted dry run or run of a method naming a model the deck does not know now says which one and what to use instead.
[v0.64.1] - 2026-09-24
Changed
- The README leads with the hosted route: its quick start is the onboarding region included from
Pipelex/.github— sign up at app.pipelex.com, then the Pipelex plugin in Claude Code or Codex or the Pipelex MCP in ChatGPT or Claude — and the runtime's own install, configuration and first run follow under their own heading, Run it yourself. The CV batch screening walkthrough moved from the README to the cookbook in the documentation. Its hero is the website's line, "Turn your expertise into an AI-powered App, MCP or API", with a paragraph on building a method with your coding agent and where it runs, and What a method looks like and Why methods? follow the install instead of preceding it. Configure AI Access no longer offers the Pipelex Gateway: it names your own provider keys or a local model. - The documentation site leads with the hosted route: its getting-started opens on a new Quick Start page — sign up at app.pipelex.com, the Pipelex plugin in Claude Code or Codex, the Pipelex MCP in ChatGPT or Claude, then a webapp from the method-app template or the hosted API over HTTP, TypeScript or Python — and the runtime's own install moves to Run It Yourself. Build with Claude Code and its
npm install -g mthdsroute are retired, and Claude Code Skills Plugin becomes Pipelex Plugin, which documents the Pipelex plugin instead of the MTHDS plugin; the old addresses redirect. The front page's buttons and the site's description lead with what Pipelex lets you do. mthdsversion: The pin moves tomthds==0.16.0, the release adding aUser-Agenton everyMthdsAPIClientrequest and themthds.versionmodule. Nothing in pipelex uses either, and the MTHDS standard version stays at2.0.0, but the pin is exact, so everyone downstream inheritsmthds0.16.0 and a consumer depending onmthdsdirectly must move in step.
Fixed
- Test runs no longer send the Pipelex Gateway telemetry stream: a runtime booted in the
pytestorciintegration mode, or in a session that loads the shared pytest plugin, which sets a test run mode when pytest configures, keeps the Gateway stream off even withpipelex_gatewayenabled, so a project's test suite no longer reports its runs and its fixture user ids to Pipelex. In a test run a boot no longer needsPIPELEX_GATEWAY_API_KEYfor telemetry,DO_NOT_TRACKno longer conflicts with the Gateway, and the Gateway's Portkey debug and tracing flags have no effect, even where the custom stream is allowed;IntegrationMode.is_test_harnessnames the two modes. The plugin's session fixtureset_run_modeis gone: the run mode is set in itspytest_configure, so a boot at test-module import or during collection is covered too.
[v0.64.0] - 2026-09-24
Added
- A caller can be known without a run:
CallerIdentity(pipelex.system.caller_identity) holds theuser_idandextrasa host would state on a run, andscoped_caller_identitymakes one the ambient caller for a block. An event emitted with noRunMetadatain hand reads the caller in scope before it falls back to the configureduser_id, andPipelexMTHDSProtocol.caller_identityexposes the caller a protocol was built for.
Changed
- A run's
analytics_groupsis renamedextras(Breaking): the base library no longer names an analytics concept in its run model or its job-preparation signatures.RunMetadata.analytics_groups,PipelexPipeRunInput.analytics_groupson the runtime-bridge payload, and theanalytics_groupsparameter ofpipeline_run_setup,prepare_pipe_job,PipelexKernel.makeand thePipelexMTHDSProtocolconstructor are nowextras: an opaquedict[str, str]of labels the host attaches to a run, which the library never reads by name. Behaviour is unchanged: the mapping is validated where it enters under the same bounds, and telemetry still forwards it whole as the PostHog groups of each capture. Migration: rename everyanalytics_groups=argument and field toextras; there is no alias, and while the functions andPipelexPipeRunInputrefuse the old name,RunMetadatasilently ignores it, so a missed rename there drops the groups without an error. Import frompipelex.system.run_extrasinstead ofpipelex.system.analytics_groups, callingvalidate_run_extrasinstead ofvalidate_analytics_groups, with the bounds now namedRUN_EXTRAS_KEY_PATTERN,RUN_EXTRAS_VALUE_PATTERNandRUN_EXTRAS_MAX_ENTRIES. A dashboard, query or exporter reading the span attributepipelex.run.analytics_groupsmust readpipelex.run.extras(PipelexSpanAttr.RUN_EXTRAS) instead. - Pipelex's own telemetry stream receives the run's
user_idas it is: the Gateway stream sent a one-way digest of the caller taken inside the Gateway-key hash, and carried adeploymentgroup instead of the run's own groups. It now resolves identity like the operator's identified stream — the run'suser_idis the PostHogdistinct_idand itsextrasride the groups facet — so a host's events and the runtime's land on the same person and the same organization. TheNAMESPACEDidentity policy and thedeploymentgroup type are removed. Anything that names no caller still reports under the Gateway-key hash. BundleValidatorProtocol.validate_bundlesrequirescaller_identity(Breaking): a validator is handed who asked for the validation,Nonemeaning nobody, so a hosted/validateattributes its telemetry to its caller.validate_bundles_in_process,validate_bundle,dry_run_pipe_in_processand theBundleValidatorsweep methods gain an optionalcaller_identityof their own. Migration: every plugin implementingBundleValidatorProtocolmust add a keyword-onlycaller_identity: CallerIdentity | Noneparameter tovalidate_bundlesand pass it on to the sweep it runs, carrying it to the worker when the validation is dispatched. Every host callingvalidate_bundlesmust now pass it explicitly:caller_identity=self.caller_identityfrom aPipelexMTHDSProtocolrunner, orNonefor a validation nobody in particular asked for.
Fixed
- A trace's first span carries its run: the trace-start
$ai_spancaptured on both PostHog streams now includespipeline_run_id, so the root of a trace joins its run's other events without going through a child span. - A validation is attributed to the caller who asked for it: the sweep's
pipe_dry_runevent carried no run, so on a hosted plane it went out under the deployment's configureduser_idwhoever validated. It is now emitted with the caller in scope, and the sweep's dry runs and the graph arm's dry run state that caller in their job metadata instead ofdry-run-no-user. A local validation names nobody and reports under the configured id as before. - A crash inside a run is attributed to that run's caller: the exception autocapture resolved one runless identity when it was built, so every
$exceptionon a hosted plane landed on the configured constant. Every pipe now runs with its run's caller in scope, an exception leaving a pipe carries that caller — read back through the__cause__and__context__chain, so a host's own error raisedfromit still resolves — and the capture resolves each error's identity when it arrives. An error raised outside every run still reports under the fallback, andmode = "anonymous"still identifies nobody.
[v0.63.0] - 2026-09-23
Added
- A run carries opaque analytics groups:
RunMetadataand the runtime-bridge payload gainanalytics_groups, a host-supplied mapping of group type to group key ({"organization": "org_acme"}on a multi-tenant host, nothing on a single-user one), threaded throughpipeline_run_setup,prepare_pipe_jobandPipelexKernel.makebesideuser_idandstorage_scope; likestorage_scope, the runtime never reads a key by name. It is validated where it enters — a lowercase snake_case group type, a group key fromA-Za-z0-9_-, and a small fixed number of group types per run — so a malformed mapping is a construction error naming the field rather than a failure inside a telemetry capture. - Telemetry is attributed to each run's caller: spans and events resolve their identity per capture through one
TelemetryIdentity, so a run'suser_idbecomes the PostHogdistinct_id, itsanalytics_groupsride PostHog's groups facet, and both are written onto the span aspipelex.run.user_idandpipelex.run.analytics_groupsfor every OpenTelemetry exporter, Langfuse'slangfuse.user.idincluded. Everything that names no caller of its own — an event outside any run, or a run whoseuser_idis a shared placeholder (local,dry-run-no-user,single-tenant) — falls back to the configureduser_idand the Gateway-key hash,mode = "anonymous"identifies nobody, and neither value ever becomes an event property or a span name. Pipelex's own Gateway stream receives neither the caller nor the groups, only a one-way digest of the caller taken inside the Gateway-key hash, so a host serving many callers from one process sees each caller on their own timeline without that identity leaving its own project.
Changed
RunMetadatais frozen (Breaking): a field can no longer be reassigned once the run exists, and an attempt raises a pydanticValidationErrorat runtime rather than merely failing a type check. Telemetry reads a run's identity off it while the run is in flight, so construct a newRunMetadatawherever one used to be mutated.
Fixed
- A crash no longer sends a
PipelexError's message as written: the exception autocapture sent the message verbatim — which may repeat a file path or a slice of the document that would not parse — even on the shipped default, because redaction applied only to an error handed over bare. EveryPipelexErrorPostHog would reach, across the whole__cause__and__context__chain and every member of an exception group, is now replaced by a same-class stand-in carrying the privacy notice, the class name and the traceback, on a copy so the exception the interpreter prints is untouched. An error of another class that copied aPipelexError's text into its own message still sends that text. - Both telemetry streams record a crash: PostHog's once-per-error stamp made the second stream — always Pipelex's own — drop every unhandled exception on a machine that also reports to an operator's project. The stamp is cleared between the streams now, and an error the host had already captured itself is still not recorded twice.
- An unhandled exception is captured under the same identity as everything else on its stream: the autocapture sent the configured
user_idand the Gateway-key hash raw, somode = "anonymous"still named you on every crash and a run's$exceptionlanded on a different PostHog person from its spans. Both streams now use the identity they resolve for any capture that belongs to no run. - An anonymous capture no longer marks the Gateway capture of the same event: both streams shared one properties dictionary, so the anonymous branch's
$process_person_profile = falsereached the Gateway capture sent with adistinct_id. Each capture now takes its own copy. - A malformed
storage_scopeis refused before the run registers itself: it was validated only inprepare_pipe_job, after the trace-start emission, so a scope carrying..announced a trace that cannot be unsent.pipeline_run_setupnow gates the caller's scope at its top and again after the local sentinel rebinds it, andprepare_pipe_jobkeeps its gate for callers reaching that seam directly.
[v0.62.0] - 2026-09-21
Added
- The previous multi-provider deck is parked in the package as a deck variant:
pipelex/kit/deck_variants/multi_provider/holds the four numbered deck files as they shipped before the default moved to Azure, so a project that wants the Anthropic, Google and OpenAI ladder back copies them over the kit's own deck directory instead of reconstructing it, and the directory's README says how. Nothing loads a variant —init,updateand the deck manifest ignore it, and it sits outsideconfigs/so no installer can pick it up — and a parity test holds every variant to the shipped deck's alias, preset and waterfall vocabulary and checks that each handle only the variant names is still declared by a kit backend file, so a parked deck goes red instead of rotting quietly.
Changed
- The shipped model deck resolves only to models the Pipelex Gateway serves from Azure (Breaking): every default alias and preset keeps its name, but the whole language ladder is now the GPT-5.6 range — the premium tier and
best-gptresolve togpt-5.6-sol, the general and large-context tiers togpt-5.6-terra, the small tiers togpt-5.6-luna— and image generation's general and premium tiers togpt-image-2;$engineering-codebase-analysisreaches its model through@default-large-context-code, the alias that names its job, instead of through the premium tier. Document extraction was alreadyazure-document-intelligenceand web search stays on Linkup, which Azure has no equivalent for.pipelex updaterefreshes the numbered deck files and backs up any you edited; a project that wants the previous models keeps them by naming them inx_custom_llm_deck.toml, which pipelex never touches. - The shipped deck declares
temperature = 1throughout (Breaking): every model in the GPT-5.6 range fixes its temperature at 1, so every preset states that value instead of one the worker would override while warning on every call, and[llm.choice_defaults].default_temperatureis 1 for the same reason — it is the temperature paired with an alias, a waterfall or a bare handle named outside a preset, thefor_textandfor_objectchoices every pipe without an explicit model takes among them. Presets are therefore distinguished by their model tier and their reasoning effort alone:$writing-factualand$writing-creativeissue the same call, and$engineering-structuredruns at temperature 1. A method that needs a temperature of its own writes an inline LLM settings table —model = { model = "gpt-4o", temperature = 0.8 }— since a bare handle is paired with the deck-wide default rather than with a value of the author's. - The shipped Gateway models reference lists the roster the Gateway actually serves:
backends/pipelex_gateway_models.mdand its plain twin are generated from the served remote config, and they had not been regenerated since July — so they lacked the GPT-5.6 range this deck now resolves every tier to, while still advertising the GPT-4.1, o-series and GPT-5.1 to 5.2 generations that v0.61.0 retired. Both files are refreshed, in the kit and in.pipelex/, so the reference a user opens to check what their default deck runs on agrees with the deck. - CLI help and documentation examples name an alias the deck still defines:
check-model's argument help, the model-reference and model-suggestion docstrings and the doc examples all used@best-claude, which this release removes, and now use@best-gpt. The "Configure AI Providers" guide also gains a section stating exactly what a fresh install resolves to — the GPT-5.6 ladder and GPT Image 2 for language and image generation, Azure Document Intelligence for extraction, and Linkup for search and for web-page extraction, the two places the deck leaves Azure. - The Elastic License 2.0 boundary is named by version rather than by date:
docs/license.mdsaid the license applies to every version released after 2026-09-14, which left a reader holding a version to work out which side of that date it fell on. It now names the first Elastic version of each artifact —pipelexfrom v0.58.0,pipelex-apifrom v0.24.0,@pipelex/mcpfrom v0.15.0 — and spells out the last MIT version of each.
Removed
best-claude,best-geminiandbest-mistralare gone from the shipped deck (Breaking): an alias naming a provider cannot honestly resolve to a GPT model, and the image deck'sbest-geminigoes with them. A method referencing one fails validation with the usual alias-not-found error; define it in anx_custom_*deck file to keep it.- GPT-6 Astra is no longer served on the Pipelex Gateway (Breaking):
gpt-6-astrais out of the remote config and out of the shipped Gateway roster, so a method naming it on the Gateway now fails the boot-time membership check. It stays declared on theazure_openaiandopenaibackends, where a project holding its own key reaches it as before. ModelDeck.final_validateis removed: it raised when a preset's temperature differed from its model's fixed temperature, but it had had no caller since 2025-09-17 and itsexceptclause could never match the exception its helper raised, so it protected nothing.
[v0.61.0] - 2026-09-20
Added
- Claude Opus 5, Claude Fable 5 and Claude Fable 5.1 join the Anthropic backend, and Opus 5 the two hosted profiles:
claude-5-opus,claude-5-fableandclaude-5.1-fableare served on Anthropic's own model ids, and all three are reachable on Pipelex Manifold through Bedrock'sglobal.anthropic.*endpoints, the way every other Claude already is. Only Opus 5 is served on the Pipelex Gateway: Bedrock refuses both Fables through Portkey withdata retention mode 'default' is not available for this model, a mode no model spec can set, so listing them there would publish handles that fail on every request. All of them take text, images and PDF, cap output at 128k tokens, and think adaptively with the temperature parameter refused. Fable 5.1 additionally declaresstructure_method = "instructor/anthropic_reasoning_tools", because it rejects the forcedtool_choicethat the backend's defaultinstructor/anthropic_toolssends and structured output fails without it. - The GPT-5.6 series and GPT-6 Astra are served on OpenAI, Azure OpenAI, the Pipelex Gateway and Pipelex Manifold:
gpt-5.6-sol,gpt-5.6-terra,gpt-5.6-lunaandgpt-6-astraare registered on the two direct backends and, through the remote config, on both hosted routing profiles. All four take text, images and PDF, produce text and structured output, and reason underthinking_mode = "manual"with a fixed temperature of 1. The Azure entries carry the dated deployment ids the hosted profiles also use as their wire model ids.
Changed
add-modelskill follows the model's nearest sibling: the skill derives which backend TOMLs, test collection and deck entries a new model belongs in from the model it succeeds, instead of a fixed provider-to-backend table, and checks each backend actually serves the model. It takes the model's facts from the provider's own documentation with OpenRouter as a cross-check, hands the live tests to/test-modelwith a test class per declared capability, and adds the deck and changelog steps andmake ugmonce the gateway catalog carries the model.
Removed
- The superseded GPT generations are retired from the rosters we ship (Breaking): the GPT-4.1 series, the
o1,o3ando4reasoning models and the GPT-5, GPT-5.1 and GPT-5.2 generations are gone fromazure_openai, fromopenai— which also losesgpt-3.5-turbo,gpt-4,gpt-4-turbo, the codex variants of those generations and the datedgpt-4oaliases — and from the gateway and manifold catalogs.o4-miniis retired fromportkeyin the same pass, since no other backend served it. None of them is unsupported: they are unlisted, and a project that still wants one keeps it by declaring it in its own backend config. What genuinely retires is their availability on the Pipelex Gateway and Pipelex Manifold, where the roster is ours.gpt-4oandgpt-4o-ministay on both direct backends; the retired names thatportkeyandblackboxaistill declare remain reachable through those, andopenrouter's bulk third-party catalog is untouched.
Fixed
- Claude Sonnet 5 is priced at its standard rate:
claude-5-sonnetcarried the $3 / $15 per million tokens it was announced at, but its $2 / $10 introductory rate became the standard price and the scheduled increase was cancelled. The entry is corrected on the Anthropic backend and in both hosted catalogs, so cost reporting and budget ceilings no longer overstate a Sonnet 5 run. - A failed publish can be recovered without damaging what already shipped:
publish-pypi.ymlscopes the signature upload todist/*.sigstore.jsonso it no longer clobbers the wheel and sdist attached just before it, turnsskip-existingon for a re-run or a manual dispatch, and refuses to touch a release whose tag names a commit other than the one being published. A retry no longer rebuilds the dists either: the build job asks PyPI what this version published and recovers those bytes from the first place that holds them — the build an earlier attempt of this run stored, an artifact an earlier run stored for the same commit, or the files PyPI itself serves — building fresh only when PyPI answers that nothing was published under this version. Whatever it holds is then checked to be one wheel and one source tarball of the version being released and, when PyPI serves that version, byte-for-byte what PyPI publishes, so the release assets are the bytes PyPI serves rather than a rebuild that probably matches them. The release is now tagged at the commit it ships, named explicitly with--target, rather than at the default branch's head. - A generated image is stored under its own media type, so a store no longer serves it as
binary/octet-stream:GeneratedContentFactory.make_image_contentresolves the effective mime type once and already used it to mint the storage key's extension, but passed nocontent_typetoStorageProviderAbstract.storeon either the direct-bytes path or the fetched-remote one. The S3 provider only setsContentTypewhen it is given one, so every generated image landed untyped and S3 answeredbinary/octet-streamon the way back out — which reached any consumer that trusts the stored type, a same-origin assets route serving the object to a browser among them. Both calls now pass the resolved mime type, so the key's extension, the object's stored type and the reportedmime_typecannot diverge. - A generated image fetched from a remote URL is stored under the media type that URL served:
GeneratedImageRawDetailsadmits a bare URL with no mime type and no image format on the promise that the type is settled when the bytes are downloaded, and nothing kept that promise — the fetch discarded the response'sContent-Typeand the resolution fell through to the requested image format or to a hardcodedimage/jpeg. The newfetch_file_and_content_type_from_url_httpxreports the declared type alongside the bytes, normalized and with its parameters stripped, and the factory stores under anyimage/<subtype>it names — not merely the formatsImageFormatcan itself produce, since what this runtime generates is a different question from what a remote URL may serve, and a GIF or an AVIF stored asimage/jpegis the very error being removed. The storage key's extension follows that subtype so it keeps agreeing with the stored type. A type the caller declared still wins, and a served type that is not an image at all — thetext/htmlof an error page dressed as a 200 — is dropped. - A remote image that fails to download degrades to its URL instead of failing the pipe:
make_image_contentguarded its fetch withexcept (httpx.HTTPStatusError, httpx.RequestError), butfetch_file_from_url_httpxconverts every httpx failure intoRemoteFileFetchError, so that clause caught nothing and a 404 or a timeout on a remote image propagated out and failed the whole run. The guard now names the exception the fetch actually raises, and the fallback it was written for — log the warning, keep the remote URL — runs. - A remote fetch honours a timeout again:
fetch_file_from_url_httpxpassed itsrequest_timeoutstraight to httpx, and httpx reads an explicittimeout=None— the parameter's own default — as no timeout at all rather than as its client default. Every caller that did not name one therefore had no timeout, so a hanging server blocked the caller indefinitely and theTimeoutExceptionbranch that raises a caller-facingRemoteFileFetchErrorwas unreachable. The helper now passes httpx'sUSE_CLIENT_DEFAULTsentinel where no timeout was asked for. -
A manifold error on the native routes keeps its request id under the gateway's pipelex-spelled trace header:
extract_manifold_metadatareads the gateway's trace id fromx-pipelex-trace-idbefore the inheritedx-portkey-trace-id, still preferring a provider's ownx-request-idover both. The gateway emits the two spellings with the same value today, so nothing changes yet; once it drops the vendor one, such an error whose provider sent nox-request-idkeeps a request id instead of reporting none.extract_gateway_metadatais unchanged and still reads the vendor spelling alone — it serves the Portkey cloud and the manifold image path, which travels onportkey_aiand keeps that spelling until it is ported off the SDK. -
The License page names the versions the Elastic License 2.0 starts at: the page said ELv2 applies "for every version released after 2026-09-14", which left the first ELv2 releases outside the boundary it drew, since all three were released on that very day. It now names them instead of dating them —
pipelexfrom v0.58.0 onward,pipelex-apifrom v0.24.0 onward and@pipelex/mcpfrom v0.15.0 onward — and spells out the last MIT version of each.
[v0.60.0] - 2026-09-19
Changed
- A dump of a working memory names each stuff's concept by ref (Breaking): every path that serializes a
Stuff—model_dump,smart_dump,dump_for_transport, theworking_memory.jsona run saves, the agent CLI's--with-memoryenvelope and the runner'sPipeOutputresponse — now emits"concept": "<domain>.<Code>"besidestuff_code,stuff_nameandcontentinstead of the fullConceptobject. A dump is one-way: aStuff, aWorkingMemoryor aPipeOutputcannot be validated back from its own dump, because the definition no longer travels with the data. Every consumer that readconcept.code,concept.domain_codeorconcept.structure_class_nameoff a dumped stuff must read the ref string and resolve it instead. - A transported stuff's concept is resolved through the loaded library (Breaking): one shared rule matches the ref against every entry the library holds for that spelling — the host or native concept keyed by the ref itself, plus each concept a dependency package contributed, which the library keys under that package's address. One match resolves, none refuses naming the ref, and a spelling that a host bundle and a dependency both declare refuses naming every key that matched, rather than binding one package's definition to the other's data. Outside any library scope the readers answer from the pinned native set alone, so a delivery worker that never loaded the crate now renders a bundle-declared concept as a raw dict where it previously got a typed render from the definition on the wire.
ConceptProviderAbstractandConceptLibraryAbstractgained abstract methods (Breaking): a provider must now implementlist_concept_keys_for_ref, and a concept library must implementadd_dependency_concept. An out-of-tree implementation of either fails to instantiate until it does.- The agent CLI's stdin resolver passes the concept ref through with its domain intact (Breaking): it used to reduce the concept object to a bare code, which let a bare code match any domain's concept of that spelling. A
--with-memoryenvelope therefore pipes into a receiving method only where that method declares the same<domain>.<Code>; the looser behaviour silently bound one domain's definition to another domain's data. A concept the resolver cannot accept is reported under the newerror_typeStdinEnvelopeShapeError— the envelope parsed, so labelling itJSONDecodeErrorsent readers hunting for syntax errors in valid JSON.
Note the limit: the MTHDS standard also defines <package_address>::<domain>.<Code> for a concept a dependency contributes, and this runtime neither emits nor resolves that form.
Fixed
PINNED_NATIVES_MTHDS_VERSIONnames the set it labels: the constant read1.0.0while the standard pins the native set at2.0.0, so anything taking it as the answer to "which pinned set is this engine's" — a downstream port gating its own native goldens, for one — read a version the standard now describes as predating the pinning regime. It now reads2.0.0, and it is no longer a value nothing reads: the standard-conformance suite holds it to the page's own "Pinned at MTHDS" statement, and a check that needs no sibling checkout refuses a label later than the standard version this engine implements.
[v0.59.0] - 2026-09-16
Added
pretty_print_modein[runtime.log](Breaking):rich(the default),poororsilent, applied at boot besidelog_mode, so a host with no console turns the "Output of pipe" panels off in configuration rather than in code; a silent printer builds no Rich renderable at all. Boot now replaces aPrettyPrinter.modeassigned in code before it, so set the key instead, and teardown restores whatever mode the process held before that boot.pipelex doctorapplies the key too.
Changed
- Remote inputs are checked the way the web checks them (Breaking): an http(s) URL on a
DocumentorImageinput is only checked for syntax when the inputs are shaped, and refused withPipelineInputUrlInvalidErrorwhen it does not parse. Whether the resource exists is decided by the operator that fetches it, which raises a caller-facingRemoteFileFetchErrornaming the URL and the answer it got, instead of a rawhttpxerror. The pre-flight HEAD probe is gone: on the hosted runner a host that stalled it could end a runTIMED_OUTwith no error. A local file path must still exist before the run starts. - The User-Agent is
Pipelex/<version>: the(https://pipelex.com)suffix is gone, because the crawler-style URL in parentheses is what bot walls key on, and sites that stalled the old string serve the new one promptly.
Fixed
- Poor pretty-print mode: a panel whose title is wider than the terminal no longer hangs the run in an endless wrapping loop, which every "Output of pipe" panel did on an 80-column headless console; an over-wide title is elided instead. Titles print without their Rich markup, a title in square brackets no longer raises, and an output prints as its plain rendering rather than a Rich object's repr.
- A trace event log the current models refuse is no longer read as an empty one: an event written by a version whose event shape this one no longer accepts now makes the read refuse with a count and a reason, instead of being skipped and the remainder reported as a success; such a run reports neither its graph nor its usage and cost total, each with its assembly error set. A line that is not JSON, left by a crash mid-write, is still skipped with a warning.
pipelex graph rendergives the same diagnosis for a graph spec saved by an earlier version instead of a raw traceback.
Removed
- The text and HTML renderings of traced data (Breaking):
stuff_text_contentandstuff_html_contentunder[interpreter.pipeline_execution.graph.data_inclusion]are gone, along with a graphIOSpec'sdata_textanddata_htmland the unusedPrettyPrinter.pretty_htmlandPrettyRenderable.rendered_pretty_html. Runs no longer pay for rendering every traced input and output on the execution path, and the Mermaid graph page now shows a data node's JSON, with a preview for image and PDF content. A graph spec saved by an earlier version is refused when read back; run the pipeline again to get one in the current shape. Migration: runpipelex migrate— ledger entrypipelex-config@4deletes both keys from~/.pipelex/and from every project.pipelex/file, keeping one timestamped backup per file.
Security
escape_script_tagescapes every<: the filter only matched the literal</script>, so</script >and</script/>closed an embedded JSON block and let the rest of the value parse as live HTML. Every<is now escaped as\u003c, which guards the graph and stuff viewer pages that embed model-generated traced content.
[v0.58.0] - 2026-09-14
Changed
- License (Breaking):
pipelexis now licensed under the Elastic License 2.0 (ELv2) instead of MIT, and the package metadata declares the SPDX expressionElastic-2.0in place of the MIT license classifier, which makes hatchling 1.27 or later a requirement for building from source; every earlier version, up to and including v0.57.0, stays under MIT. ELv2 is source-available: you may embedpipelexin your own products, including services you offer to others whose features run your methods, and in your internal tools, while its main limitation rules out hosting a service that runs methods for others, whether they send the methods or pick them from a catalog.LICENSEcarries the full terms, including its conditions on notices and redistribution, and the License page on docs.pipelex.com explains how Pipelex reads them.
Security
- Docs site built on mkdocs-material 9.7.7: the
docsextra now requiresmkdocs-material>=9.7.7, which closes a DOM XSS in the theme's search suggestions (GHSA-xvg9-69gf-fjrf) that the published site shipped. The pull-request documentation check now builds fromuv.locklike the deploy does, instead of from its own hand-pinned theme versions.
[v0.57.0] - 2026-09-07
Added
- A run's results carry the I/O artifacts that describe its data: beside
graphspec.json, and gated by the samegraphspec_jsoninclusion flag, a run now writespipe_io_contracts.json,input_form.jsonandoutput_form.json— the validation report's own artifacts under the standard's names, keyed by namespacedpipe_reffor every pipe of the library the run executed against. Until now only/validateproduced them, so a consumer of a run's graph either took the no-data floor or paired the graph with a validation of some other bundle text, and a graph viewer shows a data node's value only when it holds the contracts and the output form. They ride onPipeOutput.pipe_io_artifactsbesidegraph_spec, with a build failure of any kind landing onpipe_io_artifacts_errorrather than failing the run or skipping its delivery, and the SPI payload carries them aspipe_io_artifacts_dump.pipelex run,pipelex-agent runand the direct-mode delivery executor all write them, while a run hosted on our Temporal plugin still writesgraphspec.jsonwithout its siblings. See Execution Graph Tracing. - The
--versionhandshake reports three numbers (Breaking):pipelex --versionandpipelex-agent --versionnow print the runtime version, the MTHDS Protocol version and the MTHDS standard version, one<label> <version>line each, instead of the runtime version alone. The three move on independent cadences under the standard's own versioning rule, so none can be inferred from another, and the two MTHDS numbers are read from themthdspackage rather than restated here. The first line keeps its historical<program> <version>form, so a consumer that matches a version out of the text is unaffected — but one that treats the whole of stdout as a single version string now holds three lines, so match on the label rather than on the shape of the output. See CLI Reference.
Changed
- The MTHDS standard version this runtime implements is now
2.0.0(Breaking):mthdsmoves to==0.14.0, whoseMTHDS_STANDARD_VERSIONwas cut from1.0.0to2.0.0— the standard's first cut under its own versioning rule, accounting for breaking changes to the native set and to the manifest, lock and resolution rules that had already shipped unversioned. Two things follow for users, and no pipelex line caused either: every normalized crate this engine emits is stamped2.0.0instead of1.0.0(crate fingerprints are unaffected, becausemthds_versionis excluded from the hashed payload by design), and aMETHODS.tomlwhosemthds_versionpinned the old major, such as^1.0.0, now warns where it did not before, while a plain floor like>=1.0.0is still satisfied. The protocol version is unchanged at0.6.0, and the pin stays exact, so a consumer depending onmthdsdirectly must move in step. - The Commands section of the shipped agent rules is a compact list:
pipelex/kit/agent_rules/commands.mddrops its long-form headings and code blocks for one bullet per gate — the target, its alias, when to run it and where the full rationale lives — so theCLAUDE.mdthatmake rulesgenerates from it costs a session a third of what it did. It also now records the one thing a session could not learn from the file itself: thatCLAUDE.mdandAGENTS.mdare generated, thatmake rulesis how they move, and thatmake check-rulesis a CI gatemake agent-checkdoes not run.
Fixed
- The codegen stamp reference no longer understates what moves a fingerprint: Codegen Projections said a stamp moves only when a method's effective type surface changes, which would have a reader mistake an ordinary restamp for drift and expect a prompt edit to leave the generated tree alone. The crate fingerprint is computed over the whole normalized crate — concepts, pipes and domains — so an edited
system_promptor a swapped model restamps every generated file exactly as a renamed concept field does, with the body below the fence byte-identical.content_hash, in the stamp header and in the lock'sartifacts[]entries, is what distinguishes that harmless restamp from a real change to the generated code. - Compound version constraints written the documented way now parse:
parse_constraintstripped no whitespace around the comma that ANDs a compound constraint's clauses, so>=1.0.0, <2.0.0— the exact spelling of the MTHDS standard's own example — was rejected while the identical>=1.0.0,<2.0.0parsed. A manifest'smthds_versionwritten that way therefore got no compatibility check at all, since the load path degrades an unparseable constraint to a log line. A malformed compound such as>=1.0.0,,<2.0.0still fails, because normalizing whitespace must not repair a broken constraint.
[v0.56.0] - 2026-09-03
Added
- Pipelex Manifold (private beta):
pipelex_manifoldjoinspipelex_gatewayinbackends.tomlas a second Pipelex-operated inference backend, with a matchingall_pipelex_manifoldrouting profile. It ships disabled; since both services share model handles, switching the profile is the whole change. A new provider package supplies the manifold sdk set (manifold_completions,manifold_responses,manifold_img_gen,manifold_extract,manifold_search), the backend declares the service origin with no/v1, and the service token travels on thex-pipelex-api-keyheader on every protocol. Per-model local overrides work throughbackends/pipelex_manifold.tomlexactly as they do for the gateway. - Personal configuration overrides:
backends_override.tomlandrouting_profiles_override.tomlsit beside their base files under.pipelex/inference/, are git-ignored, and deep-merge over the base — the global~/.pipelex/inference/one first, the project's one last — so a developer switches backends locally without touching a tracked file. Every reader sees the merged document: the boot,pipelex show backends, and the doctor's rows. The.pipelex/.gitignorethatpipelex initwrites now lists every personal override file. See Personal overrides. - Fetch-on-miss for methods: a bundle referencing a method by address (
github.com/...->domain.pipe) that misses the installed methods now fetches the package, honoring an@<tag>pin, installs it into~/.mthds/methods/with provenance recorded beside the manifest, and loads it. A miss that cannot be bridged raises an error naming the address and the remedy instead of passing silently. Disable it with[interpreter.methods] fetch_on_miss = falseorPIPELEX_METHODS_FETCH_ON_MISS. See Include by Address. - TypeScript emission gates:
make test-ts-gates(aliasttg) provisions a pinned prettier and zod into.ts-toolchain/and runs the codegen suite withPIPELEX_REQUIRE_TS_GATES=1, under which the two tests that read the emitted TypeScript as TypeScript fail instead of skipping. The same target runs as a required CI job. See TypeScript Emission Gates. - Output-form descriptor: the validate report carries an
output_formbesidepipe_io_contractsandinput_form, and the output contract carries ajson_schema, closing the asymmetry between a pipe's two halves.build_output_formprojects the same concept derivation the inputs use, wraps a plural output as alistnode, and publishes the payload's schema (ListContent[...]on the plural arm, withminItems/maxItemson a fixed count). See Pipe IO Contracts. - Projection fixture corpus:
pipelex-dev generate-projection-corpuswrites the shared fixture corpus that pins the TypeScript and Python inputs-template projections byte-for-byte against each other, records every declared divergence from the engine's own renderer, and round-trips every template it pins through the input shaper so a template the runtime would refuse cannot be captured unnoticed (EXPECTED_UNSHAPEABLEdeclares the known gaps). The corpus covers the open-shaped natives, an outputs bundle, and nested and plural file slots. See Projection Fixture Corpus. bump-kajsonskill: a Claude skill and helper scripts that move thekajsonpin, re-lock, adapt the engine and write the changelog entry..worktreeinclude: the gitignored files a fresh worktree of this repo needs (.env, the personal.pipelex/overrides,.pipelex-dev/test_profiles_override.toml) are declared in a tracked file, read by Claude Code and by the workspace'swttool.
Changed
- Inference error domains (Breaking):
InferenceErrorCategorygains anerror_domainproperty and the wholeCogtErrorfamily derives its domain from it, where it declared none. A content-classified failure — a provider policy refusal, a malformed prompt image, a bad prompt parameter, every provider HTTP 400 — now answers HTTP 422 instead of 500 on any surface readingErrorReport.http_status.ModelChoiceNotFoundErrordeclaresinputon the class itself. See Error Model. - Gateway refusals classified: the Pipelex inference gateway's own refusals no longer fall through to the generic provider-error ladder. Three families carry their own classification, keyed on the gateway's error code rather than the reporting provider, and none is retried: request limits (a body or storage object over its cap, nesting past the depth limit), unresolved references (a storage key or URL the gateway cannot turn into bytes, a host the SSRF guard refuses), and routing refusals (a model the deployment does not serve, an integration switched off). Routing refusals are
CONFIGURATIONthroughout, so they answer 500 rather than 422, and a model the gateway does not serve raises the same availability error a deck miss does.ClassificationResultgainsgateway_request_limit,gateway_unresolved_referenceandgateway_routing_refusal. See Error Model. - Managed gateways by declaration (Breaking): a backend is Pipelex-managed when it declares a
model_specs_section, not because it is namedpipelex_gateway; more than one can be live, and the published configuration is fetched once and sliced per backend. A managed backend whose${…}variables do not resolve is disabled with a warning naming it, while bring-your-own-key backends keep failing the boot loudly. Service-terms acceptance is asked for any enabled managed backend, and declining disables all of them.is_gateway_backendis gone, the backend loader takesmanaged_gateway_configs, andGatewayUnknownModelErrornames its backend. - Inference loaders take a path sequence (Breaking):
InferenceBackendLibrary.load,load_active_routing_profileandModelManager.setuptake a sequence whose first path is the base file and whose rest are the overrides in merge order, built byConfigLoader.backends_file_paths()/routing_profiles_file_paths();is_pipelex_gateway_enabledtakes aconfig_dir. The routing loader now raisesRoutingProfileLibraryNotFoundErrorandRoutingProfileDisabledBackendError, the classes the boot already caught, and a file that does not parse or a backend that is not a table fails asInferenceBackendLibraryValidationErrornaming the file. - Remote config pin: the default moves to
pipelex_remote_config_13.json, the first artifact carryingmanifold_model_specs. Itsbackend_model_specsare identical to version 12's, so the Portkey-cloud path reads what it read before. - Dependencies:
mthdsmoves to==0.13.0, which carries the output-form descriptor, andkajsonto==0.7.1, whoseregister_classes_dict({})no longer raises on an empty snapshot.
Removed
InferenceBackendLibrary.check_backend_credentialsandCredentialsValidationReport: the method had no caller and read the basebackends.tomlalone, blind to the new override files. The doctor's credentials row does the same job over the merged document.DocumentFormat: the enum andpipelex/tools/misc/document_utils.pyare gone; nothing in the engine imported them, andfiletype_utils.pyalready carries the MIME and extension knowledge (Breaking for anyone importing the symbol).
Fixed
- Class registry isolation: structure classes generated for one library could leak into another load in the same process — a later bundle silently reused an earlier bundle's structured concept of the same name, and two interleaved loads overwrote each other's registration. Every
Libraryopened throughLibraryManager.open_librarynow gets its ownClassRegistry, seeded from the global one, so a load's classes live and die with it. A structure class every load must see has to be registered before any library is opened. See Per-call scoping. - Pre-boot registry starvation: a class-registry read before boot instantiated the kajson singleton around an empty registry, and the boot then handed that manager back and discarded its own, so the core models landed where nothing resolved. Any pre-boot manager is now dropped before the boot constructs its own.
- CLI error reporting: an unknown or ambiguous entry
pipe_codeand a typo'd--pipe/pipe_refslice selector now raiseEntryPipeNotFoundError/EntryPipeAmbiguousError, which carry theINPUTdomain and answer 422 instead of 500; both subclass the classes they replace, so every existing catcher still matches.--helpworks again on everymethodsubcommand, where the address example's brackets reached rich's markup parser unescaped. - TypeScript codegen stability: a dict-valued default, a key or string carrying a double quote, and an exploded choice list containing a comma or a bracket each produced a
types.tsthat prettier rewrote or that did not parse, sopipelex codegen checkreported an untouched file as hand-edited. Defaults render as TypeScript literals, every target emits the quote style its formatter keeps, and every line-break split walks string literals. An always-on invariant asserts no emitted line ends inside an unterminated literal. - Zod wire agreement (Breaking for generated TypeScript — regenerate): a non-required field was projected as
.optional(), which rejects the explicitnullthe runtime serializes for an unset field. It is now.nullish(), or.nullable().default(…)where it carries a default, and parsed optional fields type asT | null | undefined. The emitter's line-breaking now models prettier's member-chain rule. See Codegen Projections. - Native concept handling: a
native.Anythinginput no longer crashes the pipe-IO contract builder — it publishes a permissive schema and renders{}in a template; anative.JSONinput renders{"json_obj": {}}from its pinned structure instead of an empty object the runtime refused; and anative.Dynamicinput referenced in aPipeLLMorPipeImgGenprompt no longer fails dry-run validation demandingImageContent, since a dynamic concept gets no static image or document classification and renders as text. See PipeLLM. - Input-form descriptor for class-backed concepts: a concept declaring
structure = "SomeClass"collapsed tounknownthe moment one field annotation could not be mapped, hiding everydocumentandimageposition beneath it. Reflection is now field by field: an unmappable annotation isunknownat that field alone, a nested model is walked into, and aRootModelreports its root annotation. See Input-Form Descriptor. - Projection corpus divergence gate: the gate that declares every engine/projection difference missed fields the projection stopped rendering, absorbed a regressed file placeholder into an existing class, and could not tell an honoured
Concept[N]from a wrong count. All three now refuse, and a pipe declaring no inputs is captured from the projection alone instead of aborting the run. - Doctor:
pipelex doctorreported a false backend-configuration error forpipelex_gatewayon a stock install, because the Backend Files row loaded the library strictly with no gateway config, and that first failure hid real errors in later files. The row now loads leniently, the loader stampsbackend_nameon every error it raises about one backend, and both rows charge a failure to the backend it declares rather than to any backend whose name appears in the message. - Strict disclosure leak:
PipelineInputContentErrordeclared its caller-facing flag under the wrong name, so the message a caller needs on a missing url was redacted under STRICT disclosure while the unreadable-local-path message would have leaked. The family is split:PipelineInputUrlMissingErroris caller-facing, andPipelineInputContentErrorstays redacted. HtmlContentJSON rendering:rendered_jsonemittedhtmlwhere the standard's pinned concept requiresinner_html. The override is deleted and the rendering is the model's own dump.- Generated-image storage keys: the key's extension followed the requested image format while
mime_typefollowed the provider's actual answer, so the two could disagree. Both now derive from one resolved mime type. - Agent-rules templates: the
generate-projection-corpusentry was written into the generated repo-rootCLAUDE.mdinstead of its template, leavingmake check-rulesred ondev. It now lives inpipelex/kit/agent_rules/commands.mdand its Codex twin, and the check runs in CI aslint-agent-rules.
[v0.55.0] - 2026-08-29
Added
- Run a method by address: every command that resolves a method target (
pipelex run method,pipelex validate method, thepipelex buildmethod commands, and their agent-CLI twins) now accepts a method reference<address>[@<tag>]—github.com/<owner>/<repo>[/<selector>], optionally pinned at a git tag (a@<name>that names a branch rather than a tag is refused, so only tags pin a revision) — alongside installed names, local paths, and full GitHub URLs. The newpipelex/methods/package is the single implementation the CLI, the API, and tests all share: theMethodRefgrammar parser, fetch-at-tag (wiring themthdsclone_at_versionmachinery besideclone_default_branch, with the fetched commit SHA always resolved and reported as provenance), package location inside the clone by manifest identity (scan forMETHODS.toml, match the full address, case-insensitively; a miss or ambiguity is a loud error listing the packages the clone contains), env-tunable fetched-package ceilings (file count and total bytes, plus a bounded manifest scan — a size cap perMETHODS.tomlread and a cap on manifests considered per repository), and the reusable structures-refusal check — an alias-aware AST scan forStructuredContentsubclasses (bases reached throughimport ... asaliases and attribute access are caught alongside the literal name) producing the rule-naming error (hosted execution accepts MTHDS concepts and sandboxed PipeFuncs, not in-process Python), which the CLI surfaces as a hosted-parity warning and a hosted runner can apply as a hard refusal. A fetched target's default outputs (run results, graphs, generated files) anchor in the caller's working directory — the fetched clone itself is ephemeral and deleted at process exit —build runneradditionally copies the fetched package beside the generated script so the runner stays self-contained, and a method-reference failure reaches the agent CLI as its structured error envelope, like every other agent-CLI error. See Run a Method by Address. (Breaking: baregithub.com/...targets no longer resolve as local paths, and packages inside a fetched repository are located by manifest identity rather than directory path —/tree/<branch>/<dir>deep links now resolve through the manifest'saddress/name.)
Changed
- Structure-class-not-found diagnostics teach the missing-module case: when a concept's declared structure class does not resolve (
ConceptStructureClassNotFoundError, and the corresponding PipeFunc output validation error), the message now states that the class may live in a Python module that was not part of the request, and suggests including the module or expressing the type as MTHDS concepts.
Fixed
- Blueprint serialization schemas:
ConceptBlueprint,ConceptStructureBlueprint,InputSlotBlueprintandConstructBlueprintpublish their real shape again in serialization mode. Each carries a@model_serializer(mode="wrap")annotated-> dict[str, Any], and pydantic turns a serializer's return annotation into the model's serialization JSON Schema in preference to the one it would generate — so all four came back as an opaque{"type": "object", "additionalProperties": true}with no properties, norequiredand no closed shape. This is the same annotationmthds0.11.1 dropped for the input-form field descriptors. Nothing here noticed, because validation-mode schemas andmodel_dump()were both correct throughout; the damage landed in consumers that generate response-model schemas, which FastAPI always does, so a downstream OpenAPI artifact regains its field-level detail on its nextpipelexbump. A new sweep test holds every Pipelex model to the rule, and the agent rules now scope "every function return must be typed" so a serializer's return stays unannotated.
[v0.54.0] - 2026-08-28
Added
- MTHDS standard conformance CI: A new test holds the pinned native-concept set (
pinned_blueprints.py) to the standard's ownnative-concepts.md, read from a siblingmthdscheckout — definition for definition, field for field, order included. Themthds-standard-check.ymlworkflow checks the standard out at its default branch and runs it on every pull request, so the day the standard moves a definition this repo goes red instead of a downstream port. bump-mthdsClaude skill: A skill (with itsupstream_notes.pyhelper) that automates moving themthdsdependency: it finds the latest release, re-locks, adapts the engine source to what the release broke, runs the checks and drafts the changelog entry.
Changed
mthdsis pinned exactly to==0.11.1(previously>=0.8.2). Everyone downstream of Pipelex inherits that exact version, so a consumer that also depends onmthdsmust now move in step with Pipelex rather than resolving its own. (Breaking)- Multiplicity
[1]is strictly singular: a count of one (e.g.Concept[1]) is universally the single form, not a one-item list. A[1]input takes the value itself and refuses a list, a[1]output produces one object rather than aListContentof one, a re-rendered ref canonicalizes back toConcept, and a presence marker beside a count of one (Concept[1]?) is accepted on inputs as it already was on outputs. A sequence declaring a plural output whose last step yields one no longer validates, andConcept[0]is rejected at parse time with a validation error instead of surfacing later. See Understanding multiplicity. (Breaking) native.Html.css_classis now optional (str | None), matching MTHDS v0.9.0. AnHtmlvalue is its markup, and the class is presentation a producer states when it has one — composed HTML and JSON rendering no longer fabricate an emptycss_class, and structured output no longer forces a generating LLM to invent it. (Breaking)- Input-form descriptors and pipe I/O contracts are typed by the standard now: both artifacts' wire models are imported from
mthds.protocolinstead of being restated here, and re-exported frompipelex/pipeline/input_form.pyandpipelex/pipeline/pipe_io_contracts.py. The wire is unchanged; what moved is which package declares the shape. Importers are affected three ways: a field descriptor is one model per kind (TextField,ObjectField, …) under anInputFormFieldunion discriminated onkind, narrowed withisinstanceormatchrather than read off a flat model, soFieldKind.is_listandFieldKind.is_objectare gone; thedatekind's Python attribute is nowdatetime, the name it always had on the wire, rather thandatetime_flag; and the item layer is a distinctInputFormItemunion (TextItem,ObjectItem, …) that structurally forbids thenamea list item never had, renamingInputFormFieldBasetoInputFormItemBaseandTextValuedFieldBasetoTextValuedItemBase. See Input-form descriptor and Pipe I/O contracts. (Breaking) native.Dateandnative.Htmlareobjectnodes in the input-form descriptor, withfieldsderived from their pinned definitions, rather than the formerdateandprosescalars — which reported shapes the slots' own schemas rejected. (Breaking)PresenceMarkeris imported from the standard, so the three helpers that are this engine's.mthdssuffix grammar rather than wire vocabulary became module-level functions inpipelex/core/pipes/variable_multiplicity.py:presence_from_symbol,presence_symbolandis_force_presence, all keyword-only.is_optionalandis_plainstay properties on the enum. (Breaking)- Input-form descriptor conformance: a description-only concept's
refinesnow carries only the authored chain and no longer fabricates thenative.Textlink the spec forbids reconstructing — text-valuedness still reaches the wire askind: "prose". TypeAdapteragent rule refined (pipelex/kit/agent_rules/python_standards.md): the flat ban becomes a scoped one. Don't design shapes that need aTypeAdapter, but validating into a top-level union or container owned by an external contract — a standard's wire model, a log stream of one-of-many event types — is its sanctioned use, as a module-level singleton.- Bumped
pyrightto1.1.411. The release narrowshasattr()on anAnyortype[Any]operand totype[object]instead of leaving it wide, which turned two duck-typed guards into type errors; both now test what they mean —issubclass(..., BaseModel)before callingmodel_validate, andtyping.get_origininstead of reaching for__origin__.
Fixed
- Input-form field serialization:
mthds0.11.1 drops the-> dict[str, Any]return annotation from the wrap serializer every field descriptor shares. Pydantic preferred that annotation over the schema it would generate, soTextField.model_json_schema(mode="serialization")and each of its siblings came back as an opaque{"type": "object", "additionalProperties": true}— the mode a server generating response-model schemas asks for, so a FastAPI consumer embeddingPipeInputFormDescriptorpublished akind-discriminated union whose arms described nothing. The per-kind shapes are back. - Sequence output enrichment: a sequence step with
nb_output = 1on an optional boundary no longer drops the optional marker (?) during fix planning. - Parallel pipe output validation: a plural branch with
nb_output = 1no longer bypasses type checking, which previously let an incompatible singular result through to crash at run time. - Image generation multiplicity:
PipeImgGenhonors anoutput_multiplicity=1run-time override, generating a single image instead of refusing it as unresolved.
[v0.53.0] - 2026-08-25
Added
- Vacuous-presence lint (
input_presence_vacuous): New advisory warning that fires when a required method input (declared without?) points to a concept with no required fields, since an empty object{}already satisfies it. Scoped strictly to entry pipes (the bundle's declaredmain_pipe), the message suggests either marking the input optional (?) or adding a required field to the concept. See Understanding optionality and Inline structures. - Unified advisory-warnings module (
pipelex/pipeline/advisory_warnings.py): A single composition point that assembles all advisory warnings (optionality, vacuous-presence, and intent-hints) in a fixed, deterministic order. - Corpus vocabulary exclusion: Added the
input_presence_vacuousexclusion to the corpus vocabulary generator (generate_corpus_vocabulary_cmd.py). - Documentation: Updated the user guides (Inline structures, Understanding optionality, validate) to cover the new lint, its remedies, and the unified warning channels.
Changed
- Consistent warnings across all channels: Every whole-bundle validate channel (protocol report, agent CLI, builder operations, and the bare CLI's
validate bundleandvalidate --all) now carries the exact same set of advisory warnings. Previously, channels showed different subsets — intent-hints were missing from the CLI, for instance. To keep output manageable, intent-hint findings are capped per site (five before collapsing into an "...and N more" message) and long authored tokens are elided at sixty characters. - Library manager interface (Breaking):
LibraryManagerAbstractgained a new abstract methodget_accumulated_blueprints(), which the collector reads to find the entry pipes. Any custom library manager injected viaPipelex.setup(library_manager=...)must implement it or fail loudly at construction.
Fixed
- Bare CLI warning visibility: The bare CLI's
validate bundlecommand now prints its advisory warnings before the strict pending-signature gate exits. Previously, a bundle with an unimplemented placeholder would exit non-zero and swallow the warnings — which is exactly the state an author is in while building a method. - Empty structure table parsing: An authored but empty
[concept.X.structure]table is now correctly parsed as anobjectwith an emptyfieldslist, instead of falling through a truthiness check and being described asproserefiningnative.Text. - Field-less structure class reflection: A registered Python class declaring no fields is now reflected as an empty object rather than
unknown. Genuinely unmappable annotations still correctly report asunknown. - Domain splitting in locators: Hierarchical domains such as
legal.contractsare now split at the last dot rather than the first when building advisory-warning locators. - Release workflow no longer loses the tag when signing fails: The GitHub Release is created, and the dists attached, before the Sigstore step, which is now allowed to fail without failing the job. The tag exists only as a side effect of creating the release, so a refreshed Sigstore trust root previously cost the tag and the release page while PyPI published normally — as happened to v0.52.0. The action is also bumped to a version carrying a current trust root, and release creation is idempotent so the manual retry path works.
[v0.52.0] - 2026-08-24
Highlights
A pipe's inputs now describe themselves well enough to render a form. The validation report carries a per-pipe input_form descriptor built from the loaded pipes and the authored blueprints rather than from the emitted JSON Schema, so a renderer keeps the concept chain, the presence marker, gating, fixed counts, defaults and choices. pipe_io_contracts drops its boolean flags for a three-valued presence and a real multiplicity, and emitted schemas gained the concept ref as title, the authored description as description, and minItems/maxItems on fixed counts. Alongside them a non-normative hints table lets an author state presentation intent without touching semantics, and authoring got louder: an unknown structure-field key, or a required = true paired with a default_value, is now rejected at parse time instead of silently discarded. Breaking: the contract reshape, those two new rejections, and build_validation_report's new required input_form argument.
Added
- MTHDS intent hints: A non-normative
hintstable can now be authored on a concept, on a structure field, and on a pipe input slot, with one defined key —intent, over the closed vocabularyprose/label/rating/quantity. Hints travel through the library crate (a concept's effective hints merge along its refinement chain, the nearer declaration winning) and surface on the input-form descriptor, while execution, validation verdicts and pipe contracts never read them. Validation emits advisory warnings (hint_unknown_key,hint_unknown_intent,hint_inapplicable_intent) rather than failing, and unknown-but-well-formed content is preserved. See Intent hints. - Input-form descriptors on the validation report (Breaking):
PipelexValidationReport.input_formmaps everypipe_refto an ordered list of field descriptors, derived bybuild_input_formfrom the loaded pipes and blueprints — never from the emitted JSON Schema — so a UI renderer gets the namespacedconcept_refand itsrefineschain, the three-valued presence, a statedgatingfact, fixed counts, authored defaults besiderequired, and one-memberchoiceslists, withunknownas the honest escape hatch. Class-backed concepts are reflected through the newreflect_structure_class. Breaking:build_validation_reportnow requiresinput_form, so every backend assembles it or fails loudly. See Input-form descriptor. - Richer emitted JSON Schemas: A schema now carries the concept ref as its top-level
title— replacing the mangled generated-class name — and the concept's authored description as its top-leveldescription, uniformly for generated, class-backed and native concepts. A fixed-count slot (Concept[N]) emitsminItems/maxItemson its array wrap while a variable-length one (Concept[]) stays unbounded, and the native content classes (TextContent,ImageContent,DocumentContent,PageContent,NumberContent,HtmlContent,YesNoContent) gained docstrings matching their pinned native descriptions, so provider-side schemas built straight from the classes carry them too. pipelex-dev trace-input-semantics: A capture harness for the chain that turns authored structure syntax into thejson_schemaonpipe_io_contracts. Given one or more.mthdsbundles it dumps per-hop captures — parse blueprint, generated class source, raw pydantic schema, SCHEMA render, wire contract — plus a manifest of each input's wire framing, so a lost or mangled authored fact is localized to exactly one hop. A committed probe bundle exercises every construct the language accepts. See Trace input semantics.
Changed
pipe_io_contractslearns presence and real multiplicity (Breaking): An input contract now carriespresence(plain/optional/force) — the declared marker verbatim — replacing the two-valuedoptional, so lint and graph surfaces can see where!assertions live without a presentation view. Input and output contracts both carrymultiplicity(single/variable/fixed), withitem_countpresent exactly whenfixed, retiring the old ruling that reported a fixed count asvariable; the output contract keeps its two-valuedoptional, since!is rejected on outputs. See Pipe I/O contracts and Understanding multiplicity.- Unknown keys in a structure-field table are rejected (Breaking): A typo'd or hopeful key on a concept structure field (
minimum,unit, a misspelledhint) used to be dropped at parse while the bundle validated green, silently discarding authored intent.ConceptStructureBlueprintnow forbids unknown keys, making the field table strict exactly like an input slot table; hint content stays lenient, since an unknown hint key warns and is preserved. The MTHDS JSON Schema regenerates accordingly, structure-field objects carryingadditionalProperties: false. See Inline structures. required = trueanddefault_valuecan no longer be declared together (Breaking): The two are contradictory instructions — a default is applied when the caller omits the field, which makes absence legal — and the generator used to resolve the conflict silently by dropping the required-ness. The pair is now rejected at blueprint validation and at the builder's spec layer, so an authoring agent fails before emitting TOML the parser would reject, with a message naming both remedies. Relatedly, a pydantic default on a hand-written structure class now counts as an authored fact: the input-form descriptor reports such a field asrequired: falsecarrying its default, instead of claiming it required and hiding the default.- Ruff moves to 0.16.4, matching the version the VS Code extension bundles, so a rule can never fire in the editor and not on the command line — an older binary parses
pyproject.tomlas Python source and paints phantom syntax diagnostics across it. Two mechanical conversions ride along, both applied byruff check --fix: selector lists inpyproject.tomlname rules rather than code them, and# noqa: CODEsuppressions became# ruff: ignore[rule-name]with their explanations preserved. The new formatter also reformats Python code blocks inside Markdown, so the docs picked up trailing-comment and blank-line normalization, and rules newly reported underprevieware ignored deliberately rather than in bulk, each with its reason recorded beside it. This is a dev-dependency change: nothing shipped changes.
Fixed
- Builder-authored defaults no longer evaporate on re-load: The builder's TOML writer emitted the key
defaultwhere the language readsdefault_value, so a default authored through the builder — or through the agent-CLIconceptcommand — validated green and then reached nothing when the file was re-loaded. The writer now emitsdefault_value, a write-then-validate round-trip test pins the survival, and the new unknown-key rejection turns any recurrence of this bug class into a hard failure instead of silent loss. pipelex.providers.bedrockno longer imports a type-stub package at runtime:ConverseResponseTypeDefwas imported at module level as well as inside the module'sif TYPE_CHECKING:block. It resolves only throughtypes-aioboto3, which is a development dependency, so the module-level copy raisedModuleNotFoundErroron any install ofpipelex[bedrock]without the dev extras. The duplicate is gone and the surviving import is the type-checking one, whose annotation is a local variable annotation and is never evaluated at runtime.- A usage example in
pipelex.test_extras.shared_pytest_pluginshad lost itsyield: The example fixture inneeds_inference_in_pipelex's docstring showed a teardown call with nothing handing control to the test first, itsyieldline having drifted into a literalYield:docstring section on a function that only ever returns abool. The example is a working fixture again.
[v0.51.0] - 2026-08-21
Added
fails_aton everyerror.*tag in the MTHDS Test Corpus vocabulary. A non-excluded tag now declares the earliest layer of checking that catches its fault:"schema"when a pass over the raw document's shape already rejects a bundle carrying it (a pipe section with notypekey, atypeoutside the closed set of pipe kinds),"runtime"when the document has to be interpreted to notice. The consumer rule is one sentence: a structural sweep expects a diagnostic on an entry exactly when its tag saysschema, and silence on every other entry.generate-corpus-vocabularycomputes and injects the field from an internally maintained set of schema faults, so it is never hand-written.
The signal lives on the vocabulary tag rather than on each entry.toml because the layer is a property of the error type: per-entry restatement would let two entries covering one fault disagree, with nothing to catch it. Excluded tags carry no value at all — an excluded tag has no entry, so nothing was measured. Previously every consumer had to hardcode which faults it believed were structural, a downstream re-reading of this repo's validation-error registry guaranteed to drift from it; the corpus now ships the answer in the wheel.
-
schema_fault_tagsproperty onCorpusVocabulary. Structural consumers — JSON-Schema sweeps, editor diagnostics, linters — read which faults they are expected to catch instead of maintaining their own list. -
Gates keeping the signal honest. The exhaustivity gate now requires every
error.*tag that is owed an entry to declare afails_at, so a new validation error type cannot land carrying a fault a structural consumer has no way to sweep for. A newtest_mthds_corpus_plxt_exclusions.pygates.pipelex/plxt.toml's corpus exclusions against the vocabulary, so the linter config and the vocabulary cannot drift apart.
Changed
- The corpus's deliberately invalid entries are linted again — all but the two whose fault the schema is supposed to reject.
.pipelex/plxt.tomlexcluded the wholeinvalid_*population behind one blanket glob, because some of those entries carry a structural fault and the config had no way to say which. The exclusion now names exactly the schema-fault entries (invalid_missing_pipe_typeandinvalid_unknown_pipe_type), and everyruntimeentry is covered bymake lintandmake formatlike any ordinary.mthdsfile.
That makes this repo the first consumer of its own signal, and it is what keeps the signal measured rather than merely asserted: declare a fault runtime when the schema in fact rejects it and its entry stays linted, so make plxt-lint goes red naming it.
- Documentation.
docs/contribute/mthds-test-corpus.mdgains afails_atsection covering the consumer rule, why the field sits on the tag, and how the linter config closes the measurement loop. Its invalid-entry authoring warning is corrected to match the narrowed exclusion: only a schema-fault bundle has to be laid out by hand now.
[v0.50.0] - 2026-08-20
Changed
- Breaking:
JobMetadatasplits its run-constant half into a newRunMetadata.user_id,pipeline_run_id,storage_scopeandrequest_idare constant for a whole run;pipe_code,pipe_run_id,otel_contextandcontent_generation_job_idchange at every step. They sat side by side with nothing saying which was which, so a copy made for a nested pipe had to know, field by field, what to carry through. They are nowjob_metadata.run_metadata.*, and the copy carries one object.
There is deliberately no read-through accessor on JobMetadata: one spelling for one fact. Every call site reads the nested path.
- A pipe's result now carries the job it was produced under.
PipeOutput.job_metadataandTracingAssembly.run_metadataare set once at the run boundary, so a transport that offloads an oversized result to storage can key it inside the run's own namespace. Both are private attributes behind properties, not model fields:PipeOutputis on the wire —pipelex-apireturns it and publishes its schema — so as a field this would have putuser_id,request_idandotel_contextinto a public API's response body and its documented OpenAPI artifact. A transport reads the value by attribute on the live object before serializing, so it never needed to be on the wire at all.
The gap was one-directional and easy to miss: every payload a run sends in carries a job_metadata to resolve a scope through, and nothing it sent back did. Under a hosted orchestrator that meant the two largest payloads an ordinary run produces — the pipe output and the tracing assembly, both around 350 KB — were stored outside every prefix the host's erasure cascade deletes, and survived deletion of the run that produced them. The same bytes were also stored a second time, correctly scoped, inside the delivery argument that carries them.
Set at the run boundary and not at the ~17 PipeOutput(...) construction sites: only the top-level output crosses a transport boundary, and stamping every constructor to serve one of them is the same "remember to add it everywhere" fragility that caused the original bug. Optional throughout — dry runs, signature stubs and tests build outputs with no job in hand — and invisible on the wire, since serialize_completed_output names the fields it returns explicitly.
[v0.49.0] - 2026-08-20
Added
- The
error_typevalues a validation verdict can carry are now an enumerable, closed registry: a.mthdsbundle that fails validation reports each fault under anerror_type, and until now there was no way to ask the runtime which faults exist. The values were reachable only by reading raise sites, and the only curated list of them lived outside this repo, in a hand-maintained conformance suite that could silently fall behind.pipelex/validation_error_types.pynow holds the vocabulary in full and enumerates it asVALIDATION_ERROR_TYPES, so a consumer building coverage over the language surface — a test corpus, a client mapping faults onto its own UI — reads the registry instead of collecting strings from whichever diagnostics it happened to have seen. It is a union of the enums the runtime already raises, never a second list beside them: the two stage vocabularies plus a newValidationResidualErrorTypenaming the one residual channel that had no enum of its own and rode an inline string. A member added to any of them is in the registry the moment it is declared.ValidationErrorItem.error_typeis typed against their union, which is what makes the registry closed rather than merely documented — an unregistered string can no longer be constructed or parsed onto an item — and which publishes the vocabulary into the OpenAPI schema served for/validate, where the field was previously an openstring. No wire value changed, including the dry-run residual'sDryRunError: it keeps the exception-class spelling it has always had, because normalizing it would break every consumer that pins the string and would buy nothing an enumeration does not already give. Membership means a value is reachable on the wire, not that it is worth exercising — the advisory-only warning type and the twounknown_*fallbacks are in the registry and a coverage consumer excludes them on its own side, with a reason, rather than pruning the runtime's truth. - Closed registry for validation error types:
VALIDATION_ERROR_TYPES, in the newpipelex/validation_error_types.py, is an enumerable, closed registry of everyerror_typea validation verdict can carry. It is a union of the enums the runtime already raises —PipeValidationErrorType,PipeFactoryErrorType, and a newValidationResidualErrorTypenaming the one residual channel that had no enum of its own — so a member added to any of them is in the registry the moment it is declared. A consumer building coverage over the language surface now reads the registry instead of collecting strings from whichever diagnostics it happened to have seen. No wire value changed. - The corpus's invalid axis: the MTHDS Test Corpus now carries an
error.*tag namespace generated fromVALIDATION_ERROR_TYPES, plus a focused invalid entry for every tag in it that a.mthdsbundle can actually produce — each authored to trigger exactly one error, and gated on doing so. Five codes are excluded, each with the measurement behind it:circular_dependency_errorandunknown_conceptare unreachable through bundle validation,optional_force_redundantis advisory-only and rides thewarningsarray, and the twounknown_*fallbacks name the absence of a diagnosis rather than a language fault. - The exhaustivity gate now requires an invalid entry to agree with itself: such an entry states its fault twice — as
expected_error, the wire string, and as anerror.*tag incovers, the normalized form — and nothing checked that the two matched. The check joins through the vocabulary's owncodefield rather than re-normalizing, so it carries no second copy of the normalization rule. A valid entry may no longer carry anerror.*tag, since its bundle produces no diagnostic for the exhaustivity arm to count. - Validation parity test:
test_validate_bundle_entry_shape_parity.pyvalidates the same bundle through both entry shapes — in-memory contents and a library directory — and requires the two to produce identical structured diagnostics.
Fixed
- The same bundle produced different diagnostics depending on how it reached the validator, and the library-directory route — the one the CLI and every local tool take — was the lossy one. Two arms in
LibraryManager._load_mthds_files_into_librarydestroyed the structured data on the way out: a pydanticValidationErrorwas rewrapped as a bareLibraryError, which routes to the error cascade's structured-forwarding arm rather than its categorizing arm; and aConceptLibraryErrorwas re-raised message-only, dropping the per-reference items it now carries. In practiceunresolved_concept,optional_input_unguardedandoptional_output_requiredwere simply unreachable from a file on disk. Both arms now preserve what they were discarding.
Changed
ValidationErrorItem.error_typeis typed against the closed registry union instead of an open string (breaking). This is what makes the registry closed rather than merely documented — an unregistered string can no longer be constructed or parsed onto an item — and it publishes the exact vocabulary into the OpenAPI schema served for/validate, where the field was previously an openstring. As part of this,PipeValidationErrorTypeandPipeFactoryErrorTypemoved frompipelex/core/pipes/exceptions.pytopipelex/validation_error_types.py: the vocabulary is a wire contract rather than pipe machinery, and it has to sit belowbase_exceptionsfor the wire item to be typed against it at all.- Two test fixtures were naming faults that do not exist, and typing
error_typeagainst the registry is what surfaced them: one claimedMISSING_CONCEPT, a code no enum has ever defined, and others used member names (INADEQUATE_OUTPUT_CONCEPT) where the wire carries values (inadequate_output_concept). Each was accepted silently while the field was an open string. They now name registry members. - The corpus's
invalid_*entries are excluded fromplxtlinting and formatting in.pipelex/plxt.toml: their structural faults are deliberate, and what guards them instead is strictly stronger — the corpus's entry-validation gate runs the real validation engine over every entry in CI, and is red both when an invalid entry fails differently and when it accidentally validates. - Documentation:
docs/under-the-hood/error-model.mdnow describes the closederror_typeregistry, anddocs/contribute/mthds-test-corpus.mdexplains the rules for authoring an invalid corpus entry.
[v0.48.0] - 2026-08-20
Added
- MTHDS Test Corpus: Introduced a canonical, tagged set of
.mthdsmethods atpipelex/test_extras/mthds_corpus/, replacing scattered per-repo fixtures with a single source of truth that ships in the wheel for synchronized cross-repo consumption. Each entry is a directory holding the method, an optionalinputs.json, and anentry.tomldeclaring what it exercises —covers,tier,granularity,validity, and for a deliberately invalid entry the exact wireerror_typeit must produce. The MTHDS test corpus page of the contributor docs states the cross-repo contract. - Corpus Loader API: Added
iter_entries()andget_entry()inpipelex.test_extras.mthds_corpus.loaderto fetch fixtures programmatically by tag, execution tier, validity, and granularity instead of by file path. - Tag Vocabulary Generation: Added the
pipelex-dev generate-corpus-vocabularyCLI command andMakefiletargets to generate the corpus tag vocabulary (native.*,operator.*,controller.*) directly from runtime registries (NativeConceptCode,PipeType). The two pipe namespaces come from one registry walk, split byPipeType.category, so a pipe kind added later lands in the right namespace with nothing to update.PipeFuncis the corpus's first recorded exclusion: it names a Python function the runtime resolves against its function registry at validation time, so no portable entry can declare one. - Feature Tag Axis: Added a hand-maintained
feature.*tag namespace for language features spanning multiple blueprint fields or lacking a registry (e.g.feature.multi_file_library,feature.optionals,feature.smart_inputs,feature.structured_output). Each feature tag is declared in the same change that lands its first covering entry, so the vocabulary never advertises coverage the corpus does not have. - Strict Corpus CI Gates: Added test suites enforcing corpus integrity: exhaustivity (every registry code has a focused entry), vocabulary drift (committed
vocabulary.tomlmatches registries), manifest validation (strict Pydantic model perentry.toml), entry validation (valid entries compile, invalid entries fail with their declarederror_type), and packaging (builds a real wheel to confirm corpus files ship while excluding the generator script). - New Corpus Entries: Added inference-tier image payload entries (
image_from_planting_brief,image_round_trip_room_photo), a nested structured-output entry (feature_structured_output_delivery_round) for lists of structured objects with optional fields, and focused entries covering batching, conditions, sequences, parallel execution, composition, extraction, and search. No entry names a model: presets are resolved by the validation engine, so an entry pinning one would fail validation on any consumer whose deck does not define it. - Documentation: Added a contributor guide for the corpus at
docs/contribute/mthds-test-corpus.md.
Changed
- Centralized Test Fixtures: Migrated end-to-end and integration tests (Date, YesNo, Smart Inputs, Optionals, Multi-file libraries, Image round-trips) to source their
.mthdsbundles dynamically from the new corpus viaget_entry(), removing all local duplicated copies previously scattered acrosstests/e2e/andtests/integration/. Nothing in this repo holds a second copy of a language-level.mthdsmethod any more. - Makefile Targets: Added
generate-corpus-vocabulary-quietto thecc(check) andup(update) targets to keep the vocabulary current during local development. - VS Code Launch Config: Updated the integration test target in
.vscode/launch.jsonfromtest_image_out_in.mthdstotest_image_inputs.mthdsto match the refactored test structure.
[v0.47.0] - 2026-08-19
Added
codegen.locknow declares its format version, so the format can grow without breaking every reader: the lock is a cross-language interchange format — this CLI writes it, other implementations of the offline check read it — and it validated strictly with no version field at all, so the day it gained any key every consumer pinned to an older reader would have failed CI with an opaque "unknown key" error instead of a drift verdict. Each lock now opens withlock_version = 1; a reader refuses a version it does not know and names which side to upgrade, and it reads the version before the key set, so a future lock is diagnosed by its version rather than by whichever new key it happens to carry first. A lock written before the field existed is version 1 by definition and needs no migration, but every regeneration rewrites the lock once to add the key — that one-line diff is the change consumers will see. The stamp header stays deliberately unversioned, since it already ignores commented fields it does not recognise.- Line endings are pinned to LF in the git index and on checkout, on every platform: this repo carries a lot of generated-but-committed text —
CLAUDE.mdandAGENTS.md,subject_grants.toml, the drift acks,.test_durations, the error-identity snapshot, the generated error pages, the migration goldens, the gateway model references — and each is produced by a writer that passes no explicitnewline, so on Windows Python translated every\nto CRLF and the whole set churned in version control with no change of content. No gate would have caught it: they all compare with universal-newline reads, which fold CRLF back to LF before the comparison. A.gitattributescarrying* text=auto eol=lfcloses that class for tracked files whichever writer produced them.text=autoleaves git to tell text from binary, so images and other binaries are untouched, and no tracked file held CRLF when this landed — it introduces no renormalisation diff, it pins the invariant. The complementary writer-by-writer fix is the structural half and is tracked separately. -
Documentation:
docs/under-the-hood/codegen-projections.mdnow covers the strict stamp-header rules, the JSON conformance theoptionspayload must meet, the policy for evolving the lock format underlock_version, and the cross-platform line-ending guarantee. -
A configuration that will not load now says whether it is wrong or merely old: A validation error is raised against the merged configuration and carries no provenance — it names a key, not which file put it there, and certainly not whether that key was correct last month. When a configuration surface's model refuses, the error now also carries a dry-run scan of that surface over the same directories
pipelex migratewalks: a structuredmigrationblock naming the remedy, whether that remedy would actually rewrite anything, whether anything there is a person's to resolve rather than the tool's, and the plan per file — the same shapepipelex-agent migrate --dry-run --format jsonemits, so an agent that parses one has already parsed the other.error_domaindeliberately staysconfig: it is a closed cross-repo enum and the agent-hook specification routes anything else to BLOCK, so a stale configuration reported under a new domain would stop an agent instead of telling it what to run. Consumers branch on the block's presence — absent means the failure is not staleness — and then onwould_write, because presence means the migration history has something to say about these files and not that a command repairs them: a block whose only finding is a path no entry explains, or an entry blocked before anything applied, has nothing to run, and naming the remedy there would send a reader to a run that writes nothing and leaves the same error standing. Every surface points that reader atpipelex migrate --dry-runinstead, where the diagnosis is. The message gains a paragraph naming the files and the next move. No value read from a user's file appears in the block, which is the third of the three channels that rule covers. A caller validating something that is not a configuration surface — a.mthdsbundle, a model deck, a routing profile — names no surface and pays for no walk. -
Inference backend definitions are a migration surface, and the key the templating change deleted is repaired on every machine that has one:
#1104removedprompting_targetfrom the model-spec blueprint and from every backend file we ship — butpipelex initnever overwrites, so the key survives ininference/backends/*.tomlon every installation that predates it, where it is fatal twice over: in[defaults]it is copied wholesale into every model of the file, and on a single model it is rejected by name since unknown per-model keys must be header-shaped. Those files were reachable by no surface, no walk and no gate. They are now theinference-backendsurface, which required the walk to learn about subdirectories — a file is claimed by the pair (directory, name), one level deep, and only into a directory some configuration family owns — and required the fingerprint to record a document whose root is the open node, since a backend file's root keys are model names rather than a field we could name. Boot tolerance joins at the backend loader too: a stale directory is replayed in memory, boots on the models the current files would produce, and warns once naming every file it carried. Migration: runpipelex migrateafter upgrading — ledger entryinference-backend@2deletesprompting_targetfrom every root table of every backend file, in~/.pipelex/and in each project.pipelex/, keeping one timestamped backup per file. Until you run it, each boot repeats the warning; nothing is ever written by a boot. A key you added that we have never heard of is still an error rather than something the entry silently drops, and the served Pipelex Gateway specs are untouched. The one key class the report stays silent about is a per-model request header, legal by shape rather than by name — and only on a model table:[defaults]is copied into every model of a file unsplit, so a header put there breaks all of them, and the report names it like any other key the schema cannot explain. -
A configuration file the ledger can explain no longer stops the boot: A schema change used to leave every machine in the field with a boot that died on
extra="forbid", naming a key the user never chose to have. When a configuration surface fails to validate, its ledger is now replayed over the same files in memory and the result validated again; a boot that succeeds says so in a warning naming the files and, for each of them the command would actually reach, thepipelex migrateremedy. The retry replays whatever the loader merged, so a file loaded throughPipelex.make(config_dir=…)from outside the two directoriesmigratewalks is named instead as the caller's to update where it lives, rather than pointed at a command that would then report nothing to do. Nothing is written either way — a tolerated boot leaves the file and its directory exactly as it found them, which is why the warning keeps coming back until the command is run. Tolerance widens what starts and never what is accepted: the re-validation is what decides, so material anunsafeentry is about still fails the boot because the model still refuses it. Anything that goes wrong inside the retry makes it decline rather than replace the error the user already has — theirs names the key to fix. A healthy configuration pays nothing: it reads no ledger, re-parses no document, and does not even import the migration engine, which is deferred for a second reason too, since the applier lives in the interpreter layer and the configuration loaders sit in the kernel layer that must not load it. This release is where it matters most: the configuration reshape givespipelex.tomlits first ledger entry, so the file every boot reads is the one tolerance carries forward on every machine that upgrades. pipelex migrateandpipelex-agent migrate: The migration engine now has commands, so a configuration that predates the installed pipelex can be repaired instead of re-initialized. Both walk the global~/.pipelex/and the project.pipelex/— those two directories only, one level deep, and into a subdirectory only when a configuration family owns it:inference/backends/is such a directory,inference/deck/is not and is never entered. A file is claimed by the pair (directory, name), which is what keepsinference/backends/pipelex_gateway.toml— apipelex_*.tomlmatch by name — from being rewritten under the main configuration's ledger. Neither command boots: a broken configuration is the reason to reach for them, so they use the ledger, the applier and the filesystem and nothing else, and a test hands them a configuration that cannot load to keep it that way. The human command plans, shows the plan and asks; the machine one writes only with--yes, since it cannot ask, and passing--dry-runand--yestogether is refused rather than resolved. Its answer is structured, andneeds_attentionis the verdict — this run left something a person has to decide, deliberately not did anything get written — with the exit code and the Markdown rendering as presentation. Every file rewritten is backed up beside itself first, and no value read from a user's file appears in either command's output or in the structured plan.- A migration now names what it cannot explain about a file: A file newer than the running pipelex — an override migrated on one branch, then an older branch checked out — breaks the boot while a replay finds nothing to do, because every operation's source is absent. That silence beside a dead boot is the failure the migration project exists to remove, so every plan now carries an
unexplained[]listing the paths of the migrated document that the current schema does not know and no ledger entry accounts for, each with the two readings the tool cannot tell apart: a typo, or a file written by a newer pipelex. This is the one part of a migration that needs the configuration model — the question goes to the surface's fingerprint, the same projection the coverage gate diffs, so the replay engine stays model-free. The document it reads is the one the run leaves behind rather than the one it found, which is what keeps every key the ledger is about to repair out of the list, and a path a blocked entry is about is left to that entry, which reports it by name with its own guidance. A key the user chose is never rendered: beneath an open mapping a typo is reported atqueues.*.retries, naming only the segment the schema does not know. -
A configuration file may now be refused for declaring a schema version its ledger no longer reaches:
min_supported_schema_versionhad no reader, which left the floor a comment. A migration now reads the reserved[meta] schema_version— exactly as tolerantly as the configuration loader strips it, so a malformed declaration is no declaration — and refuses a file below the floor by name (unsupported_schema_version), writing nothing. Without it, a ledger whose oldest entries had been squashed away would run over a file older than them, change nothing, and report it clean, because the applier skips an operation whose target is absent. -
An
unsafemigration entry now says what it is about, and keeps saying it:unsafeis the ledger's form for a change only a human can make, and the coverage gate's only remedy for a tightened numeric bound — but the engine decided whether to report one by rehearsing its operations, so an entry with none was accepted by the accounting and then guaranteed to reach nobody, and any entry went silent the moment a latersafeentry renamed the table its material sat in (reported on the first run, clean on the second, boot still failing). An entry now declares the paths whose value domain narrowed, indeclared_narrowed_paths; an entry with no operations must carry one, asafeentry may not,make check-ledgerrefuses a path the entry's own version does not record, and the coverage gate no longer takes the bare wordunsafeas accounting for a narrowing or a lost enumerated spelling — it demands the path be named. The material of anunsafeentry is traced through the ledger before a document is questioned: back through the entry's own operations, since it never applied them, and forward through every latersafeentry, so the same file is reported on every run for as long as it carries it. A file that merely sets a narrowed path is reported under a new, weakervalue_domain_narrowedreason listing the keys to check by hand — the engine is model-free by design and cannot tell a value the new domain refuses from one it accepts — and those keys are named at the spelling the migrated file carries rather than the one that matched, since the entry is questioned where the replay reaches it and the latersafeentries rename the material before the file is written — and that report, alone among blocked entries, does not fail convergence, since a healthy reference document sets those paths like everyone else's file does. Forward tracing is deliberately confined tounsafeentries: a rehearsal may guess, an application may not. -
A migration operation can now remap every value beneath an open mapping:
remap_valueacceptskey = "*", meaning "each key of the addressed table" — the same "each of these" reading the wildcard already had in atable_path, applied to keys. A mapping from the user's own keys to an enumerated value (dict[str, LogLevel]) keeps its spellings under keys only the document can enumerate, so no fixed key reaches them and this is the only shape in which a member renamed beneath one can be repaired at all. On every other kind a*key is now refused when the operation is parsed: deleting every key of a table isdelete_table, and renaming or moving every key onto one fixed name is the many-to-one that already rules a wildcard out of amove_key. Refused rather than tolerated, because an unrefused one is a dead operation that skips forever while looking like an accounting. -
A schema change that narrows what a value may be is now caught, not waved through as additive: A change can keep every path and every enumerated spelling and still break a user's file —
intbecomesstr, a free string becomes an enum, a bound moves fromge=1toge=5. Read as a path diff those look like additions, so the coverage gate used to ask for a golden regeneration, demand no version bump and no entry, and let the next boot reject a valid file with a green gate behind it. It now splits its verdict by direction: a widened type or a relaxed bound still costs only a regeneration, while a narrowing demands a bump and an entry carrying aremap_valuefor each narrowed path — or markedunsafe, which is the only remedy a tightened numeric bound has, since no structural operation can repair a value. The same rule applies to a stored entry, compared by origin so a narrowing hidden behind a rename is still caught, and to apre_historyentry, so the flag is not a way past it. To see a bound at all the fingerprint now records one: a closed whitelist overgt,ge,lt,le,min_length,max_lengthandmultiple_of, read fromannotated_types(now a declared dependency) and from the field metadata pydantic folds a top-levelField(ge=1)into, with anything unrecognized dropped rather than serialized —patternexcluded permanently, because regex containment is not decidable and every pattern edit would read as a tightening. One correction rides along: an enumerated type relaxed into a free string no longer reads as the loss of every spelling it had, and that is read structurally, solist[enum]becominglist[str]is the same benign loosening one container down — while alist[enum]flattened to a barestr, which is no loosening at all, still loses every spelling it had. The contract states in the same change what the gate structurally cannot see — narrowing expressed in a validator, which no projection of the annotations can reach. - The telemetry configuration can now be migrated instead of thrown away, and the escape hatch that would have let it ship unverified is closed:
telemetry.tomlcarries the package's first migration entry,telemetry-config@2, which lifts a flat first-generation file —telemetry_mode,host,project_api_keyand the rest at the document root — into today's[custom_posthog]section, keeping every value the user chose; four settings the current shape has no home for are dropped, and the entry says which and why. Because that change predates the first fingerprint, no diff describes it, and such an entry is markedpre_historyand exempted from the coverage gate's accounting — so the verification that replaces it lands in the same change and neither ships without the other. A pre-history entry must declare the paths it removes (an empty declaration is refused when the ledger is parsed), none of those paths may appear in any fingerprint at or below its own version, the fingerprint pair it sits between must show no removal at all, and every operation must act inside the declaration. It also ships a hand-authoredbefore@N.tomlbeside the golden chain, which the transform check migrates and holds against the current shape and the current model — the same three claims every other entry answers. One comparator refinement came with it: a created path is now checked againstdefaults@Norfingerprint@N, because an optional key defaulting toNoneis a legal destination that no TOML document can carry. - Configuration surfaces now have a migration ledger and a gate that forces one to be written: Every configuration surface —
pipelex.tomland its tiers,telemetry.toml,pipelex_service.toml, and the inference backend definitions — carries a checked-in ledger atpipelex/migration/ledgers/<surface-id>.tomlrecording, as data, every shape change it has ever undergone, plus a golden chain (fingerprint@N,defaults@N) that pins what the schema looked like at each version. Each is cut at schema version 1 with no entries; the entries this release then ships on top of that baseline are the bullets below. The newmake check-migration-schemas(aliascmig, inmake check) recomputes each surface's fingerprint and refuses a change that would break a user's file without recording how to repair it — a removed path or a lost enumerated spelling demands a version bump and an operation accounting for it, a destination the new schema does not know is refused by name, and an entry that removes a path the schema still has is refused as over-deletion. Its counterpartmake up-migration-schemas(aliasumig, inmake up) regenerates the goldens; the regeneration diff is the review artifact. One standing invariant is checked whether or not anything changed: every required path must have a value in its surface's defaults layer, because that is what makes an added key absorbable rather than breaking. The contract the whole thing implements isdocs/migration-ledger.md. - The migration operation vocabulary: Fix operations became a discriminated union on
kind, published as two subsets — every kind for the.mthdsfix path, and the structural kinds only for ledgers, so an operation that would write a value into a user's file fails when the ledger is parsed rather than at review time. Two new kinds land with it:move_key, which relocates a key across tables (the whole subtree travels when the key is table-valued, and a missing destination parent is created), andremap_value, which rewrites a value against an explicit old-to-new mapping. A*segment addresses every entry of an open mapping, expanded over the keys the file actually holds. A newCONFLICToutcome separates "this cannot be done without choosing on your behalf" — a rename or move onto an occupied destination, or anensure_tableover a key holding something that is not a table — from the benignSKIPPEDit used to hide inside; a conflicting operation writes nothing at all. [meta] schema_versionis reserved in configuration files: Every configuration-surface reader now tolerates and strips it. Nothing writes it — the key exists so a later release can stamp a version into a file without that being a breaking change today.make check-ledger(aliascl): The second migration gate, and the first one that joinsmake agent-checkas well asmake check. It asks what a ledger entry is allowed to say and whether replaying it is harmless, wherecheck-migration-schemasasks whether a schema change was accounted for. An operation may only act on material some schema version removed — one addressing a path the schema still has would rewrite a perfectly valid file on every run — and it may never name a concrete key beneath an open mapping, whose keys are the user's and unbounded, while the*segment that addresses every entry of one is legal exactly there and nowhere else. Asafevalue remap whose old spelling is still legal, or whose target is a free string where staleness cannot be proven, is refused because it would rewrite a deliberate choice; a remap to a spelling the new schema rejects is refused because every migrated file would then fail to load with the tool reporting success. A retired path, or a remapped-away spelling, may never come back. Finally, replaying the whole ledger over each of the surface's two reference documents — the complete packaged configuration and the sparse kit starter template, deliberately different shapes — must apply nothing, report nothing, and hand back the very bytes it was given. It reads checked-in files only and fingerprints no live model, which is what makes it safe to run in the loop agents run constantly: every failure names a file the author wrote.- The migration engine: A surface's ledger can now be replayed over a real file. The engine takes a document's text and returns the text it should now have, plus a per-file plan naming the entries that applied, the entries it would not apply, and why. Two reporting rules keep a plan from being more optimistic than the file: an entry is reported once — a conflicting operation routes the whole entry into the blocked list, carrying whichever of its operations did land — and an
unsafeentry is rehearsed against the document first, so it stays silent on a file that does not need it rather than warning at every boot forever. Writing is transactional per file and always takes a backup: exactly one per file, named<file>.bak.<UTC stamp>, inheriting the source file's mode rather than the umask, written before the replacement and pruned only after it commits. A file that is unparseable, unwritable or changed during the run is reported as blocked while every sibling is migrated normally — unlike the.mthdsfix loop, whose files only make sense together and which still commits a round all-or-nothing; both now share one set of transaction primitives. Nothing calls this from the command line yet. - The transform goldens:
make check-migration-schemasnow also performs each migration it is asked to believe in. For every ledger entry it applies the entry's operations to the frozen reference document of the version before it and compares the result with the frozen document of the version after: every path the migration creates must be one the new document has, every path the two documents share must survive, and the last link's output — read the way a user's file is really read, beneath the current defaults layer — must be accepted by the current model. That closes the one defect the other two checks pass in silence: a rename whose destination is misspelled is accounted for by coverage and skipped by convergence, and then migrates every file to a key the schema rejects while the tool reports success. The comparison is deliberately one-directional in each claim, so a bump that also adds keys, edits comments, flips defaults or ships one more entry in a packaged deck stays green. - Replay neutrality is now a property rather than an assertion: The guarantee a user's machine depends on — replaying a surface's whole ledger over a file already valid at the current schema changes not one byte — is checked over generated documents, not only over the two reference documents. A reference document carries exactly one value per key, so it cannot see an operation that misbehaves on the other legal spelling of an enumerated field; a sampler that reaches those spellings can. The sampler proposes each mutation from the surface's fingerprint and lets the models decide whether it lands, because schema membership is settled by validators no type projection can see: a legal enum member can still be rejected for what it requires elsewhere in the document, and a mapping typed with arbitrary string keys can still refuse a key outside a fixed set. Idempotence — replaying twice equals replaying once — and prefix coherence — a file caught halfway lands where a replay from zero lands — are checked alongside it, and each generator carries its own vacuity check, since every one of these properties is trivially true over documents nothing acts on.
hypothesisjoins the development dependencies. pipelex doctorreports pending configuration migrations, and--fixruns them: A configuration the migration history explains now boots with a warning rather than an error — andpipelex-agentsilences logging process-wide as its first act, so a machine consumer never learns from a boot that its configuration is stale. Asking was the only channel left and there was nothing to ask. The health report gains a Configuration Migrations row that ispipelex migrate's own dry run: a structuredfinding(up_to_date,pending,needs_attention,unavailable), the files a migration would rewrite, and separately the files it will not repair on its own — both sets, because a drifted machine usually has some of each and a reader who heard only the first would stop with a broken file still in place.pipelex doctor --fixthen offers to run the migration, reaching the command's own write pass rather than a second implementation of it. The row is deliberately not scoped by--global: it answers for a command that walks both configuration directories, and a row naming fewer files than the command touches would be a surprise write. A failure inside the scan is reported as a failure to check rather than as health, and never as an exception — an exception there would replace every row of the report with a single line.
Changed
- Breaking: where a run's bytes go is now told to the runtime, not derived from who is running (
JobMetadata.storage_scope): Storage keys were built fromuser_id—{user_id}/results/{run_id}/for delivery,{primary_id}/{secondary_id}/…(always{user_id}/{pipeline_run_id}/…) for generated content, a flat top-levelnormalized/for inputs pulled off adata:URL. That ties where the bytes live to who uploaded them, which is fine until two people share the work: the second one asks for a file the first one attached and is refused, because the only thing standing between them is a string comparison on the first path segment. A host that serves teams cannot express "these two people may read the same file" in that layout at all.
JobMetadata gains a required storage_scope: one opaque, host-supplied prefix that the runtime composes assets/, results/ and payloads/ onto and never parses. The runtime does not learn what a tenant, an organization or a method is, and that is deliberate — those are host concepts with no meaning in an MIT-licensed runtime, so threading them through the transport was rejected in favour of one string the host fills however its own tenancy works. PipelexPipeRunInput carries it across the boundary alongside a now-required user_id.
It is validated at construction, on the type. The value becomes a storage key prefix, so a .. or a leading slash in it is a traversal out of the tenant's namespace; a JobMetadata that exists is one whose scope is safe, which makes every key derived downstream safe by construction rather than by each call site remembering. This replaces a per-segment sanitizer that ran over user_id and pipeline_run_id separately — collapsing them into one slash-bearing string would otherwise have deleted that control silently.
uri_format loses both {primary_id} and {secondary_id}, replaced by one {storage_scope}. Supported placeholders are now {storage_scope}, {hash} and {extension}, and the shipped default becomes {storage_scope}/{hash}.{extension}. This is not a rename of {primary_id}: keeping that name would have left the configuration saying "primary" with no "secondary" for it to be primary to, and given an operator no way to tell that the placeholder means the whole tenant prefix rather than some id. {hash} is unchanged — the first 16 hex chars of the SHA-256 of the file's own bytes, so identical output keeps one name instead of accumulating duplicates.
A configuration naming either retired placeholder is refused at boot, by name, with the supported set listed. The free-string value domain is not something the migration ledger can prove stale — per docs/migration-ledger.md a remap_value there could only be unsafe, i.e. reported and never applied — so this is a loud failure rather than a repair. Edit uri_format in your .pipelex/pipelex.toml if you customized it.
- Breaking:
"anonymous"no longer appears anywhere on the identity or storage path, and a test keeps it that way: The dry-run paths still passedOTelConstants.DEFAULT_USER_IDas auser_id, which is the last place the telemetry placeholder was still being bound as an identity. They now passDRY_RUN_USER_ID— a dry run has no caller in the identity sense either, so it says so in its own word rather than borrowing tracing's. The constant survives for telemetry, where "a span with no known caller" is a real concept.
test_storage_scope.py now scans every module under pipelex/ and fails if DEFAULT_USER_ID is ever bound to a user_id or a storage_scope again. The pattern deliberately matches an annotated parameter default (user_id: str = OTelConstants.DEFAULT_USER_ID) as well as a plain assignment — the first version of the guard did not, and passed a real injected regression while looking like it worked.
- Breaking: identity is required, and
"anonymous"goes back to being a tracing label:pipeline_run_setupcoalesced a missinguser_idtoOTelConstants.DEFAULT_USER_ID— the string"anonymous", a tracing placeholder for a span with no known caller. It leaked into the identity path and became a storage key prefix, so every run without an authenticated caller wrote into one sharedanonymous/namespace where each tenant could read the others' outputs. The fallback is what made it silent: a missing identity looked exactly like a present one.
user_id and storage_scope are now required parameters of pipeline_run_setup and prepare_pipe_job, so the same omission is a TypeError at the call site instead of a shared prefix in production. The constant survives for telemetry only.
Two callers pass explicit, greppable values rather than inheriting a default: a dry run — which provably stores nothing — passes DRY_RUN_STORAGE_SCOPE, and PipelexMTHDSProtocol defaults to LOCAL_USER_ID / LOCAL_STORAGE_SCOPE, because a run on somebody's own machine genuinely has one user and no tenancy. A constructor default stating a true fact at the boundary where it is true is a different thing from an or fallback deep in the call stack, and a multi-tenant host cannot reach these by omission — the seam it calls requires both explicitly.
Fixed
- The codegen stamp header is now read strictly:
pipelex codegen checkverified a generated file as pristine even when a line had been injected between the stamp's fence markers, because the content hash covers only the body below the fence — so an executable statement could sit inside a header that saysDO NOT EDITand the tree would still report as current. Any line inside the header that does not carry the file's comment prefix now makes the stamp unparseable, which the check already reports ashand-edited. Theoptionsvalue must also be conformant JSON: Python's non-standardNaN/Infinity/-Infinityliterals are refused, so a stamp cannot hold something only a Python reader accepts — the header is a cross-language interchange format, and the second implementation of the offline check (in@pipelex/sdk) already rejected them. Commentedkey: valuelines a reader does not recognise are still ignored, so the header can gain fields without breaking older readers. One consequence to know: a file whose header was tampered with is no longer onepipelex codegen typeswill overwrite — it now refuses a destination it cannot prove it owns rather than clobbering work, so such a file has to be removed before regenerating, which is what the check's advice already tells you to do. pipelex codegen checknow reports drift in one order, whichever half of the report it comes from: locked artifacts were listed by their full path compared as a string, while stale orphans came out in the order a directory walk found them — a component-by-component order. For a tree holding amodels/directory beside amodels.py, the two halves of a single report therefore sorted paths by different rules, and a second implementation could not mirror both with one comparator. Both halves now use the plain full-string sort, and the ordering is written down as a contract: every locked-artifact drift first, then every orphan, each group by path.-
Generated files are written with LF line endings on every platform:
save_text_to_pathlet Python translate each newline toos.linesep, so a file emitted on Windows landed as CRLF on disk while its recorded content hash had been taken over the LF text it was built from. The verdict survived, because reads translate back, but the promise that regenerating produces byte-identical output held only within one platform — a team mixing Windows and Linux saw generated trees churn in version control with no change of content. Every artifact this writes (generated code, the codegen lock, JSON outputs, input templates) is now the string verbatim. -
The scope was validated after the first thing that wrote with it.
prepare_pipe_jobran the data-url normalizer — which stores bytes at{storage_scope}/assets/…— and only then constructed theJobMetadatawhose validator is supposed to make that key safe. A scope carrying..therefore escaped the tenant's namespace, and the guard meant to stop it ran afterwards, on damage already done. The defect was ordering, not absence: this file's own "safe by construction" claim did not hold on the one path that mattered. The scope is now validated at the top of the seam, before anything composes a key from it. Pinned by a test that asserts the normalizer was never called, with real inputs — the empty-inputs harness skips the normalizer on its own, so the same test written that way passes with the guard deleted. -
Generated content had no leaf: it landed loose at the root of the scope. Three things write during a run and each built its key separately — the data-url normalizer under
assets/, storage delivery underresults/, and generated bytes (an image a pipe produced) directly at{storage_scope}/<hash>.<ext>, becauseuri_formatrendered the whole key and shipped without one. A file under no leaf is invisible to anything listing a run's content by category, cannot be retained or dropped separately from the rest, and shares the root with every leaf added later. Generated bytes now land undergenerated/.
It is its own leaf, not assets/ or results/. assets/ is what the caller supplied and this is what the run produced — a provenance split an operator can act on ("drop what we generated, keep what the user gave us"). results/ is the delivery envelope: written once at the end, under fixed names, only when storage delivery is configured — whereas generated content is written during the run, content-addressed, and usually intermediate, since a generated image is more often an input to a later step than an output of the pipeline.
Breaking: uri_format renders the FILENAME only, and {storage_scope} is no longer a supported placeholder ({hash} and {extension} remain; a leading / is now refused). The prefix moved into code because a placeholder an operator may omit is an invariant that does not hold — a config dropping it put every run's generated bytes in one namespace. What the format still decides is the filename, including any subdirectory beneath the leaf.
-
Every local run with storage delivery overwrote the previous one. The run id used to be part of the storage key (
{user_id}/{key_prefix}{pipeline_run_id}); collapsing the key onto the scope removed it, which is correct for a host — a hosted scope already identifies the run,<org>/<method>/<run>— but left the local default naming a tenant and no run. So every local run wrotelocal/results/<filename>, the same key each time, each one silently destroying the last. A local run's scope is now its run id, composed inpipeline_run_setupbecause that is where the run id is minted and the constructor default cannot name a run that does not exist yet.LOCAL_STORAGE_SCOPEis a sentinel meaning "no host supplied a scope" and never reaches a storage key: a tenancy segment on a laptop separates nothing, since there is exactly one tenant, while the run id is the only part that has to be distinct. Keyed on the sentinel exactly, never a prefix test, so a host-supplied scope is never rewritten. -
A scope ending in a newline passed validation.
validate_storage_scopeusedre.matchwith a$anchor, which admits one trailing newline, so"org/mt/run\n"was accepted and the newline travelled into every storage key and log line built from the scope — an unaddressable key and a log-forging primitive. Nowfullmatch. -
The bridge payload required the scope but never validated it.
PipelexPipeRunInput.storage_scopewas a barestr, so a payload arriving from another process could carry a traversal that was only refused frames later, wherever a key was first composed. It is validated at construction now, making a malformed payload a decoding error that names the field.
[v0.46.4] - 2026-08-18
Changed
- This repo now dogfoods its own migration
.gitignoreconvention: the project-local.pipelex/directory picked up the.gitignorethatpipelex migrate/pipelex initwrite to keep timestamped backup copies (*.bak.<timestamp>) out ofgit status— the same file every consumer project has received automatically since v0.46.2.
Fixed
- The config-sync gate no longer goes red on what the migrator leaves behind:
make checkcompares the repository's own.pipelex/against thepipelex/kit/configs/templates by contents, and it had never been taught about the two things pipelex itself writes into a configuration directory — the.gitignoreadded in v0.46.2 (which has no kit counterpart by design) and the timestamped*.bak.<stamp>copies a realpipelex migrateleaves beside every file it rewrites. Dogfooding the.gitignorein v0.46.4 brokemake checkoutright, and any developer who ranpipelex migratein a checkout broke it again — putting the dirtied-repository problem that.gitignoreexists to solve straight back, one gate over. Both the check andmake up-kit-configsnow look past those artifacts, matching backups and.rescue.copies by a glob built from the same infixes and stamp shape the backup namer writes, so a rename there cannot orphan the rule. This also closes a latent hole inmake up-kit-configs, which would have mirrored a developer's backup copy into the shipped kit.
[v0.46.3] - 2026-08-18
Fixed
- The configuration-directory
.gitignorenow reaches machines with nothing to migrate: v0.46.2 added a.gitignoreinside.pipelex/to keep migration backups out ofgit status, butpipelex migrateonly wrote it on a run that actually had a file to carry forward. A machine already at the current schema was reported clean and returned before the write pass, so it never got the rule — and that is the state every user is in between schema changes, which meant in practice almost nobody received it. Any real (non---dry-run)pipelex migratenow writes it whether or not there is anything to migrate; a run whose migration you decline still writes nothing at all.
[v0.46.2] - 2026-08-18
Fixed
- Migration backups no longer dirty your repository:
pipelex migratecopies every file it rewrites to<file>.bak.<timestamp>beside the original, and inside a project's.pipelex/those copies showed up as untracked files ingit status— a dozen of them after a single run. Pipelex now keeps a.gitignoreinside the configuration directory itself, ignoring exactly the timestamped copies it makes; it is written bypipelex initand by any realpipelex migrate, so a machine whose.pipelex/predates this gets the rule from the run that would otherwise have made the mess. An existing.gitignorethere is never modified, and a.rescue.copy is deliberately left visible because it is the one the report asks you to go and collect.
[v0.46.1] - 2026-08-18
Changed
- Test-duration map refresh is now incremental (developer tooling):
make store-test-durations(std) no longer re-runs the whole suite. It collects the tests, measures only the ones missing from.test_durations, and leaves recorded values alone — which takes seconds when little has been added, instead of several minutes. This is driven by what actually unbalances the CI shards: measured against the real eight-way split, a map with stale values but complete coverage costs about 7% of balance, while one with current values and missing entries costs over 50%, becausepytest-splitimputes an unknown test at the suite mean (~0.25s) when the median test is ~0.002s. Addedmake store-test-durations-force(stdf) for the occasional full re-measurement. The refresh now also prunes entries whose test file no longer exists, making thetest_test_durations_pathsgate self-healing, and keeps any recorded value that has not meaningfully moved, which cuts the release diff for that file by about 95% — it had grown large enough that automated PR reviewers declined to read the branch at all. New page:docs/contribute/test-duration-map.md.
Fixed
- Migrator comment fidelity: Fixed
pipelex migrateso structural operations no longer leave comment banners behind or attach them to the wrong sections. Moved keys and tables now carry their introducing comment block with them; deleted keys and tables now drop their introducing comment block; items appended under an existing table now land before that table's trailing banner; and the.mthdsfix path now drops comments above stripped native-concept redeclarations. Note: files already migrated by v0.46.0 are not rewritten automatically—tidy the comments by hand or restore the.bakfiles and re-runpipelex migrate.
[v0.46.0] - 2026-08-17
Added
- Configuration migration engine & commands: Introduced
pipelex migrateandpipelex-agent migrateto bring user configuration files up to the current schema. Migrations rewrite files in-place (preserving user values), create timestamped.bakbackups, and run without booting Pipelex, so no credentials or network access are required. Both walk the global~/.pipelex/and the project.pipelex/only, one level deep, and enter a subdirectory only when a configuration family owns it. Configuration validation errors now include a structuredmigrationblock so machine consumers can distinguish a stale config from an invalid one. - Boot tolerance for stale configs: A config that is out-of-date but explainable by the migration ledger no longer hard-crashes the boot. Pipelex replays the migration ledger in-memory, boots successfully, and warns the user to run
pipelex migrate. Nothing is ever written by a boot, so the warning returns until the command is run, and a file the in-memory re-validation still refuses fails the boot as before — tolerance widens what starts, never what is accepted. pipelex doctormigration integration: Added a "Configuration Migrations" row that dry-runs pending migrations;pipelex doctor --fixnow offers to apply them. The row reports the files a migration would rewrite and, separately, the files it will not repair on its own, because a drifted machine usually has some of each and a reader who heard only the first would stop with a broken file still in place.- Migration ledgers & gates (developer tooling): Every configuration surface —
pipelex.tomland its tiers,telemetry.toml,pipelex_service.toml, and the inference backend definitions — now carries a checked-in ledger recording each shape change as data, plus a golden chain pinning what the schema looked like at each version. Addedmake check-ledger(cl) to verify ledger legality and convergence,make check-migration-schemas(cmig) to ensure all schema changes are accounted for in the ledger, andmake up-migration-schemas(umig, withumigfto force past a refusal) to regenerate the goldens.umigdeliberately leftmake up: rewriting a head golden after deleting a field would erase the very removal the coverage gate exists to catch. - Migration operation vocabulary: Fix operations became a discriminated union on
kind, published as two subsets — every kind for the.mthdsfix path, and the structural kinds only for ledgers, so an operation that would write a value into a user's file is refused when the ledger is parsed rather than at review time.move_keyrelocates a key across tables andremap_valuerewrites a value against an explicit old-to-new mapping, with a*segment addressing every entry of an open mapping. A newCONFLICToutcome separates "this cannot be done without choosing on your behalf" from the benignSKIPPEDit used to hide inside; a conflicting operation writes nothing at all. - A migration names what it cannot explain, and refuses a file it cannot reach: Every plan now carries an
unexplained[]listing paths of the migrated document that the current schema does not know and no ledger entry accounts for — a typo, or a file written by a newer Pipelex, two readings the tool cannot tell apart. Separately, a file declaring a[meta] schema_versionbelow its ledger's floor is refused by name (unsupported_schema_version) instead of being run over, left unchanged, and reported clean. No value read from a user's file is ever rendered in either report. [meta] schema_versionis reserved in configuration files: Every configuration-surface reader now tolerates and strips it. Nothing writes it — the key exists so a later release can stamp a version into a file without that being a breaking change today.- Value domain narrowing detection: The migration engine now detects schema changes that narrow allowed values (e.g.,
inttostr, or tightening age=1bound toge=5) and forces the author to provide a migration path or anunsafemanual warning. Read as a path diff those look like additions, so they used to cost only a golden regeneration and let the next boot reject a valid file with a green gate behind it. Anunsafeentry must now name the narrowed paths indeclared_narrowed_paths, and that material is traced through later entries so the same file keeps being reported for as long as it carries it. To see a bound at all the fingerprint records one, over a closed whitelist (gt,ge,lt,le,min_length,max_length,multiple_of);patternis excluded permanently, because regex containment is not decidable and every pattern edit would read as a tightening. - Replay neutrality is checked as a property, not asserted: The guarantee a user's machine depends on — replaying a surface's whole ledger over a file already valid at the current schema changes not one byte — is now checked over generated documents rather than only the two reference documents, which carry one value per key and so cannot reach the other legal spelling of an enumerated field. Idempotence and prefix coherence are checked alongside it, each generator carrying its own vacuity check. The coverage gate also performs every migration it is asked to believe in, applying each entry to the frozen document of the version before it and holding the result against the version after, which catches a rename whose destination is misspelled — accounted for by coverage, skipped by convergence, and otherwise shipped.
hypothesisjoins the development dependencies. - Claude skills: Added the
add-migrationskill to automate ledger entry creation from coverage gate refusals, and updated thereleaseskill to enforce ledger completeness before cutting a release.
Changed
- Configuration root reshaped (Breaking): The
pipelex.tomlroot structure has been reorganized to mirror the runtime layers, and the redundant_configsuffix has been dropped from keys:[runtime](process-scoped infrastructure),[inference](the model-calling seam, formerly[cogt]),[interpreter](library-scoped method machinery), and[kit](dev tooling). So[pipelex.log_config.rich_log_config]is now[runtime.log.rich_log], and code reading the config tree moves with the keys:get_config().pipelex.storage_configisget_config().runtime.storage. Class names keep their suffix — the redundancy was in the key, not the type. Migration: runpipelex migrateafter upgrading — ledger entrypipelex-config@3rewrites~/.pipelex/and every project.pipelex/file in place, keeping every value you chose. - Templating style is now an authoring decision (Breaking): How a pipe's inputs are tagged into a prompt is declared on the pipe itself (
templating_styleonPipeLLM) rather than inferred from the model, with one runtime default for everything that declares nothing ([inference.templating].default_templating_style, shipped asxml). The global default is now XML tags (<tag>...</tag>) instead of back-ticks, so every prompt rendered for an OpenAI-family model will change shape. Compose, image-generation, search, and structuring prompts follow this same runtime default, so any template of yours using| tagchanges shape whichever model family you run on. Migration: runpipelex migrate— ledger entrypipelex-config@2carries a tuned[pipelex.prompting_config]forward, the default style travelling todefault_templating_stylewhile the per-targetprompting_stylesmap is dropped, since a style is no longer chosen per model; the backup beside the file is where the old map remains readable. - Inference backend files are a migration surface (Breaking):
prompting_targetwas removed from every backend file we ship, butpipelex initnever overwrites, so the key survives ininference/backends/*.tomlon every installation that predates this release — where it is fatal twice over: in[defaults]it is copied wholesale into every model of the file, and on a single model it is rejected by name as an unknown, non-header key. Those files were previously reachable by no surface, no walk and no gate; they now are, and the backend loader replays a stale directory in memory so the boot survives with a warning naming every file it carried. Migration: runpipelex migrate— ledger entryinference-backend@2deletesprompting_targetfrom every root table of every backend file, in~/.pipelex/and each project.pipelex/, keeping one timestamped backup per file. - Strict Jinja2 templating filters (Breaking): The
tag,format, andwith_imagesfilters no longer fall back to a built-in style when the rendering context lacks one; a missing style now raisesJinja2ContextError. Every prompt-rendering entry point now supplies a resolved style, so a missing one is a bug rather than a shape nobody chose. An explicit filter argument (| format("markdown")) is unaffected, and an unknown format name is reported as a template error instead of a bareValueError. templating_styletables reject unknown keys (Breaking):TemplatingStyleand the rich template table ofPipeComposeare nowextra="forbid"like every other blueprint. A misspelled optional key —{ tag_style = "xml", text_formt = "markdown" }, ortemplating_stileon a template table — used to be dropped silently, yielding a prompt rendered under a shape the author never asked for; it is a validation error now. The MTHDS JSON Schema gainsadditionalProperties: falseon both definitions.- Unknown per-model backend keys must be header-shaped (Breaking): Any unrecognized key in a per-model table inside
inference/backends/*.tomlis sent to the provider as an HTTP header. To prevent typos from silently becoming headers, these keys must now contain a hyphen (e.g.,x-portkey-provider); keys without hyphens fail the boot, in strict and lenient boots alike, naming the key, the model and the file. The key must also be usable on the wire — RFC 7230 token characters only, with a printable single-line string value — and a hyphenated spelling of a real field (max-tokens) is rejected by name. In the remotely served Pipelex Gateway config the same rule prunes such keys instead of failing, as version skew. - Telemetry config is migratable:
telemetry.tomlis now a migration surface. Upgrading from the legacy flat format to the[custom_posthog]format is handled bypipelex migrate, preserving user API keys and exporters instead of forcing a destructivepipelex init telemetryreset. The loader now raises through the same reporting path as every other configuration surface, so the human CLI, the agent envelope andpipelex doctorall say whether the file is old, wrong, or reported on by a migration that would not repair it, rather than each holding a hardcoded remedy. Migration: runpipelex migrate— ledger entrytelemetry-config@2lifts a flat first-generation file into[custom_posthog], keeping every value that has a home in the current shape and saying which settings it drops and why. - The migration ledger contract is published, and a config-model change owes it a review:
docs/migration-ledger.mdjoins the documentation navigation beside the other contributor contracts. Theconfig-docsdrift contract's review list gains the migration ledgers and the two modules holding the validators that narrow a configuration value domain — a narrowing expressed in a validator is invisible to the fingerprint projection the coverage gate reads, so no derived check can demand the ledger entry it owes, and a review obligation is the only place that can catch it. - Keyless dry runs:
pipelex run --dry-runno longer requires inference credentials, bringing it to parity withpipelex-agent run --dry-run. It boots keyless with real model specs, every run forced to DRY. Anyone who relied onpipelex run --dry-runas a credentials check should use a live run for that. - Reasoning budgets: Thinking budgets are now worker-owned (keyed by family constants like
"anthropic"or"gemini") rather than read from the removedprompting_target. Behavior is unchanged; the budget maps keep theiranthropicandgeminikeys. - Remote gateway config: Bumped the remote inference config to
pipelex_remote_config_12.json, whosedefaultsno longer declare the removed field; earlier releases stay pinned to_11, which is frozen. Unknown keys in remotedefaultsare now pruned rather than rejected, to tolerate version skew — local backend files stay strict.
Fixed
pipelex doctorbracket escaping: Fixed a crash wherepipelex doctorfailed with aMarkupErrorwhen a file path or error message contained brackets (e.g.,[dev]) that Rich interpreted as formatting tags — worst exactly when the report is what the user came for. Every value the report interpolates is now escaped, including configuration paths, backend names, deck filenames and the text of every exception the--fixpass reports.pipelex doctor --fixno longer offers to overwrite a telemetry config it could migrate: The fix machinery decided what it could repair by searching the row's message for"format has changed", and on a match offered to write a freshtelemetry.toml, discarding every setting in it. It now branches on a structured finding, so an out-of-date file is answered withpipelex migrateand an invalid one is left to a person. The doctor's telemetry check also strips the reserved[meta]table the way boot does, instead of reporting a perfectly bootable file as invalid.- Agent CLI double envelopes: Fixed
pipelex-agent modelsandcheck-modelprinting two JSON error envelopes on a single stream, resulting in invalid JSON output.agent_errorleaves throughtyper.Exit, which is aRuntimeErrorand not aSystemExit, so a command boundary re-raising onlySystemExitcaught the error it had just reported and reported it again. - Dotted-key renaming: Fixed a bug in the TOML fix applier where renaming a dotted key (e.g.,
a.b = 1) accidentally converted it into a block header ([a.b]), swallowing subsequent scalar values into the wrong table. The applier now also refreshes every chunk of a table written in several places, so renamingain a[a.b]…[a.c]…[a.b.d]layout no longer leaves the later chunks rendering under the old name. Both reach every caller of the applier, the.mthdsfix loop included. - Symlink migration: Migrating a symlinked configuration file now correctly replaces the file at the target of the symlink rather than replacing the symlink itself with a regular file.
- Backup rotation race conditions: Fixed backup file rotation so a failed migration run deletes its own temporary backup but never deletes a backup created by a concurrent run. A copy kept because a write could not be confirmed is renamed out of the
.bak.rotation into.rescue., so a later run cannot prune away the very file the report told the user to go and get. The backup's directory entry is nowfsync-ed before the target is replaced, so "back up first, replace second" holds across a power loss and not only a process exit. - Boot failure reporting: Fixed a failure to load
pipelex.tomlproducing a bare Python traceback instead of the intended user-friendly, field-level error message.RuntimeBootand the doctor both caughtpydantic.ValidationErroraroundsetup_config, and the main configuration raisesConfigValidationErrorinstead, so the arm never fired for the one configuration everything depends on. Both sites now catch either shape a refusal takes and share one helper. - Backend errors name the model, the backend and the file: A refusal cut down to
max_tokens: Input should be a valid integersent the reader to grep a directory of files that all have amax_tokens; the loader now says which model, which backend and which file before it quotes the pydantic analysis, and the model deck's message keeps its paths the same way. A backend table whose own fields fail the blueprint (endpoint = 42) is raised as the library's own validation error rather than escaping as a bare pydanticValidationErrorthat no clause in the boot named. A message carrying a migration block no longer also suggestspipelex init config, which would have reset every value the block had just promised to keep. - Gateway enabled truthiness: Fixed
enabled = 1inbackends.tomlbeing evaluated differently by the loader and the boot process, which silently dropped the gateway. Both readers now read the value the same way. - A blocked configuration file says which state it is in:
FileBlockedReasongainedunreadableandstate_uncertain, so an unreadable file is no longer reported as unwritable and a write whose outcome the transaction could not describe is no longer reported as one that never happened.
Removed
prompting_target(Breaking): Removed fromLLMSetting,InferenceModelSpec,InferenceModelSpecBlueprint, and all shipped backend TOML files, along with thePromptingTargetenum and the per-target style map. Deleteprompting_targetfrom your local.pipelex/inference/backends/*.tomlfiles, or the strict boot loader will reject them — or runpipelex migrate, which does it for you. Checkportkey.tomlin particular: the key appears in[defaults]and on individual model entries, and a surviving per-model one is rejected by name as an unknown, non-header key.- Legacy prompt templates: Removed
LLMPromptTemplate,LLMPromptFactoryAbstract,LLMPromptTemplateInputsandLLMPromptTemplateInputsError, which relied on hardcoded XML+Markdown styles predating the new configurable templating system and had no production consumer. - Deprecated config keys: Removed
[migration.migration_maps],session_id, andplugins.boot_orchestratorfrom the root configuration. Renames are ledger entries now, so the old-to-new name hints in validation errors go away,.mthdshints included;boot_orchestratoris strictly a boot argument — pass--orchestratororPipelex.setup(boot_orchestrator=...)and read it back at run time from theruntime_hubaccessor rather than off the config.
[v0.45.0] - 2026-08-14
Added
-
Per-node token usage and cost on the run graph (Breaking):
GraphSpecnodes gained ausageobject and the graph gained a run-levelusagerollup, so a consumer can finally answer "which pipe spent the money". The link already existed in the event stream — everyUsageReportEventnames its graph node — and the assembler was dropping it; it now folds those events into per-node totals (own inference calls, tokens by category, cost) plus asubtree_*rollup up the parentage chain, so a controller reports what its whole branch spent. Cost is deliberately three-valued and never overloaded:nullmeans unrated (no rate table — own-GPU models, dry and mock runs, so every dry-run graph is unrated), arated_inference_callscount marks a mixed rated/unrated node's cost as a lower bound rather than a total, andtotal_tokensis input_joined + output — never the sum ofnb_tokens_by_category, whereinput_cachedis a subset ofinput. Usage whose node cannot be resolved lands in a typedunattributedbucket instead of being dropped, so the graph's total cannot silently disagree with the cost report's. Every dollar comes fromcompute_tokens_usage_cost, the same engine the cost table and the API wire records use. Breaking:NodeSpecandGraphSpecareextra="forbid", so an older pipelex rejects the new JSON. See Per-node Usage Attribution. -
The run graph now records which model actually ran, per node.
NodeUsageSpecgainedby_model(andsubtree_by_model): theinference_model_name,inference_model_id, call count and cost of every model a node actually used, ordered most-used first. Until now a GraphSpec named a model in two places and neither was the outcome — the pipe blueprint holds the authored choice ($writing-factual), andexecution_data.resolved_modelholds the handleLLMSetting.modelcarried, which is frequently still an unresolved alias (@default-premium) because aliases resolve later, at inference time.by_modelreads from the usage records instead, so it survives alias resolution, deck defaults, and any fallback or retry that landed somewhere other than what was asked for. It is a list rather than a single value because a node routinely uses more than one model: aPipeLLM's text pass and its object-structuring pass resolve separately. Per-modelcostfollows the same rule as the node's —Noneiff no call to that model carried a rate table.
Fixed
-
A dry run of a
PipeConditionis now reproducible.pipe_dependencies()returns aset[str], and the dry-run arm walks every branch into the same working memory under the sameoutput_name— so the branch iterated last is the one whose stuff the enclosing pipe reports as the condition's output. Iterating the raw set made "last" a function of string-hash order, which varies per process, so the same bundle dry-run twice produced two different graphs: in a sequence routing to two branches that each name their own tail result, the sequence's output name flipped between them run to run. Nothing about a dry run should depend onPYTHONHASHSEED, and anything diffing graph specs across runs — a fixture corpus, a static-vs-dry parity check — saw phantom changes. The loop sorts now, which is what the two other order-sensitive readers of this set (signature_walk, theoutput_option_Nrenderer) already did. Live runs were never affected: they take exactly one branch. -
A
{concept, content}envelope carrying an empty list is now a value, not an error: a plural slot is never absent — when nothing is found it is the empty list — and the top-down shaper has always honoured that for a bare[](D2). The bottom-up factory did not: it infers a list's item type from the first item, so an empty list raised "Cannot create Stuff from empty list in content". The two spellings of one input therefore disagreed — a caller with no pictures could send[]and run, or send{"concept": "native.Image", "content": []}and fail — which made a plural input with nothing in it unrunnable for every caller that wraps its inputs. The envelope names the concept, so there was never anything to infer; it now builds an emptyListContent. Case 1.5 (a bare emptyListContent, no concept anywhere) still raises, and must: nothing there says what the items would have been. - Kernel step identity:
PipelexKernel.make_step_metadatanow acceptspipe_codeto name the pipe a step is running, mirroring what the interpreter stamps on its live and dry paths — so log correlation, usage accounting, and per-step labelling see a named step for kernel-driven runs. The key is omitted from the metadata update when not supplied, so a run-levelpipe_codeis never silently erased; the direct-call façade (llm_text/llm_object) stays deliberately anonymous.PipelexKernel.makealso accepts an injectablestep_id_sourcefor per-step ids, defaulting touuid4, so a kernel hosted inside a replay-based executor can supply a replay-safe source.
[v0.44.0] - 2026-08-13
Added
- Entry-point resolution affordances: Added
get_required_entry_pipe,get_optional_entry_pipe,get_required_entry_conceptandget_optional_entry_conceptto theinterpreter_huband library APIs. These are designed for human-supplied codes (CLI arguments, API payloads) and safely search across domains, unlike strict in-body references. Installed dependency packages are excluded from that search, so adding a dependency cannot make one of your own codes ambiguous. - Build-time crate qualification: Introduced a standalone
crate_qualificationpass that fully qualifies in-body references to their owner domain at library build time, ensuring the normalizer and the live library strictly agree on reference resolution. This supersedes v0.43.0's Cross-domain pipe refs in normalized crates: the two readers still agree, but on owner-domain qualification rather than on a crate-wide search.
Changed
- Strict in-body pipe reference resolution (Breaking): Bare in-body pipe references (sequence steps, parallel branches, condition outcomes,
batch_overtargets) now resolve only within their own domain. To reference a pipe in another domain, use the fully qualified form (e.g.sales.generate_tagline). This enforces[exports]visibility rules that were previously bypassable, and lets multiple domains safely reuse the same pipe code without library load failures. - Automatic search scope for concepts (Breaking): Replaced the
search_domain_codeslist parameter across the public API (PipelexMTHDSProtocol,pipeline_run_setup,prepare_pipe_job,InputShaper,WorkingMemoryFactory, etc.) with a singlesearch_scopestring derived automatically from the entry pipe. Every entry-lookup refusal — invalid string, miss, ambiguity — is now raised asConceptLibraryConceptNotFoundError, a class the input-shaping handlers catch. - Fully qualified validation error payloads (Breaking): The
missing_pipe_codefield on unresolved-dependency validation items now returns the fully qualified reference the compiler attempted to resolve (e.g.marketing.render_html) rather than the bare spelling the author wrote. - Ambiguity handling for bare code lookups: Bare pipe codes typed by users (via
pipelex run,show,which, or API requests) still search across all domains, but when two domains declare the same code the system now raises a clean ambiguity error listing the candidates instead of failing with a traceback. - Per-domain scoping in
pipelex fix: Rename collisions in the fix loop are now scoped per-domain rather than crate-wide, so two same-named pipes in different domains rename independently without bailing. PipeFunctransports qualified references:PipeFuncnow transports the fully qualifiedpipe_refto the executor instead of the bare code, preventing resolution failures in strict remote environments.- Expanded agent quality gate:
make agent-checknow includesdrift-checkandcheck-hub-layering, so open code↔docs drift contracts fail the local quality gate before reaching CI. The digest reads the git index, so stage changes for the gate to see them.
Fixed
- Verdict-neutral CLI lookup errors: The
buildandcodegenCLI commands no longer prepend a misleading "not found" verdict to stderr when a bare code is ambiguous; the prefix is now verdict-neutral. Timenative conversion inPipeCompose:TimeandTime[]native concepts are now correctly converted todatetime.timeandlist[datetime.time]when copied into native-typed fields viaPipeCompose.- Batch sub-pipe code derivation: Fixed
batch_oversynthesized pipes deriving their code from the qualified reference rather than the local code, which caused false-positive "namespace prefix" warnings on every batched run.
Removed
ConceptProviderAbstract.get_required_concept_from_concept_ref_or_code: Removed in favor of the newget_required_entry_conceptmethod to standardize entry-point concept resolution.
[v0.43.1] - 2026-08-12
Changed
PipeRunParams.batch_max_concurrencyis now required (Breaking): The field no longer defaults toNone, which previously made an omitted value indistinguishable from an explicitly authored unbounded concurrency (max_concurrency = "unbounded"). Omitting it now raises aValidationErrorearly—on both direct construction and payload decoding—to prevent accidental unbounded fan-outs.- Migration: Construct
PipeRunParamsviaPipeRunParamsFactory.make_run_params(), the single writer for run-scoped invariants (run_mode,pipe_stack_limit,batch_max_concurrency). Integration test fixtures now route through the factory via a newmake_mode_guarded_run_paramshelper. - Docs: Added a warning to
docs/under-the-hood/pipe-routing-and-execution.mdon using the factory method.
Fixed
- Empty lists in concept envelopes are now parsed as values, not errors: The bottom-up factory previously raised "Cannot create Stuff from empty list in content" when given an empty list inside a concept envelope (e.g.,
{"concept": "native.Image", "content": []}). Since the envelope names the concept, item-type inference is unnecessary, so it now correctly yields an emptyListContent. This aligns the bottom-up factory with the top-down shaper; callers can pass[]or{"concept": "...", "content": []}to represent an empty plural slot. A bare empty list with no concept still raises an error, as its type cannot be inferred. - Docs: Added a note to
docs/building-methods/pipes/provide-inputs.mdclarifying that an empty list represents an empty plural slot, whereas omitting the key fails with a "missing required inputs" error.
Security
uv.lockrefreshed onto patched dependency versions: A targeted lock refresh clears the open Dependabot alerts, with nopyproject.tomlconstraint changes — every declared range already permitted the patched version, so this was stale-lock drift rather than a constraint problem. Bumpsaiohttp,cryptography,datamodel-code-generator,pillow,pyasn1,pymdown-extensionsandsoupsieve. The alerts left open are held bytorch, reachable only through the optionaldoclingextra, whose upgrade would dragtorchvision,tritonand the NVIDIA CUDA stack along with it.
[v0.43.0] - 2026-08-12
Highlights
Every operator's execution semantics is now importable without the interpreter. The new public pipelex.kernel subpackage extracts what a step does — deck resolution, prompt assembly, generation, memory write-back — into plain functions covering every operator, all servable on a kernel-only boot with zero .mthds loaded. The layer below the interpreter takes the kernel's name, and plugins now declare which side they belong to through their entry-point group, so an interpreter-side plugin is never imported into a kernel-only process. Distributed execution sharpens alongside — a run_batch_branch router hook lets a backend isolate each batch branch, and the fan-out bound is frozen onto the run to close a replay hazard. Breaking changes are import-path moves, an error-class unification, a new required build_registrar parameter, a retired plugin entry-point group, and stricter ISO 8601 and failure classification.
Added
- Pipelex kernel (
pipelex.kernel): New public subpackage holding operator-execution semantics as importable functions — deck resolution, prompt assembly, generation, memory write-back — no longer reachable only through a fully booted interpreter with a loaded library. Every operator is covered (LLM and structuring, extraction, image generation, search, template composition, Python function calls), every call is servable on a kernel-only boot with zero.mthdsloaded, and a boot-contract test pins it. Two arms stay with the interpreter because their inputs are language or deployment artifacts rather than execution semantics:PipeCompose's construct mode andPipeFunc's pluggable executor. See Pipelex Kernel. PipelexKernelfacade with typed result envelopes: A facade class managing per-run state (JobMetadata,CogtRunParams) with ergonomic entry points such asllm_textandllm_object. Every kernel operation returns a strongly-typed envelope (LlmTextResult,LlmObjectResult,ExtractResult,ImgGenResult,SearchResult,ComposeResult,FuncResult) carrying the produced content, the updated memory, and tracer intermediates.pipelex.kernel.memory_opsstandardizes the memory boundary withshape_inputs,store_resultand typed extraction helpers.run_batch_branchrouter hook:PipeRouterProtocolgained a second dispatch entry point, used byPipeBatchfor each per-item fan-out branch — the only signal a router gets that a dispatch is a batch branch rather than an ordinary step. The default body delegates torun, so in-process execution is unchanged and every existing router keeps working untouched (routers subclassPipeRouterProtocol, so they inherit it); a distributed backend can override it to give each branch its own isolation. Documented on the "Pipe Routing and Execution" page and listed in the Orchestrator SPI.- Exceptions-aggregate layering gate: A new AST test bans kernel-layer packages from importing the all-errors aggregate
pipelex.exceptions, whose re-exports pull in the interpreter. It matches module-level, function-local and string-named imports alike, closing the blind spot neithercheck-hub-layeringnor the import-closure test can see. The fix for a violation is always to import the error class from the module that defines it. - Plugin validation errors:
PluginDeclaredInMultipleGroupsError,PluginLayerViolationErrorandRetiredPluginEntryPointGroupErrorgive loud, actionable startup failures for a misdeclared plugin, each with its own error reference page.
Changed
- Interpreter operators delegate to the kernel:
PipeLLM,PipeStructure,PipeExtract,PipeImgGen,PipeSearch,PipeComposeandPipeFuncnow delegate their core execution semantics to the kernel functions — a zero-behavior-change refactor.pipe_operators/llm/helpers.pyis gone, its structure-prompt derivation absorbed by the kernel. - "Runtime layer" is now "kernel layer" (Breaking): The layer below the method interpreter takes the name of the package that defines it.
pipelex.providers.builtinsexportsKERNEL_BUILTIN_PLUGINSandKERNEL_CORE_UNCONDITIONAL_PLUGIN_NAMES; the hub-layering guard declaresKERNEL_LAYER_PACKAGESand answersis_kernel_layer.runtime_bridgekeeps the word because it means the orchestration venue, not the layer;runtime_hubandruntime_bootstill carry the layer's former name and are renamed when the layer moves into its own package. - Plugins declare their layer through their entry-point group (Breaking): The single
pipelex.pluginsentry-point group is replaced bypipelex.plugins.kernelandpipelex.plugins.interpreter, andPLUGIN_API_VERSIONbumps to 4. A kernel-only boot queries the kernel group alone, so an interpreter-side plugin's module is never imported into a process that only runs kernel operations. The declaration is enforced at register time — a kernel-group plugin contributing an orchestrator, a bundle validator, a PipeFunc executor or an interpreter-layer hub slot fails loud, and declaring the same plugin under both groups is a startup error; the restriction is one-directional, so an interpreter-group plugin may still contribute kernel-tier capabilities.pipelex plugins listgains a Group column. This governs which plugins are discovered and what they may register, not what a plugin's module imports. build_registrargained a requiredentry_point_groupsparameter (Breaking): Every caller must now say which entry-point groups to read; there is no default, and a call at module scope fails at import rather than at first use. This affects any host embedding Pipelex that resolves plugin contributions itself — notably one readingget_http_error_mappers()to wire transport-fault handling. PassENTRY_POINT_GROUPSfrompipelex.interpreter_plugins.builtinsfor a full boot, orKERNEL_ENTRY_POINT_GROUPSfrompipelex.providers.builtinsfor a kernel-only one.- Prompt reference types moved (Breaking):
ImageReferenceandDocumentReference(with their*Kindenums) moved frompipelex.pipe_operatorstopipelex.kernel.prompt_references. They describe how a prompt resolves an image or document out of working memory — execution semantics, not a language artifact. - Prompt resolution errors unified (Breaking):
LLMPromptBlueprintValueErrorandImgGenPromptBlueprintValueErrorare replaced bypipelex.kernel.exceptions.PromptContentError, moved with the code that raises it. Both blueprints keep their fields, their validation and theirmake_*_promptsignatures. - Image-generation prompts are buildable without the interpreter:
pipelex.kernel.img_gen_prompt.assemble_img_gen_promptis the image counterpart of the kernel's LLM prompt assembly — image-reference resolution,[Image N]placeholder numbering and template rendering. Image generation was the one operator a kernel-only caller could not reach;ImgGenPromptBlueprintnow delegates to it, keeping its parse-and-validate role and themax_prompt_imageslimit. StructuredContentComposerno longer takespipe_run_params(Breaking): The constructor parameter and the attribute behind it were dead — stored, threaded down to nested composers, and never read — and the docstring's claim that they gated dry-run mode was stale. Dropping them leaves the composer importing nothing frompipelex.pipe_run.- Keyless-boot forced-DRY rule has one home:
pipelex.runtime_hub.resolve_run_mode_for_bootnow owns the coercion a keyless boot (needs_inference=False) applies to a requested run mode, and both run-params factories call it. The pipe tier is unchanged; the kernel tier'sPipelexKernel.makepreviously minted a LIVECogtRunParamson an offline boot and now gets the same forced DRY and the same warning as a pipe run. - Documentation: New Pipelex Kernel page, with Hub layering convention, the plugin pages and the under-the-hood execution pages updated for the kernel rename, the entry-point group split and the batch-branch hook.
Fixed
DateandTimecould not be produced by a livePipeLLM: APipeLLMwhoseoutputwasDate,Date[]orTimefailed every real run ondate_type/time_type— the structured-output layer validates the model's response in strict mode, where amode="before"validator forfeits pydantic's strict-JSON acceptance of ISO strings. Both natives now parse ISO 8601 themselves and return a realdate/time, so the strict and lax paths agree and the guards (no epoch-seconds, no datetime on a calendar date) still fire in both.Timegained the numeric-string guardDatealready had.- One ISO 8601 contract for the
DateandTimenatives (Breaking):pipelex.core.stuffs.iso_temporalnow owns what those natives accept — the extended forms only (2026-07-07,15:40:00+02:00) — and both the content models andStuffContentFactoryparse through it, so a model-generated value and an authored one are held to the same contract. It also rejects the end-of-day24:00form, which names the next day's midnight: Python 3.14 reads it as this day's midnight where 3.11-3.13 raise, so a value could silently land a full day early on part of the supported interpreter range. Breaking for authored inputs using a non-extended spelling:"15:40:00+0200","2026-07-07T154000","24:00:00"and surrounding whitespace are now refused. - Cross-domain pipe refs in normalized crates (Breaking): A bare pipe ref calling a pipe declared in another domain was normalized by prefixing the caller's domain, naming a pipe the crate does not hold — nothing raised on the way in, and the crate round-trip broke the method at run time with
PipeNotFoundError. Normalization now resolves a bare ref against the whole crate exactly as the live library does, and raisesCrateNormalizationErrorwhen the ref is ambiguous or matches no pipe. - Structureless concepts in the
python-structuresprojection (Breaking): A concept declaring only a description was projected as aStructuredContentsubclass while the runtime promotes the same declaration to a refinement of nativeText— so the interpreter's text-vs-object dispatch answered differently depending on which class it was handed, and the mismatch raised nothing. The projection now emitsTextContentas the base, matching the runtime. - Unimportable
python-structuresmodule for a concept refiningAnything:Anythingis the one native with no content class of its own, so a refining concept fell back to the rootStructuredContent— emitting the class name without registering its import, and the generated module raisedNameErroron the consumer's first import. The fallback now goes through the renderer that registers the import. - Format-stable import blocks in generated Python: A generated import line whose merged names crossed 88 columns was emitted flat, so the first
ruff formatin a consumer's tree rewrote it — changing the body bytes and makingpipelex codegen checkreport the file as hand-edited. Import blocks now pre-explode past the threshold like every other emitted construct. - Input shaping honors its injected concept provider (Breaking):
InputShaperconsulted its injectedConceptProviderAbstractfor concept resolution and compatibility, then resolved each value's structure class from the process-global class registry anyway — the ambient lookup the provider parameter exists to replace. It now resolves through the provider, so a caller supplying its own provider can shape inputs without a loaded library. Breaking on one failure path: a declared structure class that does not resolve raisesConceptStructureClassNotFoundErrorinstead of aStructureValidationErrorthat pointed the author at their input rather than at the concept. - A dry run now identifies the pipe it is running:
dry_run_pipestamps the running pipe onto theJobMetadatait hands down, exactly aslive_run_pipealready did — previously log correlation and the per-step labelling a distributed backend derives saw an anonymous step in DRY and a named one in LIVE. Telemetry stays live-only: the dry copy clearsotel_contextrather than inheriting a live span. - Explicit image-generation seed of
0respected: Building image-generation job params swallowed an explicitseedof0through a truthiness check and fell back to the configured default.0is a legal seed and now goes through; the"auto"literal still means the backend picks one. PipeBatchfan-out bound is frozen onto the run (Breaking): The[pipelex.pipeline_execution_config] max_concurrencysetting is now read once, when the run's parameters are built, and carried asPipeRunParams.batch_max_concurrencyinstead of being re-read inside everyPipeBatch— editing the config mid-run no longer reshapes in-flight batches. This closes a durable-execution hazard: the bound decides where a backend's task boundaries fall between branch dispatches, so a worker redeploy mid-run could make a replay group its dispatches differently from the recorded history. Breaking only for code that mutatedmax_concurrencymid-run and expected the change to take effect.
Removed
- Retired the pre-split
pipelex.pluginsentry-point group (Breaking): A plugin still published under it now fails the boot with aRetiredPluginEntryPointGroupErrornaming the plugin and the group it should move to, rather than being silently undiscovered. Republish underpipelex.plugins.kernelorpipelex.plugins.interpreter.
[v0.42.0] - 2026-08-01
Added
- New classified error types: Introduced specific error classes for clearer debugging and handling, each with dedicated documentation pages:
OutputStructureSchemaError: raised when an output structure class cannot produce the JSON schema required by a structured leaf.GatewaySearchEmptyResultError&LinkupSearchEmptyResultError: raised when a search returns no object payload to fill the output structure (pointing to the query rather than the model).LinkupError(base class) &LinkupSearchResponseError: raised when the Linkup API answers a structured search with an invalid payload shape.
Changed
- Strict
PipeFuncname collisions (Breaking): Registering two different functions under the samePipeFuncname now raises aFuncRegistryError(naming both origins) instead of silently overwriting the previous registration. Re-registering the exact same function object remains a safe no-op. Docs updated to explain the new flat namespace collision rules. - Stricter
pipelex validatefor structured search (Breaking):pipelex validatenow rejects structured-search output classes that Pydantic cannot describe as a JSON Schema (e.g.arbitrary_types_allowedfields), failing early with anOutputStructureSchemaErrorinstead of crashing mid-run inside the worker. - Dry-run object mocks built from real classes (Breaking): Dry-run leaf mocks for
PipeLLMandPipeSearchstructured outputs are now built directly from your output class rather than a schema rebuild. Constraints previously erased are now enforced at mock-build time, failing loudly with aDryRunMockBuildErrorif Polyfactory cannot satisfy them. Related "Under the Hood" docs for dry-run mock generation updated. - Consolidated leaf result conversion: Centralized the logic for converting a leaf result into the caller's class into a single shared helper (
pipelex.cogt.content_generation.object_revalidation), ensuring consistent behavior across in-process and distributed-execution boundaries. Related "Under the Hood" docs for distributed content generation updated.
Fixed
- Preserved class invariants in structured output & web search: In-process structured generation and structured web searches no longer lose your class's invariants (custom
@field_validators,json_schema_extrahints, and docstrings). The caller's live class is now passed directly down to the leaf and used as-is, preventing weaker contracts from reaching the provider. - Shadowed field names in worker-boundary round trips: Output fields named
json,copy,schema, orconstructno longer break when crossing a worker boundary. The shared conversion now serializes withby_alias=True, ensuring keys match the published schema and preventing falseField requiredvalidation errors. Our distributed-execution plugin picks this fix up with its next release, once it adopts the shared helper. - Structured web search envelope unwrapping: Fixed an issue where Linkup and Gateway search backends returned a
{data, sources}envelope that no output class could accept, making structured search unusable. The direct backend now requests the payload alone, and both backends recognize and unwrap the envelope if returned. - Accurate cost reporting for failed searches: A structured search whose response shape was rejected by the backend previously vanished from the run's cost report because it failed after the provider had answered. Completion and reporting now run in a
finallyblock, ensuring billed usage is always recorded. - Malformed Gateway responses properly classified: A Gateway relay answering with non-JSON content previously raised a bare
JSONDecodeError; it is now caught and surfaced as a classifiedGatewaySearchResponseError.
[v0.41.0] - 2026-07-30
Highlights
Importing the Pipelex runtime now loads zero interpreter modules, down from 50 — you can embed the inference engine without loading a line of the MTHDS interpreter. pipelex.hub split into pipelex.runtime_hub and pipelex.interpreter_hub, the boot sequence gained a RuntimeBoot layer seam, and the vendor adapters, the MTHDS parser and the Pipe machinery each moved to the package that owns them. The boundary is enforced in CI (make check-hub-layering) and pinned by a subprocess import-closure test. This is a large breaking cycle: many import paths moved, and PipelexInterpreterError → MthdsParserError changes a wire-visible error_type. See Hub layering convention.
Added
- Claude 5 Support:
claude-5-sonnetis now served by the Pipelex gateway. - Error Identity Snapshotting: Added a
generate-error-identity(gei) CLI command and Make target that commits a snapshot of everyPipelexErrorsubclass's(error_type, title, type_uri)triple, so renaming an error class — which changes theerror_typestring consumers branch on — surfaces as a reviewable diff instead of silently breaking them. Documented in Error Model. - Hub Layering CI Enforcement: Added
make check-hub-layering(chl), now a required CI status check (lint-hub-layering), to enforce the one-way boundary between the runtime and interpreter layers. The rule is transitive — it reports any runtime-layer module that reaches the interpreter hub, naming the shortest import chain and the line of its first hop. - Concept Registry Boundary: A golden-set AST test over
pipelex/core/concepts/**pins which modules may read the process-global class registry — only the materialization write side may. - Vendor-Neutrality Pinning: Added strict tests pinning
pipelex.cogt's third-party import roots and its exactcogt → pipelex.providersedges, so a new inference-provider SDK import or a new inversion shows up as a diff. Both packages are runtime-layer, so no other gate can see these edges. - Registry Sort Enforcement: The keyword-only guard now enforces the sort order of the
subject_grants.tomlregistry (unsorted-grantviolation) to prevent silent breakage during bulk rewrites. See Keyword-only arguments. - Test Duration Gating: Added a test ensuring
.test_durationscontains no dead paths after bulk file moves, keeping CI shard balancing accurate. - Registration Surface Docs: New Registration surface page documents every place a new pipe kind has to be registered.
Changed
- The Hub Split:
pipelex.hubhas been split into two distinct lifecycles to keep the runtime from loading the method interpreter:pipelex.runtime_hub(process-scoped infrastructure: config, console, secrets, storage, telemetry, model deck, inference workers) andpipelex.interpreter_hub(library-scoped method machinery: library manager, pipe router, pipeline manager).get_pipelex_hubsplits intoget_runtime_hub/get_interpreter_hub. There is no alias shim — a stale import fails loudly rather than silently resolving to the wrong layer. (Breaking) - Composition Root:
Pipelexis now the interpreter-layer boot composed on top of the newRuntimeBootlayer seam (pipelex/runtime_boot.py), which stands up the whole runtime while loading zero interpreter modules. Inheritance preserves every attribute address, so no consumer changes;Pipelex.__init__takes keyword-onlyconfig_dirin place of the inert positionalconfig_dir_path, and the deadis_pipelex_service_enabledattribute is gone. (Breaking) - Plugin & Provider Separation: Built-in vendor adapters (OpenAI, Anthropic, etc.) moved from
pipelex.pluginstopipelex.providers; interpreter-touching built-in plugins (direct,pipe_func) moved topipelex.interpreter_plugins;pipelex.pluginsnow strictly houses the plugin mechanism (contracts, registrars). The plugin contract itself is untouched — an out-of-tree plugin still importspipelex.plugins.contract/registrarand is still discovered through thepipelex.pluginsentry-point group.build_registrarnow requiresbuiltin_plugins=andcore_unconditional_plugin_names=instead of importing them, which is what keepspipelex.pluginsimportable without the interpreter. (Breaking) - Core Restructuring & MTHDS Parser: The MTHDS parser moved out of
core/intopipelex.mthds_parsing, withPipelexInterpreterrenamed toMthdsParserandPipelexInterpreterErrorrenamed toMthdsParserError. The rename is wire-visible: the class name is theerror_typeclients branch on, and itstype_urimoves toerrors/mthds-parser-error/(a docs redirect keeps the retired URI resolving; a consumer switching on the string does not get one). Core's Pipe machinery (abstracts, factories, blueprint, validation, rendering) moved topipelex.pipe_machinery, and the pipe-kind registration manifest moved topipelex.pipe_machinery.registry_models, leavingCoreRegistryModelswith the value model only. (Breaking) - Concept Purity:
Conceptis now pure data. Class resolution has been removed from the wire model and delegated toConceptProviderAbstract.get_structure_class(concept=...), andConcept.are_concept_compatiblebecomes the pureConcept.are_compatible_by_declaration, composed byConceptLibrary.is_compatible. Three defects close with it: cross-packagerefinesaliases now resolve at five call sites that comparedConceptvalues directly, an unresolvable structure class now raisesConceptStructureClassNotFoundErrorinstead of silently answeringFalse, andnative.Anythingis answered as a declared property rather than raised. (Breaking) - Injected Providers in
core/: Core's data model resolves concepts through an injected provider instead of reaching into the hub —StuffFactory,InputShaper,WorkingMemoryFactory,InputStuffSpecsFactoryandStuffSpecFactoryall take a requiredconcept_provider. The new read-side contractConceptProviderAbstractowns that surface, whichConceptLibraryAbstractextends, keeping library management in the interpreter layer. (Breaking) - Graph Rendering Split:
graph_rendering's bundle-driven half is nowpipelex.pipeline.bundle_graph_rendering(generate_graph_for_bundle/generate_view_for_bundle);GraphFormatandrender_graph_from_specstay put. The split is along "do I need a loaded method?" — producing a graph spec from a bundle requires dry-running it, rendering an existing spec does not.pipelex.graph,pipelex.tracing,pipelex.observer,pipelex.errorsandpipelex.test_extrasjoin the declared runtime layer. (Breaking) - Type Relocations: Leaf models moved to the packages that own them, preventing import leaks:
SpecialPipelineId/JobMetadata/JobCategory/UnitJobIdtopipelex.system.job_metadata,JobMetadataErrortopipelex.system.exceptions,PipeRunModetopipelex.system.pipe_run_mode,PipeRunParamKeytopipelex.system.pipe_run_param_key,PipeRunErrortopipelex.core.pipes.exceptions,TraceContexttopipelex.system.trace_context,DataInclusionConfigtopipelex.system.data_inclusion_config,TextFormat/TemplatingStyle/TagStyletopipelex.tools.templating.*,TemplateCategorytopipelex.tools.jinja2.template_category, and the resolved-field layer topipelex.core.concepts.resolved_fields. The two pipe renderers regrouped intopipelex.pipe_machinery.rendering. The TOML config shape is unchanged throughout — only the Python homes moved. (Breaking) - LLM Prompts:
LLMPromptBlueprintno longer has atemplating_stylefield; the style is now passed as a run-scoped parameter tomake_llm_prompt.TemplateBlueprint.templating_stylestays and now correctly takes precedence over the run-derived style. (Breaking) - Image Generation: Geometry mappings moved inward from the vendor adapters to
pipelex/cogt/img_gen/—GoogleImgGenFactory/OpenAIImgGenFactoryare nowImgGenGeminiMapping/ImgGenGptMapping, with wire-literal aliases following the model family rather than the vendor. They were never vendor adapters: they map acogt-owned taxonomy onto provider wire values, and no family is served by a single adapter. (Breaking)
Fixed
- Lint-Clean Codegen Artifacts: Generated Python and TypeScript artifacts are now lint-clean by construction, emitting modern typing (builtin generics,
X | None), double-quotedLiteralmembers, isort-grouped imports, idiomatic docstrings, pre-wrapped long lines, blank JSDoc lines carrying no trailing space, and a header-only file when there is nothing to emit. This prevents consumer formatters (Ruff/Prettier) from rewriting bytes and invalidating the[hand-edited]codegen stamp. Emitted bytes change for all three targets, so a committed projection reports drift until it is regenerated, and the lint exclusion on generated paths is no longer needed. Documented in Codegen Projections. (Breaking) - Prompt Rendering Determinism: Rendering an LLM prompt no longer mutates the library-held pipe object; a pipe's serialized form is now identical before and after runs, and in-process deck or config changes take effect immediately instead of being shadowed by the first run's cached value.
- Sandbox PipeFunc Structures: A custom PipeFunc returning a non-native concept structure failed in the sandbox with an opaque "Function not found in registry" error — the structure class neither travelled nor was generated, so its import raised
ModuleNotFoundError, which the registration path silently swallowed.structures.pyis now regenerated from the crate before customer code is registered, and a failed import logs its real cause instead of being swallowed. - Boot & Teardown Resilience: A partway-failing teardown no longer permanently wedges the process — the whole un-poisoning set now runs in a
finally, not just the singleton de-registration, since anything left behind alongside it turns a loud failure into a silent one. A failed boot no longer leaves the telemetry manager singleton behind or leaks a plugin runtime; plugin teardown is now best-effort per callback so one failing plugin cannot strand another's resources; an injected telemetry manager that raises no longer aborts the release suite; a secondmake()of an already-booted class is now explicitly refused; and the singleton lookup iterates a copy rather than a live process-global dict that grows during a run.teardown()no longer propagates a plugin callback's exception. (Breaking) - Layering Leaks: Built-in vendor adapters were pulling the entire interpreter into the runtime layer by importing
MissingDependencyErrorfrom the all-errors aggregatepipelex.exceptions; they now import directly frompipelex.system.exceptions. Both gates were blind to it, which is what motivated the guard's new transitive rule. - Runtime Structure Generator Typing: The runtime
StructureGeneratoremittedOptional[X]/List/Dictwhere the codegen emitters emit modern typing, despite both rendering the same resolved-type tree. Both now agree, with unions folded inside quoted forward references so the annotation stays a single deferred expression. - Config Initialization:
config_dirpassed at boot is no longer silently ignored and now correctly reachessetup_config. - Documentation & CLI: Fixed
pipelex build --helpadvertising a non-existenttypescriptformat (it is a downstream use of--format schema); correctedhub.set_observer()documentation, which advertised an inert API, to reflect thePipelex.make(observers=...)mechanism; rewrote the validation feature page, which promised an inter-step concept check that does not exist and cannot exist given working-memory resolution; and removed a duplicateset_library_managercall inPipelex.setup.
[v0.40.0] - 2026-07-19
Added
- Durable runs now persist token usage: the delivery executor writes a
tokens_usages.jsonresult artifact ({"tokens_usages": [...], "usage_assembly_error": null}) alongsideworking_memory.json, themain_stuff.*renders, and the graph outputs. The records use the client-facingTokensUsageRecordwire shape — the same shape the/executeresponse carries onpipe_output.tokens_usages— so a client polling a durable run's result files reads costs the same way a sync/executecaller does. The artifact is written unconditionally: a run with usage assembly off yields explicit nulls, distinguishable from a pre-artifact run (file absent). TokensUsageRecord— a deliberate client wire shape for token usage (pipelex/reporting/usage_records.py). One flat record per inference call:model_type,inference_model_name,inference_model_id,pipe_code,job_category,unit_job_id,nb_tokens_by_category(raw provider counts), computed USDcost(nullwhen the model has no rate table), and ISOstarted_at/completed_at. Enum-ish fields are open string sets on the wire, so runtime enum churn is non-breaking for clients.compute_tokens_usage_costreuses the existing cost engine (model_cost_per_token+ the cached/non-cached split), so the wirecostequals the CLI cost table's total for the same call. Documented in TokensUsage Wire Records.
Changed
- Breaking:
tokens_usagesrecords on client surfaces are nowTokensUsageRecordwire records, not dumps of the internal usage models. Both the/executeresponse (pipe_output.tokens_usages, via the newapply_tokens_usage_wire_shapehelper API servers apply to the response dump) and the durabletokens_usages.jsonartifact stop leaking runtime internals:job_metadata(with itsuser_id,session_id,request_id,pipe_run_id,otel_context,trace_context,content_generation_job_id,pipeline_run_id) and theunit_costsrate table no longer cross the wire — the keptJobMetadatafields (pipe_code, job-kind enums, timing) are flattened onto the record andcostis computed server-side. Null semantics unchanged (null= assembly off,[]= no inference). Internal crossings (runtime-bridge payloads, Temporal transport, the usage-event telemetry stream) keep full-fidelity records, untouched.
Fixed
- Validation errors stay domain-qualified: every raise site that names a
pipe_codenow also carriesdomain_code. The presentation chain (VS Code extension + mthds-ui) identifies pipes by full pipe ref (domain_code.pipe_code); a handful ofPipeValidationErrorraise sites (PipeLLM static input checks, PipeStructure input-mismatch, the pipe sorter's circular-dependency error) omitteddomain_code, degrading node decorations and click-to-navigate for those errors in multi-domain bundles. The sorter gains adomain_codeparameter threaded from the bundle spec's domain. - Hosted PipeFunc dry runs now honor output multiplicity. In sandbox-hosted mode a PipeFunc with a multiplicity output (e.g.
Foo[]) mocked a single scalar item because the function annotation is unavailable in-process; downstream pipes expecting a list (e.g.batch_over) then failed the dry-run//validateof a perfectly valid method. The mock now takes its shape from the declared output multiplicity. get_optional_config()honors its non-raising contract before any hub exists. It used to raiseRuntimeError: PipelexHub is not initializedwhen no hub had been created at all, which broke the documented safe pre-boot path — e.g. constructing aPipeFuncin an isolated unit test crashed instead of defaulting to thedirectexecution mode.- PipeFunc transport hardening: the wire contract now rejects a non-finite
timeout_seconds(float("inf")used to disable the runaway-code guard entirely), and the transported executor removes its materialized source workdir on every path (success, timeout, failure) instead of leaking a temp directory per invocation, pruning thesys.pathentries it inserted for that workdir in the same cleanup so dead entries can't accumulate (or be hijacked by a recreated path) in a reused process. - Error reference pages generated for the PipeFunc transport errors (
PipeFuncTransportError,PipeFuncExecutionError,DuplicatePipeFuncExecutorError,UnknownPipeFuncExecutionModeError) — theirtype_uris previously dereferenced to 404s.
[v0.39.2] - 2026-07-17
Added
LibraryManagerAbstract.is_crate_loaded(*, library_id, fingerprint)— public query over the per-library crate-fingerprint bookkeeping that already backsload_from_crate's idempotency. ATrueanswer means the library's ClassRegistry holds the crate's dynamic classes, so callers can hydrate within an existing scope instead of opening a fresh one — preserving dynamic-class identity with instances that scope already produced. This is the core half of the fix for the PipeParallel same-concept combine failing over Temporal withexpected X, got Xpydantic model-type errors.
Fixed
- PipeCompose construct: whole-stuff copies into native fields now convert in every promised case.
{ from = "..." }referencing a whole native stuff used to hand the content wrapper (TextContent,ListContent, ...) to the composed field in several cases, failing the dry-run runnable gate on correct methods. Three gaps fixed: optional (non-required) fields never converted (theOptional[X]annotation defeated the type detection), list-of-text targets dumped each item as a{"text": ...}dict instead of extracting the string, and scalar wrappers other thanTextwere not handled at all. The conversion matrix now coversText → text,Number → number,YesNo → boolean, andDate → date, for both required and optional target fields, scalar and list-item positions — including nullable list items (list[str | None]), which normalize like their non-nullable counterparts. Unconvertible list items now raise a clearStructuredContentComposerTypeErrorinstead of surfacing as a cryptic pydantic error downstream. One fidelity guard: copying aDatethat carries a time of day into a baredatefield raises instead of silently dropping the time and its UTC offset (breaking: this case previously truncated silently).
[v0.39.1] - 2026-07-15
Changed
- Bumped the
pipelex-tools-pyruntime dependency to>=0.1.3and thepipelex-toolsdev dependency to>=0.7.2, picking up the latest MTHDS schema updates forplxtformatting and linting.
[v0.39.0] - 2026-07-14
Highlights
- From resolved library to typed clients:
pipelex resolveandpipelex codegen—pipelex resolvedistills a bundle's whole closure into the normalized library crate — a flat, fully-qualified, fingerprinted snapshot — andpipelex codegen typesprojects it into TypeScript (ts-zod) or Python (python-pydantic,python-structures) artifacts. A trust chain of stamped headers and acodegen.lockmakes drift detectable offline:pipelex codegen checkverifies artifacts by pure hashing — no engine, no network, no API key. - Validation now fixes what it finds — When an error has a deterministic safe fix, every surface says so: a 💡 suggested-fix line in the CLI output, a structured
suggested_fixobject in agent and API payloads. The newpipelex fix bundle/pipelex-agent fix bundlecommands apply those fixes in place through a shared fix loop, with--diffto preview the changes without writing anything. - Smart Inputs: the signature decides the shape — Inputs are now interpreted top-down against the pipe's declared input signature instead of bottom-up from their shape alone: a bare string becomes the declared concept (never silently degrading to generic
Text), lists shape element-wise, bare URLs/paths resolve forImage/Documentinputs, and generated input templates default to the light bare-value form. - New native concepts:
Date,Time, andYesNo— Built-in concepts for calendar dates (preserving source precision — no fabricated times of day), times of day, and boolean judgments, plus atimestructure-field type completing the temporal triple. Every native now materializes from its pinned normative definition, so crate fingerprints byte-agree across implementations.
Added
pipelex resolveand thepipelex codegenfamily:pipelex resolveassembles a bundle closure and emits the normalized library crate — a flat, fully-qualified, natives-expanded, self-contained, fingerprinted snapshot of a resolved library — as JSON or TOML. On top of the crate:pipelex codegen types --target ts-zod|python-pydantic|python-structuresprojects the crate's concept set into typed artifacts. Thets-zodtarget emits a puretypes.ts(imports onlyzod, with wire-native snake_case keys so a schema validates a wire payload directly) plus abinder.tsexposing a typedparse/serializepair per concept;python-pydanticemits self-containedBaseModels;python-structuresemits runtimeStructuredContentclasses.pipelex codegen inputs --pipe <ref>renders a runnable inputs template for a pipe (defaulting to the closure'smain_pipe).-
Generated files are never edited — each carries an
AUTOGENERATEDheader pointing at the sibling extension-file mechanism. -
Codegen trust chain (stamps ·
codegen.lock· offlinecodegen check): Every filepipelex codegen typeswrites now carries a self-describing stamp header (source crate fingerprint, engine version, projection, options, and a content hash of the body below it), and the artifact set is recorded in a siblingcodegen.lock.pipelex codegen check [DIR]verifies generated artifacts are current offline — pure hashing, no engine, network, or API key — reporting drift by category (missing, modified, hand-edited, orphan) with the verdict on the exit code (0current ·1drift ·2no lock). Emission is idempotent (write-if-changed — unchanged output leaves files untouched) and prunes previously-generated files that drop out of the set, so a deleted concept never lingers as a stale generated file. Input templates (codegen inputs) are deliberately not stamped or locked — they are user-editable scaffolds, not tracked generated code. - Agent-CLI codegen mirror (
pipelex-agent codegen types|check): The codegen family is now available to machine consumers through the agent CLI's two-stream envelopes (--format markdown|jsonsuccess stream,--error-formaterror stream).codegen typesruns the same engine and write-if-changed emission as the bare CLI;codegen checkpresents the offline drift verdict structurally — drift is aCodegenDriftErrorenvelope enumeratingdrifts[]by category (exit1), a missing or unreadablecodegen.lockis a no-verdict error (exit2). With the pipelex runner,mthds-agent codegen …forwards to these commands. - Host-facing resolve/codegen engine cores (what the HTTP routes ride):
pipelex.pipeline.resolve_bundle.resolve_crate_from_contentsresolves in-memory MTHDS contents into the normalized crate — same engine, sameValidateBundleErrorverdict vocabulary, and the same loaded-on-success library contract asvalidate_bundle— so a host (thepipelex-api/v1/resolve+/v1/codegenroutes) serves resolution without touching the filesystem.pipelex.codegen.emission.build_stamped_projectionis the new pure stamping core (stamped files +codegen.lockcontent, no disk writes) thatwrite_stamped_projectionnow rides, so artifacts served over HTTP are byte-identical to locally written ones and pass the offlinecodegen checkverbatim. The shared bundle-loading error cascadetranslate_to_validate_bundle_erroris now public (renamed from its underscore-private form) since it gained a third entry point. - Agent bundle fixer: Added
pipelex-agent fix bundle, a deterministic in-place fixer over the shared validation fix loop. The command supports--format/--error-format, repeatable--select/--ignorefix-rule filters,--allow-signatures,--max-iterations, and the same bundle-file or directory resolution aspipelex-agent validate bundle. Successful fix results reportpending_signaturesandis_runnable; by default the command exits non-zero when a fixed bundle remains valid-but-not-runnable. - Human bundle fixer: Added
pipelex fix bundle, the human counterpart of the agent fixer over the same fix loop, with Rich-rendered output naming every change made (fix descriptions, files written, iteration count), the same--select/--ignore/--allow-signatures/--max-iterationsoptions, and the same bundle-file or directory resolution aspipelex validate bundle. A new--diffflag previews the changes as a unified diff without writing anything — the real fix loop runs against a temp-copy sandbox, and exit codes keep the verdict semantics so--diffanswers "would it converge?". - Suggested fixes surfaced in
pipelex validate: When a validation error has a deterministic safe fix, the human error output now shows a per-error💡 Suggested fix:line and ends with an actionable footer naming the exactpipelex fix bundlecommand (echoing the invocation's-Ldirs) in place of the generic tip. suggested_fixfield in/validatepayloads: Each validation error that has a deterministic safe fix now carries an additive structuredsuggested_fixobject —fix_code(the kebab-case rule id),description(the author-facing 💡 text),safety(safe/unsafe), optionalsourcefile, andops(the semantic patch operations) — on everyvalidation_errors[]item, both in thepipelex-agent validate/fixJSON envelopes and in the API/validateresponse body. This is the machine-actionable counterpart of the 💡 line, letting a software consumer apply the fix directly from the structured payload.- Smart Inputs (signature-driven input shaping): Inputs are now interpreted top-down against the pipe's declared input signature rather than bottom-up from their shape alone.
- Bare values (strings, numbers, booleans, dicts, lists) are automatically shaped into the declared concept (e.g., a bare string becomes a
legal.Question, not a genericnative.Text). - Lists shape element-wise into the declared item concept; single values auto-wrap into lists where required, and empty lists are now legal.
- Bare URLs/paths for
ImageorDocumentinputs are resolved relative to the inputs file's directory. -
Declared structured lists (e.g.,
Person[]) accept a direct path to a.csvfile by signature, without requiring an envelope. -
Comprehensive input error handling: A suite of typed errors for input shaping failures (
WrongScalarKindError,ListWhereSingularError,MultiplicityCountMismatchError,StructureValidationError,UnknownInputNameError) that provide clear hints and render the expected JSON shape. - CLI
--explicitflag: Added--explicittopipelex build inputsandpipelex-agent inputsto generate the legacy{"concept", "content"}envelope templates. - Native
YesNoconcept: Built-in concept (backed byYesNoContent) for boolean judgments. Requires a strict boolean, renders asyes/noin downstream prompts, and can be read viapipe_output.main_stuff_as_yes_no.yes_no. - Native
Dateconcept: Built-in concept (backed byDateContent) for calendar dates. Requires adatetime.dateand accepts an optionaldatetime.time. Preserves source precision (never forcing the LLM to fabricate a time of day) and retains UTC offsets verbatim. - Native
Timeconcept: Built-in concept (backed byTimeContent) for a time of day, optionally with a UTC offset — never attached to an invented date. A top-level TOML time-of-day literal in an inputs file now maps toTime(Breaking:Timejoins the reserved native concept codes, and the former bare-time rejection —InputsTimeOnlyNotSupportedError— is gone). - Structure-field
type = "time": The concept structure language gains atimefield type (a time of day, optionally with a UTC offset), completing the temporal triple besidedateanddatetime. Generates adatetime.timefield; emitted by every codegen target. - Pinned normative native definitions: Native concepts now materialize into the normalized crate by lookup into the standard's pinned per-version definitions (
pipelex/core/concepts/native/pinned_blueprints.py, mirroring the mthds spec's newnative-concepts.md), never by reflection over the runtime content classes — so crate fingerprints over materialized natives byte-agree across implementations. The old reflection is retained solely as a consistency test proving each runtime content class still matches its pinned blueprint. - Subject-grants registry — the keyword-only convention hardened: A positional first parameter ("subject") in
pipelex/source is now legal only under an explicit grant recorded insubject_grants.tomlat the repo root, carrying the subject's param name and an honest, def-specific review rationale — every grant in the registry has been individually reviewed against the grant rubric.pipelex-dev subject-grantis the sole registry writer; thecheck-keyword-onlyAST guard enforces the registry symmetrically (an ungranted subject fails, and so does a stale grant whose def was renamed, moved, demoted, or newly carved out) inmake check, CI, and the edit-time hook. Subjects typedbool/int/floatare banned outright — no grant can cover them. The registry schema is strict: exactlyparam+rationale, unknown keys fail the check. Seedocs/contribute/keyword-only-arguments.md. - Drift contracts (
pipelex-dev drift plan|check|ack): Deterministic review obligations between code and docs. A rootdrift.tomldeclares contracts ("when these trigger files change, these review targets must be re-examined and the examination recorded"); committed ack files under.drift/acks/record each fulfilled review with a digest, reviewer, and rationale. A contract is fulfilled iff its stored ack digest equals the digest recomputed from the current tree (staged blob OIDs from the git index — no base ref, no timestamps), so trigger edits, matched-file adds/deletes, and contract-definition edits all reopen it. make drift-checkgates themake checkaggregate and runs as an advisory CI job;make drift-planprints per-contract Markdown packets with exact per-file changes since the last ack.make drift-ack CONTRACT=… RATIONALE="…"runs the contract's verify commands, then records the review and stages the ack file it writes, so the local gate checks the same index the commit is built from. If staging that ack fails, the ack file is removed rather than left unstaged, so a later check reports a missing ack instead of a false green.- For a contract with verify commands, a trigger file left unstaged-modified or untracked is a hard ack error (the verify run would certify content the digest does not cover) — checked both before and after the verify commands, so a trigger the commands themselves format or generate is caught too. For other contracts it stays a warning.
- Seeded with contracts for the config docs, the CLI docs, and the keyword-only convention spec; workflow documented in
docs/contribute/drift-contracts.md.
Changed
- Keyword-only demotions across the public API (Breaking): The subject-grant review demoted every positional first parameter that failed the grant rubric to keyword-only, and external code calling these surfaces positionally (or subclassing them with positional signatures) breaks. Notable demoted families: the
PipeAbstractrun family (run_pipe,dry_run_pipe,validate_before_run, … —job_metadata/visited_pipesnow keyword-only),StuffContent.rendered_for_prompt/rendered_for_template_async(text_format) and theTextFormatRenderable/ImageRenderableprotocols with all their implementations, theLibraryManagerAbstractload family (library_id),ensure_pipelex_booted(config_overrides),pipeline_run_setup(execution_config),ConceptLibrary.is_compatible(tested_concept),GraphTracerProtocol.setup(graph_id), thePromptImageFactory/PromptDocumentFactorycontent factories (uri), the model-manager/backend-library setup+load (secrets_provider), the reporting protocol'sset_event_log/clear_event_log(context_key), and assorted tools helpers (pluralize/count_with_noun,LogLevel.from_int,set_dry_run_forced, …). pipelex build structuresis now an alias ofcodegen types --target python-structures(Breaking): The legacy always-qualified per-file generator is deleted. The alias resolves the target's whole directory closure and emits one stampedstructures.pymodule plus acodegen.lock(bare-when-unique class names, declared imprecision instead of the old silentTextContentguess for structureless concepts). The--forceflag is gone (emission is write-if-changed), a.mthdsfile target now resolves its parent directory's closure, and concepts-only loading of invalid-pipe bundles is no longer supported (the crate requires a valid library). The concepts-only loading API (load_concepts_only,load_concepts_only_from_directory, and the library-manager methods behind them) is removed with it.pipelex build runneremits its structures through the codegen engine (Breaking): The scaffoldedstructures/directory now contains the stamped single-module projection (structures.py+codegen.lock) instead of per-concept files, and the generated runner script imports and instantiates the emitted class spellings (from structures.structures import Invoice) — fixing dropped imports for nested custom concepts along the way. A user class backing astructure = "<ClassName>"concept is imported from its real module.- Human validation error rendering routed through the shared structured items:
pipelex validate's error details now render from the sameValidationErrorItems the agent CLI and API emit, which surfaces information the old renderer silently dropped: pipe-factory errors (e.g. an unknown concept) get their own section, a parse-level failure (e.g. a TOML syntax error) now shows its message instead of no detail at all, and pipe validation errors name their source file. The dry-run message remains visible alongside categorized errors. - Validation error messages now speak MTHDS author syntax: Pipe validation errors state the problem and the fix the way an author writes them — a sequence output-multiplicity mismatch now reads "declares its output as 'StoryIdea', but its last step yields 'StoryIdea[]'. Update the sequence's output to 'StoryIdea[]'", and an input-type mismatch reads "input 'dish' is declared as 'Number' but its step needs 'Text'. Update the input to 'Text'" — instead of leaking Python internals (
multiplicity=None,Concept(...)/PresenceMarkerreprs). The misleading(required: […], provided: …)suffix is suppressed for multiplicity errors (where both sides are identical by definition) and rendered as joined author-syntax refs — never Python list-repr brackets — elsewhere. - Agent
validate/fixoutput renders prose, not a JSON dump:pipelex-agent validate bundleandpipelex-agent fix bundlemarkdown output now renders validation errors as category-grouped prose (a humanized title, identity fields, the message, and a💡 Suggested fix:line) followed by a fix-aware footer naming the exactpipelex-agent fix bundlecommand when a safe fix exists — mirroring the human panel. The machine-facing JSON stream (--format json) is unchanged, so software consumers keep branching on the same structured fields. - Clean validation summary message on every surface: The top-level error
message(agent JSON, the API/validatebody) is now a clean, author-facing summary derived from the structured errors, instead of the raw pydantic dump (Validation error(s): … Value errors: '<field>': Value error, …). The API's opt-in invalid-verdictrendered_markdownnow carries the same per-error💡 Suggested fixprose as the CLI. A still-invalid fix that stops with no progress now names the re-proposed fix in author terms rather than dumping an internal fingerprint string. - Input templates default to "light" shape (Breaking):
pipelex build inputsandpipelex-agent inputsnow generate light templates by default, outputting the bare values expected by the signature rather than the verbose{"concept", "content"}envelopes. TOML templates include the declared concept as a# concept: ...comment. - Structure-field
type = "date"behavior (Breaking): Adatefield now generates adatetime.date(JSON schemaformat: date), preventing LLMs from hallucinating times for calendar dates. A newtype = "datetime"field handles full timestamps (format: date-time), and default-value validation now rejects adatetimedefault on adatefield. - Strict input validation (Breaking): Undeclared input names are now a hard error (previously silently ignored), and top-level
nullvalues are rejected — absence must be expressed by omitting the key. Known limitation: this also rejects the extra stuffs carried by apipelex-agent run --with-memoryenvelope, so piping one run's full working memory into a downstream pipe that declares only a subset of it now fails withUnknownInputNameError. Until envelope-derived inputs get their own leniency, narrow the envelope to the declared inputs before piping, or pass the inputs explicitly with--inputs. - Explicit envelope validation (Breaking): The
{"concept", "content"}escape hatch is now compatibility-checked against the pipe's declared signature; incompatible concepts raise a typed error. - TOML date/time handling (Breaking): TOML date and datetime literals are natively supported and map directly to the
Dateconcept; a bare time-of-day literal maps to the newTimeconcept (it never silently becomes aDate— shaping aTimeinto aDateslot fails the compatibility check). ImageContentflattens its pixel dimensions (Breaking):size: ImageSize | Noneis replaced by paired optionalwidth/heightinteger fields (both-or-neither, enforced by a validator) — a named nested shape in the crate is a concept, and the size grouping did not earn that citizenship.ImageSizeremains as the internal img-gen parameter type. Image wire payloads change shape accordingly.native.Datedeclares real structure (Breaking): MaterializedDateis no longer structureless — its pinned definition carries a requireddatefield plus an optionaltimefield (the newtimefield type), so generated clients get real Date types.DateContent's wire form is unchanged.
Fixed
- Library loaders now bind the target library for the whole load:
load_from_blueprints/load_from_crate/load_librariestook an explicitlibrary_idbut resolved concepts and the class registry through the ambient current-library ContextVar one hop below — so loading into a library that wasn't current read (and registered generated structure classes into) whatever library the ambient pointer happened to name. Production hosts always set the target current before loading, so this only bit multi-library callers — exactly the isolation the loaders exist to provide. The loaders (and blueprint removal) now wrap their body inscoped_current_library(library_id), restoring the caller's binding on exit, so everything beneath a load resolves against the load's target by construction. A regression test loads into a deliberately non-current library. PipeExtractbroke crate normalization (500s on every static-core route):PipeExtractBlueprint.validate_outputonly accepted the authoring spellingPage[], but normalization qualifies concept refs (Page[]→native.Page[]) and then rebuilds the blueprints — so the validator rejected its own normalized output, and any bundle containing aPipeExtractraised a raw pydanticValidationErrorout ofresolve_crate_from_contents, 500-ing/v1/build/inputs,/v1/build/output,/v1/resolve, and/v1/codegen. The validator now compares the parsed concept ref and multiplicity instead of the raw string, so it holds for both spellings. The output contract is unchanged and still exact — a variable-length list of nativePage— since a refined output would be an unverifiable promise:PipeExtractmechanically produces native pages and cannot make them more specific.resolve/codegenrejected bundles that pin a model: The resolve and codegen commands booted without model specs, so library validation could not check pipe model pins (model = "gpt-4o-mini") against the deck and rejected any bundle carrying one. They now boot likevalidate— offline, but with model specs loaded.- Native
Imageno longer materializes structureless: Every native now materializes from its pinned normative definition, so generated clients for image-producing pipes exposeurl,public_url,caption,width,height, etc. (The brief interim dict-with-imprecision mapping for the nested size model is superseded by theImageContentflatten above.) - Deterministic input-template placeholders: Mock URL placeholders in generated inputs templates (
codegen inputs/build inputs) are now deterministic (https://mock.invalid/<field>), so committed templates no longer churn on every regeneration. codegen inputsis write-if-changed: Likecodegen types, an already-current inputs template is left untouched (no mtime churn) and the console reportsUnchangedinstead of claimingGenerated.- Opaque generated classes no longer strip content: A structureless or Python-class-backed concept's generated Python class now carries
model_config = ConfigDict(extra="allow"), somodel_validatekeeps the payload verbatim instead of pydantic's defaultextra="ignore"silently dropping every field. Opaque is pass-through, never lossy — matching the ts-zodz.unknown()behavior. - Human CLI
~expansion:pipelex validate bundle(and the newpipelex fix bundle) now expand~in the bundle path and-L/--library-dirvalues, matching the agent CLI. - Validate telemetry label: The
CLI_COMMANDtelemetry tag forpipelex validatesubcommands no longer double-suffixes (previously "validate bundle bundle"). - Silent mistyping of inputs: Providing a bare string for a refined text concept (e.g.,
legal.Question) no longer silently degrades the type to a genericnative.Text; the declared concept type is now retained. - TOML inline table formatting: Light templates now use inline tables for structured values, preventing trailing scalars from being swallowed into
[sections].
Documentation
- Smart Inputs guides: Updated
provide-inputs.md,executing-pipelines.md, andnative-concepts.mdfor the Smart Inputs paradigm, leading with the bare-value approach and demoting the envelope format to an escape hatch.
[v0.38.0] - 2026-07-06
Highlights
- Optionality is a first-class language feature — Concept references in pipe
inputs/outputscan be marked?(optional) or!(force), so absent values become tracked, first-class data with provenance instead of crashes or silent nulls. Absence flows through a runtime trichotomy — a plain input skips its pipe, a?input absorbs it and runs, a!input fails loudly — and a static taint pass proves at author time that every maybe-absent value reaches an explicit sink. - Storage and secrets are now plugin seams — Both backends are selected at boot from a config-keyed registry, so third parties can ship
pipelex-storage-<backend>/pipelex-secrets-<backend>packages via thepipelex.pluginsentry point — the same discovery machinery the inference and orchestrator seams already use.
Added
- Optionals & presence markers (
?,!): Concept references in pipeinputs/outputscan now be marked?(optional) or!(force), e.g.clause = "PenaltyClause?". Absent values are first-class data, tracked in working memory via anAbsenceRecordledger (variable, producing pipe, kind = declared-absent / skipped / not-provided, reason, upstream chain) with provenance captured at the moment absence is produced: - A plain input fed an absence causes the pipe to be skipped (implicitly lifted); its own output is recorded absent, chaining back to the origin so the miss short-circuits like Swift's
a?.b.c. - A
?input absorbs the absence and the pipe runs, handling both arms. - A
!input fed an absence fails loudly with a typedOptionalValueAbsentErrornaming the variable, the consuming pipe, the producing pipe, and the original reason. - Callers may omit
?-marked method inputs; the slot starts as a recordednot_providedabsence rather than raising a missing-inputs error, and the missing-required-inputs message now names the optional inputs a caller may omit. -
Post-run reads get a tri-state resolved accessor (
WorkingMemory.resolve_stuff/resolve_main_stuff: a value or a recorded absence). A slot with neither a value nor a record is still a hard error — never-produced is a bug, not an absence. Markers round-trip through blueprints, builder specs, IO contracts, and bundle representations; grammar misuse (X[]?,!on an output, markers on concept definitions) is rejected at parse withoptional_marker_invalid. -
Static optionality validation (the absence-taint pass): Validation proves the absence-safety theorem at author time — a taint pass over each controller's dataflow computes per-slot presence (
guaranteed/maybe-absent) and rejects any maybe-absent slot that escapes without an explicit sink, with typed errors (optional_not_handled,optional_output_required,optional_branch_required_field) that each name the absence origin, the propagation path, and the fixes. The validation report also lists every liftable (skippable) pipe (liftable_pipes) — build-time visibility for "may be skipped when X is absent" — and gains a generalwarningsarray for advisory lints that never flipis_valid(first occupant:optional_force_redundant, a!whose slot is guaranteed present in every flow). - Template guard-lint (
optional_input_unguarded): Every template reference to a declared-optional (?) input must be guarded —@?var, a{% if var %}…{% endif %}block, or an inline presence conditional — or validation fails with the precise fix. Applies toPipeLLMprompts and system prompts,PipeComposetemplates, andPipeConditionexpressions. In the same motion,@?finally means what it says: an optional variable is no longer presence-required by the controller miss-gates, so the pipe runs and its guarded templates take the absent arm. - Execution graph updates:
GraphSpecnodes gain theskippedstatus with askip_reason— "why did my workflow produce nothing?" is now answerable from the graph — and data edges fed by a declared-optional (?) output carryoptional: true. Both flow through the in-process tracer and the distributed event-replay assembler (newpipe_end_skippedtrace event). - Storage provider plugin seam: The storage backend (
local/in_memory/s3/gcp) is selected at boot from aStorageProviderRegistrykeyed by the openstorage_config.methodtoken, populated by the always-on built-inStoragePlugin. Third parties can ship apipelex-storage-<backend>package that advertises itself under thepipelex.pluginsentry-point group — the same discovery/denylist machinery the inference and orchestrator seams use.storage_config.methodis now an openstr: an unknown method fails loudly at boot withUnknownStorageMethodError(listing the registered methods), not at config parse. See the newdocs/under-the-hood/storage-provider-plugins.md. - Secrets provider plugin seam: The secrets backend is now a plugin seam selected via the new open
secrets_config.methodtoken (default"env") from aSecretsProviderRegistry, populated by the always-on built-inSecretsPlugin. Third parties can ship apipelex-secrets-<backend>package (Vault, AWS Secrets Manager, …) under thepipelex.pluginsentry-point group — the same mechanism the storage seam uses. Out-of-the-box behavior is unchanged:envstays the default. An unknown method fails loudly at boot withUnknownSecretsMethodError. See the newdocs/under-the-hood/secrets-provider-plugins.md. - TOML pipeline inputs & templates: The
--inputsfile passed topipelex runandpipelex-agent run(pipe/bundle/method) can now be TOML in addition to JSON, discriminated by file extension — TOML's multi-line strings make text-heavy inputs much easier to author.run bundle <dir>auto-detectsinputs.tomlalongsideinputs.jsonwhen--inputsis omitted, erroring if both exist. Complementarily,pipelex build inputsandpipelex-agent inputsaccept--format json|toml(defaultjson) to emit the generated inputs template as TOML. Inline JSON ({…}) and agent-CLI stdin inputs stay JSON-only; a bare TOML datetime/date/time literal is rejected with an explicit "quote it as a string" error (native datetime concept support is a separate track). - Gateway models: Added support for
nano-banana-2-litethrough Pipelex Gateway.
Changed
- Dropped Python 3.10 support (Breaking):
requires-pythonis now>=3.11,<3.15. The 3.10 compatibility bridges are gone —StrEnum,Self, andTraversableare imported directly from the stdlib at every call site, thepipelex.typesre-export module that existed only to paper over 3.10 is deleted, and thebackports.strenumconditional dependency (and its type stub) is dropped. CI test/lint matrices, thepackage-checkrequires-pythonfloor gate, and the contributor docs now start at 3.11. - Plugin API version bumped to 3 (Breaking):
PLUGIN_API_VERSIONis now3(was2) to add theadd_storage_providerandadd_secrets_providerregistrar menu methods. Plugin discovery version-checks with strict equality, so every external plugin must re-declaretargets_api = 3. Our Mistral Workflows plugin registers no storage or secrets provider, so thetargets_apibump is its only change. Our Temporal plugin needs the bump and a code migration: it imports the now-removedmake_storage_provider_from_config(see Removed) and must switch to resolving the provider via the registry (get_storage_provider_registry().get_required(method=...)). PipeSignatureis not a pipe type (Breaking): A signature is now declared by omittingtype— a[pipe.x]section with notypeand nothing but the contract (description,output, optionalinputs, optionalsignature_for) is aPipeSignature. Writingtype = "PipeSignature"explicitly is rejected with a migration error ("PipeSignatureis no longer a pipe type — delete thetypeline"), and a typeless section declaring any non-contract field is a hard error naming the field. Thepipelex-agent pipeauthoring command mirrors this. MTHDS JSON Schema shape change: the signature arm no longer carries atypeproperty (an explicit tag now fails schema validation), so the downstream schema copies (mthds,vscode-pipelex,mthds-ui) must be re-synced on the next release via themthds-schema-syncskill.PipeConditioncontinuenow resolves the output as absent (Breaking): Thecontinuespecial outcome no longer passes the current main stuff through (and no longer errors when there is none). It records a declared-absentAbsenceRecordfor the condition's declared output — with the evaluated expression as the reason — and returns success, memory otherwise unchanged. Acontinue-reachablePipeConditionmust therefore declare its output optional (e.g.output = "Constraint?"), or validation rejects it (optional_output_required). Dry-run parity holds. Migration: the previous value stays in working memory under its own name, so downstream pipes consume it explicitly by that name (declared?when it may be absent); a coalescing operator is the planned ergonomic replacement for pass-through.- Controllers combine under absence:
PipeParallelno longer crashes when a branch result resolves absent (lifted branch orcontinue): with aCompositeoutput the absent component is omitted (ledger note kept), and with a structured output a non-required field absorbs the absence as its default (Noneunless the author declared another), while a required field fed an absent branch raises a typed error naming the branch, the field, and the fixes.PipeBatchcompacts: absent branch results are dropped from the aggregated list, so batching a "keep or skip" condition over items yields only the kept results. - Absent output delivery is now a first-class success: A run whose declared
?output resolves absent succeeds everywhere.main_stuff_nameonPipelexPipeRunOutputand the/executeresponse names the declared output slot even when it resolved absent — consumers branch on the absence record in the serialized working memory (absences), never on transport, and the ledger round-trips cross-process (hydrate_working_memoryreconstructs it). Themain_stuff.json/md/htmlartifact files become an explicit absence document ({"absent": true}plus the provenance chain) on both the typed and raw delivery paths,pipelex run/--save-main-stuffand the agent CLI'srunsurface it instead of crashing, and OTel/Langfuse capture serializes it. A working memory with neither a value nor a recorded absence still fails delivery loudly.
Fixed
- Template truth-tests on singular values no longer crash:
{% if var %}(and@?var, which expands to it) on a present singular value used to raiseTypeError: '…' content does not support len().— Jinja2's truth test fell through to the artefact's list-only__len__.StuffArtefactnow defines__bool__: a present non-list artefact is truthy, a list artefact follows list emptiness. This makes the optionals guard idiom safe on both arms. PipeComposeescaped-sigil literals ($$,@@) no longer double-rewrite:PipeComposealone stored its template already-preprocessed and then re-ran the sigil rewriter at guard-lint and render time. Because the escape collapse ($$name→$name) is not idempotent, the second pass resurrected escaped literals into interpolations — raising a spuriousoptional_input_unguardedon a$$namethat only looks like an optional reference, and silently rendering the value ofnameinstead of the literal$name.PipeComposenow stores authored source and rewrites exactly once, like every other pipe.--inputs ~/…now expands onrun pipe/run bundle: a quoted or=-form tilde path (--inputs "~/inputs.json", which the shell leaves unexpanded) is nowexpanduser()-ed before loading, resolving to the home directory instead of failing on a literal~— matching the existingrun methodbehavior, on both the human and agent CLIs.- Invalid storage/secrets
methoderrors stay actionable under STRICT:UnknownStorageMethodError/UnknownSecretsMethodErrorare marked caller-facing, so their "registered methods: … — checkstorage_config.method" guidance survives STRICT error disclosure instead of being redacted to a generic internal-error message. - Non-string JSON pipe
typegives an actionable error: atypethat is a list/dict/number in apipelex-agent pipeJSON spec now surfaces as anArgumentErrornaming the valid pipe types, instead of a cryptic internalTypeError: unhashable type. - GitHub Actions: Removed an invalid
environmentblock from themanual-trigger-tests-check.ymlworkflow.
Documentation
- Optionality guide: New "Understanding Optionality" page (
docs/building-methods/pipes/understanding-optionality.md) beside the multiplicity guide — presence markers, the runtime trichotomy (skip / run / fail), absence records and provenance, template guards, controllers under absence, and the static safety net. PipeBatch documents compaction under absence, PipeLLM documents the@?optional block sigil, and the run CLI page documents the absence artifact an absent main output produces.
Removed
make_storage_provider_from_config(Breaking): The module-level storage-provider factory (pipelex/tools/storage/storage_provider_factory.py) is removed — storage is now resolved through the config-selectedStorageProviderRegistryat boot. Downstream code that imported this helper must select through the registry instead.
[v0.37.0] - 2026-07-04
Highlights
PipeParallelalways combines — and every run has a main output — A parallel now always combines its branch outputs into its declaredoutputconcept (with the newnative.Compositeconcept as the ready-made combination vehicle), thecombined_outputfield is deleted from the MTHDS language, and the main-stuff invariant is enforced end to end: every completed pipe run delivers amain_stuff, so downstream surfaces (delivery, graph tracing, telemetry, wire models) can rely on it unconditionally.- Portable, statically validated image generation — A new portable
sizeparameter ("1k"/"2k"/"4k"tiers or exact pixel dimensions) carries the same size intent across providers, Gemini models gain image-to-image editing and extreme banner aspect ratios, and Google image models are now validated against declarative geometry rules at blueprint-load time — unsatisfiable requests fail fast instead of at the provider call.
Added
- Portable image size (
size) forPipeImgGen: Newsizeparameter accepting portable tiers ("1k","2k","4k") or exact pixel dimensions (e.g."2048x1152"). A tier means "this pixel class at my chosenaspect_ratio", mapped to each provider's own grid; an exact size is deterministic (declaringaspect_ratioalongside it is a validation error). Unsatisfiable requests fail as hard validation errors at blueprint-load time, never warn-and-ignore. Whensizeis unset, no size intent is sent and the provider default applies. The MTHDS JSON Schema exposes the field, and an optionalsizedefault is supported in[cogt.img_gen_config.img_gen_param_defaults]. native.Compositeconcept: New native concept backed byCompositeContent— an untyped, named composition holding sub-contents as top-level fields, serving as the default combination vehicle for parallel branches. Supports the full content surface:smart_dump, kajson/transport round-trip, and markdown/HTML rendering.- Gemini image-to-image (img2img) support: The Google img-gen worker now forwards input images to the Gemini API as inline parts, enabling image editing and multi-image composition for the
nano-bananamodels (including the newnano-banana-2-lite). Every img-gen worker now validates input images against the model's declared capability at job start, rejecting img2img on unsupported models with a cleanImgGenParameterErrorbefore any provider call. - Banner aspect ratios: Support for extreme banner formats (
landscape_4_1,landscape_8_1,portrait_1_4,portrait_1_8) from Gemini 3.1 image models. All other image-gen backends reject them with a clean parameter error. - Static validation for Google image models: Google image models now use
rulesblocks (geometry taxonomies likegemini_3_flash) in the backend deck, catching unsupported aspect-ratio and size combinations at blueprint-load time. The Google img-gen factory is keyed by taxonomy instead of hardcoded model names (Breaking: theGoogleImageGenModelname enum is removed — model handles are deck config, not code constants).
Changed
PipeParallelalways combines (Breaking): APipeParallelcontroller now always combines its branch outputs into its declaredoutputconcept and stamps it as the main output. The declaredoutputis strictly validated at author time and must beCompositeor a structured concept whose fields and types match the branchresultnames. Combination replaces the removedcombined_outputfield. Migration: pipes declaring both fields just drop thecombined_outputline;add_each_output-only pipes replace their placeholderoutputwithCompositeor a matching structured concept.add_each_outputkeeps its meaning and now defaults tofalse.- Main-stuff invariant enforced (Breaking): Every pipe run now guarantees a
main_stuff; defensive "maybe there is no main stuff" branches are removed, wire models make the field required (main_stuff_name: stronPipelexPipeRunOutputand the/executeresponse extension), andPipeOutput.optional_main_stuffis replaced byPipeOutput.main_stuff. - Orchestrator SPI delivery split (Breaking):
OrchestratorProtocol.runis now strictly the blocking arm (returning a completedPipelexPipeRunOutput). A newOrchestratorProtocol.starthandles fire-and-forget, returning aPipelexPipeDispatchAck(IDs only).PipelexPipeRunOutput.is_completedis deleted — it is always a completed output now. LLMWorkerInternalAbstractfolded (Breaking): Removed and folded intoLLMWorkerAbstract, which now owns the whole job lifecycle (capability checks, constraints, telemetry). Subclasses extendLLMWorkerAbstractdirectly and implement_gen_text/_gen_object, nothing else.- Type preservation across transport:
Compositecomponents (and their nested lists) now retain strict types across transport boundaries (dump_for_transport/ hydration) via private class markers. PipeParallelhonorsfinal_stuff_code: A requested final stuff code (e.g. from aPipeBatch) is now correctly stamped on the parallel's combined output.- Docs — inference plugins: Rewrote
using-inference-plugins.mdto demonstrate a genuinepipelex.pluginsentry-point package instead of a legacy config workaround. - Linting pipeline (repo-local): in the pipelex repo itself,
plxt lintnow validates.mthdsfiles against the locally generated schema (derived/mthds_schema.json) rather than the released schema bundled withplxt. This is a.pipelex/plxt.tomloverride for this repo only — theplxt.tomltemplate distributed bypipelex init configis unchanged. - Dry-run mocks: Reduced the default dry-run mock list generation from 3 items to 2 to cut dry-run processing time.
- Bumped
mthdsdependency from>=0.6.0to>=0.7.0.
Fixed
PipeConditionpass-through failure: Acontinueoutcome with no pre-existing main stuff now fails loudly and actionably at the pipe level instead of crashing downstream surfaces.- Stale main stuff in parallels: A pipeline ending in an
add_each_output-onlyPipeParallelno longer silently reports the previous step's output as its main result. add_each_outputoptional in the builder spec:PipeParallelSpecnow defaultsadd_each_outputtofalselike the blueprint does, so a generated always-combine spec with justbranchesandoutputno longer fails validation beforeto_blueprint().- Google img-gen size handling: The native Google worker and gateway path now send the requested
image_sizeto the Gemini API instead of hardcoding"1K", making 2K/4K generation reachable. - Img2img support checks: The capability check now keys off the model's declared
inputsrather than requiring aninput_imagesrule, which had falsely reported Gemini models as unsupported. - Routing profile optional routes:
optional_routesdeclared inrouting_profiles.tomlare no longer silently dropped by the factory. - Unknown boot orchestrator fails loud: Requesting a boot orchestrator no installed plugin provides — via
--orchestrator <name>orPipelex.make(boot_orchestrator=...)— now raisesUnknownBootOrchestratorErrorat boot instead of silently falling back to in-process execution. - Failed boot no longer leaks process-global state: A
Pipelex.makethat raises during setup now releases the process-global singletons a partial boot acquired (config, logging, the kajson class registry, template registries), so a failed boot no longer poisons a subsequent boot in the same process. - HuggingFace streaming errors: The provider error-body reader now safely tolerates unread
httpxstreaming responses without crashing withhttpx.ResponseNotRead.
Removed
combined_outputfield (Breaking): Removed from the MTHDS language forPipeParallel; pipes now declare their combination target directly in theoutputfield.- Legacy external plugin setter (Breaking): Removed
set_llm_worker_from_external_pluginfrom the inference manager, fully replaced by thepipelex.pluginsentry point.
Security
- Transformers vulnerability: Bumped
transformerspast CVE-2026-4372 (GHSA-29pf-2h5f-8g72) to resolve a high-severity RCE. To unblock this, thehuggingfaceextra now requireshuggingface_hub>=1.5.0,<2.0.0(Breaking).
[v0.36.0] - 2026-06-30
Highlights
- Orchestration plugin SPI — Pipelex opens its execution path to pluggable orchestrators: the orchestrator that runs a job is now chosen per call by an open
orchestration_modetoken, with the core shipping an in-processdirectorchestrator and the contract ready for plugins to supply distributed backends. - Orchestration plugin contract (
orchestration_mode+DeliveryMode): The contract is built on two independent axes — an openorchestration_modetoken (which orchestrator runs the job —directin core, other modes supplied by plugins) and a closedDeliveryModeenum (the wait-semantics axis —BLOCKING/FIRE_AND_FORGET). An orchestrator plugin implementsOrchestratorProtocol.runwith a requireddelivery: DeliveryModekeyword and declares asupports_fire_and_forget: boolcapability; the orchestrator registries are keyed by theorchestration_modetoken.PipelexPipeRunInputcarriesorchestration_mode+delivery. - Orchestrator-dispatched
/validate: A new per-call bundle-validator seam (BundleValidatorProtocol/BundleValidatorRegistry) makes/validatedispatch byorchestration_modethe way/startruns a pipe —directvalidates in-process, distributed modes dispatch the job to a worker — with a byte-identical verdict across backends. The coredirectplugin registers an in-processDirectBundleValidator. - Honest fire-and-forget delivery: A fire-and-forget
/startrequest is honored only when the resolved orchestrator can do genuine async; an orchestrator that cannot (e.g. the in-processdirectmode) returns a 4xx rather than running the job to completion and falsely acking.
Added
- Explicit graph targets for validation: Graph generation during validation and dry runs now accepts an explicit
pipe_codetarget, so developers can generate graphs for a specific pipe in a bundle even when nomain_pipeis declared. This is wired through the CLI (validate bundle --pipe <target> --graph/--view),dry_run_pipeline,generate_graph_for_bundle,generate_view_for_bundle, andPipelexMTHDSProtocol.validate(extra={"graph_pipe_code": "..."}). - GraphSpec source path enrichment:
GraphSpecartifacts now include the declarationsourcefile path for pipes and concepts inpipe_registryandconcept_registry, powered byLibraryCrate.source_map. - CI/environment heartbeats: Added
WAIT_WITH_HEARTBEATandRUN_WITH_HEARTBEATMakefile macros that emit a periodic heartbeat (default every 20s) to prevent CI runners and sandboxes from timing out during long tasks; applied to theagent-test,pyright,mypy, andpylinttargets. - Developer guidelines: Expanded
python_standards.mdwith rules on filesystem paths (pathlib.Pathoverstr/os.path), data-holder shapes (NamedTuplevs.pydantic.dataclassesvs.BaseModel), keyword-only arguments (enforced viamake cko/make fko), and custom exception placement (exceptions.pyor<topic>_exceptions.py).
Changed
dry_run_pipelinereturn shape:dry_run_pipelinenow returns the fully resolved, domain-qualified pipe reference alongside theGraphSpec.- Graph viewer assets bumped: The CDN-pinned
@pipelex/mthds-uigraph viewer used by the generated ReactFlow HTML is upgraded from0.6.4to0.11.0, with refreshed Subresource Integrity hashes.0.11.0addsPipeStructureto the viewer's recognized pipe types, so graphs containing aPipeStructurenode (e.g. from thepreliminary_textstructuring path) render instead of showing the GraphSpec validation error screen.
Fixed
- Graph generation on bundles without a
main_pipe: Requesting a graph or view via the CLI no longer fails with a missingmain_pipeerror when a valid pipe is targeted with--pipe. - Developer guidelines typo: Fixed a minor spacing typo in
pytest_standards.md.
[v0.35.1] - 2026-06-22
Changed
PipeImgGendocumentation: Rewrote the docs to clarify how inputs are consumed.PipeImgGenhas no dedicated "prompt concept"; it uses apromptstring template into which declaredinputsare injected at runtime —Textvariables are interpolated directly, whileImagevariables (single or lists) are injected as reference images to enable image-to-image and editing workflows. Added examples demonstrating this vision pattern.
Fixed
- Blueprint-stage validation error categorization:
PipeValidationErrors raised during blueprint parsing (e.g.PipeBatchorSubPipeitem name collisions) previously lost theirerror_typebecause Pydantic wrapped them in a genericvalue_error, degrading them to uncategorized residual errors. The categorizer now unwraps them, preserving their structurederror_type,pipe_code,domain_code, andsourcelocators.
Removed
- Native concept
ImgGenPrompt(Breaking): Removed the built-innative.ImgGenPromptconcept. It was structurally identical toText(mapped toTextContent) and added no unique semantics;PipeImgGennever depended on it. Migration: replaceImgGenPrompt(orrefines = "ImgGenPrompt") withTextin your.mthdsfiles. The internalImgGenPromptruntime model, theTemplateCategory.IMG_GEN_PROMPTcategory, andImgGenPromptErrorare unchanged. - Dead validation error type (Breaking): Removed the
PipeValidationErrorType.img_gen_input_not_text_compatibleenum value. It had no raise sites and contradicted the current design, wherePipeImgGenaccepts image inputs as a first-class feature.
[v0.35.0] - 2026-06-18
Added
- Structured validation errors: Bundle validation failures now emit categorized
validation_errors[]items with identity locators, so machine consumers can readerror_typeand locators instead of parsing text. Categories: unresolved concept reference (pipe_validation/unresolved_conceptwithpipe_code,concept_code,field_name; orblueprint_validation/unresolved_conceptwith owningconcept_code), undefined pipe dependency (pipe_validation/unresolved_pipe_dependencywithpipe_codeand the newmissing_pipe_codelocator), and unknown pipe type (blueprint_validation/unknown_pipe_typewithpipe_code). - New error locators and types: Added the
missing_pipe_codelocator field and thePipeValidationErrorTypevaluesunresolved_concept,unresolved_pipe_dependency, andunknown_pipe_type. - String utilities: Added
pluralizeandcount_with_nountopipelex.tools.misc.string_utilsfor correct CLI output grammar (e.g. "1 pipe" instead of "1 pipe(s)"). - Documentation: Documented the 0/1/2 exit-code policy in
docs/under-the-hood/error-model.md.
Changed
- Validate exit-code policy (Breaking): Both
pipelex validateandpipelex-agent validatenow use a three-tier exit code mirroring the hosted/validateAPI:0= valid (including valid-but-not-runnable with--allow-signatures),1= negative verdict (invalid, or valid-but-not-runnable without--allow-signatures),2= no verdict (bad arguments, unresolvable target, or setup/internal errors). Consumers testing zero vs. non-zero are unaffected; machine consumers should rely on the structuredis_validJSON field. - Relocated markdown renderer: Moved
format_validate_markdownfrom CLI internals to the publicpipelex.pipeline.validation_rendermodule so other surfaces (e.g. thepipelex-api/validateroute) can use it without Typer/CLI dependencies, and addedrender_invalid_validation_markdownfor invalid verdicts. - Error class hierarchy:
ConceptLibraryErrornow extendsLibraryLoadingError(instead ofLibraryError) so it can carry per-reference structured items through the existing error cascade. - Strict protocol arguments:
PipelexMTHDSProtocolnow rejectsextraextension arguments, raisingPipelineRequestError, since the local runtime defines no extension arguments. - Bumped
mthdsdependency from>=0.4.1to>=0.5.0.
Fixed
- Dropped
pipe_codeon blueprint errors: The blueprint validation-error categorizer matched the wrong Pydanticlockey (pipesinstead ofpipe), causing categorized items (e.g.missing_input_variable) to silently lose theirpipe_codelocator; it now populates correctly. - CLI output grammar: Validation CLI output now uses the new string utilities, fixing awkward pluralizations (e.g. "Validated 1 pipe(s)" is now "Validated 1 pipe").
[v0.34.0] - 2026-06-17
Highlights
- Structured validation errors — validation failures now return a typed, per-error
validation_errors[]on the error wire instead of a baredetailstring, with the same shape across the HTTP API and the agent CLI. - Structured
validation_errorson the error wire:ErrorReportgains a typedvalidation_errors: list[ValidationErrorItem] | Nonefield.ValidateBundleError.to_error_report()flattens its per-error data onto it via a single shared builder (pipelex/pipeline/validation_errors.py), so the structured error report an HTTP API surfaces carries machine-mappable per-error diagnostics —category,message, identity fields, and asource(declaring file) for cross-file mapping — instead of only adetailstring. The closedValidationErrorCategoryset isblueprint_validation/pipe_factory/pipe_validation/dry_run; a residual dry-run failure (DryRunError/PipeRunError, no structured locator) is projected as onedry_runitem only when no categorized error has data, so an invalid verdict always carries a non-emptyvalidation_errors[](the structured-info invariant) rather than a bare message. The same builder feeds the agent CLI'svalidation_errorsJSON array, so the CLI and API structured shapes cannot drift. The list is surfaced under STRICT disclosure (_STRICT_KEPT_FIELDS) because it describes the caller's own submitted bundle, not server internals. - Changed — agent CLI
validation_errorsshape: thevalidatecommands' error envelope now omits null-valued keys per entry (exclude_none) and gains the previously-droppedsource,field_name, andconcept_codefields. Entries are guaranteed to carrycategoryandmessage; all other keys are present only when populated, so consumers must treat them as optional rather than assume a fixed key set. - Per-content
sourceon the in-memory validate path:PipelexInterpreter.make_pipelex_bundle_blueprint(mthds_source=...)andvalidate_bundle(mthds_sources=...)let a caller attach a logical source to each in-memory bundle string, threaded intoblueprint.source. A sourcelessmthds_contentssubmission previously producedsource=None, breaking cross-file diagnostics; a host (e.g. an HTTP API) can now pass per-item sources so the structuredvalidation_errorscarry a real owning file. The on-disk CLI path is unchanged (it already records real file paths). -
dry_runvalidation category:ValidationErrorCategorygains adry_runvalue. A dry-run residual failure that previously produced a bare-messageValidateBundleErrorwith an emptyvalidation_errors[]now surfaces one structureddry_runitem carrying the message (graph-level, so typically nosource) — closing the structured-info gap that drove consumers to fabricate a category. -
Keyword-only argument enforcement — a mechanical guard enforces the keyword-only argument convention across
pipelex/, end to end — local autofix through the CI gate. - Keyword-only AST guard: A custom AST-based linter (
pipelex-dev check-keyword-only/make cko) that mechanically enforces the keyword-only argument convention acrosspipelex/, wired intomake checkandmake agent-check. - Keyword-only auto-fix: A non-gating
--fixmode (pipelex-dev check-keyword-only --fix/make fix-keyword-only/make fko) that rewrites every mechanically-fixable violation by inserting a bare*as far left as possible (afterself/cls), re-parsing each rewrite before writing and reporting the shapes it can't fix mechanically. It runs early inmake agent-check; the read-only check still runs last and owns the pass/fail gate. - Claude Code hook: A
PostToolUsebash hook (.claude/hooks/check-keyword-only.sh) that runs the guard on edited files for immediate blocking feedback to AI agents. - CI integration: A dedicated
lint-keyword-onlyjob in the GitHub Actions linting workflow to block non-compliant signatures from merging. - Convention documentation: New
docs/contribute/keyword-only-arguments.md, plus updates toCLAUDE.mdandAGENTS.mdestablishing the keyword-only convention as a standing rule.
Added
is_validon the canonical validation report:PipelexValidationReportgainsis_valid: Literal[True] = True, the always-true discriminant of the valid arm of the hosted/validateresponse union (mirrored by pipelex-api'sInvalidReport'sLiteral[False]). It sits besideis_runnable: a sound bundle may still be not-yet-runnable.- Offline CLI unit tests: Coverage for the pipelex-internal logic behind the CLI commands —
doctordiagnostics,runexecution and its sync wrapper, thebuildcodegen cores, the readiness gate,show/which, and the gateway/telemetry/signature error handlers (the spec'd CLI interface stays owned by our cross-repo spec suite). - Offline inference & runtime unit tests: Coverage for layers that can break without a provider — structured-output↔instructor mode mapping, model-deck reference checks, the image-gen argument and worker-routing factories, gateway request-shaping and extract parsing, the Mistral factory, the local observer sink, the output renderer, builder spec validation /
to_blueprint(), the pipeline runner's error paths, the TOML config-sync engine, and the storage config validators.
Changed
- [BREAKING] Keyword-only public API: Top-level public surfaces now require keyword arguments after the first parameter. A caller passing a second-or-later argument positionally must switch to keyword form — affected methods include
Pipelex.make(integration_mode, *, ...),Pipelex.setup(integration_mode, *, ...), andPipelexHub.setup_config(config_cls, *, ...). Downstream consumers to check:pipelex-api,n8n-nodes-pipelex, cookbook, starter, and our hosted services. - Codebase-wide keyword-only refactor: The entire
pipelex/tree (leaf tools, domain core, inference layer, execution path, framework-sensitive packages) now places a bare*after the subject parameter, requiring all later arguments to be named at the call site (e.g.copy_file(src, target_path=dst, overwrite=False)). - Signature subject corrections: Reordered parameters so the true semantic subject is the first positional argument, e.g.
parse_pipe_spec(spec_data, *, pipe_type),hydrate_content(raw_content, *, concept), andwrite_manifest(manifest, *, deck_dir).parse_pipe_specis consumed by the hosted runner API (pipelex-api), updated in lockstep. - [BREAKING] Canonical protocol validation report:
PipelexValidationReportis reworked into the canonical typed shape shared by every backend and moved topipelex/pipeline/validation_report.py— a typedbundle_blueprint(replacing the untyped single-or-listblueprintdump),pipe_io_contractskeyed by namespacedpipe_ref(newpipelex/pipeline/pipe_io_contracts.py),validated_pipes, and thepending_signatures/is_runnableverdict. It is assembled in exactly one place (build_validation_report) with a single shared primary-blueprint rule (select_primary_blueprint). - [BREAKING]
validated_pipeskeypipe_code→pipe_ref: the entry has always carried the namespacedpipe_ref(domain.code) under a key named for the wrong identity; the key now says what the value is. - Protocol
validatenow produces a best-effortgraph_spec: when the batch declares amain_pipe, the local runtime dry-runs it in-process and ships the resultingGraphSpecvia one shared implementation (best_effort_graph_spec); a graph-arm domain failure degrades tograph_spec=Nonewith validation still successful. (dry_run_pipe_in_processmoved topipelex/pipe_run/dry_run_in_process.pyto break an import cycle.) - [BREAKING]
MTHDS_PROTOCOL_VERSIONdeleted: the hardcoded duplicate is gone fromrunner.py; the SDK'sPROTOCOL_VERSION(mthds.protocol.protocol) is the single source of truth, imported directly by consumers including the hosted API. - Additive multi-file library construction: Same-domain
.mthdslibraries can now be built as separate, additive files (forward-declaredPipeSignatureheaders plus concrete definitions in sibling files), enabling parallel top-down construction. APipeSignatureand a same-code concrete pipe reconcile (contracts compared by normalized concept identity) instead of colliding; concept and qualified pipe references resolve against the merged library across sibling and separately-loaded files; domaindescription/system_promptmerge order-independently; and a successfulvalidatereports library-widepending_signaturesplus anis_runnableverdict in both JSON and markdown. - [BREAKING]
PipeSignatureevicted from the executable pipe taxonomy: a signature is a contract, not a way of running, soPipeType.PIPE_SIGNATUREandPipeCategory.PIPE_SIGNATUREare removed (any code matching on them must drop that arm). It stays aPipeAbstractsubclass withpipe_category = None, andis_signatureis now a class fact rather than an enum read. - [BREAKING] Signatures are never a validation error (signatures-as-data): the validator no longer raises when a pipe reaches an unimplemented
PipeSignature. Strict and lenient modes return the same report body — the assembled library's outstanding signatures viapending_signaturesandis_runnable = not pending_signatures.allow_signaturesnarrows to a sweep-mechanics flag (whether signature pipes are mock-run and listed invalidated_pipes); it no longer changes the verdict. The "is this a failure?" decision moves to the consumer: the whole-bundle / whole-library surfaces (pipelex validate,pipelex validate --all,pipelex-agent validate bundle/validate method/validate pipe --all) derive the exit code from the library-wide runnability verdict — strict by default, exiting non-zero onnot is_runnableunless--allow-signatures, while still emitting the success envelope (carryingpending_signatures+is_runnable: false). Single-pipe surfaces make no library-wide runnability claim and never gate: barevalidate pipe <code>, andvalidate bundle/validate methodinvoked with--pipe(the slice can be fully implemented even when unrelated placeholders remain elsewhere). The execute/run path is unchanged — running a stub still raisesPipeSignatureNotExecutableError. - [BREAKING] Host-wiring guards reclassified to
PipelexUnexpectedError:validate_bundle/load_concepts_only's "provide exactly one ofmthds_contents/mthds_file_path" guards (and the existingmthds_sources-length-mismatch guard) now raisePipelexUnexpectedError(→ 500), notValidateBundleError— a caller wiring bug is a programmer error, not a content verdict to be reported as an invalid bundle. The empty-mthds_contentsguard stays caller-facing.
Fixed
- Protocol model deck no longer loses aliases to cross-category collisions:
PipelexModelDeck.aliases/waterfallsare now keyed by model category ({category: {alias: model}}) instead of being flattened withupdate(), which silently kept only the last category's entry for an alias name shared across categories. Breaking for the deck's extension shape. - JSON-Schema rendering failures on the validate surfaces are now structured errors: a pydantic schema-generation failure while building
pipe_io_contractsnow raises the newPipeIOContractError(naming the offending pipe/input/concept) instead of a raw exception. - Protocol
validateteardown can no longer mask the real error: a raising library teardown is suppressed while a body error is propagating, and the validation library id is captured once so the graph arm and the teardown always target the same library. - Empty
mthds_contentsis rejected with a structured error:validate_bundleandselect_primary_blueprintnow raiseValidateBundleErrorinstead of crashing primary-blueprint selection with a rawIndexError; the hand-rolled first-declaring-main_pipeloops inexecution_seams,dry_run_pipeline, andinputs_opsare folded into the singleselect_primary_blueprint. - Structured-info invariant is now total — parse-level failures carry a structured item: a malformed
.mthdsfile (a TOML-syntax error, an empty blueprint, or a bundle-elaborator failure) is raised with only a message and no categorized data, so an invalid verdict previously rode an emptyvalidation_errors[]for the single most common failure mode. The shared builder (build_validation_error_items) gains a last-resortfallback_messageresidual: when no categorized error and nodry_runresidual produced an item, it emits oneblueprint_validationitem carrying the message (nosource, noerror_type— the bundle could not become a blueprint at all). Both surfaces —ValidateBundleError.to_error_report()and the agent CLI'sextract_validation_errors()— pass it, so every invalid verdict now carries a non-emptyvalidation_errors[], never a bare message. - Qualified same-domain pipe references resolve across files: a controller referencing a sibling-file pipe by qualified name (
research.find_key_findings) is now deferred to the merged library like bare references; a reference no file declares is still rejected at load. - Concepts-only loading validates concept references: the
pipelex structures/load_concepts_onlypath now runs the cross-file concept-reference check (batching sibling files into one pass) instead of silently accepting a structure field pointing at an undeclared concept. - Multi-file dependency packages reconcile signatures with their definitions: a dependency package split into a
PipeSignatureheader plus its concrete sibling now goes through the same additive merge instead of colliding on the duplicate code and dropping one declaration by load order. - OpenAI image moderation mapping was inverted:
is_moderated=truesent the less restrictivemoderation="low"andfalsesent"auto"— the mapping now matches the flag (enabled →"auto", disabled →"low").ImgGenSetting.is_moderatedalso defaults toNone, so workers omit the parameter and the provider's own default applies (which also stops force-disabling FAL'senable_safety_checker). - Storage config validation now covers every provider and requires a real
{hash}slot: theuri_formatcheck is enforced uniformly across local / in-memory / S3 / GCP — every placeholder must be a plain supported{name}, a{hash}slot is required (GCP previously accepted the bare substringhash, silently overwriting every stored object), and buckets must set a positivesigned_urls_lifespan_seconds. A misconfigured format now fails fast at Pipelex boot instead of at the first content store. - Mistral chat requests now send the system message before the user message:
make_simple_messagespreviously appended it after the user message, contradicting its own docstring and the OpenAI-typed sibling. - String concept values in bundle specs are constructible again: a bare string in the
ConceptSpec | strunion now validates (no longer crashing themode="before"validator) and passes through to the blueprint as the concept's description instead of intostructure, where the loader rejected it. - LocalObserver JSONL records always carry the true lifecycle event name: a payload's own
event_typekey can no longer overwrite the event name in the written record, which had broken event-type filtering for JSONL consumers. - Tests under
tests/**/build/are no longer silently skipped: pytest's defaultnorecursedirsincludesbuild, so the wholetests/unit/pipelex/cli/commands/build/tree was never collected; the config now overridesnorecursedirsand setstestpaths = ["tests"]. Anything-output option numbering is now deterministic: the output renderer sorts aPipeCondition's possible outputs by pipe code instead of iterating a set, sooutput_option_N/schema_option_Nnumbering is stable across runs.- Documentation examples: Updated code snippets in
docs/under-the-hood/to match the new keyword-only signatures. - Test mock assertions: Migrated mock assertions from
.call_args.argsto.call_args.kwargsto reflect keyword-only calls.
Removed
- [BREAKING]
SignaturesNotAllowedErrordeleted: with signatures no longer a validation error, the strict-mode signature pre-pass (BundleValidator._signature_pre_pass), thesignature_check_errorfield onValidateBundleError, thehandle_signatures_not_allowed_errorCLI renderer, and theSignaturesNotAllowedErrorexception class itself are removed. Code that caughtSignaturesNotAllowedErroron the validate path should read the report'spending_signatures/is_runnableinstead; the execute path'sPipeSignatureNotExecutableErroris unaffected. - Keyword-only migration scaffolding: With the whole
pipelex/tree converted, the temporary baseline file and thepipelex-dev check-keyword-only --regen-baselineflag are removed; the guard now hard-blocks on any new violation rather than burning one down.
[v0.33.0] - 2026-06-11
Breaking Changes
PipelexRunner→PipelexMTHDSProtocol— the runner class now implements the MTHDS Protocol (mthds.protocol.protocol.MTHDSProtocol, mthds 0.4.1). Method renames:execute_pipeline→execute,start_pipeline→start(stillNotImplementedErrorlocally).execute/startcarry the protocol's basic args plus a genericextrapassthrough — server-specific args (a client-supplied run id, callbacks, a storedmethod_id) rideextra, never named params. Response classes:PipelexPipelineExecuteResponse→PipelexRunResultExecute(subclassesRunResultExecute—pipeline_run_id+pipe_output),PipelexPipelineStartResponse→PipelexRunResultStart(subclassesRunResultStart—pipeline_run_idonly).state/created_at/finished_at/main_stuff_name/workflow_idare pipelex extension fields on the protocol's base responses; the run identifier keeps the namepipeline_run_ideverywhere.mthdspin → 0.4.1 — the restructured SDK (mthds.protocol+mthds.runners.api). The agent CLI's API run path usesMthdsAPIClient(mthds.runners.api.client) directly; domain shapes import frommthds.protocol.*, Dict wire models frommthds.runners.api.models.
Changed
--dry-runnow mocks at the cogt leaf instead of swapping out the operators.run_moderides a newCogtRunParamscarrier (derived fromPipeRunParams.run_mode, stamped on every cogt assignment), so a DRY run mocks inside the inference leaf at zero AI cost, with no API keys and no storage IO (the img/extract DRY branches sit above the store step).ContentGeneratorDryis deleted: operators no longer swap generators (the base operator's dry path simply reuses the live path), and the in-process validation scopes pass an inlineContentGenerator. Object mocks are now schema-built; classes with exotic format constraints should declareexamples/mock_format(a re-validation failure surfaces as a typedDryRunObjectFidelityError, and a deterministic mock-build failure as the newDryRunMockBuildError).- Unified dry run —
--mock-inferenceremoved (breaking). There is now exactly one non-live run mode:--dry-run. The mock-inference mode is retired; its capability — non-zero synthetic usage so the cost report renders — survives as an internalis_mock_usagesub-flag of DRY onCogtRunParams(replacingis_mock_inference; setting it on a LIVE run is a validation error). It is exposed on the Python surface (PipelexMTHDSProtocol/execute/prepare_pipe_jobviais_mock_usage=...) and as a hidden test-only CLI trigger, deliberately undocumented. Dry-run leaf coverage was already uniform across every operator, soMockInferenceUnsupportedErrorand its img-gen/extract/search guards are deleted;MockInferenceObjectFidelityErroris renamedDryRunObjectFidelityError(mock-built objects are DRY-only now); themock_inferenceusage sentinel is renamedmock_usage. - Keyless boot (
needs_inference=False) now forces every run to DRY instead of installing a mock generator. Generator selection is purely backend-keyed; the forced-DRY flag is consumed atPipeRunParamsFactory.make_run_params— the single writer ofrun_mode— so every execution entry point is covered, including the runtime bridge.
Removed
dry_run_config.apply_to_jinja2_renderingconfig key: it was dead — PipeCompose renders templates directly and the jinja2 parse check survives in the templating leaf's DRY branch. Remove it from your.pipelex/pipelex.tomloverride if present (config is strict and will reject the unknown key).
Fixed
-
Resubmitting a
pipeline_run_idafter its run finished no longer fails permanently.PipelineManagerwas the one per-run registry with no per-run removal: every run permanently stranded its key, so a second submission of the samepipeline_run_idagainst a long-lived server process raisedPipelineManagerAlreadyExistsError— surfacing as an unrecoverable 500 from the hosted runner API until process restart. The registry entry is now freed on every exit path:execute'sfinallyremoves it after each run (success or failure), andpipeline_run_setupremoves its own registration when setup fails after registering (the caller never learns the id on that path, so setup must self-clean). Serial resubmission of a completed/failed run id now succeeds; only genuinely concurrent same-id runs still collide — deliberately, because the collision raise fires beforeopen_tracerand shields the live direct-mode tracer (keyed by the caller-suppliablepipeline_run_id) fromopen_tracer's stale-key pop-and-replace healing. The hosted API maps the remaining concurrent-duplicate case to 409 Conflict (pipelex-api change). -
dry_run_pipelinenow owns its graph transport — graph generation no longer depends on the host'stracing_config. The function requests a graph explicitly (generate_graph=True) but relied on the host's configured tracing backend as the emit/assemble channel, so a host withtracing_config.is_enabled = false(e.g. pipelex-api's/validatein direct mode, where the web-app's Dry Run button lands) always got "Pipeline execution did not produce a graph spec" and a response withoutgraph_spec. The run now traces through a scopedInMemoryEventLog— the graph comes back under any tracing config, and validation dry-runs stop writing NDJSON/DynamoDB trace events as a side effect when tracing IS enabled. The no-graph contract violation is now a typedDryRunGraphNotProducedError(was a barePipelexError, ambiguous in host logs that record only the exception type) — raised by bothdry_run_pipelineanddry_run_pipe_in_process. - tz-aware datetime payloads no longer fail to decode on hosts without a timezone database — kajson bumped to 0.7.0. kajson ≤ 0.6.0 decoded timezone-aware datetimes via
ZoneInfo(...), which needs the IANA tz database: system tz files or thetzdatapackage. On hosts with neither (slim containers, uv-managed standalone Pythons — including the CI test runner, wheretzdataonly arrived transitively via pandas on Python < 3.11), decoding any aware-datetime payload raisedZoneInfoNotFoundError. Fixed upstream in kajson 0.7.0: its self-sufficient timezone wire format carries the UTC offset (decoding degrades gracefully to a fixed offset instead of raising, and plain UTC decodes with zero tz-database dependence), and kajson now declarestzdataas its own runtime dependency, so a tz database is always available for named zones.
Added
- Protocol
validate/models/versiononPipelexMTHDSProtocol—validatewrapsvalidate_bundle(blueprints + per-pipe structures into the protocolValidationReport);modelswraps the builder'slist_modelsinto aModelDeck;versionreportsprotocol_version0.6.0 with the installed pipelex version.
[v0.32.1] - 2026-06-09
Added
render_cost_report_for_output(pipe_output)submitter helper. A one-arg convenience overrender_run_cost_reportthat unpackspipeline_run_idandtokens_usagesfrom a finishedPipeOutputand derives the--costsgate from the output itself —pipe_output.tokens_usages is Noneis exactly the signal that cost reporting was off for the run, the decision the runner already resolved (with all--costs/--no-costsoverrides applied) and recorded on the output. Embedders and thepipelex runCLI no longer re-derive the three primitive arguments by hand or re-read global config to reconstruct the gate.render_run_cost_reportis unchanged as the low-level primitive for paths where the three values come from different places (the agent CLI'sbuild_cost_summaryJSON envelope, distributed reassembly).
Changed
typeris now capped (>=0.16,<0.27). pipelex subclasses typer'sTyperGroupand overridesmake_context, so it is coupled to typer/click internals and a future typer minor can break the CLI. The cap is tested through 0.26; a new dependency-canary CI job resolves the latest dependencies on a schedule so the bound is bumped deliberately rather than discovered in a user's install.
Fixed
- CLI no longer crashes under newer typer/click. Every
pipelexsubcommand (build,run,validate,init,worker, …) raisedRuntimeError: There is no active click contextand exited 1 when run againsttyper >= 0.26/click >= 8.4— the versions a freshpip install pipelexresolves. The rootapp_callbackfetched the context via the globalclick.get_current_context()instead of thectxTyper already injects; the global context stack isn't populated when a subcommand is dispatched under those versions. It now uses the injectedctx, which is version-robust. The sibling--tracebackflag had the same flaw —is_traceback_requested()read it viaclick.get_current_context(), so under those versions it was silently ignored on handled errors — and is now recorded at parse time in a context-independent flag. A subprocess CLI smoke test (tests/unit/pipelex/cli/test_cli_entrypoint_smoke.py) guards every subcommand against regressing.
[v0.32.0] - 2026-06-09
This cycle reworks cost reporting so it stops leaking. Cost now rides on the run result instead of a side buffer: the success-path registry leak is gone by removal, and a new --costs switch decouples cost collection from --graph. It also extracts the framework-agnostic runtime bridge so any host runtime — not just Mistral Workflows — can embed Pipelex through one boundary.
Added
-
--costs/--no-costs(default on).pipelex run pipe|method|bundlegains a dedicated cost switch that emits usage tracing events and renders the end-of-run cost report. It rides the shared trace-event transport independently of--graph, so--no-graph --costsreports cost without building a graph and--graph --no-costsbuilds a graph with no cost report. It replaces the removed--cost-report(see Changed). -
tokens_usagesonPipeOutput. Newtokens_usages: list[AnyTokensUsage] | Noneandusage_assembly_error: str | Nonefields, mirroringgraph_spec/graph_assembly_error. Cost is now part of the run result and is exposed automatically wherever aPipeOutputis returned, including the Pipelex API response. Render it withCostRegistry.generate_report(tokens_usages=...). -
--mock-inference. A LIVE run that fakes AI calls at the inference leaf with reportable synthetic usage, so cost reporting can be validated cheaply and deterministically without billing tokens. Mutually exclusive with--dry-run. It covers the LLM leaf; image-generation / extract / search under--mock-inferencefail loud withMockInferenceUnsupportedError(pointing at--dry-run) rather than silently calling the real provider. -
Cost report in the agent CLI JSON.
pipelex-agent run ... --with-memoryattaches a best-effort structuredcost_report({total_cost, by_model}, real USD) to its JSON envelope when the run did reportable work and summary aggregation succeeds — treat it as optional (absent for dry runs,--no-costs, the API-runner path, or an aggregation failure). The agent surface stays JSON-only — no Rich table on stderr; compact mode is unchanged.
Changed
-
Mistral Workflows integration extracted into a dedicated package. The optional
pipelex[mistralai-workflows]extra and thepipelex.plugins.mistralai_workflows.*modules have been removed frompipelex. The Mistral Workflows integration now ships separately, as our Mistral Workflows plugin. The framework-agnostic runtime-bridge core (boundary types,run_pipe_via_bridge,PipelexExecutionMode,ensure_pipelex_booted) has been promoted frompipelex.plugins.mistralai_workflows.*topipelex.runtime_bridge.*so any host runtime — not just Mistral Workflows — can embed Pipelex. No behavior changes; activities, boundary types, and execution modes are identical. -
--cost-reportremoved, folded into--costs. Breaking:--cost-report/--no-cost-reportis gone fromrun pipe|method|bundle. Use--costs(default on) instead. -
Cost reporting is event-sourced; the submitter-side
UsageRegistryis removed. The cost report is rendered fromPipeOutput.tokens_usages, not from an in-process registry.ReportingProtocol(andReportingNoOp) no longer exposeopen_registry/close_registry/generate_report/inject_tokens_usages, and theUsageRegistrymodel is gone. Embedders that rendered cost viaget_report_delegate().generate_report()must render from the returnedpipe_output.tokens_usagesinstead. -
is_log_costs_to_consolenow defaultstrue. With--costson, the CLI prints the cost table at end of run by default (parity with--graphproducing visible output). Per-inference-job console logging is removed — the report renders once at the end, not per call. Library embedders who don't want console output set itfalse. -
Dry runs never emit a cost report; free-model runs still do. Suppression keys on whether the run did reportable work (any tokens or any cost), not on total cost alone: a dry run (zero tokens, zero cost) is suppressed, while a real run on a free / zero-price model (e.g. Ollama) still reports its token usage with a zero total.
Fixed
-
Docs deploy crashed on a fresh runner with
InferenceSetupRequiredError.pipelex-dev generate-error-pages(a prerequisite of everydocs-*make target) bootstrapped Pipelex withneeds_inference=True, so on a CI runner with the gateway enabled but no on-disk service config it hit the first-run inference-setup gate and exited non-zero — breaking theDeploy docsworkflow onmain. The command only introspectsPipelexErrorsubclasses to write markdown and never calls inference, so it now bootstraps withneeds_inference=False. -
UsageRegistrysuccess-path leak. The per-run cost registry was opened during run setup but closed only on the failure path, so every successful run leaked its registry — and in a long-lived process (e.g. the Pipelex API) reusing apipeline_run_idcould collide on the orphaned registry. The registry is removed entirely, so the leak is structurally impossible.
[v0.31.0] - 2026-06-04
This release hardens Pipelex at its edges. The headliners: a full error-handling overhaul that gives every error a stable, RFC 7807-shaped identity and carries it all the way out to webhooks and external surfaces; lenient validation with pipe signatures, so you can sketch and dry-run a whole pipeline top-down before a single pipe is implemented; and a first, experimental cut of CSV tabular support for reading and writing typed lists straight from .csv.
Added
-
Stable error identity (RFC 7807). Every
PipelexErrornow exposestitle()and a stabletype_uri()— a real, dereferenceablehttps://docs.pipelex.com/latest/errors/<error>/URL — and everyErrorReportcarries both as populated fields, so consumers readreport.title/report.type_uridirectly instead of humanizing class names.ErrorReport.to_problem_document()renders anapplication/problem+jsondocument with no web-framework dependency, and a newDisclosureMode(VERBOSE/STRICT) controls how much leaks onto external surfaces:STRICTdrops provider/model attribution and redacts any message that wasn't authored to be caller-facing, keeping only the stable identifiers. -
Structured error payloads on failure webhooks. When a run fails, the delivery webhook now includes an
errorobject — the fullErrorReportas a dict — so receivers can rehydrate it withErrorReport.from_dict(...), render an RFC 7807 response, or route onerror_domain/retryable. AWebhookTarget.payloadthat collides with a Pipelex-reserved key (pipeline_run_id,status,result_url,error) is now rejected at construction. -
Per-class error reference pages. Every
PipelexErrorsubclass now has a generated reference page underdocs/errors/, surfaced in the docs site as "Error Reference" — so atype_uridereferences straight to a populated page. Regenerate with the newpipelex-dev generate-error-pagescommand (make generate-error-pages, aliasmake gep); pages a maintainer claims with a<!-- pipelex:authored -->marker are preserved across runs. -
request_idonJobMetadata. An optional caller-supplied request id, threaded through the run and into the log context. Set it at dispatch withpipeline_run_setup(..., request_id="...")and read it back offjob_metadata.request_id. -
PipeSignature— contract-only pipes for top-down design. A new pipe type (type = "PipeSignature") declares a pipe'sinputs,output, anddescriptionwith no implementation, so an author or agent can sketch a complete pipeline before committing to the operator that will eventually do the work. At dry-run time a signature mints a mock output matching its declared type and multiplicity; at runtime it raisesPipeSignatureNotExecutableError. The optionalsignature_forfield hints which pipe type the stub stands in for. See Signature Pipes. -
Lenient validation with
--allow-signatures.pipelex validate pipe|bundleand everypipelex-agent validatesubcommand take--allow-signatures(default off) to dry-run a pipeline that still containsPipeSignaturestubs. Strict by default: without the flag, validation refuses any pipeline whose dependency graph reaches a signature and raisesSignaturesNotAllowedError, reporting every reachable stub plus the controller chain that leads to it — so you know exactly which pipes are still placeholders. -
CSV tabular support (experimental). Read a
.csvstraight into a typedListContent[YourConcept]: point aninputs.jsonreference at a.csvwhose column headers match the concept's field names, and each row becomes an instance, with cells coerced via Pydantic (birth_year→int, ISO dates →date; an empty cell becomesNone, so the field must be optional). Write a flat-list output back out with the new--save-csv <path>flag onpipelex run pipe|bundle|method. v1 is deliberately narrow — local file paths only (remote URLs with a tabular suffix are rejected), the row concept must be flat (scalar fields only; a nested/list/concept-typed field is rejected with a clear error naming it), and.xlsxis recognized but routed to a "needspipelex[tabular]" message (the Excel backend isn't built yet). See the CSV Input & Output guide. -
--tracebackCLI flag for full stack traces. By default CLI commands print a friendly one-line error;--tracebackalso prints the Rich-rendered stack trace before it. It's position-agnostic —pipelex run --traceback pipe ...andpipelex run pipe ... --tracebackboth work. -
Test tooling for hanging runs. New
make agent-test-debug(aliasmake atd) runs the suite with upfront stale-process cleanup, an outer wall-clock timeout, and live per-test logging — for whenmake agent-testhangs or fails opaquely. Paired with a debugging playbook atdocs/agents/debugging-hanging-pytest-runs.md.
Changed
-
ErrorReportis now a frozen PydanticBaseModel(was a frozen Pydantic dataclass) — still immutable and still round-trips throughto_dict()/from_dict(), but an attempted mutation now raisespydantic.ValidationErrorinstead ofdataclasses.FrozenInstanceError. -
Dry-run and validation consolidated into
BundleValidator. The standalonedry_run_pipe/dry_run_pipesfunctions and the modulespipelex/pipe_run/dry_run.py,dry_run_with_graph.py, anddry_pipe_router.pyare gone; their work now lives inBundleValidator, andvalidate_bundle/BundleValidatorgained theallow_signatures: bool = Falseflag (strict by default). Validation ordering and outputs are unchanged — only the import surface moved. Callers importing frompipelex.pipe_run.dry_runmust switch toBundleValidator. -
validated_pipesnow identifies every pipe by its qualifiedpipe_ref(domain.code). Previouslyvalidate pipereported the bare code whilevalidate bundle/validate allreported the namespaced ref, so the same pipe could appear under two identities depending on which command produced it. Every validate surface now emits the qualified ref, which can't collide across domains. Consumers that parsedpipe_codeexpecting the bare form (e.g. the MTHDS skills) must match on the namespaced ref. -
Gemini deck refresh. Google shut down
gemini-3-pro-preview, so thegemini-3.0-prohandle is removed across thegoogle,portkey, andopenrouterbackends (and the test-profile collections) — usegemini-3.1-pro(→gemini-3.1-pro-preview) instead, now registered onportkeytoo for parity withgoogle. Thegemini-3.0-flash-previewhandle is renamed togemini-3.0-flash. The Pipelex Gateway lists models from its own remote config, so dropgemini-3.0-prothere separately. -
Filesystem path helpers moved to
pathlib.Path. Helpers inpipelex.tools.misc.file_utilsandpipelex.tools.misc.json_utils(save_text_to_path,load_json_from_path,load_binary,copy_file,ensure_directory_exists,get_incremental_file_path, and the rest) now take and returnPathinstead ofstr; path handling isPath-based throughoutpipelex/, converting to/fromstronly at boundaries. Callers passing bare strings must wrap them withPath(...). -
teardown_current_library()renamed toclear_current_library()inpipelex.hub, pairing it cleanly withset_current_library(). -
Dev tooling:
pyrightbumped1.1.408 → 1.1.410(drove two behavior-neutral internal adjustments).
Fixed
-
The generated MTHDS schema now requires
typeon every pipe. In the schema consumed byplxtlint and the VS Code Taplo LSP, each pipe blueprint variant declaredtypewith a literal default — so it was optional, and because the Draft-4 export drops the union discriminator, a pipe table written withouttypematched severaloneOfbranches at once and was rejected with an ambiguous multi-match (worse, a type-less table carrying fields unique to one variant validated silently).typeis now required on every variant, so a type-less pipe table fails with a clear "missing type" and a typed table resolves to exactly one variant. The patched set derives fromPipeBlueprintUnion, so new pipe types are covered automatically. Regenerate withpipelex-dev generate-mthds-schema. -
InputStuffSpecsFactoryErrorwas shadowed by a duplicate class definition.input_stuff_specs_factory.pydeclared a local class with the same name as the canonical one inexceptions.py, leaving two distinct class objects in play so anexcepton one would miss the other. Consolidated to the single canonical class.
Security
- urllib3 floor
>=2.7.0to patch two high-severity issues: CVE-2026-44431 (GHSA-qccp-gfcp-xxvc) forwards sensitive headers across origins on proxied low-level redirects, and CVE-2026-44432 (GHSA-mf9v-mfxr-j63j) bypasses the decompression-bomb safeguards in parts of the streaming API. Floor added to the runtimedependencies. - pymdown-extensions floor
>=10.21.3to patch CVE-2026-46338 (GHSA-62q4-447f-wv8h, medium): a regression inpymdownx.snippetsreintroduced the sibling-prefix path-traversal bypass despiterestrict_base_path. Docs-only; floor added to thedocsextra. - idna floor
>=3.15(lockfile resolves to 3.18) to patch CVE-2026-45409 (GHSA-65pc-fj4g-8rjx, medium): specially crafted inputs toidna.encode()could bypass the CVE-2024-3651 fix. Floor added to the runtimedependencies. - Proactive floor bumps in the same hardening pass, past known-vulnerable releases: pillow
>=12.1.1, protobuf>=6.33.5, python-dotenv>=1.2.2, requests>=2.33.0(runtime); Pygments>=2.20.0(docsextra).
[v0.30.3] - 2026-05-28
Added
claude-4.8-opusmodel added to theanthropicandbedrockbackends. Adaptive thinking mode, supports text / images / pdf in and structured outputs,max_prompt_images = 100,max_tokens = 128000, costs{ input = 5.0, output = 25.0 }per million tokens. Carries thetemperature_unsupportedconstraint likeclaude-4.7-opus. Also available via the Pipelex Gateway.
[v0.30.2] - 2026-05-26
Changed
pipelex-agentmarkdown error envelope no longer includes the## Error sourcestack-frame section. Markdown is the agent / human-facing channel; internal frames likeLibraryError @ pipelex/libraries/library.py:140are noise for an LLM trying to fix a.mthdsfile and forced every consumer (e.g. themthds-pluginsvalidate hook) to strip them. Theerror_sourcefield remains in the JSON envelope (--error-format json) untouched for programmatic consumers — same shape, same cause-chain ordering — so any tooling that parses JSON keeps the full diagnostic surface. The## Detailssection (which carrieserror_domainand other structured fields) is unchanged. No change to error categorization,error_domainvalues, message wording, or stdout/stderr routing.
[v0.30.1] - 2026-05-26
Fixed
pipelex-agentnow silences every Python logger on stderr regardless of user TOML. The agent CLI is machine-consumed: stdout is reserved for the structured success envelope (JSON / markdown) and stderr for the structured error envelope. Free-floatinglog.*calls — alog.debugfromtelemetry_factory.py, alog.warningfromvalidation_error_categorizer.py, or an INFO/WARNING line from any third-party dep (anthropic,httpx,botocore,openai, anything a transitive dep configures) — would corrupt the stderr channel for downstream parsers (mthds-js'sPipelexRunnerdoingJSON.parse(stderr)on the validate hook). Two layers, defense-in-depth:- Layer 1 — pin pipelex's own logs off via config.
make_pipelex_for_agent_cliinjectsconfig_overridesintoPipelex.make()that pindefault_log_level = OFFandpackage_log_levels.pipelex = OFFfrom the very firstlog.configurecall. A user setting[pipelex.log_config.package_log_levels] pipelex = "DEBUG"in~/.pipelex/pipelex.tomlcan no longer leak its own DEBUG/INFO/WARNING lines. - Layer 2 — process-global cutoff that covers every logger. New
silence_logging_for_agent_cli()callslogging.disable(sys.maxsize)— a process-global threshold checked insideLogger.isEnabledForBEFORE any per-logger level. No record gets created for any logger at any level (including custom levels aboveCRITICAL), regardless of which package emits or what level the user configured. Wired into the Typerapp_callbackso every subcommand — includinginitandaccept-gateway-terms, which bypassmake_pipelex_for_agent_cli— silences logging before any command body runs; idempotent per-call invocations remain insidemake_pipelex_for_agent_cliandagent_doctor_cmdas belt-and-braces for direct library callers.
A new e2e regression test pins the contract by setting anthropic, httpx, botocore, openai to DEBUG in the user TOML and asserting both stdout and stderr stay clean.
Changed
-
pipelex-agentno longer accepts--log-level. Log suppression is unconditional by design — there is no verbosity setting on the agent CLI. Thelog_levelparameter is removed frommake_pipelex_for_agent_cli()andapply_agent_cli_output_discipline(), and the correspondingctx.obj["log_level"]plumbing is gone from every caller. For verbose debugging, use the humanpipelexCLI, which honors the user's TOML log config. -
pipelex/cli/commands/doctor_cmd.py::setup_doctor_runtimenowdeep_updateslog_config_overridesinto the loaded config (was a flat-merge that silently replaced nested dicts likepackage_log_levels). With the flat merge, pinningpackage_log_levels.pipelex = OFFfor the agent doctor path would have wiped out all the third-party package levels (anthropic,asyncio,botocore, ...) shipped in the default config. The deep merge preserves them. -
Removed
ctx: typer.Contextfrom 8 agent CLI commands that no longer used it.validate/{pipe,bundle,method}_cmd.py,inputs/{pipe,bundle,method}_cmd.py,models_cmd.py,check_model_cmd.pyhadctx.obj["log_level"]as their only ctx usage; with--log-levelgone, the parameter became dead weight. Therun/commands keepctxbecause they still readctx.obj["runner"]. Test callers (and the now-unusedagent_ctxconftest fixture) updated to match.
[v0.30.0] - 2026-05-25
Fixed
-
console_log_targetpackage default is nowstderr(wasstdout). Logs now stay off the data channel by default, matching the intent of PR #452 ("default to stderr for outputs happening before initialization"). Downstream tooling that parsespipelex/pipelex-agentstdout as JSON (e.g.mthds-js'sPipelexRunner) is no longer at risk of stdout pollution from package-level logs — the bug was latent for stock installs because the agent-CLI JSON paths happen not to log at INFO+, but surfaced for anyone who raisedpackage_log_levels.pipelexto DEBUG or added a setup-time log on the command path. The same flip is applied to the kit template (pipelex/kit/configs/pipelex.toml) thatpipelex initcopies to~/.pipelex/. Note:console_print_targetis intentionally left atstdout— the mainpipelexCLI emits human-facing tables (show backends,show models,which,doctor) via that channel, and downstream piping (pipelex show backends > out.txt) must keep working. -
pipelex-agentnow pins both console targets tostderrregardless of user config.make_pipelex_for_agent_cliinjectsconfig_overridesintoPipelex.make()that forceconsole_log_target = "stderr"andconsole_print_target = "stderr"from the very first log/print fired during init, so a user override of either knob in~/.pipelex/pipelex.tomlcan no longer leak diagnostics onto the agent CLI's JSON data channel. Defense-in-depth post-init calls tolog.redirect_to_stderr()andget_pipelex_hub().set_console_print_target(STDERR)remain. A new adversarial E2E test (tests/e2e/agent_cli/test_stdout_is_clean_json.py::test_models_json_stdout_resists_user_targets_override_to_stdout) pins the contract by overriding both targets tostdoutandpackage_log_levels.pipelextoDEBUGand assertingjson.loads(stdout)still parses.
[v0.29.1] - 2026-05-21
Fixed
pipelex runnow prints the aggregated cost table when[pipelex.reporting_config].is_log_costs_to_console = true, with--cost-report/--no-cost-reportto override per invocation. Thecost-tracking.mdandreporting-config.mddocs both promised a summary cost table at the end of a CLI run, butpipelex/cli/commands/run/_run_core.pynever calledget_report_delegate().generate_report()— the flag only triggeredlog.verbose(...)lines per inference job, which the defaultINFOlog level swallows, so the table never appeared. The CLI now callsgenerate_report()after a successful run when eitheris_log_costs_to_consoleoris_generate_cost_report_file_enabledis true, so the Rich table (one row per model, plus a totals row) prints right before the "✓ Pipeline execution completed successfully" recap — and the CSV export branch finally fires too. The new--cost-report/--no-cost-reporttri-state flag (default unset → use config) lets you force the cost table on for a single invocation (--cost-report) or skip reporting entirely — no Rich table and no CSV file (--no-cost-report) — without touching.pipelex/pipelex.toml. Applies topipelex run bundle,pipelex run pipe, andpipelex run method; works in dry-run mode as well (synthetic usage rows).
[v0.29.0] - 2026-05-20
Added
-
gemini-3.5-flashmodel added to thegooglebackend. Adaptive thinking mode, supports text / images / pdf in and structured outputs,max_prompt_images = 3000, costs{ input = 1.5, output = 9.0 }per million tokens. -
New
PipeStructureoperator that turns text into a structured concept via a single LLM call. Takes oneText-compatible input (or a domain concept thatrefines = "Text"), produces any structured output with the usual multiplicity options (Foo,Foo[],Foo[N]). Useful whenever the source text comes from a PDF extraction, a search result, an upstream pipe, or any non-LLM origin. Documented at building-methods/pipes/pipe-operators/PipeStructure.md. -
Documented
render_jsandinclude_raw_htmlonPipeExtract. Both fields were already shipping onPipeExtractBlueprint; only docs were missing.render_js = trueasks the extraction backend to render JavaScript before fetching web-page content;include_raw_html = truepopulates each extractedPage'sraw_htmlfield with the fetched HTML. Added to building-methods/pipes/pipe-operators/PipeExtract.md. -
Documented
XHIGHvalue onReasoningEffort. The enum already shipped seven levels; theunder-the-hood/reasoning-controls.mdtable listed only six.XHIGHsits betweenHIGHandMAXand maps to provider-specific xhigh values where supported. -
Bounded fan-out concurrency for
PipeBatch. APipeBatchover many items no longer spawns every branch — every coroutine, every deep-copied working memory, every inference call — at once. Branches now run in bounded chunks driven by the new[pipelex.pipeline_execution_config]settingmax_concurrency(default8; set to the literal"unbounded"for unbounded fan-out, the previous behavior). It keeps a large workload (one pipe over thousands of documents) from overwhelming asyncio, memory, and provider rate limits. Results still preserve input order, and a branch failure still propagates (first error by input index wins).PipeParallel, which fans over a fixed pipe-defined branch set, is unchanged. -
Authoritative
error_domain→ HTTP-status mapping.pipelex/base_exceptions.pynow owns the mapping that downstream HTTP APIs need to render anErrorReportas an HTTP response:error_domain_to_http_status()(the pure domain table —INPUT→ 422,CONFIG/RUNTIME/unknown → 500) and theErrorReport.http_statusproperty, which adds a provider-429 passthrough so the API can emit aRetry-Afterheader fromprovider_metadata.retry_after_seconds. The library stays HTTP-agnostic — no web-framework dependency, just the mapping table; downstream FastAPI handlers call the helper instead of reinventing the contract. -
AMBIGUOUSinference-error category for outcome-uncertain failures. NewInferenceErrorCategory.AMBIGUOUSfor failures where the error type is known but the outcome is not — the operation may or may not have committed (e.g. a connection dropped mid-request). It is non-retryable, likeCONFIGURATION/CONTENT/CAPACITY, but semantically distinct fromUNKNOWN, which means the error could not be classified at all. Theazure_restimage-generation worker now raisesAMBIGUOUSfor mid-request transport failures —ReadError/WriteError/RemoteProtocolErrorandReadTimeout/WriteTimeout— so an automatic-retry layer won't re-fire a non-idempotent, billable image submit whose outcome is unknown; pre-request failures (ConnectError/ConnectTimeout/PoolTimeout) stayTRANSIENTand retryable. -
Explicit, uniform Tier 1 transport retry across every inference worker. A new
[cogt]settingtransport_max_retries(default2) is now wired explicitly into every inference SDK client factory — Anthropic, OpenAI / Azure OpenAI, the Portkey-backed gateway clients, Mistral, and Google — instead of each factory silently inheriting whatever retry posture its SDK happens to default to. The two SDK families that default to no transport retry are brought up to the same floor: the Mistral client gets a bounded-backoffRetryConfig(retry_connection_errors=True) and the Google GenAI client getsHttpOptions(retry_options=...). The genuinely SDK-less path — theazure_restimage-generation worker, which talks to Azure over rawhttpx— gets atenacity-based transport-retry wrapper (new modulepipelex/cogt/inference/transport_retry.py) that retries connection failures and transient HTTP statuses (408/409/429/5xx) and honorsRetry-After. When noRetry-Afterheader is present, the fallback backoff uses full jitter (wait_random_exponential) so a burst of failures retried together does not re-fire in lockstep. On a non-idempotent submit-style POST it narrows the retry to failures that prove the server did no billable work — a request that was never delivered, or a 408/429 rejection — withholding an ambiguous 5xx, a 409 conflict, and a post-delivery timeout such as aReadTimeout. Transport retry is now a deliberate, configured, uniform policy rather than a per-provider accident. This is the transport-level floor of the retry model — pipeline execution makes a single attempt on top of it. (Theportkey-aiSDK does not expose a retry knob — it carries its own internal retry — so the gateway'sAsyncPortkeyclient is left as-is; only its underlying OpenAI clients are wired.) -
Offline mode for Pipelex Gateway setup and dry-run. When the gateway is enabled but the remote config service is temporarily unreachable, Pipelex now falls back to a previously primed on-disk cache (
~/.pipelex/cache/remote_config.json, schema-versioned) instead of failing setup outright. Dry-run, validation, andpipelex-agent run bundle --dry-runcomplete normally; only the actual inference call still needs the network at runtime. The cache is primed on every successful fetch and onpipelex initwhile online. When the gateway is disabled (BYOK), no remote fetch is attempted at all — setup is fully offline. A newRemoteConfigStaleWarning(UserWarning) is emitted whenever stale cache is in use; the agent CLI surfaces it on the JSON envelope aswarnings: [{"type": "RemoteConfigStale", ...}]. Telemetry is suppressed (no-op) when running on a cached config so stale model identities don't pollute metrics. The doc/fixture generators (pipelex-dev update-gateway-models,preprocess_test_models_cmd) refuse the cache fallback via a newrequire_fresh=Trueflag, so committed reference docs and test fixtures never bake in stale data. -
GatewayUnknownModelError(pipelex.cogt.exceptions). Raised at setup time when the active model deck references a gateway model handle that isn't present in the (fresh or cached) gateway specs. Carries the model name and the config source (RemoteConfigSource.FRESH|CACHED); the message branches on source so a cached-source failure suggestspipelex initwhile online and a fresh-source failure points at deck/typo fixes. Wired through both the Rich CLI (handle_gateway_unknown_model_errorinerror_handlers.py) and the agent CLI (AGENT_ERROR_HINTS/AGENT_ERROR_DOMAINS). -
RemoteConfigUnavailableError(pipelex.system.pipelex_service.exceptions). User-facing offline-mode error: raised only when the network fetch fails AND no usable cached fallback exists. The message names the cache file path and the two remediation paths (runpipelex initwhile online to prime the cache; or disablepipelex_gatewayinbackends.tomlfor permanent BYOK operation). Distinct from the internalRemoteConfigFetchError, which is kept as the retry-layer exception. -
PIPELEX_REMOTE_CONFIG_URLenvironment variable. Overrides the default remote-config URL. Useful for staging/testing environments; defaults to the production URL when unset.
Changed
-
Inference error handling refactored to Extract / Classify / Render (internal). Every inference worker's SDK-exception block now collapses to
metadata = extract_*_metadata(exc); classification = classify_inference_error(metadata); raise render_*_error(...) from exc. New modules:pipelex/cogt/inference/error_classify.py(single sharedclassify_inference_error()returningClassificationResult(category, user_action_kind, is_model_not_found)),pipelex/cogt/inference/error_render.py(single sharedrender_llm_error/render_img_gen_error/render_extract_error/render_search_error, picking theCogtErrorsubclass from anInferenceErrorFamilytag plus theis_model_not_foundflag), andpipelex/cogt/inference/provider_name.py(theProviderNameenum keying the extract-fn registry).ProviderErrorMetadatagains amessagefield plusis_quota_exhaustion/is_content_policy_violation/is_network_error@propertyaccessors; the per-providerextract_*_metadatafunctions are now the only plugin-local piece — Classify and Render live once. Mistral and the gateway-search worker specialize HTTP 404 toExtractModelNotFoundError/SearchModelNotFoundError; Azure img-gen keeps two worker-specificAMBIGUOUSbranches for mid-request transport failures. Removed: the per-provider*_error_classification.pymodules and their tests,AnthropicCredentialsError,GatewayFactory.classify_error_category/make_user_action_from_portkey_error/make_error_summary_from_portkey_error, and every inline_classify_*_error/_raise_categorized_*worker method. New unit-testtests/unit/pipelex/cogt/inference/test_provider_classification_parity.pywalks everyProviderNameagainst the extract-fn registry + worker-family map, so an unwired new provider fails fast. No user-facing API change — the structuredErrorReportcontract is unchanged. Documented at under-the-hood/error-model.md. -
instructor's structured-output retry no longer re-runs completions on transport errors. Passed a bareint,instructor'smax_retriesbuilds a retry loop whose predicate retries any exception — so the structured-output path (PipeLLM/PipeStructure) was re-running the whole completion on transport / API errors, a second retry loop nested on top of the SDK client's own transport retry. Eachinstructorcall site now passes atenacity.AsyncRetrying(built by the newpipelex/cogt/llm/instructor_retry.pyhelper) whose retry predicate matches only validation failures (pydantic.ValidationError,json.JSONDecodeError, andinstructor's own validation-error types). A transport error now propagates immediately as the raw SDK exception for the worker'sexceptclause to classify — transport retry is the SDK client floor (Tier 1) alone, andinstructor's retry is confined to genuine schema re-ask. As part of this the Google and Mistral structured-generation workers gained anhttpx.TransportErrorexceptclause: those SDKs let raw connection / timeout errors propagate outside their own exception hierarchies, so withinstructorno longer wrapping them they must be caught and classified directly. The Mistral structured path also now gets schema re-ask at all — it previously passed nomax_retries. -
Inference schema-retry setting moved and renamed:
[cogt.llm_config.llm_job_config] max_retries→[cogt.llm_config] schema_reask_max_attempts(breaking). The setting isinstructor's schema re-ask budget for structured-output validation failures. The old namemax_retriesgave no hint of that scope and collided conceptually with the new top-levelcogt.transport_max_retries— which counts retries beyond the initial attempt, whereas this one is a total attempt count (stop_after_attempt). The new name makes both the scope (schema re-ask) and the unit (attempts, not retries) explicit. The single-key[cogt.llm_config.llm_job_config]sub-table is also dropped — the setting now sits directly under[cogt.llm_config]. Projects that override this key in their ownpipelex.tomlmust move and rename it. -
Agent CLI
run/validate/initdefault to markdown output, with independent success/error format options (breaking). These commands previously always emitted JSON; they now accept--format markdown|jsonand default to markdown, matchingmodels/check-model/doctor. A second flag--error-format markdown|jsoncontrols error reporting on stderr independently from success output — it defaults to the value of--format, so--format jsonstill flips both, but the two can now be set separately (e.g.--format markdown --error-format jsonfor human-readable success with machine-parseable errors). Internally, only the error format is carried in aContextVar; the success format is threaded explicitly toagent_success_formatted(). Theinputs/concept/pipe/fmt/lint/accept-gateway-termscommands are unaffected (always JSON / raw passthrough). -
pipelex-agent validate bundlegraph-format option renamed--format→--graph-format(breaking). The--format/-fflag that selected the graph renderer (mermaidflow/reactflow/both) is now--graph-format/-f, freeing--formatto be the uniform markdown/json output-format flag across every agent-CLI command. -
structuring_method = "preliminary_text"onPipeLLMworks again, via build-time elaboration. WhenPipelexInterpreterparses a.mthdsfile with aPipeLLMcarryingstructuring_method = "preliminary_text", the newBundleElaboratorrewrites it before any pipe runs into aPipeSequenceof two synthetic pipes: aPipeLLMproducingText(step 1, inheriting the original prompt + inputs + model) and aPipeStructureproducing the original output (step 2, optionally usingmodel_to_structure). The synthetic codes are tracked in aPipelexBundleBlueprint.elaboration_metadataside-table (excluded from serialization). Output multiplicity is preserved: step 1 always emits a singleText; step 2 emitsFoo,Foo[], orFoo[N]according to the original output. Two LLM calls are issued per invocation. The user-facing pipe code is unchanged, so callers,main_pipe, and the run API keep working as before. Mechanism documented at under-the-hood/build-time-elaboration.md. -
PipeLLMruntime no longer knows aboutstructuring_method. The field is now a pure build-time directive. The runtimePipeLLMclass no longer carriesstructuring_method, the validator that rejected mismatched output concepts has moved toPipeLLMBlueprint(where it fires at parse time), and theNotImplementedErrorpreviously raised at runtime whenpreliminary_textwas selected is gone — the elaborator handles it.PipeLLMSpecexposesstructuring_methodso AI agents authoring via specs can still opt in. -
StructuringMethodenum import path moved. The enum is now defined inpipelex.pipe_operators.llm.pipe_llm_blueprint(it lives next to the blueprint that consumes it) instead ofpipelex.pipe_operators.llm.pipe_llm. Direct importers must update theirfrom pipelex.pipe_operators.llm.pipe_llm import StructuringMethodline tofrom pipelex.pipe_operators.llm.pipe_llm_blueprint import StructuringMethod. -
Retry removed from the gateway workers;
[cogt.tenacity_config]removed (breaking). Thetenacity-based retry insideGatewayExtractWorkerandGatewaySearchWorkeris gone, along with the[cogt.tenacity_config]config block and itsTenacityConfigmodel. Because config models forbid unknown keys, an existing~/.pipelex/pipelex.toml(or any layered override) that still carries[cogt.tenacity_config]will fail to load — remove that block from your config. Transient transport blips are left to the provider SDK clients' own retry; thetenacitylibrary itself is still used elsewhere (FAL job polling, remote-config fetch) and remains a dependency. -
PipeRouter.run()now reportsCogtErrorfailures to the observer. ACogtErrorraised out of pipe execution previously propagated pastrun()without anobserve_after_failing_runnotification (onlyPipeRunErrorwas caught). It is now observed on the failing path, then re-raised as-is — the cause chain is preserved and it is not wrapped intoPipeRouterError(only thePipeRunErrorpath still wraps). -
Provider HTTP 404s now raise dedicated
*ModelNotFoundErroracross LLM, image-gen, extract, and search. A model-or-deployment-not-found 404 from any LLM or image-gen provider, or from the Pipelex Gateway extract / search workers, now raises a dedicated*ModelNotFoundError(LLMModelNotFoundError,ImgGenModelNotFoundError,ExtractModelNotFoundError,SearchModelNotFoundError— allModelNotFoundErrorsubclasses,CONFIGURATIONcategory) instead of a genericLLMCompletionError/ImgGenGenerationError/ExtractJobFailureError/GatewaySearchResponseError. Because these are siblings of the generic errors — not subclasses — a 404 propagates past the operator's genericexcepttoexcept ModelNotFoundErrorinPipeOperator._live_run_pipe, which re-raisesPipeOperatorModelAvailabilityErrorcarrying the unavailablemodel_handle. -
RemoteConfigFetcher.fetch_remote_config()now returns aRemoteConfigResultcarryingconfig,source(fresh|cached), andcached_at, instead of a bareRemoteConfig. Callers unwrap.configfor the payload and may branch on.sourceto know whether the config is fresh or restored from cache. The fetcher accepts a new keyword-onlyrequire_fresh: bool = False— whenTrue, a cached fallback raisesRemoteConfigUnavailableErrorinstead.RemoteConfigValidationErroris never satisfied by the cache (server-side schema breaks must surface loudly). -
ModelManager.setup()andBackendLibrary._load_gateway_model_specs()accept a newgateway_config_source: RemoteConfigSource | Noneparameter. Passed through fromPipelex.setup()so the deck-level gateway membership check can branch its error message onFRESHvsCACHED.GatewayConfigitself staysextra="forbid"and source-free — provenance is plumbed alongside, not baked in. -
RemoteConfigUnavailableErrormessage branches on whether the cache was refused vs absent. Whenrequire_fresh=True(dev-CLI generators) refuses to fall back to an existing cache, the message now reads "the local cache at<path>was refused because a fresh fetch is required" instead of the previously misleading "no local cache is available at<path>". The cache-truly-missing path keeps its original wording.
Fixed
-
Inference-failure
ErrorReports now carrymodelandproviderin production. A real LLM / image-gen / extract / search failure used to surface anErrorReportwithmodel = Noneandprovider = None: the leaf errors (LLMCompletionError,ImgGenGenerationError,ExtractJobFailureError,SearchJobFailureError,ExtractOutputError) carry no model or provider of their own, andCogtError.to_error_report()only duck-typed whatever attributes happened to be set on the exception. Each inference worker family now fillsmodel_handle/backend_namefrom the worker — where both are unambiguously known — at its public-method chokepoint (gen_text/gen_object,gen_image/gen_image_list,extract_pages,search_sourced_answer/search_structured), via the newCogtError.fill_model_and_provider(). The fill never overwrites a value an inner error already set and skips the"unknown"placeholder external plugins report.model_handle/backend_nameare now declared onCogtError(soto_error_report()reads them directly instead ofgetattr), making them uniformlystr | Noneacross the exception hierarchy. -
Two worker error-classification miscategorizations corrected.
LinkupNoResultError(a search or fetch that returned nothing) had no explicit branch in either Linkup worker's_classify_linkup_errorand fell through to theTRANSIENTcatch-all — marking a query that cannot succeed by retrying as retryable. It is now classifiedCONTENT+CHANGE_INPUTin both the search and the extract worker. Behavior change: a no-result search is now non-retryable (CONTENT.is_retryableisFalse), so an automatic-retry layer no longer retries a query that returns nothing on every attempt. Separately, theFileNotFoundErrorbranch in the Docling and pypdfium2 extract workers was classifiedCONFIGURATION— inconsistent with its sibling branches (CONTENT+CHANGE_INPUT) and with the rest of the codebase, whereCONFIGURATIONis reserved for setup problems. A missing input file is a content problem; it is nowCONTENT. This second change does not alter retry behavior —CONFIGURATIONandCONTENTare both non-retryable. -
Wrapped exceptions now surface the underlying inference error's classification.
PipelexError.to_error_report()enriches its report from the__cause__chain, so aPipelineExecutionError— or any wrapper around a transientCogtError— now reportserror_category,retryable,model, andproviderinstead of dropping them. Previously the agent-CLI JSON / markdown error output for a failed pipeline run lost the worker's category and retryability once the error had been wrapped by the pipe operator → router → runner layers, leaving an agent unable to tell a transient failure from a fatal one. -
MTHDS JSON schema no longer leaks the
elaboration_metadataside-table.PipelexBundleBlueprint.elaboration_metadatacarriedField(exclude=True)so it stayed out ofmodel_dump()/model_validate()round-trips, but Pydantic v2'sexclude=Truedoes not affectmodel_json_schema()— soderived/mthds_schema.jsonended up declaring a top-levelelaboration_metadataproperty plus$defs/ElaborationMetadataand$defs/StepRoleentries, even though the side-table is process-local in the reference runtime. Wrapped the field type withSkipJsonSchema[...]so it is hidden from the schema. After regenerating, the three entries are gone from the public schema, matching the spec's silence on them. -
A directory containing a
.pipelex/config dir is now recognized as a project root..pipelexwas added toPROJECT_ROOT_MARKERS, so project-level config (e.g..pipelex/inference/backends.toml) is honored even when the directory has no.git,pyproject.toml, or other source-project marker. Previously such a directory fell through to the global~/.pipelex/config, silently ignoring the project's own overrides — so a backend disabled in the project'sbackends.tomlcould still demand credentials because the global config (where it was enabled) was loaded instead. The home directory remains excluded from project-root detection, so the global~/.pipelex/is unaffected. -
PipeLLMoutputting a concept that refines the nativeJSONconcept no longer crashes withNameError: name 'Any' is not defined. On a LIVE run, such a concept resolves to a structured-output model carrying adict[str, Any]field inherited fromJSONContent.SchemaToModelFactorygenerates that model as source withfrom __future__ import annotations(every annotation becomes a string) and then rebuilds each class to resolve the string annotations. The rebuild namespace was assembled from the exec'd user types plus a hand-listedLiteral, buttyping.Anyis a special form, not atype, so it was filtered out —model_rebuildthen raisedPydanticUndefinedAnnotationevaluating"dict[str, Any]". The rebuild namespace is now the exec namespace itself (minus__builtins__), so it carries exactly the names the generated source was written against and cannot drift as codegen emits other typing constructs. Fixed inpipelex/cogt/content_generation/schema_to_model_factory.py; covers both the sender path (make_from_json_schema) and the cross-process receiver path (make_types_from_source).
[v0.28.0] - 2026-05-13
Changed
- Breaking (rendering): graph viewer assets are no longer bundled into generated HTML. The ReactFlow HTML output now references
@pipelex/mthds-ui(the standalone IIFE bundle + its CSS) andelkjsfromcdn.jsdelivr.netwith pinned versions andsha384Subresource Integrity hashes. Three external requests now load from jsDelivr at view time; the HTML itself is still generated offline by Pipelex — only viewing the rendered graph requires network access. The previousmake sync-graph-ui/make check-graph-ui-syncworkflow and the committedpipelex/graph/reactflow/assets/graph-viewer.{js,css}files have been removed; the manifest-stylepackage.jsonand theGraph UI sync checkGitHub Actions workflow are gone. A single Python module (pipelex/graph/reactflow/standalone_assets.py) is the source of truth for pinned versions and integrity hashes — the template reads them via Jinja2, so a bump touches one file plus the regenerated SRI constants. The newpipelex-dev refresh-graph-ui-sricommand re-fetches the URLs, recomputessha384, and rewrites the constants module; theupdate-graph-uiskill drives that flow. Previously, the "self-contained HTML" was already not offline (elkjs loaded fromunpkg.comwithout integrity), and bumping the vendored bundle meant committing ~2 MB of regenerated JS+CSS on every mthds-ui release — SRI hashes give the same tamper-evidence with none of the diff churn. - Breaking:
@var/@?varsigils must be alone on their own line, and inline@<ident>raises at load time only when<ident>collides with a declared input of the surrounding pipe. The@/@?rewriters wrap the value in a tag-shaped block envelope (<var>...</var>via thetag()filter), so the inline shapes never produced sensible output and are now rejected at load time with a newTemplateSigilSyntaxError— surfaced through pydantic validation with the line number, the offending span, and a migration hint. To stay friendly to coding agents authoring HTML/CSS templates (<style>@media (...) { ... }</style>,@font-face { ... }, Python/Java decorators like@deprecated/@Override), the validator is gated on the surrounding pipe's declaredinputs: an inline@<ident>raises only when the candidate's root identifier matches a declared input (a real typo). Every inline candidate on a line is inspected, so a later collision can't hide behind an earlier CSS at-rule. Other inline@candidates pass through silently. The line-bounded rewriter (alone-on-line@var→{{ var|tag("var") }}, leading/trailing whitespace preserved) is unchanged and remains inputs-agnostic. The strict rule replaces the heuristic regex tightening from the prior CSS-collision fix: there are no residual rewrites for@font-face/@counter-style/@view-transition/@property --x, no special-case lookbehind/lookahead arms. The$varinline sigil is unchanged. API:preprocess_template(template, *, declared_inputs=...)gained a keyword-onlydeclared_inputs: set[str] | Noneparameter (defaultNone= lenient, no validation); a new pair of helpersvalidate_template_sigils(template, declared_inputs)andrewrite_template_sigils(template)exposes the validate and rewrite steps separately.render_template(runtime) callsrewrite_template_sigilsdirectly — templates are already validated at load time, so the render path doesn't need thedeclared_inputscontext. All pipe blueprints (PipeLLMBlueprint,PipeComposeBlueprint,PipeSearchBlueprint,PipeImgGenBlueprint,PipeComposeFactory) and template analyzers (TemplateDocumentAnalyzer,TemplateImageAnalyzer) thread their declared inputs into the validator; the baseTemplateBlueprintandConstructBlueprint-discovery validators remain lenient (no inputs context at that layer — the surrounding pipe's validator catches typos). Migration: for inline values, switch from@varto$var; for block-shaped content, move the sigil onto its own line; for literal@or$characters, use the@@/$$escapes documented below.
Added
@@and$$template escapes. Authors can opt out of sigil interpolation per occurrence:@@varrenders as the literal string@var(no{{ ... }}), and$$varrenders as the literal$var. The escapes are non-overlapping left-to-right (@@@@var→@@var, two literal@s). Use these for any literal@/$the author needs to keep in the rendered output — most commonly@@font-face,@@namespace,@@mediainside CSS<style>blocks, or$$10for a literal dollar amount.TemplateSigilSyntaxError(pipelex/cogt/templating/template_errors.py): raised by the preprocessor when a candidate@/@?sigil is not alone on its line. Pipe blueprints (PipeLLMBlueprint,PipeComposeBlueprint,PipeImgGenBlueprint,PipeSearchBlueprint,TemplateBlueprint,ConstructBlueprint,pipe_compose_factory) and theTemplateBlueprintvalidator catch and re-raise as pydanticValueErrorwith pipe-specific context, so the diagnostic surfaces through normal MTHDS validation errors and IDE/plxt checkdiagnostics.
Fixed
- Template preprocessor:
$<var>no longer substitutes when$is adjacent to a word character on the left (e.g.micro$oft,user$host.com,P@ssw$rd123,a$b$cnow pass through unchanged instead of producing mid-word{{ ... }}substitutions), and no longer emits invalid Jinja for shapes like$name..(now renders{{ name|format() }}..with both trailing dots preserved as literal punctuation). The$arm now mirrors the@candidate pattern's word-boundary lookbehind ((?<!\w)) and uses a strict segmented identifier[a-zA-Z_][a-zA-Z0-9_]*(?:\.[a-zA-Z_][a-zA-Z0-9_]*)*that rules out a leading digit and consecutive dots structurally. The trailing-dot kludge in the$replacement function is gone (unreachable under the strict identifier shape).$varkeeps its silent pass-through posture for non-candidate shapes — it does not raise like the@arm does. - Project
.pipelex/telemetry.tomlcreated bypipelex initno longer silently disables globally-enabled telemetry. The kit previously shipped onetelemetry.tomltemplate carrying explicitenabled = false/mode = "off"defaults, andpipelex init telemetrycopied it into both~/.pipelex/and the project's.pipelex/. Because the layered loader appends the project base file after~/.pipelex/telemetry_override.toml, a user who enabled Langfuse/PostHog globally in their override still had it turned off in every initialized project. The kit now ships a separatetelemetry.project.tomltemplate with every setting commented out, used only by project-level init. The global init still copies the activetelemetry.toml. As a safety net,telemetry_allowed_modesdefaults (ci=False,cli=True,docker=True,fastapi=True,mcp=True,n8n=True,pytest=False,python=False) now live in theTelemetryConfigPydantic model rather than the TOML template, so a project file that omits the section can't disable custom telemetry for everyone. - Global
pipelex.tomlandtelemetry.tomloverrides no longer silently disappear when a project.pipelex/exists.ConfigLoader.load_config()previously sourced all override files (pipelex_local.toml,pipelex_{env}.toml,pipelex_{run_mode}.toml,pipelex_override.toml,pipelex_temporary_override.toml) from a single "effective config dir" — project.pipelex/if present, else~/.pipelex/. As a result, the user's machine-wide overrides in~/.pipelex/were ignored entirely the moment a project shipped its own.pipelex/.load_telemetry_confighad the same shadowing behavior fortelemetry.tomlandtelemetry_override.toml, which was particularly painful because telemetry holds Langfuse/PostHog/OTLP secrets that users typically set once globally. Both loaders now layer global → project: package defaults → global base → global overrides → project base → project overrides, deep-merged so machine-wide personal preferences keep applying unless explicitly overridden by the project. The unit-testing runmode behavior is preserved:./tests/pipelex*{run_mode}.tomlis still the only run_mode file loaded under unit testing, replacing both layers to keep test runs hermetic. Migration: if a project previously relied on its.pipelex/to fully replace* the user's global config rather than merging over it, the project must now redeclare any keys it wants to set explicitly — silent inheritance from~/.pipelex/is the new default. pipelex build structuresnow emits domain-qualified class names and cross-references.ConceptFactoryregisters each concept's structure class undermake_qualified_structure_class_name(domain, concept_code)(e.g.expense_validator__SpendingLimitCheck), but the CLI generator was passing the bareconcept_codetoStructureGenerator. Generated files therefore containedclass ConceptCode(StructuredContent)and unqualified cross-imports/types, while the registry expected the qualified name.PipeFuncvalidation againstconcept.structure_class_namethen rejected the mismatch withoutput concept expects structure class 'domain__X', but the function return type is 'X', breaking library load for any project that had regenerated structures. Fixed inpipelex/cli/commands/build/structures_cmd.py: the class-definition call sites now callmake_qualified_structure_class_name(blueprint.domain, concept_code);_build_concept_ref_to_class_infoqualifies cross-reference class names while keeping the file stem unqualified so output filenames staydomain__concept_code.py; therefines:branch resolves the refined concept's domain viaQualifiedRef.parse_stripping_cross_packageand qualifies the base-class name for non-native refines, mirroringConceptFactory._handle_refines.pipelex build structurescross-package refines now emit the base-class import. A concept that refines a class defined in another already-installed package (e.g.RefiningConceptwithrefines: some_alias->other_domain.OtherConcept) was generating a file that inherited fromother_domain__OtherConceptwithout importing it.StructureGenerator._validate_executioninjects the base class intoexec_globalsfrom the class registry, so codegen-time validation passed; the failure surfaced only when the generated structures module was later loaded via the normal Python import system, raisingNameError: name 'other_domain__OtherConcept' is not defined. The generator now falls back to the class registry's__module__whenconcept_ref_to_class_infohas no entry for the base class and emits the corresponding import (guarded oncls.__name__ == base_classso we never emit a name the target module won't expose). Regression test intests/unit/pipelex/cli/commands/build/test_structures_cmd_cross_package_refines.pywas tightened to assert the import statement is present.
[v0.27.0] - 2026-05-07
Changed
RunEnvironmentenv var renamed fromENVIRONMENTtoPIPELEX_ENV. The variable that selects the environment-specificpipelex_<env>.tomloverlay (and is stamped on OTel spans asdeployment.environment) is now namespaced to avoid collisions with unrelatedENVIRONMENTvariables already set in deployment environments. Update any shell, CI, or container config that exportedENVIRONMENT=...to exportPIPELEX_ENV=...instead.
[v0.26.4] - 2026-05-06
Fixed
clean_json_content()no longer crashes on nested Pydantic instances inworking_memory_raw. Whenworking_memory_raw(typeddict[str, Any]) held rehydratedBaseModelinstances (e.g.PageContentfromPage[]outputs),clean_json_content()walked only dicts/lists/scalars and letBaseModelinstances reachjson.dumps, which raisedTypeError: Object of type PageContent is not JSON serializable.clean_json_contentnow reduces anyBaseModelit encounters viamodel_dump(serialize_as_any=True)(the canonicalsmart_dumppath) before continuing the recursive walk.
[v0.26.3] - 2026-05-06
Fixed
- OpenAI SDK 2.34.0 compatibility for Portkey/Gateway clients. The new SDK adds both a constructor-time check (
_enforce_credentials) and a request-time_validate_headersguard that reject an emptyapi_key. Pipelex's gateway and Portkey factories were passingapi_key=""because auth is delivered viax-portkey-api-keyandx-portkey-configheaders, not the OpenAIAuthorizationheader. Replaced the empty string with a non-empty placeholder ("unused-auth-via-portkey-headers") ingateway_completions_factory,gateway_responses_factory,portkey_completions_factory, andportkey_responses_factory. Same fix applied toopenai_client_factoryfor the no-auth case (e.g. local Ollama backends), wherebackend.api_keyis intentionally unset. Bumps theopenairequirement to>=2.0.0.
[v0.26.2] - 2026-05-06
Fixed
choicesfields no longer fail validation with'EnumName.MEMBER_NAME'errors. A concept declared withchoices = [...]produces aLiteral[...]field on the dynamic Pydantic class. That schema is round-tripped throughSchemaToModelFactory.make_from_json_schema(used to feed structured-output schemas to LLM providers). Previously the round-trip silently re-emitted the field as a plain PythonEnumclass — e.g.Literal["Strong Match", "Good Match", "Partial Match", "Poor Match"]becameclass Recommendation(Enum): Poor_Match = "Poor Match"; .... LLMs filling that schema then returned the enum's Python repr ("Recommendation.Poor_Match") instead of the literal string ("Poor Match"), which failed Pydantic validation against the original choice set with errors likeInvalid choice errors: 'recommendation': got 'Recommendation.Poor_Match', expected one of 'Strong Match', 'Good Match', 'Partial Match' or 'Poor Match'._generate_source_from_schemanow passesenum_field_as_literal=LiteralType.Alltodatamodel-code-generator, soenum: [strings]schema nodes round-trip asLiteral[...]instead of being regenerated asEnumclasses._exec_source_to_typesnow also exposesLiteralin the rebuild namespace somodel_rebuildresolves the deferred annotations.
[v0.26.1] - 2026-05-05
Changed
- Live-run graph output:
pipelex-agent runnow emits the assembledGraphSpecaslive_run_graph.jsonalongsidelive_run.jsonand the ReactFlow HTML.
[v0.26.0] - 2026-05-04
Highlights
- Distributed tracing — pluggable event-log backends (NDJSON, DynamoDB, in-memory, buffering) emit
TraceEvents that reassemble into a completeGraphSpec. Enabled by default; works in single-process mode too.
Added
Pipeline execution & delivery
PipeRunlayer (pipelex/pipe_run/): newPipeRunProtocolwith the in-processPipeRunimplementation. Wraps pipe execution + delivery in a single orchestration unit.- Delivery framework:
DeliveryAssignmentmodel with webhook (HTTP POST) and storage (S3 / local) variants, run inline after a pipe completes. LibraryCrateIR (pipelex/libraries/library_crate.py,library_crate_factory.py): serializable intermediate representation capturing the exact set of libraries (concepts, pipes, structures) required to execute a job — no shared filesystem assumed.- Schema-based dynamic model construction (
pipelex/cogt/content_generation/schema_to_model_factory.py): rebuilds dynamic Pydantic models from the JSON schema embedded inObjectAssignment, instead of relying on class identity. Cache bounded by an LRU (_SCHEMA_CACHE_MAX_SIZE = 1024); codegen serialized through a file lock to kill thundering-herd duplication when many distinct schemas arrive. Addsdatamodel-code-generator>=0.55.0as a runtime dependency. - Pre-generated
pipeline_run_idsupport:pipeline_run_setup(),PipelineFactory, andPipelineManageraccept an optionalpipeline_run_idparameter for external run-record creation.
Distributed tracing (pipelex/tracing/)
EventLogProtocolwith pluggable backends:NdjsonEventLog,DynamoDBEventLog,InMemoryEventLog,BufferingEventLog. EmitsTraceEvents as pipes start/finish across processes. Configured via[pipelex.tracing_config](backend = "ndjson" | "dynamodb"). Tracing is enabled by default — single-process runs also produce a trace.pipelex[dynamodb]extra:pip install "pipelex[dynamodb]"for the DynamoDB event log backend.writer_idonTraceEvent(default"primary"): every event records which writer (worker / process) produced it, propagated through every backend. Separate writer processes use a distinctwriter_idso multiple writers can emit into the same backend partition without colliding on(workflow_id, sequence). Stamped at construction time (no copy-on-emit). Enables cross-writer dedup and per-writer aggregation.- NDJSON multi-writer key scheme: dedup key is
(workflow_id, writer_id, type, sequence); sort key is(workflow_id, sequence, writer_id)— sequence is primary so the runner'sUsageReportEventdoes not sort before the router'sPipeStartEvent. The file-handle cache key includeswriter_id, so two writers for the same(pipeline_run_id, workflow_id)cannot share a stale handle. - DynamoDB sort key format:
EVENT#{workflow_id}#{writer_id}#{sequence:010d}. ContentGeneratorDryemitsUsageReportEvents: invokesreport_inference_jobwith a syntheticLLMTokensUsageso dry-run mode produces real events observable through the same backend path as live runs.GraphSpecAssembler: rebuilds a completeGraphSpecoffline from a stream ofTraceEvents — equivalence with the liveGraphTraceris pinned by tests.UsageAggregator: aggregates per-pipeline LLM/extract/img-gen usage across workers from the event log.- Hub DI:
PipelexHub.set_event_log()/get_event_log()for dependency injection of custom event log backends.make_event_log()factory selects the backend from config or hub injection; the 3 prior call sites that hardcodedNdjsonEventLognow go through it.
Inference error classification
InferenceErrorCategoryenum (transient,configuration,content,capacity) with per-provider helpers that distinguish quota exhaustion from rate limits and detect content-policy violations — covers OpenAI, Anthropic, Google, Mistral, AWS Bedrock, and Pipelex Gateway.- All inference workers attach error category + user action: every LLM, image-gen, extract, and search worker across all providers raises exceptions with an
InferenceErrorCategoryand actionableuser_actionhint. - Structured
ErrorReport:PipelexError.to_error_report()returns a dataclass with error type, category, retryable flag, user-action hint, model, and provider. CLI error handlers consumeto_error_report()for consistent, structured display. SecurityError(PipelexError): new base class inpipelex.base_exceptionsfor security-policy violations.UnsafeSchemaErrornow extends it (wasContentGenerationError) so security signals are not silently swallowed by domainexcepthandlers.
Documentation
- New "Under the Hood" pages:
pipe-routing-and-execution.mdanddistributed-content-generation.md. Cover routing/dispatch and the dynamic-model serialization story. - New
AGENTS.mdat repo root, generated from the same source asCLAUDE.mdviamake rules.
Other
PipeOutput.graph_assembly_error: optionalstrfield set when graph assembly raises, so consumers can distinguish "graph never produced" from "graph assembly failed". Previously the failure was logged as a warning andgraph_specsilently stayedNone.pipe extractweb URL support with full test coverage intests/integration/pipelex/pipes/pipe_operators/pipe_extract/.needs_inferenceflag on CLI commands: non-inference subcommands (e.g.doctor) skip inference setup, shaving cold-start time.- Environment-specific config:
RunEnvironmentis now loaded from theENVIRONMENTenv var (wasENV). Accepted values:local,dev,staging,prod.
Changed
- Tightened inference quota classification patterns to reduce false positives on auth/setup errors:
- Anthropic: bare
"credit"token replaced with"credit balance","out of credits","insufficient credit". No longer misclassifies"your credit card was declined"or"we will credit your account"as quota exhaustion. - Google: bare
"billing"token replaced with"billing limit","billing quota","billing exceeded", and"billing account". Continues to match real 429-class messages ("Billing account is not active","billing limit reached"); stops matching unrelated billing-mentioning text. - Mistral / Gateway: bare
"billing"likewise narrowed to"billing limit"/"billing quota". Mistral additionally distinguishes"out of credits"vs"insufficient credits". - Removed duplicate
ContentGenerationErrorclass: the unusedpipelex.cogt.content_generation.exceptions.ContentGenerationErrorwas deleted (had zero external imports). Its subclasses re-parented:NeitherUrlNorDataError→PipelexError,UnsafeSchemaError→ newSecurityError. - Storage providers unified:
StorageConfigflattened to inheritance-based config so providers (local,s3) share a single shape across delivery and trace storage. make rulesnow generates bothCLAUDE.mdandAGENTS.mdby default: thepipelex.kit_config.preferred_agent_targetsetting has been renamed topreferred_agent_targetsand is now a list. The default is["claude", "agents"]. Cursor remains exclusive (["cursor"]) and cannot be combined with single-file targets. Downstream projects overriding this setting must rename the key and wrap the value in a list.- Cold start trimmed: heavy SDK imports (
boto3,huggingface_hub, etc.) deferred behindTYPE_CHECKINGor function-local guards across multiple plugins. - Graph viewer updated to mthds-ui v0.3.4: bumped from v0.3.0 — additional polish atop the resizable detail panel, escape-to-close, sticky header, prompt expand/collapse with copy button, concept refinement display.
- Default models: small-vision and creative defaults now point to
gemini-3.0-flash-preview. Addedclaude-4.6-sonnetand Bedrock token-auth support. kajsonupgraded from0.3.1to0.5.0— required for the dynamic-class source registry that backs schema-to-model reconstruction.- README install instructions: replaced step-by-step Claude Code setup with single copy-paste messages for Claude Code and Codex, added manual install section.
Fixed
- Anthropic structured generation no longer flattens all errors to
CONTENT.AnthropicLLMWorker._gen_objectruns requests throughinstructor, which wraps SDK exceptions inInstructorRetryException. The previous handler categorized every wrapped error asCONTENT, so genuineRateLimitError,APITimeoutError,APIConnectionError,PermissionDeniedError, andAuthenticationErrorcases never reached the typed branches — they were reported as content failures (non-retryable) instead ofTRANSIENT/CAPACITY/CONFIGURATION. The handler now unwrapsInstructorRetryException.failed_attempts[-1].exception(with a__cause__/RetryError.last_attemptfallback) and routes recognized SDK exceptions through a shared categorization helper; truly unrecognized errors keep theCONTENTfallback. - Managed-deck detection no longer misclassifies user TOMLs.
_is_managed_deck_filenamepreviously treated any non-x_custom_*.tomlas kit-managed, so a user file likemy_overrides.tomldropped into the deck dir would have been reported (and onpipelex updateoverwritten) as if it were a kit file. Now requires the numbered^\d+_.*\.toml$prefix that the kit actually ships. PipeLLMoutput structure prompt used the unresolved concept ref. Whenis_structure_prompt_enabled,get_output_structure_promptwas called withpipe_run_params.dynamic_output_concept_ref or output_stuff_spec.concept.concept_ref. The first form can be a bare code (no domain), which broke downstream concept resolution. Now passes the already-resolvedoutput_stuff_spec.concept.concept_refdirectly.PIPELEX_NO_DECK_NOTICEsuppression was too lax. Any non-empty value silenced the deck-staleness notice — includingPIPELEX_NO_DECK_NOTICE=0, which users might reasonably expect to enable the notice. Now requires the documentedPIPELEX_NO_DECK_NOTICE=1.- Shipped
.kit_manifest.jsoncarriedkit_version: "0.25.0"for the 0.25.1 release, which would have triggered false stale-detection on fresh installs. Bumped to0.25.1and re-synced intopipelex/kit/configs/. - Stored-XSS guard: HTML escaped in fallback
main_stuff.html. GatewayExtractWorkerteardown: guarded done-callbacks against cancelled tasks to stop noisy teardown errors.- URL checker: 401/403 are now treated as OK for auth-walled URLs.
- Pipeline duplicate guard: registering the same pipeline twice raises explicitly instead of silently shadowing.
Documentation
docs/tools/cli/update.mdno longer claims the deck advisory fires on every CLI invocation. Clarified that it is suppressed forlogin,init,doctor,update, andwhich.
Security
schema_to_modelrejectsx-python-*codegen extensions and restricts__import__to an allowlist.datamodel-code-generatorhonors thex-python-importJSON Schema extension by emitting arbitraryfrom <module> import <name>statements with no sanitization. Combined with the prior gap that_make_restricted_builtins()did not block__import__, an attacker able to plant a craftedobject_class_schema(whereObjectAssignment.object_class_schema: dict[str, Any]round-trips untouched) could cause arbitrary modules to be imported duringexec()of generated code. Two-layer defense added: (1)_reject_unsafe_schema_extensionsraisesUnsafeSchemaErrorif anyx-python-*key is present anywhere in the schema; (2)__import__in the exec namespace is now wrapped to allow onlypydantic,typing,typing_extensions,enum,datetime,decimal,uuid,__future__,collections, andre.- Restricted exec builtins, path sanitization, and atomic fingerprint caching in the schema-to-model pipeline.
- No leaked exception strings on error paths; prefix guards on storage URIs.
[v0.25.2] - 2026-04-30
Fixed
pipelex updateno longer flags user-added deck files for removal. The "managed file" filter accepted any.tomlnot prefixed withx_custom_, so project-local additions likepipelex-cookbook'scookbook.tomlwere reported as "removed upstream" with action "back up + remove". The filter now matches the documented numbered convention only (<digits>_*.toml); any other filename —cookbook.toml,x_custom_*.toml, etc. — is invisible to the update planner.
[v0.25.1] - 2026-04-29
Added
pipelex updatecommand + model-deck staleness detection. New top-level command refreshes the installed model deck (~/.pipelex/inference/deck/) to match the kit shipped with the running pipelex version. Tracks per-file SHA-256 hashes plus the kit version in a.kit_manifest.jsonwritten next to the deck files. Supports--dry-run,--yes(non-interactive),--no-backup(skip.bakfiles for locally-modified deck files), and--local(project-local deck instead of global).- Boot-time deck staleness warn. Every
pipelexCLI invocation prints a one-line yellow advisory when the installed deck's recorded kit version is older than the running pipelex (or when no manifest exists yet). Cost is one file read + one string compare. Suppress withPIPELEX_NO_DECK_NOTICE=1. Skipped automatically forpipelex login,init,doctor,update, andwhich. pipelex doctordeck section. Reports per-file deck status (up_to_date,kit_added,kit_removed,clean_behind,locally_modified) and offerspipelex updateas an auto-fix under--fix.- Manifest written by
pipelex init. Fresh installs land with a current.kit_manifest.jsonso future updates can detect drift cleanly. --dynamic-output-concept/-Oflag onpipelex run. All three subcommands (run bundle,run pipe,run method) accept a concept ref (e.g.document_qa.ReferenceCount) used to resolve a pipe whose output is declared asDynamic. Threaded through_run_core.execute_runtoPipelexRunner.execute_pipeline(dynamic_output_concept_ref=...). Until now, Dynamic-output pipes were only callable from the Python runner.- Line-length-safe wrapping in the structures generator.
pipelex build structuresnow wraps long descriptions so every emitted line stays under the 150-char ruff limit. Long class docstrings become a multi-line triple-quoted block; long Field descriptions become a parenthesized implicit-string-concatenation block (description=("first chunk " "second chunk")). Short descriptions still emit the compact single-line form. New unit tests intests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_wrapping.pycover both lengths and the combined long-everything case. Previously, descriptions above ~140 chars produced files that failedruff checkwith E501.
Changed
- Numbered deck files (
N_*_deck.toml) are pipelex-managed. Each file now carries a header banner explaining that customizations belong inx_custom_*.toml(which pipelex never tracks or overwrites). Local edits to numbered files are preserved with a timestamped.bak.<UTC-timestamp>backup onpipelex updatebut will not survive future updates.
Fixed
-
PipeLLMDynamic-output detection compared the wrong fields.pipe_llm.pycheckedself.output.concept.code == "native.Dynamic", butconcept.codeis the bare code ("Dynamic") not the qualified ref. The check never matched, so when a caller passeddynamic_output_concept_reffor aDynamic-output pipe, the resolver branch was skipped silently: the output structure stayedDynamicContent(an emptyStuffContentsubclass), the LLM produced JSON shaped like the requested concept, and the result deserialized to{}. Detection now usesconcept.code == NativeConceptCode.DYNAMIC and concept.domain_code == SpecialDomain.NATIVE. -
Dynamic-output concept resolution rejected qualified refs. When the runtime override was supplied (e.g.
"document_qa.ReferenceCount"), the previous code calledmake_concept_ref_with_domain(domain_code=self.domain_code, concept_code=output_concept_ref), producing"document_qa.document_qa.ReferenceCount"and a missing-concept lookup. Now usesmake_concept_ref_with_domain_from_concept_ref_or_code, which extracts the domain from the input when it's already qualified and falls back to the pipe's domain only for bare codes. Callers can pass either form.
[v0.25.0] - 2026-04-28
Added
-
GPT-5.5 model support. New
gpt-5.5entry on theopenai,azure_openaiand gateway backends. -
gpt-image-2image generation (OpenAI). New model entry with img-gen routing and constraints onopenai,azure_openai.
Fixed
- PDF input declared on Azure OpenAI and gateway for the GPT-5.4 series.
gpt-5.4,gpt-5.4-mini,gpt-5.4-nano,gpt-5.4-pro(andgpt-5.5) now advertisepdfin theirinputsonazure_openaiand the gateway, matching the existing OpenAI direct entries. Verified end-to-end with live document tests against both backends.
Changed
- Img-gen taxonomies aligned for
gpt-image-2. Convention:gpt_imagefor taxonomies/values shared across all OpenAI GPT Image models (legacy + gpt-image-2);gpt_image_legacywhen the value applies only to gpt-image-1/-1-mini/-1.5 (gpt-image-2 usesunavailablewhere a legacy-only option does not apply). Renames: AspectRatioTaxonomy.OPENAI_GPT_IMAGE_LEGACY→GPT_IMAGE_LEGACY(value"openai_gpt_image_legacy"→"gpt_image_legacy")AspectRatioTaxonomy.OPENAI_GPT_IMAGE_2→GPT_IMAGE_2(value"openai_gpt_image_2"→"gpt_image_2")OutputFormatTaxonomy.GPT→GPT_IMAGE_LEGACY(value"gpt"→"gpt_image_legacy")OutputCompressionTaxonomy.GPT_IMAGE→GPT_IMAGE_LEGACY(value"gpt_image"→"gpt_image_legacy")InputFidelityTaxonomy.OPENAI_IMAGE→GPT_IMAGE_LEGACY(value"openai_image"→"gpt_image_legacy")NumImagesTaxonomy.GPT→GPT_IMAGE(value"gpt"→"gpt_image")InferenceTaxonomy.GPT→GPT_IMAGE(value"gpt"→"gpt_image")- Removed dead
AspectRatioTaxonomy.GPT. -
InputImagesTaxonomy.GPT_IMAGEunchanged (already correct — shared across legacy and gpt-image-2). -
HuggingFace and Fal img-gen workers honor
model_choicerule.huggingface_img_genandfalworkers now pop"model"fromargs_dictinstead of hardcodinginference_model.model_id, mirroring howpromptis already extracted. Themodel_choice = "model_id"rule is now required on every model under these backends; missing it raisesImgGenParameterErrorat call time. Rules added toqwen-image(HF) and to all Fal models (flux-pro,flux-pro/v1.1,flux-pro/v1.1-ultra,flux-2,fast-lightning-sdxl).
[v0.24.1] - 2026-04-22
Security
- lxml floor
>=6.1.0to patch CVE-2026-41066 (GHSA-vfmq-68hx-4jfw): default configuration ofiterparse()andETCompatXMLParser()allowed XXE to local files (resolve_entities=True). lxml 6.1.0 changes the default toresolve_entities='internal'. Transitive viadocling; floor added to thedoclingextra inpyproject.tomlso downstream installs ofpipelex[docling]cannot resolve a vulnerable version. - cryptography floor
>=46.0.7to patch CVE-2026-39892 (GHSA-p423-j2cm-9vmq): non-contiguous Python buffers passed to hashing APIs (e.g.Hash.update()) could read past the end of the buffer on Python >3.11. Transitive viagoogle-auth(pulled bygoogle,gcp-storage,google-genaiextras) andmoto(dev). Floor added to each affected extra inpyproject.toml— previous bump was lockfile-only, which did not protect downstream users resolving fresh from PyPI metadata. - pytest bumped to 9.0.3 to patch CVE-2025-71176 (GHSA-6w46-j5rx-g56g): vulnerable
/tmp/pytest-of-{user}directory handling on UNIX could let a local user cause DoS or gain privileges. Dev-only dependency;pyproject.tomlminimum bumped from>=9.0.2to>=9.0.3. - transformers CVE-2026-1839 (GHSA-69w3-r845-3855) risk-accepted, alert dismissed. The vulnerability requires calling
transformers.Trainer._load_rng_state()with an attacker-controlled checkpoint file. Pipelex only pullstransformerstransitively throughdocling-ibm-modelsfor PDF layout inference; theTrainerclass is never imported or executed. Upgrade path is blocked upstream:docling-ibm-models3.13.0 pinstransformers!=5.0.*,!=5.1.*,!=5.2.*,!=5.3.*,<6.0.0,>=4.42.0, explicitly excluding the patched 5.0.0rc3 release. Revisit whendocling-ibm-modelsadds support fortransformers>=5.4. - Release-publishing GitHub Actions pinned to SHAs:
pypa/gh-action-pypi-publishandsigstore/gh-action-sigstore-pythoninpublish-pypi.ymlare now pinned to full commit SHAs (version kept as a trailing comment) so a compromised tag on a third-party action cannot silently alter a PyPI release. Dependabot keeps them fresh. .github/dependabot.ymladded: declarespipandgithub-actionsecosystems, weekly cadence, with dev and runtime deps grouped to reduce PR noise. Security updates fire immediately regardless of schedule.dependency-review.ymlworkflow added: runs GitHub'sdependency-review-actionon PRs tomain,dev, and release branches. Fails the PR if it introduces a dependency with a moderate-or-higher CVE. Respects the existing transformers (GHSA-69w3-r845-3855) risk-acceptance viaallow-ghsas. Enable as a required status check in branch protection formainto block vulnerable merges.
[v0.24.0] - 2026-04-16
Added
- claude-opus-4-7 model: Registered on anthropic, bedrock, and gateway backends with 128k max output tokens, $5/$25 per MTok pricing, adaptive thinking, and PDF/vision support
XHIGHreasoning effort level: New effort tier across all providers, mapped to Anthropic'sxhigh(recommended for coding/agentic work), OpenAI'sxhigh, and best-available equivalents for Google and MistralTEMPERATURE_UNSUPPORTEDconstraint: New listed constraint for models that reject sampling parameters entirely, checked in both Anthropic and OpenAI completions workers- claude-4.6-sonnet model: Registered on anthropic, bedrock, and gateway backends
- LLM deck cheap presets: Added cheap variants for writing-factual, retrieval, and engineering-code presets, with retrieval tiers from
gemini-2.5-flash-litetoclaude-4.7-opus - Bedrock bearer token authentication: New
bedrock_access_variantconfig option supports"bedrock_token"auth usingAWS_BEARER_TOKEN_BEDROCKenv var, alongside the existing"aws_access"method (default)
Changed
- Anthropic adaptive thinking rejects
reasoning_budget:_build_thinking_params_for_budgetnow raisesLLMCapabilityErrorfor adaptive thinking models, guiding users toreasoning_effortinstead — extended thinking (type: "enabled") is removed on Opus 4.7+ - LLM deck: Updated
best-claudealias toclaude-4.7-opus, switchedimg-gen-prompting-cheapto@default-small-creative, removed deprecated builder presets, standardized cheap preset descriptions - Google
MINIMALreasoning level:ReasoningEffort.MINIMALnow maps to Google'sThinkingLevel.MINIMAL(was previously collapsed toLOW);GoogleThinkingLevelenum gainsMINIMALand the defaultgoogle_config.effort_to_level_mapupdates accordingly
[v0.23.9] - 2026-04-14
Added
- Graph UI asset sync workflow:
package.jsonpins@pipelex/mthds-uito a git tag,make sync-graph-uiclones and builds standalone assets,make check-graph-ui-syncverifies version alignment - CI check:
graph-ui-check.ymlworkflow validates graph viewer assets match the pinned version on PRs to main /update-graph-uiskill: automates bumping the mthds-ui version, syncing assets, and running tests
Changed
- Graph viewer updated to mthds-ui v0.3.0: resizable detail panel, escape-to-close, sticky header, prompt expand/collapse with copy button, concept refinement display
- README install instructions: Replaced step-by-step Claude Code setup with single copy-paste messages for Claude Code and Codex, added manual install section
[v0.23.8] - 2026-04-07
Changed
- Standalone ReactFlow graph rendering: Replaced Jinja2 template-based HTML generation with a single standalone HTML asset, simplifying the graph rendering pipeline and removing the Jinja2 template dependency for ReactFlow output.
[v0.23.7] - 2026-04-06
Added
- Graph tracing for pipe run data: Pipe run data and concept are now included inside the flowchart graph spec, enabling richer visualization of pipe execution results across all pipe types (LLM, extract, compose, search, image gen, sequence, condition, batch, parallel).
- Assignment pipe: New
pipe_assignmentpattern for direct value assignment within pipe execution.
[v0.23.6] - 2026-04-06
Changed
- Sub-pipe input normalization: Silently drop unsupported
inputsfield from step/branch dicts instead of logging a warning.
[v0.23.5] - 2026-04-04
Added
- Gateway config: Introduced
GatewayConfigto bundle gateway model specs with AWS region, propagating it through the backend library so bedrock backends use the correct region. - Config coverage tests: Integration tests that validate one model per Portkey config for each model type (LLM, image gen, extract, search), with
all_configs_gwtest profile andmake ticctarget. - nano-banana-2 model: Added
gemini-3.1-flash-image-previewasnano-banana-2with updated Google image gen costs. - DeepSeek models on bedrock: Added DeepSeek models to the bedrock backend configuration.
Changed
- Image gen deck aliases: Updated aliases to nano-banana model variants and removed
flux-2-pro. - Remote config: Bumped to v08.
- Gateway model docs: Regenerated, removing retired models (claude-3.7-sonnet, deepseek-v3.1, deepseek-v3.2-speciale, flux-2-pro).
Fixed
- deepseek-v3.1 structured output: Removed unsupported
structuredoutput capability from the bedrock deepseek-v3.1 model spec — thebedrock_aioboto3worker does not implement object generation, so structured calls would fail at runtime.
[v0.23.4] - 2026-04-02
Changed
- Pipe spec output alias: Removed
output_typealias fromparse_pipe_spec, keeping onlyoutput_conceptas the single alias for theoutputfield. Simplified the alias resolution logic accordingly.
[v0.23.3] - 2026-04-02
Changed
- Pipe spec output aliases:
parse_pipe_specnow acceptsoutput_conceptandoutput_typeas aliases for theoutputfield, with smart fallback when both alias and canonical field are present.
Fixed
- Gateway terms check: Terms acceptance is now only required for inference operations, not for read-only operations like model spec fetching during validation.
[v0.23.2] - 2026-03-30
Changed
- Claude Code plugin install command: Fixed as →
/plugin install mthds@mthds-pluginsacross README and docs. - Claude Code plugin reload instructions: Added
/reload-pluginsas the primary method to activate the plugin, with exit/reopen as fallback.
[v0.23.1] - 2026-03-30
Changed
- Concept spec:
concept_codereplacesthe_concept_codeas the canonical field name in concept specs and working memory factory. - Shared spec parsing:
concept_cmdandpipe_cmdnow delegate to the sharedparse_concept_specandparse_pipe_spechelpers, removing stale duplicate parsing logic while preserving compatibility and fixing alias/dict-mutation edge cases. concept_ref/pipe_refaliases:parse_concept_specandparse_pipe_specnow acceptconcept_refandpipe_refas input aliases for better AI-agent compatibility.- Replace
pipwithuvin install commands across config files and error messages. - Docs links: Updated mthds.ai links to include
/latest/path.
Fixed
- Concept alias bug: Concept alias handling previously listed
concept_codeas an alias instead ofthe_concept_code, causing valid input to be silently dropped. Fixed by the new sharedparse_concept_spechelper.
[v0.23.0] - 2026-03-29
Added
- Builder Operations module (
pipelex/builder/operations/): New standalone operation functions for the build agent —concept_ops,inputs_ops,models_ops,output_ops,pipe_ops,runner_code_ops,validate_ops. These decouple build logic from CLI commands so it can be reused by agents and the API. dry_run_pipeline(pipelex/pipe_run/dry_run_pipeline.py): New shared entrypoint to dry-run a pipeline from MTHDS contents and produce aGraphSpec. Used by both CLI graph commands and the API.
Changed
mthds_content→mthds_contents: Unified singularmthds_content: str | Noneintomthds_contents: list[str] | NoneacrossPipelexRunner,pipeline_run_setup,validate_bundle, CLI commands, and builder operations. Callers now pass a list of bundle content strings.bundle_uri→bundle_uris: Renamedbundle_uri: str | Nonetobundle_uris: list[str] | NoneacrossPipelexRunner.__init__,pipeline_run_setup,dry_run_pipeline, and all CLI call sites. Multiple bundles can now be loaded simultaneously.- Bump
mthdsdependency to>=0.2.0(from>=0.1.1) to match the updatedRunnerProtocolinterface. - Graph rendering refactor: Extracted dry-run logic from
graph_rendering._dry_run_bundleinto the new shareddry_run_pipelinefunction. - Agent CLI inputs: Refactored
_inputs_coreto delegate tobuilder.operations.inputs_ops.build_inputs_for_pipeinstead of duplicating bundle validation and input rendering logic. - Agent CLI run: Added graph generation support with ReactFlow HTML output and side-effect metadata tracking in
_run_core. - Deduplicate
models_cmd→models_ops:models_cmd.pyis now a thin CLI wrapper delegating tolist_models()andformat_models_markdown()inmodels_ops.py, consistent with other agent CLI commands.
Removed
BuilderLoopand its iterative build-validate-fix cycle (builder_loop.py,builder.py,builder_errors.py). The build-agent CLI now drives spec construction directly.pipelex build pipeCLI command and associated MTHDS workflow files (builder.mthds,agentic_builder.mthds,pipe_design.mthds,concept_fixer.mthds,synthesize_image.mthds).-
pipelex-toolsruntime dependency: Moved to dev-only dependency;plxtis now invoked via subprocess passthrough. -
Talent system: Removed talent enums, config mappings (
talent_preset_mappings), and talent preset tests. Pipe specs accept model presets directly via themodelfield, making the talent indirection unnecessary.
Fixed
- Per-bundle dedup in pipeline run setup: Fixed duplicate bundle loading when the same bundle appears in multiple sources.
- HTTP utils: Use HEAD-first strategy for URL validation to avoid downloading large payloads unnecessarily.
library_dirspassthrough: Fixedlibrary_dirsnot being forwarded in builder operations (inputs_ops,validate_ops).- Dry-run status: Fixed validation status reporting for dry-run results.
[v0.22.0] - 2026-03-25
Added
- MiniMax Backend Support: Added integration for MiniMax models (M2, M2.1, M2.5, M2.7 series including high-speed variants) with an
all_minimaxrouting profile. check-modelAgent CLI Command: Introducedpipelex-agent check-modelto validate model references. Provides fuzzy matching suggestions, cross-collection suggestions, and wrong-sigil hints when an invalid model is provided.- Domain-Qualified Pipe References (
pipe_ref): Added apipe_refproperty (domain.code) as the primary pipe index, allowing pipes with the same code to coexist across different domains. Bare code lookups remain supported as a fallback when unambiguous. - Duplicate Concept Detection: Added validation to warn about duplicate concept declarations across different bundles within the same library.
- Shared TOML Formatter: Extracted a shared
format_toml_stringutility for consistent multi-line string formatting across the MTHDS factory and Agent CLI.
Changed
- Agent CLI Output Formats: Overhauled
pipelex-agentCLI output for LLM agents:conceptandpipecommands now output raw TOML to stdout;modelsanddoctorcommands output Markdown by default (JSON still available via--format json). - Unified
modelField in Pipe Specs: Replaced type-specific talent fields (llm_talent,extract_talent,img_gen_talent,search_talent) with a singlemodelfield accepting model presets directly. Talent mappings removed frompipelex-agent modelsoutput accordingly. - Initialization Checks:
check_is_initializednow additionally verifies the presence ofplxt.toml. - Documentation & Handoffs: Updated Claude Code skills plugin docs, CLI references.
Fixed
- Domain Metadata Conflicts: Loading multiple bundles declaring the same domain with differing descriptions or system prompts now emits a warning and keeps the first declaration instead of throwing an error.
- Cross-Domain Execution Tracking: Pipe controllers and the dry-run engine now track visited pipes using
pipe_refinstead of bare codes, preventing false-positive loop detections when crossing domains.
Removed
assembleAgent CLI Command: Removed thepipelex-agent assemblecommand.
[v0.21.0] - 2026-03-19
Added
- Web Page Extraction:
PipeExtractnow supports extracting content from web pages via the newlinkup-fetchmodel, withrender_jsandinclude_raw_htmlparameters. Added@default-extract-web-pagealias to the extraction model deck. - New AI Models: OpenAI / Azure:
gpt-5.4,gpt-5.4-pro,gpt-5.4-mini,gpt-5.4-nano,gpt-5.2-codex; Mistral:mistral-small-4 - DocumentContent Enhancements: Added
public_url,filename,title, andsnippetfields to support web pages and search citations. - AI Agent Documentation: Integrated
mkdocs-llmstxt-mdplugin to generate/llms.txtand/llms-full.txtfor AI agents. - Claude Code Skills: Added internal
.claude/skills(add-model,test-model) for registering and testing new inference models.
Changed
- Web Fetching → Extract Domain: Moved web fetching from the Search domain into the Extract domain. Replaced
FetchJob,FetchWorkerAbstract, and related classes with native handling inPipeExtractand a newGatewayExtractWorker. SplitLinkupWorkerinto dedicatedLinkupSearchWorkerandLinkupExtractWorker. - Search Results Unification:
SearchResultContentnow usesDocumentContentinstead of the removedSearchSourceContent. - Search Job Refactoring: Encapsulated
SearchJobparameters into a newSearchJobParamsobject; updatedSearchWorkerAbstractsignatures accordingly. - Documentation Build: Replaced inline HTML generation with a
docs/root-index.htmltemplate. Updatedrobots.txtto allow AI crawler access tollms.txt.
Fixed
- Remote Image Fetching: Added error handling (
httpx.HTTPStatusError,httpx.RequestError) inGeneratedContentFactorywhen downloading remote images, with graceful fallback to using the original URL aspublic_url.
Removed
page_image_captionsparameter: Removed fromPipeExtractSpecandExtractJobParams.
[v0.20.13] - 2026-03-16
Removed
- Remove unused
ViewSpecclass and related code from graph/reactflow — dead code that was no longer referenced.
[v0.20.12] - 2026-03-16
Fixed
- Fix Google "Sitemap could not be read" error by adding
Allow: /sitemap.xmltoROOT_ROBOTS_TXT—Disallow: /was blocking Googlebot from fetching the root sitemap even thoughSitemap:pointed to it.
[v0.20.11] - 2026-03-16
Fixed
- Fix
docs-deploy-rootsilently failing due to shell comments breaking the\continuation chain in the Makefile recipe — rootsitemap.xmland updatedrobots.txtwere never deployed in v0.20.10.
[v0.20.10] - 2026-03-16
Fixed
- Fix sitemap double-path bug (
latest/0.20.9/page/) caused bysite_urlincluding/latest/while mike also inserts the version prefix during deployment. - Override canonical URLs,
og:url, and JSON-LDurlto always point to/latest/via template override. - Add root-level
sitemap.xmlgeneration with/latest/URLs todocs-deploy-root.
[v0.20.9] - 2026-03-16
Changed
- Reorganize documentation site architecture and navigation.
- Add SEO meta descriptions to all doc pages.
- Redesign README: lead with value, collapse details.
[v0.20.8] - 2026-03-13
- Add
needs_model_specs=Trueto agent CLI run and models commands.
[v0.20.7] - 2026-03-12
- Bump
mthdsdependency to>=0.1.1.
[v0.20.6] - 2026-03-12
Added
needs_model_specsparameter: New option onPipelex.make()and CLI factories to load real gateway model specs without enabling full inference. Validation commands (pipelex validate,pipelex-agent validate) now fetch live model specs, improving accuracy of pipe/method/bundle validation.
Removed
- Deprecated
pipelex_inferencebackend: Removed all traces of the legacy backend — config files, enum values, migration logic, deprecation warnings, and documentation admonitions. The transition topipelex_gatewayis complete.
Fixed
pipelex init inference --local: Fixed file filter to also copy.mdfiles (e.g. Gateway model lists) to the backends directory, not just.tomlfiles.- Gateway terms persistence: Hardened failure handling — when terms acceptance can't be persisted (e.g. unwritable global directory), gateway is now properly disabled in
backends.tomlas a fallback, preventingGatewayTermsNotAcceptedErrorat runtime.
[v0.20.5] - 2026-03-09
Changed
- Config paths use
pathlib.Path: AllConfigLoaderproperties and methods now returnPathinstead ofstr. Consumer code across the config system (doctor, init, backends, routing, credentials, telemetry, agent CLI) updated accordingly. - Layered config resolution for
pipelex doctorandpipelex init: Config files are now resolved with project-first, global-fallback layering per file. Fixedreplace_backend_fileusing CWD-relative path that broke when run from another directory. - Gateway terms are explicitly global:
update_service_terms_acceptancenow targets~/.pipelex/by default, since gateway terms are a user-level agreement, not project-level. - Tweaks to the
pipelex-agentCLI to play well with the Claude Code plugin and skills.
[v0.20.4] - 2026-03-04
- Improve README.md instructions for Claude Code and MTHDS skills.
[v0.20.3] - 2026-03-04
Fixed
- Rewrote README with updated CV Batch Screening example, corrected CLI command, inputs schema, Python snippet, and pipe descriptions.
[v0.20.2] - 2026-03-04
Fixed
- Fixed removal of the --reset flag from the init command docs.
- Fixed the name of the
PipeSequenceresults within a batch.
[v0.20.1] - 2026-03-03
Changed
- Cost Report Consistency: Renamed
platform_llm_idfield toplatform_model_idinLLMTokenCostReport, aligning with all other report types (ImgGen, Extract, Search, Fetch) that already useplatform_model_id. - Test Coverage: Added
linkupbackend to the coverage test profile.
[v0.20.0] - 2026-03-03
Added
- CLI Authentication:
pipelex logincommand that initiates a browser-based OAuth flow (GitHub/Google) to authenticate with the Pipelex Gateway and save the API key locally. - Search Filtering:
from_date,to_date,include_domains, andexclude_domainsoptions on thePipeSearchoperator, routed through a newGatewaySearchWorker. - Expanded Cost Reporting: Token usage and cost tracking now covers Search, Fetch, Extraction, and Image Generation jobs, in addition to LLMs.
- New Models: Added
gpt-5.3-codex(text, images, pdf inputs) andnano-banana-2.
Changed
- Search Configuration: Renamed Linkup model IDs from
linkup/standardtolinkup-standard(anddeepvariant), simplified presets to$standardand$deep. - Default Pipeline Execution: Changed from
in_memorytolocal. - Default Extraction Model: Changed from
mistral-document-ai-2505toazure-document-intelligence; removedmistral-document-ai-2505from the supported Gateway models list. - Content Handling:
TextAndImagesContentnow supports an optionalraw_htmlfield. - Dev Experience:
Makefilecommands (generate-mthds-schema,update-gateway-models) run in quiet mode by default.
[v0.19.0] - 2026-03-02
Added
- Web Search Integration: Introduced a new
PipeSearchoperator, native support for Linkup as a search backend provider,SearchResultandSearchResultContentconcepts for handling answers with citations, and Model Deck support for search models. - Graph View Generation: Added a
--viewoption topipelex validate bundlethat generates aViewSpecJSON (compatible with ReactFlow) for client-side graph rendering without writing files to disk.
Changed
- Test Configuration: Added a
searchpytest marker, excluded from default test runs. mthdsbumped to>=0.1.0.pipelex-toolsbumped to>=0.2.3.linkup-sdk>=0.12.0added as optional dependency for the Linkup search provider.
[v0.18.6] - 2026-03-01
Added
- GitHub URL support for CLI method targets — All CLI commands accepting a method target (
pipelex validate method,pipelex run method,pipelex build inputs method, etc.) now accept a public GitHub URL (e.g.,https://github.com/org/repo/tree/main/methods/my-method). The repository is cloned automatically, and the method package is discovered and validated. Subdirectory URLs are supported. - Local path support for CLI method targets — Method targets can now also be local filesystem paths pointing to a directory containing a
METHODS.toml. This works for bothpipelexandpipelex-agentCLIs. - Added
clone_default_branch()inmthdspackage for shallow-cloning a git repository's default branch.
[v0.18.5] - 2026-03-01
Changed
pipelex-agent assemblecommand now outputs JSON to stdout by default.
[v0.18.4] - 2026-03-01
Added
- Introduced a lenient loading mode for
InferenceBackendLibraryandRoutingProfileLoader: when enabled, logs warnings instead of raising errors for missing credentials, variable fallback failures, or disabled backends — skipping individual backends gracefully rather than crashing the entire load process. - Added
needs_inferenceparameter toPipelex.make,make_pipelex_for_cli, andmake_pipelex_for_agent_clito control whether full inference setup (credentials, gateway checks, telemetry) is required. - Added
needs_inference_in_pipelexhelper toshared_pytest_pluginsto replace the inverted logic of the previous helper.
Changed
- CLI Robustness:
pipelex build,validate,show,graph,inputs, andwhichcommands now run withneeds_inference=False, allowing them to succeed even when backend credentials are missing or incomplete. - Refactored
Pipelex.make: renameddisable_inferencetoneeds_inference(defaultTrue). WhenFalse, enables lenient backend loading, uses a dummy remote config, disables telemetry, and skipsvalidate_model_deck().
Deprecated
is_inference_disabled_in_pipelexinshared_pytest_plugins— migrate toneeds_inference_in_pipelex.
[v0.18.3] - 2026-02-27
- Updated
pipelex-toolsdependency to0.2.1
[v0.18.2] - 2026-02-27
Added
- Inputs path resolution — Relative file paths (e.g.,
"url": "data/invoice.pdf") insideinputs.jsonare now resolved relative to the inputs file's parent directory, making bundle directories self-contained and portable. Applies to both user and agent CLIs. Handles HTTP, data:, pipelex-storage://, and absolute paths by leaving them unchanged.
Changed
pipelex-agent initdefaults to local project — Theinitcommand now targets the project-level.pipelex/directory by default (at detected project root) instead of auto-detecting. The--localflag has been removed. Use--global/-gto target~/.pipelex/. Errors out if no project root is found without-g.pipelex-agent doctorsupports--global/-g— Thedoctorcommand now accepts--global/-gto check the global~/.pipelex/directory. Without the flag, it auto-detects project.pipelex/if present, else falls back to~/.pipelex/.pipelex-agent init --configimproved help — The--configoption help text now shows the JSON schema with field types upfront for better discoverability.
Fixed
pipelex-agent init -gservice agreement — The--globalflag now correctly writes the gateway service terms acceptance (pipelex_service.toml) to the target directory instead of always writing to the auto-detected config dir.- Deterministic file discovery order —
find_files_in_dirnow sortsrglob/globresults for consistent ordering across platforms and Python versions. On Linux with Python < 3.13,rglobreturned filesystem-order results, which could causetest_validate_allto fail by picking up a test package'sMETHODS.tomlbefore pipelex internal bundles.
[v0.18.1] - 2026-02-25
Fixed
- Fix docs deployment failure caused by shell metacharacters (parentheses) in Makefile
PRINT_TITLEmacro argument fordocs-deploy-roottarget - Fix
mikeunable to findmkdocsbinary by adding venvbin/to PATH in all mike-based Makefile targets
[v0.18.0] - 2026-02-25
Highlights
- Pipelex Gateway — The deprecated
pipelex_inferencebackend is now replaced bypipelex_gateway, featuring remote model configuration fetching so you always have access to the latest models without updating Pipelex.
Getting your API key:
1. Get your API key at app.pipelex.com
2. Add it to your .env file: PIPELEX_GATEWAY_API_KEY=your-key-here
Gateway Supported models — included with the Free API Key - Language Models (LLM): - OpenAI: all models up to GPT-5.2 and Codex - Anthropic: Claude 3.7 Sonnet through Claude 4.5 Haiku/Sonnet/Opus - Google: Gemini 2.0/2.5 Flash, Gemini 2.5 Pro, Gemini 3.0 Flash/Pro - xAI: Grok 3/mini, Grok 4 (+ fast reasoning variants) - Open-source: Mistral Large 3, DeepSeek v3.1/3.2/Speciale, GPT-OSS 20B/120B1, Kimi K2, Phi-4, Qwen3-VL 235B - Document Extraction: Mistral Document AI, Azure Document Intelligence, DeepSeek OCR - Image Generation: GPT-Image 1/1.5, Flux 2 Pro, Nano Banana/Pro
Accepting the Terms of Service:
When you run pipelex init, you'll be prompted to accept the Gateway terms of service. By using Pipelex Gateway, telemetry is automatically enabled and identified by your API key (hashed for security) to monitor service quality and enforce fair usage. We only collect technical data (model names, token counts, latency, error rates)—never your prompts, completions, or business data. See our Privacy Policy.
⚠️ Migration deadline: If you were using pipelex_inference, please migrate soon—the legacy service will be shut down within few days. Get your new Gateway key at app.pipelex.com.
- Execution Graph Visualization System (preview feature) — Comprehensive tracing and visualization for pipeline executions.
CLI: --graph flag on pipelex run generates execution graphs. New pipelex graph render <graph.json> command for post-run rendering.
Viewers: Interactive ReactFlow viewer (reactflow.html) with pan/zoom and node inspector. Mermaid diagrams (mermaidflow.html) with subgraphs and clickable nodes.
Makefile: make view-graph (vg) and make serve-graph (sg) to start a local graph viewer.
- Pydantic Structure Generation — Two new CLI commands bridge Pipelex's declarative concepts with your Python code:
pipelex build structures /library_dir/ — Generates Pydantic models from all concept definitions found in the specified library directory. Now you have your structures as Python code: you can iterate on them, add custom validation functions, or use them as type hints in your code.
pipelex build runner — Now automatically generates both the Python runner file AND the required Pydantic structures. When you run this command, it creates a complete, ready-to-execute Python script that imports the generated structures, so you can immediately use typed objects in your pipeline code.
See the Build Commands documentation for usage examples.
- New Backends & Models:
- Hugging Face Inference — Support for Hugging Face Inference API, including
qwen-imagetext-to-image model. - Google
gemini-3.0-flash-preview - Mistral OCR latest model
mistral-ocr-2512 - Scaleway inference provider support for open-source models
-
Portkey AI backend integration for unified access to multiple models through a single API key
-
Document Support in
PipeLLM— IncludeDocumentobjects (like PDFs) directly in prompts using@variableor$variablesyntax. Supports single documents, multiple documents, and lists, combinable with text and image inputs. -
PipeComposeConstruct Mode — New mode for deterministically buildingStructuredContentobjects without an LLM. Compose fields from working memory variables, fixed values, templates, and nested structures. -
Content Storage System — Configurable storage for generated artifacts (images, extracted pages). Supports
localfilesystem (.pipelex/storage/),in_memory, AWS S3 (pip install pipelex[s3]), and Google Cloud Storage (pip install pipelex[gcp-storage]). Cloud providers support both public URLs and time-limited signed URLs. Content referenced via stablepipelex-storage://URIs. -
Langfuse & OpenTelemetry Observability — New OpenTelemetry-based observability system enables powerful tracing and Evals through Langfuse integration. Also supports OTLP-compatible backends (Datadog, Honeycomb, etc.). Configured via
.pipelex/telemetry.toml. -
Python 3.14 Support — Officially tested and supported.
-
Agent CLI (
pipelex-agent): New machine-first CLI for AI agents with structured JSON output for all commands (build,run,validate,inputs,concept,pipe,assemble,graph,models,doctor). - LLM Reasoning Controls: Unified support for "Thinking" models (Chain of Thought) with
reasoning_effort,reasoning_budget, andthinking_modeparameters. Supports Anthropic Extended Thinking, Google Gemini Thinking, OpenAI Reasoning (o1/o3), and Mistral/Magistral models. Includes new presets:$deep-analysisand$quick-reasoning. - Image-to-Image Generation:
PipeImgGennow supports input images viainput_imagesfield, withInputImagesTaxonomyfor provider-specific handling and variable reference detection ({{ var }},$var,@var) in prompts. - PipeCompose "Construct" Mode: New
constructmode for building structured objects (dictionaries/Pydantic models) directly from variables. - Nested concepts in inline structures: You can now define nested structures for your concepts in your
.plxfiles. Learn more here: Nested Concepts in Inline Structures.
Breaking Changes
- CLI restructure:
method/pipesubcommands —pipelex run,pipelex validate, and allpipelex buildsubcommands (runner,inputs,output) now require an explicitmethodorpipekeyword. For example:pipelex run method my-methodorpipelex run pipe scoring.compute. The oldpipelex run <target>form is no longer supported. The agent CLI (pipelex-agent) follows the same structure.
Added
- Method resolution (
cli/method_resolver.py) — resolves installed method names to pipe codes and library directories. Integrates withmthdsdiscovery module to find methods in~/.mthds/methods/and./.mthds/methods/. run methodcommand — run an installed method by name, optionally overriding the pipe with--pipe.validate methodcommand — validate all bundles in an installed method.build runner/inputs/output methodcommands — generate build artifacts for installed methods.- Persistent Credential Storage (
~/.pipelex/.env):pipelex initnow prompts for missing API keys after backend selection and saves them to~/.pipelex/.envwith0600permissions. Credentials are loaded automatically at startup (global defaults, overridden by project.env). pipelex init credentialsCommand: New standalone focus to re-enter API keys without resetting configuration. Scans enabled backends, detects missing env vars, and prompts only for those.- Package Management System: Introduced a full package manager for MTHDS with
METHODS.tomlmanifests,methods.locklockfile generation, local path and remote Git-based dependency resolution (MVS), a local package cache (~/.mthds/packages), and a newpipelex pkgCLI command group (init,list,add,lock,install,update,publish,search,inspect,graph). - Hierarchical Domains: Support for dot-separated domain namespaces (e.g.,
legal.contracts.shareholder) and cross-package references viaalias->domain.pipesyntax. - PipelexRunner: Introduced the
PipelexRunnerclass as the primary entry point for executing pipelines, replacingPipelexClientand standaloneexecute_pipeline/start_pipelinefunctions. - Linting & Formatting: Integrated
plxt(Pipelex Tools) for formatting and linting.mthdsand.tomlfiles, including CI/CD checks alongsideruff. - JSON Schema: Automatic generation of
mthds_schema.jsonfor standard definition, IDE validation, and auto-completion. - Dev CLI: Added
pipelex-devCLI for internal development tasks (e.g., schema generation). - CSP Nonce Support: Added Content Security Policy nonce support for generated HTML graphs (Mermaid/ReactFlow) to enable secure rendering in VS Code webviews.
- OpenRouter Backend: Added OpenRouter as an inference backend with 337 chat model definitions and 14 image generation models.
- Builder Auto-Repair: Self-healing capabilities including auto-generation of undeclared concepts, multiplicity mismatch fixes, and pruning of unreachable pipes/unused concepts.
- Concept Field Features: Added
choicessupport (compiles toLiteraltypes/Enums) and explicitlisttype withitem_type/item_concept_ref. - PipeFunc Execution Error Context: When a
@pipe_funcfunction fails during execution, the error message now includes detailed context: the function name, actual input values from working memory, expected output type, and the original error with its type. This makes debugging PipeFunc errors much easier. pipelex build outputCLI Command: New command to generate example output JSON for a pipe, complementing the existingpipelex build inputscommand. Shows the expected output structure based on the pipe's output concept type, with multiplicity support. For pipes withnative.Anythingoutput (e.g.,PipeConditionwith different mapped pipe outputs), displays all possible outputs from mapped pipes.- Telemetry System: Introduced anonymous usage tracking and exception capture for CLI commands (
graph render), reporting to both user-configured and Pipelex analytics endpoints. - PipeExtract Operator Validation: Added strict input validation that raises configuration errors for incompatible input types or when document-specific parameters are used with image inputs.
- PipeCondition Output Auto-Fix in Builder Loop: The pipe builder now automatically fixes
PipeConditionoutput concept errors during validation. If all mapped pipes have the same output, thePipeConditionoutput is set to that concept; otherwise it's set tonative.Anything. - PipeFunc Return Type Validation: Added validation to ensure that a
PipeFuncfunction's return type matches the output concept's structure class. - URL Validation on
ImageContentandDocumentContent: Both models validate external resources (HTTP/HTTPS URLs via HEAD request, local file paths via existence check) through avalidate_resources()method called during pipeline execution (validate_before_run) rather than at model instantiation. Internal URIs (data:,pipelex-storage://) skip validation entirely. Validation is skipped in dry-run mode where inputs use mock URLs. - Literal Type Support in Dry Run Mocks:
DryRunFactorynow detectsLiteraltype annotations (includingOptional[Literal[...]]) and generates valid mock values by randomly picking from the allowed choices instead of producing invalid random strings. - Literal Type Support in
pipelex build inputs/pipelex build output: The concept representation generator now handlesLiteralfields by picking a random value from the allowed choices, and generates mock URL patterns forurlfields. - Literal Error Handling in Validation Messages: Pydantic validation error formatting now recognizes
literal_errortypes and displays them as "Invalid choice errors" with the actual value and expected options. PipeRunErrorCatching in Bundle Validation:validate_bundlenow catchesPipeRunErrorduring dry run and wraps it inValidateBundleErrorwith a clear message.ValidationErrorCatching in Pipeline Execution:execute_pipelinenow catches PydanticValidationErrorfrom input construction, formats the errors, and raises aPipeExecutionErrorwith a clear message.- Broader Error Handling in CLI
pipelex run: The CLI run command now catchesPipelexErrorin addition toPipelineExecutionError, providing better error messages for failures that occur outside the pipeline execution itself. - Content Filenames: Added
filenamefield toImageContentandDocumentContent, with auto-population from local file paths via newextract_filename_from_urihelper. - Batch Validation Error Type: Introduced
PipeValidationErrorType.BATCH_ITEM_NAME_COLLISIONfor naming conflicts in batch operations. - Documentation: Added naming convention rules to
builder.plxandpipe_design.plx(batch input lists should be plural, item names singular). --library-dirCLI Option:--library-dir/-Loption forpipelex run,pipelex validate, andpipelex buildsubcommands (one-shot-pipe,partial-pipe) to specify additional directories for searching pipe definitions. Can be specified multiple times.- Automatic File Loading: The core pipeline execution functions (
pipelex.execute_pipeline,pipelex.start_pipeline) can now directly load a pipeline from a file path via a newbundle_uriparameter. - Dry Run Mode:
pipelex run --dry-runexecutes pipeline logic without API calls, useful for validating structure and generating orchestration graphs. Combine with--mock-inputsto generate mock data for missing required inputs. | with_imagesJinja2 Filter: Explicitly extract and include all nested images from complex data structures (e.g.,Pageobjects or custom concepts withImagefields). Renders the object's text representation while making associated images available to the LLM.- System Prompt Media Support: Reference images and documents in
system_promptusing the same$variableand@variablesyntax as the userprompt. - Pipelex Gateway Service: Terms of service management via
.pipelex/pipelex_service.tomland interactive acceptance flow inpipelex initandpipelex init agreement. - Gateway Available Models Documentation: Auto-generated reference of all LLM, Document Extraction, and Image Generation models available through the Gateway.
- Configurable Retry Logic: Exponential backoff for inference API calls, configurable in
pipelex.tomlunder[cogt.tenacity_config] - Context Manager Support: The
Pipelexclass now supportswith Pipelex.make(): ...for graceful shutdown - Validation Improvements:
- Pipelex Bundle concept keys: prevent bundles from re-creating a native concept
PipeSequence: output multiplicity must match the last step's output multiplicityPipeFunc: output multiplicity must match the function return type (ListContentsubclass for multiplicity=true)- Rendering Protocols: Three new
@runtime_checkableprotocols (ImageRenderable,TagRenderable,TextFormatRenderable) to formalize the interaction between data types and Jinja2 filters. - Unified URI Handling System: New
pipelex.tools.urimodule providing type-safe parsing for HTTP/HTTPS URLs, local file paths, file URIs,pipelex-storage://URIs, and base64 data URLs. - Automatic Input Data Storage: Pipeline pre-processing step that converts large
data:URLs in anImageContentorDocumentContentintopipelex-storage://URIs for improved performance, by storing the data with the configured storage provider. Configurable viais_normalize_data_urls_to_storageinpipelex.toml. pipelex build inputsCommand: New CLI command to generate example input JSON files for pipes. Supports--library-dir(-L) to specify library directories.- Pipe Code Syntax Validation: Bundle validation now checks that pipe codes and
main_pipevalues use valid snake_case syntax, with proper error categorization (INVALID_PIPE_CODE_SYNTAX).
Changed
- CI Runner Optimization: Split self-hosted runners into dedicated test (D32) and lint (D4) pools with pre-baked Docker image (Python 3.10-3.14, UV, dependency cache) for faster job startup.
- MTHDS Light Client Extraction: The light client protocol (runner, pipeline models, pipe output abstractions) has been extracted from pipelex into the new
mthdspackage on PyPI. Pipelex now depends onmthds>=0.0.1and implements itsRunnerProtocol. - Global/Local Config Split:
pipelex initnow creates configuration in~/.pipelex/(global) by default. Usepipelex init --localto create project-level overrides in{project_root}/.pipelex/. Config loading merges: package defaults → global → project → overrides. - IDE Extension Detection: Extension check now uses
code --list-extensions/cursor --list-extensionsfor reliable detection instead of folder scanning. Shows separate marketplace links for VS Code (Microsoft Marketplace) and Cursor (Open VSX Registry). - Quieter
pipelex init: Removed verbose file listing and reset messages from config initialization output. pipelex initHelp Text: Detailed focus option descriptions now shown inpipelex init --helpexplaining each focus (all,config,credentials,inference,routing,telemetry,agreement).- File Extension: Pipeline definitions now use
.mthdsinstead of.plx; the project refers to "Methods" (MTHDS) rather than "Pipelex workflows". - Syntax: In
PipeParalleldefinitions,parallelsfield renamed tobranches;plx_configinpipelex.tomlrenamed tomthds_config. - Documentation: Comprehensive update to reflect
.mthdsfile format, package management, and new project structure. - Model Deck Updates: Default premium model now
claude-4.6-opus; added Mistral models (mistral-small-3.2,mistral-large,magistralseries) andgpt-5placeholders. - Jinja2 Rendering: Image objects now replaced with placeholders (e.g.,
[Image 1]) during text generation; async filters only registered in async environments. - Backend Configurations: Added
effort_to_level_mapandeffort_to_budget_mapsfor reasoning translation; disabled Google Vertex AI by default. - Dependencies: Updated
pypdfium2,anthropic, andmistralaiversion constraints. pipelex run --dry-run: No longer pretty prints the main_stuff output, matching the expected behavior for dry runs where no actual inference occurs.pipelex build structuresCommand: Now uses a lightweight loading mechanism that only processes domains and concepts, skipping pipe loading and validation entirely. This fixes the chicken-and-egg problem where structure generation would fail due to pipe validation errors before the structures were even created. Added--force/-fflag to regenerate all structures without checking if classes already exist.- Test Profile System: Refactored integration tests to use a new configuration system (
.pipelex/test_profiles.toml) withdev,ci, andfullprofiles for controlling which AI models are used in parametrized tests, replacing runtime filtering and hardcoded model lists. pipelex run --graphFlag: Now acts as an override forpipelex.tomlsettings instead of defaulting totrue.- Default Image Generation Models: Updated in
base_deck.toml:base-img-gen:flux-2-pro,best-img-gen:nano-banana-pro,fast-img-gen:gpt-image-1-mini - Remote Configuration: Updated service URL to version 3.
- GatewayExtractWorker: Now checks model capabilities before attempting image captioning.
- Change the output validation of
PipeCondition: If all mapped pipes have the same output concept,PipeCondition's output MUST be that same concept. If mapped pipes have different output concepts,PipeCondition's output MUST be the native conceptAnything. - CLI: Changed
pipelex validate alltopipelex validate --all(or-a). - StructuredContent.rendered_html(): Now recursively calls
rendered_html()on nestedStuffContentfields instead of using json2html conversion. Also skipsNonevalues and uses HTML table format. - Batch Pipe Validation: Enforced stricter naming rules for batch specs—
input_item_namemust differ frominput_list_nameand not shadow existing input keys, with clearer error messages suggesting plural/singular conventions. - Jinja2 Integration Refactored to Protocol-Based Approach: Replaced the
Jinja2Registrysingleton and handler functions with a decoupled protocol-based system, eliminating circular dependencies between the template layer and core domain logic. StuffArtefactRedesigned as Delegation Adapter: Now a lightweight, immutable adapter that delegates attribute access directly to underlyingStuffandStuffContentobjects, improving performance and providing more intuitive field access.- Image Extraction Moved to Content Types:
StructuredContent,ListContent,ImageContent, andTextAndImagesContentnow implement theImageRenderableprotocol, replacing centralized handler logic. - ⚠️ Breaking: User override config renamed from
pipelex_super.tomltopipelex_override.toml. - Image Generation Architecture: Refactored to taxonomy-based approach. Standardizes parameter translation (
aspect_ratio,quality,output_format) to provider-specific APIs. - Document Extraction Improvements:
pypdfium2extractor now extracts embedded images from PDFs. Response parsing uses dedicated Pydantic schemas for validation. - Default Model Change:
extract_text_from_visualsdeck now defaults toazure-document-intelligence pipelex_inferencereplaced bypipelex_gateway: See Highlights for migration details. NewPIPELEX_GATEWAY_API_KEYenvironment variable; default routing profiles updated topipelex_gateway_first.-
Telemetry System Split: Now two separate streams: 1. Pipelex Gateway telemetry for service monitoring (never collects prompts/completions/business data) 2. Custom telemetry to user-configured backends 3. Config updated accordingly (
telemetry.toml):- Renamed
[posthog]to[custom_posthog]to distinguish user's PostHog from Pipelex Gateway telemetry - Added new
[custom_portkey]section withforce_debug_enabledandforce_tracing_enabledsettings
- Renamed
-
Main Configuration Overrides Updated (
.pipelex/pipelex.toml): pipelex_override.toml(final override) renamed frompipelex_super.tomltopipelex_override.tomland moved from repo root to.pipelex/directorytelemetry_override.toml(personal telemetry settings)-
is_generate_cost_report_file_enableddefault changed fromtruetofalse -
Documentation:
- Clarified Setup (first run) vs Configuration (TOML reference), added a Setup overview page, and added contributor docs for configuration defaults/overrides.
- Added the "Under the Hood" page documenting the execution graph tracing system.
pipelex init: Now creates a documentedtelemetry.tomltemplate instead of prompting for preferences- Model Catalog Updated: Latest models (gpt-5.1, claude-4.5-opus, gemini-3.0-pro, etc.) and updated waterfalls in
base_deck.toml - Model Constraints Refactored: From simple lists to structured
valued_constraintsdictionaries (e.g.,valued_constraints = { fixed_temperature = 1 }) - OpenAI Responses API: New implementation now differentiates between
openai_completionsandopenai_responses - CLI Initialization: Commands refactored to use centralized
Pipelexinitialization factory for improved error handling pipelex doctor: Enhanced to detect outdatedtelemetry.tomlformats and suggest fixes--output-dirOption: Therunner,structures, andinputsCLI commands now accept this option- Cost Report: Now displays a note clarifying that it only includes LLM costs
descriptionField Now Required: InPipeAbstract,PipeBlueprint, andPipeSpecclasses.- Configuration: New
[pipelex.pipeline_execution_config.graph_config]section inpipelex.tomlfor fine-grained control over graph generation, data embedding, and rendering options. - CLI: All
pipelexcommands now accept--no-logoto suppress the Pipelex banner in the terminal — useful to reduce tokens.
Fixed
find_project_rootHome Directory Bug: The project root walker no longer considers the home directory (~) as a project root, even if it contains stray marker files likepackage.json.- Python 3.10 Compatibility: Fixed
datetime.UTCimport (Python 3.11+) to usedatetime.timezone.utc. - Graph Rendering: Fixed dashed edge rendering for
PipeBatchandPipeParallelrelationships. - Image Generation Response Parsing: Hardened image response parsing to handle varied provider response formats more robustly.
- Helpful Error for
get_stuff_as(ListContent[T]): When users incorrectly callget_stuff_as("name", ListContent[Something])instead ofget_stuff_as_list("name", Something), the error message now explicitly suggests usingget_stuff_as_list(). - PipeFunc
ListContent[T]Validation: Fixed validation rejecting validListContent[T]return types for array outputs (T[]). Previously, a function returningListContent[Expense]would fail validation foroutput = "Expense[]"with a misleading error. The validation now correctly extracts and validates the generic type parameter from Pydantic's metadata. - PipeFunc Class Name Matching: Fixed validation failing when the return type class and concept structure class are logically the same but loaded from different contexts. The validation now uses class name matching as a fallback, allowing
ListContent[Expense]to matchExpense[]even if theExpenseclass objects differ. - Fixed
PipeImgGennot properly convertingImageContentto custom subclasses (e.g.,Receipt(ImageContent)). The pipe now usessmart_dump()beforemodel_validate()to correctly instantiate the output concept's structure class. - Corrected output directory creation logic in
pipelex runto properly respect the--no-graphflag and configuration settings. - Fixed a bug when trying to print HTML content in a TextContent object.
- Fixed the Pipelex CLI for generating structures, inputs, runner files.
- Fixed
@pipe_funcdecorated functions showing "function not found" instead of explaining why the function is ineligible (e.g., missing return type annotation). - Fixed PipeLLM with list output (e.g.,
output = "Item[]") not producingListContentwhen run inside a nested PipeSequence withbatch_over. - Duplicate Pipe Error Message: When a pipe code is declared in multiple
.plxfiles (or twice in the same file), the error message now shows which bundle file(s) contain the conflicting declarations instead of a misleading message about "running the same pipe twice in the same pipeline". - Fixed
pipelex build runnerandpipelex build inputsgenerating string placeholders (e.g.,"number_int | float") instead of numeric values for Number concepts withint | floatunion type fields. - Fixed structure generation failing with
PydanticUserErrorwhen a concept structure references native concepts (e.g.,native.Html). The generator now properly resolves native concept refs to their content classes (e.g.,HtmlContent) with correct imports. - Nested Image Handling: Images nested within structured data are now properly replaced with
[Image N]tokens. The| with_imagesfilter andImageRegistrysystem correctly extract images from complex nested structures. PipeConditionValidation: Output multiplicity validation now works correctly.- Error Reporting:
PipeComposevalidation errors now include formatted details and failing field values. - Duplicate Pipeline Registration: Running a pipeline from a file that was also part of a pre-loaded library (via
PIPELEXPATH) no longer causes a duplicate domain registration error. The system now tracks absolute paths of loaded library files and skips files already loaded. pipelex build structures: Corrected output file naming and resolved import path generation.ConceptFactory.make_from_blueprint: Now correctly handles native concepts.
Removed
- Legacy Client & Execution Modules: Removed
PipelexClient, protocol files, and standalonepipelex.pipeline.execute/pipelex.pipeline.startmodules (replaced byPipelexRunnerandmthdspackage). - Legacy Config: Removed
plx_config.pyand.plx-specific configuration references. pipelex kitCommand: The kit commands have been removed from the main CLI. They are now internal tools for Pipelex contributors only, available viapipelex-dev kit rules.pipelex kit migrationsCommand: Removed entirely.pipelex kit remove-rulesCommand: Removed entirely.- PLX Syntax Agent Rules: Removed
write_pipelex.mdandrun_pipelex.mdagent rules. These PLX syntax guides are no longer installed in client projects. [pipelex.kit_config]Configuration: Removed from client project configuration (.pipelex/pipelex.toml).openai_utilsModule: Removedpipelex.plugins.openai.openai_utils; logic now in centralized image preparation utilities.- Pipeline Tracking feature: Removed entirely, including the
pipelex/pipeline/trackmodule,PipelineTrackercomponents, related configuration, tracker calls in pipe controllers, and associated documentation. FlowGenerator: The old flow generator has been removed.
Security
- CSP Nonce Support: Added Content Security Policy nonce for generated HTML graph outputs.
Deprecated
pipelex_inferencebackend in favor ofpipelex_gateway(marked as "🛑 Legacy" in configuration template)
For Contributors
- Technical Documentation: Added a new "Under the Hood" page documenting the
StuffArtefactdelegation pattern and image rendering architecture. - Enhanced Testing: Added extensive unit and integration tests for the protocol-based rendering system, including nested image extraction and filter error conditions.
- Agent Rules: Added
pipelex_standards.mdoutlining standards for the Pipelex configuration system, also included as rules for AI development agents. - Agent Rules: New target
make agent-checkfor faster linting. - ⚠️ Breaking — Content Handling Overhaul:
GeneratedImagereplaced by internalGeneratedImageRawDetails;ImageContentis now standard (withoutbase_64field).PipeExtractoutputsPageContentlist directly. Content persistence now handled automatically by storage system. - ⚠️ Breaking — Image Prompt Representation: Redesigned
PromptImagemodels—consolidatedPromptImagePathandPromptImageUrlintoPromptImageUri; now uses Pydantic discriminated union for URI, base64, and raw bytes sources. - Centralized Image Preparation: Moved image fetching and base64 conversion logic to
pipelex.cogt.image.prompt_image_utils, simplifying LLM provider plugins (Anthropic, Google, Mistral, OpenAI). - Unified Resource Loading: Updated all file/URL reading components (PDF renderers, document extractors) to use the new URI handling system, replacing the legacy
pipelex.tools.misc.path_utilsmodule. - Async HTTP Fetching: Renamed
fetch_file_from_url_httpx_asynctofetch_file_from_url_httpx; removed redundant synchronous version. PreparedImageAbstraction: New models (PreparedImageHttpUrl,PreparedImageBase64) representing images ready for LLM provider APIs.- Pipelex Gateway Model Management: New CLI commands (
pipelex-dev update-gateway-models,pipelex-dev check-gateway-models) and correspondingmaketargets (ugm,cgm) to generate and verify gateway model catalog. CI now validates this documentation is up-to-date. - Test Suite Pre-flight Check: Verifies Gateway terms acceptance before running tests, providing clear error messages.
- Content Rendering:
StuffContent.rendered_*methods now provide both synchronous and asynchronous variants.
Migration Notes
- Telemetry configuration migration: If you have an existing
telemetry.toml, rename: [posthog]→[custom_posthog][posthog.tracing]→[custom_posthog.tracing][posthog.tracing.capture]→[custom_posthog.tracing.capture]- Or run
pipelex init telemetry --resetto regenerate the file with the new structure
Refactored
- Anthropic Backend: Internal streaming for standard completions to prevent SDK timeouts, configurable structured output timeout (
structured_output_timeout_seconds), improved error mapping, and increased Bedrock Claudemax_tokens(8K → 64K) with removal ofmax_output_tokens_limitconstraint. - ⚠️ Breaking — Pipe I/O Specification: The output (and inputs) of a pipe is now a
StuffSpecobject that holds the concept and the multiplicity. - Naming Convention: Renamed
domaintodomain_codewhere relevant. - Dry Run Methods: Refactored the dry run methods of the
PipeAbstractclass.
[v0.18.0b4] - 2026-02-23
Added
- Persistent Credential Storage (
~/.pipelex/.env):pipelex initnow prompts for missing API keys after backend selection and saves them to~/.pipelex/.envwith0600permissions. Credentials are loaded automatically at startup (global defaults, overridden by project.env). pipelex init credentialsCommand: New standalone focus to re-enter API keys without resetting configuration. Scans enabled backends, detects missing env vars, and prompts only for those.- Package Management System: Introduced a full package manager for MTHDS with
METHODS.tomlmanifests,methods.locklockfile generation, local path and remote Git-based dependency resolution (MVS), a local package cache (~/.mthds/packages), and a newpipelex pkgCLI command group (init,list,add,lock,install,update,publish,search,inspect,graph). - Hierarchical Domains: Support for dot-separated domain namespaces (e.g.,
legal.contracts.shareholder) and cross-package references viaalias->domain.pipesyntax. - PipelexRunner: Introduced the
PipelexRunnerclass as the primary entry point for executing pipelines, replacingPipelexClientand standaloneexecute_pipeline/start_pipelinefunctions. - Linting & Formatting: Integrated
plxt(Pipelex Tools) for formatting and linting.mthdsand.tomlfiles, including CI/CD checks alongsideruff. - JSON Schema: Automatic generation of
mthds_schema.jsonfor standard definition, IDE validation, and auto-completion. - Dev CLI: Added
pipelex-devCLI for internal development tasks (e.g., schema generation). - CSP Nonce Support: Added Content Security Policy nonce support for generated HTML graphs (Mermaid/ReactFlow) to enable secure rendering in VS Code webviews.
- OpenRouter Backend: Added OpenRouter as an inference backend with 337 chat model definitions and 14 image generation models.
Changed
- CI Runner Optimization: Split self-hosted runners into dedicated test (D32) and lint (D4) pools with pre-baked Docker image (Python 3.10-3.14, UV, dependency cache) for faster job startup.
- MTHDS Light Client Extraction: The light client protocol (runner, pipeline models, pipe output abstractions) has been extracted from pipelex into the new
mthdspackage on PyPI. Pipelex now depends onmthds>=0.0.1and implements itsRunnerProtocol. - Global/Local Config Split:
pipelex initnow creates configuration in~/.pipelex/(global) by default. Usepipelex init --localto create project-level overrides in{project_root}/.pipelex/. Config loading merges: package defaults → global → project → overrides. - IDE Extension Detection: Extension check now uses
code --list-extensions/cursor --list-extensionsfor reliable detection instead of folder scanning. Shows separate marketplace links for VS Code (Microsoft Marketplace) and Cursor (Open VSX Registry). - Quieter
pipelex init: Removed verbose file listing and reset messages from config initialization output. pipelex initHelp Text: Detailed focus option descriptions now shown inpipelex init --helpexplaining each focus (all,config,credentials,inference,routing,telemetry,agreement).- File Extension: Pipeline definitions now use
.mthdsinstead of.plx; the project refers to "Methods" (MTHDS) rather than "Pipelex workflows". - Syntax: In
PipeParalleldefinitions,parallelsfield renamed tobranches;plx_configinpipelex.tomlrenamed tomthds_config. - Documentation: Comprehensive update to reflect
.mthdsfile format, package management, and new project structure.
Fixed
find_project_rootHome Directory Bug: The project root walker no longer considers the home directory (~) as a project root, even if it contains stray marker files likepackage.json.- Python 3.10 Compatibility: Fixed
datetime.UTCimport (Python 3.11+) to usedatetime.timezone.utc. - Graph Rendering: Fixed dashed edge rendering for
PipeBatchandPipeParallelrelationships. - Image Generation Response Parsing: Hardened image response parsing to handle varied provider response formats more robustly.
Removed
- Legacy Client & Execution Modules: Removed
PipelexClient, protocol files, and standalonepipelex.pipeline.execute/pipelex.pipeline.startmodules (replaced byPipelexRunnerandmthdspackage). - Legacy Config: Removed
plx_config.pyand.plx-specific configuration references.
Security
- CSP Nonce Support: Added Content Security Policy nonce for generated HTML graph outputs.
[v0.18.0b3] - 2026-02-11
Highlights
- Agent CLI (
pipelex-agent): New machine-first CLI for AI agents with structured JSON output for all commands (build,run,validate,inputs,concept,pipe,assemble,graph,models,doctor). - LLM Reasoning Controls: Unified support for "Thinking" models (Chain of Thought) with
reasoning_effort,reasoning_budget, andthinking_modeparameters. Supports Anthropic Extended Thinking, Google Gemini Thinking, OpenAI Reasoning (o1/o3), and Mistral/Magistral models. Includes new presets:$deep-analysisand$quick-reasoning. - Image-to-Image Generation:
PipeImgGennow supports input images viainput_imagesfield, withInputImagesTaxonomyfor provider-specific handling and variable reference detection ({{ var }},$var,@var) in prompts. - PipeCompose "Construct" Mode: New
constructmode for building structured objects (dictionaries/Pydantic models) directly from variables. - Nested concepts in inline structures: You can now define nested structures for your concepts in your
.plxfiles. Learn more here: Nested Concepts in Inline Structures.
Added
- Builder Auto-Repair: Self-healing capabilities including auto-generation of undeclared concepts, multiplicity mismatch fixes, and pruning of unreachable pipes/unused concepts.
- Concept Field Features: Added
choicessupport (compiles toLiteraltypes/Enums) and explicitlisttype withitem_type/item_concept_ref. - PipeFunc Execution Error Context: When a
@pipe_funcfunction fails during execution, the error message now includes detailed context: the function name, actual input values from working memory, expected output type, and the original error with its type. This makes debugging PipeFunc errors much easier. pipelex build outputCLI Command: New command to generate example output JSON for a pipe, complementing the existingpipelex build inputscommand. Shows the expected output structure based on the pipe's output concept type, with multiplicity support. For pipes withnative.Anythingoutput (e.g.,PipeConditionwith different mapped pipe outputs), displays all possible outputs from mapped pipes.- Telemetry System: Introduced anonymous usage tracking and exception capture for CLI commands (
graph render), reporting to both user-configured and Pipelex analytics endpoints. - PipeExtract Operator Validation: Added strict input validation that raises configuration errors for incompatible input types or when document-specific parameters are used with image inputs.
- PipeCondition Output Auto-Fix in Builder Loop: The pipe builder now automatically fixes
PipeConditionoutput concept errors during validation. If all mapped pipes have the same output, thePipeConditionoutput is set to that concept; otherwise it's set tonative.Anything. - PipeFunc Return Type Validation: Added validation to ensure that a
PipeFuncfunction's return type matches the output concept's structure class. - URL Validation on
ImageContentandDocumentContent: Both models validate external resources (HTTP/HTTPS URLs via HEAD request, local file paths via existence check) through avalidate_resources()method called during pipeline execution (validate_before_run) rather than at model instantiation. Internal URIs (data:,pipelex-storage://) skip validation entirely. Validation is skipped in dry-run mode where inputs use mock URLs. - Literal Type Support in Dry Run Mocks:
DryRunFactorynow detectsLiteraltype annotations (includingOptional[Literal[...]]) and generates valid mock values by randomly picking from the allowed choices instead of producing invalid random strings. - Literal Type Support in
pipelex build inputs/pipelex build output: The concept representation generator now handlesLiteralfields by picking a random value from the allowed choices, and generates mock URL patterns forurlfields. - Literal Error Handling in Validation Messages: Pydantic validation error formatting now recognizes
literal_errortypes and displays them as "Invalid choice errors" with the actual value and expected options. PipeRunErrorCatching in Bundle Validation:validate_bundlenow catchesPipeRunErrorduring dry run and wraps it inValidateBundleErrorwith a clear message.ValidationErrorCatching in Pipeline Execution:execute_pipelinenow catches PydanticValidationErrorfrom input construction, formats the errors, and raises aPipeExecutionErrorwith a clear message.- Broader Error Handling in CLI
pipelex run: The CLI run command now catchesPipelexErrorin addition toPipelineExecutionError, providing better error messages for failures that occur outside the pipeline execution itself. - Content Filenames: Added
filenamefield toImageContentandDocumentContent, with auto-population from local file paths via newextract_filename_from_urihelper. - Batch Validation Error Type: Introduced
PipeValidationErrorType.BATCH_ITEM_NAME_COLLISIONfor naming conflicts in batch operations. - Documentation: Added naming convention rules to
builder.plxandpipe_design.plx(batch input lists should be plural, item names singular).
Changed
- Model Deck Updates: Default premium model now
claude-4.6-opus; added Mistral models (mistral-small-3.2,mistral-large,magistralseries) andgpt-5placeholders. - Jinja2 Rendering: Image objects now replaced with placeholders (e.g.,
[Image 1]) during text generation; async filters only registered in async environments. - Backend Configurations: Added
effort_to_level_mapandeffort_to_budget_mapsfor reasoning translation; disabled Google Vertex AI by default. - Dependencies: Updated
pypdfium2,anthropic, andmistralaiversion constraints. pipelex run --dry-run: No longer pretty prints the main_stuff output, matching the expected behavior for dry runs where no actual inference occurs.pipelex build structuresCommand: Now uses a lightweight loading mechanism that only processes domains and concepts, skipping pipe loading and validation entirely. This fixes the chicken-and-egg problem where structure generation would fail due to pipe validation errors before the structures were even created. Added--force/-fflag to regenerate all structures without checking if classes already exist.- Test Profile System: Refactored integration tests to use a new configuration system (
.pipelex/test_profiles.toml) withdev,ci, andfullprofiles for controlling which AI models are used in parametrized tests, replacing runtime filtering and hardcoded model lists. pipelex run --graphFlag: Now acts as an override forpipelex.tomlsettings instead of defaulting totrue.- Default Image Generation Models: Updated in
base_deck.toml:base-img-gen:flux-2-pro,best-img-gen:nano-banana-pro,fast-img-gen:gpt-image-1-mini - Remote Configuration: Updated service URL to version 3.
- GatewayExtractWorker: Now checks model capabilities before attempting image captioning.
- Change the output validation of
PipeCondition: If all mapped pipes have the same output concept,PipeCondition's output MUST be that same concept. If mapped pipes have different output concepts,PipeCondition's output MUST be the native conceptAnything. - CLI: Changed
pipelex validate alltopipelex validate --all(or-a). - StructuredContent.rendered_html(): Now recursively calls
rendered_html()on nestedStuffContentfields instead of using json2html conversion. Also skipsNonevalues and uses HTML table format. - Batch Pipe Validation: Enforced stricter naming rules for batch specs—
input_item_namemust differ frominput_list_nameand not shadow existing input keys, with clearer error messages suggesting plural/singular conventions.
Fixed
- Helpful Error for
get_stuff_as(ListContent[T]): When users incorrectly callget_stuff_as("name", ListContent[Something])instead ofget_stuff_as_list("name", Something), the error message now explicitly suggests usingget_stuff_as_list(). - PipeFunc
ListContent[T]Validation: Fixed validation rejecting validListContent[T]return types for array outputs (T[]). Previously, a function returningListContent[Expense]would fail validation foroutput = "Expense[]"with a misleading error. The validation now correctly extracts and validates the generic type parameter from Pydantic's metadata. - PipeFunc Class Name Matching: Fixed validation failing when the return type class and concept structure class are logically the same but loaded from different contexts. The validation now uses class name matching as a fallback, allowing
ListContent[Expense]to matchExpense[]even if theExpenseclass objects differ. - Fixed
PipeImgGennot properly convertingImageContentto custom subclasses (e.g.,Receipt(ImageContent)). The pipe now usessmart_dump()beforemodel_validate()to correctly instantiate the output concept's structure class. - Corrected output directory creation logic in
pipelex runto properly respect the--no-graphflag and configuration settings. - Fixed a bug when trying to print HTML content in a TextContent object.
- Fixed the Pipelex CLI for generating structures, inputs, runner files.
- Fixed
@pipe_funcdecorated functions showing "function not found" instead of explaining why the function is ineligible (e.g., missing return type annotation). - Fixed PipeLLM with list output (e.g.,
output = "Item[]") not producingListContentwhen run inside a nested PipeSequence withbatch_over. - Duplicate Pipe Error Message: When a pipe code is declared in multiple
.plxfiles (or twice in the same file), the error message now shows which bundle file(s) contain the conflicting declarations instead of a misleading message about "running the same pipe twice in the same pipeline". - Fixed
pipelex build runnerandpipelex build inputsgenerating string placeholders (e.g.,"number_int | float") instead of numeric values for Number concepts withint | floatunion type fields. - Fixed structure generation failing with
PydanticUserErrorwhen a concept structure references native concepts (e.g.,native.Html). The generator now properly resolves native concept refs to their content classes (e.g.,HtmlContent) with correct imports.
Removed
pipelex kitCommand: The kit commands have been removed from the main CLI. They are now internal tools for Pipelex contributors only, available viapipelex-dev kit rules.pipelex kit migrationsCommand: Removed entirely.pipelex kit remove-rulesCommand: Removed entirely.- PLX Syntax Agent Rules: Removed
write_pipelex.mdandrun_pipelex.mdagent rules. These PLX syntax guides are no longer installed in client projects. [pipelex.kit_config]Configuration: Removed from client project configuration (.pipelex/pipelex.toml).
[v0.18.0b2] - 2026-01-20
Highlights
- Pipelex Gateway — The deprecated
pipelex_inferencebackend is now replaced bypipelex_gateway, featuring remote model configuration fetching so you always have access to the latest models without updating Pipelex.
Getting your API key:
1. Get your API key at app.pipelex.com
2. Add it to your .env file: PIPELEX_GATEWAY_API_KEY=your-key-here
Gateway Supported models — included with the Free API Key - Language Models (LLM): - OpenAI: all models up to GPT-5.2 and Codex - Anthropic: Claude 3.7 Sonnet through Claude 4.5 Haiku/Sonnet/Opus - Google: Gemini 2.0/2.5 Flash, Gemini 2.5 Pro, Gemini 3.0 Flash/Pro - xAI: Grok 3/mini, Grok 4 (+ fast reasoning variants) - Open-source: Mistral Large 3, DeepSeek v3.1/3.2/Speciale, GPT-OSS 20B/120B1, Kimi K2, Phi-4, Qwen3-VL 235B - Document Extraction: Mistral Document AI, Azure Document Intelligence, DeepSeek OCR - Image Generation: GPT-Image 1/1.5, Flux 2 Pro, Nano Banana/Pro
Accepting the Terms of Service:
When you run pipelex init, you'll be prompted to accept the Gateway terms of service. By using Pipelex Gateway, telemetry is automatically enabled and identified by your API key (hashed for security) to monitor service quality and enforce fair usage. We only collect technical data (model names, token counts, latency, error rates)—never your prompts, completions, or business data. See our Privacy Policy.
⚠️ Migration deadline: If you were using pipelex_inference, please migrate soon—the legacy service will be shut down within few days. Get your new Gateway key at app.pipelex.com.
- Execution Graph Visualization System (preview feature) — Comprehensive tracing and visualization for pipeline executions.
CLI: --graph flag on pipelex run generates execution graphs. New pipelex graph render <graph.json> command for post-run rendering.
Viewers: Interactive ReactFlow viewer (reactflow.html) with pan/zoom and node inspector. Mermaid diagrams (mermaidflow.html) with subgraphs and clickable nodes.
Makefile: make view-graph (vg) and make serve-graph (sg) to start a local graph viewer.
- Pydantic Structure Generation — Two new CLI commands bridge Pipelex's declarative concepts with your Python code:
pipelex build structures /library_dir/ — Generates Pydantic models from all concept definitions found in the specified library directory. Now you have your structures as Python code: you can iterate on them, add custom validation functions, or use them as type hints in your code.
pipelex build runner — Now automatically generates both the Python runner file AND the required Pydantic structures. When you run this command, it creates a complete, ready-to-execute Python script that imports the generated structures, so you can immediately use typed objects in your pipeline code.
See the Build Commands documentation for usage examples.
- New Backends & Models:
- Hugging Face Inference — Support for Hugging Face Inference API, including
qwen-imagetext-to-image model. - Google
gemini-3.0-flash-preview - Mistral OCR latest model
mistral-ocr-2512 - Scaleway inference provider support for open-source models
-
Portkey AI backend integration for unified access to multiple models through a single API key
-
Document Support in
PipeLLM— IncludeDocumentobjects (like PDFs) directly in prompts using@variableor$variablesyntax. Supports single documents, multiple documents, and lists, combinable with text and image inputs. -
PipeComposeConstruct Mode — New mode for deterministically buildingStructuredContentobjects without an LLM. Compose fields from working memory variables, fixed values, templates, and nested structures. -
Content Storage System — Configurable storage for generated artifacts (images, extracted pages). Supports
localfilesystem (.pipelex/storage/),in_memory, AWS S3 (pip install pipelex[s3]), and Google Cloud Storage (pip install pipelex[gcp-storage]). Cloud providers support both public URLs and time-limited signed URLs. Content referenced via stablepipelex-storage://URIs. -
Langfuse & OpenTelemetry Observability — New OpenTelemetry-based observability system enables powerful tracing and Evals through Langfuse integration. Also supports OTLP-compatible backends (Datadog, Honeycomb, etc.). Configured via
.pipelex/telemetry.toml. -
Python 3.14 Support — Officially tested and supported.
Added
--library-dirCLI Option:--library-dir/-Loption forpipelex run,pipelex validate, andpipelex buildsubcommands (one-shot-pipe,partial-pipe) to specify additional directories for searching pipe definitions. Can be specified multiple times.- Automatic File Loading: The core pipeline execution functions (
pipelex.execute_pipeline,pipelex.start_pipeline) can now directly load a pipeline from a file path via a newbundle_uriparameter. - Dry Run Mode:
pipelex run --dry-runexecutes pipeline logic without API calls, useful for validating structure and generating orchestration graphs. Combine with--mock-inputsto generate mock data for missing required inputs. | with_imagesJinja2 Filter: Explicitly extract and include all nested images from complex data structures (e.g.,Pageobjects or custom concepts withImagefields). Renders the object's text representation while making associated images available to the LLM.- System Prompt Media Support: Reference images and documents in
system_promptusing the same$variableand@variablesyntax as the userprompt. - Pipelex Gateway Service: Terms of service management via
.pipelex/pipelex_service.tomland interactive acceptance flow inpipelex initandpipelex init agreement. - Gateway Available Models Documentation: Auto-generated reference of all LLM, Document Extraction, and Image Generation models available through the Gateway.
- Configurable Retry Logic: Exponential backoff for inference API calls, configurable in
pipelex.tomlunder[cogt.tenacity_config] - Context Manager Support: The
Pipelexclass now supportswith Pipelex.make(): ...for graceful shutdown - Validation Improvements:
- Pipelex Bundle concept keys: prevent bundles from re-creating a native concept
PipeSequence: output multiplicity must match the last step's output multiplicityPipeFunc: output multiplicity must match the function return type (ListContentsubclass for multiplicity=true)- Rendering Protocols: Three new
@runtime_checkableprotocols (ImageRenderable,TagRenderable,TextFormatRenderable) to formalize the interaction between data types and Jinja2 filters. - Unified URI Handling System: New
pipelex.tools.urimodule providing type-safe parsing for HTTP/HTTPS URLs, local file paths, file URIs,pipelex-storage://URIs, and base64 data URLs. - Automatic Input Data Storage: Pipeline pre-processing step that converts large
data:URLs in anImageContentorDocumentContentintopipelex-storage://URIs for improved performance, by storing the data with the configured storage provider. Configurable viais_normalize_data_urls_to_storageinpipelex.toml. pipelex build inputsCommand: New CLI command to generate example input JSON files for pipes. Supports--library-dir(-L) to specify library directories.- Pipe Code Syntax Validation: Bundle validation now checks that pipe codes and
main_pipevalues use valid snake_case syntax, with proper error categorization (INVALID_PIPE_CODE_SYNTAX).
Fixed
- Nested Image Handling: Images nested within structured data are now properly replaced with
[Image N]tokens. The| with_imagesfilter andImageRegistrysystem correctly extract images from complex nested structures. PipeConditionValidation: Output multiplicity validation now works correctly.- Error Reporting:
PipeComposevalidation errors now include formatted details and failing field values. - Duplicate Pipeline Registration: Running a pipeline from a file that was also part of a pre-loaded library (via
PIPELEXPATH) no longer causes a duplicate domain registration error. The system now tracks absolute paths of loaded library files and skips files already loaded. pipelex build structures: Corrected output file naming and resolved import path generation.ConceptFactory.make_from_blueprint: Now correctly handles native concepts.
Changed
- Jinja2 Integration Refactored to Protocol-Based Approach: Replaced the
Jinja2Registrysingleton and handler functions with a decoupled protocol-based system, eliminating circular dependencies between the template layer and core domain logic. StuffArtefactRedesigned as Delegation Adapter: Now a lightweight, immutable adapter that delegates attribute access directly to underlyingStuffandStuffContentobjects, improving performance and providing more intuitive field access.- Image Extraction Moved to Content Types:
StructuredContent,ListContent,ImageContent, andTextAndImagesContentnow implement theImageRenderableprotocol, replacing centralized handler logic. - ⚠️ Breaking: User override config renamed from
pipelex_super.tomltopipelex_override.toml. - Image Generation Architecture: Refactored to taxonomy-based approach. Standardizes parameter translation (
aspect_ratio,quality,output_format) to provider-specific APIs. - Document Extraction Improvements:
pypdfium2extractor now extracts embedded images from PDFs. Response parsing uses dedicated Pydantic schemas for validation. - Default Model Change:
extract_text_from_visualsdeck now defaults toazure-document-intelligence pipelex_inferencereplaced bypipelex_gateway: See Highlights for migration details. NewPIPELEX_GATEWAY_API_KEYenvironment variable; default routing profiles updated topipelex_gateway_first.-
Telemetry System Split: Now two separate streams: 1. Pipelex Gateway telemetry for service monitoring (never collects prompts/completions/business data) 2. Custom telemetry to user-configured backends 3. Config updated accordingly (
telemetry.toml):- Renamed
[posthog]to[custom_posthog]to distinguish user's PostHog from Pipelex Gateway telemetry - Added new
[custom_portkey]section withforce_debug_enabledandforce_tracing_enabledsettings
- Renamed
-
Main Configuration Overrides Updated (
.pipelex/pipelex.toml): pipelex_override.toml(final override) renamed frompipelex_super.tomltopipelex_override.tomland moved from repo root to.pipelex/directorytelemetry_override.toml(personal telemetry settings)-
is_generate_cost_report_file_enableddefault changed fromtruetofalse -
Documentation:
- Clarified Setup (first run) vs Configuration (TOML reference), added a Setup overview page, and added contributor docs for configuration defaults/overrides.
- Added the "Under the Hood" page documenting the execution graph tracing system.
pipelex init: Now creates a documentedtelemetry.tomltemplate instead of prompting for preferences- Model Catalog Updated: Latest models (gpt-5.1, claude-4.5-opus, gemini-3.0-pro, etc.) and updated waterfalls in
base_deck.toml - Model Constraints Refactored: From simple lists to structured
valued_constraintsdictionaries (e.g.,valued_constraints = { fixed_temperature = 1 }) - OpenAI Responses API: New implementation now differentiates between
openai_completionsandopenai_responses - CLI Initialization: Commands refactored to use centralized
Pipelexinitialization factory for improved error handling pipelex doctor: Enhanced to detect outdatedtelemetry.tomlformats and suggest fixes--output-dirOption: Therunner,structures, andinputsCLI commands now accept this option- Cost Report: Now displays a note clarifying that it only includes LLM costs
descriptionField Now Required: InPipeAbstract,PipeBlueprint, andPipeSpecclasses.- Configuration: New
[pipelex.pipeline_execution_config.graph_config]section inpipelex.tomlfor fine-grained control over graph generation, data embedding, and rendering options. - CLI: All
pipelexcommands now accept--no-logoto suppress the Pipelex banner in the terminal — useful to reduce tokens.
Removed
openai_utilsModule: Removedpipelex.plugins.openai.openai_utils; logic now in centralized image preparation utilities.- Pipeline Tracking feature: Removed entirely, including the
pipelex/pipeline/trackmodule,PipelineTrackercomponents, related configuration, tracker calls in pipe controllers, and associated documentation. FlowGenerator: The old flow generator has been removed.
Deprecated
pipelex_inferencebackend in favor ofpipelex_gateway(marked as "🛑 Legacy" in configuration template)
For Contributors
- Technical Documentation: Added a new "Under the Hood" page documenting the
StuffArtefactdelegation pattern and image rendering architecture. - Enhanced Testing: Added extensive unit and integration tests for the protocol-based rendering system, including nested image extraction and filter error conditions.
- Agent Rules: Added
pipelex_standards.mdoutlining standards for the Pipelex configuration system, also included as rules for AI development agents. - Agent Rules: New target
make agent-checkfor faster linting. - ⚠️ Breaking — Content Handling Overhaul:
GeneratedImagereplaced by internalGeneratedImageRawDetails;ImageContentis now standard (withoutbase_64field).PipeExtractoutputsPageContentlist directly. Content persistence now handled automatically by storage system. - ⚠️ Breaking — Image Prompt Representation: Redesigned
PromptImagemodels—consolidatedPromptImagePathandPromptImageUrlintoPromptImageUri; now uses Pydantic discriminated union for URI, base64, and raw bytes sources. - Centralized Image Preparation: Moved image fetching and base64 conversion logic to
pipelex.cogt.image.prompt_image_utils, simplifying LLM provider plugins (Anthropic, Google, Mistral, OpenAI). - Unified Resource Loading: Updated all file/URL reading components (PDF renderers, document extractors) to use the new URI handling system, replacing the legacy
pipelex.tools.misc.path_utilsmodule. - Async HTTP Fetching: Renamed
fetch_file_from_url_httpx_asynctofetch_file_from_url_httpx; removed redundant synchronous version. PreparedImageAbstraction: New models (PreparedImageHttpUrl,PreparedImageBase64) representing images ready for LLM provider APIs.- Pipelex Gateway Model Management: New CLI commands (
pipelex-dev update-gateway-models,pipelex-dev check-gateway-models) and correspondingmaketargets (ugm,cgm) to generate and verify gateway model catalog. CI now validates this documentation is up-to-date. - Test Suite Pre-flight Check: Verifies Gateway terms acceptance before running tests, providing clear error messages.
- Content Rendering:
StuffContent.rendered_*methods now provide both synchronous and asynchronous variants.
Migration Notes
- Telemetry configuration migration: If you have an existing
telemetry.toml, rename: [posthog]→[custom_posthog][posthog.tracing]→[custom_posthog.tracing][posthog.tracing.capture]→[custom_posthog.tracing.capture]- Or run
pipelex init telemetry --resetto regenerate the file with the new structure
Refactored
- Anthropic Backend: Internal streaming for standard completions to prevent SDK timeouts, configurable structured output timeout (
structured_output_timeout_seconds), improved error mapping, and increased Bedrock Claudemax_tokens(8K → 64K) with removal ofmax_output_tokens_limitconstraint. - ⚠️ Breaking — Pipe I/O Specification: The output (and inputs) of a pipe is now a
StuffSpecobject that holds the concept and the multiplicity. - Naming Convention: Renamed
domaintodomain_codewhere relevant. - Dry Run Methods: Refactored the dry run methods of the
PipeAbstractclass.
[v0.17.6] - 2026-02-14
Added
- Claude Code GitHub Actions: Added
claude.ymlworkflow for interactive Claude Code assistance on issues and PR comments, andclaude-code-review.ymlworkflow for automated code review on pull requests.
[v0.17.5] - 2026-01-16
- Added target
make docs-deploy-404to deploy the 404.html file to the gh-pages root for versionless URL redirects.
[v0.17.4] - 2026-01-16
Added
- Added the
mikedependency to support mutiple docs versions.versionplugin added to the MkDocs configuration, make targets and CI scripts.
[v0.17.3] - 2025-12-01
Fixed
- Fixed the issue with the
find_files_in_dirforce including virtual environment directories: Now it force includes thepipelex.builderdirectory. - Fixed a bug with the comparison of Concept structures.
[v0.17.2] - 2025-12-01
Added
- New AI models support: Added GPT-5.1, Claude 4.5 Opus, and Gemini 3 Preview to the available models.
- Codex Cloud support: Added support for running Pipelex in Codex Cloud environments with appropriate configuration and testing capabilities.
- Enhanced file discovery: Added
force_include_dirsparameter to thefind_files_in_dirfunction. This allows specific directories to be force included in the search even when they are nested within excluded directories. For example, you can now exclude.venvwhile still including.venv/lib/python3.11/site-packages/pipelexfor loading Pipelex libraries from installed packages. - Added validation of the PipeLLM inputs at the blueprint level.
- Added a xfailed test for
PipeCondition: if one of the outcome of the pipe iscontinue, it does nothing, but the main stuff still points towards the last step. Therefore when trying to get the main stuff out of the working memory as a specific type, it fails.
Changed
- Backend fallback now only activates when explicitly opted-in, giving users more control over model selection.
- Renamed and improved the Azure Image Generation SDK implementation.
- Enhanced language spec examples, operator details, and added Viewpoint documentation.
Fixed
- Fixed kit rules to be idempotent and work correctly across multiple executions.
[v0.17.1] - 2025-11-27
Fixed
- Fixed a bug in the
find_files_in_dirfunction.
[v0.17.0] - 2025-11-27
Highlights
- Previously, in the pipelex config files (
.tomlfiles in the.pipelex/directory, such as.pipelex/pipelex.toml, but also the routing profiles files, backends, etc.), when an array was overridden, the new array was concatenated to the old array. Now, the new array overrides the old array.
Fixed
- Relaxed concept structure field naming restrictions: Users can now use field names like
content,stuff_code,stuff_name, andconceptin their concept structures without conflicts. Internal metadata fields in stuff artefacts now use underscore prefixes (_stuff_name,_content_class,_concept_code,_stuff_code,_content) to avoid collisions with user-defined fields. Reserved field names (Pydantic BaseModel attributes likemodel_config,model_fields, etc.) and field names starting with underscore remain forbidden with improved error messages that clearly specify which fields are problematic.
Changed
- Modified the GHA
version-check.ymlso that the check of the version is only applying to release branches. - Removed the
pyproject.tomlfile from the build. - No more implicit concepts. A concept reference has a domain and a code. If there is no domain, it should be a native concept, or it is declared in the same bundle.
Refactored
- The
find_files_in_dirfunction was coded in 3 different places, now it's inpipelex/tools/misc/file_utils.py, and acceptsexcluded_dirs. - Refactored the Pipe factories: Centralized everything in the
PipeFactoryclass.
[v0.16.0] - 2025-11-25
Highlights
- Library manager now supports multiple libraries. You can now have multiple libraries in your project, each with its own set of concepts, pipes, and stuffs. You can run the same pipe at the same times as much as you want, with different inputs. Side effets: Unit tests now run in 30s.
Fixed
- Fixed some issues with inputs of pipes: The validation methods was not detecting misconceptions with implicit concepts.
Changed
- Improved pipe builder by auto-fixing errors, forcing consistency in the inputs and outputs of the pipes.
Refactor
- PipeCondition: Moved the expression/expression_template choosing to the factory.
- Moved a lot of validation to blueprints instead of pipe instances.
- Refactored the Blueprint validation errors, and validation functions.
- Refactored the PipelexInterpreter validation errors.
- Refactored the pipe builder validation loop.
- Reorganized the unit tests, and added new ones.
- Reorganized the config files.
- Refactored methods
execute_pipelineandstart_pipeline. - Moved
dev_clitocli.dev_cli.
[v0.15.7] - 2025-11-18
Fixed
- Fixed issue with
get_console()function returningNoneif Pipelex is not initialized. Now always defaults tostderrif not set.
[v0.15.6] - 2025-11-18
Contributors
- Welcome to our new contributor @0x090909 (yup, that's his github username) for his work on Groq support in PR #445! 🎉
Added
- Improved configuration repair: New
--fixoption forpipelex doctorcommand that interactively detects and repairs outdated or invalid backend configuration files using latest templates from the Pipelex kit. - Developer CLI & tooling: New internal
pipelex-devCLI for project maintenance withcheck-config-synccommand to verify user-facing configuration templates (.pipelex/) are synchronized with package's internal kit configs. Includesmake check-config-synccommand and CI check (lint-check.yml) to enforce synchronization. - Enhanced test infrastructure: Integration tests now automatically parameterized to run against all supported backend routing profiles. Tests are intelligently skipped at collection time if a model is not supported by the active backend profile, with a summary of skipped tests provided at session end.
- New routing profiles: Added
all_groqandall_pipelex_inferencerouting profiles. - Vision support flag: Added
is_vision_supportedproperty toLLMWorkerAbstractclass for explicit checks of model vision capabilities. - New type of
StuffContent:JSONContentto support an arbitrary JSON object as input or output of a pipe. - Azure image generation: Support for image generation models via Azure OpenAI backend using
gpt-image-1.
Changed
- Unified structured output: Complete overhaul of structured generation settings. Replaced global configuration setting with new
structure_methodparameter in backend.tomlfiles (configurable at backend level in[defaults]or per individual model). ExpandedStructureMethodenum to include dozens of modes supported byinstructor, enabling fine-grained control over provider-specific features like OpenAI Structured Outputs or Anthropic Tools, and various JSON-based modes. - Groq integration: Updated to use standard
openaiSDK, simplifying integration. - Default configurations: All official backend providers now enabled by default after
pipelex init. Default prompting style changed fromtickstoxml. - Code & test organization: Unit test suite reorganized from
tests/unit/coreand other directories into unifiedtests/unit/pipelex/structure. Integration test fixtures modularized fromconftest.pyinto separate files withintests/integration/pipelex/fixtures/. - Console output settings: Added
console_print_targetandconsole_log_targetsettings inpipelex.tomlfor redirecting output tostdoutorstderr, with CLI and logging refactored to use centralized console instance. This makes it easier to support MCP communication based on stdio.
Fixed
- Improved Pydantic validation error messages when loading backend configurations to clearly indicate the specific file and model containing the error.
Removed
- Perplexity backend: Default configuration for Perplexity AI backend (
perplexity.toml) removed from kit (it was obsolete, it will come back). - Groq plugin: Dedicated
pipelex/plugins/groqplugin removed (now uses standardopenaiSDK). - Global instructor config: Global
is_openai_structured_output_enabledsetting, replaced by per-modelstructure_methodapproach.
[v0.15.4] - 2025-11-12
Added
- Enhanced
pipelex buildCommand: Now generates a self-contained directory (e.g.,results/pipeline_01/) containingbundle.plx,inputs.json,run_{pipe_code}.py,bundle_view.html, andbundle_view.svg. New CLI options:--output-name (-o)for custom base name,--output-dirfor custom directory, and--no-extrasto generate only the.plxfile. - CLI Readiness Check: Verifies that a virtual environment is active for development installations.
- Model Deck Presets: Added
llm_for_creativityandcheap_llm_for_creativitymodel waterfalls, plus[cogt.model_deck_config]section inpipelex.tomlfor configuring model fallback behavior. - WIP: Groq Inference Backend Support: Integrated full support for the Groq API with configuration file (
.pipelex/inference/backends/groq.toml), model specifications, costs, capabilities, new model aliases (base-groq,fast-groq,vision-groq), and routing profile (all_groq).
Changed
- CLI Output and Visualization: Overhauled command-line output with rich, table-based layouts for pipeline components. Final output of
pipelex runis now pretty-printed and adapts to content type. - Documentation: Updated "Get Started" and "Build Reliable AI Workflows" to reflect new directory-based build output and CLI options.
- Internal Code Refactoring: Reorganized exception hierarchy into dedicated
exceptions.pyfiles per module, centralized validation logic intovalidation.pymodules, addedValueErrorto blueprints, and removed unused exceptions for improved maintainability. - Updated pytest to
>=9.0.1to support their newpyproject.tomlconfig format.
Fixed
- Adjusted default temperature for
llm_for_testing_gen_objectpreset from0.5to0.1for more deterministic structured data generation. - Corrected
LLM_FOR_VISUAL_DESIGNskill inpipe_llm_specto point tocheap_llm_for_creativitypreset. - Standardized input variable names in
pipe_llm_vision.plxfromimageA/imageBtoimage_a/image_b.
Removed
- Deleted
pipelex/core/validation_errors.pyfile as part of exception hierarchy refactoring.
[v0.15.3] - 2025-11-07
Fixed
- Fixed weird import issues with
posthogandStrEnum
[v0.15.2] - 2025-11-07
Fixed
- Fixed resetting routing profile when calling with
--resetflag inpipelex init
[v0.15.1] - 2025-11-07
Fixed
- Bumped OpenAI dependency to
>=1.108.1to support their breaking change: "change optional parameter type fromNotGiventoOmit" get_selected_backend_keys()now correctly considers backends enabled by default (like before v0.15.0)
[v0.15.0] - 2025-11-07
Highlights
This release dramatically simplifies onboarding with interactive CLI setup, comprehensive documentation relaunch, and intelligent model fallbacks, making Pipelex more accessible and resilient than ever.
Added
- Model Waterfalls: Define prioritized model lists in
base_deck.toml(e.g.,smart_llm = ["gpt-4o", "claude-4.5-sonnet", "grok-3"]). Pipelex automatically falls back to the next model if the preferred one is unavailable. - Advanced Routing Profiles: New capabilities in
routing_profiles.toml:fallback_order(Global fallback sequence specifying which backends to try if a model isn't found) andoptional_routes(Routes that activate only when their target backend is enabled) - New Models: Anthropic
claude-4.5-haiku(Pipelex Inference, Anthropic, and Bedrock backends) and Azure OpenAIo3 - Comprehensive Documentation Relaunch: Complete restructure under
/home/with new "Get Started" guides forpipelex buildand manual workflows, plus in-depth sections on Domains, Bundles, Concepts, and Pipe lifecycle. - Enhanced CLI:
pipelex initnow interactively guides backend selection and automatically configures routing profiles, including primary backend and fallback order. Addedpipelex init routingfocus. - Enhanced CLI: Improved error reporting across all commands (
build,validate,run,show) with clear, actionable feedback for configuration errors, missing models, and invalid presets. - Enhanced CLI:
pipelex doctornow validates model deck configuration. - New Routing Profiles: Full suite of
all_*profiles (e.g.,all_openai,all_anthropic,all_google) to route all requests to a single provider.
Changed
- BREAKING: for inline concept structures, the fields are now optional by default: the
requiredproperty defaults tofalse. Explicitly setrequired = trueto make fields mandatory, which we discourage as it increases risks of hallucinations. - LLM Presets Overhaul: Rationalized and renamed default presets in
base_deck.toml. Single-model aliases replaced with waterfall aliases. Key renames:llm_for_complex_reasoning→engineering-structured,llm_to_answer_hard_questions→llm_to_answer_questions,llm_to_write_questions→llm_for_writing_cheap. Removed redundant older presets. - Stricter Configuration Validation: Pipelex validates model deck on startup and raises errors if presets reference unavailable models.
Fixed
- Local OpenAI-Compatible Endpoints: OpenAI plugin now handles empty API keys, enabling seamless integration with local servers like Ollama.
Removed
- Old Documentation Structure: Previous
/pages/directory documentation removed in favor of new structure.
[v0.14.3] - 2025-10-29
Added
- Image generation models via BlackBoxAI backend:
flux-pro,flux-pro/v1.1,flux-pro/v1.1-ultra(Black Forest Labs),fast-lightning-sdxl(ByteDance), andnano-banana(Google). Implemented using newopenai_alt_img_genSDK worker with chat completion-style API. - Language model:
claude-4.5-sonnet(Anthropic) via BlackBoxAI backend. - Routing profile:
all_blackboxaiprofile routes all supported model requests to BlackBoxAI backend.
Changed
- Model aliases in
base_deck.toml:base-img-gen→flux-pro/v1.1-ultra,best-img-gen→nano-banana,llm_for_large_codebasenow includesclaude-4.5-sonnet. - Configuration file:
BLACKBOX_RULES.mdrenamed to.blackboxrules.
Fixed
- Image generation schema:
ImgGenJobParams.seedfield now explicitly defined withdefault=None. - CLI bundle validation:
pipelex validatecommand now accepts bundle path (.plxfile) which are in the package and already loaded and performs dry run on all pipes in the bundle.
[v0.14.2] - 2025-10-29
Chaged
- Improved pipe builder.
Added
- CLI to generate inputs JSON.
[v0.14.1] - 2025-10-27
Added
- Tutorial GIF on the README.md file.
[v0.14.0] - 2025-10-27
Added
pipelex doctorcommand: Diagnoses and fixes common configuration issues including missing files, invalid telemetry settings, and unset environment variables for enabled backends.- Interactive backend selection in
pipelex init: Multi-select menu for enabling/disabling inference backends (OpenAI, Anthropic, Amazon Bedrock, etc.). - JSON input support:
pipelex run --inputsflag accepts a JSON file path for passing structured data to pipelines. pretty_printmethods: Added toPipeSpec,ConceptSpec, andStuffobjects for readable debugging output.- VS Code debug configuration: "Debug run pipe" launch configuration for debugging pipeline executions.
display_nameattribute: Added to all inference backends inbackends.tomlfor better UI presentation.- Documentation headers: All default
.tomlconfiguration files now include headers with links to documentation and support channels.
Changed
pipelex initredesign: Transformed into a unified, interactive setup wizard with rich terminal UI for configuration files, backend selection, and telemetry preferences. Telemetry is now configured here instead of via first-run prompt.README.mdrewrite: Complete overhaul featuring a simplified 5-step quick-start guide highlighting thepipelex buildcommand.- Documentation updates: "Quick Start" guide renamed to "Writing Workflows" with simplified content. Python examples updated to use JSON input method, removing manual
StuffandWorkingMemoryobject creation boilerplate. Developer guides and AI assistant rules now recommendpipelex validateovermake validate. Added instructions emphasizing.venvactivation before running commands. - Error handling improvements: Pipelines now validate required inputs upfront and fail early with
PipeRunInputsError.pipelex runprints full rich-formatted exception tracebacks on error. - Default enabled backends: Amazon Bedrock, Google AI, and Google Vertex AI are now enabled by default.
- Naming consistency: "AWS Bedrock" renamed to "Amazon Bedrock" throughout codebase, configuration, and documentation.
Fixed
- Some documentation links were broken.
[v0.13.2] - 2025-10-25
Added
- Added the
n8ndocumentation page for the n8n-nodes-pipelex package. - Added optional telemetry system with first-run interactive prompt offering three modes: off (no data collected), anonymous (usage data without identification), and identified (usage data with user identification). Automatically respects
DO_NOT_TRACKenvironment variable and redacts sensitive data (prompts, responses, file paths, URLs). Configuration stored in.pipelex/telemetry.toml. - Added telemetry documentation: user-friendly setup guide and comprehensive configuration reference.
Changed
- Updated the
PipelexClientand changed the route of the API calls tov1/pipeline/executeandv1/pipeline/start. - Changed the parameter
input_memorytoinputsin the documentaton.
[v0.13.1] - 2025-10-22
Changed
- Changed the
pydanticdependency from==2.10.6to>=2.10.6,<3.0.0to avoid compatibility issues.
[v0.13.0] - 2025-10-21
Highlights
This release focuses on making Pipelex more accessible and easier to use, with major improvements to the CLI, simplified syntax for multiplicity, and a complete documentation overhaul:
- New CLI commands: Run pipelines directly with
pipelex run, generate Python runners withpipelex build runner, and inspect your AI backend configuration withpipelex show backends - Simplified pipeline inputs: The new
inputsparameter replacesinput_memoryand accepts strings, lists, or content objects directly - no more complex dictionary structures - Getting started faster: Completely rewritten quick-start guide and new documentation sections help you go from installation to your first pipeline in minutes
Added
- CLI command
pipelex run: Top-level command to execute pipelines directly from the CLI. Can run pipes from the package or from any.plxbundle file, with options to provide inputs from a JSON file and save the output - CLI command
pipelex build runner: Generates Python script with imports and example input structures for any pipe - CLI command
pipelex show backends: Displays configured AI providers, their status, and active routing rules - Model presets: Added task-oriented presets including
llm_to_write_questions,llm_to_code,llm_for_basic_vision,llm_for_visual_analysis - Documentation: Complete quick-start guide rewrite, new guides for "Understanding Multiplicity", "API Guide", "Executing Pipelines with Inputs", and updated README with video demo
- Migration guide: Updated guide at
pipelex/kit/migrations/migrate_0.11.0_0.12.x.md
Changed
- Unified bracket notation for multiplicity: Single items use
"Concept", variable lists use"Concept[]", fixed-count lists use"Concept[3]". Applies to bothinputsandoutputfields in.plxfiles - Pipeline input format:
input_memoryparameter renamed toinputs; now accepts strings, lists of strings,StuffContentobjects, or explicit concept dictionaries instead ofCompactMemory - Bundle
main_pipeattribute: Pipelex bundles (.plxfiles) now support amain_pipeattribute to designate the primary entry point of the bundle. Used bypipelex runandpipelex build runnercommands to simplify execution - Model preset names:
llm_to_reason→llm_for_complex_reasoning,base_ocr_mistral→extract_text_from_visuals,base_extract_pypdfium2→extract_text_from_pdf,base_img_gen→gen_image_basic,fast_img_gen→gen_image_fast,high_quality_img_gen→gen_image_high_quality - Unified model parameter:
PipeExtractandPipeImgGennow usemodelparameter for consistency across all operator pipes PipeExtractoperator: Output is now consistently validated to be thePageconcept, simplifying its usage for document processing- CLI improvements:
pipelex runandpipelex validatenow auto-detect pipe code vs.plxbundle files;pipelex validatepromoted to top-level command with improved error reporting and syntax-highlighted code snippets - CLI reorganization: Main command-line interface restructured for better usability with improved help texts and more logical command order
- Python API:
Pipelex.make()now accepts dependency injection arguments directly - Python coding standards: Updated internal coding standards to recommend declaring variables with a type but no default value to better leverage linters for bug detection
- Default configuration: Azure and AWS inference backends now disabled by default in template configuration
Fixed
- Structure generation: Special characters (double quotes, backslashes) in concept field descriptions or default values no longer produce invalid Python code
Removed
- Legacy multiplicity syntax:
nb_output,multiple_outputparameters, and complex input dictionary syntax withmultiplicityfield - Pipe-specific model parameters:
ocrparameter fromPipeExtractandimg_genparameter fromPipeImgGen prompt_template_to_structureandsystem_prompt_to_structureconfigurations at the pipe and domain level- Project Name discovery from Configuration
- Temporary design document for the new inference backend system (feature now fully implemented and documented)
[v0.12.0] - 2025-10-15
Highlights
Moving fast and breaking things:
- Added the new builder pipeline system for auto-generating Pipelex bundles from user briefs
- it's a pipeline to generate pipelines, and it works!
- the pipeline definitions are in
pipelex_libraries/pipelines/base_library/builder/ - removed the previous draft which was named
meta_pipeline.plx
Breaking changes... for good!
We tried to group all the renamings we wanted to do which impact our language, so that you get one migration to apply and then we will be way more stable in the future releases.
This is all in the spirit of making Pipelex a declarative language, where you express what you want to do, and the system will figure out how to do it. So our focus inwas to make the Pipelex language easier to understand and use for non-technical users, and at the same time use more consistent and obvious words that developers are used to.
💡 Pro tip: To make migration easier, pass the migration guide to your favorite SWE agent (Cursor, Claude Code, github copilot, etc.) and let it handle the bulk of the changes!
- Removed centralized
pipelex_librariesfolder system - Pipelines are now auto-discovered from anywhere in your project—no special directory required
- No config path parameters needed in
Pipelex.make()or CLI commands (just callPipelex.make()) - Custom functions require
@pipe_func()decorator for auto-discovery - Structure classes auto-discovered (must inherit from
StructuredContent) - Configuration stays at repository root in
.pipelex/directory -
See migration guide for details on reorganizing your project structure
-
General changes
-
renamed
definitionfields todescriptionacross all cases -
Renamed PipeJinja2 to PipeCompose
- the fact that our templating engine is Jinja2 is a technnical detail, not fundamental to the language, especially since we included a pre-processor enabling insertion of variables in prompts using
@variableor$variable, in addition to the jinja2 syntax{{ variable }} - renamed
jinja2field totemplatefor the same reason -
for more control, instead of providing a string for the
templatefield, you can also use a nestedtemplatesection withtemplate,categoryandtemplating_stylefields -
Renamed PipeOCR to PipeExtract
- this is to account for various text extraction techniques from images and docs, including but not only OCR; e.g. we now have integrated the
pypdfium2package which can extract text and images from PDF, when it's actually real text (not an image), and soon we'll add support for other document extraction models solutions - removed obligation to name your document input
ocr_input, it can now be named whatever you want as long as it's a single input and it's either anImageor aPDFor some concept refining PDF or Image - renamed
ocr_page_contents_from_pdftoextract_page_contents_from_pdf - renamed
ocr_page_contents_and_views_from_pdftoextract_page_contents_and_views_from_pdf - introduced model settings and presets for extract models like we had for LLMs
-
renamed
ocr_modeltomodelfor choice of model, preset, or explicit setting and introducedbase_ocr_mistralas an alias tomistral-ocr -
PipeLLM field renames
- image inputs must now be tagged in the prompt like all other inputs; you can just drop their names at the beginning or end of the prompt, or you can reference them in meaningful sentences to guide the Visual LLM, e.g. "Analyze the colors in $some_photo and the shapes in $some_painting."
- renamed
prompt_templatefield toprompt - renamed
llmfield tomodel -
renamed
llm_to_structurefield tomodel_to_structure -
PipeImgGen field renames
- renamed
img_genfield tomodelfor choice of model, preset, or explicit setting - removed some technical settings such as
nb_stepsfrom the pipe attributes, instead you can set these as model settings or model presets -
introduced model settings and presets for image generation models like we had for LLMs
-
PipeCondition field renames
- renamed
pipe_maptooutcomes -
renamed
default_pipe_codetodefault_outcomeand it's now a required field, because we need to know what to do if the expression doesn't match any key in the outcomes map; if you don't know what to do in that case, then it's a failure and you can use thefailvalue -
Configuration file changes (
.pipelex/directory) - Renamed parameter
llm_handletomodelacross all LLM presets in deck files - Renamed parameter
img_gen_handletomodelacross all image generation presets in deck files - Renamed parameter
ocr_handletomodelin extraction presets - Renamed
ocrsection toextractthroughout configuration files - Renamed
ocr_configtoextract_configinpipelex.toml - Renamed
base_ocr_pypdfium2tobase_extract_pypdfium2 - Renamed
is_auto_setup_preset_ocrtois_auto_setup_preset_extract - Renamed
nb_ocr_pagestonb_extract_pages - Updated pytest marker from 'ocr' to 'extract'
Added
- Added
cheap-gptmodel alias forgpt-4o-mini - Added
cheap_llm_for_visionpreset usinggemini-2.5-flash-lite - Added
llm_for_testing_visionandllm_for_testing_vision_structuredpresets for vision testing - Added
is_dump_text_prompts_enabledandis_dump_response_text_enabledconfiguration flags to have the console display everything that goes in and out of the LLMs - Added
generic_templatessection inllm_configwith structure extraction prompts - Added useful error messages with migration configuration maps pin-pointing the fields to rename for config and plx files
- Added improved error message for
PipeFuncwhen function not found in registry, mentioning@pipe_func()decorator requirement since v0.12.0 - Added pytest filterwarnings to ignore deprecated class-based config warnings
- Added
Flowclass that represents the flow of pipe signatures - Added
pipe-buildercommandflowto generate flow view from pipeline brief - Added
FlowFactoryclass to create Flow from PipelexBundleSpec or PLX files - Added
sort_pipes_by_dependencies()function for topological sorting of pipes - Added
pipe_sorter.pymodule for pipe dependency sorting utilities - Added
search_for_nested_image_fields_in_structure_class()method to Concept class - Added
image_field_search.pymodule with utilities to search for image fields in structure classes - Added
pipe_dependenciesproperty to PipeBlueprint and controller blueprints - Added
ordered_pipe_dependenciesproperty to PipeBlueprint for ordered dependencies - Added
get_native_concept()function to hub - Added
get_pipes()function to hub - Added
remove_concepts_by_codes()method to ConceptLibraryAbstract - Added
remove_pipes_by_codes()method to PipeLibraryAbstract - Added template preprocessing with
preprocess_template()function - Added better dependency checking for optional SDK packages (anthropic, mistralai, boto3, aioboto3)
- Added
MissingDependencyErrorexception for missing optional dependencies - Added
library_utils.pymodule with utility functions for PLX file discovery usingimportlib.resources - Added
class_utils.pymodule withare_classes_equivalent()andhas_compatible_field()functions - Added comprehensive unit tests for
CostRegistry,WorkingMemory, andModuleInspector - Added
ScanConfigclass with configurable excluded directories for library scanning - Added CSV export capabilities to
CostRegistrywithsave_to_csv()andto_records()methods - Added default configuration template in
pipelex/kit/configs/pipelex.toml
Changed
- Replaced package
tomlbytomliwhich is more modern and faster - Updated Gemini 2.0 model from
gemini-2.0-flash-exptogemini-2.0-flashwith new pricing (input: $0.10, output: $0.40 per million tokens) - Updated Gemini 2.5 Series comment from '(when available)' to stable release
- Updated
best-claudefromclaude-4-sonnettoclaude-4.5-sonnetacross all presets - Updated kajson dependency from version
0.3.0to0.3.1 - Updated httpx dependency to
>=0.23.0,<1.0.0for broader compatibility - Cleanup env example and better explain how to set up keys in README and docs
- Changed Gemini routing from
googlebackend topipelex_inferencebackend - BREAKING: Major module reorganization - moved
tools/config/,tools/exceptions.py,tools/environment.py,tools/runtime_manager.pytosystem/package structure (system/configuration/,system/exceptions.py,system/environment.py,system/runtime.py) - BREAKING: Reorganized registry modules from
tools/tosystem/registries/(affectsclass_registry_utils,func_registry,func_registry_utils,registry_models) - BREAKING: Split
pipelex.core.stuffs.stuff_contentmodule into individual files per content type (affects imports:StructuredContent,TextContent,ImageContent,ListContent,PDFContent,PageContent,NumberContent,HtmlContent,MermaidContent,TextAndImagesContent) - BREAKING: Renamed package
pipelex.pipe_workstopipelex.pipe_runand movedPipeRunParamsclasses into it - BREAKING: Cost reporting changed from Excel (xlsx) to CSV format using native Python csv module instead of pandas
- Renamed
ConfigManagertoConfigLoader - Renamed
PipelexRegistryModelstoCoreRegistryModels - Renamed
PipelexTestModelstoTestRegistryModels - Renamed
generate_jinja2_context()togenerate_context()inWorkingMemoryandContextProviderAbstract - Renamed
ConceptProviderAbstracttoConceptLibraryAbstract - Renamed
DomainProviderAbstracttoDomainLibraryAbstract - Renamed
PipeProviderAbstracttoPipeLibraryAbstract - Renamed
PipeInputSpectoInputRequirements - Renamed
PipeInputSpecFactorytoInputRequirementsFactory - Renamed
pipe_input.pytoinput_requirements.py - Renamed
pipe_input_factory.pytoinput_requirements_factory.py - Renamed
pipe_input_blueprint.pytoinput_requirement_blueprint.py - Changed hub methods from
get_*_provider()toget_*_library()pattern - Changed hub methods from
set_*_provider()toset_*_library()pattern - Changed
PipeLLMvalidation to check all inputs are in required variables - Updated
LLMPromptSpecto handle image collections (lists/tuples) in addition to single images - Changed Mermaid diagram URL generation from
/img/to/svg/endpoint - Changed
PipeLLMPromptTemplate.make_llm_prompt()to private method_make_llm_prompt() - Updated pipe-builder prompts to include concept specs for better context
- Updated
PipelexBundleSpec.to_blueprint()to sort pipes by dependencies before creating bundle - Changed exception base class from
PipelexErrortoPipelexErrorthroughout codebase - Updated Makefile pyright target to use
--pythonpathflag correctly - Enhanced
LibraryManagerto useimportlib.resourcesfor reliable PLX file discovery across all installation modes (wheel, source, relative path) - Simplified
FuncRegistryUtilsto exclusively register functions with@pipe_funcdecorator (removeddecorator_namesandrequire_decoratorparameters) - Updated
ReportingManagerto get config directly instead of via constructor parameter - Updated PipeFunc documentation to reflect
@pipe_func()decorator requirement and auto-discovery from anywhere in project - Added warnings about module-level code execution during auto-discovery to PipeFunc and StructuredContent documentation
Fixed
- Fixed Makefile target
pyrightto use correct pythonpath flag - Fixed bug with inputs of the
PipeLLMwhere image inputs couldn't be used and tagged in prompts - Fixed image input handling in
LLMPromptSpecto support both single images and image collections - Fixed template preprocessing to handle jinja2 templates correctly
- Fixed hard dependencies by moving imports to function scope in model_lists.py
- Updated README badge URL to point to main branch instead of feature/pipe-builder branch
Removed
- Removed centralized
pipelex_librariesfolder system andpipelex init librariescommand - Removed config path parameters from
Pipelex.make()(relative_config_folder_path,config_folder_path,from_file) - Removed Gemini 1.5 series models:
gemini-1.5-pro,gemini-1.5-flash, andgemini-1.5-flash-8b - Removed
base_templates.tomlfile (generic prompts moved topipelex.toml) - Removed
gpt-5-minifrom possible models in pipe-builder - Removed useless functions in
LLMJobFactory:make_llm_job_from_prompt_factory(),make_llm_job_from_prompt_template(),make_llm_job_from_prompt_contents() - Removed
add_or_update_pipe()method from PipeLibrary - Removed
get_optional_library_manager()method from PipelexHub - Removed
get_optional_domain_provider()andget_optional_concept_provider()methods from hub - Removed unused test fixtures (apple, cherry, blueberry, concept_provider, pretty) from conftest.py
- Removed some Vision/Image description pipes from the base library, because we doubt they were useful as they were
- Removed pandas and openpyxl dependencies (including stubs: pandas-stubs, types-openpyxl)
- Removed Excel file generation for cost reports and
to_dataframe()method fromCostRegistry - Removed
should_warn_if_already_registeredparameter fromfunc_registry.register_function() - Removed
decorator_namesandrequire_decoratorparameters fromFuncRegistryUtilsmethods - Removed
_find_plx_files_in_dir()and_get_pipelex_plx_files_from_dirs()methods fromLibraryManager(refactored tolibrary_utilsmodule) - Removed hardcoded excluded directories from
ClassRegistryUtilsandFuncRegistryUtils(now useScanConfig) - Removed
are_classes_equivalent()andhas_compatible_field()methods fromClassRegistryUtils(moved toclass_utilsmodule)
[v0.11.0] - 2025-10-01
Highlights
- New pipe builder pipeline to generate Pipes based on a brief in natural language: use the cli
pipelex build pipe "Your task"to build the pipe. - New observer system: inject your own class to observe and trace all details before and after each pipe run. We also provide a local observer that dumps the payloads to local JSONL files = new-line delilmited json, i.e. one json object per line.
- Full refactoring of OCR and Image Generation to use the same patterns as
LLMworkers and pipes.
Added
- Added
claude-4.5-sonnetto the model deck. - Added a badge on the
README.mdto display the number of tests. - Added new test cases for environment variable functions
- Added new documentation for
PipeFuncon how to register functions. - Added
pipelex show models [BACKEND_NAME]command to list available models from a specific backend.
Changed
- Renamed
llm_deckterminology tomodel_deckthroughout codebase and documentation, now that it's also used for OCR and Image Generation models - Renamed
is_gha_testingproperty tois_ci_testingin RuntimeManager - Refactored
all_env_vars_are_set()function to only accept a list of keys, single string support now usesis_env_var_set() - Modified
any_env_var_is_placeholder()to use new placeholder detection logic - Updated test environment setup to use dynamic placeholder generation instead of hardcoded values
Fixed
- Fixed logic error in
any_env_var_is_placeholder()function - now correctly returns False when no placeholders are found
Removed
- Removed
get_rooted_path()andget_env_rooted_path()utility functions which were not used - Removed hardcoded placeholder dictionary and
ENV_DUMMY_PLACEHOLDER_VALUEconstant in test setup - Removed function
run_pipe_codein pipe router because it was not relevant (used mostly in tests) - Remove the use of
PipeComposeinPipeCondition, to only use jinja2 directly, through theContentGenerator - Remove the template libraries from the pipelex libraries.
- Removed
claude-3.5-sonnetandclaude-3.5-sonnet-v2from the model deck.
[v0.10.2] - 2025-09-18
Added
- Unified OCR system using model handles instead of separate OcrHandle enum
- ModelType enum supporting LLM and TEXT_EXTRACTOR types
- Enhanced error handling in library loading with better validation messages
- Config template management with
config-templateandcftMakefile targets to update templates from the.pipelex/directory
Changed
- ⚠️ Breaking changes:
- Renamed
ocr_handletoocr_modelinPipeExtractblueprint, so you'll need to update your PLX code accordingly - Updated .env.example file with slightly modified key names (more standard).
- OCR system now uses InferenceModelSpec with unified model handles
- Renamed
get_llm_deck()toget_model_deck()and updated parameter names fromllm_handletomodel_handle - Simplified OCR worker factory using plugin SDK matching
- Enhanced plugin system compatibility with InferenceModelSpec
- Improved error messages throughout system
- Improved management of placeholder environment variables for unit tests
Removed
- Legacy OCR classes: OcrHandle, OcrPlatform, OcrEngine, OcrEngineFactory
- Obsolete configuration fields and setup methods
- PipelexFileError exception class
[v0.10.1] - 2025-09-17
Changed
- Enabled all backends, still required to pass all unit tests.
- A few tweaks to the base model deck.
[v0.10.0] - 2025-09-17
Highlights
New Inference Backend Configuration System — We've completely redesigned how LLMs are configured and accessed in Pipelex, making it more flexible and easier to get started:
- Get started in seconds with Pipelex Inference: Use a single API key to access all major LLM providers (OpenAI, Anthropic, Google, Mistral, and more)
- Flexible backend configuration: Configure multiple inference backends (Azure OpenAI, Amazon Bedrock, Vertex AI, etc.) through simple TOML files in
.pipelex/inference/ - Smart model routing: Automatically route models to the right backend using routing profiles with pattern matching
- User-friendly aliases: Define shortcuts like
best-claude→claude-4.1-opuswith optional fallback chains - Cost-aware model specs: Each model includes detailed pricing, capabilities, and constraints for better cost management
For complete details, see the Inference Backend Configuration documentation.
Added
- New inference backend configuration system in
.pipelex/inference/directory - Support for 10+ inference backends: OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Mistral, Vertex AI, XAI, BlackboxAI, Perplexity, Ollama, and Pipelex Inference
- Model routing profiles with pattern matching (
*model*,model*,*model) - Model aliases with waterfall fallback chains
- Environment variable and secret substitution in TOML configs (
${VAR}and${secret:KEY}) - Comprehensive model specifications with detailed cost categories
- Unified plugin SDK registry for all backends
- CI environment detection with automatic placeholder API keys for testing
- Improved
pipelex init configcommand to copy entire configuration template directory structure to.pipelex/with smart file handling (skips existing files, shows clear progress messages) - Added
FuncRegistryUtilsto register functions in a pipelex folder that have a specific signature. - Added
mistral-mediumandmistral-medium-2508to the Mistral backend configuration. - Added
gemini-2.5-flashto the VertexAI backend configuration.
Changed
- LLM configuration moved from
pipelex_libraries/llm_deck/to.pipelex/inference/deck/ - LLM handles simplified to direct model names or user-defined aliases
- Model deck completely redesigned with inference models, aliases, and presets
- Plugin system refactored to use backend-specific TOML configuration
- Token categories renamed to cost categories with expanded types
Fixed
- Improved error messages for missing environment variables
- Enhanced TOML configuration validation
- More robust model routing and backend selection
Removed
- Legacy LLM model library system (
llm_integrations/directory) - Platform-specific configuration classes (AnthropicConfig, OpenAIConfig, etc.)
- Deprecated LLM engine blueprint and factory classes
- Old LLM platform and family enumerations
Security
- Enhanced secret management with secure fallback patterns
- Improved API key handling through centralized backend configuration
[v0.9.5] - 2025-09-12
Highlights
- Pinned
instructorto version<1.10.0to avoid errors withmypy
Added
- Added
PIPELEX_INFERENCELLM family enum value - Added support for
PIPELEX_INFERENCEin OpenAI LLM worker - Added Azure OpenAI platform support for Grok models (
grok-3andgrok-3-mini) - Added debug logging for
PipeParalleloutput contents - Added
TOMLfile filtering in LLM model library loading - Added error handling for Unicode decode errors in LLM model library
- Added new test model configurations for
pipelexandvertex_aiplatforms
Changed
- Improved error messages in
StuffFactoryto include concept code and stuff name - Disabled
is_gen_object_supportedfor all Grok models (grok-3,grok-3-mini,grok-3-fast) - Updated test configurations to use different LLM models and platforms
- Modified
Jinja2filter to use defaultTagStyle.TICKSinstead of raising error - Added proper error handling for Unicode decode errors when loading model libraries
- Improved error handling in Anthropic plugin tests with specific
AuthenticationErrorhandling - Image handling in
AnthropicFactorynow converts image URLs tobase64data URLs with proper MIME type prefix - Put back Discord link in
README.md
Fixed
- Pinned
instructorto version<1.10.0to avoid errors withmypy
[v0.9.4] - 2025-09-06
Added
- Added support for BlackboxAI models
[v0.9.3] - 2025-09-06
Added
- Better support for BlackboxAI IDE
- VS Code extensions recommendations file with Pipelex, Ruff, and MyPy extensions
- File association for .plx files in VS Code settings
[v0.9.2] - 2025-09-05
Fixed
- Fix the rules of all agents.
Added
- Added agent rule for copilot
- Added a rule to forbidden structuring basic text concepts
[v0.9.1] - 2025-09-05
Fixed
- Fixed many inconsistencies in the documentation.
[v0.9.0] - 2025-09-02
Refacto
- Changed the pipeline file extension from
.tomlto.plx: Updated the LibraryManager in consequence.
Fixed
- Fixed the
structuring_methodbehavior in thePipeLLMpipe: Putting it topreliminary_text, thePipeLLMwill always generate text before generating the structure -> Reliability increased by a lot.
Fixed
- Fixed a bug in the
needed_inputsmethod of thePipeSequencepipe.
Changed
dry_run_pipenow returns aDryRunOutputobject instead of astrwith additional information.- Updated
cocodedependency from versionv0.0.10tov0.0.15.
Added
- Added the
FuncRegistryUtilsclass to register functions in the library.
[v0.8.1] - 2025-08-27
Bugfix
- Bugfix: Fixed the
PipeFuncoutput concept code and structure class name in the dry run.
[v0.8.0] - 2025-08-27
Refactor
- Refactored the concepts: Blueprints are now more explicit, and hold only concept strings or code. Pipes hold concept instances.
- Organized code: Created subfolders for controller and operator pipes.
- Say goodbye to
PipeLLMPrompt. - Removed the
PipeComposeandPipeLLMPromptfrom thePipeLLM.
Added
- Added a lot of unit tests.
- Loading the library can now be done from toml file or from
PipelexBundleBlueprint.
Fixed
- Backported
backports.strenumto>=1.3.0to support Python 3.10 now in dependencies and not in optional dependencies.
[v0.7.0] - 2025-08-20
Refactor
- Refactored the Blueprints. Introduces the
PipelexInterpreterthat interprets the Pipelex language and creates the Pipelex Blueprints (and vice versa) - Modified the way we declare pipes. Use the field
type = "PipeLLM"instead of fieldPipeLLM. (Same for all pipes) - Refactored the
LibraryManager. - Refactored CLI commands and added new ones. Modified CLI command structure:
pipelex init- Initialization commandspipelex init libraries [DIRECTORY]- Initialize pipelex libraries (createspipelex_librariesfolder)pipelex init config- Initialize pipelex configuration (createspipelex.toml)
pipelex validate- Validation and dry-run commandspipelex validate all -c pipelex/libraries- Validate all libraries and dry-run all pipespipelex validate pipe PIPE_CODE- Dry run a single pipe by its code
pipelex show- Show and list commandspipelex show config- Show the pipelex configurationpipelex show pipes- List all available pipes with descriptionspipelex show pipe PIPE_CODE- Show a single pipe definition
pipelex migrate- Migration commandspipelex migrate run- Migrate TOML files to new syntax (with--dry-runand--backupsoptions)
pipelex build- Build artifacts like pipeline blueprintspipelex build draft PIPELINE_NAME- Generate a draft pipelinepipelex build blueprint PIPELINE_NAME- Generate a pipeline blueprint
- Organized
concept,pipe,working_memory,stufffiles into folders.
Changed
- Allow
aiofilesversion>=23.2.1 - GHA Cla assistant fixed with Github App
Added
- New LLM families
LLMFamily.GPT_5,LLMFamily.GPT_5_CHATandLLMFamily.CLAUDE_4_1 - Added support for Claude 4.1 and GPT 5 models (inc. mini, nano, chat)
- New Pipe that generates pipe. Pipe code:
build_blueprint - New tests. Especially for the
PipelexInterpreter. - Migration files and cli commands to migrate Pipelex language to new syntax.
- Introduces
PipelexBundle, which correspond to the python paradigm of the Pipelex TOML syntax.
[v0.6.10] - 2025-08-02
Added
- New test file for source code manipulation functions (tests/cases/source_code.py)
- New integration test for PipeFunc functionality (tests/integration/pipelex/pipes/pipe_operator/pipe_func/test_pipe_func.py)
- New package structure file for pipe_func tests (init.py)
- Simplified input memory creation for native concepts (Text, Image, PDF) in pipeline execution
- Added Pipeline requests link to GitHub issue template config
Changed
- Updated pipeline execution documentation and examples to use input_memory instead of working_memory
- Renamed pipeline from 'extract_page_contents_from_pdf' to 'ocr_page_contents_from_pdf'
- Renamed pipeline from 'extract_page_contents_and_views_from_pdf' to 'ocr_page_contents_and_views_from_pdf'
- Updated cocode dependency from version 0.0.6 to 0.0.9
Fixed
- Fixed typo in pipeline description ('aspage views' to 'as full page views')
Removed
- Removed WorkingMemoryFactory and StuffFactory imports from pipeline execution examples
- Removed working memory creation code from pipeline examples
[v0.6.9] - 2025-07-26
Changed
Simplified input memory:
- The concept code can now be provided with arg named
conceptin addition toconcept_code - You can pass a simple string to create a
Textstuff
[v0.6.8] - 2025-07-25
Added
- New method
make_stuff_using_concept_name_and_search_domainsinStuffFactoryfor creating stuff using concept names and search domains. - New method
make_stuff_from_stuff_content_using_search_domainsinStuffFactoryfor creating stuff from stuff content using search domains. - New method
make_from_implicit_memoryinWorkingMemoryFactoryfor creating working memory from implicit memory. - New method
create_mock_contentinWorkingMemoryFactoryfor creating mock content for requirements.
Changed
- Refactored
PipeInputto useInputRequirementandTypedNamedInputRequirementclasses instead of plain strings for input specifications. - Updated
WorkingMemoryFactoryto handlePipelineInputsinstead ofCompactMemory. - Replaced
ExecutePipelineExceptionwithPipelineInputErrorinexecute_pipelinefunction. - Updated
PipeBatch,PipeCondition,PipeParallel,PipeSequence,PipeFunc,PipeImgGen,PipeCompose,PipeLLM, andPipeExtractclasses to useInputRequirementfor input handling. - Updated
PipeInputcreation in various test files to usemake_from_dictmethod. - Updated
pyproject.tomlto excludepypdfium2version4.30.1. - Updated
Jinja2TemplateCategoryto handle HTML and Markdown templates differently.
Fixed
- Corrected error messages in
StuffFactoryandStuffContentFactoryto provide more detailed information about exceptions.
[v0.6.7] - 2025-07-24
Removed
- Removed the
structure_classesparameter from thePipelexclass.
[v0.6.6] - 2025-07-24
Added
- Added a new method
verify_content_typein theStuffclass to verify and convert content to the expected type. - Added
cocode==0.0.6to the development dependencies inpyproject.toml.
Changed
- Updated
Stuffclass methods to use the newverify_content_typemethod for content verification. - Updated
vertexai.tomlto change LLM IDs from preview models to released models:gemini-2.5-proandgemini-2.5-flash.
Removed
- Removed
reinitlibraries,rl,v, andinittargets from the Makefile.
[v0.6.5] - 2025-07-21
Fixed
- In the documentation, fixed the use of
execute_pipeline.
[v0.6.4] - 2025-07-19
- Fixed the
README.mdlink to the documentation
[v0.6.3] - 2025-07-18
Changed
- Enhanced
Stuff.content_as()method with improved type validation logic - now attempts model validation whenisinstancecheck fails
[v0.6.2] - 2025-07-18
Added
- New
dry-run-pipecli command to dry run a single pipe by its code - New
show-pipecli command to display pipe definitions from the pipe library - New
dry_run_single_pipe()function for running individual pipe dry runs
Changed
- Updated
init-librariescommand to accept a directory argument and createpipelex_librariesfolder in specified location - Updated
validatecommand to use-cflag for the config folder path
[v0.6.1] - 2025-07-16
- Can execute pipelines with
input_memory: It is aCompactMemory: Dict[str, Dict[str, Any]]
[v0.6.0] - 2025-07-15
Changed
- Enhanced
Pipelex.make()method: Complete overhaul of the initialization method with new path configuration options and robust validation: - Added
relative_config_folder_pathandabsolute_config_folder_pathparameters for flexible config folder specification - The
from_fileparameter controls path resolution: ifTrue(default), relative paths are resolved relative to the caller's file location; ifFalse, relative to the current working directory (useful for CLI scenarios) - Renamed Makefile targets like
make doctomake docsfor consistency
Added
- Added github action for inference tests
load_json_list_from_pathfunction inpipelex.tools.misc.file_utils: Loads a JSON file and ensures it contains a list.- Added issue templates
- Updated Azure/OpenAI integrations, using dated deployment names systematically
[v0.5.2] - 2025-07-11
- log a warning when dry running a
PipeFunc - Update Readme.md
[v0.5.1] - 2025-07-09
Fixed
- Fixed the
ConceptFactory.make_from_blueprintmethod: Concepts defined in single-line format no longer automatically refineTextContentwhen a structure class with the same name exists ConceptFactory.make_concept_from_definitionis nowConceptFactory.make_concept_from_definition_str
Added
- Bumped
kajsontov0.3.0: IntroducingMetaSingletonfor better singleton management - Unit tests for
ConceptLibrary.is_compatible_by_concept_code
[v0.5.0] - 2025-07-01
Highlights
Vibe Coding an AI workflow becomes a reality — Create AI workflows from natural language without writing code: the combination of Pipelex's declarative language, comprehensive Cursor rules, and robust validation tools enables AI assistants to autonomously iterate on pipelines until all errors are resolved and workflows are ready to run.
Added
- Complete Dry Run & Static Validation System - A comprehensive validation framework that catches configuration and pipeline errors before any expensive inference operations.
- WorkingMemoryFactory Enhancement: New
make_for_dry_run()method creates working memory with realistic mock objects for zero-cost pipeline testing - Enhanced Dry Run System: Complete dry run support for all pipe controllers (
PipeCondition,PipeParallel,PipeBatch) with mock data generation usingpolyfactory - Comprehensive Static Validation: Enhanced static validation with configurable error handling for missing/extraneous input variables and domain validation
- TOML File Validation: Automatic detection and prevention of trailing whitespaces, formatting issues, and compilation blockers in pipeline files
- Pipeline Testing Framework: New
dry_run_all_pipes()method enables comprehensive testing of entire pipeline libraries - Enhanced Library Loading: Improved error handling and validation during TOML file loading with proper exception propagation
Configuration
- Dry Run Configuration: New
allowed_to_fail_pipessetting allows specific pipes (like infinite loop examples that fail on purpose) to be excluded from dry run validation - Static Validation Control: Configurable error reactions (
raise,log,ignore) for different validation error types
Documentation & Development Experience
- Cursor Rules Enhancement: Comprehensive pipe controller documentation covering
PipeSequence,PipeCondition,PipeBatch, andPipeParallel, improved PipeOperator documentation forPipeLLM,PipeOCR - Pipeline Validation CLI: Enhanced
pipelex validate all -c pipelex/librariescommand with better error reporting and validation coverage - Improved Error Messages: Better formatting and context for pipeline configuration errors
Changed
- Error Message Improvements: Updated PipeCondition error messages to reference
expression_templateinstead of deprecatedexpression_jinja2
[v0.4.11] - 2025-06-30
- LLM Settings Simplification: Streamlined LLM choice system by removing complex
for_object_direct,for_object_list, andfor_object_list_directoptions. LLM selection now uses a simpler fallback pattern: specific choice → text choice → overrides → defaults. - Image Model Updates: Renamed
image_bytesfield tobase_64inPromptImageTypedBytesfor better consistency. Updated to useCustomBaseModelbase class to benefit from bytes truncation when printing.
[v0.4.10] - 2025-06-30
- Fixed a bad import statement
[v0.4.9] - 2025-06-30
Highlights
Plugin System Refactoring - Complete overhaul of the plugin architecture to support external LLM providers.
Added
- External Plugin Support: New
LLMWorkerAbstractbase class for integrating custom LLM providers, and we don't mean only an OpenAI-SDK-based LLM with a custom endpoint, now the implementation can be anything, as long as it implements theLLMWorkerAbstractinterface. - Plugin SDK Registry: Better management of SDK instances with proper teardown handling
- Enhanced Error Formatting: Improved Pydantic validation error messages for enums
Changed
- Plugin Architecture: Moved plugin system to dedicated
pipelex.pluginspackage - LLM Workers: Split into
LLMWorkerInternalAbstract(for built-in providers) andLLMWorkerAbstract(for external plugins) - Configuration: Plugin configs moved from main
pipelex.tomlto separatepipelex_libraries/plugins/plugin_config.toml(⚠️ breaking change) - Error Handling: Standardized credential errors with new
CredentialsErrorbase class
[v0.4.8] - 2025-06-26
- Added
StorageProviderAbstract - Updated the changelog of
v0.4.7: MovedAdded StorageProviderAbstracttov0.4.8
[v0.4.7] - 2025-06-26
- Added an API serializer: introducing the
compact_memory, a new way to encode/decode the working memory as json, for the API. - When creating a Concept with no structure specified and no explicit
refines, set it to refinenative.Text JobMetadata: addedjob_name. Removedtop_job_idandwfidPipeOutput: addedpipeline_run_id
[v0.4.6] - 2025-06-24
- Changed the link to the doc in the
README.md: https://docs.pipelex.com
[v0.4.5] - 2025-06-23
Changed
- Test structure overhaul: Reorganized test directory structure for better organization:
- Tests now separated into
unit/,integration/, ande2e/directories - Created
tests/cases/package for pure test data and constants - Created
tests/helpers/package for test utilities - Cleaned up test imports and removed empty
__init__.pyfiles - Class registry refactoring: Updated kajson from 0.1.6 to 0.2.0, adapted to changes in Kajson's class registry with new
ClassRegistryUtils(better separation of concerns) - Dependency updates:
- Added pytest-mock to dev dependencies for improved unit testing
Added
- Coverage commands: New Makefile targets for test coverage analysis:
make cov: Run tests with coverage reportmake cov-missing(ormake cm): Show coverage with missing lines- Test configuration: Set
xfail_strict = truein pytest config for stricter test failure handling - Pydantic validation errors: Enhanced error formatting to properly handle model_type errors
Fixed
- External links: Removed broken Markdown target="_blank" syntax from MANIFESTO.md links
- Variable naming consistency: Fixed redundant naming in OpenAI config (openai_openai_config → openai_config)
- Makefile optimization: Removed parallel test execution (
-n auto) from codex-tests, works better now
Tests
- Unit tests added: New comprehensive unit tests for:
ClassRegistryUtilsFuncRegistryModuleInspector- File finding utilities
[v0.4.4] - 2025-06-20
Fixed
- Changed the allowed base branch names in the GHA
guard-branches.yml:doc->docs - Fixed
kajsondependency (see kajson v0.1.6 changelog)
Cursor rules
- Added Cursor rules for coding best practices and standards (including linting methods). Added TDD (Test Driven Development) rule on demand.
- Various changes
Documentation
- Added documentation for referencing images in PipeLLM.
- Fixed typos
Refactor
- Removed the
imagesfield from PipeLLM - images can now be referenced directly in theinputs - Moved the list-pipes CLI function to the
PipeLibraryclass.
[v0.4.3] - 2025-06-19
Fixed
- Removed deprecated Gemini 1.5 models: Removed
gemini-1.5-flashandgemini-1.5-profrom the VertexAI integration as they are no longer supported - Fixed multiple import statements across the codebase
Documentation
- Enhanced MkDocs search: Added search functionality to the documentation site
- Proofreading improvements: Fixed various typos and improved clarity across documentation
Refactor
- Mini refactor: changed kajson dependency to
kajson==0.1.5(instead of>=) to tolerate temporary breaking changes from kajson
[v0.4.2] - 2025-06-17
- Fixed the inheritance config manager method (Undocumented feature, soon to be removed)
- Fixed the
deploy-doc.ymlGitHub Action - Grouped the mkdocs dependencies in a single group
docsin thepyproject.tomlfile
[v0.4.1] - 2025-06-16
- Changed discord link to the new one: https://go.pipelex.com/discord
- Added
hello-worldexample in thecookbook-examplesof the documentation.
[v0.4.0] - 2025-06-16
Highlights
Complete documentation overhaul:
- MkDocs setup for static web docs generation
- Material for MkDocs theme, custom styling and navigation
- Other plugins: meta-manager, glightbox
- GitHub Pages deployment, mapped to docs.pipelex.com
- Added GHA workflows for documentation deployment and validation
- Added to docs:
- Manifesto explaining the Pipelex viewpoint
- The Pipelex Paradigm explaining the fundamentals of Pipelex's solution
- **Cookbook examples** presented and explained, commented code, some event with mermaid flow charts
- And plenty of details about using Pipelex and developing for Pipelex, from structured generation to PipeOperators (LLM, Image generation, OCR…) to PipeControllers (Sequence, Parallel, Batch, Condition…), workflow optimization, workflow static validation and dry run… there's still work to do, but we move fast!
- Also a major update of Cursor rules
Tooling Improvements
- Pipeline tracking: restored visual flowchart generation using Mermaid
- Enhanced dry run configuration: added more granular control with
nb_list_items,nb_extract_pages, andimage_urls - New feature flags: better control over pipeline tracking, activity tracking, and reporting
- Improved OCR configuration: handle image file type for Mistral-OCR, added
default_page_views_dpisetting - Enhanced LLM configuration: better prompting for structured generation with automatic schema insertion for two-step structuring: generate plain text and then structure via Json
- Better logging: Enhanced log truncation and display for large objects like image bytes (there are still cases to deal with)
Refactor
Concept system refactoring
- Improved concept code factory with better domain handling, so you no longer need the
nativedomain prefix for native domains, you can just call them by their names:Text,Image,PDF,Page,Number… - Concept
refinesattribute can now be a string for single refined concepts (the most common case)
Breaking Changes
- File structure changes: documentation moved from
doc/todocs/ - Configuration changes: some configuration keys have been renamed or restructured
StuffFactory.make_stuff()argumentconcept_coderenamed toconcept_strto explicitly support concepts without fully qualified domains (e.g.,TextorPDFimplicitlynative)- Some method signatures have been updated
Tests
- Added Concept refinement validation:
TestConceptRefinesValidationFunctionandTestConceptPydanticFieldValidationensure proper concept inheritance and field validation
[v0.3.2] - 2025-06-13
- Improved automatic insertion of class structure from BaseModel into prompts, based on the PipeLLM's
output_concept. New unit test included. - The ReportingManager now reports costs for all pipeline IDs when no
pipeline_run_idis specified. - The
make_from_strmethod from theStuffFactoryclass now usesTextcontext by default.
[v0.3.1] - 2025-06-10
Added
- New pytest marker
dry_runnablefor tests that can run without inference. - Enhanced
maketargets with dry-run capabilities for improved test coverage: make test-xdist(ormake t): Runs all non-inference tests plus inference tests that support dry-runs - fast and resource-efficientmake test-inference(ormake ti): Runs tests requiring actual inference, with actual inference (slow and costly)- Parallel test execution using
pytest-xdist(-n auto) enabled for: - GitHub Actions workflows
- Codex test targets
Changed
- Domain validation is now less restrictive in pipeline TOML: the
descriptionattribute is nowOptional
[v0.3.0] - 2025-06-09
Highlights
- Structured Input Specifications: Pipe inputs are now defined as a dictionary mapping a required variable name to a concept code (
required_variable->concept_code). This replaces the previous singleinputfield and allows for multiple, named inputs, making pipes more powerful and explicit. This is a breaking change. - Static Validation for Inference Pipes: You can now catch configuration and input mistakes in your pipelines before running any operations. This static validation checks
PipeLLM,PipeExtract, andPipeImgGen. Static validation for controller pipes (PipeSequence, PipeParallel…) will come in a future release. - Configure the behavior for different error types using the
static_validation_configsection in your settings. For each error type, choose toraise,log, orignore. - Dry Run Mode for Zero-Cost Pipeline Validation: A powerful dry-run mode allows you to test entire pipelines without making any actual inference calls. It's fast, costs nothing, works offline, and is perfect for linting and validating pipeline logic.
- The new
dry_run_configlets you control settings, like disabling Jinja2 rendering during a dry run. - This feature leverages
polyfactoryto generate mock Pydantic models for simulated outputs. - Error handling for bad inputs during
run_pipehas been improved and is fully effective in dry-run mode. - One limitation: currently, dry running doesn't work when the pipeline uses a PipeCondition. This will be fixed in a future release.
Added
native.AnythingConcept: A new flexible native concept that is compatible with any other concept, simplifying pipe definitions where input types can vary.- Added dependency on
polyfactoryfor mock Pydantic model generation in dry-run mode.
Changed
- Refactored Cognitive Workers: The abstraction for
LLM,ImgGen, andOcrworkers has been elegantly simplified. The old decorator-based approach (..._job_func) has been replaced with a more robust pattern: a public base method now handles pre- and post-execution logic while calling a private abstract method that each worker implements. - The
b64_image_bytesfield inPromptImageByteswas renamed tobase_64for better consistency.
Fixed
- Resolved a logged error related to the pipe stack when using
PipeParallel. - The pipe tracker functionality has been restored. It no longer crashes when using nested object attributes (e.g.,
my_object.attribute) as pipe inputs.
Tests
- A new pytest command-line option
--pipe-run-modehas been added to switch betweenliveanddryruns (default isdry). All pipe tests now respect this mode. - Introduced the
pipelex_apipytest marker for tests related to the Pipelex API client, separating them from generalinferenceorllmtests. - Added a
make test-pipelex-apitarget (shorthand:make ta) to exclusively run these new API client tests.
Removed
- The
llm_job_func.pyfile and the associated decorators have been removed as part of the cognitive worker refactoring.
[v0.2.14] - 2025-06-06
- Added a feature flag for the
ReportingManagerin the config:
[pipelex]
[pipelex.feature_config]
is_reporting_enabled = true
- Moved the reporting config form the
cogtconfig to the Pipelex config.
[v0.2.13] - 2025-06-06
- Added Discord badge on the Readme. Join the community! -> https://go.pipelex.com/discord
- Added a client for the Pipelex API. Join the waitlist -> https://www.pipelex.com/signup
- Removed the
run_pipe_codefunction. Replaced byexecute_pipelineinpipelex.pipeline.execute. - Added llm deck
llm_for_img_to_text. - Renamed
InferenceReportManagertoReportingManager: It can report more than Inference cost. RenamedInferenceReportDelegatetoReportingProtocol. - Added an injection of dependency for
ReportingManager - pipelex cli: fixed some bugs
[v0.2.12] - 2025-06-03
- pipelex cli: Split
pipelex initinto 2 separate functions:pipelex init-librariesandpipelex init-config - Fixed the inheritance config manager method
- Rename Mission to Pipeline
- Enable to start a pipeline and let in run in the background, getting it's run id, but not waiting for the output
Makefile: avoid defaulting pytest to verbose. Setup targetmake test-xdist= Run unit tests withxdist, make it the default for shorthandmake t. The oldmake tis nowmake tp(test-with-prints)- Added
mistral-small-3.1andqwen3:8b - Fix template pre-processor: don't try and substitute a dollar numerical like $10 or @25
- Refactor with less "OpenAI" naming for non-openai stuff that just uses the OpenAI SDK
[v0.2.11] - 2025-06-02
- HotFix for v0.2.10 👇 regarding the new pipelex/pipelex_init.toml`
[v0.2.10] - 2025-06-02
Highlights
Python Support Expansion - We're no longer tied to Python 3.11! Now supporting Python 3.10, 3.11, 3.12, and 3.13 with full CI coverage across all versions.
Major Model Additions - Claude 4 (Opus & Sonnet), Grok-3, and GPT-4 image generation are now in the house.
Pipeline Base Library update
- New pipe -
ocr_page_contents_and_views_from_pdftransferred from cookbook to base library (congrats on the promotion!). This pipe extracts text, linked images, AND page_view images (rendered pages) - it's very useful if you want to use Vision in follow-up pipes
Added
- Template preprocessor - New
@?token prefix for optional variable insertion - if a variable doesn't exist, we gracefully skip it instead of throwing exceptions - Claude 4 support - Both Opus and Sonnet variants, available through Anthropic SDK (direct & Bedrock) plus Bedrock SDK. Includes specific max_tokens limit reduction to prevent timeout/streaming issues (temporary workaround)
- Grok-3 family support - Full support via OpenAI SDK for X.AI's latest models
- GPT-4 image generation - New
gpt-image-1model through OpenAI SDK, available via PipeImgGen. Currently saves local files (addressing in next release) - Gemini update - Added latest
gemini-2.5-proto the lineup - Image generation enhancements - Better quality controls, improved background handling options, auto-adapts to different models: Flux, SDXL and now gpt-image-1
Refactored
- Moved subpackage
pluginto the same level ascogtwithin pipelex for better visibility - Major cleanup in the unit tests, hierarchy significantly flattened
- Strengthened error handling throughout inference flows and template preprocessing
- Added
make test-quiet(shorthandtq) to Makefile to run tests without capturing outputs (i.e. without pytest-soption) - Stopped using Fixtures for
pipe_routerandcontent_generator: we're now always getting the singleton frompipelex.hub
Fixed
- Perplexity integration - Fixed breaking changes from recent updates
Dependencies
- Added pytest-xdist to run unit tests in parallel on multiple CPUs. Not yet integrated into the Makefile, so run it manually with
pytest -n auto(without inference) orpytest -n auto -m "inference"(inference only). - Swapped pytest-pretty for pytest-sugar - because readable test names > pretty tables
- Updated instructor to v1.8.3
- All dependencies tested against Python 3.10, 3.11, 3.12, and 3.13
Tests
- TestTemplatePreprocessor
- TestImgGenByOpenAIGpt
- TestImageGeneration
- TestPipeImgGen
[v0.2.9] - 2025-05-30
- Include
pyproject.tomlinside the project build. - Fix
ImgGenEngineFactory: image generation (imgg) handle required format isplatform/model_name - pipelex cli: Added
list-pipesmethod that can list all the available pipes along with their descriptions. - Use a minimum version for
uvinstead of a fixed version - Implement
AGENTS.mdfor Codex - Add tests for some of the
tools.misc - pipelex cli: Rename
pipelex run-setuptopipelex validate all -c pipelex/libraries
[v0.2.8] - 2025-05-28
- Replaced
poetrybyuvfor dependency management. - Simplify llm provider config: All the API keys, urls, and regions now live in the
.env. - Added logging level
OFF, prevents any log from hitting the console
[v0.2.7] - 2025-05-26
- Reboot repository
[v0.2.6] - 2025-05-26
- Refactor: use
ActivityManagerProtocol, renameBaseModelTypeVar
[v0.2.5] - 2025-05-25
- Add custom LLM integration via OpenAI sdk with custom
base_url
[v0.2.4] - 2025-05-25
- Tidy tools
- Tidy inference API plugins
- Tidy WIP feature
ActivityManager
[v0.2.2] - 2025-05-22
- Simplify the use of native concepts
- Include "page views" in the outputs of Ocr features
[v0.2.1] - 2025-05-22
- Added
OcrWorkerAbstractandMistralOcrWorker, along withPipeExtractfor OCR processing of images and PDFs. - Introduced
MissionManagerfor managing missions, cost reports, and activity tracking. - Added detection and handling for pipe stack overflow, configurable with
pipe_stack_limit. - More possibilities for dependency injection and better class structure.
- Misc updates including simplified PR template, LLM deck overrides, removal of unused config vars, and disabling of an LLM platform id.
[v0.2.0] - 2025-05-19
- Added OCR, thanks to Mistral
- Refactoring and cleanup
[v0.1.14] - 2025-05-13
- Initial release 🎉