Inference family surface
An inference family is a kind of model call — LLM completion, image generation, extraction, web search, judgment. Adding one is not one package: the family's name is spelled in several enums, its usage flows through reporting and cost, its models live in a deck, and each of those places is separate, with nothing inferring one from another. This page is the checklist, written from the judgment family's addition, and a sibling of the registration surface for pipe kinds.
Most of the list is enforced by the type checker: the matches over the family enums are exhaustive with no case _, so adding the enum value walks you to their arms. The entries marked silent are the ones nothing walks you to, and they are the reason this page exists.
A new family
-
The family's package,
pipelex/cogt/<family>/, mirroringpipelex/cogt/search/: the question/request and answer models, the setting and its model-choice union, the job, the job factory, the usage report, the worker contract (<Family>WorkerAbstractwith one template method and one abstract hook) and the worker factory, which resolves its worker through the inference-backend registry and holds nomatchover SDK strings. -
The family enums, together.
InferenceFamilyinpipelex/plugins/inference_backend_registry.py,ModelTypeinpipelex/cogt/model_backends/model_type.py,InferenceErrorFamilyinpipelex/cogt/inference/error_classify.py, with an arm in itsis_idempotent(a family whose call is not idempotent, such as image generation, reports a failure after the request was sent asprovider.outcome_unknown) and an entry in both the failure-class and not-found-class tables ofpipelex/cogt/inference/error_render.py, andModelCategoryinpipelex/cogt/models/model_listing.py, which is whatpipelex-agent modelsandcheck-modellist from. -
The error classes in
pipelex/cogt/exceptions.py: a job failure, a model-not-found (subclassingModelNotFoundError) and a handle-not-found. Then regenerate the identity snapshot (make gei); the snapshot test fails until you do. A code the family's errors need beyond those the error registry already declares is added topipelex/errors/registry/pipelex-error-registry.yaml, followed bymake gerto regenerate its outputs andmake certo check them (The Error Registry). -
The job identity in
pipelex/system/job_metadata.py: aJobCategorymember, aUnitJobIdmember and its display arm. -
The deck. A
<Family>DeckBlueprint, its flat fields onModelDeck, the setting getter, the preset validator, the choice check, the suggestion arms, and the model manager's handle collection and flattening. The kit deck ispipelex/kit/configs/inference/deck/<n>_<family>_deck.toml— the numeric prefix is what makespipelex updatemanage it — mirrored into the repository's own.pipelex/inference/deck/. -
Reporting and cost. The usage and cost-report classes join both unions in
pipelex/reporting/reporting_types.py,ReportingManagergets its dispatch arm, andCostRegistry.compute_cost_reportgets an arm in itsmatch, which closes withassert_neverso a family that falls through fails the type check. Update themodel_typecomment onModelUsageSpecinpipelex/graph/graphspec.pyto say how the family is billed. -
The content-generation leaf. An assignment model in
pipelex/cogt/content_generation/assignment_models.py, a<family>_generate.pymodule whose coroutine opens with the dry branch, the dry mock indry_mock.py, and the method onContentGeneratorProtocolwith its@overrideinContentGenerator. A host runtime's durable execution needs an activity and a queue for the new leaf in its own plugin. -
Test infrastructure. A pytest marker in
pyproject.tomland its term in the default deselect expression; the family in every table ofpipelex/cli/dev_cli/commands/preprocess_test_models_cmd.py; a<family>_modelskey on every profile in.pipelex-dev/test_profiles.toml; a combo getter intests/integration/pipelex/fixtures/model_selection.py, a combo fixture incombo_fixtures.pyand its re-export intests/integration/pipelex/conftest.py. -
The migration golden. A new
ModelTypevalue changes the inference-backend surface's fingerprint;make umigrecords it, and the real-surfaces test fails until you do. -
Silent: fakes that stand in for the deck. Any test double that impersonates
ModelDeckby attribute — the fake deck intests/unit/pipelex/cli/test_agent_models_cmd.pyis one — needs the new family's presets, aliases and waterfalls, or every case using it fails with anAttributeErrorfar from the cause. -
The MTHDS protocol's model categories.
PipelexMTHDSProtocol.models()inpipelex/pipeline/runner.pytypes each preset by the protocol's ownModelCategory, which carries every family this runtime serves,judgmentincluded, through two exhaustive matches beside it:_protocol_category_offor the deck's entries and_listing_category_offor the?type=filter. A new member of the model listing'sModelCategory(step 2) stops the type checker at the first of them, because a new family needs a protocol category too, which the standard adds in a protocol minor for each settings family it adds. Until the protocol defines one, the family's presets go under an extension propertyPipelexModelDeckadds for them, never under a category the protocol does not define, since a runner must not put such a value in a deck entry'stype. -
Files, when the family reads them. The job and the assignment carry them inside the prompt that numbers them, as
JudgmentJobandJudgmentAssignmentcarry aJudgmentPromptwhoseimagesanddocumentsare the files itstextmarks with[Image N]and[Document N]tokens, and the assignment'sreferenced_uris()returns each one, or a scoped run's read authorization never sees them. A family whose operator renders a template shapes its contract around the rendered prompt rather than the step's inputs: the operator assembles it throughpipelex/kernel/prompt_assembly.py, which PipeLLM, PipeImgGen and PipeJudge share, so the worker receives one text and its ordered files and never reads an input by name. The worker's template method checks that the model reads them before its hook runs, raising the family's capability error (JudgmentWorkerAbstract._check_can_read_files), and runs the per-file format checks inpipelex/cogt/inference/prompt_file_checks.py, which the LLM family shares. What a model reads is theinputsof its spec in the backend's model file, so the capability is configuration, and an operator that knows its model at load time refuses a file input there too, asPipeJudgedoes. -
Outcomes the worker may return besides an answer. A model may decline a question, and the worker reports that as an outcome rather than as an error or a made-up answer:
judgereturns aJudgmentOutcomeper question, an answer of the question's kind or aJudgmentRefusal, and the base checks an answer's key, kind and offered option while accepting a refusal for any question it was asked. What a refusal means is the caller's policy, never the worker's: the kernel's single-question judgment raisesJudgmentRefusedError, a content error naming the step and the model, its several-question judgment leaves an optional field absent and raises the same error naming the question behind a required one, and neither reads a refusal as a no, a first option or a lowest level. -
Silent: a default nothing serves boots cleanly and fails at run time. A boot checks only that each alias a deck names exists, not that an enabled backend serves the model it resolves to, so a
default-*alias pointing at a model no backend declares loads without complaint and fails the first pipe that relies on it with aModelNotFoundError, for everyone running on their own keys.tests/unit/pipelex/kit/test_shipped_deck_defaults.pyis the guard: everydefault-*alias of the shipped deck must reach, through aliases, presets and every entry of a waterfall, a model some backend file of the kit declares. Give a new family's defaults such a model, or ship no default, asjudgmentships nochoice_default.
A new backend for a family
-
The provider package,
pipelex/providers/<vendor>/: the plugin, the worker, the translation to and from the vendor's vocabulary, and a<vendor>_exceptions.py. The vendor's own limits are enforced in the translation, not in a blueprint. -
Error classification. An
extract_<vendor>_metadatafunction and aProviderNamemember inpipelex/cogt/inference/, with the member added to every exhaustive match overProviderNameand to the parity test intests/unit/pipelex/cogt/inference/test_provider_classification_parity.py. When the vendor's status codes cannot separate its failures, the worker branches on the body before handing the rest to the shared classifier. -
Registration. The plugin joins
KERNEL_BUILTIN_PLUGINSinpipelex/providers/builtins.py. Itsmake_workerclosure callsrequire_sdkbefore importing the worker, and imports no SDK at module level. Record theregistermethod's subject grant beforemake agent-check, whose keyword-only fixer would otherwise rewrite it into a signature the plugin protocol refuses. -
Configuration. The backend's table in the kit's
backends.toml, its model file underbackends/, and anall_<vendor>routing profile — which is also what the integration combo fixtures route to — all mirrored into.pipelex/inference/. Its API key variable goes in.env.example. -
Silent: the routing profiles. The default
all_enabled_backendsprofile tries the backends itsfallback_orderlists first and any other enabled backend after them, so list the vendor there when it should be tried before the end. A profile whosedefaultis a single backend sends every handle there, and a handle that backend does not serve is silently dropped from the deck — so a user who has the vendor's key still cannot use its models. Give such a profile anoptional_routesentry sending the vendor's models to its backend; an optional route applies only while that backend is enabled. -
Silent: the CI placeholder list. CI runners hold no provider keys, and every test module boots live, so each variable an enabled kit backend references must be in
ENV_VAR_KEYS_WHICH_MAY_NEED_PLACEHOLDERS_IN_CIinpipelex/test_extras/shared_pytest_plugins.py.tests/unit/pipelex/test_extras/test_ci_placeholder_keys.pyfails when one is missing. The agent CLI's end-to-end tests build their own hermetic environment, with a dummy value per variable, in theoffline_subprocess_envfixture oftests/e2e/agent_cli/conftest.py, and nothing checks that list: a dry run resolves no key, so a variable missing there only reaches the tests of commands that check credentials, such aspipelex doctor. -
Plugin surface tests. The
(family, sdk)pair intests/unit/pipelex/plugins/test_inference_backend_coverage.py, a missing-extra guard test mirroring Linkup's, the SDK's import name in the blocked list oftests/unit/pipelex/plugins/test_import_light_boot.py, and an arm intests/integration/pipelex/system/test_keyless_boot_forced_dry.pyshowing that a keyless boot does not need the key. -
Live tests under a test profile of their own, marked with the family's marker and
inference, so thatmake ti PROF=<profile>runs them and every other profile skips them.