Packages

A package is a collection of .mthds bundles with a METHODS.toml manifest at the project root. The manifest gives the package an identity and defines which pipes are exported to the outside world.

What the Manifest Does

METHODS.toml currently serves two main purposes:

• Identity: give the package an address, version, and description
• Visibility: declare which pipes are exported through [exports]

The Package Manifest: METHODS.toml

Here is a representative manifest:

[package]
address = "github.com/acme/legal-tools"
version = "1.0.0"
description = "Legal document analysis and contract review methods."
authors = ["Acme Corp"]
license = "MIT"
mthds_version = ">=0.8.0"

[exports.legal.contracts]
pipes = ["extract_clause", "analyze_contract"]

[exports.scoring]
pipes = ["compute_weighted_score"]

Field Reference

Exports and Visibility

The [exports] section controls which pipes are visible outside the package.

Default Behavior

• Without METHODS.toml, or with one that declares no [exports]: every pipe of the package is visible from outside
• With an [exports] section: pipes are private by default — only the pipes listed in [exports] (and each bundle's main_pipe) are visible from outside

Declaring Exports

Exports are grouped by domain path:

[exports.legal.contracts]
pipes = ["extract_clause", "analyze_contract"]

[exports.scoring]
pipes = ["compute_weighted_score"]

This manifest exports two pipes from legal.contracts and one pipe from scoring. Exports are read by domain: exporting compute_weighted_score from scoring does not export a pipe of the same name declared in another domain of the package.

Pipes Call Their Own Package

Inside a package, a pipe calls the package's own pipes, private ones included. A sequence exported by the package can call a helper the manifest does not export, and that helper is loaded with it. A private pipe that no exported pipe calls is not loaded at all, so a problem in it never stops a consumer from loading the package.

A reference written inside a package always reaches that package's pipe. If the consumer declares a pipe with the same domain and code, or another loaded package does, the package's own call is never redirected to it.

What a Consumer Is Refused

When a consumer references a loaded package, two mistakes are refused when the consumer loads, rather than surfacing later as an unrelated error:

• A pipe the package does not have is refused as an unresolved pipe dependency (unresolved_pipe_dependency), naming the reference and the package. The same refusal names the error when the package declares the pipe but it failed to build.
• A pipe the package does not export is refused as an unexported pipe dependency (unexported_pipe_dependency). The pipe exists, so the remedy is to call one of the package's exported pipes, or to export this one in the package's manifest.

These checks apply to the consumer's references and to a package's references to its own pipes. A package's reference to one of its own dependencies is not checked when the consumer loads, because a package's dependencies are not loaded with it.

A reference by bare code, alias->code, ignores the package's private pipes when it also matches an exported one, so a private helper sharing an exported pipe's code in another domain does not make the reference ambiguous.

Naming a private pipe by hand, as the pipe to run at the command line or in an API request, is not a reference from inside a method, so [exports] does not apply to it: a loaded private pipe can be run that way by its alias->domain.code.

Cross-Package References

Pipelex can resolve cross-package references when the required packages are available to the runtime. The reference form is:

alias->domain.code

Example:

[pipe.analyze_item]
type = "PipeSequence"
description = "Analyze an item using another package"
steps = [
    { pipe = "scoring_lib->scoring.compute_weighted_score" },
]

This runtime-level capability exists separately from the current public mthds package CLI workflow documented on this page. If you rely on cross-package references, validate them against the specific runtime setup you are using.

Include by Address: Fetch-on-Miss

The alias can also be a full package address, so a bundle can reference a method from a public GitHub repository directly — the same reference grammar as running a method by address:

[pipe.summarize_report]
type = "PipeSequence"
description = "Extract a document then summarize it"
inputs = { doc = "PDF" }
output = "Summary"
steps = [
    { pipe = "github.com/Pipelex/methods/documents->documents.extract_page_contents_and_views", result = "pages" },
    { pipe = "summarize_pages" },
]

An address-based reference resolves against the installed methods — ~/.mthds/methods/ (global), .mthds/methods/ in the working directory (project-local), and a .mthds/methods/ directory beside the bundle being loaded or in a directory above it — matched by the manifest's address + name, case-insensitively. When no installed method matches, Pipelex fetches on miss: it fetches the package by address — honoring an @<tag> pin, through the same grammar, bounds, and tags-only rules as a direct fetch — installs it into ~/.mthds/methods/, records the fetch provenance (address, tag, the commit SHA of what was actually cloned, and the manifest's display_name) in a .provenance.json sidecar beside the manifest, and loads it so resolution proceeds. The next load finds the installed copy and never touches the network again.

A few behaviors worth knowing:

• Installed wins over a fetch. Once a method is installed, it is used as-is — including when a reference pins a different @<tag> than what is installed (a warning tells you so). Remove the installed directory to re-fetch.
• A fetched package is checked against the calling method's methods.lock. When the method calling a package by address carries a methods.lock at the root of its tree, beside its METHODS.toml, and Pipelex resolves that package from a copy it fetched itself, whether for this load or by an earlier fetch whose copy carries a .provenance.json sidecar, the tree digest of that copy is compared with the digest the lock records for the package's address (matched without regard to case, any @<tag> stripped). A different digest refuses the load with security.integrity_mismatch, whose message names the address, the digest and version the lock records, and the version and digest that were fetched. The digest is compared whatever version the lock names: the version is part of the tree, in METHODS.toml, so a lock recording another version than the one fetched refuses the load too, and the message says to pin the reference to the locked version or to lock the method again. When the reference pins a @<tag> but an installed copy fetched at another tag or at the default branch answered it (installed wins, above), the message says so instead, naming both, and says to remove that installed copy so the pinned tag is fetched. A bundle loaded in a later batch of the same library, such as the contents a run submits beside its library directories, is checked against the tree an earlier batch fetched and loaded for the same reference, since it runs that tree. Pipelex never fetches the locked version in place of the reference, since a lock verifies what was fetched and never chooses it. A lock that does not name the address, or no lock at all, checks nothing. A package shipped with the method (under .mthds/methods/<name>/ or in a request's dependencies) is not checked, as it travels with the lock from the same author, and neither is a copy another installer wrote without a provenance sidecar, such as mthds install, which checks the lock itself. The lock is read only when a dependency the method calls resolves to a copy Pipelex fetched, for this load or an earlier one; a load whose dependencies are all shipped, or installed otherwise, never reads it. A lock Pipelex then cannot read, including one written before tree digests that records a hash, one whose bytes cannot be read, and one that changed since the method's tree was listed, refuses the load with method.package_invalid, reason lock_invalid, path methods.lock.
• A package can travel with the method. A method sent over HTTP to a Pipelex API server as files can carry the packages it calls in dependencies, each a map of file entries keyed by its address, and one sent as a bundle_b64 archive under .mthds/methods/<name>/ at its root, each with its METHODS.toml. The server searches them before its installed copies and before fetch-on-miss, for that request alone, so a shipped package wins over an installed copy and runs where fetching is disabled; see Sending the method packages the method calls. On disk, the .mthds/methods/<name>/ layout beside the bundle is one of the local stores above.
• A package bundle that does not parse refuses the load. When a bundle of a referenced package does not parse, the bundle calling it is refused with the parser's items, each naming the file as <address>/<path inside the package>, never by its path on the machine, whether the package was shipped with the bundle, installed or fetched.
• A miss that cannot be bridged refuses the bundle, never a silent pass. Fetch disabled, an address this runtime cannot fetch, or a failed fetch (a repository that does not exist, a @tag that is not a tag, a repository holding no package at that address) refuses the load with one unresolved_package_dependency item per package, located on the first pipe that calls it, with the reference as written in missing_pipe_code and the reason and the remedy in its message. It is the same invalid verdict as any other mistake in the bundle: pipelex validate reports it, POST /v1/validate answers 200 with is_valid: false, and a run is refused with a 422 before any pipe runs. Every package of the bundle is tried before the refusal, so two bad references give two items.
• Hosted parity warning. A fetched method declaring Python structure classes runs locally but triggers the same warning as a direct CLI fetch: hosted execution accepts MTHDS concepts and sandboxed PipeFuncs, not in-process Python. On a sandbox-hosted deployment the fetch refuses such a package outright, with the same rule-naming error. The refusal is not specific to fetching: a sandbox-hosted deployment refuses every method whose Python declares structure classes, however it arrived (see Python classes).

To disable network fetches at load time, set the switch in your pipelex.toml (or use the environment variable, which takes precedence):

[interpreter.methods]
fetch_on_miss = false

export PIPELEX_METHODS_FETCH_ON_MISS=0

With fetching disabled, a miss refuses the bundle with that item, whose message names the address and says to ship the package with the bundle under .mthds/methods/<name>/ or to install it where the runtime runs (for example with mthds install <address>). See Methods Configuration.

Example Layout

your-project/
├── METHODS.toml
├── my_project/
│   ├── finance/
│   │   ├── invoices.mthds
│   │   └── invoices_struct.py
│   └── legal/
│       ├── contracts.mthds
│       └── contracts_struct.py
├── .pipelex/
│   └── pipelex.toml
└── requirements.txt

The METHODS.toml file sits at the project root. Pipelex discovers it by walking up from bundle locations until it finds the manifest, stopping at a repository's root, and the folder holding it roots the package's method tree: the files of the package its steps can read.

Current Public CLI Workflow

The public package-management commands documented in this repo are provided by the lowercase mthds CLI:

mthds package init
mthds package list
mthds package validate

Initialize a Manifest

mthds package init

This command creates METHODS.toml interactively. It asks for package metadata and writes a manifest with an empty [exports] section.

Inspect a Manifest

mthds package list

This command reads the manifest from the target package directory and prints the parsed package metadata and exports.

Validate a Manifest

mthds package validate

This command validates the manifest syntax and package fields.

Target a Different Directory

All three commands accept -C, --package-dir <path>:

mthds package init -C ./my-package
mthds package list -C ./my-package
mthds package validate -C ./my-package

Important Compatibility Note

The current public mthds package validate command focuses on package identity and exports. It does not document a public dependency-management workflow here, and it rejects a [dependencies] section in the current JS CLI validation path.

For this reason, this documentation only describes the stable public package-manifest workflow exposed by the current mthds CLI.

Related Documentation

• Domain — How domains organize concepts and pipes
• Libraries — How libraries load and validate bundles
• Pipelex Bundle Specification — The .mthds file format
• Package Commands — mthds package command reference