Init CLI Flows

pipelex init sets up the .pipelex/ configuration directory and decides where runs execute: on the hosted Pipelex API, with a Pipelex API key, or on this machine, with the user's own provider keys. It handles four independent concerns — config files, inference backends, routing profiles, and telemetry — through a focus-based dispatch system, in three stages: inspect, choose, execute. Each concern owns its own file-copying and customization logic, so they can be run together or individually without interference.

Why This Design

The .pipelex/ directory contains two categories of files with different lifecycles:

1. Config files (pipelex.toml, plxt.toml) — static templates copied verbatim from the kit (no interactive customization); re-running init overwrites them (full reset).
2. Inference files (inference/backends.toml, inference/routing_profiles.toml, inference/backends/*.toml, inference/deck/*.toml) — interactive setup, customized per-project based on which AI backends the user selects.

These two categories are managed by separate steps. init_config() copies only config files (skipping the inference/ directory entirely). The inference step handles its own template copying and then runs interactive backend selection and routing customization. Each file is owned by exactly one step — init_config() explicitly skips the inference/ directory via INIT_SKIP_DIRS, and skips telemetry.toml via INIT_SKIP_FILES. This separation ensures that re-running pipelex init config never overwrites a user's carefully tuned inference setup.

Interfaces

CLI Commands

Every one of them first shows the usage statistics notice on stderr when statistics are on, through show_usage_stats_notice(), which the init Typer command in pipelex/cli/_cli.py calls before init_cmd(), so pipelex doctor --fix, which calls init_cmd() itself, does not show it. A first run that has just printed the notice does not print it again, and showing it records no marker.

All commands except credentials perform a full reset (overwrite existing files) — every setting in the file is replaced by the template's. That is why init is not the answer to a configuration file that has fallen behind the current schema: pipelex migrate rewrites such a file in place and keeps what is in it.

Inputs

• focus (InitFocus enum): Determines which steps run. Derived from the CLI subcommand.
• skip_confirmation (bool): When True, asks nothing: skips the confirmation, keeps the [run] execution the target already sets (local for a pipelex.toml that sets none), takes the hosted Pipelex API only for a brand-new home with no pipelex.toml, and prints pipelex login instead of opening a browser. Used when called from pipelex doctor --fix.
• local (bool): When True, targets the project-level .pipelex/ directory instead of the global ~/.pipelex/. Maps to the --local CLI flag.

Outputs / Side Effects

Architecture

Three Stages

init_cmd() runs every focus but credentials through three functions:

1. Inspect — inspect_initialization(focus=, local=) resolves the target directory, reads what is on disk and returns an InitInspection: which steps are needed, whether this run asks where runs execute (asks_setup_path), the [run] execution the target pipelex.toml sets now (configured_execution), and what a former release left (former_release_findings and former_release_boot_blockers, see below). It asks nothing and writes nothing.
2. Choose — choose_initialization(console=, inspection=, skip_confirmation=) first offers the cleanup of what a former release left, when the inspection found any and skip_confirmation is false. Then it shows the confirmation panel, then, when asks_setup_path, the question prompt_setup_path() asks. It returns an InitChoices: the SetupPath (HOSTED or LOCAL, None when this run does not decide it) and whether anyone answers prompts.
3. Execute — execute_initialization(console=, inspection=, choices=) writes the files and runs the steps the choices call for.

asks_setup_path is true when the run sets up inference and its focus is all or config: the full setup always, and a configuration reset only on a first setup. inference configures this machine's backends on purpose and never asks.

A Former Release's Configuration

A machine set up by a release that ran models through the Pipelex Gateway still carries what that took: its backend table, routing profiles that send models to it, its per-backend file and model lists, model_specs_section keys and pipelex_service.toml. What those releases left for Pipelex Manifold, which is not retired, stays as it is. The boot refuses such a machine with FormerReleaseConfigError, naming pipelex migrate and pipelex init.

The inspect stage runs detect_former_release_across() (pipelex/migration/former_release.py) over the configuration directories a boot reads (config_manager.existing_config_dirs, the home's then the project's, kept in former_release_config_dirs), reading them together as the boot merges them, and keeps the directories with findings in former_release_findings. Whether what they hold stops this machine's boot is a separate reading, former_release_boot_blockers() over config_manager.backends_file_paths() and routing_profiles_file_paths(), the files the boot itself merges, kept in former_release_boot_blockers: a project booting on bases of its own is not told it cannot start because of the home's, and a profile one directory activates from the other is not missed. The choose stage then calls offer_former_release_cleanup(), before the confirmation: it shows a panel naming what stops the boot, when anything does, and the files it is in, asks "Clean it up now?" (default yes), and on a yes runs apply_former_release_cleanup() over former_release_config_dirs, the write pass of pipelex migrate's first step, with the same copies kept. When that pass leaves a file for a hand edit, or its check after writing finds that the boot still cannot start, init says so and the setup goes on. A no leaves the files as they are and the setup goes on, with "Pipelex will not start until then" added only when former_release_boot_blockers is not empty. Under skip_confirmation (pipelex doctor --fix) the offer is skipped and the cleanup never runs: nobody is there to answer, and the doctor asks about the cleanup in its own migrations row.

The cleanup is needed even though a full reset rewrites the target's backends.toml and routing_profiles.toml: the reset copies the kit's backend files over the target's but removes none, so the retired backend files, the model lists and pipelex_service.toml would stay, and so would everything in the other directory and in the personal overrides.

Overall Flow

flowchart TD
    START([pipelex init]) --> FOCUS{focus?}

    FOCUS -- credentials --> CREDS_DIRECT["prompt_credentials<br/>Prompt for missing API keys"]
    CREDS_DIRECT --> DONE

    FOCUS -- "all / config / inference / routing / telemetry" --> INSPECT["inspect_initialization<br/>determine_needs, first-time detection,<br/>current run execution, former release"]
    INSPECT --> FORMER{"former release found<br/>and someone to answer?"}
    FORMER -- Yes --> CLEANUP["offer_former_release_cleanup<br/>clean up on yes"]
    FORMER -- No --> SKIP
    CLEANUP --> SKIP{skip_confirmation?}
    SKIP -- "Yes (doctor --fix)" --> DEFAULT["Setup path: the configured one, local for a<br/>pipelex.toml without it, hosted for a new home<br/>(when the run asks it)"]
    SKIP -- No --> CONFIRM{User confirms?}
    CONFIRM -- No --> CANCEL([Cancelled])
    CONFIRM -- Yes --> ASKS{asks_setup_path?}
    ASKS -- Yes --> QUESTION["prompt_setup_path<br/>1 hosted (Enter) / 2 this machine"]
    ASKS -- No --> EXEC
    QUESTION --> EXEC
    DEFAULT --> EXEC[execute_initialization]

    EXEC --> S1{needs_config?}
    S1 -- Yes --> INITCFG["init_config()<br/>Copies non-inference files<br/>(writes back a kept run execution)"]
    S1 -- No --> S2
    INITCFG --> S2

    S2{needs_inference?}
    S2 -- Yes --> COPY_INF["Copy inference templates<br/>(backends, deck, routing)"]
    COPY_INF --> PATH{setup path?}
    PATH -- hosted --> HOSTED["Write execution = hosted<br/>Suggest IDE extension"]
    PATH -- "local / none" --> CUST_BE[customize_backends_config<br/>Interactive backend selection]
    CUST_BE --> CHK_RT{check_routing?}
    CHK_RT -- "No (auto-route)" --> CUST_RT[customize_routing_profile<br/>Auto-routing based on selection]
    CHK_RT -- "Yes (focus=routing)" --> LOCAL_SET
    CUST_RT --> LOCAL_SET["Write execution = local<br/>(when the run asked)"]
    HOSTED --> S25
    LOCAL_SET --> S25

    S2 -- No --> S25

    S25{check_credentials<br/>and not hosted?}
    S25 -- Yes --> CREDS["prompt_credentials<br/>Prompt for missing API keys"]
    S25 -- No --> S3
    CREDS --> S3

    S3{needs_routing?}
    S3 -- Yes --> ROUTE[customize_routing_profile<br/>Standalone routing setup]
    S3 -- No --> S4

    ROUTE --> S4

    S4{needs_telemetry?}
    S4 -- Yes --> TELEM[setup_telemetry<br/>Copy telemetry template]
    S4 -- No --> S5
    TELEM --> S5

    S5{setup path hosted?}
    S5 -- Yes --> KEY["ensure_pipelex_api_key<br/>key set: keep it / interactive: pipelex login /<br/>doctor --fix: print pipelex login"]
    S5 -- No --> DONE2([Done])
    KEY --> DONE2

Implementation

Determine Needs

determine_needs(), called by the inspect stage, evaluates the current state of .pipelex/ to decide which steps are required:

nb_missing_config_files = init_config(reset=False, dry_run=True) if check_config else 0
needs_config = check_config and (nb_missing_config_files > 0 or reset)
needs_inference = check_inference and (not path_exists(backends_toml_path) or reset)
needs_routing = check_routing and (not path_exists(routing_profiles_toml_path) or reset)
needs_telemetry = check_telemetry and (not path_exists(telemetry_config_path) or reset)

The check_* flags are derived from the focus parameter:

First-Time Inference Detection

init_config() never copies inference/, so a configuration reset on a directory with no inference/backends.toml is a first setup. The inspect stage sets up inference too in that case, whatever the focus:

if needs_config and is_first_time_backends_setup:
    needs_inference = True

Where Runs Execute

pipelex/cli/commands/init/setup_path.py holds the choice: SetupPath (HOSTED, LOCAL), DEFAULT_SETUP_PATH (hosted, the answer Enter takes, and the one doctor --fix takes for a brand-new home with no pipelex.toml; otherwise InitInspection.unattended_setup_path keeps the configured [run] execution, and local for a pipelex.toml that sets none), and the tomlkit edit of [run] execution, write_run_execution() and read_run_execution(), which keep every other line of pipelex.toml, comments included. The question itself is prompt_setup_path() in ui/setup_path_ui.py.

• Hosted writes execution = "hosted" in Step 2 and reconfigures nothing after the reset, so the inference files are the kit's defaults, any customised backends and routing included, until pipelex init inference reconfigures them; it skips the credentials step, and, once every file is written (Step 5), calls ensure_pipelex_api_key(): a key already in PIPELEX_API_KEY or saved in the home .env is kept with no login and no network call; otherwise an interactive run calls login_with_browser() from pipelex/cli/commands/login/ (see pipelex login), and a run with nobody to answer prints pipelex login. An OSError in that step, such as a loopback listener that cannot bind, is reported as a saved setup with no key, naming pipelex login and pipelex login --paste, since every file is already written.
• Local runs the backend and routing steps unchanged, writes execution = "local", and runs the credentials step.

A configuration reset that does not ask the question (pipelex init config on an existing setup) copies the kit's pipelex.toml, whose execution is "local", so the execute stage writes back the configured_execution the inspection read, and skips the credentials step when that is "hosted".

Step 1: Config Step — init_config()

Copies the config template tree from kit/configs/ to .pipelex/, with two skip mechanisms:

INIT_SKIP_FILES: frozenset[str] = GIT_IGNORED_CONFIG_FILES | {TELEMETRY_CONFIG_FILE_NAME, ".DS_Store"}
INIT_SKIP_DIRS: frozenset[str] = frozenset({"inference"})

The recursive copy_directory_structure function checks both sets before processing each entry:

if item in INIT_SKIP_FILES:
    continue
if os.path.isdir(src_item):
    if item in INIT_SKIP_DIRS:
        continue
    # recurse...

Step 2: Inference Step

When needs_inference is True and reset is True, the inference step copies its own template files independently:

1. backends.toml — main backend registry
2. backends/*.toml — per-backend configuration files
3. deck/*.toml — model deck configurations
4. routing_profiles.toml — routing profile definitions

A reset therefore puts customised backends and routing back to the kit's defaults, which the confirmation panel says. Then, on the hosted path, it writes execution = "hosted" and offers the IDE extension, customizing nothing, so the defaults stay until pipelex init inference. On the local path, or when the run does not decide where runs execute (focus=inference), it runs interactive customization:

1. customize_backends_config() — prompts user to select backends and suggests IDE extension installation via suggest_extension_install_if_needed()
2. customize_routing_profile() — auto-configures routing based on selected backends (only when check_routing is False, i.e. when routing is not the specific focus)
3. On the local path, execution = "local" is written

When focus=routing, the inference step skips routing entirely because Step 3 handles it as a standalone operation.

Step 2.5: Credentials Step

When check_credentials is True (for focus in all, config, inference) and runs do not execute on the hosted Pipelex API, prompt_credentials() checks which backends are enabled and prompts the user for any missing API keys. Credentials are stored in ~/.pipelex/.env with mode 0600.

Step 3: Routing Step

If needs_routing is True (only for focus=routing), runs customize_routing_profile() as a standalone step.

Step 4: Telemetry Step

Copies a telemetry template and prints instructions. No interactive prompts. Which template is copied depends on the target:

• Global init (target is ~/.pipelex/): copies pipelex/kit/configs/telemetry.toml — the active template with disabled-by-default settings.
• Project init (target is {project_root}/.pipelex/): copies pipelex/kit/configs/telemetry.project.toml — every setting commented out so the project file does not shadow ~/.pipelex/telemetry.toml or ~/.pipelex/telemetry_override.toml during layered loading.

setup_telemetry() selects the template via a for_project: bool arg passed down from the --local flag on pipelex init.

Step 5: Pipelex API Key

On the hosted path only, ensure_pipelex_api_key() runs last, so every file is written before a sign-in that can take minutes. A login that saves nothing leaves the setup as written and names pipelex login.

Scenario Matrix

File Reference

Template Directory (kit/configs/)

Skip Lists

Source Modules

Config Directory Resolution

The ConfigLoader (singleton: config_manager) resolves where configuration files live. Understanding this logic is essential because bugs in directory resolution cause files to be written or read from the wrong location.

Source: pipelex/system/configuration/config_loader.py

Three Directory Properties

Project Root Detection

find_project_root() walks up from cwd looking for marker files:

• .pipelex, .git, pyproject.toml, setup.py, setup.cfg, package.json, .hg

A directory containing only a .pipelex/ directory is therefore itself a valid project root.

The home directory (~) is explicitly excluded — even if it contains a package.json, it is never treated as a project root.

File Resolution Order

resolve_config_file(relative_path, config_dir) uses layered resolution:

flowchart TD
    START([resolve_config_file]) --> EXPLICIT{config_dir<br/>provided?}
    EXPLICIT -- Yes --> USE_EXPLICIT["Return config_dir / relative_path"]
    EXPLICIT -- No --> PROJECT{project_config_dir<br/>exists?}
    PROJECT -- Yes --> FILE_EXISTS{file exists in<br/>project dir?}
    FILE_EXISTS -- Yes --> USE_PROJECT["Return project_dir / relative_path"]
    FILE_EXISTS -- No --> USE_GLOBAL["Return global_dir / relative_path"]
    PROJECT -- No --> USE_GLOBAL

This means a project-level file wins over the global one, but only if it actually exists on disk.

Global Config Bootstrap

ensure_global_config_exists(), called automatically during load_config(), lays the kit's template files into the home configuration directory (~/.pipelex/, or PIPELEX_HOME) wherever they are missing. It creates the directory when it does not exist and copies each kit file the directory lacks, never overwriting one it holds, so a home holding only a retired pipelex_service.toml, a .env or a personal override boots like a fresh one, and a home holding every kit file is not written at all. A file counts as there when anything stands at its path, a symbolic link included even when its target is gone, and nothing is ever written through a link: where the kit has a directory, a link standing in its place, valid or dangling, or a regular file, is left alone with everything under it, the manifests included. Each file is written under a temporary name beside its destination and renamed into place, so it appears whole or not at all and a copy cut off mid-write leaves no truncated file for later boots to keep. It skips all GIT_IGNORED_CONFIG_FILES (files like pipelex_override.toml, and .DS_Store), which are never part of the bootstrap copy.

The inference/ directory is one unit, as the inference step of pipelex init treats it. It is filled only when inference/backends.toml is absent, the same signal pipelex init reads as a first-time backend setup, and the deck and backends kit manifests are then stamped for each area that has none; the kit's own .kit_manifest.json files are never copied. Because backends.toml is that signal, it is written last, after every other inference file and the manifests: a fill cut short part-way, by a full disk or a killed process, leaves it absent, and the next boot completes the fill instead of taking a half-copied directory for a complete one. A home with a backends.toml of its own keeps its inference directory exactly as it is, because kit routing profiles, deck or backend files copied beside it could route to, or alias the models of, backends it disables.

A home the process cannot write to, such as a read-only mount or a directory another user owns, is read as it is, and the boot then reports whatever configuration is missing. Any other error that stops a copy is raised.

The copy is copy_kit_templates() in pipelex/kit/template_copy.py, the same walk init_config() copies the configuration files with, so the bootstrap and pipelex init config agree on which files a directory receives. Both write each file with copy_file_atomically(), which is also how the bootstrap writes backends.toml, and the kit manifests are written the same way, through write_text_atomically() in the same module, so a manifest cut short leaves no truncated .kit_manifest.json for the next boot to keep. The two differ on what already stands at a destination. The bootstrap copies without overwriting, so it keeps every file there and never follows a link. init_config() copies with overwrite=reset, and every pipelex init or pipelex init config run passes reset=True, as pipelex-agent init does, because configuration updates are not supported and there is no flag that keeps the existing files; only the dry run that counts missing files for pipelex init and pipelex doctor passes reset=False, and it writes nothing. Overwriting follows a link as a plain copy does, so the template lands at a linked file's target, written beside it and renamed into place, and the link survives. A configuration file linked from a dotfiles repository, as chezmoi, stow or yadm lay them out, therefore stays linked through a reset.

Config Loading Chain

load_config() merges configuration in this order (later values override earlier ones):

1. Package defaults (pipelex/pipelex.toml)
2. Global config (~/.pipelex/pipelex.toml)
3. Project config ({project_root}/.pipelex/pipelex.toml, if different from global)
4. Override files from the effective config dir (local, environment, run mode, override)

Agent CLI Init

pipelex-agent init is the non-interactive counterpart to pipelex init. It runs the same steps but takes all inputs from a JSON config argument instead of interactive prompts, and never signs in: a hosted setup reads its key from PIPELEX_API_KEY.

Source: pipelex/cli/agent_cli/commands/init_cmd.py

Target Directory Resolution

_resolve_target_dir(global_) determines where files are written:

If no project root is found and --global is not set, the command fails with an error.

Config Input

--config / -c accepts a JSON string (inline or file path) with these optional fields:

Flow

flowchart TD
    START([pipelex-agent init]) --> PARSE["Parse --config JSON"]
    PARSE --> RESOLVE["Resolve target dir<br/>(project or --global)"]
    RESOLVE --> STEP1["Step 1: init_config()<br/>Copy config templates (skips inference/)"]
    STEP1 --> STEP15["Step 1.5: Copy inference templates<br/>(backends.toml, backends/*, deck/*, routing_profiles.toml)"]
    STEP15 --> STEP16["Step 1.6: Copy telemetry template<br/>(global: active defaults; project: commented-out)"]
    STEP16 --> STEP17["Step 1.7: Write [run] execution"]
    STEP17 --> EXECUTION{execution?}
    EXECUTION -- hosted --> HOSTED_OUT["Report execution and api_key_set"]
    EXECUTION -- "local (or absent)" --> STEP2["Step 2: Configure backends<br/>Enable requested backends in backends.toml"]
    STEP2 --> STEP3["Step 3: Configure routing<br/>Auto-derive routing profile<br/>(the template's, when the config names neither field)"]
    STEP3 --> OUTPUT["Output result<br/>(Markdown, or JSON via --format json)"]
    HOSTED_OUT --> OUTPUT

Key Differences from Interactive Init

Doctor Fix: Config Dir Handling

pipelex doctor and pipelex doctor --fix use the config directory resolution to find and fix configuration issues wherever they live.

Where Runs Execute

Once the configuration files are healthy, resolve_doctor_run_execution() reads the effective [run] execution, local when the configuration does not load. A hosted run boots nothing on this machine, so under "hosted" the backend credentials and models rows, which load this machine's backends, are shown as information, needed only for --local runs: they count toward neither the overall status nor the exit code, and no provider key is asked for. A Pipelex API Key row counts instead: check_pipelex_api_key_set() finds the key where a run reads it, PIPELEX_API_KEY and then the home .env, healthy when it is set and starts with plx_sk_, otherwise naming pipelex login. It makes no network call and never prints the key. pipelex-agent doctor reports the same: execution at the top level, checks.pipelex_api_key with a finding of set, not_a_pipelex_key or missing, and informational: true on the two rows it does not count.

Source: pipelex/cli/commands/doctor_cmd.py

How Each Check Resolves Files

check_pending_migrations() is the one check that takes no config_dir at all, and that is deliberate. It also reports the command's first step, the cleanup of what a former release left: former_release_files names the files the cleanup would rewrite or remove, and former_release_blocks_boot says whether what they hold stops this machine's boot, read by former_release_boot_blockers() over config_manager.backends_file_paths() and routing_profiles_file_paths() inside the row's own error handling, so a failure reading them leaves the row unchecked rather than escaping. The dry run of the replay leaves out the files the cleanup removes, as pipelex migrate does. While what a former release left stops the boot, check_models() does not load anything: the boot would stop on that before any backend, so the row says so and names pipelex migrate rather than quoting whichever refusal about one backend or one profile it would meet first. check_backend_credentials() skips a retired backend's table, since its key is no remedy. Every other row reports on a file and is scoped to the directory the doctor was pointed at, --global included. That one reports on a command — it is pipelex migrate's own dry run — and pipelex migrate has no --global: it walks the global ~/.pipelex/ and the project .pipelex/ both. Scoping the row narrower would name a command that then rewrites a file the row never mentioned. Every file it reports is named with its full path, so the wider scope stays legible.

check_backend_files() loads the backend library once per enabled backend, and loads it without resolving credentials (CredentialResolution.SKIP, the keyless boot's mode). The row reports on file shape — an unknown key, a spec that is not a table, a missing per-backend file — so a backend's credentials are the Credentials row's finding, not this one's. Resolving them here would report an enabled backend whose key is not set yet as a backend-configuration error, and because the loader stops at the first backend it cannot load, a malformed file listed after that backend in backends.toml would go unreported. Skipping credentials skips nothing else: every enabled backend is loaded, a malformed file stays fatal, and so do the refusals of an enabled backend that declares no model or still carries model_specs_section. Every probe sees that same first failure, so a failure is charged to the backend the error declares (backend_name, stamped by the loader on every error about one backend) or, for an error that declares none, to the backend whose file it names — never to a backend whose name merely appears in the message's prose. check_models() attributes the same way and for the same reason: its own load resolves credentials, which the probe's does not, so it can be the first thing to name a broken backend, and it writes into the very reports this row produced.

Fix Targeting

When --fix replaces an outdated backend file, it derives the config directory from the resolved file path:

# File lives at {config_dir}/inference/backends/{name}.toml
# Walk up 3 levels to get the config_dir
resolved_config_dir = Path(backend_file_report.file_path).parent.parent.parent
replace_backend_file(backend_name, config_dir=resolved_config_dir)

This ensures the fix targets the same directory where the issue was found — if the broken file was in the project .pipelex/, the replacement goes there too, not to ~/.pipelex/.

The migration fix is the exception that follows from the row above it: it calls apply_former_release_cleanup() first when the row lists files a former release left, then apply_pending_migrations(config_dirs=config_directories_to_migrate(), skipped_paths=...) with the files the cleanup removed left out, and so writes to both directories, because that is what the command it is running does. When the cleanup leaves something for a person — a file it could not clean, or a boot its check after writing finds still refused — the fix says so rather than printing a plain success. It calls that helper rather than migrate_cmd itself — the command exits the process when a run leaves something for a person, which would cut off the doctor's remaining fixes, its remaining rows and its own exit code.

Next Steps

• Configuration Internals
• Architecture Overview