Skip to content

Under the Hood

Welcome to the technical deep-dives of Pipelex. This section is for contributors, curious developers, AI agents, and anyone doing due diligence on how Pipelex works internally.

What You'll Find Here

  • Architecture Overview - The two-layer design and how components fit together
  • Build-time Elaboration - How shorthand directives like structuring_method = "preliminary_text" are rewritten into concrete pipe trees before any pipe runs
  • Execution Graph Tracing - How pipeline executions are captured as graphs for visualization
  • Image Handling in LLM Prompts - How images flow from inputs to LLM calls
  • Reasoning Controls - How reasoning effort/budget flows to each provider's SDK
  • Error Model - How errors are classified, carried across every layer, and reported to humans, agents, and HTTP APIs
  • StuffArtefact & Image Rendering - How template access and image extraction work
  • Test Profile Configuration - How to configure which models are used in tests
  • Dry Run Mock Generation - How mock objects satisfy field validation constraints
  • Init CLI Flows - How pipelex init sets up the configuration directory
  • Runtime Bridge & Transport - The backend-neutral seam that carries a pipe run across process boundaries — boundary DTOs, LibraryCrate propagation, deferred hydration, and per-call isolation
  • Content Generation Across Boundaries - How dynamic classes and large payloads cross a worker boundary
  • Technical Design Decisions - Why we chose X over Y
  • Module Deep-Dives - Detailed explanations of specific subsystems

Not Required for Using Pipelex

You don't need to read this section to use Pipelex effectively. The Home section covers everything you need to build methods.

Looking for distributed execution?

For running Pipelex methods as durable, distributed workflows, see the user-facing Distributed Execution overview. The pages below cover the runtime mechanics that make distributed execution work under the hood.

Start Exploring