Skip to content

Under the Hood

Welcome to the technical deep-dives of Pipelex. This section is for contributors, curious developers, AI agents, and anyone doing due diligence on how Pipelex works internally.

What You'll Find Here

  • Architecture Overview - The two-layer design and how components fit together
  • Build-time Elaboration - How shorthand directives like structuring_method = "preliminary_text" are rewritten into concrete pipe trees before any pipe runs
  • Execution Graph Tracing - How pipeline executions are captured as graphs for visualization
  • Image Handling in LLM Prompts - How images flow from inputs to LLM calls
  • Reasoning Controls - How reasoning effort/budget flows to each provider's SDK
  • Error Model - How errors are classified, carried across every layer, and reported to humans, agents, and HTTP APIs
  • StuffArtefact & Image Rendering - How template access and image extraction work
  • Test Profile Configuration - How to configure which models are used in tests
  • Dry Run Mock Generation - How mock objects satisfy field validation constraints
  • Init CLI Flows - How pipelex init sets up the configuration directory
  • Distributed Content Generation - How dynamic classes and large payloads cross Temporal boundaries
  • Technical Design Decisions - Why we chose X over Y
  • Module Deep-Dives - Detailed explanations of specific subsystems

Not Required for Using Pipelex

You don't need to read this section to use Pipelex effectively. The Home section covers everything you need to build methods.

Looking for Temporal deployment and operations?

For running Pipelex on Temporal — cluster setup, workers, task-queue routing, and dashboard observability — see the user-facing Distributed Execution with Temporal guide. The pages below cover the runtime mechanics that make distributed execution work under the hood.

Start Exploring