PipeCompose

The PipeCompose operator composes data from your pipeline's working memory into new outputs. It supports two modes:

1. Template mode: Render templates to produce Text-like output
2. Construct mode: Build structured objects by mapping fields from inputs

In both modes, PipeCompose produces a single output stuff: the output concept cannot carry multiplicity (output = "Report[]" is rejected at validation).

Template Mode

Template mode uses Jinja2 templates to dynamically generate text by combining data from working memory. This is ideal for creating formatted reports, HTML content, or constructing complex, multi-part prompts for LLMs.

How Template Mode Works

PipeCompose takes all the data currently in the WorkingMemory and uses it as the context for rendering a Jinja2 template. The resulting text is then saved back to the working memory as a new Text or Html output. In template mode, the output concept must refine the native Text or Html concept — use construct mode for structured outputs.

Template mode supports two syntax variants:

• Simple: template = "Hello $name" — an inline string, defaults to basic category
• Rich: A [pipe.name.template] section with template, category, and optionally templating_style

Template Context

The Jinja2 template has access to all the "stuffs" currently in the working memory. You can access them by the names they were given in previous pipeline steps. For example, if a previous step produced an output named user_profile, you can access its attributes in the template like {{ user_profile.name }} or {{ user_profile.email }}.

A template reads data and calls methods of plain values, and nothing else: it can read fields, call isoformat() on a date or upper() on a string, and apply any filter, but it cannot call methods of Pipelex objects or read names starting with an underscore, other than an input's metadata fields such as _stuff_name. This applies to every template a method contains, PipeLLM prompts and PipeCondition expressions included. The Template Sandbox page lists what is allowed and how a refusal is reported.

Template Mode Configuration

Every input the pipe declares must be read by the template, and every variable the template reads must be declared in inputs. Validation refuses an input the template never reads as extraneous_input_variable, naming the input, so you either reference it in the template or remove it from inputs; an undeclared variable is refused as missing_input_variable. A name the template sets with {% set %} is its own, not an input, from that statement on: after an {% if %} it stays set only when every branch sets it, the {% else %} included, so a name only some branches set is read from the input of that name on the other paths; and a loop, a macro or a block keeps what it sets to its own body.

*Template mode requires template. When using the rich form ([pipe.name.template] section), the following sub-fields are available:

Template Mode Examples

Simple inline template:

[pipe.compose_greeting]
type = "PipeCompose"
description = "Compose a greeting message"
inputs = { user = "User" }
output = "Text"
template = "Hello $user.name, welcome to our platform!"

Markdown template with category:

[pipe.compose_report]
type = "PipeCompose"
description = "Format data as a markdown report"
inputs = { summary = "Text", items = "Item[]" }
output = "Text"

[pipe.compose_report.template]
category = "markdown"
template = """
# Report

$summary

{% for item in items %}
- {{ item.name }}: {{ item.value }}
{% endfor %}
"""

HTML template with templating style:

[pipe.format_html_report]
type = "PipeCompose"
description = "Format data as HTML"
inputs = { summary = "Text", items = "Item[]" }
output = "Html"

[pipe.format_html_report.template]
category = "html"
templating_style = { tag_style = "xml", text_format = "html" }
template = """
<h1>Report</h1>
<p>{{ summary }}</p>
<ul>
{% for item in items %}
  <li>{{ item.name }}: {{ item.value }}</li>
{% endfor %}
</ul>
"""

The markdown filter. An HTML template can turn Markdown held in a plain text field into HTML with {{ order.notes | markdown }}. Raw HTML inside the Markdown is shown as text rather than passed through. A bare URL becomes a link only when it has a scheme, such as https://example.com, so a file name like README.md stays text; a link written in Markdown, such as [the guide](docs/guide.md), is a link whatever its target. A Markdown input needs no filter: it renders as HTML by itself in an HTML template. Converting Markdown is charged to the template's render budget, so one render converts at most about 65,000 characters of it, each conversion counting.

Construct Mode

Construct mode builds structured objects by mapping fields from inputs. Use this when you need to assemble a complex output concept from multiple inputs without using an LLM.

How Construct Mode Works

Instead of rendering a template, construct mode creates a structured object by specifying how each field should be populated. Fields can be:

• Referenced from inputs: Copy a value from working memory
• Templated: Generate a string using template interpolation
• Fixed: Use a static value

Construct Mode Configuration

*Either template or construct must be provided, but not both.

The input rule of template mode applies to a construct too: what reads the inputs is every from path and every field template, nested constructs included. A declared input none of them reads is refused as extraneous_input_variable, and a path whose input is not declared as missing_input_variable.

Construct Field Methods

Each field in the [pipe.name.construct] section can use one of these methods:

A nested construct is written as a table whose keys are the sub-object's own field names — each sub-field uses any of the four methods, recursively:

[pipe.assemble_invoice.construct]
customer = { name = { from = "order.customer_name" }, tier = "standard" }

A from reference also accepts one modifier, list_to_dict_keyed_by: when the target field is a dict, it converts the referenced list into a dict keyed by the named attribute of each item:

[pipe.index_products.construct]
products_by_sku = { from = "products", list_to_dict_keyed_by = "sku" }

The referenced value must be a list, and every item must carry the key attribute with a string value — otherwise the composer raises an error.

A from path and a list_to_dict_keyed_by name read public fields only: validation refuses a segment starting with an underscore, such as { from = "order._stuff" }, for the reason the Template Sandbox gives.

A from path reaching a field that holds nothing, such as an optional order.note left unset, gives the target field nothing. A target field with a default_value is left unset and takes its default, since it never holds nothing; an optional one holds nothing; and a required one is refused. A Python class field typed X | None holds nothing whether it is required or defaulted. With list_to_dict_keyed_by, a path holding nothing is refused, since the modifier needs a list.

Copying Whole Inputs Into Native Fields

The from reference is not limited to dotted paths like "customer.name" — it can name a whole input variable. When the referenced input is a native stuff (Text, Number, YesNo, Date, Time, or a list of them) and the target field is native-typed, the composer automatically converts the content wrapper into the field's native value. This works for required and optional fields alike.

Conversion matrix:

When the target field expects a content object rather than a native value (e.g. a field typed with a concept), the object is kept as-is — the conversion only fires when the field expects the native type.

Choice and Rating are not in the matrix: reach their verdict by path, { from = "team.choice" } or { from = "severity.level" }. A YesNo converts to its boolean and leaves its probability behind, so copy { from = "approved.probability" } into a number field when you need it.

One fidelity guard: a Date stuff that carries a time of day cannot be copied into a bare date field — that would silently drop the time and its UTC offset, so the composer raises an error instead. Target a Date-typed field to keep the full timestamp. The same guard applies per item when copying a Date[] into a list of date items.

Time needs no such guard, and the asymmetry is deliberate: a Time is a single value that carries its UTC offset inside the time itself, so copying it into a time field is lossless and nothing can be dropped. In the other direction, a whole Date does not convert into a time field at all — that would drop the date. Use a dotted path instead, { from = "deadline.time" }, to take the time of day out of a Date.

Worked example — assembling a report from whole stuffs produced by earlier steps:

[concept.ScreeningReport]
description = "The final screening report"

[concept.ScreeningReport.structure]
match_score = { type = "number", description = "The match score", required = true }
rejection_email = { type = "text", description = "The rejection email — optional" }
interview_questions = { type = "list", item_type = "text", description = "Questions to ask — optional" }

[pipe.assemble_report]
type = "PipeCompose"
description = "Assemble the screening report from previously generated pieces"
inputs = { score = "Number", email = "Text", questions = "Text[]" }
output = "ScreeningReport"

[pipe.assemble_report.construct]
match_score = { from = "score" }
rejection_email = { from = "email" }
interview_questions = { from = "questions" }

Here email is a whole Text stuff copied into an optional text field, questions is a whole Text[] stuff copied into an optional list of text items, and score is a whole Number stuff copied into a required number field. Each lands as its native value (str, list[str], float).

Construct Mode Example

[pipe.assemble_invoice]
type = "PipeCompose"
description = "Assemble invoice from order and customer data"
inputs = { order = "Order", customer = "Customer" }
output = "Invoice"

[pipe.assemble_invoice.construct]
invoice_number = { template = "INV-$order.id" }
customer_name = { from = "customer.name" }
customer_email = { from = "customer.email" }
line_items = { from = "order.items" }
total = { from = "order.total" }
status = "pending"
version = 1

In this example:

• invoice_number is generated from a template using the order ID
• customer_name, customer_email, line_items, and total are copied from inputs
• status and version are fixed values

Choosing Between Modes

Related Documentation

• Working Memory - How pipes access and share data
• Provide Inputs - Passing inputs to pipes and pipelines
• Templating Style - Tag styles and text formats, and the runtime default