PipeCompose

The PipeCompose operator composes data from your pipeline's working memory into new outputs. It supports two modes:

1. Template mode: Render templates to produce Text-like output
2. Construct mode: Build structured objects by mapping fields from inputs

In both modes, PipeCompose produces a single output stuff: the output concept cannot carry multiplicity (output = "Report[]" is rejected at validation).

Template Mode

Template mode uses Jinja2 templates to dynamically generate text by combining data from working memory. This is ideal for creating formatted reports, HTML content, or constructing complex, multi-part prompts for LLMs.

How Template Mode Works

PipeCompose takes all the data currently in the WorkingMemory and uses it as the context for rendering a Jinja2 template. The resulting text is then saved back to the working memory as a new Text or Html output. In template mode, the output concept must refine the native Text or Html concept — use construct mode for structured outputs.

Template mode supports two syntax variants:

• Simple: template = "Hello $name" — an inline string, defaults to basic category
• Rich: A [pipe.name.template] section with template, category, and optionally templating_style

Template Context

The Jinja2 template has access to all the "stuffs" currently in the working memory. You can access them by the names they were given in previous pipeline steps. For example, if a previous step produced an output named user_profile, you can access its attributes in the template like {{ user_profile.name }} or {{ user_profile.email }}.

Template Mode Configuration

*Template mode requires template. When using the rich form ([pipe.name.template] section), the following sub-fields are available:

Template Mode Examples

Simple inline template:

[pipe.compose_greeting]
type = "PipeCompose"
description = "Compose a greeting message"
inputs = { user = "User" }
output = "Text"
template = "Hello $user.name, welcome to our platform!"

Markdown template with category:

[pipe.compose_report]
type = "PipeCompose"
description = "Format data as a markdown report"
inputs = { summary = "Text", items = "Item[]" }
output = "Text"

[pipe.compose_report.template]
category = "markdown"
template = """
# Report

$summary

{% for item in items %}
- {{ item.name }}: {{ item.value }}
{% endfor %}
"""

HTML template with templating style:

[pipe.format_html_report]
type = "PipeCompose"
description = "Format data as HTML"
inputs = { summary = "Text", items = "Item[]" }
output = "Html"

[pipe.format_html_report.template]
category = "html"
templating_style = { tag_style = "xml", text_format = "html" }
template = """
<h1>Report</h1>
<p>{{ summary }}</p>
<ul>
{% for item in items %}
  <li>{{ item.name }}: {{ item.value }}</li>
{% endfor %}
</ul>
"""

Construct Mode

Construct mode builds structured objects by mapping fields from inputs. Use this when you need to assemble a complex output concept from multiple inputs without using an LLM.

How Construct Mode Works

Instead of rendering a template, construct mode creates a structured object by specifying how each field should be populated. Fields can be:

• Referenced from inputs: Copy a value from working memory
• Templated: Generate a string using template interpolation
• Fixed: Use a static value

Construct Mode Configuration

*Either template or construct must be provided, but not both.

Construct Field Methods

Each field in the [pipe.name.construct] section can use one of these methods:

A nested construct is written as a table whose keys are the sub-object's own field names — each sub-field uses any of the four methods, recursively:

[pipe.assemble_invoice.construct]
customer = { name = { from = "order.customer_name" }, tier = "standard" }

A from reference also accepts one modifier, list_to_dict_keyed_by: when the target field is a dict, it converts the referenced list into a dict keyed by the named attribute of each item:

[pipe.index_products.construct]
products_by_sku = { from = "products", list_to_dict_keyed_by = "sku" }

The referenced value must be a list, and every item must carry the key attribute with a string value — otherwise the composer raises an error.

Copying Whole Inputs Into Native Fields

The from reference is not limited to dotted paths like "customer.name" — it can name a whole input variable. When the referenced input is a native stuff (Text, Number, YesNo, Date, or a list of them) and the target field is native-typed, the composer automatically converts the content wrapper into the field's native value. This works for required and optional fields alike.

Conversion matrix:

When the target field expects a content object rather than a native value (e.g. a field typed with a concept), the object is kept as-is — the conversion only fires when the field expects the native type.

One fidelity guard: a Date stuff that carries a time of day cannot be copied into a bare date field — that would silently drop the time and its UTC offset, so the composer raises an error instead. Target a Date-typed field to keep the full timestamp. The same guard applies per item when copying a Date[] into a list of date items.

Worked example — assembling a report from whole stuffs produced by earlier steps:

[concept.ScreeningReport]
description = "The final screening report"

[concept.ScreeningReport.structure]
match_score = { type = "number", description = "The match score", required = true }
rejection_email = { type = "text", description = "The rejection email — optional" }
interview_questions = { type = "list", item_type = "text", description = "Questions to ask — optional" }

[pipe.assemble_report]
type = "PipeCompose"
description = "Assemble the screening report from previously generated pieces"
inputs = { score = "Number", email = "Text", questions = "Text[]" }
output = "ScreeningReport"

[pipe.assemble_report.construct]
match_score = { from = "score" }
rejection_email = { from = "email" }
interview_questions = { from = "questions" }

Here email is a whole Text stuff copied into an optional text field, questions is a whole Text[] stuff copied into an optional list of text items, and score is a whole Number stuff copied into a required number field. Each lands as its native value (str, list[str], float).

Construct Mode Example

[pipe.assemble_invoice]
type = "PipeCompose"
description = "Assemble invoice from order and customer data"
inputs = { order = "Order", customer = "Customer" }
output = "Invoice"

[pipe.assemble_invoice.construct]
invoice_number = { template = "INV-$order.id" }
customer_name = { from = "customer.name" }
customer_email = { from = "customer.email" }
line_items = { from = "order.items" }
total = { from = "order.total" }
status = "pending"
version = 1

In this example:

• invoice_number is generated from a template using the order ID
• customer_name, customer_email, line_items, and total are copied from inputs
• status and version are fixed values

Choosing Between Modes

Related Documentation

• Working Memory - How pipes access and share data
• Provide Inputs - Passing inputs to pipes and pipelines