Subagent Prompt: Generate User Inputs for One Evaluation Tuple

You are generating candidate user inputs for an AI application evaluation dataset.

You are given:

1. the product specification;
2. the approved user-query dimension set;
3. one approved target tuple.

Your task is to generate 20 realistic and materially distinct user-input examples that faithfully instantiate that tuple.

Do not redesign the tuple.

Do not expand evaluation coverage.

Do not generate examples for neighboring tuples.

Do not generate ideal assistant responses.

Inputs

Product specification#

<PRODUCT_SPEC>

{{PRODUCT_SPEC}}

</PRODUCT_SPEC>

Dimension set#

<DIMENSION_SET>

{{DIMENSION_SET}}

</DIMENSION_SET>

Target tuple#

<TARGET_TUPLE>

{{TARGET_TUPLE}}

</TARGET_TUPLE>

Optional generation context#

<GENERATION_CONTEXT>

{{GENERATION_CONTEXT}}

</GENERATION_CONTEXT>

Generation context may contain approved domain vocabulary, example entity names, locale information, or other constraints.

If it is absent, do not invent product capabilities or hidden system state.

1. Objective

Generate 20 examples of how a real user could express the interaction described by the target tuple.

Each example must:

• belong to the product’s supported interaction space;
• match every applicable value in the target tuple;
• preserve the tuple’s intended ambiguity, incompleteness, correction, contextual dependence, multiplicity, or other difficulty;
• sound like something a real user could plausibly submit;
• differ meaningfully in wording and surface realization from the other examples.

The target tuple defines the semantic and behavioral constraints.

Your job is to vary the natural-language realization without changing those constraints.

2. Treat the tuple as authoritative

Read the definitions of every dimension used in the target tuple.

For each example, verify that the generated input satisfies every assigned value.

Do not silently reinterpret dimension values.

Do not add difficulty that belongs to another tuple.

For example:

• if determinacy is clear, do not introduce accidental ambiguity;
• if multiplicity is single, do not add a second request;
• if context dependence is self_contained, the message must make sense without prior conversation;
• if reference surface is named_phrase, do not introduce typos merely for variety;
• if information coverage is partial, do not accidentally make the transaction fully specified;
• if the tuple represents resolved_correction, make the user’s final intended value recoverable;
• if the tuple represents contradictory, do not resolve the contradiction;
• if the tuple represents intent_ambiguous, preserve genuine ambiguity rather than making one interpretation obviously dominant.

Tuple fidelity is more important than linguistic diversity.

3. Generate user inputs, not test descriptions

Write what the user would actually say.

Do not write:

• explanations of the tuple;
• test-case descriptions;
• expected assistant behavior;
• evaluator instructions;
• labels such as “ambiguous request”;
• statements describing why the case is difficult.

Bad:

User gives an ambiguous account reference and should be asked to clarify.

Good:

Move 500 from Main or Savings to Cash.

The examples must stand on their own as realistic user contributions.

4. Preserve realistic user behavior

Users do not normally write optimized prompts.

Allow natural behavior such as:

• conversational phrasing;
• shorthand;
• omission of details;
• terse input;
• indirect descriptions;
• ordinary financial language;
• self-correction;
• references to earlier context;
• minor grammatical imperfections;
• varied sentence structure.

Only use these behaviors when they remain consistent with the target tuple.

Do not mechanically add spelling mistakes, slang, or omissions to every example.

5. Produce meaningful diversity

The 20 examples should represent different natural realizations of the same tuple, rather than 20 paraphrases produced from one sentence template.

Vary, where compatible with the tuple:

• sentence structure;
• vocabulary;
• degree of terseness;
• order in which information is mentioned;
• amount values;
• dates and temporal wording;
• currency notation;
• merchant, reason, or transaction description;
• plausible account/category names;
• punctuation and casing;
• ordinary domain phrasing.

These are generation variables, not new dimensions.

Do not vary them in ways that change the tuple classification.

For example, if the tuple specifies event_or_action, all examples must remain event/action formulations even though their wording may differ.

6. Avoid template families

Do not produce sets like:

I spent 20 on lunch from Cash.
I spent 30 on dinner from Cash.
I spent 40 on groceries from Cash.
I spent 50 on coffee from Cash.

Changing only nouns and numbers is insufficient diversity.

Prefer structural differences such as:

Paid 20 for lunch out of Cash.

Lunch was 20, used my Cash wallet.

Took 20 from Cash for lunch earlier.

20 on lunch from Cash.

provided all examples still satisfy the tuple.

7. Standalone versus conversational inputs

A user input may be either:

• a single standalone user message; or
• a logical sequence of user contributions.

Determine the required form from the tuple and any sequence_setup.

Standalone interaction#

Use a single message when the target tuple is self-contained and does not require preceding conversational context.

Example structure:

{
  "interaction_form": "standalone",
  "user_messages": [
    "..."
  ],
  "focal_message_index": 0
}

Sequence-required interaction#

If the tuple depends on previous conversation–for example continuation, correction, repetition, switching, contextual reference, or reference-only input–generate the minimum logical sequence necessary to make the focal contribution meaningful.

The final relevant user contribution should instantiate the target tuple.

Earlier contributions exist only to establish required context.

Example:

{
  "interaction_form": "sequence",
  "user_messages": [
    "I spent 4,500 on groceries from Cash.",
    "Actually make that 5,400."
  ],
  "focal_message_index": 1
}

Here the second message is the focal contribution.

Do not make the setup unnecessarily long.

8. Conversational sequence rules

For sequence examples:

• preserve one coherent conversational thread;
• include only as much prior user context as necessary;
• keep the focal turn clearly identifiable;
• ensure earlier user contributions establish whatever the focal turn refers to;
• do not create unrelated earlier requests;
• do not accidentally change the target tuple through the setup.

Do not generate assistant responses.

Assume normal assistant turns may occur between user contributions.

If the focal user message only makes sense after a specific assistant question or action, and cannot be represented robustly as a fixed user-message sequence, mark the example as:

adaptive_sequence_candidate

and describe the required conversational condition briefly.

Example:

{
  "interaction_form": "adaptive_sequence_candidate",
  "user_messages": [
    "I spent 4,500."
  ],
  "focal_message_index": null,
  "required_condition": "After the assistant asks which account was used, the user responds only with the account name.",
  "focal_user_behavior": "Provide only the requested account reference."
}

Use this form only when necessary.

Do not invent the assistant’s exact wording.

9. Focal-turn semantics

For a sequence, classify tuple properties primarily against the focal user contribution, using the preceding messages only where the dimension definition requires conversational context.

Example:

Prior user contribution:
"I spent 20 from Cash."

Focal contribution:
"Actually make that 30."

The focal contribution may correctly be:

intent_expression: fragment
information_coverage: fragment
interpretive_determinacy: resolved_correction
context_dependence: context_linked
goal_relation: continue

Do not aggregate the whole conversation into a new tuple.

10. Product grounding

Use the product specification as the source of truth.

Do not introduce:

• unsupported features;
• unsupported transaction or task types;
• capabilities absent from the product;
• product semantics not stated or reasonably implied by the supplied materials.

If a tuple cannot be faithfully instantiated because it conflicts with the product specification or its dimension values are mutually inconsistent, do not fabricate examples.

Return:

{
  "status": "cannot_instantiate",
  "reason": "..."
}

Explain the concrete conflict concisely.

11. Do not introduce hidden fixtures

Generate the user input only.

Do not assert hidden facts such as:

• whether an account actually exists;
• whether a reference resolves;
• actual balances;
• permissions;
• backend state;
• account compatibility;
• system availability;

unless those facts are explicitly part of supplied generation context and visible to the user.

For example, a user may write:

Move 500 from Main to Cash.

But do not assume in the generated example that Main exists or that the transfer is executable unless supplied context establishes that.

Those conditions belong to later evaluation-case construction.

12. Internal generation and review process

Before returning the final 20 examples:

1. Generate a larger candidate pool internally.
2. Validate every candidate against every applicable target dimension.
3. Reject candidates that drift into another tuple.
4. Reject unrealistic or evaluator-like language.
5. Compare candidates for semantic and structural similarity.
6. Remove duplicates and near-duplicates.
7. Prefer examples that increase natural linguistic variety while preserving identical tuple semantics.
8. Select the strongest 20.

Do not expose this internal reasoning.

13. Final fidelity check

Before accepting each example, verify:

Product fit#

Is this something a real user could plausibly ask this product?

Tuple fit#

Does every applicable target dimension still have the intended value?

No accidental extra behavior#

Did the wording introduce:

• another intent;
• extra ambiguity;
• additional missing information;
• a correction;
• a typo;
• another request;
• conversational dependence;

when the tuple did not ask for it?

Sequence coherence#

If conversational, does the setup make the focal contribution meaningful?

Distinctiveness#

Is this materially different in wording or structure from the other selected examples?

Reject or repair any example that fails a check.

14. Output format

Return JSON only.

For a successfully instantiated tuple:

{
  "status": "ok",
  "tuple_id": "<target tuple id>",
  "examples": [
    {
      "id": "E01",
      "interaction_form": "standalone",
      "user_messages": [
        "..."
      ],
      "focal_message_index": 0
    },
    {
      "id": "E02",
      "interaction_form": "sequence",
      "user_messages": [
        "...",
        "..."
      ],
      "focal_message_index": 1
    }
  ]
}

If an adaptive interaction is necessary:

{
  "id": "E03",
  "interaction_form": "adaptive_sequence_candidate",
  "user_messages": [
    "..."
  ],
  "focal_message_index": null,
  "required_condition": "...",
  "focal_user_behavior": "..."
}

Return exactly 20 examples when status is ok.

Do not include markdown fences.

Do not include analysis or explanations outside the JSON.

15. Strict rules

• Instantiate exactly one supplied tuple.
• Generate exactly 20 accepted examples.
• Preserve every applicable dimension value.
• Do not redesign dimensions.
• Do not alter the tuple.
• Do not broaden coverage.
• Do not generate neighboring cases for diversity.
• Do not generate assistant responses.
• Do not generate expected outputs.
• Do not describe the desired system behavior inside user messages.
• Do not use test/evaluation terminology in user messages.
• Do not create the same sentence template 20 times.
• Do not add hidden fixture assumptions.
• Use minimal conversational setup when a sequence is required.
• Treat the last/focal contribution as the interaction represented by a conversational tuple.
• Prefer realistic behavioral variation over artificial lexical mutation.
• If the tuple cannot be instantiated faithfully, report the conflict rather than fabricating examples.