In the current landscape of prompt engineering, the ‘XML-tagging’ pattern has become a de facto standard for structuring complex instructions. By wrapping context, instructions, and few-shot examples in tags like or , developers provide clear boundaries for the model to parse. However, this approach assumes the model’s underlying tokenizer and training objective treat XML as a primary structural element. For models trained heavily on conversational datasets or JSON-based API interactions, XML tags can introduce unnecessary noise and cognitive load, leading to instruction drift.

The fundamental issue is not the utility of boundaries, but the semantic weight the model assigns to the delimiter itself. When a model lacks significant exposure to XML-heavy documentation, it may perceive tags as arbitrary strings rather than structural anchors. This often results in the model attempting to ‘complete’ the tags or failing to recognize the hierarchical relationship between sections, ultimately undermining the precision of the prompt.

Identifying Structural Mismatch

To determine if your model is struggling with XML, monitor the attention patterns during the prompt processing phase. If the model is failing to adhere to instructions contained within specific tags, or if it hallucinates closing tags in its output when not requested, you are likely dealing with a model that views XML as a syntactic distraction.

Models optimized for JSON, such as those tuned for tool calling or structured data extraction, often treat curly braces and key-value pairs with higher fidelity. For these models, shifting from XML to a structured JSON object or a clean markdown hierarchy often yields immediate improvements in instruction following.

Refactoring to JSON-Based Delimiters

When moving away from XML, the goal is to maintain clear separation while utilizing tokens that the model is more likely to interpret as logical containers. JSON is generally the safest alternative due to its ubiquitous presence in training corpora.

{
  "instruction": "Summarize the provided technical documentation.",
  "context": "...",
  "constraints": [
    "No jargon",
    "Use bullet points"
  ]
}

By wrapping instructions in a JSON object, you leverage the model’s latent understanding of key-value relationships. This reduces the ambiguity of where one section ends and another begins, as the model recognizes the syntax of the language itself rather than having to parse custom delimiters.

The Markdown Alternative

If JSON feels too restrictive for your specific task—such as when passing long-form text—markdown headers provide a highly effective middle ground. Models are trained extensively on markdown, making headers (’#’, ’##’) natural separators that the model implicitly understands as hierarchical markers.

### Task
Summarize the following log entry.

### Input Data
[Log content here]

### Constraints
- Focus on error codes only.
- Output format: JSON.

Markdown headers have the distinct advantage of being ‘soft’ delimiters. They do not force the model into a rigid schema, yet they provide enough visual and semantic weight to delineate sections effectively across almost any transformer-based architecture.

Trade-offs and Considerations

• Token overhead: JSON serialization adds more tokens than simple XML tags.
• Parsing complexity: Your backend code must now handle JSON decoding rather than simple regex-based tag extraction.
• Model-specific tuning: Always test if the model’s system prompt already enforces a specific style; overriding it can cause performance degradation.

Conclusion

The effectiveness of prompt structure is entirely dependent on the model’s pre-training data distribution. While XML-tagging is a powerful tool for models like Claude or specialized research models, it is not a universal solution. Engineers should treat prompt structure as a tunable parameter—if the model is struggling to follow instructions, the first step should be to align the delimiter syntax with the model’s training bias.