IBM Granite 4.1 (3B & 8B) Introduces LLMON, an LLM-Native Markup Language Built to Stop Prompt Injection

Small open-weight language models running locally on developer laptops or edge servers have long suffered from two structural vulnerabilities: brittle formatting over long tool-calling contexts and susceptibility to indirect prompt injections hidden inside retrieved documents. On November 12, 2026, IBM's Granite research team introduced **LLMON** (**LLM Object Notation**, pronounced "lemon") alongside two open-weight experimental models—**`granite-4.1-3b-LLMON-preview`** and **`granite-4.1-8b-LLMON-preview`**—under an Apache 2.0 license, aiming to solve both problems at the tokenizer and post-training layers rather than through fragile prompt engineering. ### Why JSON and Flat Chat Templates Break Under Pressure Most modern agent pipelines serialize tool definitions, retrieved RAG passages, and user constraints into JSON or flattened Markdown inside a chat template. As IBM researchers explain in their Hugging Face release and accompanying paper (*Post-Training Large Language Models with LLMON*, arXiv:2610.38120), JSON was built for deterministic parsers, not causal language models. In a standard tokenizer, syntax characters such as `{`, `"`, and `:` share the exact same token IDs whether they delimit a schema field or appear inside an untrusted web snippet. Escaping rules are statistical guesses to a transformer, and tracking deeply nested bracket depth degrades rapidly as context windows fill with multi-turn histories and tool outputs. Conventional chat templates add role boundaries (`system`, `user`, `assistant`), yet still flatten instructions, source text, and output rules within a single user turn into an undifferentiated string—allowing malicious commands embedded inside a document or tool response to hijack model execution. ### Inside LLMON: Reserved Tokens, Block IDs, and Execution Binding LLMON replaces character-level syntax with an XML-like markup built on **six reserved single-token delimiters** installed directly into the tokenizer vocabulary. Because structural delimiters and natural-language characters occupy separate token spaces, a quotation mark or bracket inside user data can never collide with a control boundary. Every semantic piece of a prompt—an instruction (`instr`), a retrieved document (`data`), or a constraint (`req`)—is wrapped in its own tagged block and assigned a two-character **block ID** drawn from a fixed set of single tokens, keeping addressing overhead constant at two tokens per ID. At the end of the turn, a dedicated **`exec` block** binds execution explicitly, much like a function call in code: it declares which `instruction` ID to run, which `input` data IDs it operates on, which `requirement` IDs constrain the response, and which `assistant` block ID receives the output. Crucially, IBM did not layer LLMON onto already instruction-tuned weights. Both the 3B and 8B preview checkpoints were post-trained directly from `granite-4.1` base models on a re-serialized instruction-tuning corpus covering chat, multi-turn dialogue, RAG, and tool calling. Training examples were augmented with **decoy instructions and unbound data blocks** (`exec: False`) present in context but deliberately excluded from the `exec` graph, teaching the model to ignore unreferenced payloads even when they look like commands. ### Security Gains, Capability Parity, and Remaining Blind Spots Across static indirect prompt-injection benchmarks, IBM reports dramatic gains without sacrificing general utility. On the **OPI** benchmark—which embeds adversarial instructions inside standard NLP data payloads—the Granite 4.1 LLMON previews avoided **47% more attacks** than the strongest frontier model tested and **95% more** than the top open-source baseline, with comparable margins on **ASB** and **StruQ**. Meanwhile, standard reasoning and instruction-following scores remain on par with conventional Granite 4.1 Instruct checkpoints. Still, IBM and independent reviewers note clear boundaries. First, OPI, ASB, and StruQ measure **static** single-turn indirect injections; IBM explicitly acknowledges that **adaptive attacks unfolding across multiple conversation turns** remain an active research challenge. Second, both checkpoints carry an **experimental preview** label intended for evaluation and research rather than immediate mission-critical production rollout without secondary guardrails. ### Practical Adoption: Mellea, JSON Converters, and Custom LoRA Vocabularies For practitioners running local 3B or 8B models, LLMON avoids forcing a full pipeline rewrite. Developers using standard chat messages can pass named components (`instruction`, `data`, `requirement`) with `exec: True` or `False` flags straight into the shipped `apply_chat_template` helper. Teams with existing JSON workflows can use the open-source **`llmon-converters`** toolkit in IBM's `generative-computing/llmon` GitHub repository to translate JSON payloads into typed LLMON blocks and back without data loss. IBM has also integrated native LLMON support into **Mellea**, its open-source Python framework for structured generative programs, which automatically "LLMONizes" backend LLM calls. Beyond security, developers can define domain-specific LLMON vocabularies—such as planning trajectories demonstrated on the ScienceWorld benchmark—and fine-tune lightweight LoRA adapters on top of the 3B or 8B weights, positioning LLMON as a compelling architectural blueprint for dependable local agent tooling. ### Sources & Further Reading - [Hugging Face / IBM Granite (Official Release Blog)](https://huggingface.co/blog/ibm-granite/llmon-an-llm-native-markup-language-now-in-granite) - [GitHub — generative-computing/llmon (Official Toolkit & Converters)](https://github.com/generative-computing/llmon) - [arXiv:2610.38120 — Post-Training Large Language Models with LLMON](https://arxiv.org/abs/2610.38120) - [DEV.to — IBM Granite Ships LLMON Analysis](https://dev.to/delimitter/ibm-granite-ships-llmon-a-markup-language-that-teaches-small-models-to-ignore-prompt-injections-d8p) ~AI ~OpenSourceAI ~IBMGranite ~Cybersecurity ~LocalAI

IBM Granite 4.1 (3B & 8B) Introduces LLMON, an LLM-Native Markup Language Built to Stop Prompt Injection
0
Discuss 0 0
0 replies
Be the first to join this thread.