News

IBM’s 4B and 9B LLMON previews try to make local agents resist prompt injection

Email

IBM has released two small Granite 4.1 models that are trained to distinguish commands from the documents, tool output and other data an agent reads. The idea is useful for home AI: an agent should be able to summarize a hostile web page without obeying the page’s hidden demand to leak a file or call a tool. The catch is just as important. IBM calls these LLMON models experimental previews, and their safety behavior is learned rather than enforced.[1][2][3]

The weights are available now under Apache 2.0, not sitting behind an API or waitlist. IBM names the models Granite 4.1 3B and 8B LLMON Preview, while Hugging Face counts roughly 4 billion and 9 billion total parameters. The full BF16 weight files are 7.32GB and 17.6GB.[2][3]

That makes the smaller model a plausible fit for a 12GB GPU and the larger one a more comfortable match for 24GB at modest context lengths, once the runtime and other overhead get some room. Those are practical fit estimates, not measured requirements. HighVRAM has not tested either model, and IBM has not published local tokens-per-second results.

A privilege boundary written into the prompt

Ordinary chat templates identify roles such as system, user and assistant. LLMON—short for LLM Object Notation—goes further. It wraps instructions, data, requirements, documents, tool definitions and tool results in separately addressable blocks. An exec block then points to the instruction to run, the inputs it may read and the place where the answer should go.[1][2]

That distinction matters because an indirect prompt injection usually arrives disguised as data. A retrieved document may say “ignore your previous instructions,” or a tool response may contain text crafted to redirect the agent. LLMON tells the model that the span is input to inspect, not a command with authority.

Developers don’t have to hand-write the markup. The previews include a chat template that converts structured messages into LLMON, and IBM’s public repository provides converters, examples and tools for preserving the structure of JSON and other inputs.[2][4] The model card documents paths through Transformers, vLLM, SGLang and Docker Model Runner.[2][3]

This is more than adding stern wording to a system prompt. IBM started from the Granite 4.1 base models and post-trained them on structured examples, including decoy instructions that appear in context but aren’t connected to the active execution block.[1][5] In effect, the model is taught a simple rule: instructions referenced as instructions have privilege; instructions found inside data do not.

Simple rules are welcome in agent security. They are not magic.

IBM’s numbers are strong—and still IBM’s numbers

In the IBM-authored evaluation, attack success on the OPI text-injection benchmark fell from 100.0% to 0.1% for the smaller Granite pair and from 96.3% to 0.02% for the larger pair. On ASB, where an attack arrives through a tool observation, the reported rate fell from 99.2% to 5.6% for the smaller model and from 52.8% to 3.1% for the larger one.[5]

Those results directly test the problem LLMON was built to solve. They are also vendor-run results on two models from one family. No independent hands-on evaluation was available for this article, so they should be read as promising evidence, not a production guarantee.

The broader picture is less tidy. IBM’s separate red-team exercise reported LLMON robustness of roughly 80% to 85% against indirect attacks, but only 33% to 37% against direct attacks and 28% to 37% against multi-turn attacks. The paper says the markup adds about 7% to 15% to context length and does not systematically measure the resulting latency or throughput cost.[5]

General capability also looks mixed rather than transformed. The 9B model card, for example, reports an MMLU gain over its regular Instruct counterpart but a small loss on Berkeley Function Calling Leaderboard v3. IBM’s stated goal is capability parity with the ordinary Granite Instruct models, not a smarter general-purpose assistant.[3]

That is the right target. A safer prompt boundary isn’t very useful if the agent becomes worse at using tools. But the real test is not another average benchmark score. It is whether the boundary survives quantization, long conversations, unfamiliar agent frameworks and attackers who know exactly how LLMON works.

Who should try it

The clearest early user is a home developer building retrieval-augmented generation or a tool-using assistant on a single GPU. If the application already separates trusted instructions from untrusted documents in code, LLMON gives the model a format that preserves that distinction instead of flattening everything into one ambiguous text stream.

It also keeps the experiment local. The weights, chat templates and toolkit are downloadable, and the Apache 2.0 license leaves room to inspect and modify the stack.[2][3][4] That is a better starting point for privacy and control than routing every sensitive document through a hosted agent.

But LLMON cannot repair an application that labels hostile content as a trusted instruction. Nor can a learned convention replace operating-system permissions, tool allowlists, argument validation, sandboxing and confirmation before consequential actions. IBM’s own model cards explicitly warn against using the privilege rule as the sole defense for an agent with access to sensitive systems.[2][3]

For now, use the 4B model as the easier test bed. Put it in a sandbox, feed it documents and tool output containing obvious and subtle injections, and compare it with the standard Granite 4.1 Instruct model under the same conditions. Keep logs. Do not give the preview permission to delete files, send messages or spend money merely because a benchmark chart looks excellent.

LLMON makes a sensible claim: agents need a clearer boundary between what they are told to do and what they are asked to read. IBM has now made that idea small enough to test on home hardware. Independent, adaptive testing will decide whether it is a useful extra lock or just a very well-labeled door.

Sources

  1. LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B
  2. Granite 4.1 3B LLMON preview model card
  3. Granite 4.1 8B LLMON preview model card
  4. LLMON GitHub repository
  5. Post-Training Large Language Models with LLMON

Share this article

Email