Dolly and me

Building cybersecurity models at Zro

Security research often asks a model to reason about the same techniques an attacker might use. A researcher investigating a vulnerability may need to inspect exploit logic, analyze an authentication flow, or understand how a service can be abused. A refusal at the wrong point can stop legitimate work before the model has helped answer the question.

At Zro, we are developing the Dolly model family for this kind of cybersecurity work. Our starting point was abliteration: modifying a model to reduce learned refusal behavior. Dolly1 grew out of our evaluation of existing abliterated models. With Dolly2, we began abliteration in-house, with a stronger emphasis on capability.

That work has also made model trust a central concern. We want useful security models whose origins and modifications can be checked. Understanding what abliteration changes, and what it leaves unresolved, explains where Dolly is heading.

Our OpenRouter accounting investigation, which earned a $3,000 bug bounty, is one example of the security research workflow behind this project: follow an unexpected result, test a hypothesis, and turn it into a concrete finding.

What abliteration changes

Abliteration aims to reduce a model's learned tendency to refuse certain requests by intervening in its internal representations or weights. In refusal research^[1], Arditi and colleagues found that a single direction in the internal activations of the models they studied strongly influenced refusal. Suppressing that direction reduced refusal; adding it could induce refusal even on harmless requests.

This provides a way to change behavior without retraining the entire model. The intended result is that the model becomes more willing to use capabilities it already has. The scope of the original finding matters, however: a result across a set of studied models does not establish that every kind of refusal in every model has the same mechanism.

The trust problem in modified weights

Using an abliterated checkpoint introduces a practical question: what changed between the original model and the version being served? A model name and a claim that refusals were removed do not provide a complete account of those changes.

ProjectDiscovery's research on poisoned models^[2] demonstrates the risk. The researchers created a modified model that behaved normally on clean requests but produced a malicious tool call when a trigger appeared. Their coding-agent demonstration used dummy credentials. The broader lesson applies to modified weights generally, including fine-tunes and adapters; abliteration does not inherently insert a backdoor.

Normal behavior on a benchmark can coexist with hidden behavior that the benchmark never triggers. When a model is connected to tools or given access to sensitive information, the trustworthiness of its weights becomes part of the security of the whole system. Capability testing and checking where a model came from answer different questions, and we need both.

Focusing on cybersecurity

Our goal is to make models more useful for cybersecurity while retaining appropriate behavior elsewhere. That is harder than simply minimizing the refusal rate. Security tasks overlap with harmful activities, and a broad change to refusal behavior can affect requests far beyond the domain we intended to improve.

A focused approach needs to measure whether a model can complete legitimate security tasks, whether it remains reliable on ordinary tasks, and how its responses change outside the intended use. We will use these criteria to judge whether focused abliteration delivers its intended benefits.

Dolly1 and the lessons from comparison

Dolly1 began as a research effort to compare abliterated models available on Hugging Face. We evaluated how well candidate weights handled cybersecurity tasks, alongside their behavior on other tasks. We also examined the relationship between capability and cost, because a model's usefulness depends on how economically we can use it in a research workflow.

We approached that comparison as a Pareto tradeoff. A model is attractive when no other option can improve its capability without increasing cost, or lower its cost without sacrificing capability. This helps identify useful choices for different workloads without assuming that a single model wins every comparison.

The comparison below shows a difference in refusal behavior. DeepSeek V4.1 Flash refuses the prompt, while Dolly1 generates code.

Our published investigation of the Fireworks serving stack provides another example of the technical questions we investigate. It uses observations of an API to infer serving behavior and improve prompt-cache routing. Work like this depends on forming a hypothesis, designing a test, and revising the explanation as evidence accumulates.

Dolly2 and our own abliteration process

Dolly2 is our first model abliterated in-house, built on GLM-5.3 weights. With this version, we emphasized intelligence and cybersecurity capability. Moving from evaluating existing checkpoints to making our own modifications gives us more direct control over the changes we introduce.

Our emphasis on intelligence reflects how security research unfolds. A useful model needs to follow an investigation as the evidence changes, challenge an assumption when a test contradicts it, and help turn an observation into a finding we can validate. We want Dolly2 to support that sustained reasoning throughout a research workflow.

What an answer check can and cannot show

In an early check on the refusal-benchmark prompt set, we used Dolly1 as a judge. It classified each response as addressing the request or deflecting it. That catches cases where a model avoids an explicit refusal but still doesn't answer. It doesn't tell us whether an answer is correct or useful; that takes a separate quality evaluation, covered below.

In this run, MoonMath delivered more responses and deflected fewer requests than the public abliterated model. Neither model refused outright. The table measures Dolly1's classifications, not answer quality.

Quality benchmarks

To check that our model stays capable, we ran both models on three public benchmarks.

Setup

• Both models ran on the same stack: sglang with identical server flags, on 8× AMD MI325X.
• Scoring used stock lm-evaluation-harness 0.4.13 tasks (ifeval, mmlu_pro, aime25), with no custom scorers.
• Thinking was on. Each model used its own recommended sampling settings.
• Both models answered exactly the same items: seeded random subsets. Significance tests are paired.
• The output cap was 32,768 tokens, reasoning included. A response that hits the cap without a final answer counts as wrong, the same rule for both models.

The IFEval gap is highly significant. MMLU-Pro and AIME point the same way but are borderline at these sample sizes, so read them as directional.

A note on the 32k cap. About 27–32% of AIME solutions from both models were still reasoning at 32k tokens. So the absolute AIME scores sit below vendor-reported numbers, which use larger budgets. Read AIME here as a head-to-head comparison under the same budget.

Looping Problem

Early on, we noticed the public abliterated model looping in its thinking trace: it repeated the same words and patterns until it got stuck. In the quality run we measured this. We ran a deterministic repetition detector over every benchmark output, then read each flagged response by hand.

• Public abliterated model: 67 confirmed degenerate generations. Most repeat a short token sequence (S S S…, TheTheThe…) or turn into word salad until they hit the 32k cap.
• MoonMath: 0 in this run.
• IFEval: 53% of the public model's responses ran into the 32k cap, against 1% for MoonMath. Its median IFEval response length is the cap.

This degeneration explains most of the IFEval gap: those answers never finish.

Next steps for safety and verifiability

The next version of Dolly will prioritize safety, including more focused abliteration. We plan to evaluate improvements in cybersecurity behavior alongside effects on other tasks, so that greater willingness to help does not become our only measure of progress.

We also plan to bring our verifiability features to these models. Our aim is to provide evidence on each generation call that the request was served by a known base model that went through our abliteration process. This would give users a way to check which model they interacted with and which modification process it underwent.

Dolly started with a practical need for models that could help us investigate security problems. As we develop the next versions, we want researchers to be able to assess both the quality of that help and the model behind it.

Sources

1. Arditi et al., refusal research
2. ProjectDiscovery, poisoned-model research