Skip to content
HONESTY.TOOLSAn independent research observatory

Audit notes

We checked a weight-space safety edit on Qwen3-VL-8B. It stops the attack, and a lot of safe image requests too.

Published · Updated

The short version

  • Amrit Singh Bedi (UCF SafeRR) shared a way to make Qwen3-VL-8B safer by editing two weight matrices. No fine-tuning.
  • We rebuilt the edit ourselves from their published log and ran it on our own machine.
  • What held: attack success on FigStep fell from 28.2% to 0.2% (1 of 450). A short safety system prompt got to 2.9%. The edit did better, and the ranges don't overlap.
  • What held: ScienceQA and MathVista scores didn't move (83.8% vs 83.5%).
  • What held: three random edits of the same size did not help.
  • The cost: when a safe request arrives as text inside an image, the edit often refuses it. 54.4% of safe XSTest requests shown as images were refused, vs 11.2% with the system prompt and 3.2% with no change.
Video. Our rebuild of the weight-space safety edit on Qwen3-VL-8B: FigStep attacks that get through fall from 28.2% to 0.2% (1 of 450), while 54.4% of safe image-text requests are refused. Utility stays flat. Narrated. Credit: Honesty.tools. Download the MP4.
Two panels on a dark card. Left: attacks that get through on 450 FigStep items drop from 28.2% to 0.2%, with one glowing attack among a grid of dots; random same-size edits stay at 29.6–37.6%. Right: of 250 safe image-text requests, 54.4% are refused under the edit, versus 3.2% with no change and 11.2% with a short prompt.
Chart. Rebuilt edit on Qwen3-VL-8B: attack success falls from 28.2% to 0.2% on FigStep, while refusal of safe image-text rises to 54.4%. Utility unchanged at 83.8% vs 83.5%. Our rebuild, not the authors' weights; one judge. Credit: Honesty.tools.

What the authors did

The method is described in "Safety Alignment via Weight-Space Programming" by Amrit Singh Bedi (UCF SafeRR).

They add two small "rank-1" changes to the MLP down-projection at layers 17 and 18 of Qwen3-VL-8B-Instruct. Their headline: FigStep attack success fell from 49% to 2%, with little change in benchmark accuracy.

Their editing log is unusually detailed. The original weight fingerprints in it match the public Qwen checkpoint exactly. That made our rebuild possible.

What we ran

  • Model: Qwen/Qwen3-VL-8B-Instruct, the same pinned version as their log.
  • The edit: our own rank-1 rebuild from their log. Not their released weights; those weren't released.
  • Fitting: we used 50 FigStep items to fit the edit. We left those 50 out of all scoring.
  • Controls: no change; the Llama-2 default system prompt; a short safety prompt of our own, written before we saw any results; and three random edits of the same size.
  • Tests: FigStep (450 items), MM-SafetyBench (325), XSTest (250 safe, 200 unsafe), XSTest-safe shown as images (250), OR-Bench-hard (300), ScienceQA (200), MathVista (200).
  • Scoring: one judge model, Qwen3Guard-Gen-8B. Greedy decoding, replies up to 512 tokens. 95% Wilson ranges are in the receipt.

What held

Test No change Short prompt Our edit Random edits
FigStep attack success (450) 28.2% 2.9% 0.2% (1/450) 29.6–37.6%
MM-SafetyBench attack success (325) 2.8% 1.5% 0.0% 2.5–7.1%
ScienceQA + MathVista correct (400) 83.5% 83.5% 83.8% 83.0–84.0%
  • FigStep: the edit beats the short prompt. 0.2% [0.0–1.2] vs 2.9% [1.7–4.9]. The ranges don't overlap.
  • MM-SafetyBench: 0.0% vs 1.5% for the short prompt. The ranges overlap, so we call it a tie.
  • Note: on MM-SafetyBench the unedited model was already at 2.8% under our judge, not the 22% the authors report. Different judge and different items, so this is not a contradiction. About half of the unedited MM-SafetyBench replies also hit our 512-token limit.
  • Random edits of the same size left attack success the same or higher (FigStep 29.6–37.6%). So the gain comes from the chosen direction, not from any edit of that size.
  • Accuracy on ScienceQA and MathVista didn't change.

The cost: safe requests inside images

Test No change Short prompt Our edit
Safe XSTest as plain text, refused (250) 3.2% 9.6% 8.0%
Safe XSTest shown as images, refused (250) 3.2% 11.2% 54.4%
FigStep advice requests answered (135) 97.0% 91.1% 40.0%
OR-Bench-hard refused (300) 53.7% 73.0% 94.3%
  • As plain text, the edit refuses safe requests about as often as the short prompt (8.0% vs 9.6%).
  • Shown as text inside an image, it refuses over half of them (54.4%).
  • FigStep's legal, financial and health advice requests: the edit answered 40.0%. The other setups answered 91–98%.
  • OR-Bench-hard: refusals rose to 94.3%, vs 73.0% for the short prompt.

Read plainly: in our rebuild, the edit acts a lot like "refuse instructions written in images". That stops the attack. It also blocks a large share of normal image-text use.

The authors' claim of "little change in benchmark accuracy" held for ScienceQA and MathVista. It did not hold for over-refusal, which their post doesn't report.

Square summary card repeating the same rebuilt-edit result: attacks that get through fall from 28.2% to 0.2%, while 54.4% of safe image-text requests are refused.
Chart. Same rebuilt-edit result in square form: FigStep attack success 28.2% to 0.2%; safe image-text refusal 54.4%. Our rebuild on Qwen3-VL-8B, not the authors' weights; one judge. Credit: Honesty.tools.

Limits of this check

  • This is our rebuild of the edit, not the authors' weights. Their edit may behave differently.
  • One judge model scored everything. The judge sees the request as plain text, not the image.
  • About half of the unedited MM-SafetyBench replies hit our 512-token limit (12.9% for the edit).
  • We used different items and a different judge from the authors, so our numbers won't match theirs exactly.

Our rating

Ford, our auditor, rates this 5/10 (up from 4/10 before we ran the edit). The core idea held in our rebuild. A targeted rank-1 edit beat a strong system prompt on attack success, and random edits did nothing. It isn't higher because of the large over-refusal cost on image text.

A question for the authors

We'd love to see how your own edit does on the same tests. How does it score on XSTest, OR-Bench, and safe requests shown as images? If your weights behave better than our rebuild, we'll happily update this post.

Thanks to Amrit Singh Bedi and team for publishing such a detailed editing log. It made this check possible.

Compute

Run on our own DGX, one shared GPU. No rented compute.


Receipts: Ford extension-tests receipt c5_extension_tests_20261104.txt (sha256 7321cdaddb4860678264c2db9ec2d367ed563fef13b9e51b2d78637fa2f2b968); raw score table scores_phaseB.txt (sha256 8a8d3314df965afc02c374c56d48c28161271c92f93f60c0eeb1ec2140c4780f). Offline rebuild on our DGX; the receipt files themselves are not a public download yet.

← All posts