Anthropic · Claude Code
| Thinking | Agentic | |
|---|---|---|
| Prompt | The one above, word for word, plus a line on how to answer: reply with the SVG only, or save it as painting.svg in the folder. | |
| Tools offered | None | A shell (with Python), an SVG-to-PNG renderer, and a way to view the PNG |
| Sees its own picture | Never | As often as it likes |
| Broken SVG | Shown as broken | It can notice and fix it |
| Time limit | 20 minutes | 25 minutes |
| Web, other attempts | None: no tools at all | None: sandboxed, network off, reading outside its folder denied |
| Effort setting | high, for every model (Gemini takes a token budget instead: 24,576 thinking tokens, Google’s own mapping of “high”). A run at any other effort says so in its caption. | |
| Audit | 22 of 22 runs made zero tool calls | 8 of 8 runs used only their own folder; every call is logged |
Harnesses. Anthropic models ran in Claude Code and OpenAI models in the Codex CLI, each on its subscription login, batch 2026-10-20. Thinking mode ran Claude Code with no tools offered and Codex with its shell and every other tool switched off; agentic mode ran both inside their own sandboxes with the network denied.
Google, xAI, DeepSeek and Meta have no such CLI here, so they were called through their APIs. Thinking mode is one request with no tools declared. Agentic mode is a small agent loop of ours that offers exactly two tools, a shell and an image viewer, and runs every shell command under macOS’s sandbox with the network denied and every file outside the model’s own folder unreadable. Before the runs, one model per family (and both DeepSeeks and both Muses) was shown a test image none of them drew; all described it correctly except DeepSeek V4 Pro, which cannot see images: it gets the same tools, but renders blind. Meta’s API can run a web search on its own; a response that did was thrown away unseen and the same request sent again.
Reading a caption. Time is wall-clock for the whole run. A tool call is any shell command or file operation; rendered counts how many times the model turned its SVG into a PNG it could look at. All of it is read from the run logs by the audit, not reported by the model.