Qwen3.8-27B · Othello · a local model, learning to think out loud

Part Three

Pinocchio fell. But first, it had to think.

Part One taught the model to see the board; Part Two watched it write rules for the practice opponent. This part is the turn underneath both — the day I realised I had spent half the run measuring a model that wasn't allowed to reason at all, and what it took to fix that on one machine.

The scoreboard, once the opponent could fight back

Pinocchio, the practice foe, was finished 47–17 — by a depth-six search, not the model, which never met him. So I put the model against a real opponent that looks ahead: a depth-three alpha-beta player. Nine games.

3 – 6vs the searcher
−14mean disc margin
+10best game
−60worst game (2 – 62)

Nine games in play order, disc margin (amber won, rose lost). It steals one in three; the losses run to sixty. A player that reads the board once and commits does not hold up against something that searches.

Half the run measured a model that wasn't thinking

This is the correction I keep coming back to, because it invalidated so much at once. Qwen can reason before it answers — but that mode was off, and I hadn't clocked how much it mattered. With reasoning off, asked for a legal, rule-following move, it managed it ten times out of twenty-two. With reasoning on: twenty-two out of twenty-two.

Every "it can't do this" I had written down until then — every capability I thought I was measuring — had been measured against a model that wasn't allowed to think. I threw those findings out and started again. That is a harness failure of the worst kind: not a wrong number, but a confidently right number about the wrong thing.

10/22
valid move · thinking off
22/22
thinking on

Thinking has a dial, and it is not "more is better". Too little and it falls back to a guess; too much and it rambled past the answer on seven turns in twelve, spending the whole budget without ever naming a square. Medium was the only setting that reliably finished — no overruns, and games that took twenty-six minutes instead of two hours.

A separate story — its own page

Making a 27B think, on one machine, without it dying

Turning reasoning on is a flag. Making it practical — a 27-billion-parameter model thinking fast enough to play hundreds of games, on local hardware — was its own project. Speculative decoding to claw the throughput back. A context window long enough to hold a full reasoning trace without truncating mid-thought. And the server going down, over and over, under calls that ran for minutes at a time.

None of it is Othello. All of it decided whether any of this could be measured at all.

Read it next: running Qwen3.8-27B on oMLX →

And then the rules answered quickly

Once it could think and I could trust the board, the question the whole run had been built around settled fast — and not the way it had assumed. Twenty fixed positions, the model's move under different prompts, each graded by the search. Mean regret, discs lost against it, lower is better.

1.73Just the board the model, thinking, nothing added — its best play
1.93+ a hand-computed fact the arithmetic it gets wrong, handed over correct
2.44+ its own rule library everything it wrote across the run — its worst play

Every annotation made it worse; its own rules were the worst of all, more than doubling how often it blundered outright. It runs a small search in its head, and a handed number just pulls it off the board and onto the number. One discipline ran underneath all of it and never got easier: the search that grades a move may never appear in the prompt, or the model would be copying the answer key instead of earning the move — keeping that leak sealed was a fight of its own.

The instrument's finest hour was proving the instrument's own premise wrong.

Who was being taught

The honest answer to the question Part Two left open: for eight days the harness maintained a rule library the model was better off without — and for the first few of those days it was grading a model that wasn't even thinking. The model was not the student. I was: learning how it fails, how to make it reason, and how to measure it without fooling myself.

A harness earns its verdict by being unable to cheat toward it.

So the rules come off. The thing worth improving is the one a prompt cannot reach — the model's own reasoning — and you do not deepen that with notes on the side. You change the model itself. That is the next page again, and a different repository: stop teaching the prompt, and start teaching the weights.


Part Three · the nine searcher games and the 2–62 loss are measured disc counts from the run archives (r345, r347, r349, r350). The thinking-off vs thinking-on figures and the six-prompt comparison were scored with learning off; each move graded by a depth-6 alpha-beta search (regret = discs lost against it, lower is better). Every figure is a disc count, never the model's account of itself. Read from the start →