human-or-machine preference test
Blind side-by-side comparison asking viewers which of two similar compositions is human-made, and which they prefer, to test algorithmic imitation.
Run it
What it looks like when it goes wrong (2)
- With only a handful of simulated judges, the 95% uncertainty bracket around each rate swallows most of the 0–1 axis and the dot rows are too sparse to read as a rate at all — any apparent split between the two questions is indistinguishable from coin-flip sampling noise, not a real effect.
- At algorithmStrength = 1 the generated panel is built to be indistinguishable from the reference, so both the identify rate and the prefer rate collapse toward 50% together — there is no longer anything to tell apart or to prefer, so the two-questions-diverge point the sketch exists to show disappears.
Clean-room implementation — written from the described algorithm, not from source. Reuse policy
What it means
A blind side-by-side comparison: show a viewer two similar compositions, one made by a person and one generated by a computer, and ask two separate questions — which one do you think is human-made, and which one do you prefer. The write-up describes it as mimicking the Turing Test: the subject’s job is to identify which picture they think is human and which is computer-generated, not to rate quality directly.
Why it matters in practice
The point is that the two questions can pull apart. In the instance this page’s evidence describes — a computer-generated composition set against a picture produced by Piet Mondrian — 59% of subjects preferred the computer-generated image, while only 28% correctly identified which picture Mondrian had actually made. Note what those two numbers do and do not say. They are separate tallies, not a cross-tabulation: a majority misidentified the authorship, and a majority preferred the machine image, but the source does not report how far the two groups overlap. Arithmetic alone guarantees only about a third of subjects did both. If you only ever ask “can they tell,” you miss the more useful result: whether they like what the machine made regardless of whether they can spot it.
How it shows up in code
There’s no algorithm to implement here, only a test rig: generate (or collect) a matched pair of images from different processes, randomize left/right assignment so position carries no signal, and log two separate tallies per trial — the identify-the-human answer and the prefer answer — rather than one combined score. Run it on your own output before you run it on anyone else’s: pair a hand-composed piece against a generative one built to resemble it, and see whether “looks machine-made” and “I like it less” are actually the same judgment for your viewers, or not.
Works that embody it
The only documented instance in this evidence is the Mondrian comparison above, known as Noll’s human-or-machine experiment.
Arguments against it
The test tells you nothing about authorship in the legal sense — it measures audience perception and taste, not process. That’s a different question from the one algorithmic authorship for copyright registration asks, which is whether the generating algorithm can be shown deterministic enough to support a copyright claim regardless of what any viewer guesses.
Connected to
Pioneered by: A. Michael Noll