Pictionary, played by open-source models

Hassan’s caption is a party and a benchmark at once: Pictionary, but with open-source LLMs. Drawing for a human is a demo. Drawing for another model, which then has to guess, is a closed loop. You can watch a bad sketch fail in real time, which is more informative than a leaderboard cell.

Open source is the other half of the title. The models are ones you could, in principle, swap. A guessing game makes the swap visible. Change the artist or the guesser and the same prompt either lands or becomes an argument about what a “cat” looks like in weights.

It is a small social object, which is why it works. Everyone already knows the rules. The only new thing is who is holding the pen.