What Dreams Are Made Of
Picture this: a projector-backlit screen sits a top a beautiful wooden, glowing box in the middle of a music festival. People walk up, touch it, sketch on it with their fingers. And the canvas responds—not with simple paint strokes, but with AI dreams.
The Dreambox is a touch-reactive canvas inhabited by a collective of artificial intelligences. It displays an ever-evolving stream of imagery as the AIs interpret and incorporate your sketches into their dream-like visions in real-time. Your finger becomes the bridge between human imagination and machine hallucination.
We installed this interactive exhibit at the BassCoast music festival for the 2024-2025 season, where hundreds of festival-goers sketched mountains, faces, abstract shapes, and… well, let’s just say the AIs had some interesting interpretations of what people draw at 2 AM.
The Technical Dream
Under the hood, the Dreambox runs a custom-built touch detection system feeding into our own image generation pipeline running…so many ML models it makes me head hurt.
The challenge wasn’t just making the AI respond to sketches (that part is relatively straightforward with modern diffusion models). The real trick was making it feel responsive and alive. Nobody wants to wait 30 seconds for their doodle to become art. We hand-coded everything for performance, from Kalman filters for robust touch tracking to frame-to-frame interpolation steps to WebGL transition animations to create the illusion of a living, breathing canvas that’s always one step ahead of you.
Early Sketching Sessions
Before we took it to BassCoast, we spent weeks tuning the software. These are some of the early recordings—just us, a tablet, canvas, and an AI trying to figure out what the hell we were drawing.
Watch how the AI interprets different types of sketches. Sometimes it gets it perfectly. Sometimes it goes completely off the rails. Both outcomes are equally fascinating.