← All work

From the lab · Conversational AI

Vincent: talk to Van Gogh, and he'll paint you

What would it be like to sit down with Vincent van Gogh? This prototype lets you talk to him. He answers in his own voice and character, drawing on his letters, and if you ask, he paints your portrait in his style.

TypePersonal R&D
RoleConcept & engineering
StackUnity, C#, Node.js, LLMs
StatusIn development

How it works

  • 01 · inputVoice & webcamSpeech-to-text; optional photo of you
  • 02 · clientUnity appConversation UI, state, media capture
  • 03 · backendNode.js servicePersona, history, model calls
  • 04 · modelLLM + image toolReplies in character; paints on request
  • 05 · outputVoice & canvasText-to-speech; painting on screen

Vincent stays in character. He talks about Saint-Rémy, Arles and the stars the way he wrote about them in his letters. When you ask him to paint you and attach a photo, the model calls an image-generation tool and Unity shows the result in a frame on screen.

Paintings

Julian painted in the style of The Starry Night
Show him your face and he paints you under his stars.
Julian and his future wife painted together in Van Gogh's style, under a crescent moon with cypress, blossoms and sunflowers
Ask him to paint you with someone. Prompt: "Paint me with my future wife."

Local first, then hosted

The first version ran on a locally hosted LLM trained on Van Gogh's letters and writing. For faster iteration on the experience itself, I moved the demo to a hosted frontier model with built-in image generation.

Why it matters

A believable character who guides, explains and responds is the same thing that makes VR training and educational apps work. Museums, classrooms and training simulators all benefit from a guide you can actually talk to. Vincent is my testbed for that: persona design, speech, and multimodal input in a real-time engine.

Next

Speech in and out already works. Next is a 3D Vincent, streamed and talking, instead of a text window.

Prototype

The Vincent prototype: conversation history, a generated portrait, the Unity editor and the backend log
Conversation, generated portrait, Unity editor and backend log (secrets blurred).