From the lab · Conversational AI
Vincent: talk to Van Gogh, and he'll paint you
What would it be like to sit down with Vincent van Gogh? This prototype lets you talk to him. He answers in his own voice and character, drawing on his letters, and if you ask, he paints your portrait in his style.
How it works
- 01 · inputVoice & webcamSpeech-to-text; optional photo of you
- 02 · clientUnity appConversation UI, state, media capture
- 03 · backendNode.js servicePersona, history, model calls
- 04 · modelLLM + image toolReplies in character; paints on request
- 05 · outputVoice & canvasText-to-speech; painting on screen
Vincent stays in character. He talks about Saint-Rémy, Arles and the stars the way he wrote about them in his letters. When you ask him to paint you and attach a photo, the model calls an image-generation tool and Unity shows the result in a frame on screen.
Paintings
Local first, then hosted
The first version ran on a locally hosted LLM trained on Van Gogh's letters and writing. For faster iteration on the experience itself, I moved the demo to a hosted frontier model with built-in image generation.
Why it matters
A believable character who guides, explains and responds is the same thing that makes VR training and educational apps work. Museums, classrooms and training simulators all benefit from a guide you can actually talk to. Vincent is my testbed for that: persona design, speech, and multimodal input in a real-time engine.
Next
Speech in and out already works. Next is a 3D Vincent, streamed and talking, instead of a text window.
Prototype