// MEDIA LAB

Giving the minds faces, voices, and motion

Grounding makes a persona think. Media makes it present. Some media ships in Revive today: real voices, video replies, retrieved artwork, and AI-generated portraits. The rest is a media pipeline that a sister project proved end to end. It is ready to be pointed at the personas.

What the minds can do today

None of this is a mockup. Each capability below runs in production, together with the RAG APIs and Discord bots.

A voice per mind

Each persona speaks through its own ElevenLabs voice. Send a voice message, and it returns through Whisper transcription, a grounded RAG answer, and text-to-speech in character. Sagan and Walter Russell use voice clones, not stock voices. Russell's clone comes from a lecture tape he recorded at Swannanoa in 1953.

8 personas · Whisper in → ElevenLabs out

Gabriella 2.0, tuned by ear

Gabriella 2.0's "Gabriella Pro" voice was not set once. We calibrated it across several listening sessions until it sounded correct.

stability 0.75 · similarity 1.0 · style 0.20 · speed 0.88

Real-time voice channels

Gabriella 2.0, Sagan, Decode, and Walter Russell hold live spoken conversations. The channel streams speech-to-text into Claude, and it streams ElevenLabs audio back out, at about two seconds of latency. Sagan's channel mixes background music under his answers.

Gabi · Sagan · Decode · Russell · ~2 s round trip

Gabriella 2.0's /vid replies

Gabriella 2.0 answers with video. ffmpeg merges a looping mp4 of her portrait with a new TTS answer. A spoken reply arrives as a short talking clip, not an audio file. This shows that persona video works here today.

looping mp4 + TTS · merged with ffmpeg

Leonardo pairs his works

Leonardo has an image-retrieval system. It pulls pieces from his own catalog and pairs them with his replies. When he speaks about anatomy or flight, the applicable sketch comes with the answer.

image catalog · retrieved alongside text

Portraits, generated or sourced

Some faces on this site are AI-generated portraits from the image stack below. Not all are. A persona's face can also come from the record. Imagery a mind owns (artwork from its corpus, photographs, sketches) can enter the system too, so the mind can answer with its own pictures. The avatars in the strip below this grid are those exact files, live on each page.

generated & sourced imagery · gpt-image-2 + corpus

The portraits, as shipped

Gabriella 2.0 portrait Benjamin Franklin portrait Leonardo da Vinci portrait Carl Sagan portrait Decode portrait Emily Dickinson portrait Sir David Attenborough portrait Walter Russell portrait

These are the real avatar files served across the site. The pipeline below can set these portraits in motion.

From still frame to finished reel

A sister project proved this five-stage media pipeline end to end. Each script and API key is in place. It has not been pointed at the personas yet, but nothing new must be built.

1 · Stills

Generate the key frames.

gpt-image-2
→

2 · Narration

Voice the script in the persona's voice.

ElevenLabs
→

3 · Timing

Align frames to the narration.

whisper-1
→

4 · Motion

Animate the stills into moving shots.

Grok Imagine
→

5 · Assembly

Cut it all into the finished reel.

reel sequencer
Stage Tool What it does Discipline worth keeping
1 · Stills OpenAI gpt-image-2 Manifest-driven generation. A TSV lists each frame and its prompt, and the script renders them in a batch. Raw PNGs stay archived. Finals ship as web JPEGs. Each prompt ends with a "no lettering / signage / watermarks" guard sentence. Without it, the model invents text in the frame. --quality high runs about 6 times slower than low.
2 · Narration ElevenLabs eleven_turbo_v2_5 Reads a script line by line from a JSON manifest, in the persona's own voice. This produces one recorded narration track. An --estimate mode prices a run from the character count, with no API call. You know the cost before you spend a credit.
3 · Timing OpenAI whisper-1 Forced alignment. It transcribes the recorded narration with word-level timestamps. It then aligns the transcript to the known script, and it recovers a real start and end time per line. All sync derives from this data. On-screen text tracks the voice exactly, and no timing is estimated by hand. Whisper bills per minute of audio, not per character.
4 · Motion xAI Grok Imagine grok-imagine-video Turns a still into a short motion clip. The system submits an image-to-video job and polls until it finishes. This is how a portrait becomes animated: a face that breathes and blinks. $0.05/s flat ($0.08/s on v1.5), with a cap of 15 s per clip. Real cost is about 1.4 times the estimate, because the quote omits an image-input fee. Trust the per-job log, not the estimate.
5 · Assembly JS reel sequencer Plays the narration track against the sequence of stills and clips, cut to the voice. Subtitles render on the video itself. Each asset is base64-inlined, so the page makes no external requests. Beats lock to narration onset times, not to a timer. The picture does not drift ahead of the voice. Short clips slow down with playbackRate to fill their beat, and they do not freeze early.

Current status

Stages 1 through 3 can run in any order. Stages 4 and 5 are sequential. The chain is a sister project's proven production path. It is not a Revive feature yet, but the scripts, the manifests, and each API key are in place. To point it at a persona is a configuration job, not a build. The stage-by-stage write-up is in the public repo as media-pipeline.md. The voice side, async and real-time, is voice-pipeline.md.

Concrete next moves

Each item below is assembled from the stages above. The questions are which stages, and what the cost is. Where a per-second price exists, the estimate is included.

Animated portraits

Each mind's portrait, breathing and blinking on the personas page. An idle loop replaces a static headshot. The stills exist. Only the motion is new.

Stages 1 (done) → 4 · a 5 s loop ≈ $0.25 estimated, ~$0.35 actual

"How a mind is rebuilt" reel

A narrated short that shows how a persona is rebuilt from its sources. The persona's own voice tells it, over its own portrait and stills. This uses the full pipeline.

Stages 1 → 5 · motion is the only per-second cost

Debate highlight reels

The real roundtable transcripts, cut into short reels, with word-synced subtitles over the speakers' portraits. The audio is recorded persona voice. The timing comes from forced alignment.

Stages 2 → 3 → 5 · no motion needed for a subtitle cut

Emily's poems as audio cards

Emily retrieves her actual verses. Each one becomes a typographic card. Her voice reads the poem, and the text appears as she speaks. There is no performance and no motion. The poem is heard and seen.

Stages 2 → 5 · TTS + typographic assembly, zero video spend

Leonardo narrating his sketches

Leonardo retrieves works from his catalog with his replies. Add narration and light motion, and a retrieved sketch becomes a short piece, with his commentary on his own work.

Image catalog (done) → Stages 2 (+ optional 4) · a 4 s clip ≈ $0.20 est, ~$0.28 actual

The same discipline applies to media

Grounding is not only a text idea. A persona's face and voice should come from what the persona was, and they should be calibrated with intent, not set to a default. Gabriella 2.0's voice shows this. We did not guess the tuning values (stability 0.75, similarity 1.0, style 0.20, speed 0.88). We reached them by ear over several sessions. Portraits, motion, and narration need the same care before they represent a mind.

← Previous
The Corpus