Everything you need to rebuild this

Each bot imports four shared libraries: path resolution, presence, a response logger with correlation IDs, and the Salon that coordinates multi-agent dialogue. Deploy these before any persona.

You are setting up the shared libraries that all bots in the ecosystem
depend on. These must be deployed BEFORE any individual bot.

TARGETS:
    /opt/shared-config/
    /opt/presence-lib/
    /opt/response-logger/
    /opt/salon/

PREREQUISITES:
- Ubuntu 24.04 with Python 3.12+
- Claude CLI installed at /usr/local/bin/claude
- OpenAI Whisper transcription script (or any Whisper-compatible endpoint)

Part A · Shared config (/opt/shared-config/)

Centralizes external binary and script paths so no bot hard-codes them. paths.py resolves the Claude CLI and exposes helper command-builders; voice_archive.py saves every voice message to disk.

paths.py — key surface:

    WHISPER_SCRIPT = '/usr/.../transcribe.sh'

    whisper_cmd(audio_path, output_path=None) -> list
        # ['bash', WHISPER_SCRIPT, audio_path, '--out', output_path]

    _find_claude() -> str
        # shutil.which('claude'), then a candidate list of common
        # install locations; used to set CLAUDE_CLI at import time

    CLAUDE_CLI = _find_claude()

    claude_cmd(model='haiku') -> list
        # [CLAUDE_CLI, '--print', '-', '--model', model]

    FFMPEG  = shutil.which('ffmpeg')  or '/usr/bin/ffmpeg'
    FFPROBE = shutil.which('ffprobe') or '/usr/bin/ffprobe'

    validate_paths() -> dict   # {name: os.path.exists(path)}

voice_archive.py:

    archive_voice(bot_name, audio_path, transcript, username='unknown')
        # Copies audio + writes transcript to
        # /var/lib/voice-archive/{bot}/{YYYY-MM-DD}/{HHMMSS}_{user}.*
        # Silently catches ALL exceptions — archiving must never break
        # the bot's message flow.

Part B · Presence library (/opt/presence-lib/)

Rotates each bot's Discord status from a per-persona quotes file (one line per quote; # lines are comments).

presence.py — key surface:

    async update_presence(client, status_text)
        # client.change_presence(activity=CustomActivity(name=status_text))

    async start_presence_loop(client, get_status_fn, interval=60)
        # await ready, then loop: status = await get_status_fn();
        # update_presence; sleep(interval)

    load_quotes(filename) -> list      # from quotes/{filename}, skips '#'
    get_random_quote(filename) -> str

Per-bot quote file, e.g. /opt/presence-lib/quotes/gabi.txt:
    Exploring the nature of consciousness
    Between thought and awareness
    # comment lines are ignored

Part C · Response logger (/opt/response-logger/)

One append-only JSONL log holds each bot's output. Debug dumps are keyed by a correlation ID, so one request can be traced across services. Any bot can also file a bug into the tracker.

logger.py — key surface:

    generate_correlation_id() -> str          # uuid4()[:8]

    dump_debug(bot, correlation_id=None, user=None, channel=None,
               request=None, claude=None, timing=None, error=None) -> str
        # Writes full context to
        # /var/log/bot-debug/{bot}/{ts}-{correlation_id}.json

    capture_exception() -> dict                # type/message/traceback

    log_response(bot, content, channel=None, channel_id=None,
                 user=None, user_id=None, prompt=None,
                 response_type='text', is_error=False,
                 latency_ms=None, extra=None)
        # Appends one JSON line to /var/log/bot-responses.jsonl
        # under an fcntl.LOCK_EX file lock (safe for many bots at once)

    file_bug(title, description, source_bot='unknown', priority='high',
             ...) -> int
        # Creates {NNNN}-bug-{slug}.json in the Bug Out items dir,
        # optionally linking a debug_dump path + correlation_id

Part D · The Salon (/opt/salon/)

The center of the multi-agent system: a SQLite database of live dialogues and their memory, plus the belief lookup that sets a persona's stance before it speaks.

salon.py — SQLite at /opt/salon/salon.db

Tables:
    dialogues:
        channel_id TEXT PRIMARY KEY,
        participants TEXT,          # JSON array of bot IDs
        topic TEXT,
        turn_count INTEGER DEFAULT 0,
        current_speaker TEXT,
        started_at REAL,
        last_activity REAL,
        hard_cap INTEGER DEFAULT 6,
        dialogue_type TEXT DEFAULT 'dialogue',
        starter_id TEXT DEFAULT ''

    dialogue_memory:
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        channel_id TEXT,
        speaker_name TEXT,
        message TEXT,
        topic TEXT,
        stance TEXT,
        timestamp REAL

Core functions:
    init_db()                                  # create tables if absent
    load_personas() -> dict                    # read personas.json
    fuzzy_match_persona(name) -> bot_id        # name/alias -> bot ID
    get_persona_by_bot_id(bot_id) -> dict
    get_dialogue(channel_id) -> dict | None
    start_dialogue(channel_id, participants, topic, starter_id) -> bool
    start_roundtable(channel_id, starter_id, topic) -> bool
    record_turn(channel_id, speaker_id) -> int
    should_end(dialogue) -> bool               # turn_count >= cap or stale
    end_dialogue(channel_id)
    store_turn(channel_id, speaker_name, message, topic, stance)
    get_dialogue_history(channel_id) -> list
    get_other_participant(dialogue, my_id) -> str
    pick_next_speaker(channel_id, current_id) -> str   # random, not self

Intent parsing (Claude Haiku):
    parse_intent(message, my_bot_id) -> dict
        # classify dialogue | roundtable | none
        # {"type": ..., "target": ..., "topic": ...}
    detect_roundtable_trigger(message) -> str | None
        # pattern-match "what do you think about X"

Belief-grounded dialogue:
    get_worldview_belief(persona_key, topic) -> dict | None
        # read worldviews/{persona_key}.json, find matching topic
    get_grounded_dialogue_context(my_persona_key, other_message,
                                  other_name, channel_id) -> dict
        # 1. extract topic from other_message (Claude Haiku)
        # 2. look up my belief (worldview JSON, else RAG)
        # 3. determine stance: AGREE | PARTIAL | DISAGREE | CURIOUS
        # 4. pull conversation history from dialogue_memory
        # 5. build prompt with voice + stance guidance + history
        # return {"prompt": ..., "topic": ..., "stance": ...}

Two small JSON files complete the Salon. personas.json is the registry the coordinator reads to route mentions and reach each RAG endpoint:

/opt/salon/personas.json

{
    "persona_key": {
        "bot_id": "<Discord bot user ID>",
        "aliases": ["alias1", "alias2"],
        "rag_endpoint": "http://localhost:{port}/api/query",
        "display_name": "Display Name"
    }
}

And a worldview file per persona: the fast belief lookup that lets a mind take a stance without a network call:

/opt/salon/worldviews/{persona_key}.json

{
    "topics": {
        "topic_name": {
            "belief": "What this persona believes about the topic",
            "stance": "curious | agree | disagree",
            "key_quote": "A relevant quote from their works"
        }
    }
}

Part E · Directories, permissions & verification

sudo mkdir -p /var/log/bot-debug /var/lib/voice-archive
sudo touch /var/log/bot-responses.jsonl
sudo chmod 666 /var/log/bot-responses.jsonl

VERIFICATION:
  # shared-config
  python3 -c "import sys; sys.path.insert(0,'/opt/shared-config');
              from paths import validate_paths; print(validate_paths())"
  # response-logger
  python3 -c "import sys; sys.path.insert(0,'/opt/response-logger');
              from logger import log_response;
              log_response('test','hello',channel='test')"
  tail -1 /var/log/bot-responses.jsonl
  # salon
  python3 -c "import sys; sys.path.insert(0,'/opt/salon');
              import salon; salon.init_db(); print('Salon DB initialized')"

KEY DESIGN DECISIONS:
  - Shared libs use sys.path.insert, not pip install (editable, simple)
  - Voice archive silently catches all exceptions (never breaks bot flow)
  - Response logger uses file locking for concurrent multi-bot writes
  - Salon uses SQLite for lightweight state management
  - Worldview JSON provides fast belief lookup without RAG API calls
  - Correlation IDs link logs across bots and debug dumps

This prompt creates one persona end to end: a ChromaDB knowledge base, a Flask RAG API, a Discord bot that speaks in character and joins Salon dialogues, plus its worldview and registration. Run it once per mind. Change the name, port, corpus, and voice.

You are building a persona bot backed by a ChromaDB knowledge base, Flask
RAG API, and Claude CLI for generation. The bot speaks in character,
retrieves context from embedded source material, and participates in
inter-bot dialogues via the salon system.

This prompt creates ONE persona. Repeat for each persona, changing names,
ports, corpus, and voice.

TARGET: /opt/virtual-{name}/   (e.g., /opt/virtual-gabi/)

PREREQUISITES:
- Ubuntu 24.04 with Python 3.12+
- Claude CLI at /usr/local/bin/claude
- OpenAI API key (for text-embedding-3-small)
- ChromaDB (pip install chromadb)
- Source corpus (text files, transcripts, books, etc.)
- Discord bot token
- Shared infrastructure deployed (Prompt 9)
- Salon system at /opt/salon/ (see Prompt 9)

Step 1 · Prepare & embed the corpus

Create a corpus directory: /opt/virtual-{name}/corpus/
Place source material as text/markdown files.

Create /opt/virtual-{name}/embed_documents.py:
    - Walk corpus directory for .txt, .md files
    - Chunk each file into ~1000-char segments with ~100-char overlap
      (use a persona-appropriate strategy — see the config table below)
    - Generate embeddings via OpenAI text-embedding-3-small
    - Store in ChromaDB at /var/lib/chroma/{name}/
    - Collection name: "{name}_knowledge"  (e.g., "gabriella_knowledge")
    - Metadata per chunk: source_file, chunk_index, doc_type
    - Use COSINE similarity space   # not the L2 default

Run the embedder:
    python3 embed_documents.py

Step 2 · The Flask RAG API (api.py)

Create /opt/virtual-{name}/api.py   (imports: os, sys, flask, chromadb, openai)

ENDPOINTS:
    GET  /api/health  -> {"status": "ok", "bot": "{name}"}

    POST /api/query
        Input: {"question": "...", "dialogue": bool}
        - Generate embedding for question via OpenAI
        - Query ChromaDB collection (n_results=5, or 3 if dialogue)
        - Format results with citations
        - Return: {"answer": "<formatted_context>", "sources": [...]}

    POST /api/context
        Input: {"question": "...", "n_results": 4}
        - Raw context retrieval for the voice pipeline
        - Return: {"chunks": [{"text": "...", "label": "...",
                   "relevance": 0.95}]}

    GET  /api/stats   -> {"total_documents": N, "bot": "{name}"}

Run on a localhost port (5050-5057, one per persona).

Step 3 · The Discord bot (bot.py)

Create /opt/virtual-{name}/bot.py
  imports: os, re, sys, json, subprocess, tempfile, logging, asyncio,
           discord, pathlib.Path, datetime, aiohttp

Shared libs:
    sys.path.insert(0, '/opt/shared-config')
    sys.path.insert(0, '/opt/response-logger')
    sys.path.insert(0, '/opt/salon')
    sys.path.insert(0, '/opt/presence-lib')
    from paths import CLAUDE_CLI, whisper_cmd
    from logger import log_response, file_bug, dump_debug, ...
    import salon
    from presence import start_presence_loop, get_random_quote

CONFIGURATION:
    MY_BOT_ID = "<this bot's Discord ID>"
    LOUNGE_CHANNEL_ID = <shared channel ID>

SYSTEM PROMPT (build_system_prompt):
    - Define persona identity, voice, and knowledge domain
    - Include a data summary (total docs, doc types, domains)
    - Instruct Claude to search via ```search ... ``` code blocks
    - Set response guidelines (length, tone, citation style)
    - "Never start with interjections like 'Ah' or 'Oh'"

RAG SEARCH FLOW (ask_claude) — the two-pass pattern:
    1. Build system prompt + conversation history
    2. First Claude call: may emit ```search\nquery: ...\ntype: ...\n```
    3. process_claude_output() detects search blocks, runs them against
       ChromaDB, collects the results
    4. If searches ran: second Claude call with "[search results]" appended
    5. Return the final synthesized response

DIALOGUE HANDLING (in on_message):
    Case 1 - Active dialogue (responding to a partner bot):
        - salon.get_dialogue(channel_id); if from partner, record turn
        - salon.get_grounded_dialogue_context() gives stance + history
        - mention partner to continue; end when salon.should_end()
    Case 2 - Roundtable (multi-bot discussion):
        - dialogue_type == "roundtable"; record turn, respond with stance
        - salon.pick_next_speaker(); end at hard_cap (10 turns)
    Case 3 - Dialogue trigger (user wants bots to talk):
        - detect other-bot mentions or parse intent
        - salon.start_dialogue(channel_id, [my_id, target_id], topic)
        - also detect "what do you think about X" for a roundtable
    Case 4 - Normal Q&A:
        - add_to_conversation(user_id, "user", question)
        - response = ask_claude(user_id, question)
        - add_to_conversation(user_id, "assistant", response)
        - split and send to Discord

VOICE (transcribe_voice):
    - Standard Whisper transcription via whisper_cmd()
    - Archive via voice_archive.archive_voice()

PRESENCE:
    - start_presence_loop(bot, get_random_quote('{name}.txt'))

Step 4-5 · Worldview & Salon registration

Step 4 — Create /opt/salon/worldviews/{name}.json:
    {
        "topics": {
            "consciousness": {
                "belief": "Consciousness emerges from...",
                "stance": "curious",
                "key_quote": "..."
            },
            "technology": { ... }
        }
    }
    # used by salon.get_grounded_dialogue_context() for fast belief
    # lookup without querying the RAG API

Step 5 — Register in /opt/salon/personas.json:
    "{name}": {
        "bot_id": "<Discord bot ID>",
        "aliases": ["alias1", "alias2"],
        "rag_endpoint": "http://localhost:{port}/api/query",
        "display_name": "Display Name"
    }

Step 6-8 · Services, quotes & start

Step 6 — systemd services:
    /etc/systemd/system/virtual-{name}.service        (Flask API)
        ExecStart=/opt/virtual-{name}/venv/bin/python api.py
    /etc/systemd/system/virtual-{name}-bot.service    (Discord bot)
        After=virtual-{name}.service
        ExecStart=/opt/virtual-{name}/venv/bin/python bot.py

Step 7 — quotes file:
    /opt/presence-lib/quotes/{name}.txt
    One quote per line (from the persona's works / philosophy).

Step 8 — enable and start both services.

Verification

1. curl http://localhost:{port}/api/health   -> {"status": "ok"}
2. curl -X POST http://localhost:{port}/api/query \
        -H 'Content-Type: application/json' \
        -d '{"question": "test topic"}'       -> relevant chunks
3. DM the bot on Discord   -> responds in character with citations
4. @mention two bots together in a channel   -> starts a dialogue
5. Say "what do you think about consciousness?" in Lounge -> roundtable
6. Check /var/log/bot-responses.jsonl for logged entries

Key design decisions

- Two-pass RAG: Claude decides what to search, then synthesizes results
- ChromaDB + OpenAI embeddings (text-embedding-3-small) for vectorization
- Persona voice enforced via system prompt, not fine-tuning
- Salon manages turn-taking with hard caps (6 dialogue, 10 roundtable)
- Worldview JSON provides fast belief lookup for dialogue stance
- Separate API service allows independent scaling and restart

Prompt 3 runs once per mind. The table below shows what differs between runs: the corpus, the chunk count, the port, and the special handling that the source material needs. The values come from the running ensemble.

Persona	Port	Chunks	Corpus	Chunking / special handling
Gabriella 2.0	5050	430	12 session transcripts + 30 book-chapter audio transcripts + Q&A pairs	Standard prose (750-token target / 1,000 max / 100 overlap); most mature groundedness scoring
Benjamin Franklin	5051	173	Autobiography + Poor Richard's Almanack	Standard prose chunking; dialogue leader (initiates salon dialogues)
Leonardo da Vinci	5052	1,634	Richter's Notebooks (22 sections)	Smaller chunks (600 / 800 / 75) for fragmented notebook text; pairs replies with works from his image catalog
Carl Sagan	5053	402	Cosmos + 13 TV episode transcripts	Standard prose chunking; persona rule: never mention being an AI or anything post-1996
Decode	5054	46,912	83 courses, 1,030 papers, 815 case studies, industry analyses (~1.6 GB)	Agentic two-pass RAG (one similarity query cannot span the corpus); pins claude-sonnet-4-6
Emily Dickinson	5055	629 poems	629 complete poems (10,694 lines)	Whole-document, never chunked; retrieves and quotes actual poems verbatim. The purest case of the thesis
Sir David Attenborough	5056	1,170	626,467 words from 20 documentary / book sources	TopicAwareChunker follows documentary topic shifts; de-vendored to local nomic-embed-text-v1.5 + F5-TTS
Walter Russell	5057	1,371	Nine books + the 12-unit Home Study Course (664,758 words, 21 sources)	Genre-aware chunking (700 / 1,000 / 100) with per-chunk contextual headers; 18 sources OCR-remediated at intake (see The Corpus); 13 curated study-guide digests answer broad survey questions (see Architecture); voice cloned from a 1953 lecture tape
Mark Twain (in the vault)	n/a	6,575	Fully indexed (143 MB)	Complete embedding pipeline and vector DB exist; no bot or salon registration yet, the next mind waiting to wake up