Mini's World

AI agent platform where Minis (AI employees) work inside company workspaces. Talk to them by voice or text, make AI-powered phone calls, run multi-step agentic workflows with human approval, execute trades, generate images, send emails, build websites, produce podcasts, automate browser tasks, sign documents, schedule recurring workflows, and more — 62 tool integrations, zero build step, enterprise-grade security.

15
AI Minis
62
Tools
195+
API Endpoints
37
Route Files
70
UI Components
31
Security Controls

Quick Start

# 1. Clone and install
git clone <repo-url> && cd minisworld
npm install

# 2. Set required env vars
cp .env.example .env
# Edit .env: OPENAI_API_KEY (required), SESSION_SECRET, JWT_SECRET

# 3. Start
npm run dev          # development (file watching)
npm start            # production
npm test             # run 815+ unit tests

# 4. Open
open http://localhost:4040

Required Environment Variables

VariablePurpose
OPENAI_API_KEYOpenAI API key (chat, embeddings, DALL-E, realtime)
SESSION_SECRETSession cookie signing (change from default in prod)
JWT_SECRETHMAC-SHA256 token signing (change from default in prod)

All other variables are optional — tools auto-disable when their credentials are missing.

What It Is

Mini's World is a full-stack AI automation platform built around Minis — 15 specialized AI employees, each with a distinct role, personality, knowledge base, and assigned tools. Minis live inside Companies (isolated workspaces) where they collaborate through Workflows, respond to Triggers, execute on Schedules, and use Tools (62 integrations).

Users interact with Minis through text chat (HTTP + SSE streaming, default), voice chat (WebSocket relay to OpenAI Realtime API), or phone calls (Twilio Media Streams + OpenAI Realtime, sub-second latency).

Core Primitives

PrimitiveDefinition
MiniAn AI employee with a role, personality, voice, knowledge base, and assigned tools
CompanyAn isolated workspace with its own contacts, memories, workflows, and tool configurations
WorkflowA multi-step orchestrated task that passes through a state machine with human approval gates
ToolAn integration (email, social, CRM, etc.) executed by Minis — 62 total across 3 tiers
TriggerAn event listener that auto-creates workflows when matching events arrive
ScheduleA cron-based timer that executes workflows on a recurring basis

High-Level Architecture

┌─────────────────────────────────────────────────────────────────────────────┐ │ BROWSER (Client) │ │ main.js → UI Components (70 files) → Stores (26 fetch wrappers) │ │ RealtimeClient (WS) · AudioCapture/Player (PCM16) · CredentialVault │ │ EventBus (pub/sub) · Analytics (PostHog) │ └────────────┬──────────────────┬──────────────────┬─────────────────────────┘ │ HTTP/SSE │ WebSocket │ WebSocket │ (text chat) │ (voice chat) │ (phone calls) ▼ ▼ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ EXPRESS SERVER (:4040) │ │ │ │ ┌──────────┐ ┌──────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐ │ │ │ 37 Route │ │ AI │ │ Tool │ │ Workflow │ │ Storage │ │ │ │ Files │ │ Module │ │ System │ │ Engine │ │ Layer │ │ │ │ 195+ EP │ │ gpt-4.1 │ │ 62 tools │ │ State │ │ Per-user │ │ │ │ SSE+REST │ │ embed │ │ 3-tier │ │ Machine │ │ Per-co. │ │ │ │ │ │ DALL-E │ │ routing │ │ Approval │ │ JSON/JSONL│ │ │ └──────────┘ └──────────┘ └───────────┘ └───────────┘ └───────────┘ │ │ │ │ Middleware: Helmet · CORS · CSRF · Rate Limit · Auth · Path Sanitize │ │ Background: TimeoutChecker · ScheduleChecker · GmailPoller · CalendarPoller│ │ WebSocket: RealtimeRelay (voice) · CallRelay (Twilio) │ └────────────┬──────────────────┬──────────────────┬─────────────────────────┘ │ │ │ ▼ ▼ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ EXTERNAL SERVICES │ │ │ │ OpenAI (Chat/Realtime/DALL-E/Embeddings) · Twilio (Calls+SMS) │ │ Recall.ai (Meeting Bot) · Microsoft Teams Bot · PostHog Analytics │ │ Google OAuth · Microsoft OAuth · Stripe · LinkedIn (Personal+Pages) │ │ Facebook/Instagram · Slack · GitHub · Notion · HubSpot │ │ Shopify · Apollo.io · Salesforce · HeyGen · Cloudinary │ │ Browserless.io · Vercel · 34 MCP Servers │ └─────────────────────────────────────────────────────────────────────────────┘

Connection types: HTTP (routes) · SSE (chat streaming) · WSS (voice/calls) · Webhooks (Twilio, Stripe, Teams, triggers)

Sequence: Text Chat

User ──► Client ──[POST /api/chat/:cid/:aid/send]──► Server │ embed(query) searchMemories() searchKnowledge() buildSystemPrompt() │ ┌─────────▼─────────┐ │ OpenAI gpt-4.1 │ │ (tool-call loop) │◄──┐ └─────────┬─────────┘ │ │ │ [tool_calls?]────► execute tools │ (max 10 iter) [finish: stop] │ SSE events: thinking → tool_start → tool_done → done │ User ◄── Client ◄──[SSE stream]──────────────────── Server

Sequence: Workflow Execution

User creates workflow ──► PENDING │ executeWorkflow() │ ┌─────────▼─────────┐ │ RUNNING │ │ callMiniAsAgent() │◄──── revision loop (max 2) └─────────┬─────────┘ │ ┌─────────▼─────────┐ │ AWAITING_HUMAN │ │ (approval card) │ └───┬───────────┬────┘ approve reject │ │ ▼ ▼ EXECUTING CLOSED │ callMiniForFinalAction() │ ▼ COMPLETED

Project Structure

agents/ — 15 AI agent configurations
agents/
├── founder/config.js          # Co-Founder & Vision
├── developer/config.js        # Lead Engineer
├── marketing/config.js        # Head of Growth
├── designer/config.js         # Head of Design
├── ops/config.js              # Head of Operations
├── research/config.js         # Head of Research
├── copywriter/config.js       # Head of Copy
├── outreach/config.js         # Head of Outreach
├── client/config.js           # Client Success Lead
├── analyst/config.js          # Head of Analytics
├── trading/config.js          # Head of Trading
├── crypto/config.js           # Head of Crypto Trading
├── studio/config.js           # Podcast Producer
├── vision/config.js           # Browser Automation
├── aria/config.js             # Executive Assistant
└── <id>/knowledge/*.md        # Per-agent knowledge bases
client/ — Vanilla JS browser app (no framework, no build)
client/
├── index.html                 # Entry point (33 CSS, loads main.js)
├── src/main.js                # Bootstrap: auth, company sync, routing
├── src/ui/                    # 70 UI components (vanilla JS classes)
│   ├── pages/                 # CompanyHub, CompanyWorkspace, LoginPage, etc.
│   ├── chat/                  # VoiceInterface, Transcript, CallPanel
│   ├── workflows/             # WorkflowLauncher, WorkflowPage, HumanReviewModal
│   ├── tools/                 # ToolSettings, ToolSetupPanel
│   ├── features/              # CampaignManager, PodcastCard, WebsitePreview
│   ├── trading/               # TradingSettings, TradeCard
│   ├── admin/                 # AuditDashboard, ImportPanel, ExportPanel
│   ├── common/                # MiniCard, Toast, ConfirmDialog, MiniAvatar
│   └── onboarding/            # OnboardingFRE, ApiKeyModal
├── src/store/                 # 26 fetch wrappers
├── src/styles/                # 33 CSS files (glass morphism theme)
├── src/realtime/              # RealtimeClient (WS), EventBus
├── src/audio/                 # AudioCapture (PCM16 mic), AudioPlayer
├── src/crypto/                # credentialVault.js (AES-256-GCM)
└── src/api/                   # agentApi.js, sessionApi.js
server/ — Express + WebSocket server
server/
├── index.js                   # Boot: register agents, mount routes, start WS
├── config.js                  # Zod-validated env vars (80+ variables)
├── logger.js                  # Winston logger
├── agentRegistry.js           # Agent Map + prompt builders
├── routes/                    # 37 Express router files (195+ endpoints)
├── ai/                        # openaiClient, completionLoop, guardrails, usageTracker
├── workflows/                 # workflowEngine, miniOrchestrator, timeoutChecker
├── tools/                     # toolDefinitions (62), local/ (15), api/ (13), toolConfigStore
├── ws/                        # realtimeRelay (voice), callRelay (Twilio)
├── auth/                      # 8 OAuth providers, userStore
├── middleware/                # security.js (Helmet, CORS, CSRF, rate limit)
├── memory/                    # store.js (per-company embeddings + cosine search)
├── triggers/                  # eventListener, gmailPoller, calendarPoller
├── schedules/                 # scheduleStore, scheduleChecker, cronParser
├── mcp/                       # mcpClient (3-tier routing), mcpServerManager
├── stores/                    # BaseStore (PerUserStore, PerCompanyStore, GlobalStore)
├── studio/                    # Podcast pipeline (HeyGen → DALL-E → FFmpeg)
├── companies/                 # companyContextStore, knowledgeStore
├── contacts/                  # contactStore (per-company)
├── trading/                   # tradingStore, capitalRules, indicators
├── calls/                     # callStore, callTelemetry
├── gallery/                   # galleryStore, imageResolver
├── integrations/teams/        # Microsoft Teams bot
└── telemetry/                 # telemetryStore, telemetryMiddleware

Core Systems

🧠 RAG & Prompt Engineering

7-layer prompt pipeline: agent personality → company context → training data → knowledge docs (RAG, cosine > 0.4) → contacts → memory → tool definitions. Embeddings: text-embedding-3-small (1536 dim).

⚙️ Workflow Engine

State machine: PENDING → RUNNING → AWAITING_HUMAN → APPROVED → EXECUTING → COMPLETED. Safety: 10-min timeout, max 2 revisions, 100 iteration loop cap, approval gating.

🔧 Tool System (3-Tier)

Local executors (15) → API tools (13) → MCP servers (34). Per-user defaults + per-company overrides. AES-256-GCM encrypted credentials with scrypt key derivation.

📞 Phone Calls

Twilio Media Streams + OpenAI Realtime API. Sub-second latency, barge-in support, hold tone during tool execution, live whisper injection, call transcripts.

🎙️ Podcast Studio

Pipeline: AI script → HeyGen avatar video → DALL-E cover art → FFmpeg assembly → YouTube auto-publish. SSE progress streaming throughout.

🌐 Website Builder

Natural language → HTML generation → live preview → one-click deploy (Vercel or self-hosted with Nginx subdomain routing).

🎯 Lead Generation

Apollo.io + Google Maps + Hunter.io for lead sourcing. Email verification, lead scoring, 4-email campaign sequences with reply tracking.

🖼️ Image Gallery

DALL-E 3 generation with prompt tracking. [[IMG-1035]] inline reference syntax in chat. Tags, favorites, search, archive.

📋 Meeting Bot

Recall.ai integration — join Zoom, Google Meet, or Teams meetings. Listen, take notes, provide transcripts.

💬 Microsoft Teams

Direct agent interaction in Teams. JWT-verified webhooks, Adaptive Cards for approvals, proactive notifications.

🗓️ Schedule System

Cron-based recurring workflows. Natural language parsing ("every Monday at 9 AM" → cron). One-time and trigger-now execution.

🔔 Trigger System

Event-driven automation. Types: email, call, message, form, payment, webhook. Modes: once (fire and complete) or continuous.

✦ Avatar (AI Self)

Persistent, always-on super-entity above all companies. Dual-model: Claude for deep reasoning, GPT-4o for speed. Cross-company monitoring (5-min cycles), morning briefings (chat/email/phone), opportunity research, draft company builder, data-grounded conversation.

📊 Eval System (LLM-as-Judge)

Production-grade evaluation scoring every AI interaction on 8 dimensions. Failure categorization (RAI refusal, tool error, hallucination, context gap). Admin dashboard with trace detail, filters, re-run. Clickable trace explorer.

Agent Roster

IDNameRoleVoiceKey Tools
founderFounder MiniCo-Founder & Visioncorallinkedin, gmail, slack, calendar, notion, minisign
developerDev MiniLead Engineerechogmail, github, azure_devops, website_build
marketingMarketing MiniHead of Growthcorallinkedin, gmail, image_gen, twitter, canva, mailchimp
designerDesign MiniHead of Designalloyimage_gen, figma, canva, notion
opsOps MiniHead of Operationssagegmail, whatsapp, slack, twilio, shopify, gorgias
researchResearch MiniHead of Researchfableweb_search, apollo, brave, puppeteer, reddit
copywriterCopy MiniHead of Copynovagmail, image_gen, wordpress, notion, document
outreachOutreach MiniHead of Outreachonyxgmail, linkedin, hubspot, salesforce, lead_gen
clientClient MiniClient Success Leadjunipergmail, stripe, shopify, gorgias, mailchimp
analystAnalyst MiniHead of Analyticsembergmail, sheets, airtable, search_console, document
tradingTrading MiniHead of Tradingechoalpaca_trading, web_search
cryptoCrypto MiniHead of Cryptoonyxalpaca_crypto, web_search, brave_search
studioStudio MiniPodcast Produceralloyheygen, youtube, image_gen, web_search
visionVision MiniBrowser Automationechovision_browse, web_search, document
ariaAriaExecutive Assistantsagegmail, calendar, slack, twilio, meeting_bot

Environment Variables

Core AI & Models
VariableDefaultRequiredDescription
OPENAI_API_KEYYesOpenAI API key
OPENAI_CHAT_MODELgpt-4.1NoText chat & workflow model
OPENAI_CHAT_MODEL_MINIgpt-4.1-miniNoLighter model for specific tasks
OPENAI_RT_MODELgpt-realtime-1.5NoVoice chat (realtime API)
OPENAI_CALL_RT_MODELgpt-4o-mini-realtime-previewNoPhone call relay model
ANTHROPIC_API_KEYNoAnthropic API key (Avatar deep reasoning via Claude)
AVATAR_ENABLEDtrueNoEnable Avatar background monitoring
Server
VariableDefaultDescription
PORT4040HTTP port
NODE_ENVdevelopmentdevelopment | production | test
DATA_DIRPersistent data directory (for ephemeral platforms)
ALLOWED_ORIGINSlocalhost:5173,4040CORS origins (CSV)
LOG_LEVELinfoerror | warn | info | debug
SESSION_SECRETchange-me…Session cookie signing
JWT_SECRETchange-me…Token signing (HMAC-SHA256)
ENCRYPTION_SECRETCredential encryption (falls back to JWT_SECRET)
MAX_CONCURRENT_SESSIONS50Max WebSocket sessions
APP_BASE_URLhttps://minisworld.aiPublic URL
Rate Limiting & Security
VariableDefaultDescription
RATE_LIMIT_SESSION_PER_MIN10Session endpoint rate limit
RATE_LIMIT_WS_PER_MIN10000WebSocket messages/min
LOGIN_ALERT_THRESHOLD5Failed logins before alert
INVITE_ONLYtrueRequire invite for signup
ADMIN_EMAILAdmin dashboard access
MAX_HOURLY_COST2 (USD)AI spend circuit breaker
OAuth & Integrations (Google, Microsoft, LinkedIn, Twilio, etc.)
CategoryVariables
Google OAuthGOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET
MicrosoftMICROSOFT_CLIENT_ID, MICROSOFT_APP_ID, MICROSOFT_APP_SECRET, MICROSOFT_TENANT_ID
LinkedInLINKEDIN_CLIENT_ID/SECRET (personal), LINKEDIN_PAGES_CLIENT_ID/SECRET (pages)
WhatsAppWHATSAPP_PHONE_NUMBER_ID, WHATSAPP_ACCESS_TOKEN
TwilioTWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_FROM_NUMBER
Social & CRMFACEBOOK_APP_*, SLACK_CLIENT_*, GITHUB_CLIENT_*, NOTION_OAUTH_*, HUBSPOT_CLIENT_*
MediaHEYGEN_API_KEY, CLOUDINARY_*, YOUTUBE_CLIENT_*
TradingALPACA_API_KEY, ALPACA_SECRET_KEY, ALPACA_PAPER_MODE
Lead GenAPOLLO_API_KEY, GOOGLE_MAPS_API_KEY, HUNTER_API_KEY
PaymentsSTRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET
HostingSELF_HOSTING, BASE_DOMAIN, VERCEL_API_TOKEN, VERCEL_TEAM_ID
VisionVISION_MINI_ENABLED, VISION_MINI_MAX_STEPS, BROWSERLESS_API_KEY
PollingPOLL_INTERVAL_MS (5s), GMAIL_POLL_INTERVAL_MS (30s), CALENDAR_POLL_INTERVAL_MS (60s)
AnalyticsPOSTHOG_API_KEY, POSTHOG_HOST, RECALL_AI_API_KEY

API Reference

Core Chat & Voice
MethodPathDescription
POST/api/chat/:companyId/:agentId/sendText chat with SSE streaming + tool-call loop
POST/api/chat/:companyId/:agentId/approve-actionApprove pending action (trades, emails)
POST/api/voice/:companyId/:agentId/sendVoice chat (Whisper STT → GPT → TTS)
GET/api/agentsList all agents
GET/api/agents/:idGet agent details
POST/api/sessionGet WebSocket session path
Workflows
MethodPathDescription
POST/api/workflowsCreate and start workflow
GET/api/workflowsList workflows (?companyId=)
GET/api/workflows/:idGet single workflow
POST/api/workflows/:id/human-decisionSubmit approval/rejection
POST/api/workflows/:id/rerunRerun completed workflow
POST/api/workflows/:id/pausePause running workflow
POST/api/workflows/:id/resumeResume paused workflow
PUT/api/workflows/:idUpdate workflow fields
DEL/api/workflows/:idDelete workflow
Tools & Credentials
MethodPathDescription
GET/api/tools/catalogTool catalog (?companyId= for override status)
GET/api/tools/credentials/:toolIdMasked user-level credentials
PUT/api/tools/credentials/:toolIdSave user-level credentials
POST/api/tools/credentials/:toolId/validateTest API connection
GET/api/tools/:companyId/:agentIdGet Mini's assigned tools
PUT/api/tools/:companyId/:agentIdUpdate tool assignments
Companies, Contacts & Context
MethodPathDescription
GET/api/companiesList companies
PUT/api/companies/:idUpsert company
DEL/api/companies/:idDelete company (cascade)
GET/api/contacts/:companyIdList/search contacts
POST/api/contacts/:companyIdCreate contact
POST/api/contacts/:companyId/importImport CSV/vCard
GET/api/company-context/:companyIdGet company context
POST/api/company-context/:companyId/knowledgeUpload knowledge document
Triggers, Schedules & Webhooks
MethodPathDescription
GET/api/triggersList triggers (?companyId=)
POST/api/triggersCreate trigger
POST/api/triggers/:id/toggleToggle active/paused
GET/api/schedulesList schedules
POST/api/schedulesCreate schedule
POST/api/schedules/:id/trigger-nowExecute immediately
POST/api/schedules/parse-timingNatural language → cron
POST/wh/form/:companyIdForm submission webhook
POST/wh/payment/:companyIdPayment webhook
Trading, Gallery, Studio & Features
MethodPathDescription
POST/api/trading/:companyId/credentialsSet Alpaca keys
GET/api/trading/:companyId/settingsCapital rules, watchlist
GET/api/galleryList images
POST/api/gallery/uploadUpload image
GET/api/studio/:companyId/episodesList podcast episodes
POST/api/vision/runStart browser automation
GET/api/campaignsList campaigns
GET/api/websites/:companyIdList web projects
Admin (Admin Only)
MethodPathDescription
GET/api/admin/telemetryFull metrics
GET/api/admin/ai-usageAI usage by period
GET/api/admin/ai-usage/by-userPer-user breakdown
POST/api/admin/ai-usage/reset-haltReset cost breaker
GET/api/admin/usersList all users
POST/api/admin/invites/generateGenerate invite codes
Avatar
MethodPathDescription
GET/api/avatarFull avatar config (creates default if missing)
POST/api/avatar/configUpdate avatar config (partial)
POST/api/avatar/chatChat with Avatar
POST/api/avatar/chat/streamSSE streaming chat
GET/api/avatar/briefingGet/generate briefing
POST/api/avatar/briefing/deliverTrigger briefing delivery
GET/api/avatar/portfolioCross-company portfolio snapshot
GET/api/avatar/contactsMerged contacts across companies
GET/api/avatar/pendingPending approvals across companies
POST/api/avatar/researchRun opportunity research pipeline
POST/api/avatar/build-companyBuild draft company from candidate
GET/api/avatar/historyConversation history
GET/api/avatar/connected-channelsCheck Gmail/Twilio connection status
POST/api/avatar/actionDirect action execution
Eval (Admin)
MethodPathDescription
GET/api/eval/tracesList traces (filters: company_id, failure_category)
GET/api/eval/traces/:traceIdGet full trace detail (global search)
POST/api/eval/traces/:traceId/run-evalRe-run judge evaluation
GET/api/eval/statsAggregate stats + failure category breakdown
GET/api/eval/admin-companiesAdmin's own companies
Chat History & Import/Export
MethodPathDescription
GET/api/chat-history/:companyId/:agentIdGet history (paginated)
POST/api/chat-history/:companyId/:agentIdAppend message
DEL/api/chat-history/:companyId/:agentIdClear history
GET/api/import-export/export/workflow/:idExport workflow JSON
POST/api/import-export/import/workflowImport workflow

Server Boot Sequence

  1. Load & validate config (Zod schema)
  2. Register all 15 agents + preload knowledge
  3. Ensure data directory exists
  4. Load all stores from disk (users, invites, workflows, triggers, schedules, trading, telemetry, etc.)
  5. Mount middleware: requestId → CSP nonce → Helmet → CORS → JSON → telemetry → sanitizeParams
  6. Mount pre-CSRF routes: Stripe webhook, Teams webhook, Twilio webhooks, trigger webhooks
  7. Mount CSRF protection
  8. Mount auth routes (public) + all protected /api/* routes
  9. Mount static file serving (client/)
  10. Create WebSocket servers: RealtimeRelay (/ws/session) + CallRelay (/media-stream)
  11. Start HTTP server on PORT
  12. Launch background tasks: timeoutChecker, scheduleChecker, gmailPoller, calendarPoller, reminderChecker, cleanupTimer
  13. Start Avatar monitor (5-min cycle for all users with Avatar enabled)
  14. Initialize MCP server connections (non-fatal)
  15. Cleanup orphaned triggers/schedules from deleted companies

Scripts & Dependencies

Commands

CommandDescription
npm run devStart with --watch (auto-restart on changes)
npm startProduction start
npm testRun all unit tests (Node.js built-in test runner)

Key Dependencies

PackagePurpose
expressHTTP server + routing
wsWebSocket (voice + calls)
googleapisGoogle APIs (Gmail, Calendar, YouTube)
helmetSecurity headers
corsCross-origin resource sharing
express-rate-limitRate limiting
winstonLogging
zodConfig validation
dotenvEnvironment variables
@modelcontextprotocol/sdkMCP server integration
@anthropic-ai/sdkAnthropic Claude API (Avatar deep reasoning)

Conventions

Security (31 Controls)

🔐 Authentication

HMAC-SHA256 tokens, httpOnly/Secure/SameSite cookies, OAuth 2.0 + PKCE, 8 providers

🛡️ CSRF

Double-submit cookie with timing-safe comparison

🔒 XSS Prevention

CSP with per-request nonce, Helmet security headers, HSTS

🗝️ Cryptography

AES-256-GCM (scrypt N=16384), per-credential salt+IV

🤖 AI Safety

Prompt injection detection, tool argument validation, daily rate limits, output filtering, cost circuit breaker

📦 Data Isolation

Per-user directories, per-company PerCompanyStore, encrypted credentials

See Security Reference for all 31 verified controls.

Documentation