Data Models & Database Architecture

Overview

Bike4Mind uses MongoDB as its primary database with a rich, well-structured data model designed to support AI agents, user collaboration, and complex workflows. The system follows a repository pattern with clear separation between data models and business logic.

Database Architecture

Connection & Configuration

• Database: MongoDB (managed via connection string)
• Connection Management: Mongoose ODM with connection pooling
• Environment: Configurable via MONGODB_URI secret
• Migrations: Automated schema updates via SST scripts

Repository Pattern

// Repository Structure
b4m-core/packages/core/database/
├── src/models/           # Mongoose models and schemas
├── src/utils/           # Database utilities and helpers
└── index.ts            # Exported models and repositories

Core Data Models

User Management

UserModel

interface IUser {
  _id: ObjectId;
  username: string;
  email: string;
  password?: string;           // Hashed password for local auth
  isAdmin: boolean;
  isActive: boolean;
  
  // OAuth Integration
  googleId?: string;
  githubId?: string;
  oktaId?: string;
  
  // Profile Information
  firstName?: string;
  lastName?: string;
  avatar?: string;
  timezone?: string;
  
  // Organization & Permissions
  organizationId?: ObjectId;
  roles: string[];
  permissions: string[];
  
  // Usage & Analytics
  counters: {
    sessions: number;
    quests: number;
    filesUploaded: number;
    creditsUsed: number;
  };
  
  // Preferences
  preferences: {
    theme: 'light' | 'dark';
    language: string;
    notifications: {
      email: boolean;
      slack: boolean;
      push: boolean;
    };
    ai: {
      preferredModel: string;
      autonomyLevel: 'low' | 'medium' | 'high';
      confidenceThreshold: number;
    };
  };
  
  createdAt: Date;
  updatedAt: Date;
  lastLoginAt?: Date;
}

OrganizationModel

interface IOrganization {
  _id: ObjectId;
  name: string;
  slug: string;
  description?: string;
  
  // Billing & Subscription
  stripeCustomerId?: string;
  subscriptionStatus: 'active' | 'inactive' | 'trial' | 'cancelled';
  subscriptionTier: 'free' | 'pro' | 'enterprise';
  
  // Settings
  settings: {
    allowGuestAccess: boolean;
    defaultUserRole: string;
    dataRetentionDays: number;
    aiModelsEnabled: string[];
  };
  
  // Usage Limits
  limits: {
    maxUsers: number;
    maxStorage: number;
    maxCreditsPerMonth: number;
  };
  
  createdAt: Date;
  updatedAt: Date;
}

AI Agent Models

AgentModel

interface IAgent {
  _id: ObjectId;
  name: string;
  description?: string;
  userId: ObjectId;
  organizationId?: ObjectId;
  
  // Agent Configuration
  config: {
    model: string;              // Primary LLM model
    fallbackModel?: string;     // Fallback LLM model
    temperature: number;        // Creativity/randomness
    maxTokens: number;         // Response length limit
    systemPrompt?: string;     // Custom system instructions
  };
  
  // Capabilities & Tools
  capabilities: string[];       // Available tool categories
  enabledTools: string[];      // Specific enabled tools
  mcpServers: ObjectId[];      // Connected MCP servers
  
  // Autonomy Settings
  autonomy: {
    level: 'low' | 'medium' | 'high';
    confidenceThreshold: number;
    requireApproval: string[];   // Actions requiring approval
    autoExecute: string[];       // Actions to execute automatically
  };
  
  // Performance Metrics
  metrics: {
    totalQuests: number;
    successfulQuests: number;
    averageConfidence: number;
    userSatisfaction: number;
  };
  
  isActive: boolean;
  createdAt: Date;
  updatedAt: Date;
}

QuestModel

interface IQuest {
  _id: ObjectId;
  title: string;
  description?: string;
  
  // Ownership & Context
  userId: ObjectId;
  sessionId?: ObjectId;
  agentId?: ObjectId;
  organizationId?: ObjectId;
  
  // Quest Execution
  status: 'pending' | 'running' | 'completed' | 'failed' | 'cancelled';
  priority: 'low' | 'medium' | 'high' | 'urgent';
  
  // AI Processing
  model: string;               // LLM model used
  prompt: string;             // Original user prompt
  systemPrompt?: string;      // System instructions
  
  // Results & Output
  result?: string;            // Final result
  artifacts: {                // Generated artifacts
    type: 'text' | 'image' | 'file' | 'data';
    content: string;
    metadata?: any;
  }[];
  
  // Execution Tracking
  steps: {
    name: string;
    status: 'pending' | 'running' | 'completed' | 'failed';
    startTime?: Date;
    endTime?: Date;
    result?: string;
    error?: string;
  }[];
  
  // Performance Metrics
  metrics: {
    executionTime: number;      // Total execution time (ms)
    tokensUsed: number;        // Total tokens consumed
    toolsCalled: number;       // Number of tools used
    confidenceScore: number;   // AI confidence in result
  };
  
  // Human Interaction
  humanReviewRequired: boolean;
  humanFeedback?: {
    approved: boolean;
    comments?: string;
    modifications?: string;
  };
  
  createdAt: Date;
  updatedAt: Date;
  completedAt?: Date;
}

QuestMasterPlanModel

interface IQuestMasterPlan {
  _id: ObjectId;
  questId: ObjectId;
  
  // Plan Structure
  plan: {
    goal: string;
    approach: string;
    steps: {
      id: string;
      name: string;
      description: string;
      dependencies: string[];    // Step IDs this depends on
      tools: string[];          // Tools required for this step
      estimatedTime: number;    // Estimated execution time
      riskLevel: 'low' | 'medium' | 'high';
    }[];
  };
  
  // Execution State
  currentStep?: string;
  completedSteps: string[];
  failedSteps: string[];
  
  // Approval & Review
  requiresApproval: boolean;
  approvedBy?: ObjectId;
  approvedAt?: Date;
  
  createdAt: Date;
  updatedAt: Date;
}

Session & Memory Models

SessionModel

interface ISession {
  _id: ObjectId;
  title?: string;             // Auto-generated or user-defined
  
  // Ownership & Context
  userId: ObjectId;
  organizationId?: ObjectId;
  projectId?: ObjectId;
  
  // Session Configuration
  model: string;              // Primary LLM model
  systemPrompt?: string;      // Session-specific instructions
  
  // Conversation History
  messages: {
    role: 'user' | 'assistant' | 'system';
    content: string;
    timestamp: Date;
    metadata?: {
      tokensUsed?: number;
      model?: string;
      tools?: string[];
      confidence?: number;
    };
  }[];
  
  // Session State
  status: 'active' | 'archived' | 'deleted';
  isStarred: boolean;
  tags: string[];
  
  // Analytics
  metrics: {
    messageCount: number;
    totalTokens: number;
    averageResponseTime: number;
    toolsUsed: string[];
  };
  
  // Auto-generated Content
  summary?: string;           // AI-generated session summary
  keyTopics: string[];       // Extracted key topics
  
  createdAt: Date;
  updatedAt: Date;
  lastMessageAt?: Date;
}

MementoModel

interface IMemento {
  _id: ObjectId;
  title: string;
  content: string;
  
  // Ownership & Context
  userId: ObjectId;
  organizationId?: ObjectId;
  sessionId?: ObjectId;
  projectId?: ObjectId;
  
  // Content Classification
  type: 'note' | 'insight' | 'fact' | 'procedure' | 'reference';
  category?: string;
  tags: string[];
  
  // AI Enhancement
  summary?: string;           // AI-generated summary
  keyPoints: string[];       // Extracted key points
  relatedTopics: string[];   // AI-identified related topics
  
  // Vector Search
  embedding?: number[];       // Vector embedding for semantic search
  embeddingModel?: string;   // Model used for embedding
  
  // Metadata
  source?: {
    type: 'manual' | 'ai_generated' | 'imported';
    reference?: string;
    confidence?: number;
  };
  
  // Usage Tracking
  accessCount: number;
  lastAccessedAt?: Date;
  
  isActive: boolean;
  createdAt: Date;
  updatedAt: Date;
}

File & Content Models

FabFileModel

interface IFabFile {
  _id: ObjectId;
  filename: string;
  originalName: string;
  
  // File Properties
  mimeType: string;
  size: number;              // File size in bytes
  checksum: string;          // File integrity check
  
  // Storage Information
  bucketName: string;        // S3 bucket name
  key: string;              // S3 object key
  url?: string;             // Public access URL
  
  // Ownership & Context
  userId: ObjectId;
  organizationId?: ObjectId;
  projectId?: ObjectId;
  sessionId?: ObjectId;
  
  // Processing State
  status: 'uploaded' | 'processing' | 'processed' | 'failed';
  processingSteps: {
    step: 'upload' | 'chunk' | 'vectorize' | 'analyze';
    status: 'pending' | 'running' | 'completed' | 'failed';
    startTime?: Date;
    endTime?: Date;
    error?: string;
  }[];
  
  // Content Analysis
  metadata: {
    pageCount?: number;       // For documents
    wordCount?: number;       // Text content
    language?: string;        // Detected language
    contentType?: string;     // Detected content type
  };
  
  // AI Processing Results
  chunks: ObjectId[];        // Related FabFileChunk documents
  summary?: string;          // AI-generated summary
  keyTopics: string[];      // Extracted topics
  tags: string[];           // Auto-generated tags
  
  // Access Control
  isPublic: boolean;
  sharedWith: ObjectId[];   // User IDs with access
  
  createdAt: Date;
  updatedAt: Date;
  processedAt?: Date;
}

FabFileChunkModel

interface IFabFileChunk {
  _id: ObjectId;
  fabFileId: ObjectId;
  
  // Chunk Properties
  chunkIndex: number;        // Order within the file
  content: string;          // Extracted text content
  startPage?: number;       // For documents
  endPage?: number;
  
  // Vector Embedding
  embedding?: number[];      // Vector representation
  embeddingModel?: string;   // Model used for embedding
  
  // Metadata
  metadata: {
    wordCount: number;
    characterCount: number;
    language?: string;
  };
  
  // Processing State
  status: 'pending' | 'processed' | 'failed';
  processedAt?: Date;
  
  createdAt: Date;
  updatedAt: Date;
}

Project & Collaboration Models

ProjectModel

interface IProject {
  _id: ObjectId;
  name: string;
  description?: string;
  
  // Ownership & Access
  userId: ObjectId;          // Project owner
  organizationId?: ObjectId;
  
  // Collaboration
  members: {
    userId: ObjectId;
    role: 'owner' | 'admin' | 'editor' | 'viewer';
    addedAt: Date;
    addedBy: ObjectId;
  }[];
  
  // Project Settings
  settings: {
    isPublic: boolean;
    allowComments: boolean;
    allowFileSharing: boolean;
    defaultAgentConfig?: ObjectId;
  };
  
  // Content Organization
  tags: string[];
  category?: string;
  
  // Related Content
  sessions: ObjectId[];      // Associated sessions
  files: ObjectId[];        // Associated files
  mementos: ObjectId[];     // Associated mementos
  
  // Project State
  status: 'active' | 'archived' | 'completed';
  isStarred: boolean;
  
  // Analytics
  metrics: {
    totalSessions: number;
    totalFiles: number;
    totalMembers: number;
    lastActivity: Date;
  };
  
  createdAt: Date;
  updatedAt: Date;
}

System & Configuration Models

ApiKeyModel

interface IApiKey {
  _id: ObjectId;
  name: string;
  description?: string;
  
  // Ownership
  userId: ObjectId;
  organizationId?: ObjectId;
  
  // Key Properties
  keyHash: string;           // Hashed API key
  keyPreview: string;        // First/last few characters for display
  
  // Service Configuration
  service: 'openai' | 'anthropic' | 'google' | 'custom';
  endpoint?: string;         // Custom endpoint URL
  
  // Usage & Limits
  usageCount: number;
  lastUsedAt?: Date;
  rateLimit?: {
    requestsPerMinute: number;
    requestsPerDay: number;
  };
  
  // Status
  isActive: boolean;
  expiresAt?: Date;
  
  createdAt: Date;
  updatedAt: Date;
}

McpServerModel

interface IMcpServer {
  _id: ObjectId;
  name: string;
  description?: string;
  
  // Server Configuration
  serverPath: string;        // Path to MCP server executable
  args: string[];           // Command line arguments
  envVariables: {           // Environment variables
    [key: string]: string;
  };
  
  // Ownership & Access
  userId?: ObjectId;         // User-specific server
  organizationId?: ObjectId; // Organization-wide server
  isGlobal: boolean;        // System-wide availability
  
  // Capabilities
  tools: {
    name: string;
    description: string;
    schema: any;             // JSON schema for tool parameters
  }[];
  
  // Status & Health
  status: 'active' | 'inactive' | 'error';
  lastHealthCheck?: Date;
  errorMessage?: string;
  
  // Usage Statistics
  metrics: {
    totalCalls: number;
    successfulCalls: number;
    averageResponseTime: number;
    lastUsedAt?: Date;
  };
  
  createdAt: Date;
  updatedAt: Date;
}

CreditTransactionModel

interface ICreditTransaction {
  _id: ObjectId;
  
  // Transaction Details
  userId: ObjectId;
  organizationId?: ObjectId;
  
  // Credit Information
  amount: number;            // Credits used (positive) or added (negative)
  balance: number;          // Balance after transaction
  
  // Transaction Context
  type: 'usage' | 'purchase' | 'refund' | 'bonus';
  description: string;
  
  // Usage Metadata
  metadata: {
    service?: string;        // AI service used
    model?: string;         // Specific model
    tokensUsed?: number;    // Tokens consumed
    questId?: ObjectId;     // Related quest
    sessionId?: ObjectId;   // Related session
  };
  
  // Billing
  costInCents?: number;     // Actual cost in cents
  stripeTransactionId?: string;
  
  createdAt: Date;
}

Database Relationships

Entity Relationship Overview

erDiagram
    User ||--o{ Session : creates
    User ||--o{ Quest : initiates
    User ||--o{ Project : owns
    User ||--o{ FabFile : uploads
    User ||--o{ Memento : creates
    User }o--|| Organization : belongs_to
    
    Session ||--o{ Quest : contains
    Session ||--o{ Memento : generates
    
    Quest ||--|| QuestMasterPlan : has
    Quest }o--|| Agent : executed_by
    
    FabFile ||--o{ FabFileChunk : contains
    
    Project ||--o{ Session : includes
    Project ||--o{ FabFile : contains
    Project ||--o{ Memento : organizes
    
    Agent }o--o{ McpServer : uses
    
    Organization ||--o{ ApiKey : manages
    Organization ||--o{ CreditTransaction : tracks

Key Relationships

User-Centric Relationships

• User → Organization: Many-to-one (users belong to organizations)
• User → Sessions: One-to-many (users create multiple sessions)
• User → Quests: One-to-many (users initiate multiple quests)
• User → Projects: One-to-many (users own multiple projects)

Content Relationships

• FabFile → FabFileChunk: One-to-many (files split into chunks)
• Session → Quests: One-to-many (sessions contain multiple quests)
• Project → Content: One-to-many (projects organize sessions, files, mementos)

AI Agent Relationships

• Quest → QuestMasterPlan: One-to-one (each quest has a plan)
• Agent → McpServer: Many-to-many (agents use multiple tools)
• Quest → Agent: Many-to-one (quests executed by agents)

Indexing Strategy

Performance Indexes

// User lookups
User: { email: 1, organizationId: 1 }
User: { username: 1, organizationId: 1 }

// Session queries
Session: { userId: 1, createdAt: -1 }
Session: { organizationId: 1, status: 1 }

// Quest processing
Quest: { status: 1, createdAt: -1 }
Quest: { userId: 1, status: 1 }

// File processing
FabFile: { userId: 1, status: 1 }
FabFileChunk: { fabFileId: 1, chunkIndex: 1 }

// Vector search (if using MongoDB Atlas Search)
FabFileChunk: { embedding: "vectorSearch" }
Memento: { embedding: "vectorSearch" }

Compound Indexes

// Multi-tenant queries
Session: { organizationId: 1, userId: 1, createdAt: -1 }
Quest: { organizationId: 1, status: 1, priority: -1 }
Project: { organizationId: 1, status: 1, updatedAt: -1 }

// Analytics queries
CreditTransaction: { userId: 1, createdAt: -1 }
Quest: { userId: 1, status: 1, createdAt: -1 }

Data Migration & Versioning

Migration Strategy

• Automated Migrations: Run via SST deployment scripts
• Version Tracking: MigrationModel tracks applied migrations
• Rollback Support: Reversible migration scripts
• Environment Isolation: Separate migration state per environment

Schema Evolution

• Backward Compatibility: New fields are optional
• Deprecation Process: Gradual removal of unused fields
• Data Transformation: Automated data format updates
• Validation: Mongoose schema validation for data integrity

This comprehensive data model provides the foundation for Bike4Mind's AI agent capabilities, user collaboration features, and scalable multi-tenant architecture.