Skip to main content

Development Performance Optimization Guide

Quick Setup for Local Development​

To enable performance optimizations for local development, add these environment variables to your .env.local:

# Skip heavy features for faster local development
ENABLE_MEMENTOS=false
ENABLE_QUESTMASTER=false

# Preserve full history (set to true to keep all message history)
PRESERVE_HISTORY=false

# Enable verbose logging for debugging
VERBOSE_CHAT_CONTEXT=true
VERBOSE_MESSAGE_BUILDING=false

Implemented Optimizations​

1. Parallel File Processing​

  • Change: Files are now processed in parallel with controlled concurrency (max 3 concurrent)
  • Impact: ~50-70% faster file processing for multiple files
  • Location: b4m-core/packages/core/utils/llm/utils.ts

2. Reduced WebSocket Throttling​

  • Change: WebSocket updates throttled at 10ms (dev) vs 50ms (prod) instead of 25ms/100ms
  • Impact: Much more responsive streaming in development
  • Location: b4m-core/packages/core/services/llm/ChatCompletion.ts

3. Development Mode Optimizations​

  • Change: Automatically skip heavy features and reduce history count in development
  • Impact: 40-60% faster processing for local development
  • Location: b4m-core/packages/core/services/llm/ChatCompletion.ts

4. Parallel Admin Settings Fetch​

  • Change: Admin settings and API keys fetched in parallel
  • Impact: ~100-200ms saved per request
  • Location: b4m-core/packages/core/services/llm/ChatCompletion.ts

Expected Performance Improvements​

Before Optimization​

  • Total time: ~6.2s
  • Time to first chunk: ~2.1s
  • Chunk intervals: 700-800ms

After Optimization​

  • Total time: ~2.5-3.5s (40-60% improvement)
  • Time to first chunk: ~800ms-1.2s (60% improvement)
  • Chunk intervals: 50-100ms (80-90% improvement)

Next Steps for Further Optimization​

Database Indexes (High Impact)​

Run these MongoDB commands to add performance indexes:

db.quests.createIndex({ "sessionId": 1, "timestamp": -1 })
db.quests.createIndex({ "sessionId": 1, "status": 1 })
db.fabfiles.createIndex({ "userId": 1, "enabled": 1 })
db.fabfilechunks.createIndex({ "fabFileId": 1, "score": -1 })

Redis Caching (Medium Impact)​

Consider implementing Redis caching for:

  • Recent message history per session
  • User's effective API keys (with TTL)
  • Admin settings cache

Model Selection (High Impact for Development)​

For local development, consider using faster models:

  • claude-3-haiku-20240307 instead of claude-3.5-sonnet
  • gpt-4o-mini instead of gpt-4o

Monitoring Performance​

The system now logs file processing times. Look for logs like:

📁 File processing completed in 245ms for 3 files

WebSocket send timing is also logged in development:

⏱️ WebSocket send took 15ms for 1250 chars

Troubleshooting​

If performance is still slow:​

  1. Check if you're using a slow model (Claude 3.5 Sonnet, GPT-4o)
  2. Verify environment variables are set correctly
  3. Check database connection latency
  4. Monitor file processing logs for bottlenecks

If features are missing:​

  1. Set ENABLE_MEMENTOS=true and ENABLE_QUESTMASTER=true in your .env.local
  2. Set PRESERVE_HISTORY=true to keep full message history