User Settings Architecture

Overview

Bike4Mind's user settings system manages multiple types of configuration data across client, server, and database layers. This document provides a comprehensive analysis of the current architecture, identifies key issues, and proposes architectural improvements for better separation of concerns, performance, and maintainability.

Current Architecture Analysis

Architecture Overview

The current user settings implementation follows a hybrid approach that conflates three distinct types of settings:

graph TB
    Client[Client Layer] --> LocalStorage[localStorage]
    Client --> ServerAPI[Server API]

    ServerAPI --> AdminDB[(AdminSettings DB)]
    ServerAPI --> UserDB[(User Model)]

    LocalStorage --> UIPrefs[UI Preferences]
    AdminDB --> GlobalConfig[Global Configuration]
    UserDB --> UserData[User Account Data]

    Client --> MixedContext[UserSettingsContext]
    MixedContext --> LocalStorage
    MixedContext --> ServerAPI

Current Implementation Structure

// Current UserSettingsContext.tsx - MIXED CONCERNS
interface UserSettings {
  // 🔴 LOCAL UI STATE (localStorage)
  showDebug: boolean;
  showHelp: boolean;
  maxVisibleLines: number;
  enableAutoScroll: boolean;

  // 🔴 GLOBAL SERVER CONFIG (AdminSettings DB)
  serverSettings: IAdminSettings[];

  // 🔴 LOCAL FEATURE TOGGLES (localStorage)
  experimentalFeatures: {
    enableQuestMaster: boolean;
    enableMementos: boolean;
    enableArtifacts: boolean;
    enableOllama: boolean;
    enableAgents: boolean;
  };
}

Current Data Flow

Client-Side Settings Management

sequenceDiagram
    participant U as User
    participant C as Client Component
    participant USC as UserSettingsContext
    participant LS as localStorage
    participant API as Settings API
    participant DB as Database

    U->>C: Change Setting
    C->>USC: setSettings()
    USC->>LS: Save to localStorage
    USC->>API: Fetch serverSettings
    API->>DB: Query AdminSettings
    DB->>API: Return settings
    API->>USC: Update serverSettings
    USC->>C: Re-render with new data

Server-Side API Structure

// Current API Endpoints
/api/settings/
├── fetch.ts          // Public settings (non-admin)
├── index.ts          // Admin-only settings (full access)
├── update.ts         // Admin settings modification
├── serverStatus.ts   // Server status check
└── serverConfig.ts   // Server configuration

Database Layer

AdminSettings Collection

interface IAdminSettings {
  _id: ObjectId;
  settingName: SettingKey;           // e.g., "EnableQuestMaster"
  settingValue: string | boolean | number;
  createdAt: Date;
  updatedAt: Date;
}

User Model (Scattered Preferences)

interface IUser {
  // ... other user fields

  // 🔴 SCATTERED USER PREFERENCES
  preferredLanguage: string | null;
  lastNotebookId: ObjectId | null;
  tags: string[];

  // 🔴 NO DEDICATED PREFERENCES FIELD
}

Architectural Issues

1. Conceptual Confusion

Problem: The system conflates three fundamentally different types of settings:

Type	Storage	Scope	Examples
UI State	localStorage	Session/Device	showHelp
Global Config	Database	Application	EnableQuestMaster, maxFileSize
User Preferences	localStorage*	User Account	debugMode, theme, language

Should be database-backed for persistence across devices

2. Feature Flag Inconsistency

// 🔴 CURRENT: Confusing dual toggle system

// Local toggle in UserSettingsContext
experimentalFeatures: {
  enableQuestMaster: false,  // User's local preference
}

// Global flag in AdminSettings database
{
  settingName: "EnableQuestMaster",
  settingValue: true          // Global feature availability
}

// 🔴 LOGIC UNCLEAR: Which takes precedence?

3. Data Synchronization Problems

• Lost Settings: localStorage preferences lost when switching devices
• Stale Data: No invalidation strategy for cached settings
• Race Conditions: Multiple components fetching same settings data
• Inconsistent State: Client and server settings can diverge

4. Performance Issues

// 🔴 CURRENT: Multiple redundant API calls
const ComponentA = () => {
  const { data } = useSettingsFromServer(); // API call 1
};

const ComponentB = () => {
  const { data } = useExperimentalFeatureSettings(); // API call 2 (same data)
};

const ComponentC = () => {
  const { serverSettings } = useServerSettings(); // API call 3 (same data)
};

5. Lack of User-Specific Persistence

// 🔴 CURRENT: No dedicated user preferences storage
interface IUser {
  // User preferences scattered across multiple fields
  preferredLanguage: string | null;
  lastNotebookId: ObjectId | null;
  // ... no structured preferences object
}

Recommended Architecture

1. Clear Separation of Concerns

// ✅ PROPOSED: Three distinct setting types

// 1. GLOBAL CONFIGURATION (Admin-managed, cached)
interface GlobalConfig {
  // Feature flags
  features: {
    enableQuestMaster: boolean;
    enableMementos: boolean;
    enableArtifacts: boolean;
    enableAgents: boolean;
  };

  // System limits
  limits: {
    maxFileSize: number;
    maxContentLength: number;
    vectorThreshold: number;
  };

  // API configurations
  integrations: {
    openaiApiKey: string;
    anthropicApiKey: string;
    enableWeatherService: boolean;
  };
}

// 2. USER PREFERENCES (User-managed, server-backed)
interface UserPreferences {
  // UI preferences
  ui: {
    theme: 'light' | 'dark' | 'auto';
    language: string;
    debugMode: boolean;
    maxVisibleLines: number;
    defaultView: 'list' | 'grid' | 'compact';
  };

  // Feature opt-ins (subset of globally enabled features)
  enabledFeatures: string[];

  // AI preferences
  ai: {
    preferredModel: string;
    autonomyLevel: 'low' | 'medium' | 'high';
    confidenceThreshold: number;
  };

  // Notification preferences
  notifications: {
    email: boolean;
    slack: boolean;
    questUpdates: boolean;
    systemAlerts: boolean;
  };

  // Workflow preferences
  workflow: {
    autoTagging: boolean;
    autoSummarization: boolean;
    autoSave: boolean;
    defaultProjectVisibility: 'private' | 'organization' | 'public';
  };
}

// 3. UI STATE (Session-only, localStorage)
interface UIState {
  // Transient UI state
  showHelp: boolean;
  enableAutoScroll: boolean;
  sidebarCollapsed: boolean;
  activeTab: string;

  // Session-specific state
  lastVisitedProject: string | null;
  temporaryFilters: Record<string, any>;
}

2. Improved Database Schema

// ✅ Enhanced User Model with structured preferences
interface IUser {
  _id: ObjectId;
  username: string;
  email: string;
  // ... existing fields

  // ✅ NEW: Structured preferences object
  preferences: UserPreferences;

  // ✅ Migrate scattered fields into preferences
  // preferredLanguage -> preferences.ui.language
  // lastNotebookId -> preferences.workflow.lastNotebookId
}

// ✅ Keep AdminSettings for global configuration
interface IAdminSettings {
  _id: ObjectId;
  settingName: SettingKey;
  settingValue: any;
  category: 'features' | 'limits' | 'integrations' | 'system';
  isPublic: boolean;          // Can non-admin users read this?
  isSensitive: boolean;       // Should this be redacted in logs?
  createdAt: Date;
  updatedAt: Date;
}

3. Clean API Structure

// ✅ PROPOSED: Separate endpoints by concern

// Global configuration (cached, public subset)
GET  /api/config/global          // Public global settings
GET  /api/admin/settings         // Admin-only global settings
PUT  /api/admin/settings/:key    // Update global setting

// User preferences (user-scoped, persistent)
GET  /api/users/me/preferences   // User's preferences
PUT  /api/users/me/preferences   // Update user preferences
PATCH /api/users/me/preferences  // Partial update

// Server status and config
GET  /api/config/server          // Server status and public config

4. Client Architecture

// ✅ PROPOSED: Separate contexts by concern

// Global configuration context (cached, read-only for users)
const GlobalConfigProvider = ({ children }) => {
  const { data: config, isLoading } = useQuery({
    queryKey: ['global-config'],
    queryFn: () => api.get('/api/config/global'),
    staleTime: 5 * 60 * 1000,  // 5 minutes cache
  });

  return (
    <GlobalConfigContext.Provider value={{ config, isLoading }}>
      {children}
    </GlobalConfigContext.Provider>
  );
};

// User preferences context (user-scoped, read-write)
const UserPreferencesProvider = ({ children }) => {
  const queryClient = useQueryClient();

  const { data: preferences, isLoading } = useQuery({
    queryKey: ['user-preferences'],
    queryFn: () => api.get('/api/users/me/preferences'),
    staleTime: 2 * 60 * 1000,  // 2 minutes cache
  });

  const updatePreferences = useMutation({
    mutationFn: (updates: Partial<UserPreferences>) =>
      api.patch('/api/users/me/preferences', updates),
    onSuccess: () => {
      queryClient.invalidateQueries(['user-preferences']);
    },
  });

  return (
    <UserPreferencesContext.Provider value={{
      preferences,
      isLoading,
      updatePreferences
    }}>
      {children}
    </UserPreferencesContext.Provider>
  );
};

// UI state context (session-only, localStorage)
const UIStateProvider = ({ children }) => {
  const [uiState, setUIState] = useLocalStorage<UIState>('ui-state', {
    showHelp: false,
    enableAutoScroll: true,
    sidebarCollapsed: false,
    activeTab: 'general',
    lastVisitedProject: null,
    temporaryFilters: {},
  });

  return (
    <UIStateContext.Provider value={{ uiState, setUIState }}>
      {children}
    </UIStateContext.Provider>
  );
};

5. Feature Flag Implementation

// ✅ PROPOSED: Clear feature flag logic

const useFeatureFlag = (featureName: string): boolean => {
  const { config } = useGlobalConfig();
  const { preferences } = useUserPreferences();

  // Feature must be globally enabled AND user must have opted in
  const globallyEnabled = config?.features?.[featureName] ?? false;
  const userEnabled = preferences?.enabledFeatures?.includes(featureName) ?? false;

  return globallyEnabled && userEnabled;
};

// Usage in components
const QuestMasterSection = () => {
  const questMasterEnabled = useFeatureFlag('questMaster');

  if (!questMasterEnabled) {
    return null;
  }

  return <QuestMasterUI />;
};

Migration Strategy

Phase 1: Database Schema Updates

// 1. Add preferences field to User model
db.users.updateMany(
  {},
  {
    $set: {
      preferences: {
        ui: {
          theme: 'light',
          language: '$preferredLanguage',  // Migrate existing field
          debugMode: false,
          maxVisibleLines: 25,
          defaultView: 'list',
        },
        enabledFeatures: [],
        ai: {
          preferredModel: 'gpt-4',
          autonomyLevel: 'medium',
          confidenceThreshold: 0.8,
        },
        notifications: {
          email: true,
          slack: false,
          questUpdates: true,
          systemAlerts: true,
        },
        workflow: {
          autoTagging: false,
          autoSummarization: false,
          autoSave: true,
          defaultProjectVisibility: 'private',
        },
      },
    },
  }
);

// 2. Remove migrated fields
db.users.updateMany(
  {},
  {
    $unset: {
      preferredLanguage: '',
      // ... other migrated fields
    },
  }
);

Phase 2: API Implementation

// packages/client/pages/api/users/me/preferences.ts
const handler = baseApi()
  .get(async (req, res) => {
    const user = await User.findById(req.user.id).select('preferences');
    return res.json(user.preferences || getDefaultPreferences());
  })
  .put(async (req, res) => {
    const preferences = UserPreferencesSchema.parse(req.body);
    const user = await User.findByIdAndUpdate(
      req.user.id,
      { $set: { preferences } },
      { new: true }
    );
    return res.json(user.preferences);
  })
  .patch(async (req, res) => {
    const updates = PartialUserPreferencesSchema.parse(req.body);
    const user = await User.findByIdAndUpdate(
      req.user.id,
      { $set: flattenUpdates('preferences', updates) },
      { new: true }
    );
    return res.json(user.preferences);
  });

Phase 3: Client Migration

// 1. Create new contexts alongside existing ones
// 2. Gradually migrate components to use new contexts
// 3. Add feature flags to control rollout
// 4. Remove old UserSettingsContext once migration complete

// Migration helper hook
const useMigratedSettings = () => {
  const oldSettings = useUserSettings();  // Legacy
  const newPreferences = useUserPreferences();  // New
  const globalConfig = useGlobalConfig();  // New
  const uiState = useUIState();  // New

  // Feature flag to control which system to use
  const useNewSettingsSystem = useFeatureFlag('newSettingsSystem');

  if (useNewSettingsSystem) {
    return {
      debugMode: newPreferences.ui.debugMode,
      theme: newPreferences.ui.theme,
      // ... map new structure
    };
  }

  return {
    debugMode: oldSettings.showDebug,
    theme: 'light', // fallback
    // ... map old structure
  };
};

Performance Optimizations

1. Caching Strategy

// Global config: Long cache (rarely changes)
const useGlobalConfig = () => useQuery({
  queryKey: ['global-config'],
  queryFn: fetchGlobalConfig,
  staleTime: 10 * 60 * 1000,    // 10 minutes
  cacheTime: 30 * 60 * 1000,    // 30 minutes
});

// User preferences: Medium cache (changes occasionally)
const useUserPreferences = () => useQuery({
  queryKey: ['user-preferences'],
  queryFn: fetchUserPreferences,
  staleTime: 2 * 60 * 1000,     // 2 minutes
  cacheTime: 10 * 60 * 1000,    // 10 minutes
});

// UI state: No cache (local only)
const useUIState = () => useLocalStorage('ui-state', defaultUIState);

2. Request Deduplication

// Automatic request deduplication with React Query
const GlobalConfigProvider = ({ children }) => {
  // Multiple components calling useGlobalConfig will share the same request
  const queryClient = useQueryClient();

  // Prefetch on app load
  useEffect(() => {
    queryClient.prefetchQuery(['global-config'], fetchGlobalConfig);
  }, []);

  return <Provider>{children}</Provider>;
};

3. Optimistic Updates

const useUpdatePreferences = () => {
  const queryClient = useQueryClient();

  return useMutation({
    mutationFn: updatePreferences,
    onMutate: async (updates) => {
      // Cancel outgoing refetches
      await queryClient.cancelQueries(['user-preferences']);

      // Snapshot previous value
      const previousPreferences = queryClient.getQueryData(['user-preferences']);

      // Optimistically update
      queryClient.setQueryData(['user-preferences'], (old) => ({
        ...old,
        ...updates,
      }));

      return { previousPreferences };
    },
    onError: (err, updates, context) => {
      // Rollback on error
      queryClient.setQueryData(['user-preferences'], context.previousPreferences);
    },
    onSettled: () => {
      // Always refetch to ensure sync
      queryClient.invalidateQueries(['user-preferences']);
    },
  });
};

Security Considerations

1. Settings Access Control

// Global settings with visibility control
interface IAdminSettings {
  settingName: string;
  settingValue: any;
  isPublic: boolean;        // Can regular users see this setting?
  isSensitive: boolean;     // Should this be redacted in logs?
  adminOnly: boolean;       // Can only admins modify this setting?
}

// API endpoint with proper filtering
export default baseApi().get(async (req, res) => {
  const isAdmin = req.user?.isAdmin ?? false;

  const settings = await AdminSettings.find({
    $or: [
      { isPublic: true },           // Public settings
      { adminOnly: false },         // User-modifiable settings
      ...(isAdmin ? [{}] : []),     // All settings if admin
    ],
  });

  // Redact sensitive values for non-admins
  const filteredSettings = settings.map(setting => ({
    ...setting.toJSON(),
    settingValue: setting.isSensitive && !isAdmin
      ? '[REDACTED]'
      : setting.settingValue,
  }));

  return res.json(filteredSettings);
});

2. User Preferences Validation

// Strong validation for user preferences
const UserPreferencesSchema = z.object({
  ui: z.object({
    theme: z.enum(['light', 'dark', 'auto']),
    language: z.string().min(2).max(5),
    debugMode: z.boolean(),
    maxVisibleLines: z.number().min(10).max(100),
    defaultView: z.enum(['list', 'grid', 'compact']),
  }),
  enabledFeatures: z.array(z.string()).max(20),  // Prevent abuse
  ai: z.object({
    preferredModel: z.string(),
    autonomyLevel: z.enum(['low', 'medium', 'high']),
    confidenceThreshold: z.number().min(0).max(1),
  }),
  // ... other preferences with validation
});

Monitoring & Analytics

1. Settings Usage Analytics

// Track preference changes for product insights
const useUpdatePreferences = () => {
  return useMutation({
    mutationFn: updatePreferences,
    onSuccess: (data, variables) => {
      // Analytics tracking
      analytics.track('Preferences Updated', {
        userId: user.id,
        changes: Object.keys(variables),
        timestamp: new Date(),
      });
    },
  });
};

2. Performance Monitoring

// Monitor settings API performance
const fetchGlobalConfig = async () => {
  const startTime = performance.now();

  try {
    const response = await api.get('/api/config/global');
    const loadTime = performance.now() - startTime;

    // Log performance metrics
    console.log(`Global config loaded in ${loadTime.toFixed(2)}ms`);

    return response.data;
  } catch (error) {
    // Error monitoring
    console.error('Failed to load global config:', error);
    throw error;
  }
};

Future Considerations

1. Real-time Settings Updates

// WebSocket integration for real-time setting changes
const useRealtimeSettings = () => {
  const queryClient = useQueryClient();

  useEffect(() => {
    const ws = new WebSocket('/ws/settings');

    ws.onmessage = (event) => {
      const { type, data } = JSON.parse(event.data);

      if (type === 'GLOBAL_CONFIG_UPDATED') {
        queryClient.invalidateQueries(['global-config']);
      } else if (type === 'USER_PREFERENCES_UPDATED') {
        queryClient.setQueryData(['user-preferences'], data);
      }
    };

    return () => ws.close();
  }, [queryClient]);
};

2. Settings Import/Export

// Settings backup and migration between environments
const useSettingsBackup = () => {
  const exportSettings = async () => {
    const preferences = await api.get('/api/users/me/preferences');
    const backup = {
      version: '1.0',
      timestamp: new Date().toISOString(),
      preferences: preferences.data,
    };

    // Download as JSON file
    downloadJSON(backup, `settings-backup-${Date.now()}.json`);
  };

  const importSettings = async (backupFile: File) => {
    const backup = await parseJSON(backupFile);
    await api.put('/api/users/me/preferences', backup.preferences);
  };

  return { exportSettings, importSettings };
};

3. A/B Testing Framework

// Settings-based A/B testing
const useExperimentalSetting = (experimentName: string, defaultValue: any) => {
  const { preferences } = useUserPreferences();
  const { config } = useGlobalConfig();

  // Check if user is in experimental group
  const userInExperiment = config.experiments?.[experimentName]?.includes(preferences.userId);

  return userInExperiment
    ? config.experiments[experimentName].value
    : defaultValue;
};

Summary

The proposed user settings architecture provides:

1. Clear Separation: Distinct handling of global config, user preferences, and UI state
2. Better Performance: Proper caching, request deduplication, and optimistic updates
3. Data Persistence: Server-backed user preferences that sync across devices
4. Type Safety: Strong TypeScript interfaces and validation
5. Security: Proper access control and sensitive data handling
6. Scalability: Architecture that supports future features like real-time updates and A/B testing

This architecture resolves the current conceptual confusion while providing a solid foundation for future enhancements.