Skip to main content

Artifact Generation Architecture Review

Executive Summary​

This document provides a comprehensive architectural review of how artifacts are generated, processed, and displayed in the Bike4Mind system. Based on analysis of the codebase, this review identifies the complete flow from LLM response generation to artifact viewing and highlights critical architectural decisions and gaps.

Current Artifact Generation Flow​

1. LLM Response Generation​

Location: b4m-core/packages/core/services/llm/ChatCompletion.ts

The artifact generation begins in the ChatCompletionService.process() method:

// Process artifacts from responses if enabled
if (getSettingsValue('EnableArtifacts', defaultAdminSettings)) {
  const { parseArtifacts, convertCodeBlocksToArtifacts } = await import('../../utils/artifactParser');

  quest.replies = quest.replies?.map(reply => {
    // First, try to detect and convert code blocks to artifacts as fallback
    const processedReply = convertCodeBlocksToArtifacts(reply);

    // Then parse any artifacts that exist
    const { artifacts } = parseArtifacts(processedReply);

    if (artifacts.length > 0) {
      logger.info(`Found ${artifacts.length} artifacts in response`);
      // TODO: Save artifacts to knowledge system here
      // For now, we keep the artifacts in the response
    }

    return processedReply; // Return the content with artifact tags
  });
}

Key Architectural Decisions:

  • Artifacts are processed after LLM completion, not during streaming
  • Two-stage processing: code block detection → artifact parsing
  • Artifacts remain embedded in quest replies as XML tags
  • Critical Gap: No persistent artifact storage system

2. Artifact Parsing System​

Location: b4m-core/packages/core/utils/artifactParser.ts & packages/client/app/utils/artifactParser.ts

The system uses regex-based parsing to extract Claude-style artifact syntax:

const ARTIFACT_REGEX = /<artifact\s+(.*?)>([\s\S]*?)<\/artifact>/gi;

Supported Artifact Types:

  • React components (application/vnd.ant.react)
  • HTML pages (text/html)
  • SVG graphics (image/svg+xml)
  • Mermaid diagrams (application/vnd.ant.mermaid)
  • Code blocks (application/vnd.ant.code)

Fallback Processing: Automatically converts code blocks to artifacts:

export function convertCodeBlocksToArtifacts(content: string): string {
  // Detects React, HTML, SVG code blocks and converts to artifact tags
}

3. WebSocket Streaming Architecture​

Location: packages/client/app/hooks/useSubscribeChatCompletion.ts

Artifacts flow through the WebSocket streaming system via streamed_chat_completion actions:

const StreamedChatCompletionAction = z.object({
  action: z.literal('streamed_chat_completion'),
  quest: z.object({
    id: z.string(),
    sessionId: z.string(),
    reply: z.string().nullable().optional(),
    replies: z.array(z.string()).optional(),
    // ... artifact content embedded in replies
  })
});

Performance Optimizations:

  • Throttled updates to prevent UI blocking
  • Real-time streaming with 250ms update intervals
  • Optimistic UI updates

4. Client-Side Artifact Extraction​

Location: packages/client/app/components/Session/PromptReplies.tsx

The client extracts artifacts from streamed quest replies in real-time:

import { parseArtifacts, convertCodeBlocksToArtifacts } from '@client/app/utils/artifactParser';

// Artifacts are parsed from quest replies during rendering
const { artifacts } = parseArtifacts(replyContent);

5. Knowledge Viewer Integration​

Location: packages/client/app/components/Knowledge/KnowledgeViewer.tsx

Artifacts are displayed through the Knowledge Viewer system via session layout:

// Artifacts are stored in session layout state
const { artifactData } = useSessionLayout();

// Different viewer components based on artifact type
switch (artifactData?.type) {
  case 'react':
    return <ReactArtifactViewer artifact={content} onSave={handleSave} />;
  case 'html':
    return <HtmlArtifactViewer artifact={content} />;
  case 'svg':
    return <SvgArtifactViewer artifact={content} />;
  // ...
}

Critical Architectural Issues​

1. Artifact Persistence Gap​

Issue: Artifacts are not persistently stored in a dedicated system.

Current State:

  • Artifacts remain embedded as XML tags in quest replies
  • No dedicated artifact storage or indexing
  • No artifact versioning or history
  • No artifact relationships or dependencies

Impact:

  • Saving edited artifacts requires complex quest reply parsing
  • No way to reference artifacts across sessions
  • No artifact reusability or sharing
  • Performance issues with large artifact content

Issue: No systematic way to discover, search, or catalog artifacts.

Current State:

  • Artifacts are only accessible through their originating chat sessions
  • No artifact metadata indexing
  • No full-text search within artifact content
  • No artifact categorization or tagging

3. Edit and Save Architecture Problems​

Issue: The save functionality attempts to modify quest replies, which is architecturally incorrect.

Current Implementation (Problematic):

// Tries to find and update artifact tags within quest replies
const questWithArtifact = quests.find(quest => {
  const replyContent = quest.replies?.[0] || quest.reply || '';
  return artifactIdentifierRegex.test(replyContent);
});

Problems:

  • Quest replies may not contain artifact tags (as seen in debugging)
  • Artifacts might be generated separately from replies
  • No transactional consistency between artifact edits and quest history

4. Artifact Lifecycle Management​

Issue: No clear artifact lifecycle or state management.

Missing Capabilities:

  • Artifact creation timestamps
  • Artifact modification history
  • Artifact ownership and permissions
  • Artifact publishing and sharing workflows
  • Artifact deprecation and archival

1. Dedicated Artifact Storage System​

Implement a proper artifact storage layer:

interface ArtifactDocument {
  id: string;
  type: ArtifactType;
  title: string;
  content: string;
  metadata: ArtifactMetadata;
  
  // Lifecycle
  createdAt: Date;
  updatedAt: Date;
  version: number;
  status: 'draft' | 'published' | 'archived';
  
  // Relationships
  sourceQuestId: string;
  sessionId: string;
  userId: string;
  projectId?: string;
  
  // Content
  contentHash: string;
  dependencies: string[];
  size: number;
}

2. Artifact Service Layer​

Create dedicated artifact services:

class ArtifactService {
  async createArtifact(data: CreateArtifactRequest): Promise<ArtifactDocument>
  async updateArtifact(id: string, updates: UpdateArtifactRequest): Promise<ArtifactDocument>
  async getArtifact(id: string): Promise<ArtifactDocument>
  async searchArtifacts(query: ArtifactSearchQuery): Promise<ArtifactDocument[]>
  async deleteArtifact(id: string): Promise<void>
  async forkArtifact(id: string, updates: Partial<ArtifactDocument>): Promise<ArtifactDocument>
}

3. Artifact Processing Pipeline​

Implement a proper artifact processing pipeline:

// 1. Extract artifacts during LLM processing
const artifacts = parseArtifacts(reply);

// 2. Store artifacts separately
for (const artifact of artifacts) {
  const storedArtifact = await artifactService.createArtifact({
    sourceQuestId: quest.id,
    sessionId: quest.sessionId,
    ...artifact
  });
  
  // 3. Replace artifact content with references
  reply = reply.replace(artifact.fullMatch, 
    `<artifact-ref id="${storedArtifact.id}" />`);
}

4. Improved Knowledge Viewer Architecture​

Enhance the knowledge viewer with proper artifact loading:

const KnowledgeViewer = () => {
  const { data: artifacts } = useArtifacts(sessionId);
  const { data: quest } = useQuest(questId);
  
  // Load artifacts referenced in quest replies
  const referencedArtifacts = extractArtifactReferences(quest.reply);
  
  return (
    <div>
      {artifacts.map(artifact => (
        <ArtifactViewer 
          key={artifact.id}
          artifact={artifact}
          onSave={handleArtifactSave}
        />
      ))}
    </div>
  );
};

5. WebSocket Artifact Events​

Add dedicated WebSocket events for artifact operations:

export const ArtifactCreatedAction = z.object({
  action: z.literal('artifact_created'),
  artifact: ArtifactDocumentSchema,
  questId: z.string(),
});

export const ArtifactUpdatedAction = z.object({
  action: z.literal('artifact_updated'),
  artifactId: z.string(),
  changes: z.record(z.unknown()),
});

Security Considerations​

Current Security Model​

The artifact system includes several security measures:

  1. Content Validation: Artifacts are validated based on type
  2. Dependency Restrictions: Only allowed dependencies for React artifacts
  3. Sandboxing: React artifacts run in isolated sandboxes
  4. CSP Integration: Content Security Policy controls artifact execution
  1. Artifact Signing: Cryptographically sign artifacts to prevent tampering
  2. Access Control: Implement proper permissions for artifact viewing/editing
  3. Content Scanning: Scan artifact content for malicious code
  4. Audit Logging: Log all artifact operations for security monitoring

Performance Considerations​

Current Performance Characteristics​

  1. Streaming Performance: Well-optimized with throttling and batching
  2. Memory Usage: Artifacts stored in session layout state (memory-bound)
  3. Network Efficiency: Artifacts transmitted as part of quest replies
  1. Lazy Loading: Load artifact content on demand
  2. Caching: Implement artifact-specific caching strategies
  3. Compression: Compress large artifact content
  4. CDN Integration: Store artifact assets in CDN for faster delivery

Migration Strategy​

Phase 1: Backward Compatibility​

  • Implement artifact storage alongside existing system
  • Maintain quest reply embedding for compatibility
  • Add artifact reference extraction

Phase 2: New Artifact Creation​

  • Route new artifacts through dedicated storage
  • Update viewers to handle both embedded and referenced artifacts
  • Implement artifact editing through new system

Phase 3: Migration and Cleanup​

  • Migrate existing embedded artifacts to dedicated storage
  • Remove artifact content from quest replies
  • Clean up legacy parsing code

Conclusion​

The current artifact generation architecture is functional but has significant limitations around persistence, discoverability, and editing. The system would benefit from a dedicated artifact storage and management layer that treats artifacts as first-class entities rather than embedded content within chat responses.

The recommended improvements would provide:

  • Better User Experience: Faster loading, reliable saving, artifact discovery
  • Improved Developer Experience: Cleaner APIs, better testing, easier maintenance
  • Enhanced Security: Proper access controls, content validation, audit trails
  • Better Performance: Optimized loading, caching, and network usage

Implementation should follow the phased migration approach to maintain backward compatibility while gradually moving to the improved architecture.