Best Practices
Production deployment guidance for the B4M Completions API, covering security, performance, reliability, and cost optimization.
Security
Never Expose API Keys Client-Side
❌ Bad: Client-side code with API key
// Frontend code (NEVER DO THIS)
const apiKey = 'b4m_live_xxxxxxxxxxxx'; // Exposed to users!
fetch('https://app.bike4mind.com/api/ai/v1/completions', {
headers: { 'Authorization': `Bearer ${apiKey}` }
});
✅ Good: Proxy through your backend
// Frontend code
fetch('/api/my-completion-proxy', {
method: 'POST',
headers: { 'Authorization': `Bearer ${userJWT}`, 'Content-Type': 'application/json' },
body: JSON.stringify({ model: 'claude-sonnet-5', messages })
});
// Backend code (Node.js/Express)
import { Readable } from 'node:stream';
app.post('/api/my-completion-proxy', authenticateUser, async (req, res) => {
// Validate user has permission
if (!req.user.canUseAI) {
return res.status(403).json({ error: 'Forbidden' });
}
// Forward to B4M API with YOUR API key (server-side)
const response = await fetch('https://app.bike4mind.com/api/ai/v1/completions', {
method: 'POST',
headers: {
'Authorization': `Bearer ${process.env.B4M_API_KEY}`, // Safe on server
'Content-Type': 'application/json',
},
body: JSON.stringify(req.body)
});
// Stream response back to client
res.status(response.status);
res.setHeader('Content-Type', response.headers.get('content-type') ?? 'text/event-stream');
if (response.headers.has('Retry-After')) {
res.setHeader('Retry-After', response.headers.get('Retry-After'));
}
Readable.fromWeb(response.body).pipe(res);
});
Use Environment Variables
Store API keys in environment variables, never in code:
Node.js:
const apiKey = process.env.B4M_API_KEY;
Python:
import os
api_key = os.environ.get("B4M_API_KEY")
Docker:
docker run -e B4M_API_KEY=b4m_live_xxx myapp
Rotate Keys Regularly
Recommended schedule:
- Production: Every 90 days
- Development: Every 180 days
- Immediately after: suspected compromise, team member departure
Rotation process:
- Generate new API key
- Update application configuration
- Deploy and test
- Revoke old key
- Monitor for errors
Use Separate Keys per Environment
Create one key per environment and name it for that environment (for example "production", "staging", "development"), and keep each in that environment's secret store.
Benefits:
- Limit blast radius of compromises
- Easier tracking and debugging
- Independent rate limits per environment
Rate Limit Management
Read the Headers on a 429
The streaming completions endpoint sends the X-RateLimit-* headers (per-minute and per-day limit, remaining and reset, reset as Unix epoch seconds) only on an API-key 429. Successful responses do not carry them, so you cannot track remaining quota from them. On a 429, wait the Retry-After seconds before retrying:
if (response.status === 429) {
const retryAfter = Number(response.headers.get('Retry-After')) || 1;
console.warn(`Rate limited; day remaining: ${response.headers.get('X-RateLimit-Remaining-Day')}`);
await sleep(retryAfter * 1000);
}
Implement Client-Side Rate Limiting
Prevent hitting server-side limits:
class RateLimiter {
constructor(requestsPerMinute) {
this.requestsPerMinute = requestsPerMinute;
this.requests = [];
}
async waitForSlot() {
const now = Date.now();
const oneMinuteAgo = now - 60000;
// Remove old requests
this.requests = this.requests.filter(time => time > oneMinuteAgo);
if (this.requests.length >= this.requestsPerMinute) {
const oldestRequest = this.requests[0];
const waitTime = 60000 - (now - oldestRequest);
console.log(`Rate limit: waiting ${Math.ceil(waitTime / 1000)}s`);
await new Promise(resolve => setTimeout(resolve, waitTime));
return this.waitForSlot(); // Try again
}
this.requests.push(now);
}
}
const limiter = new RateLimiter(requestsPerMinute); // your key's per-minute limit
async function makeRequest() {
await limiter.waitForSlot();
return fetch(...);
}
Use Request Queuing
Queue requests during high traffic:
class RequestQueue {
constructor(maxConcurrent = 5) {
this.maxConcurrent = maxConcurrent;
this.running = 0;
this.queue = [];
}
async add(requestFn) {
if (this.running >= this.maxConcurrent) {
await new Promise(resolve => this.queue.push(resolve));
}
this.running++;
try {
return await requestFn();
} finally {
this.running--;
if (this.queue.length > 0) {
const resolve = this.queue.shift();
resolve();
}
}
}
}
const queue = new RequestQueue(5); // Max 5 concurrent requests
async function makeQueuedRequest(messages) {
return queue.add(() => client.complete('claude-sonnet-5', messages));
}
Credit Optimization
Choose Appropriate Models
Smaller models cost fewer credits than larger ones. Call GET /api/v1/models for the model ids available to you.
Set Reasonable max_tokens
Avoid over-reserving credits:
// Bad: Unlimited or very high limits
{ max_tokens: 100000 }
// Good: Appropriate for task
const tokenLimits = {
shortAnswer: 500,
codeGeneration: 2000,
essay: 4000,
conversation: 1024,
};
Cache Responses
Cache identical requests:
const cache = new Map();
const CACHE_TTL = 60 * 60 * 1000; // 1 hour
async function getCachedCompletion(prompt) {
const cacheKey = JSON.stringify(prompt);
const cached = cache.get(cacheKey);
if (cached && Date.now() - cached.timestamp < CACHE_TTL) {
console.log('Cache hit');
return cached.response;
}
const response = await complete(prompt);
cache.set(cacheKey, {
response,
timestamp: Date.now()
});
return response;
}
Monitor Credit Usage
Track usage patterns:
let totalTokensUsed = 0;
let totalCreditsUsed = 0;
function trackUsage(event) {
if (event.usage) {
const inputTokens = event.usage.inputTokens || 0;
const outputTokens = event.usage.outputTokens || 0;
totalTokensUsed += inputTokens + outputTokens;
// Estimate credits (varies by model)
const estimatedCredits = (inputTokens + outputTokens) * 0.001;
totalCreditsUsed += estimatedCredits;
console.log({
totalTokens: totalTokensUsed,
estimatedCredits: totalCreditsUsed.toFixed(2)
});
}
}
Error Handling
Always Handle All Event Types
function handleEvent(event) {
switch (event.type) {
case 'content':
// Display content
displayContent(event.text);
break;
case 'tool_use':
// Execute tools
executeTools(event.tools);
break;
case 'error':
// Handle error
handleError(event.code, event.message, event.requestId);
break;
default:
// Log unknown types for future compatibility
console.warn('Unknown event type:', event.type);
logUnknownEvent(event);
}
}
Implement Exponential Backoff
Retry only transient failures, honor Retry-After on a 429, and never retry 400, 401, 403, 413 or 422. The retry table is in Errors, and a ready-made helper is in the JavaScript and Python examples.
Log Errors Properly
Log enough for debugging, but not sensitive data:
function logError(error, context) {
console.error({
timestamp: new Date().toISOString(),
error: {
message: error.message,
stack: error.stack,
status: error.status,
},
context: {
model: context.model,
messageCount: context.messages?.length,
hasTools: Boolean(context.options?.tools),
},
rateLimit: {
minuteRemaining: context.headers?.['x-ratelimit-remaining-minute'],
dayRemaining: context.headers?.['x-ratelimit-remaining-day'],
},
// NEVER log: API key, message content, user data
});
}
Tool Execution
Validate Tool Inputs
Never trust tool inputs from the model:
function validateToolInput(toolName, input) {
const schemas = {
get_weather: {
location: (val) => typeof val === 'string' && val.length > 0 && val.length < 100
},
search_database: {
query: (val) => typeof val === 'string' && val.length > 0,
limit: (val) => typeof val === 'number' && val > 0 && val <= 100
}
};
const schema = schemas[toolName];
if (!schema) {
throw new Error(`Unknown tool: ${toolName}`);
}
for (const [field, validator] of Object.entries(schema)) {
if (!validator(input[field])) {
throw new Error(`Invalid ${field} for tool ${toolName}`);
}
}
}
Set Timeouts
Don't let tool execution hang:
async function executeToolWithTimeout(toolName, toolInput, timeoutMs = 5000) {
const timeoutPromise = new Promise((_, reject) => {
setTimeout(() => reject(new Error('Tool execution timeout')), timeoutMs);
});
const executionPromise = executeTool(toolName, toolInput);
try {
return await Promise.race([executionPromise, timeoutPromise]);
} catch (error) {
return {
error: true,
message: `Tool '${toolName}' timed out after ${timeoutMs}ms`
};
}
}
Whitelist Tools
Only allow execution of explicitly defined tools:
const ALLOWED_TOOLS = new Set(['get_weather', 'search_database']);
function executeTool(toolName, toolInput) {
if (!ALLOWED_TOOLS.has(toolName)) {
throw new Error(`Tool ${toolName} not allowed`);
}
// Validate inputs
validateToolInput(toolName, toolInput);
// Execute tool
switch (toolName) {
case 'get_weather':
return getWeather(toolInput.location);
case 'search_database':
return searchDatabase(toolInput.query, toolInput.limit);
default:
throw new Error(`Unknown tool: ${toolName}`);
}
}
Performance
Stream Responses to Users
Display content as it arrives:
client.streamCompletion(model, messages, null, (event) => {
if (event.type === 'content') {
updateUI(event.text); // Update immediately (better UX)
}
});
Use Connection Pooling
Reuse HTTP connections:
import https from 'https';
const agent = new https.Agent({
keepAlive: true,
maxSockets: 10,
keepAliveMsecs: 30000
});
fetch(url, { agent });
Implement Request Deduplication
Avoid duplicate concurrent requests:
const pendingRequests = new Map();
async function deduplicatedRequest(key, fetchFn) {
if (pendingRequests.has(key)) {
console.log('Deduplicating request:', key);
return pendingRequests.get(key);
}
const promise = fetchFn();
pendingRequests.set(key, promise);
try {
return await promise;
} finally {
pendingRequests.delete(key);
}
}
// Usage
const response = await deduplicatedRequest(
`completion:${JSON.stringify(messages)}`,
() => client.complete('claude-sonnet-5', messages)
);
Monitor Latency
Track request performance:
async function timedRequest(url, options) {
const start = Date.now();
try {
const response = await fetch(url, options);
const duration = Date.now() - start;
console.log({
duration_ms: duration,
status: response.status,
url: url
});
return response;
} catch (err) {
const duration = Date.now() - start;
console.error({
duration_ms: duration,
error: err.message,
url: url
});
throw err;
}
}
Production Checklist
Before deploying to production, ensure:
Security
- API keys stored securely (environment variables, secrets manager)
- API keys never exposed client-side
- Server-side proxy implemented for web apps
- Key rotation schedule established
- Separate keys for each environment
- Security review completed
Rate Limiting
- Client-side rate limiting implemented
- Rate limit headers monitored
- Alerts configured for approaching limits
- Request queuing implemented
- Fallback strategy for rate limits
Error Handling
- Exponential backoff retry logic in place
- All event types handled
- Error logging configured
- Alerts configured for error spikes
- Graceful degradation implemented
Cost Management
- Credit usage monitoring set up
- Appropriate models selected
-
max_tokenslimits configured - Response caching implemented (if applicable)
- Budget alerts configured
Tool Execution
- Tool inputs validated
- Timeouts configured
- Whitelist implemented
- Error handling in place
- Security review of tool execution
Performance
- Connection pooling enabled
- Request deduplication implemented
- Latency monitoring set up
- Streaming implemented for UX
- Load testing completed
Reliability
- Health checks implemented
- Monitoring and alerting configured
- Rollback plan prepared
- Documentation for team members
- On-call runbook created
Testing
- Unit tests passing
- Integration tests passing
- Load tests passing
- Error scenarios tested
- Tool calling tested end-to-end
Monitoring & Alerting
Key Metrics to Track
Request Metrics:
- Requests per minute/hour/day
- Success rate (%)
- Error rate (%)
- Average latency (ms)
- P95/P99 latency (ms)
Credit Metrics:
- Credits used per hour/day
- Cost per request
- Token usage (input/output)
- Model distribution
Rate Limit Metrics:
- Rate limit utilization (%)
- Rate limit violations count
- Time spent waiting for rate limits
Error Metrics:
- Error rate by type
- 4xx vs 5xx errors
- Retry success rate
- Timeout rate
Example Monitoring Setup
class MetricsCollector {
constructor() {
this.metrics = {
requests: 0,
errors: 0,
tokens: 0,
credits: 0,
latencies: [],
rateLimitWaits: 0
};
}
recordRequest(duration, success, tokens, credits) {
this.metrics.requests++;
this.metrics.latencies.push(duration);
if (!success) this.metrics.errors++;
if (tokens) this.metrics.tokens += tokens;
if (credits) this.metrics.credits += credits;
// Report to monitoring service
this.report();
}
recordRateLimitWait() {
this.metrics.rateLimitWaits++;
}
report() {
// Send to monitoring service (Datadog, Prometheus, etc.)
console.log({
timestamp: Date.now(),
requests: this.metrics.requests,
errorRate: (this.metrics.errors / this.metrics.requests * 100).toFixed(2) + '%',
avgLatency: this.avg(this.metrics.latencies).toFixed(0) + 'ms',
totalTokens: this.metrics.tokens,
totalCredits: this.metrics.credits.toFixed(2),
rateLimitWaits: this.metrics.rateLimitWaits
});
}
avg(arr) {
return arr.reduce((a, b) => a + b, 0) / arr.length || 0;
}
}
const metrics = new MetricsCollector();
Next Steps
- API Reference - Review complete API spec
- Error Handling - Handle errors properly
- Code Examples - See implementations
- Authentication - Secure API keys