livedash-node/docs/scheduler-architecture.md

Scheduler Architecture for Horizontal Scaling

This document describes the extracted scheduler architecture that enables horizontal scaling of background processing tasks.

Architecture Overview

The scheduler system has been refactored from a monolithic approach to a service-oriented architecture that supports:

• Individual Scheduler Services - Each scheduler runs as a separate service
• Horizontal Scaling - Multiple instances of the same scheduler can run across different machines
• Health Monitoring - Built-in health checks for load balancers and orchestrators
• Graceful Shutdown - Proper handling of shutdown signals for zero-downtime deployments
• Centralized Management - Optional scheduler manager for coordinated operations

Components

1. BaseSchedulerService

Abstract base class providing common functionality:

export abstract class BaseSchedulerService extends EventEmitter {
  // Common scheduler functionality
  protected abstract executeTask(): Promise<void>;

  async start(): Promise<void>;
  async stop(): Promise<void>;
  pause(): void;
  resume(): void;
  getHealthStatus(): HealthStatus;
  getMetrics(): SchedulerMetrics;
}

Features:

• Status management (STOPPED, STARTING, RUNNING, PAUSED, ERROR)
• Metrics collection (run counts, timing, success/failure rates)
• Event emission for monitoring
• Configurable intervals and timeouts
• Automatic retry handling

2. Individual Scheduler Services

CsvImportSchedulerService

Handles periodic CSV data import from companies:

const csvScheduler = new CsvImportSchedulerService({
  interval: "*/10 * * * *", // Every 10 minutes
  batchSize: 10,
  maxConcurrentImports: 5,
  timeout: 300000, // 5 minutes
});

Features:

• Batch processing with configurable concurrency
• Duplicate detection
• Company-specific error handling
• Progress monitoring

Additional Schedulers (To Be Implemented)

• ImportProcessingSchedulerService - Process imported CSV data into sessions
• SessionProcessingSchedulerService - AI analysis and categorization
• BatchProcessingSchedulerService - OpenAI Batch API integration

3. SchedulerManager

Orchestrates multiple schedulers in a single process:

const manager = new SchedulerManager();

manager.registerScheduler({
  id: "csv-import",
  name: "CSV Import Scheduler",
  service: new CsvImportSchedulerService(),
  autoStart: true,
  critical: true, // Auto-restart on failure
});

await manager.startAll();

Features:

• Automatic restart of failed critical schedulers
• Health monitoring across all schedulers
• Coordinated start/stop operations
• Event aggregation and logging

4. Standalone Scheduler Runner

Runs individual schedulers as separate processes:

# Run CSV import scheduler as standalone process
npx tsx lib/services/schedulers/StandaloneSchedulerRunner.ts --scheduler=csv-import

# List available schedulers
npx tsx lib/services/schedulers/StandaloneSchedulerRunner.ts --list

Features:

• Independent process execution
• Environment variable configuration
• Graceful shutdown handling
• Health reporting for monitoring

Deployment Patterns

1. Single Process (Current Default)

All schedulers run within the main Next.js server process:

// server.ts
import { initializeSchedulers } from "./lib/services/schedulers/ServerSchedulerIntegration";

await initializeSchedulers();

Pros:

• Simple deployment
• Lower resource usage
• Easy local development

Cons:

• Limited scalability
• Single point of failure
• Resource contention

2. Separate Processes

Each scheduler runs as an independent process:

# Terminal 1: Main application
npm run dev

# Terminal 2: CSV Import Scheduler
npm run scheduler:csv-import

# Terminal 3: Session Processing Scheduler
npm run scheduler:session-processing

Pros:

• Independent scaling
• Fault isolation
• Resource optimization per scheduler

Cons:

• More complex deployment
• Higher resource overhead
• Inter-process coordination needed

3. Container Orchestration (Recommended for Production)

Each scheduler runs in separate containers managed by Kubernetes/Docker Swarm:

# docker-compose.yml
version: "3.8"
services:
  app:
    build: .
    environment:
      - SCHEDULER_ENABLED=false # Disable in-process schedulers

  csv-import-scheduler:
    build: .
    command: npx tsx lib/services/schedulers/StandaloneSchedulerRunner.ts --scheduler=csv-import
    environment:
      - CSV_IMPORT_INTERVAL=*/10 * * * *
      - CSV_IMPORT_BATCH_SIZE=10

  session-processing-scheduler:
    build: .
    command: npx tsx lib/services/schedulers/StandaloneSchedulerRunner.ts --scheduler=session-processing
    environment:
      - SESSION_PROCESSING_INTERVAL=*/5 * * * *

Pros:

• Full horizontal scaling
• Independent resource allocation
• Health monitoring integration
• Zero-downtime deployments

Cons:

• Complex orchestration setup
• Network latency considerations
• Distributed system challenges

Configuration

Environment Variables

# Global Scheduler Settings
SCHEDULER_ENABLED=true
SCHEDULER_AUTO_RESTART=true

# CSV Import Scheduler
CSV_IMPORT_INTERVAL="*/10 * * * *"
CSV_IMPORT_BATCH_SIZE=10
CSV_IMPORT_MAX_CONCURRENT=5
CSV_IMPORT_TIMEOUT=300000

# Import Processing Scheduler
IMPORT_PROCESSING_INTERVAL="*/2 * * * *"
IMPORT_PROCESSING_TIMEOUT=120000

# Session Processing Scheduler
SESSION_PROCESSING_INTERVAL="*/5 * * * *"
SESSION_PROCESSING_BATCH_SIZE=50

# Batch Processing Scheduler
BATCH_PROCESSING_INTERVAL="*/5 * * * *"
BATCH_PROCESSING_CHECK_INTERVAL="*/2 * * * *"

Package.json Scripts

{
  "scripts": {
    "scheduler:csv-import": "tsx lib/services/schedulers/StandaloneSchedulerRunner.ts --scheduler=csv-import",
    "scheduler:import-processing": "tsx lib/services/schedulers/StandaloneSchedulerRunner.ts --scheduler=import-processing",
    "scheduler:session-processing": "tsx lib/services/schedulers/StandaloneSchedulerRunner.ts --scheduler=session-processing",
    "scheduler:batch-processing": "tsx lib/services/schedulers/StandaloneSchedulerRunner.ts --scheduler=batch-processing"
  }
}

Health Monitoring

Health Check Endpoints

# Overall scheduler health
GET /api/admin/schedulers/health

# Scheduler management
GET /api/admin/schedulers
POST /api/admin/schedulers

Response Format

{
  "healthy": true,
  "status": "healthy",
  "timestamp": "2024-01-15T10:30:00.000Z",
  "schedulers": {
    "total": 4,
    "running": 4,
    "errors": 0
  },
  "details": {
    "csv-import": {
      "status": "RUNNING",
      "healthy": true,
      "lastSuccess": "2024-01-15T10:25:00.000Z"
    }
  }
}

Kubernetes Integration

apiVersion: apps/v1
kind: Deployment
metadata:
  name: csv-import-scheduler
spec:
  template:
    spec:
      containers:
        - name: scheduler
          image: livedash:latest
          command:
            [
              "npx",
              "tsx",
              "lib/services/schedulers/StandaloneSchedulerRunner.ts",
              "--scheduler=csv-import",
            ]
          livenessProbe:
            httpGet:
              path: /api/admin/schedulers/health
              port: 3000
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /api/admin/schedulers/health
              port: 3000
            initialDelaySeconds: 5
            periodSeconds: 5

Scaling Strategies

1. Vertical Scaling

Increase resources for scheduler processes:

# docker-compose.yml
csv-import-scheduler:
  deploy:
    resources:
      limits:
        cpus: "2.0"
        memory: 2G
      reservations:
        cpus: "1.0"
        memory: 1G

2. Horizontal Scaling

Run multiple instances of the same scheduler:

# Kubernetes
apiVersion: apps/v1
kind: Deployment
metadata:
  name: csv-import-scheduler
spec:
  replicas: 3 # Multiple instances
  template:
    spec:
      containers:
        - name: scheduler
          env:
            - name: SCHEDULER_INSTANCE_ID
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name

Note: Ensure scheduler logic handles multiple instances correctly (e.g., using database locks or partitioning).

3. Geographic Distribution

Deploy schedulers across different regions:

# Region-specific scheduling
csv-import-scheduler-us:
  environment:
    - REGION=us
    - CSV_COMPANIES_FILTER=region:us

csv-import-scheduler-eu:
  environment:
    - REGION=eu
    - CSV_COMPANIES_FILTER=region:eu

Migration Guide

From Current Architecture

1. Phase 1: Extract Schedulers

• ✅ Create BaseSchedulerService
• ✅ Implement CsvImportSchedulerService
• ✅ Create SchedulerManager
• ⏳ Implement remaining scheduler services

2. Phase 2: Deployment Options

• ✅ Add ServerSchedulerIntegration for backwards compatibility
• ✅ Create StandaloneSchedulerRunner
• ✅ Add health check endpoints

3. Phase 3: Container Support

• ⏳ Create Dockerfile for scheduler containers
• ⏳ Add Kubernetes manifests
• ⏳ Implement distributed coordination

4. Phase 4: Production Migration

• ⏳ Deploy separate scheduler containers
• ⏳ Monitor performance and stability
• ⏳ Gradually increase horizontal scaling

Breaking Changes

• Scheduler initialization moved from server.ts to ServerSchedulerIntegration
• Individual scheduler functions replaced with service classes
• Configuration moved to environment variables

Benefits

1. Scalability: Independent scaling of different scheduler types
2. Reliability: Fault isolation prevents cascading failures
3. Performance: Optimized resource allocation per scheduler
4. Monitoring: Granular health checks and metrics
5. Deployment: Zero-downtime updates and rollbacks
6. Development: Easier testing and debugging of individual schedulers

Next Steps

1. Implement remaining scheduler services (ImportProcessing, SessionProcessing, BatchProcessing)
2. Add distributed coordination for multi-instance schedulers
3. Create Kubernetes operators for automatic scaling
4. Implement scheduler-specific metrics and dashboards
5. Add scheduler performance optimization tools