feat: implement comprehensive database connection pooling optimization

🎯 SESSION POOLING PERFORMANCE BREAKTHROUGH!

 Critical Issues Fixed:
- Eliminated multiple PrismaClient instances across schedulers
- Fixed connection pool exhaustion risk in processing modules
- Implemented singleton pattern for all database connections
- Added graceful shutdown and connection cleanup

🚀 Enhanced Pooling Features:
- Dual-mode connection pooling (standard + enhanced)
- PostgreSQL native pooling with @prisma/adapter-pg
- Advanced connection monitoring and health checks
- Configurable pool limits and timeouts via environment variables
- Real-time connection statistics and metrics

📊 Performance Optimizations:
- Single shared connection pool across all schedulers
- Configurable connection limits (DATABASE_CONNECTION_LIMIT=20)
- Idle timeout management (DATABASE_POOL_TIMEOUT=10)
- Connection cycling and health validation
- Process termination signal handling

🛠️ New Infrastructure:
- lib/database-pool.ts - Advanced pooling configuration
- app/api/admin/database-health/route.ts - Connection monitoring
- Enhanced lib/prisma.ts with dual-mode support
- Comprehensive documentation in docs/database-connection-pooling.md
- Graceful shutdown handling in lib/schedulers.ts

🎛️ Environment Configuration:
- USE_ENHANCED_POOLING=true for production optimization
- DATABASE_CONNECTION_LIMIT for pool size control
- DATABASE_POOL_TIMEOUT for idle connection management
- Automatic enhanced pooling in production environments

📈 Expected Impact:
- Eliminates connection pool exhaustion under load
- Reduces memory footprint from idle connections
- Improves scheduler performance and reliability
- Enables better resource monitoring and debugging
- Supports horizontal scaling with proper connection management

Production-ready connection pooling with monitoring and health checks!
This commit is contained in:
2025-06-29 09:40:57 +02:00
parent 664affae97
commit 0e526641ce
10 changed files with 508 additions and 25 deletions

View File

@ -1,19 +1,14 @@
// SessionImport to Session processor
import {
PrismaClient,
ProcessingStage,
SentimentCategory,
} from "@prisma/client";
import { ProcessingStage, SentimentCategory } from "@prisma/client";
import cron from "node-cron";
import { getSchedulerConfig } from "./env";
import { prisma } from "./prisma.js";
import { ProcessingStatusManager } from "./processingStatusManager";
import {
fetchTranscriptContent,
isValidTranscriptUrl,
} from "./transcriptFetcher";
const prisma = new PrismaClient();
interface ImportRecord {
id: string;
companyId: string;