feat: implement comprehensive database connection pooling optimization

🎯 SESSION POOLING PERFORMANCE BREAKTHROUGH!

 Critical Issues Fixed:
- Eliminated multiple PrismaClient instances across schedulers
- Fixed connection pool exhaustion risk in processing modules
- Implemented singleton pattern for all database connections
- Added graceful shutdown and connection cleanup

🚀 Enhanced Pooling Features:
- Dual-mode connection pooling (standard + enhanced)
- PostgreSQL native pooling with @prisma/adapter-pg
- Advanced connection monitoring and health checks
- Configurable pool limits and timeouts via environment variables
- Real-time connection statistics and metrics

📊 Performance Optimizations:
- Single shared connection pool across all schedulers
- Configurable connection limits (DATABASE_CONNECTION_LIMIT=20)
- Idle timeout management (DATABASE_POOL_TIMEOUT=10)
- Connection cycling and health validation
- Process termination signal handling

🛠️ New Infrastructure:
- lib/database-pool.ts - Advanced pooling configuration
- app/api/admin/database-health/route.ts - Connection monitoring
- Enhanced lib/prisma.ts with dual-mode support
- Comprehensive documentation in docs/database-connection-pooling.md
- Graceful shutdown handling in lib/schedulers.ts

🎛️ Environment Configuration:
- USE_ENHANCED_POOLING=true for production optimization
- DATABASE_CONNECTION_LIMIT for pool size control
- DATABASE_POOL_TIMEOUT for idle connection management
- Automatic enhanced pooling in production environments

📈 Expected Impact:
- Eliminates connection pool exhaustion under load
- Reduces memory footprint from idle connections
- Improves scheduler performance and reliability
- Enables better resource monitoring and debugging
- Supports horizontal scaling with proper connection management

Production-ready connection pooling with monitoring and health checks!
This commit is contained in:
2025-06-29 09:40:57 +02:00
parent 664affae97
commit 0e526641ce
10 changed files with 508 additions and 25 deletions

View File

@ -1,5 +1,6 @@
// Combined scheduler initialization
// Combined scheduler initialization with graceful shutdown
import { prisma } from "./prisma.js";
import { startProcessingScheduler } from "./processingScheduler";
import { startCsvImportScheduler } from "./scheduler";
@ -16,4 +17,44 @@ export function initializeSchedulers() {
startProcessingScheduler();
console.log("All schedulers initialized successfully");
// Set up graceful shutdown for schedulers
setupGracefulShutdown();
}
/**
* Set up graceful shutdown handling for schedulers and database connections
*/
function setupGracefulShutdown() {
const shutdown = async (signal: string) => {
console.log(`\nReceived ${signal}. Starting graceful shutdown...`);
try {
// Disconnect from database
await prisma.$disconnect();
console.log("Database connections closed.");
// Exit the process
process.exit(0);
} catch (error) {
console.error("Error during shutdown:", error);
process.exit(1);
}
};
// Handle various termination signals
process.on("SIGTERM", () => shutdown("SIGTERM"));
process.on("SIGINT", () => shutdown("SIGINT"));
process.on("SIGUSR2", () => shutdown("SIGUSR2")); // Nodemon restart
// Handle uncaught exceptions
process.on("uncaughtException", (error) => {
console.error("Uncaught Exception:", error);
shutdown("uncaughtException");
});
process.on("unhandledRejection", (reason, promise) => {
console.error("Unhandled Rejection at:", promise, "reason:", reason);
shutdown("unhandledRejection");
});
}