mirror of
https://github.com/kjanat/livedash-node.git
synced 2026-01-16 15:32:10 +01:00
Kaj Kowalski
e2301725a3
- Set up pre-commit hooks with husky and lint-staged for automated code quality - Improved TypeScript type safety by replacing 'any' types with proper generics - Fixed markdown linting violations (MD030 spacing) across all documentation - Fixed compound adjective hyphenation in technical documentation - Fixed invalid JSON union syntax in API documentation examples - Automated code formatting and linting on commit - Enhanced error handling with better type constraints - Configured biome and markdownlint for consistent code style - All changes verified with successful production build
131 lines
4.2 KiB
Markdown
131 lines
4.2 KiB
Markdown
# PostgreSQL Migration Documentation
|
|
|
|
## Overview
|
|
|
|
Successfully migrated the livedash-node application from SQLite to PostgreSQL using Neon as the database provider. This migration provides better scalability, performance, and production-readiness.
|
|
|
|
## Migration Summary
|
|
|
|
### What Was Changed
|
|
|
|
1. **Database Provider**: Changed from SQLite to PostgreSQL in `prisma/schema.prisma`
|
|
2. **Environment Configuration**: Updated to use environment-based database URL selection
|
|
3. **Test Setup**: Configured separate test database using `DATABASE_URL_TEST`
|
|
4. **Migration History**: Reset and created fresh PostgreSQL migrations
|
|
|
|
### Database Configuration
|
|
|
|
#### Production/Development
|
|
|
|
- **Provider**: PostgreSQL (Neon)
|
|
- **Environment Variable**: `DATABASE_URL`
|
|
- **Connection**: Neon PostgreSQL cluster
|
|
|
|
#### Testing
|
|
|
|
- **Provider**: PostgreSQL (Neon - separate database)
|
|
- **Environment Variable**: `DATABASE_URL_TEST`
|
|
- **Test Setup**: Automatically switches to test database during test runs
|
|
|
|
### Files Modified
|
|
|
|
1. **`prisma/schema.prisma`**
|
|
|
|
- Changed provider from `sqlite` to `postgresql`
|
|
- Updated URL to use `env("DATABASE_URL")`
|
|
|
|
2. **`tests/setup.ts`**
|
|
|
|
- Added logic to use `DATABASE_URL_TEST` when available
|
|
- Ensures test isolation with separate database
|
|
|
|
3. **`.env`** (created)
|
|
|
|
- Contains `DATABASE_URL` for Prisma CLI operations
|
|
|
|
4. **`.env.local`** (existing)
|
|
|
|
- Contains both `DATABASE_URL` and `DATABASE_URL_TEST`
|
|
|
|
### Database Schema
|
|
|
|
All existing models and relationships were preserved:
|
|
|
|
- **Company**: Multi-tenant root entity
|
|
- **User**: Authentication and authorization
|
|
- **Session**: Processed session data
|
|
- **SessionImport**: Raw CSV import data
|
|
- **Message**: Individual conversation messages
|
|
- **Question**: Normalized question storage
|
|
- **SessionQuestion**: Session-question relationships
|
|
- **AIProcessingRequest**: AI cost tracking
|
|
|
|
### Migration Process
|
|
|
|
1. **Schema Update**: Changed provider to PostgreSQL
|
|
2. **Migration Reset**: Removed SQLite migration history
|
|
3. **Fresh Migration**: Created new PostgreSQL migration
|
|
4. **Client Generation**: Generated new Prisma client for PostgreSQL
|
|
5. **Database Seeding**: Applied initial seed data
|
|
6. **Testing**: Verified all functionality works with PostgreSQL
|
|
|
|
### Benefits Achieved
|
|
|
|
✅ **Production-Ready**: PostgreSQL is enterprise-grade and scalable
|
|
✅ **Better Performance**: Superior query performance and optimization
|
|
✅ **Advanced Features**: Full JSON support, arrays, advanced indexing
|
|
✅ **Test Isolation**: Separate test database prevents data conflicts
|
|
✅ **Consistency**: Same database engine across all environments
|
|
✅ **Cloud-Native**: Neon provides managed PostgreSQL with excellent DX
|
|
|
|
### Environment Variables
|
|
|
|
```env
|
|
# Production/Development Database
|
|
DATABASE_URL="postgresql://user:pass@host/database?sslmode=require"
|
|
|
|
# Test Database (separate Neon database)
|
|
DATABASE_URL_TEST="postgresql://user:pass@test-host/test-database?sslmode=require"
|
|
```
|
|
|
|
### Test Configuration
|
|
|
|
Tests automatically use the test database when `DATABASE_URL_TEST` is set:
|
|
|
|
```typescript
|
|
// In tests/setup.ts
|
|
if (process.env.DATABASE_URL_TEST) {
|
|
process.env.DATABASE_URL = process.env.DATABASE_URL_TEST;
|
|
}
|
|
```
|
|
|
|
### Verification
|
|
|
|
All tests pass successfully:
|
|
|
|
- ✅ Environment configuration tests
|
|
- ✅ Transcript fetcher tests
|
|
- ✅ Database connection tests
|
|
- ✅ Schema validation tests
|
|
- ✅ CRUD operation tests
|
|
|
|
### Next Steps
|
|
|
|
1. **Data Import**: Import production data if needed
|
|
2. **Performance Monitoring**: Monitor query performance in production
|
|
3. **Backup Strategy**: Configure automated backups via Neon
|
|
4. **Connection Pooling**: Consider connection pooling for high-traffic scenarios
|
|
|
|
### Rollback Plan
|
|
|
|
If rollback is needed:
|
|
|
|
1. Revert `prisma/schema.prisma` to SQLite configuration
|
|
2. Restore SQLite migration files from git history
|
|
3. Update environment variables
|
|
4. Run `prisma migrate reset` and `prisma generate`
|
|
|
|
## Conclusion
|
|
|
|
The PostgreSQL migration was successful and provides a solid foundation for production deployment. The application now benefits from PostgreSQL's advanced features while maintaining full test isolation and development workflow compatibility.
|