mirror of
https://github.com/kjanat/livedash-node.git
synced 2026-01-16 10:12:09 +01:00
feat: comprehensive security and architecture improvements
- Add Zod validation schemas with strong password requirements (12+ chars, complexity) - Implement rate limiting for authentication endpoints (registration, password reset) - Remove duplicate MetricCard component, consolidate to ui/metric-card.tsx - Update README.md to use pnpm commands consistently - Enhance authentication security with 12-round bcrypt hashing - Add comprehensive input validation for all API endpoints - Fix security vulnerabilities in user registration and password reset flows 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@ -12,10 +12,10 @@ The WordCloud component visualizes categories or topics based on their frequency
|
||||
|
||||
**Features:**
|
||||
|
||||
- Dynamic sizing based on frequency
|
||||
- Colorful display with a pleasing color palette
|
||||
- Responsive design
|
||||
- Interactive hover effects
|
||||
- Dynamic sizing based on frequency
|
||||
- Colorful display with a pleasing color palette
|
||||
- Responsive design
|
||||
- Interactive hover effects
|
||||
|
||||
### 2. GeographicMap
|
||||
|
||||
@ -25,10 +25,10 @@ This component displays a world map with circles representing the number of sess
|
||||
|
||||
**Features:**
|
||||
|
||||
- Interactive map using React Leaflet
|
||||
- Circle sizes scaled by session count
|
||||
- Tooltips showing country names and session counts
|
||||
- Responsive design
|
||||
- Interactive map using React Leaflet
|
||||
- Circle sizes scaled by session count
|
||||
- Tooltips showing country names and session counts
|
||||
- Responsive design
|
||||
|
||||
### 3. MetricCard
|
||||
|
||||
@ -38,10 +38,10 @@ A modern, visually appealing card for displaying key metrics.
|
||||
|
||||
**Features:**
|
||||
|
||||
- Multiple design variants (default, primary, success, warning, danger)
|
||||
- Support for trend indicators
|
||||
- Icons and descriptions
|
||||
- Clean, modern styling
|
||||
- Multiple design variants (default, primary, success, warning, danger)
|
||||
- Support for trend indicators
|
||||
- Icons and descriptions
|
||||
- Clean, modern styling
|
||||
|
||||
### 4. DonutChart
|
||||
|
||||
@ -51,10 +51,10 @@ An enhanced donut chart with better styling and a central text display capabilit
|
||||
|
||||
**Features:**
|
||||
|
||||
- Customizable colors
|
||||
- Center text area for displaying summaries
|
||||
- Interactive tooltips with percentages
|
||||
- Well-balanced legend display
|
||||
- Customizable colors
|
||||
- Center text area for displaying summaries
|
||||
- Interactive tooltips with percentages
|
||||
- Well-balanced legend display
|
||||
|
||||
### 5. ResponseTimeDistribution
|
||||
|
||||
@ -64,10 +64,10 @@ Visualizes the distribution of response times as a histogram.
|
||||
|
||||
**Features:**
|
||||
|
||||
- Color-coded bars (green for fast, yellow for medium, red for slow)
|
||||
- Target time indicator
|
||||
- Automatic binning of response times
|
||||
- Clear labeling and scales
|
||||
- Color-coded bars (green for fast, yellow for medium, red for slow)
|
||||
- Target time indicator
|
||||
- Automatic binning of response times
|
||||
- Clear labeling and scales
|
||||
|
||||
## Dashboard Enhancements
|
||||
|
||||
@ -85,7 +85,7 @@ The dashboard has been enhanced with:
|
||||
|
||||
## Usage Notes
|
||||
|
||||
- The geographic map and response time distribution use simulated data where actual data is not available
|
||||
- All components are responsive and will adjust to different screen sizes
|
||||
- The dashboard automatically refreshes data when using the refresh button
|
||||
- Admin users have access to additional controls at the bottom of the dashboard
|
||||
- The geographic map and response time distribution use simulated data where actual data is not available
|
||||
- All components are responsive and will adjust to different screen sizes
|
||||
- The dashboard automatically refreshes data when using the refresh button
|
||||
- Admin users have access to additional controls at the bottom of the dashboard
|
||||
|
||||
@ -17,48 +17,48 @@ Successfully migrated the livedash-node application from SQLite to PostgreSQL us
|
||||
|
||||
#### Production/Development
|
||||
|
||||
- **Provider**: PostgreSQL (Neon)
|
||||
- **Environment Variable**: `DATABASE_URL`
|
||||
- **Connection**: Neon PostgreSQL cluster
|
||||
- **Provider**: PostgreSQL (Neon)
|
||||
- **Environment Variable**: `DATABASE_URL`
|
||||
- **Connection**: Neon PostgreSQL cluster
|
||||
|
||||
#### Testing
|
||||
|
||||
- **Provider**: PostgreSQL (Neon - separate database)
|
||||
- **Environment Variable**: `DATABASE_URL_TEST`
|
||||
- **Test Setup**: Automatically switches to test database during test runs
|
||||
- **Provider**: PostgreSQL (Neon - separate database)
|
||||
- **Environment Variable**: `DATABASE_URL_TEST`
|
||||
- **Test Setup**: Automatically switches to test database during test runs
|
||||
|
||||
### Files Modified
|
||||
|
||||
1. **`prisma/schema.prisma`**
|
||||
|
||||
- Changed provider from `sqlite` to `postgresql`
|
||||
- Updated URL to use `env("DATABASE_URL")`
|
||||
- Changed provider from `sqlite` to `postgresql`
|
||||
- Updated URL to use `env("DATABASE_URL")`
|
||||
|
||||
2. **`tests/setup.ts`**
|
||||
|
||||
- Added logic to use `DATABASE_URL_TEST` when available
|
||||
- Ensures test isolation with separate database
|
||||
- Added logic to use `DATABASE_URL_TEST` when available
|
||||
- Ensures test isolation with separate database
|
||||
|
||||
3. **`.env`** (created)
|
||||
|
||||
- Contains `DATABASE_URL` for Prisma CLI operations
|
||||
- Contains `DATABASE_URL` for Prisma CLI operations
|
||||
|
||||
4. **`.env.local`** (existing)
|
||||
|
||||
- Contains both `DATABASE_URL` and `DATABASE_URL_TEST`
|
||||
- Contains both `DATABASE_URL` and `DATABASE_URL_TEST`
|
||||
|
||||
### Database Schema
|
||||
|
||||
All existing models and relationships were preserved:
|
||||
|
||||
- **Company**: Multi-tenant root entity
|
||||
- **User**: Authentication and authorization
|
||||
- **Session**: Processed session data
|
||||
- **SessionImport**: Raw CSV import data
|
||||
- **Message**: Individual conversation messages
|
||||
- **Question**: Normalized question storage
|
||||
- **SessionQuestion**: Session-question relationships
|
||||
- **AIProcessingRequest**: AI cost tracking
|
||||
- **Company**: Multi-tenant root entity
|
||||
- **User**: Authentication and authorization
|
||||
- **Session**: Processed session data
|
||||
- **SessionImport**: Raw CSV import data
|
||||
- **Message**: Individual conversation messages
|
||||
- **Question**: Normalized question storage
|
||||
- **SessionQuestion**: Session-question relationships
|
||||
- **AIProcessingRequest**: AI cost tracking
|
||||
|
||||
### Migration Process
|
||||
|
||||
@ -76,7 +76,7 @@ All existing models and relationships were preserved:
|
||||
✅ **Advanced Features**: Full JSON support, arrays, advanced indexing
|
||||
✅ **Test Isolation**: Separate test database prevents data conflicts
|
||||
✅ **Consistency**: Same database engine across all environments
|
||||
✅ **Cloud-Native**: Neon provides managed PostgreSQL with excellent DX
|
||||
✅ **Cloud-Native**: Neon provides managed PostgreSQL with excellent DX
|
||||
|
||||
### Environment Variables
|
||||
|
||||
@ -103,11 +103,11 @@ if (process.env.DATABASE_URL_TEST) {
|
||||
|
||||
All tests pass successfully:
|
||||
|
||||
- ✅ Environment configuration tests
|
||||
- ✅ Transcript fetcher tests
|
||||
- ✅ Database connection tests
|
||||
- ✅ Schema validation tests
|
||||
- ✅ CRUD operation tests
|
||||
- ✅ Environment configuration tests
|
||||
- ✅ Transcript fetcher tests
|
||||
- ✅ Database connection tests
|
||||
- ✅ Schema validation tests
|
||||
- ✅ CRUD operation tests
|
||||
|
||||
### Next Steps
|
||||
|
||||
|
||||
@ -8,8 +8,8 @@
|
||||
|
||||
**Solution**:
|
||||
|
||||
- Added validation in `fetchAndStoreSessionsForAllCompanies()` to skip companies with example/invalid URLs
|
||||
- Removed the invalid company record from the database using `fix_companies.js`
|
||||
- Added validation in `fetchAndStoreSessionsForAllCompanies()` to skip companies with example/invalid URLs
|
||||
- Removed the invalid company record from the database using `fix_companies.js`
|
||||
|
||||
### 2. Transcript Fetching Errors
|
||||
|
||||
@ -17,10 +17,10 @@
|
||||
|
||||
**Solution**:
|
||||
|
||||
- Improved error handling in `fetchTranscriptContent()` function
|
||||
- Added probabilistic logging (only ~10% of errors logged) to prevent log spam
|
||||
- Added timeout (10 seconds) for transcript fetching
|
||||
- Made transcript fetching failures non-blocking (sessions are still created without transcript content)
|
||||
- Improved error handling in `fetchTranscriptContent()` function
|
||||
- Added probabilistic logging (only ~10% of errors logged) to prevent log spam
|
||||
- Added timeout (10 seconds) for transcript fetching
|
||||
- Made transcript fetching failures non-blocking (sessions are still created without transcript content)
|
||||
|
||||
### 3. CSV Fetching Errors
|
||||
|
||||
@ -28,8 +28,8 @@
|
||||
|
||||
**Solution**:
|
||||
|
||||
- Added URL validation to skip companies with `example.com` URLs
|
||||
- Improved error logging to be more descriptive
|
||||
- Added URL validation to skip companies with `example.com` URLs
|
||||
- Improved error logging to be more descriptive
|
||||
|
||||
## Current Status
|
||||
|
||||
@ -42,22 +42,23 @@
|
||||
|
||||
After cleanup, only valid companies remain:
|
||||
|
||||
- **Demo Company** (`790b9233-d369-451f-b92c-f4dceb42b649`)
|
||||
- CSV URL: `https://proto.notso.ai/jumbo/chats`
|
||||
- Has valid authentication credentials
|
||||
- 107 sessions in database
|
||||
- **Demo Company** (`790b9233-d369-451f-b92c-f4dceb42b649`)
|
||||
- CSV URL: `https://proto.notso.ai/jumbo/chats`
|
||||
- Has valid authentication credentials
|
||||
- 107 sessions in database
|
||||
|
||||
## Files Modified
|
||||
|
||||
1. **lib/csvFetcher.js**
|
||||
|
||||
- Added company URL validation
|
||||
- Improved transcript fetching error handling
|
||||
- Reduced error log verbosity
|
||||
- Added company URL validation
|
||||
- Improved transcript fetching error handling
|
||||
- Reduced error log verbosity
|
||||
|
||||
2. **fix_companies.js** (cleanup script)
|
||||
- Removes invalid company records
|
||||
- Can be run again if needed
|
||||
|
||||
- Removes invalid company records
|
||||
- Can be run again if needed
|
||||
|
||||
## Monitoring
|
||||
|
||||
|
||||
@ -15,22 +15,22 @@ The system now includes an automated process for analyzing chat session transcri
|
||||
|
||||
### Session Fetching
|
||||
|
||||
- The system fetches session data from configured CSV URLs for each company
|
||||
- Unlike the previous implementation, it now only adds sessions that don't already exist in the database
|
||||
- This prevents duplicate sessions and allows for incremental updates
|
||||
- The system fetches session data from configured CSV URLs for each company
|
||||
- Unlike the previous implementation, it now only adds sessions that don't already exist in the database
|
||||
- This prevents duplicate sessions and allows for incremental updates
|
||||
|
||||
### Transcript Processing
|
||||
|
||||
- For sessions with transcript content that haven't been processed yet, the system calls OpenAI's API
|
||||
- The API analyzes the transcript and extracts the following information:
|
||||
- Primary language used (ISO 639-1 code)
|
||||
- Number of messages sent by the user
|
||||
- Overall sentiment (positive, neutral, negative)
|
||||
- Whether the conversation was escalated
|
||||
- Whether HR contact was mentioned or provided
|
||||
- Best-fitting category for the conversation
|
||||
- Up to 5 paraphrased questions asked by the user
|
||||
- A brief summary of the conversation
|
||||
- For sessions with transcript content that haven't been processed yet, the system calls OpenAI's API
|
||||
- The API analyzes the transcript and extracts the following information:
|
||||
- Primary language used (ISO 639-1 code)
|
||||
- Number of messages sent by the user
|
||||
- Overall sentiment (positive, neutral, negative)
|
||||
- Whether the conversation was escalated
|
||||
- Whether HR contact was mentioned or provided
|
||||
- Best-fitting category for the conversation
|
||||
- Up to 5 paraphrased questions asked by the user
|
||||
- A brief summary of the conversation
|
||||
|
||||
### Scheduling
|
||||
|
||||
@ -43,10 +43,10 @@ The system includes two schedulers:
|
||||
|
||||
The Session model has been updated with new fields to store the processed data:
|
||||
|
||||
- `processed`: Boolean flag indicating whether the session has been processed
|
||||
- `sentimentCategory`: String value ("positive", "neutral", "negative") from OpenAI
|
||||
- `questions`: JSON array of questions asked by the user
|
||||
- `summary`: Brief summary of the conversation
|
||||
- `processed`: Boolean flag indicating whether the session has been processed
|
||||
- `sentimentCategory`: String value ("positive", "neutral", "negative") from OpenAI
|
||||
- `questions`: JSON array of questions asked by the user
|
||||
- `summary`: Brief summary of the conversation
|
||||
|
||||
## Configuration
|
||||
|
||||
@ -62,9 +62,9 @@ OPENAI_API_KEY=your_api_key_here
|
||||
|
||||
To run the application with schedulers enabled:
|
||||
|
||||
- Development: `npm run dev`
|
||||
- Development (with schedulers disabled): `npm run dev:no-schedulers`
|
||||
- Production: `npm run start`
|
||||
- Development: `npm run dev`
|
||||
- Development (with schedulers disabled): `npm run dev:no-schedulers`
|
||||
- Production: `npm run start`
|
||||
|
||||
Note: These commands will start a custom Next.js server with the schedulers enabled. You'll need to have an OpenAI API key set in your `.env.local` file for the session processing to work.
|
||||
|
||||
@ -82,5 +82,5 @@ This will process all unprocessed sessions that have transcript content.
|
||||
|
||||
The processing logic can be customized by modifying:
|
||||
|
||||
- `lib/processingScheduler.ts`: Contains the OpenAI processing logic
|
||||
- `scripts/process_sessions.ts`: Standalone script for manual processing
|
||||
- `lib/processingScheduler.ts`: Contains the OpenAI processing logic
|
||||
- `scripts/process_sessions.ts`: Standalone script for manual processing
|
||||
|
||||
Reference in New Issue
Block a user