Project Management Workflow¶
The project management system allows researchers to organize evidence collection across multiple searches and maintain persistent research contexts for AI-powered analysis.
Flow Overview¶
Project Selection (/agent/projects)
- Users can create new research projects or select existing ones
- Each project serves as a container for related evidence and search results
- Active project is displayed in the sidebar and persists across sessions
Evidence Collection (/agent → /agent/results)
- All searches automatically associate results with the active project
- Multiple searches can contribute evidence to the same project
- Results page dynamically loads evidence from the active project
- Real-time updates when new searches complete
AI-Powered Analysis (chatbot integration)
- RAG chatbot restricts search context to active project evidence
- Conversations are project-aware and context-specific
- Historical evidence informs AI responses and recommendations
Architecture Overview¶
Frontend Components¶
Project Store (frontend/lib/projectStore.ts)¶
interface Project {
id: string;
name: string;
description?: string;
evidence_count: number;
last_search_date?: string;
created_at: string;
updated_at: string;
}
- Zustand state management for projects and active project selection
- Local storage persistence for active project across browser sessions
- Type-safe interfaces for project data
API Client (frontend/lib/api.ts)¶
- Authentication wrapper around fetch() with Clerk JWT tokens
- Centralized error handling for 401 authentication failures
- Environment-agnostic base URL configuration
- Type-safe methods for all project CRUD operations
Results Page (frontend/app/agent/results/page.tsx)¶
- Dynamic content loading based on active project
- Auto-refresh mechanism when searches complete
- Graceful fallbacks for no project/no evidence states
- Real-time document reloading with 1-second delay for backend synchronization
Backend Architecture¶
Project API (backend/app/api/projects.py)¶
# CRUD operations
POST /projects/ # Create new project
GET /projects/ # List all projects
GET /projects/{id} # Get single project
PUT /projects/{id} # Update project
DELETE /projects/{id} # Delete project + evidence
# Evidence management
GET /projects/{id}/documents # Get project documents
POST /projects/{id}/update-stats # Update project statistics
Search Integration (backend/app/api/agent.py)¶
- Automatic project association for all search results
- Real-time statistics updates (evidence_count, last_search_date)
- Project-aware RAG context for chatbot responses
Vectorization Service (backend/app/services/vectorization.py)¶
- Supabase integration for document and chunk storage
- Project-scoped vector search for RAG functionality
- Automatic cleanup when projects are deleted
Supabase Database Schema¶
Core Tables¶
projects Table¶
CREATE TABLE projects (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(255) NOT NULL,
description TEXT,
evidence_count INTEGER DEFAULT 0,
last_search_date TIMESTAMP WITH TIME ZONE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT now(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT now()
);
documents Table¶
-- Existing table with added project association
ALTER TABLE documents ADD COLUMN project_id UUID REFERENCES projects(id);
CREATE INDEX idx_documents_project_id ON documents(project_id);
chunks Table¶
-- Existing table with added project association
ALTER TABLE chunks ADD COLUMN project_id UUID REFERENCES chunks(id);
CREATE INDEX idx_chunks_project_id ON chunks(project_id);
Database Features¶
pgvector Extension¶
- Vector similarity search for RAG functionality
- Project-scoped embeddings for contextual AI responses
- Efficient indexing for large document collections
Row Level Security (RLS)¶
-- Currently permissive for MVP (all users see all projects)
CREATE POLICY "Allow all operations" ON projects FOR ALL USING (true);
Automatic Triggers¶
-- Auto-update timestamps
CREATE TRIGGER update_projects_updated_at
BEFORE UPDATE ON projects
FOR EACH ROW
EXECUTE FUNCTION update_updated_at_column();
Data Flow¶
Search → Storage Flow¶
- User performs search from Search Hub (
/agent) - Results retrieved from Overton API and processed
- Documents vectorized and stored in Supabase with
project_id - Project statistics updated (evidence_count, last_search_date)
- Results page auto-refreshes to show new evidence
RAG Chat Flow¶
- User sends message with active project context
- Vector search scoped to project documents in Supabase
- Relevant chunks retrieved using pgvector similarity
- AI response generated with project-specific context
- Conversation maintains project awareness throughout
Project Management Flow¶
- User selects/creates project in Projects page
- Active project stored in Zustand + localStorage
- Sidebar updates to show active project name
- All subsequent operations use active project context
- Evidence accumulates across multiple search sessions
Key Features¶
Multi-Search Evidence Collection¶
- Projects can accumulate evidence from multiple searches
- Each search adds to the project's document pool
- No evidence is lost when switching between projects
Persistent Context¶
- Active project persists across browser sessions
- Chatbot maintains project-specific conversation history
- Search results are permanently associated with projects
Real-Time Updates¶
- Results page automatically refreshes when searches complete
- Project statistics update immediately after evidence storage
- UI reflects current state without manual refresh
Graceful Error Handling¶
- Authentication failures provide clear user guidance
- Network errors show helpful retry messages
- Missing projects/evidence display appropriate placeholders
Performance Optimizations¶
- Zustand state management for efficient re-renders
- Local storage for instant active project loading
- Delayed document reloading to ensure backend consistency
Future Enhancements¶
User Personalization¶
- Row Level Security implementation for user-specific projects
- User ID association for project ownership
- Shared project collaboration features
Advanced Analytics¶
- Project-level evidence quality metrics
- Search effectiveness tracking
- Evidence gap analysis
Enhanced Search Context¶
- Cross-project evidence discovery
- Project similarity recommendations
- Automated evidence categorization