555 lines
13 KiB
Markdown
555 lines
13 KiB
Markdown
# Modaic Server - FastAPI Backend
|
|
|
|
**AI Agent Platform Backend Service**
|
|
|
|
This is the FastAPI backend server for the Modaic platform, providing a comprehensive REST API for AI agent management, user authentication, Git repository operations, and collaborative features.
|
|
|
|
   
|
|
|
|
## Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
- **Python 3.13** or higher
|
|
- **UV** package manager
|
|
- **PostgreSQL** database
|
|
- **Stytch** account for authentication
|
|
- **AWS S3** bucket for file storage
|
|
- **Gitea** server for Git operations
|
|
|
|
### Installation
|
|
|
|
1. **Navigate to server directory**
|
|
|
|
```bash
|
|
cd server
|
|
```
|
|
|
|
2. **Install dependencies**
|
|
|
|
```bash
|
|
uv sync
|
|
```
|
|
|
|
3. **Set up environment variables**
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env with your actual values
|
|
```
|
|
|
|
4. **Start the development server**
|
|
|
|
```bash
|
|
uv run uvicorn src.main:app --reload
|
|
```
|
|
|
|
5. **Access the API**
|
|
- **API Server**: http://localhost:8000
|
|
- **Interactive Docs**: http://localhost:8000/docs
|
|
- **ReDoc**: http://localhost:8000/redoc
|
|
|
|
## Architecture
|
|
|
|
### Application Structure
|
|
|
|
```
|
|
src/
|
|
api/ # API endpoints organized by feature
|
|
v1/
|
|
auth/ # Authentication endpoints
|
|
agent/ # Agent management endpoints
|
|
user/ # User management endpoints
|
|
tokens/ # API token management
|
|
core/ # Core configuration
|
|
config.py # Application settings
|
|
db/ # Database configuration
|
|
index.py # Database connection
|
|
pg.py # PostgreSQL setup
|
|
lib/ # External service integrations
|
|
gitea.py # Gitea API client
|
|
s3.py # AWS S3 operations
|
|
stytch.py # Stytch authentication
|
|
logger.py # Logging configuration
|
|
objects/ # Data models and schemas
|
|
models/ # SQLAlchemy models
|
|
schemas/ # Pydantic schemas
|
|
service/ # Business logic services
|
|
email.py # Email service
|
|
utils/ # Utility functions
|
|
date.py # Date utilities
|
|
gitea.py # Gitea helpers
|
|
user.py # User utilities
|
|
main.py # FastAPI application entry point
|
|
```
|
|
|
|
### Key Components
|
|
|
|
**FastAPI Application** (`src/main.py`)
|
|
|
|
- CORS configuration for frontend integration
|
|
- API route registration
|
|
- Middleware setup
|
|
- Error handling
|
|
|
|
**Database Layer** (`src/db/`)
|
|
|
|
- PostgreSQL connection with SQLAlchemy
|
|
- Async database operations
|
|
- Connection pooling
|
|
|
|
**Authentication** (`src/lib/stytch.py`)
|
|
|
|
- Stytch integration for user management
|
|
- JWT token validation
|
|
- Session management
|
|
|
|
**Git Operations** (`src/lib/gitea.py`)
|
|
|
|
- Gitea API integration
|
|
- Repository CRUD operations
|
|
- File content management
|
|
- Branch and commit operations
|
|
|
|
## API Endpoints
|
|
|
|
### Authentication (`/api/v1/auth`)
|
|
|
|
| Method | Endpoint | Description |
|
|
| ------ | --------------- | ----------------------------- |
|
|
| POST | `/register` | User registration with Stytch |
|
|
| POST | `/login` | User authentication |
|
|
| GET | `/authenticate` | OAuth callback handling |
|
|
| GET | `/me` | Get current user information |
|
|
| POST | `/onboarding` | Complete user onboarding |
|
|
| DELETE | `/session` | User logout |
|
|
|
|
### User Management (`/api/v1/user`)
|
|
|
|
| Method | Endpoint | Description |
|
|
| ------ | ---------------------- | --------------------- |
|
|
| GET | `/{userId}` | Get user by ID |
|
|
| GET | `/username/{username}` | Get user by username |
|
|
| PUT | `/{userId}` | Update user profile |
|
|
| DELETE | `/{userId}` | Delete user account |
|
|
| GET | `/check/email/` | Check email existence |
|
|
|
|
### Agent Management (`/api/v1/agents`)
|
|
|
|
| Method | Endpoint | Description |
|
|
| ------ | --------------------------------------- | -------------------- |
|
|
| GET | `/user/{username}` | Get user's agents |
|
|
| POST | `/create` | Create new agent |
|
|
| GET | `/user/{username}/agent/{name}` | Get agent details |
|
|
| GET | `/search` | Search public agents |
|
|
| POST | `/star/owner/{username}/agent/{name}` | Star an agent |
|
|
| DELETE | `/unstar/owner/{username}/agent/{name}` | Unstar an agent |
|
|
|
|
### Repository Operations
|
|
|
|
| Method | Endpoint | Description |
|
|
| ------ | ----------------------------------------------- | ----------------- |
|
|
| GET | `/user/{username}/agent/{name}/contents/{path}` | Get file contents |
|
|
| GET | `/user/{username}/agent/{name}/raw/{path}` | Get raw file |
|
|
| GET | `/user/{username}/agent/{name}/branches` | List branches |
|
|
| GET | `/user/{username}/agent/{name}/commits` | Get commits |
|
|
| GET | `/user/{username}/agent/{name}/git/trees/{sha}` | Get git tree |
|
|
|
|
### Token Management (`/api/v1/tokens`)
|
|
|
|
| Method | Endpoint | Description |
|
|
| ------ | ---------------------------------- | ---------------- |
|
|
| POST | `/user/{user_id}` | Create API token |
|
|
| DELETE | `/user/{user_id}/token/{token_id}` | Delete token |
|
|
| GET | `/user/{user_id}` | Get user tokens |
|
|
|
|
## Database Models
|
|
|
|
### Core Models
|
|
|
|
**User Model**
|
|
|
|
```python
|
|
class User(Base):
|
|
userId: str # Primary key from Stytch
|
|
username: str # Unique username
|
|
email: str # User email
|
|
fullName: str # Display name
|
|
profilePictureUrl: str
|
|
bio: str
|
|
# Gitea integration fields
|
|
giteaUserId: int
|
|
giteaPassword: str # Encrypted
|
|
giteaToken: str # Encrypted
|
|
# Social links
|
|
githubUrl: str
|
|
linkedinUrl: str
|
|
xUrl: str
|
|
websiteUrl: str
|
|
```
|
|
|
|
**Agent Model**
|
|
|
|
```python
|
|
class Agent(Base):
|
|
agentId: str # Primary key
|
|
name: str # Agent name
|
|
description: str # Agent description
|
|
adminId: str # Owner user ID
|
|
configYaml: str # Agent configuration
|
|
readmeContent: str # README content
|
|
version: str # Agent version
|
|
visibility: str # public/private
|
|
created: datetime
|
|
updated: datetime
|
|
```
|
|
|
|
**Supporting Models**
|
|
|
|
- **Star**: User-Agent many-to-many relationship
|
|
- **Fork**: Fork tracking with original/forked agent IDs
|
|
- **Contributor**: Repository collaboration management
|
|
- **ImageKey**: S3 asset management
|
|
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
**Required Variables**
|
|
|
|
```bash
|
|
# Authentication
|
|
STYTCH_PROJECT_ID="your-project-id"
|
|
STYTCH_SECRET="your-secret-key"
|
|
STYTCH_PROJECT_DOMAIN="your-domain"
|
|
|
|
# Database
|
|
POSTGRES_CONNECTION_URL="postgresql://user:pass@host:port/db"
|
|
POSTGRES_DATABASE="modaic_db"
|
|
|
|
# Gitea
|
|
GITEA_URL="http://localhost:4000"
|
|
GITEA_ADMIN_TOKEN="admin-token"
|
|
GITEA_ADMIN_USERNAME="admin"
|
|
GITEA_ADMIN_PASSWORD="password"
|
|
|
|
# AWS S3
|
|
S3_BUCKET_NAME="your-bucket"
|
|
CLOUDFRONT_DOMAIN="your-cloudfront-domain"
|
|
|
|
# Email
|
|
GMAIL_APP_PASSWORD="app-password"
|
|
|
|
# Security
|
|
ENCRYPTION_KEY="base64-encoded-key"
|
|
GUEST_TOKEN="guest-access-token"
|
|
```
|
|
|
|
**Optional Variables**
|
|
|
|
```bash
|
|
# Development
|
|
DEBUG="true"
|
|
LOG_LEVEL="info"
|
|
CACHE_DIR="./cache"
|
|
MAX_FILE_SIZE="10485760"
|
|
|
|
# Performance
|
|
CORS_ORIGINS="http://localhost:3000"
|
|
```
|
|
|
|
### Database Configuration
|
|
|
|
The application uses SQLAlchemy with PostgreSQL:
|
|
|
|
```python
|
|
# Connection URL format
|
|
postgresql://username:password@host:port/database?sslmode=require
|
|
|
|
# Connection pooling (optional)
|
|
DB_POOL_SIZE=10
|
|
DB_MAX_OVERFLOW=20
|
|
DB_POOL_TIMEOUT=30
|
|
```
|
|
|
|
## Security Features
|
|
|
|
### Authentication & Authorization
|
|
|
|
**Multi-layer Authentication**
|
|
|
|
- Bearer token authentication (API access)
|
|
- Session cookie authentication (browser)
|
|
- Stytch JWT validation
|
|
|
|
**Permission Levels**
|
|
|
|
- Public endpoints (no auth required)
|
|
- Authenticated user endpoints
|
|
- Owner-only operations
|
|
- Admin-level operations
|
|
|
|
### Data Protection
|
|
|
|
**Encryption**
|
|
|
|
- Gitea tokens encrypted with Fernet
|
|
- Sensitive user data encrypted at rest
|
|
- Secure password hashing
|
|
|
|
**Input Validation**
|
|
|
|
- Comprehensive Pydantic schemas
|
|
- SQL injection prevention
|
|
- XSS protection through escaping
|
|
|
|
**Security Headers**
|
|
|
|
- CORS configuration
|
|
- Rate limiting (configurable)
|
|
- Secure cookie settings
|
|
|
|
## Testing
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# Install test dependencies
|
|
uv sync --group dev
|
|
|
|
# Run all tests
|
|
uv run pytest
|
|
|
|
# Run with coverage
|
|
uv run pytest --cov=src
|
|
|
|
# Run specific test file
|
|
uv run pytest tests/test_auth.py
|
|
```
|
|
|
|
|
|
## Development
|
|
|
|
### Code Quality
|
|
|
|
**Linting and Formatting**
|
|
|
|
```bash
|
|
# Format code with Black
|
|
uv run black src/
|
|
|
|
# Lint with Ruff
|
|
uv run ruff check src/
|
|
|
|
# Type checking (if using mypy)
|
|
uv run mypy src/
|
|
```
|
|
|
|
**Pre-commit Hooks**
|
|
The project uses Ruff for linting and Black for formatting:
|
|
|
|
```toml
|
|
[tool.ruff]
|
|
line-length = 88
|
|
target-version = "py313"
|
|
select = ["E", "W", "F", "I", "B", "C4", "UP", "PD"]
|
|
|
|
[tool.ruff.format]
|
|
quote-style = "double"
|
|
indent-style = "space"
|
|
```
|
|
|
|
### Database Operations
|
|
|
|
**Migrations** (if using Alembic)
|
|
|
|
```bash
|
|
# Generate migration
|
|
alembic revision --autogenerate -m "description"
|
|
|
|
# Apply migrations
|
|
alembic upgrade head
|
|
|
|
# Downgrade
|
|
alembic downgrade -1
|
|
```
|
|
|
|
**Database Reset** (development only)
|
|
|
|
```bash
|
|
# Drop and recreate tables
|
|
python -c "from src.db import recreate_tables; recreate_tables()"
|
|
```
|
|
|
|
## Deployment
|
|
|
|
### Production Setup
|
|
|
|
**Environment Configuration**
|
|
|
|
```bash
|
|
# Set production environment
|
|
ENVIRONMENT="production"
|
|
|
|
# Use production database
|
|
POSTGRES_CONNECTION_URL="postgresql://prod_user:secure_pass@prod_host:5432/prod_db?sslmode=require"
|
|
|
|
# Configure secure domains
|
|
CORS_ORIGINS="https://yourdomain.com,https://app.yourdomain.com"
|
|
```
|
|
|
|
**Docker Deployment**
|
|
|
|
```dockerfile
|
|
FROM python:3.13-slim
|
|
|
|
WORKDIR /app
|
|
COPY pyproject.toml uv.lock ./
|
|
RUN pip install uv && uv sync --frozen
|
|
|
|
COPY src/ ./src/
|
|
CMD ["uv", "run", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
|
```
|
|
|
|
### Performance Optimization
|
|
|
|
**Database Optimization**
|
|
|
|
- Connection pooling configuration
|
|
- Query optimization with proper indexes
|
|
- Database query monitoring
|
|
|
|
**Caching Strategy**
|
|
|
|
- Response caching for read-heavy endpoints
|
|
- Redis integration (optional)
|
|
- CDN integration for static assets
|
|
|
|
## Monitoring & Logging
|
|
|
|
### Logging Configuration
|
|
|
|
```python
|
|
# Structured logging
|
|
LOG_LEVEL="info" # debug, info, warning, error
|
|
|
|
# Log format includes:
|
|
# - Timestamp
|
|
# - Log level
|
|
# - Request ID
|
|
# - User ID (if authenticated)
|
|
# - Endpoint
|
|
# - Response time
|
|
```
|
|
|
|
### Health Checks
|
|
|
|
```bash
|
|
# Health check endpoint
|
|
GET /health
|
|
|
|
# Database connectivity check
|
|
GET /health/db
|
|
|
|
# External services check
|
|
GET /health/services
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
**Database Connection Errors**
|
|
|
|
```bash
|
|
# Check PostgreSQL status
|
|
pg_isready -h localhost -p 5432
|
|
|
|
# Test connection string
|
|
python -c "from src.db import engine; print(engine.connect())"
|
|
```
|
|
|
|
**Stytch Authentication Issues**
|
|
|
|
```bash
|
|
# Verify Stytch credentials
|
|
curl -X GET "https://test.stytch.com/v1/users" \
|
|
-H "Authorization: Bearer YOUR_SECRET_KEY"
|
|
```
|
|
|
|
**Gitea Integration Problems**
|
|
|
|
```bash
|
|
# Test Gitea API connectivity
|
|
curl -H "Authorization: token YOUR_GITEA_TOKEN" \
|
|
http://localhost:4000/api/v1/user
|
|
```
|
|
|
|
**File Upload Issues**
|
|
|
|
```bash
|
|
# Check S3 permissions
|
|
aws s3 ls s3://your-bucket-name
|
|
|
|
# Verify file size limits
|
|
MAX_FILE_SIZE=10485760 # 10MB
|
|
```
|
|
|
|
### Debug Mode
|
|
|
|
```bash
|
|
# Start with debug logging
|
|
DEBUG=true uv run uvicorn src.main:app --reload --log-level debug
|
|
|
|
# Enable SQL query logging
|
|
echo "SQLALCHEMY_ECHO=true" >> .env
|
|
```
|
|
|
|
## API Documentation
|
|
|
|
The FastAPI application automatically generates interactive API documentation:
|
|
|
|
- **Swagger UI**: http://localhost:8000/docs
|
|
- **ReDoc**: http://localhost:8000/redoc
|
|
- **OpenAPI JSON**: http://localhost:8000/openapi.json
|
|
|
|
### API Client Examples
|
|
|
|
**Python Client**
|
|
|
|
```python
|
|
import requests
|
|
|
|
# Authentication
|
|
response = requests.post("http://localhost:8000/api/v1/auth/login",
|
|
json={"email": "user@example.com", "password": "password"})
|
|
|
|
# Get user agents
|
|
response = requests.get("http://localhost:8000/api/v1/agents/user/username",
|
|
headers={"Authorization": f"Bearer {token}"})
|
|
```
|
|
|
|
**JavaScript Client**
|
|
|
|
```javascript
|
|
// Authentication
|
|
const response = await fetch('http://localhost:8000/api/v1/auth/login', {
|
|
method: 'POST',
|
|
headers: { 'Content-Type': 'application/json' },
|
|
body: JSON.stringify({ email: 'user@example.com', password: 'password' }),
|
|
});
|
|
|
|
// Get agents
|
|
const agents = await fetch(
|
|
'http://localhost:8000/api/v1/agents/user/username',
|
|
{
|
|
headers: { Authorization: `Bearer ${token}` },
|
|
}
|
|
);
|
|
```
|
|
|
|
---
|
|
|
|
**For more information, see the [main project README](../README.md)**
|