(no commit message)
This commit is contained in:
554
README.md
554
README.md
@@ -1,2 +1,554 @@
|
||||
# my-preferences
|
||||
# Modaic Server - FastAPI Backend
|
||||
|
||||
**AI Agent Platform Backend Service**
|
||||
|
||||
This is the FastAPI backend server for the Modaic platform, providing a comprehensive REST API for AI agent management, user authentication, Git repository operations, and collaborative features.
|
||||
|
||||
   
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **Python 3.13** or higher
|
||||
- **UV** package manager
|
||||
- **PostgreSQL** database
|
||||
- **Stytch** account for authentication
|
||||
- **AWS S3** bucket for file storage
|
||||
- **Gitea** server for Git operations
|
||||
|
||||
### Installation
|
||||
|
||||
1. **Navigate to server directory**
|
||||
|
||||
```bash
|
||||
cd server
|
||||
```
|
||||
|
||||
2. **Install dependencies**
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
```
|
||||
|
||||
3. **Set up environment variables**
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# Edit .env with your actual values
|
||||
```
|
||||
|
||||
4. **Start the development server**
|
||||
|
||||
```bash
|
||||
uv run uvicorn src.main:app --reload
|
||||
```
|
||||
|
||||
5. **Access the API**
|
||||
- **API Server**: http://localhost:8000
|
||||
- **Interactive Docs**: http://localhost:8000/docs
|
||||
- **ReDoc**: http://localhost:8000/redoc
|
||||
|
||||
## Architecture
|
||||
|
||||
### Application Structure
|
||||
|
||||
```
|
||||
src/
|
||||
api/ # API endpoints organized by feature
|
||||
v1/
|
||||
auth/ # Authentication endpoints
|
||||
agent/ # Agent management endpoints
|
||||
user/ # User management endpoints
|
||||
tokens/ # API token management
|
||||
core/ # Core configuration
|
||||
config.py # Application settings
|
||||
db/ # Database configuration
|
||||
index.py # Database connection
|
||||
pg.py # PostgreSQL setup
|
||||
lib/ # External service integrations
|
||||
gitea.py # Gitea API client
|
||||
s3.py # AWS S3 operations
|
||||
stytch.py # Stytch authentication
|
||||
logger.py # Logging configuration
|
||||
objects/ # Data models and schemas
|
||||
models/ # SQLAlchemy models
|
||||
schemas/ # Pydantic schemas
|
||||
service/ # Business logic services
|
||||
email.py # Email service
|
||||
utils/ # Utility functions
|
||||
date.py # Date utilities
|
||||
gitea.py # Gitea helpers
|
||||
user.py # User utilities
|
||||
main.py # FastAPI application entry point
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
||||
**FastAPI Application** (`src/main.py`)
|
||||
|
||||
- CORS configuration for frontend integration
|
||||
- API route registration
|
||||
- Middleware setup
|
||||
- Error handling
|
||||
|
||||
**Database Layer** (`src/db/`)
|
||||
|
||||
- PostgreSQL connection with SQLAlchemy
|
||||
- Async database operations
|
||||
- Connection pooling
|
||||
|
||||
**Authentication** (`src/lib/stytch.py`)
|
||||
|
||||
- Stytch integration for user management
|
||||
- JWT token validation
|
||||
- Session management
|
||||
|
||||
**Git Operations** (`src/lib/gitea.py`)
|
||||
|
||||
- Gitea API integration
|
||||
- Repository CRUD operations
|
||||
- File content management
|
||||
- Branch and commit operations
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Authentication (`/api/v1/auth`)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
| ------ | --------------- | ----------------------------- |
|
||||
| POST | `/register` | User registration with Stytch |
|
||||
| POST | `/login` | User authentication |
|
||||
| GET | `/authenticate` | OAuth callback handling |
|
||||
| GET | `/me` | Get current user information |
|
||||
| POST | `/onboarding` | Complete user onboarding |
|
||||
| DELETE | `/session` | User logout |
|
||||
|
||||
### User Management (`/api/v1/user`)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
| ------ | ---------------------- | --------------------- |
|
||||
| GET | `/{userId}` | Get user by ID |
|
||||
| GET | `/username/{username}` | Get user by username |
|
||||
| PUT | `/{userId}` | Update user profile |
|
||||
| DELETE | `/{userId}` | Delete user account |
|
||||
| GET | `/check/email/` | Check email existence |
|
||||
|
||||
### Agent Management (`/api/v1/agents`)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
| ------ | --------------------------------------- | -------------------- |
|
||||
| GET | `/user/{username}` | Get user's agents |
|
||||
| POST | `/create` | Create new agent |
|
||||
| GET | `/user/{username}/agent/{name}` | Get agent details |
|
||||
| GET | `/search` | Search public agents |
|
||||
| POST | `/star/owner/{username}/agent/{name}` | Star an agent |
|
||||
| DELETE | `/unstar/owner/{username}/agent/{name}` | Unstar an agent |
|
||||
|
||||
### Repository Operations
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
| ------ | ----------------------------------------------- | ----------------- |
|
||||
| GET | `/user/{username}/agent/{name}/contents/{path}` | Get file contents |
|
||||
| GET | `/user/{username}/agent/{name}/raw/{path}` | Get raw file |
|
||||
| GET | `/user/{username}/agent/{name}/branches` | List branches |
|
||||
| GET | `/user/{username}/agent/{name}/commits` | Get commits |
|
||||
| GET | `/user/{username}/agent/{name}/git/trees/{sha}` | Get git tree |
|
||||
|
||||
### Token Management (`/api/v1/tokens`)
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
| ------ | ---------------------------------- | ---------------- |
|
||||
| POST | `/user/{user_id}` | Create API token |
|
||||
| DELETE | `/user/{user_id}/token/{token_id}` | Delete token |
|
||||
| GET | `/user/{user_id}` | Get user tokens |
|
||||
|
||||
## Database Models
|
||||
|
||||
### Core Models
|
||||
|
||||
**User Model**
|
||||
|
||||
```python
|
||||
class User(Base):
|
||||
userId: str # Primary key from Stytch
|
||||
username: str # Unique username
|
||||
email: str # User email
|
||||
fullName: str # Display name
|
||||
profilePictureUrl: str
|
||||
bio: str
|
||||
# Gitea integration fields
|
||||
giteaUserId: int
|
||||
giteaPassword: str # Encrypted
|
||||
giteaToken: str # Encrypted
|
||||
# Social links
|
||||
githubUrl: str
|
||||
linkedinUrl: str
|
||||
xUrl: str
|
||||
websiteUrl: str
|
||||
```
|
||||
|
||||
**Agent Model**
|
||||
|
||||
```python
|
||||
class Agent(Base):
|
||||
agentId: str # Primary key
|
||||
name: str # Agent name
|
||||
description: str # Agent description
|
||||
adminId: str # Owner user ID
|
||||
configYaml: str # Agent configuration
|
||||
readmeContent: str # README content
|
||||
version: str # Agent version
|
||||
visibility: str # public/private
|
||||
created: datetime
|
||||
updated: datetime
|
||||
```
|
||||
|
||||
**Supporting Models**
|
||||
|
||||
- **Star**: User-Agent many-to-many relationship
|
||||
- **Fork**: Fork tracking with original/forked agent IDs
|
||||
- **Contributor**: Repository collaboration management
|
||||
- **ImageKey**: S3 asset management
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
**Required Variables**
|
||||
|
||||
```bash
|
||||
# Authentication
|
||||
STYTCH_PROJECT_ID="your-project-id"
|
||||
STYTCH_SECRET="your-secret-key"
|
||||
STYTCH_PROJECT_DOMAIN="your-domain"
|
||||
|
||||
# Database
|
||||
POSTGRES_CONNECTION_URL="postgresql://user:pass@host:port/db"
|
||||
POSTGRES_DATABASE="modaic_db"
|
||||
|
||||
# Gitea
|
||||
GITEA_URL="http://localhost:4000"
|
||||
GITEA_ADMIN_TOKEN="admin-token"
|
||||
GITEA_ADMIN_USERNAME="admin"
|
||||
GITEA_ADMIN_PASSWORD="password"
|
||||
|
||||
# AWS S3
|
||||
S3_BUCKET_NAME="your-bucket"
|
||||
CLOUDFRONT_DOMAIN="your-cloudfront-domain"
|
||||
|
||||
# Email
|
||||
GMAIL_APP_PASSWORD="app-password"
|
||||
|
||||
# Security
|
||||
ENCRYPTION_KEY="base64-encoded-key"
|
||||
GUEST_TOKEN="guest-access-token"
|
||||
```
|
||||
|
||||
**Optional Variables**
|
||||
|
||||
```bash
|
||||
# Development
|
||||
DEBUG="true"
|
||||
LOG_LEVEL="info"
|
||||
CACHE_DIR="./cache"
|
||||
MAX_FILE_SIZE="10485760"
|
||||
|
||||
# Performance
|
||||
CORS_ORIGINS="http://localhost:3000"
|
||||
```
|
||||
|
||||
### Database Configuration
|
||||
|
||||
The application uses SQLAlchemy with PostgreSQL:
|
||||
|
||||
```python
|
||||
# Connection URL format
|
||||
postgresql://username:password@host:port/database?sslmode=require
|
||||
|
||||
# Connection pooling (optional)
|
||||
DB_POOL_SIZE=10
|
||||
DB_MAX_OVERFLOW=20
|
||||
DB_POOL_TIMEOUT=30
|
||||
```
|
||||
|
||||
## Security Features
|
||||
|
||||
### Authentication & Authorization
|
||||
|
||||
**Multi-layer Authentication**
|
||||
|
||||
- Bearer token authentication (API access)
|
||||
- Session cookie authentication (browser)
|
||||
- Stytch JWT validation
|
||||
|
||||
**Permission Levels**
|
||||
|
||||
- Public endpoints (no auth required)
|
||||
- Authenticated user endpoints
|
||||
- Owner-only operations
|
||||
- Admin-level operations
|
||||
|
||||
### Data Protection
|
||||
|
||||
**Encryption**
|
||||
|
||||
- Gitea tokens encrypted with Fernet
|
||||
- Sensitive user data encrypted at rest
|
||||
- Secure password hashing
|
||||
|
||||
**Input Validation**
|
||||
|
||||
- Comprehensive Pydantic schemas
|
||||
- SQL injection prevention
|
||||
- XSS protection through escaping
|
||||
|
||||
**Security Headers**
|
||||
|
||||
- CORS configuration
|
||||
- Rate limiting (configurable)
|
||||
- Secure cookie settings
|
||||
|
||||
## Testing
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Install test dependencies
|
||||
uv sync --group dev
|
||||
|
||||
# Run all tests
|
||||
uv run pytest
|
||||
|
||||
# Run with coverage
|
||||
uv run pytest --cov=src
|
||||
|
||||
# Run specific test file
|
||||
uv run pytest tests/test_auth.py
|
||||
```
|
||||
|
||||
|
||||
## Development
|
||||
|
||||
### Code Quality
|
||||
|
||||
**Linting and Formatting**
|
||||
|
||||
```bash
|
||||
# Format code with Black
|
||||
uv run black src/
|
||||
|
||||
# Lint with Ruff
|
||||
uv run ruff check src/
|
||||
|
||||
# Type checking (if using mypy)
|
||||
uv run mypy src/
|
||||
```
|
||||
|
||||
**Pre-commit Hooks**
|
||||
The project uses Ruff for linting and Black for formatting:
|
||||
|
||||
```toml
|
||||
[tool.ruff]
|
||||
line-length = 88
|
||||
target-version = "py313"
|
||||
select = ["E", "W", "F", "I", "B", "C4", "UP", "PD"]
|
||||
|
||||
[tool.ruff.format]
|
||||
quote-style = "double"
|
||||
indent-style = "space"
|
||||
```
|
||||
|
||||
### Database Operations
|
||||
|
||||
**Migrations** (if using Alembic)
|
||||
|
||||
```bash
|
||||
# Generate migration
|
||||
alembic revision --autogenerate -m "description"
|
||||
|
||||
# Apply migrations
|
||||
alembic upgrade head
|
||||
|
||||
# Downgrade
|
||||
alembic downgrade -1
|
||||
```
|
||||
|
||||
**Database Reset** (development only)
|
||||
|
||||
```bash
|
||||
# Drop and recreate tables
|
||||
python -c "from src.db import recreate_tables; recreate_tables()"
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Production Setup
|
||||
|
||||
**Environment Configuration**
|
||||
|
||||
```bash
|
||||
# Set production environment
|
||||
ENVIRONMENT="production"
|
||||
|
||||
# Use production database
|
||||
POSTGRES_CONNECTION_URL="postgresql://prod_user:secure_pass@prod_host:5432/prod_db?sslmode=require"
|
||||
|
||||
# Configure secure domains
|
||||
CORS_ORIGINS="https://yourdomain.com,https://app.yourdomain.com"
|
||||
```
|
||||
|
||||
**Docker Deployment**
|
||||
|
||||
```dockerfile
|
||||
FROM python:3.13-slim
|
||||
|
||||
WORKDIR /app
|
||||
COPY pyproject.toml uv.lock ./
|
||||
RUN pip install uv && uv sync --frozen
|
||||
|
||||
COPY src/ ./src/
|
||||
CMD ["uv", "run", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
```
|
||||
|
||||
### Performance Optimization
|
||||
|
||||
**Database Optimization**
|
||||
|
||||
- Connection pooling configuration
|
||||
- Query optimization with proper indexes
|
||||
- Database query monitoring
|
||||
|
||||
**Caching Strategy**
|
||||
|
||||
- Response caching for read-heavy endpoints
|
||||
- Redis integration (optional)
|
||||
- CDN integration for static assets
|
||||
|
||||
## Monitoring & Logging
|
||||
|
||||
### Logging Configuration
|
||||
|
||||
```python
|
||||
# Structured logging
|
||||
LOG_LEVEL="info" # debug, info, warning, error
|
||||
|
||||
# Log format includes:
|
||||
# - Timestamp
|
||||
# - Log level
|
||||
# - Request ID
|
||||
# - User ID (if authenticated)
|
||||
# - Endpoint
|
||||
# - Response time
|
||||
```
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Health check endpoint
|
||||
GET /health
|
||||
|
||||
# Database connectivity check
|
||||
GET /health/db
|
||||
|
||||
# External services check
|
||||
GET /health/services
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Database Connection Errors**
|
||||
|
||||
```bash
|
||||
# Check PostgreSQL status
|
||||
pg_isready -h localhost -p 5432
|
||||
|
||||
# Test connection string
|
||||
python -c "from src.db import engine; print(engine.connect())"
|
||||
```
|
||||
|
||||
**Stytch Authentication Issues**
|
||||
|
||||
```bash
|
||||
# Verify Stytch credentials
|
||||
curl -X GET "https://test.stytch.com/v1/users" \
|
||||
-H "Authorization: Bearer YOUR_SECRET_KEY"
|
||||
```
|
||||
|
||||
**Gitea Integration Problems**
|
||||
|
||||
```bash
|
||||
# Test Gitea API connectivity
|
||||
curl -H "Authorization: token YOUR_GITEA_TOKEN" \
|
||||
http://localhost:4000/api/v1/user
|
||||
```
|
||||
|
||||
**File Upload Issues**
|
||||
|
||||
```bash
|
||||
# Check S3 permissions
|
||||
aws s3 ls s3://your-bucket-name
|
||||
|
||||
# Verify file size limits
|
||||
MAX_FILE_SIZE=10485760 # 10MB
|
||||
```
|
||||
|
||||
### Debug Mode
|
||||
|
||||
```bash
|
||||
# Start with debug logging
|
||||
DEBUG=true uv run uvicorn src.main:app --reload --log-level debug
|
||||
|
||||
# Enable SQL query logging
|
||||
echo "SQLALCHEMY_ECHO=true" >> .env
|
||||
```
|
||||
|
||||
## API Documentation
|
||||
|
||||
The FastAPI application automatically generates interactive API documentation:
|
||||
|
||||
- **Swagger UI**: http://localhost:8000/docs
|
||||
- **ReDoc**: http://localhost:8000/redoc
|
||||
- **OpenAPI JSON**: http://localhost:8000/openapi.json
|
||||
|
||||
### API Client Examples
|
||||
|
||||
**Python Client**
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
# Authentication
|
||||
response = requests.post("http://localhost:8000/api/v1/auth/login",
|
||||
json={"email": "user@example.com", "password": "password"})
|
||||
|
||||
# Get user agents
|
||||
response = requests.get("http://localhost:8000/api/v1/agents/user/username",
|
||||
headers={"Authorization": f"Bearer {token}"})
|
||||
```
|
||||
|
||||
**JavaScript Client**
|
||||
|
||||
```javascript
|
||||
// Authentication
|
||||
const response = await fetch('http://localhost:8000/api/v1/auth/login', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ email: 'user@example.com', password: 'password' }),
|
||||
});
|
||||
|
||||
// Get agents
|
||||
const agents = await fetch(
|
||||
'http://localhost:8000/api/v1/agents/user/username',
|
||||
{
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**For more information, see the [main project README](../README.md)**
|
||||
|
||||
Reference in New Issue
Block a user