jmp-stack/PRODUCTION_DEPLOYMENT.md

# Production Deployment Guide

This document outlines the changes made to prepare JMP Server for production deployment on a server.

## Key Changes Summary

### 1. **Removed Direct Port Exposures**
- Removed direct port mappings for:
  - `homepage:3001` (was exposing internal port)
  - `bookstack:6875` (was exposing internal port)
  - `actual_server:5006` (was exposing internal port)
- All services now route exclusively through Traefik reverse proxy
- Only SSH port `2222` (Gitea) exposed directly for Git operations

### 2. **Resource Limits & Reservations**
All containers now have CPU and memory limits:
- **Traefik**: 2 CPU / 512M limit, 0.5 CPU / 256M reserved
- **Gitea**: 2 CPU / 1G limit, 1 CPU / 512M reserved
- **Gitea Runner**: 4 CPU / 2G limit (for CI/CD workloads)
- **Bookstack**: 2 CPU / 512M limit, 1 CPU / 256M reserved
- **Databases**: 2 CPU / 1G limit, 1 CPU / 512M reserved
- **OpenCloud**: 4 CPU / 4G limit, 2 CPU / 2G reserved

Adjust these based on your server's available resources.

### 3. **Health Checks**
All services now include health checks:
- **Databases**: `pg_isready` or `mysqladmin ping` checks
- **Web services**: HTTP endpoint checks
- Service dependencies now use `condition: service_healthy` instead of just `depends_on`

This ensures proper startup ordering and service reliability.

### 4. **Log Rotation**
Configured JSON file logging with rotation:
- Max file size: 50-100MB per log file
- Max files retained: 3-5 files
- Prevents logs from filling disk

### 5. **Traefik Authentication**
- **Before**: Had invalid placeholder `$$apr1$$...`
- **After**: Uses environment variable `${TRAEFIK_BASICAUTH_USERS}`

**Generate credentials:**
```bash
openssl passwd -apr1 admin
# Output: $apr1$r61.qW2G$Lc3uT7c5p...
```

Set in `.env`:
```
TRAEFIK_BASICAUTH_USERS=admin:$apr1$r61.qW2G$Lc3uT7c5p...
```

### 6. **Database Restart Policies**
- **Databases**: Changed to `restart: always` (critical services)
- **Applications**: Keep `restart: unless-stopped` (graceful restart handling)

### 7. **OpenCloud Network Configuration**
Fixed network issues for OpenCloud:
- Explicitly declared `proxy_net` as external in opencloud-compose
- Added network driver specification
- Prevents network conflicts when deploying separately

### 8. **Container Dependencies**
Updated all service dependencies to use health conditions:
```yaml
depends_on:
  gitea-db:
    condition: service_healthy
```

This ensures services wait for databases to be healthy before starting.

### 9. **Actual Server Configuration**
- Removed direct port exposure `5006:5006`
- Fixed Traefik loadbalancer port (was `5232`, now `5006`)
- Added to `proxy_net` network
- Added health check

## Environment Variables

### Required Changes for Production

Copy `.env.production.example` to `.env` and update:

1. **DOMAIN**: Your actual domain name
2. **TRAEFIK_BASICAUTH_USERS**: Generated htpasswd hash
3. **All database passwords**: Strong, random passwords
4. **API keys & tokens**: Generate new secure values
5. **OpenCloud settings**: Domain and admin password

### Sensitive Variables

These should **never** be committed to git:
- All `*PASSWORD*` variables
- `*TOKEN*` variables
- `*SECRET*` variables
- `APP_KEY` values

Keep `.env` in `.gitignore` and use `.env.production.example` as template.

## Pre-Deployment Checklist

- [ ] Copy `.env.production.example` to `.env`
- [ ] Generate Traefik basicauth credentials with `openssl passwd -apr1`
- [ ] Set all database passwords (random, 16+ characters)
- [ ] Generate API tokens (Gitea runner, Vaultwarden admin, etc.)
- [ ] Configure domain in `.env` (DOMAIN variable)
- [ ] Review resource limits for your server capacity
- [ ] Ensure backup storage path is accessible (`./backups/`)
- [ ] Verify SSL/TLS certificates will work (Let's Encrypt requirements)

## Deployment Steps

1. **Prepare server**:
   ```bash
   mkdir -p ~/jmp-server
   cd ~/jmp-server
   # Copy all files here
   ```

2. **Configure environment**:
   ```bash
   cp .env.production.example .env
   # Edit .env with your values
   ```

3. **Start services**:
   ```bash
   ./start.sh
   # or: docker compose up -d
   ```

4. **Verify health**:
   ```bash
   docker compose ps
   # Check all services are "Up" and health checks pass
   
   docker compose logs -f traefik
   # Watch for ACME certificate provisioning
   ```

5. **Test routing**:
   ```bash
   curl https://traefik.${DOMAIN}/
   curl https://gitea.${DOMAIN}/
   curl https://bookstack.${DOMAIN}/
   ```

## Monitoring & Maintenance

### View Service Status
```bash
docker compose ps
docker compose stats
```

### Check Health
```bash
docker compose exec gitea-db pg_isready -h localhost -U ${POSTGRES_USER_GITEA}
docker compose exec bookstack-db mysqladmin ping -h localhost -u root -p${MYSQL_PASSWORD_BOOKSTACK}
```

### View Logs
```bash
# Traefik routing
docker compose logs -f traefik

# Gitea
docker compose logs -f gitea

# Backup status
docker compose logs -f backup
```

### Backup Management
Backups are automated (see `backup.env`). Stored in `./backups/`.

## Security Considerations

1. **Firewall**: Only expose ports 80, 443, and 2222 (SSH) to internet
2. **HTTPS**: Let's Encrypt certificates auto-renewed by Traefik
3. **Database access**: Internal networks prevent direct DB access
4. **Secrets**: Never store passwords in compose files; use `.env` only
5. **Log retention**: Configure log rotation in Docker daemon or use central logging
6. **Backups**: Store backups on separate storage/cloud

## Troubleshooting

### Service won't start
```bash
docker compose logs -f [service_name]
# Check: Missing dependencies, failed health checks, port conflicts
```

### Services can't communicate
```bash
docker network ls
docker network inspect proxy_net
# Verify all services are on correct networks
```

### Traefik routing issues
```bash
docker compose exec traefik traefik api dashboard
# Check router/service configurations
```

### Memory/CPU pressure
```bash
docker compose stats
# Review resource limits in docker-compose.yaml
# Adjust based on actual usage
```

## Resource Recommendations

For a typical small server (4GB RAM, 2 CPU):
- **Gitea + Runner**: 3G RAM, 2.5 CPU
- **OpenCloud**: 2G RAM, 2 CPU
- **Other services**: 1G RAM, 1.5 CPU
- **Total**: ~6G RAM recommended, 6 CPU recommended

Adjust limits down if server is smaller, test under load for your workload.

## Next Steps

1. Configure OpenCloud separately if needed: `docker compose -f opencloud-compose/docker-compose.yml up -d`
2. Set up external backup storage for production data protection
3. Configure monitoring/alerting (optional external monitoring)
4. Document your domain names and admin credentials securely