chore: bootstrap Atay Makhzan ops repo

2026-06-19 15:35:48 +01:00
commit f6ba9ab02d
14 changed files with 639 additions and 0 deletions
@@ -0,0 +1,22 @@
+# Copy to .env on the VPS and fill real values there.
+# Never commit the real .env file.
+
+GITEA_VERSION=1.26.2
+POSTGRES_VERSION=16-alpine
+
+GITEA_DOMAIN=ataymakhzan.com
+GITEA_SSH_DOMAIN=ataymakhzan.com
+GITEA_ROOT_URL=https://ataymakhzan.com/
+GITEA_HTTP_PORT=3001
+GITEA_HTTP_BIND=127.0.0.1
+GITEA_SSH_PORT=2222
+
+GITEA_USER_UID=1000
+GITEA_USER_GID=1000
+
+POSTGRES_DB=gitea
+POSTGRES_USER=gitea
+POSTGRES_PASSWORD=change-me-before-deploy
+
+GITEA_CONTAINER_NAME=gitea
+POSTGRES_CONTAINER_NAME=gitea-db
@@ -0,0 +1,34 @@
+# Secrets
+.env
+.env.*
+!.env.example
+app.ini
+*.pem
+*.key
+*.crt
+id_rsa*
+id_ed25519*
+*_token*
+*secret*
+
+# Backups and dumps
+backups/
+*.dump
+*.sql
+*.sqlite
+*.zip
+*.tar
+*.tar.gz
+*.tgz
+
+# Runtime data
+gitea-data/
+postgres-data/
+log/
+logs/
+
+# Local/editor
+.DS_Store
+.idea/
+.vscode/
+*.swp
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Saad ibn Zoubayr
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
@@ -0,0 +1,82 @@
+# Atay Makhzan Ops
+
+Open-source operations repository for **Atay Makhzan**, Saad ibn Zoubayr's self-hosted Gitea forge.
+
+Atay Makhzan is currently a sovereign Git forge running Gitea on a VPS with Docker Compose, PostgreSQL, Nginx, and SSH Git access.
+
+## Current production snapshot
+
+| Area | Current value |
+|---|---|
+| Public domain | `ataymakhzan.com` |
+| Forge | Gitea |
+| Gitea image | `gitea/gitea:1.26.2` |
+| Database | PostgreSQL via `postgres:16-alpine` |
+| Stack path | `/opt/gitea` |
+| Web proxy | Nginx + Certbot TLS |
+| Local Gitea HTTP | `127.0.0.1:3001` / container port `3001` |
+| Git SSH | `ataymakhzan.com:2222` |
+
+## What belongs in this repo
+
+- Sanitized Docker Compose templates
+- Nginx reverse-proxy templates
+- Backup, verification, and upgrade scripts
+- Restore and maintenance runbooks
+- Architecture decision records
+- Public roadmap for future Atay Makhzan evolution
+
+## What must never be committed
+
+- `.env` with real secrets
+- Gitea `app.ini` with secrets
+- PostgreSQL passwords
+- SSH private keys
+- Gitea dumps or database dumps
+- Repository backups
+- API tokens or access tokens
+- TLS private keys
+
+See [`SECURITY.md`](SECURITY.md).
+
+## Quick commands
+
+Verify a live instance:
+
+```bash
+DOMAIN=ataymakhzan.com \
+SSH_PORT=2222 \
+OWNER=ibnezzoubayr \
+PROBE_REPO=Empire-OS \
+./scripts/verify-gitea.sh
+```
+
+Create a rollback backup on the VPS:
+
+```bash
+sudo STACK_DIR=/opt/gitea ./scripts/backup-gitea.sh
+```
+
+Prepare an upgrade dry-run:
+
+```bash
+sudo TARGET_VERSION=1.26.2 STACK_DIR=/opt/gitea ./scripts/upgrade-gitea.sh
+```
+
+Apply an upgrade intentionally:
+
+```bash
+sudo TARGET_VERSION=1.26.2 STACK_DIR=/opt/gitea APPLY=1 ./scripts/upgrade-gitea.sh
+```
+
+## Strategic direction
+
+This repo starts as **ops/infrastructure** for the official Gitea-based Atay Makhzan deployment.
+
+Later, if Atay Makhzan needs product behavior that Gitea cannot cleanly support through configuration, themes, plugins, or external automation, we can create a separate source fork and maintain it as its own product.
+
+Until then, the CTO rule is:
+
+> Do not fork Gitea prematurely. First make the deployment reproducible, observable, backed up, and safe to upgrade.
+
+See [`docs/FUTURE-GITEA-FORK.md`](docs/FUTURE-GITEA-FORK.md).
@@ -0,0 +1,37 @@
+# Security Policy
+
+This repository is public. Treat it as documentation and reproducible operations code, not as a secret store.
+
+## Never commit
+
+- Real `.env` files
+- Gitea `app.ini` secrets
+- Database passwords
+- SMTP credentials
+- OAuth secrets
+- SSH private keys
+- API tokens
+- Gitea dump archives
+- PostgreSQL dump files
+- TLS private keys
+- Full production logs containing sensitive data
+
+## Safe to commit
+
+- `.env.example` with placeholders
+- Docker Compose templates with variable references
+- Nginx templates without private keys
+- Runbooks
+- Scripts that read secrets from the environment
+- Architecture records
+
+## If a secret is committed
+
+1. Rotate the secret immediately.
+2. Remove it from current files.
+3. Rewrite public Git history only if the exposure is severe and worth the operational risk.
+4. Assume the secret was copied by someone else.
+
+## Operational rule
+
+For production maintenance, create backups before upgrades and verify both web and SSH Git paths after changes.
@@ -0,0 +1,38 @@
+services:
+  server:
+    image: gitea/gitea:${GITEA_VERSION:-1.26.2}
+    container_name: ${GITEA_CONTAINER_NAME:-gitea}
+    restart: unless-stopped
+    environment:
+      - USER_UID=${GITEA_USER_UID:-1000}
+      - USER_GID=${GITEA_USER_GID:-1000}
+      - GITEA__database__DB_TYPE=postgres
+      - GITEA__database__HOST=db:5432
+      - GITEA__database__NAME=${POSTGRES_DB:-gitea}
+      - GITEA__database__USER=${POSTGRES_USER:-gitea}
+      - GITEA__database__PASSWD=${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}
+      - GITEA__server__DOMAIN=${GITEA_DOMAIN:-ataymakhzan.com}
+      - GITEA__server__SSH_DOMAIN=${GITEA_SSH_DOMAIN:-ataymakhzan.com}
+      - GITEA__server__ROOT_URL=${GITEA_ROOT_URL:-https://ataymakhzan.com/}
+      - GITEA__server__HTTP_PORT=${GITEA_HTTP_PORT:-3001}
+      - GITEA__server__SSH_PORT=${GITEA_SSH_PORT:-2222}
+    volumes:
+      - ./gitea-data:/data
+      - /etc/timezone:/etc/timezone:ro
+      - /etc/localtime:/etc/localtime:ro
+    ports:
+      - "${GITEA_HTTP_BIND:-127.0.0.1}:${GITEA_HTTP_PORT:-3001}:${GITEA_HTTP_PORT:-3001}"
+      - "${GITEA_SSH_PORT:-2222}:22"
+    depends_on:
+      - db
+
+  db:
+    image: postgres:${POSTGRES_VERSION:-16-alpine}
+    container_name: ${POSTGRES_CONTAINER_NAME:-gitea-db}
+    restart: unless-stopped
+    environment:
+      - POSTGRES_DB=${POSTGRES_DB:-gitea}
+      - POSTGRES_USER=${POSTGRES_USER:-gitea}
+      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}
+    volumes:
+      - ./postgres-data:/var/lib/postgresql/data
@@ -0,0 +1,49 @@
+# ADR-0001: Current Gitea Docker Compose Architecture
+
+## Status
+
+Accepted.
+
+## Context
+
+Atay Makhzan needs a sovereign Git forge controlled by Saad ibn Zoubayr. The current operational need is reliability, simple maintenance, backups, and controlled upgrades.
+
+## Decision
+
+Run official Gitea in Docker Compose with PostgreSQL, reverse-proxied by Nginx with Certbot TLS.
+
+Current production shape:
+
+- Gitea image pinned to `gitea/gitea:1.26.2`
+- PostgreSQL image `postgres:16-alpine`
+- Gitea HTTP served locally on port `3001`
+- Public HTTPS via Nginx on `ataymakhzan.com`
+- Git SSH exposed on port `2222`
+- Persistent data mounted under `/opt/gitea`
+
+## Consequences
+
+### Positive
+
+- Simple architecture
+- Easy backups
+- Easy rollback through Docker image pinning and database dumps
+- Low operational burden
+- Enough for current private forge needs
+
+### Negative
+
+- Single VPS is a single point of failure
+- Scaling and HA are manual future work
+- Public customization is limited unless we theme, extend, or fork
+- Production safety depends on disciplined backups and upgrade procedure
+
+## Future trigger for revisiting
+
+Revisit this decision if Atay Makhzan needs:
+
+- Multi-node availability
+- Custom product features inside the forge
+- Organization-wide policy automation not available in Gitea
+- Deep UI/UX changes that themes cannot support
+- Integrated CI/package registry workflows beyond Gitea's native capabilities
@@ -0,0 +1,60 @@
+# Future Gitea Fork Strategy
+
+Saad's long-term intent is valid: Atay Makhzan may eventually differ enough from upstream Gitea that we maintain our own source code separately.
+
+But the sequencing matters.
+
+## CTO verdict
+
+Do not fork Gitea just because we can.
+
+Fork only when one of these is true:
+
+1. The required feature cannot be achieved with configuration.
+2. It cannot be achieved cleanly with Gitea Actions, webhooks, external automation, or themes.
+3. The feature is core to Atay Makhzan's identity as a product.
+4. Maintaining the patch is cheaper than working around upstream.
+5. We have time to track upstream security releases and merge conflicts.
+
+## Recommended stages
+
+### Stage 1 — Ops sovereignty
+
+This repository.
+
+Goal: make deployment reproducible, documented, backed up, and safe to upgrade.
+
+### Stage 2 — External extensions
+
+Use webhooks, scripts, bots, custom templates, and integrations around Gitea without touching upstream source.
+
+### Stage 3 — Theming and UX identity
+
+Customize branding, templates, and public surfaces while remaining close to upstream.
+
+### Stage 4 — Source fork
+
+Create a dedicated source repository only when Atay Makhzan needs product behavior that upstream Gitea will not support.
+
+Suggested future repo name:
+
+```text
+ibnezzoubayr/Atay-Makhzan
+```
+
+This ops repo should remain separate:
+
+```text
+ibnezzoubayr/Atay-Makhzan-Ops
+```
+
+## Fork discipline
+
+If we fork:
+
+- Track upstream Gitea releases.
+- Keep a changelog of every divergence.
+- Keep custom patches small and isolated.
+- Add tests for every custom behavior.
+- Maintain an upstream merge schedule.
+- Never delay critical upstream security patches for cosmetic customization.
@@ -0,0 +1,32 @@
+# Restore Notes
+
+Restoring Atay Makhzan can destroy or overwrite live data. Do not run restore actions without explicit owner approval.
+
+## Backup artifacts expected
+
+A complete backup directory should contain:
+
+- `gitea-dump-<timestamp>.zip`
+- `gitea-postgres-<timestamp>.dump`
+- `docker-compose.yml`
+- `app.ini`
+- `metadata.txt`
+- `SHA256SUMS`
+
+## Safer restore principle
+
+Prefer restoring to a fresh VPS or staging directory first, then verifying repository data before touching production.
+
+## High-level restore sequence
+
+1. Provision VPS and install Docker/Nginx.
+2. Restore sanitized Compose configuration.
+3. Restore PostgreSQL dump into a fresh database.
+4. Restore Gitea data from Gitea dump according to the Gitea version documentation.
+5. Start containers.
+6. Verify API, web, SSH auth, and `git ls-remote` on known repositories.
+7. Switch DNS/proxy only after verification.
+
+## Production warning
+
+Do not remove `/opt/gitea/gitea-data` or `/opt/gitea/postgres-data` unless a verified backup exists and Saad explicitly approved the restore.
@@ -0,0 +1,71 @@
+# Atay Makhzan Runbook
+
+## Deployment shape
+
+Atay Makhzan currently runs as a Docker Compose stack on a VPS:
+
+- Stack directory: `/opt/gitea`
+- Gitea container: `gitea`
+- PostgreSQL container: `gitea-db`
+- Public HTTPS: `https://ataymakhzan.com`
+- Local Gitea HTTP: `http://127.0.0.1:3001`
+- Git SSH: `ssh://git@ataymakhzan.com:2222/<owner>/<repo>.git`
+
+## Normal health check
+
+From the VPS:
+
+```bash
+cd /opt/gitea
+docker compose ps
+curl -fsS http://127.0.0.1:3001/api/v1/version
+docker exec -u git gitea gitea doctor check -c /data/gitea/conf/app.ini -w /data/gitea
+```
+
+From outside:
+
+```bash
+curl -fsS https://ataymakhzan.com/api/v1/version
+ssh -p 2222 -o BatchMode=yes -T git@ataymakhzan.com
+git ls-remote --heads ssh://git@ataymakhzan.com:2222/ibnezzoubayr/Empire-OS.git
+```
+
+## Backup before maintenance
+
+```bash
+sudo STACK_DIR=/opt/gitea ./scripts/backup-gitea.sh
+```
+
+A proper backup should include:
+
+- Gitea built-in dump
+- PostgreSQL `pg_dump -Fc`
+- `docker-compose.yml`
+- `app.ini`
+- metadata and checksums
+
+## Upgrade policy
+
+1. Inspect current state.
+2. Create backup.
+3. Pull target image.
+4. Pin explicit Gitea version in Compose.
+5. Recreate only the Gitea service.
+6. Verify web, API, SSH, `git ls-remote`, and doctor check.
+
+Do not run production on `gitea/gitea:latest`.
+
+## Rollback policy
+
+Rollback can involve code image rollback, config rollback, or database restore.
+
+- Re-tagged Docker images are low-risk.
+- Restoring database dumps is destructive and requires explicit owner approval.
+- Never delete volumes during an emergency unless a verified backup exists.
+
+## Routine cleanup candidates
+
+- Remove obsolete Compose `version:` key from the live stack.
+- Move deprecated Gitea `[picture]` options out of `app.ini` if still present.
+- Add backup retention and offsite backup storage.
+- Add uptime/health monitoring.
@@ -0,0 +1,38 @@
+# Nginx reverse proxy template for Atay Makhzan / Gitea.
+# Replace DOMAIN and local port values if they differ.
+
+server {
+    listen 80;
+    server_name ataymakhzan.com www.ataymakhzan.com;
+
+    location /.well-known/acme-challenge/ {
+        root /var/www/html;
+    }
+
+    location / {
+        return 301 https://$host$request_uri;
+    }
+}
+
+server {
+    listen 443 ssl http2;
+    server_name ataymakhzan.com www.ataymakhzan.com;
+
+    ssl_certificate /etc/letsencrypt/live/ataymakhzan.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/ataymakhzan.com/privkey.pem;
+
+    client_max_body_size 512M;
+
+    location / {
+        proxy_pass http://127.0.0.1:3001;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_set_header X-Forwarded-Host $host;
+        proxy_set_header X-Forwarded-Port 443;
+
+        proxy_read_timeout 300s;
+        proxy_send_timeout 300s;
+    }
+}
@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+STACK_DIR="${STACK_DIR:-/opt/gitea}"
+GITEA_CONTAINER="${GITEA_CONTAINER:-gitea}"
+DB_CONTAINER="${DB_CONTAINER:-gitea-db}"
+POSTGRES_USER="${POSTGRES_USER:-gitea}"
+POSTGRES_DB="${POSTGRES_DB:-gitea}"
+
+cd "$STACK_DIR"
+TS="$(date -u +%Y%m%d-%H%M%S)"
+BK="$STACK_DIR/backups/$TS"
+mkdir -p "$BK"
+
+cp docker-compose.yml "$BK/docker-compose.yml"
+if [ -f "$STACK_DIR/gitea-data/gitea/conf/app.ini" ]; then
+  cp "$STACK_DIR/gitea-data/gitea/conf/app.ini" "$BK/app.ini"
+fi
+
+{
+  echo "backup_utc=$TS"
+  echo "host=$(hostname)"
+  echo "date=$(date -u --iso-8601=seconds)"
+  echo "docker=$(docker --version)"
+  echo "compose=$(docker compose version 2>/dev/null || true)"
+  echo "gitea_version=$(docker exec -u git "$GITEA_CONTAINER" gitea --version 2>/dev/null || true)"
+  docker ps --format '{{.Names}} | {{.Image}} | {{.Status}} | {{.Ports}}'
+} > "$BK/metadata.txt"
+
+docker exec -u git "$GITEA_CONTAINER" mkdir -p /data/gitea/backup-tmp
+DUMP_NAME="gitea-dump-$TS.zip"
+docker exec -u git "$GITEA_CONTAINER" gitea dump \
+  -c /data/gitea/conf/app.ini \
+  -w /data/gitea \
+  -f "/data/gitea/backup-tmp/$DUMP_NAME" \
+  --quiet
+
+cp "$STACK_DIR/gitea-data/gitea/backup-tmp/$DUMP_NAME" "$BK/$DUMP_NAME"
+rm -f "$STACK_DIR/gitea-data/gitea/backup-tmp/$DUMP_NAME"
+
+docker exec "$DB_CONTAINER" pg_dump -U "$POSTGRES_USER" -d "$POSTGRES_DB" -Fc > "$BK/gitea-postgres-$TS.dump"
+sha256sum "$BK"/* > "$BK/SHA256SUMS"
+
+echo "Backup created: $BK"
+du -sh "$BK" "$BK"/*
@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+STACK_DIR="${STACK_DIR:-/opt/gitea}"
+TARGET_VERSION="${TARGET_VERSION:?set TARGET_VERSION, e.g. TARGET_VERSION=1.26.2}"
+GITEA_CONTAINER="${GITEA_CONTAINER:-gitea}"
+APPLY="${APPLY:-0}"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+cd "$STACK_DIR"
+
+if [ ! -f docker-compose.yml ]; then
+  echo "docker-compose.yml not found in $STACK_DIR" >&2
+  exit 1
+fi
+
+echo "== Current version =="
+docker exec -u git "$GITEA_CONTAINER" gitea --version || true
+
+echo "== Creating backup first =="
+"$SCRIPT_DIR/backup-gitea.sh"
+
+echo "== Pulling target image =="
+docker pull "gitea/gitea:$TARGET_VERSION"
+docker run --rm --user git --entrypoint /usr/local/bin/gitea "gitea/gitea:$TARGET_VERSION" --version || true
+
+OLD_IMAGE_ID="$(docker image inspect "$(docker inspect --format '{{.Config.Image}}' "$GITEA_CONTAINER")" --format '{{.Id}}' 2>/dev/null || true)"
+if [ -n "$OLD_IMAGE_ID" ]; then
+  ROLLBACK_TAG="gitea/gitea:rollback-$(date -u +%Y%m%d-%H%M%S)"
+  docker tag "$OLD_IMAGE_ID" "$ROLLBACK_TAG"
+  echo "Rollback image tag: $ROLLBACK_TAG"
+fi
+
+echo "== Pinning docker-compose.yml to target version =="
+python3 - <<PY
+from pathlib import Path
+import re
+p = Path('$STACK_DIR/docker-compose.yml')
+s = p.read_text()
+s2 = re.sub(r'image:\s*gitea/gitea:[^\s]+', 'image: gitea/gitea:$TARGET_VERSION', s, count=1)
+if s2 == s:
+    raise SystemExit('Could not find gitea/gitea image line to replace')
+p.write_text(s2)
+PY
+
+docker compose config >/dev/null
+
+if [ "$APPLY" != "1" ]; then
+  echo "Dry-run complete. Compose file was pinned, but service was not recreated."
+  echo "Review the diff, then run with APPLY=1 to recreate the Gitea service."
+  exit 0
+fi
+
+echo "== Recreating Gitea service only =="
+docker compose up -d server
+
+echo "== Waiting for readiness =="
+for _ in $(seq 1 90); do
+  if curl -fsS "http://127.0.0.1:3001/api/v1/version" | grep -q "$TARGET_VERSION"; then
+    echo
+    echo "Gitea $TARGET_VERSION is ready."
+    exit 0
+  fi
+  sleep 2
+done
+
+echo "Gitea did not report target version in time" >&2
+exit 1
@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+DOMAIN="${DOMAIN:-ataymakhzan.com}"
+SSH_PORT="${SSH_PORT:-2222}"
+OWNER="${OWNER:-ibnezzoubayr}"
+PROBE_REPO="${PROBE_REPO:-Empire-OS}"
+LOCAL_URL="${LOCAL_URL:-http://127.0.0.1:3001}"
+GITEA_CONTAINER="${GITEA_CONTAINER:-gitea}"
+RUN_DOCKER_CHECKS="${RUN_DOCKER_CHECKS:-1}"
+
+echo "== Local API version =="
+curl -fsS "$LOCAL_URL/api/v1/version"
+echo
+
+echo "== External API version =="
+curl -fsS "https://$DOMAIN/api/v1/version"
+echo
+
+echo "== External homepage status =="
+curl -fsS -I -L --max-time 20 "https://$DOMAIN/" | sed -n '1,12p'
+
+echo "== Git SSH authentication =="
+SSH_OUT=$(ssh -p "$SSH_PORT" -o BatchMode=yes -o ConnectTimeout=10 -T "git@$DOMAIN" 2>&1 || true)
+echo "$SSH_OUT"
+echo "$SSH_OUT" | grep -Eiq 'successfully authenticated|Hi .*!|Welcome' || {
+  echo "Could not confirm successful SSH authentication from output" >&2
+  exit 1
+}
+
+echo "== Git ls-remote probe =="
+git ls-remote --heads "ssh://git@$DOMAIN:$SSH_PORT/$OWNER/$PROBE_REPO.git" >/dev/null
+
+echo "== Optional Docker/native checks =="
+if [ "$RUN_DOCKER_CHECKS" = "1" ] && command -v docker >/dev/null 2>&1; then
+  docker exec -u git "$GITEA_CONTAINER" gitea --version
+  docker exec -u git "$GITEA_CONTAINER" gitea doctor check -c /data/gitea/conf/app.ini -w /data/gitea
+else
+  echo "Skipping Docker checks. Set RUN_DOCKER_CHECKS=1 on the VPS to enable."
+fi
+
+echo "Atay Makhzan verification passed."