diff --git a/README.md b/README.md index d54c9f6..f9320d7 100644 --- a/README.md +++ b/README.md @@ -135,11 +135,78 @@ Use `.env.example` as your baseline env configuration. ## Storage Backend Notes -- This project defaults to **SeaweedFS S3 API** for object transit in development and compose deployments. -- The Python server uses the `minio` Python SDK, which is intentional because SeaweedFS is S3-compatible. +- Local development defaults to **SeaweedFS** (S3-compatible) via Docker Compose. +- Production can use any S3-compatible provider; **AWS S3** is the expected choice. +- The Python server uses the `minio` Python SDK against the S3 API. - Runtime configuration uses `S3_*` environment variables. - All conversions share one bucket (`S3_BUCKET`, required). Each conversion's objects live under a `{conversion_id}/` key prefix (for example `{conversion_id}/input/source.pptx` and `{conversion_id}/output/slide-0001.jpg`). +### AWS setup + +**Bucket** + +1. Create one bucket (for example `officeconvert-prod`) in the region where the server runs. +2. Leave **Block Public Access** enabled. Presigned URLs work without a public bucket. +3. Optional: add a lifecycle rule to expire objects after a few days as a safety net if cleanup fails. + +**Server environment** + +Set at minimum: + +```bash +S3_BUCKET=officeconvert-prod +S3_ENDPOINT=s3.us-east-1.amazonaws.com +S3_PUBLIC_ENDPOINT=s3.us-east-1.amazonaws.com +S3_REGION=us-east-1 +S3_USE_SSL=true +S3_PUBLIC_USE_SSL=true +S3_ACCESS_KEY=... +S3_SECRET_KEY=... +``` + +Use your bucket's regional hostname for both endpoints unless you deliberately split internal vs client-facing access. `S3_PUBLIC_ENDPOINT` must be reachable by whatever uploads and downloads via presigned URLs (clients, not just the server). + +On startup the server calls `CreateBucket` if the bucket is missing. In AWS it is simpler to **pre-create the bucket** and grant object permissions only (see IAM below). + +**IAM permissions** + +Scope access to the single bucket. Object keys are per-conversion prefixes, so list/delete can target the whole bucket: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": ["s3:ListBucket"], + "Resource": "arn:aws:s3:::officeconvert-prod" + }, + { + "Effect": "Allow", + "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"], + "Resource": "arn:aws:s3:::officeconvert-prod/*" + } + ] +} +``` + +Add `s3:CreateBucket` on `arn:aws:s3:::officeconvert-prod` only if you want the server to create the bucket on first boot. + +**CORS** + +Required only if uploads or downloads go **directly from a browser** to presigned URLs. Server-side clients (`curl`, the Go client) do not need CORS. Allow `PUT` and `GET` for your web origin on the bucket. + +**IAM roles vs IAM users** + +AWS recommends **roles** over long-lived **IAM user** access keys when the server runs on AWS compute (ECS, EC2, Lambda): a role grants **temporary** credentials that rotate automatically, with no static keys to store or leak. + +For this project today, the server reads explicit `S3_ACCESS_KEY` and `S3_SECRET_KEY` via the MinIO SDK. That maps cleanly to: + +| Where you run | Practical choice | +|---------------|------------------| +| Docker on a VPS, bare metal, or outside AWS | IAM **user** with the policy above; store keys in env or a secrets manager. Fine for a single service at low volume. | +| ECS / EC2 / EKS on AWS | Prefer an IAM **role** attached to the task or instance. Your orchestrator injects short-lived credentials; you still pass them into `S3_ACCESS_KEY` / `S3_SECRET_KEY` (and a session token if your runtime provides one — the server does not yet read a dedicated `S3_SESSION_TOKEN` env var). | + ## Conversion Tuning Notes If conversion fails on larger decks, tune these environment variables: