Containerise once, deploy anywhere · Building AI Applications

Where should an AI application run? Chapter 7 compares five deployment targets and shows the habits that keep every option open.

Containerise first

Package every component in a container: the API, the agent workers, the UI and the supporting services. One docker-compose.yml then runs the complete stack on a laptop, and the same images run on any target later. Containers are the cheapest insurance against lock-in.

Run agent work in queue-backed workers

LLM calls take seconds and agent runs take minutes, which is longer than many platform request timeouts. Do not run agents inside web requests. Put tasks on a queue, process them in workers, stream progress to the UI and store results. Workers can then scale on queue length, for example with KEDA on Kubernetes.

Five targets

Target	Choose it for	Watch out for
Local server	Development, demos, small private tools	No redundancy; you are the operations team
PaaS (Render, Railway, Fly.io, App Service)	Speed: from repository to URL in an afternoon	Request timeouts, limited GPU options, cost at scale
Cloud VM	Control at a moderate scale	Patching, backups and monitoring are your job
Kubernetes	Many services, autoscaling, larger teams	Operational complexity
On-premise	Data sovereignty, regulation, existing data centres	Hardware, GPUs and staff

There is no universally right answer. A pilot on PaaS can move to Kubernetes later without rewriting anything, if it was containerised from day one.

Infrastructure as code and readiness

Define infrastructure as code (Terraform, Bicep, Pulumi) so environments are reproducible and reviewable. Keep a production-readiness checklist: health checks, secrets management, backups, monitoring, autoscaling limits, and rollback.

Key takeaways

Containerise every component to keep all options open.
Run agent work in queue-backed workers.
Define infrastructure as code.