Why I Switched to Custom AWS Infra

When Serverless Stops Being the Answer (And What Comes Next)

Vercel is phenomenal. I mean it - the developer experience is best-in-class, deployment is magical, and for 90% of web applications, it's the right choice. But I hit the 10% edge case: building an AI agent platform that runs long-running inference workloads, maintains WebSocket connections for hours, and processes GPU-intensive vision tasks. Serverless hit its limits, and I had to go deeper into the infrastructure stack.

This matters to developers because we're increasingly building applications that don't fit the "stateless request-response" model. AI agents that think for 10 minutes on complex tasks. Real-time collaborative editing. Live data processing pipelines. WebSocket-heavy applications. Serverless was designed for HTTP APIs, not persistent processes. Understanding when and how to break out of the serverless paradigm is becoming a core skill.

The Specific Problems Serverless Couldn't Solve

Let me be concrete about what broke:

1. Execution Time Limits

The Problem:
Vercel serverless functions: 10-second timeout (Hobby), 60-second timeout (Pro), 300-second max (Enterprise).

My Use Case:
AI agent running a multi-step research task - fetch documents, analyze with LLM, synthesize findings, generate report. Total execution time: 3-15 minutes depending on complexity.

What Broke:

// This works on Vercel
export default async function handler(req: Request) {
const summary = await quickSummarize(req.body.text); // 5 seconds
return Response.json({ summary });
}

// This times out
export default async function handler(req: Request) {
const research = await deepResearch(req.body.topic); // 8 minutes
return Response.json({ research }); // ❌ Function timeout after 5 minutes
}

Workarounds exist (queue + polling, streaming responses, background jobs), but they add complexity and fragility.

2. WebSocket Persistence

The Problem:
Serverless functions are stateless and short-lived. WebSockets require persistent connections.

My Use Case:
Real-time agent status updates - client connects via WebSocket, receives streaming thoughts from agent as it works, maintains connection for duration of task (5-30 minutes).

What Broke:

// This doesn't work on Vercel Edge
const wss = new WebSocketServer({ port: 8080 });

wss.on('connection', (ws) => {
// Connection would be terminated after function execution
agent.on('progress', (update) => {
ws.send(JSON.stringify(update)); // ❌ Connection dies when function completes
});
});

Vercel doesn't support long-lived WebSocket connections. You need a persistent server process.

3. Cost Unpredictability at Scale

The Problem:
Serverless pricing is per-request and per-GB-second. For high-frequency, compute-intensive workloads, costs scale linearly (or worse) with usage.

My Use Case:
Running local LLM inference (Llama 3 70B) for privacy-sensitive tasks. Each request needs 40GB RAM and 30-120 seconds of processing.

Cost Math:

Vercel Pro: ~$0.0002 per request-second + RAM costs

1000 inference requests/day × 60 seconds avg × $0.0002 = $12/day base

Plus RAM (40GB): 1000 × 60 × 40GB × $0.000012 = $28.8/day

Total: ~$1,200/month for just compute

Compare to dedicated AWS instance:

g5.2xlarge (GPU instance, 24GB VRAM, 32GB RAM): ~$1.20/hour

Running 24/7: $864/month

Can handle unlimited requests within capacity

For high-throughput, the dedicated instance wins.

4. Custom Runtime Requirements

The Problem:
Serverless environments provide standard runtimes (Node.js, Python, Go). Custom system dependencies, GPU drivers, or specific binary tools are difficult or impossible.

My Use Case:
Running vision models that require CUDA, specific PyTorch versions, custom compiled libraries for optimal inference.

What Broke:

This level of customization doesn't work in Vercel functions

FROM nvidia/cuda:12.1-runtime-ubuntu22.04
RUN apt-get install -y libvips ffmpeg
RUN pip install torch==2.2.0+cu121 --index-url https://download.pytorch.org/whl/cu121
COPY custom_compiled_lib.so /usr/local/lib/

Serverless platforms abstract away the runtime - which is usually good, but becomes a constraint for specialized workloads.

The Solution: AWS ECS with Fargate

After evaluating options (ECS, EKS, Lambda with extensions, Google Cloud Run), I landed on AWS ECS with Fargate for this specific use case.

Why ECS:

Full control over Docker containers (custom runtimes, dependencies)

Persistent services (long-running processes, WebSockets)

Predictable pricing (pay for vCPU/RAM hours, not per-request)

Auto-scaling based on actual metrics

Integrated with AWS ecosystem (ALB, CloudWatch, Secrets Manager)

Why Fargate over EC2:

No server management (Fargate is serverless containers)

Pay-per-use (only when tasks are running)

Automatic scaling without managing EC2 instances

Security (no SSH access needed, less attack surface)

The Architecture

┌─────────────────┐
│ Application │
│ Load Balancer │ ← Public HTTPS endpoint
└────────┬────────┘
│
┌────▼─────┐
│ Target │
│ Groups │ ← Health checks, routing
└────┬─────┘
│
┌────────▼──────────┐
│ ECS Fargate Tasks │
│ ┌───────────────┐ │
│ │ Container 1 │ │ ← Agent service (persistent)
│ │ Container 2 │ │ ← Worker service (auto-scale)
│ │ Container 3 │ │ ← WebSocket server
│ └───────────────┘ │
└───────────────────┘

The Implementation

1. Dockerfile for the Service

FROM node:20-slim

Install system dependencies

RUN apt-get update && apt-get install -y \
python3 python3-pip \
&& rm -rf /var/lib/apt/lists/*

Install Python dependencies for AI workloads

COPY requirements.txt .
RUN pip3 install -r requirements.txt

Install Node dependencies

WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

Copy application code

COPY . .

Expose ports

EXPOSE 3000 8080

Start application

CMD ["node", "server.js"]

2. ECS Task Definition (Terraform)

resource "aws_ecs_task_definition" "agent_service" {
family = "ai-agent-service"
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = "4096" # 4 vCPU
memory = "16384" # 16 GB RAM

execution_role_arn = aws_iam_role.ecs_execution_role.arn
task_role_arn = aws_iam_role.ecs_task_role.arn

container_definitions = jsonencode([
{
name = "agent-service"
image = "your-ecr-repo/agent-service:latest"

portMappings = [
{
containerPort = 3000
protocol = "tcp"
},
{
containerPort = 8080 # WebSocket
protocol = "tcp"
}
]

environment = [
{ name = "NODE_ENV", value = "production" },
{ name = "OLLAMA_HOST", value = "http://localhost:11434" }
]

secrets = [
{
name = "OPENAI_API_KEY"
valueFrom = aws_secretsmanager_secret.openai_key.arn
}
]

logConfiguration = {
logDriver = "awslogs"
options = {
"awslogs-group" = "/ecs/agent-service"
"awslogs-region" = "us-east-1"
"awslogs-stream-prefix" = "ecs"
}
}
}
])
}

3. ECS Service with Auto-Scaling

resource "aws_ecs_service" "agent_service" {
name = "agent-service"
cluster = aws_ecs_cluster.main.id
task_definition = aws_ecs_task_definition.agent_service.arn
desired_count = 2 # Min 2 tasks for availability
launch_type = "FARGATE"

network_configuration {
subnets = aws_subnet.private[*].id
security_groups = [aws_security_group.ecs_tasks.id]
assign_public_ip = false
}

load_balancer {
target_group_arn = aws_lb_target_group.agent_service.arn
container_name = "agent-service"
container_port = 3000
}
}

Auto-scaling based on CPU

resource "aws_appautoscaling_target" "agent_service" {
max_capacity = 10
min_capacity = 2
resource_id = "service/${aws_ecs_cluster.main.name}/${aws_ecs_service.agent_service.name}"
scalable_dimension = "ecs:service:DesiredCount"
service_namespace = "ecs"
}

resource "aws_appautoscaling_policy" "agent_service_cpu" {
name = "agent-service-cpu-scaling"
policy_type = "TargetTrackingScaling"
resource_id = aws_appautoscaling_target.agent_service.resource_id
scalable_dimension = aws_appautoscaling_target.agent_service.scalable_dimension
service_namespace = aws_appautoscaling_target.agent_service.service_namespace

target_tracking_scaling_policy_configuration {
target_value = 70.0
predefined_metric_specification {
predefined_metric_type = "ECSServiceAverageCPUUtilization"
}
}
}

4. Application Load Balancer

resource "aws_lb" "main" {
name = "agent-service-alb"
internal = false
load_balancer_type = "application"
security_groups = [aws_security_group.alb.id]
subnets = aws_subnet.public[*].id
}

resource "aws_lb_target_group" "agent_service" {
name = "agent-service-tg"
port = 3000
protocol = "HTTP"
vpc_id = aws_vpc.main.id
target_type = "ip" # Required for Fargate

health_check {
path = "/health"
healthy_threshold = 2
unhealthy_threshold = 10
timeout = 60
interval = 120
matcher = "200"
}
}

resource "aws_lb_listener" "https" {
load_balancer_arn = aws_lb.main.arn
port = "443"
protocol = "HTTPS"
ssl_policy = "ELBSecurityPolicy-TLS-1-2-2017-01"
certificate_arn = aws_acm_certificate.main.arn

default_action {
type = "forward"
target_group_arn = aws_lb_target_group.agent_service.arn
}
}

The Migration Weekend (and What Broke)

Migrating from Vercel to AWS was not trivial. Here's what I learned:

Networking is Hard

Setting up VPCs, subnets (public/private), NAT gateways, route tables, security groups - this is infrastructure complexity that Vercel abstracts away. I spent 4 hours debugging "why can't my tasks pull from ECR" (answer: private subnet needs NAT gateway for internet access).

The networking setup that took 4 hours to get right

resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
enable_dns_hostnames = true
enable_dns_support = true
}

resource "aws_subnet" "public" {
count = 2
vpc_id = aws_vpc.main.id
cidr_block = "10.0.${count.index}.0/24"
availability_zone = data.aws_availability_zones.available.names[count.index]
map_public_ip_on_launch = true
}

resource "aws_subnet" "private" {
count = 2
vpc_id = aws_vpc.main.id
cidr_block = "10.0.${count.index + 10}.0/24"
availability_zone = data.aws_availability_zones.available.names[count.index]
}

NAT Gateway for private subnets to access internet (ECR, etc.)

resource "aws_nat_gateway" "main" {
allocation_id = aws_eip.nat.id
subnet_id = aws_subnet.public[0].id
}

IAM Roles and Permissions

ECS requires two types of IAM roles:

Execution Role: Allows ECS to pull images, write logs, fetch secrets

Task Role: Allows your application code to access AWS services

I spent 2 hours on "access denied pulling from ECR" before realizing my execution role was missing ecr:GetAuthorizationToken.

Logging and Monitoring

With Vercel, logs are automatic and searchable. With ECS, you need to configure CloudWatch, set up log groups, manage retention, and build dashboards.

// Application-side structured logging for CloudWatch Insights
import winston from 'winston';

const logger = winston.createLogger({
format: winston.format.combine(
winston.format.timestamp(),
winston.format.json()
),
transports: [
new winston.transports.Console() // CloudWatch captures stdout
]
});

logger.info('Agent task started', {
taskId: taskId,
userId: userId,
model: 'gpt-4'
});

Then query in CloudWatch Insights:

fields @timestamp, taskId, model, @message
| filter @message like /Agent task started/
| stats count() by model

Cost Comparison: 3 Months of Real Data

Before (Vercel Pro + external compute for AI):

Vercel Pro plan: $20/month

Modal.com GPU hours for inference: $800-1200/month

Upstash Redis: $40/month

Total: ~$900/month

After (AWS ECS + self-hosted inference):

2x g5.2xlarge Fargate tasks (GPU, 24/7): ~$1,728/month

Application Load Balancer: $25/month

NAT Gateway: $45/month

Data transfer: $30/month

CloudWatch: $15/month

Total: ~$1,843/month

Wait, that's more expensive! But here's the key: unlimited usage within capacity. On Vercel + Modal, I was rate-limiting users to control costs. On AWS, I can handle 10x more traffic for the same $1,843.

Usage Scaling:

At current usage (1000 inferences/day): AWS costs more

At 5000 inferences/day: Break-even

At 10,000+ inferences/day: AWS is 60% cheaper

Plus: full control over runtime, no timeout limits, native WebSocket support, custom optimizations.

When To Choose What

After this experience, here's my decision matrix:

Choose Vercel (or similar serverless) when:

Request-response HTTP APIs (most web apps)

Execution time under 30 seconds

Unpredictable or spiky traffic (serverless handles this well)

Stateless operations

You want minimal DevOps overhead

Team is small, wants to focus on features not infrastructure

Choose AWS ECS/Fargate when:

Long-running processes (minutes to hours)

WebSocket or persistent connections

Custom runtime requirements (specific libraries, GPU, system tools)

Predictable baseline traffic (dedicated instances are cost-effective)

Need fine-grained control over environment

Team has infrastructure expertise or wants to build it

Choose Kubernetes (EKS) when:

Everything ECS does + multi-cloud portability

Complex orchestration requirements

Large team with k8s expertise

Need advanced scheduling, batch jobs, sophisticated networking

My Honest Take

Migrating from Vercel to AWS was painful. A weekend project became a week project. I debugged networking, fought with IAM policies, learned Terraform, set up monitoring dashboards, and questioned my life choices.

But it was the right move for my specific use case. I now have:

✅ Unlimited execution time for agent tasks

✅ Native WebSocket support for real-time updates

✅ Full control over GPU inference runtime

✅ Predictable costs at scale

✅ Custom optimizations (shared model loading, connection pooling)

The trade-off: I'm now responsible for infrastructure. Security updates, scaling policies, networking, monitoring - all on me. Vercel abstracts this away beautifully, which is why most projects should stay there.

Bottom line: Serverless is amazing until it isn't. When you hit the edge cases (long execution, WebSockets, custom runtime, cost at scale), moving to container orchestration is the next evolution. It's more complex, but it unlocks capabilities that serverless fundamentally can't provide.

Start with Vercel. Graduate to AWS when you have to, not because it's trendy. And when you do migrate, block out a week, learn Terraform, and embrace the infrastructure complexity - it's worth it for the right workloads.

The future isn't "serverless wins" or "containers win." It's knowing which tool to use for which job.