AI Capacity Planning with Python Time-Series Models

AI Capacity Planning with Python Time-Series Models

Reactive autoscaling — add a pod when CPU crosses 70% — has a structural lag baked in. By the time utilization crosses the threshold, a scale-up event fires, a new instance boots, warms its connection pools and caches, and becomes ready to serve traffic. For a lightweight stateless service that might be 15-30 seconds. For anything with a heavier cold start — a JVM service, a container that needs to establish database connections, an ML inference server loading model weights — it can be minutes. During that window, the traffic that triggered the scale-up is still arriving, and existing capacity is already saturated. Reactive autoscaling doesn’t prevent the overload; it just eventually resolves it.

Forecasting-based capacity planning attacks the lag directly: predict tomorrow’s 9am spike using the pattern from the last eight Tuesdays, and scale up ahead of the threshold being crossed, not after. This doesn’t replace reactive autoscaling — it handles the predictable part of demand, leaving reactive scaling to do what it’s actually good at: catching the unpredictable deviation from the forecast.

Preparing the historical signal

Aggregate request volume (or whatever metric best proxies required capacity — CPU-seconds, active connections) at a consistent interval, with enough history to capture both daily and weekly seasonality:

import pandas as pd

def prepare_forecast_input(raw_metrics: pd.DataFrame) -> pd.DataFrame:
    """raw_metrics: columns [timestamp, request_count], 1-minute granularity"""
    hourly = (
        raw_metrics.set_index("timestamp")["request_count"]
        .resample("1h")
        .sum()
        .reset_index()
    )
    hourly.columns = ["ds", "y"]  # Prophet's required column names
    return hourly

At least 8-10 weeks of history is a reasonable minimum — enough for the model to see multiple instances of both the weekly pattern (weekday vs weekend) and any monthly billing-cycle or payday-driven spikes that recur.

Fitting a forecast with Prophet

Prophet handles multiple overlapping seasonalities (daily + weekly) and holiday effects out of the box, which is most of what infrastructure load actually exhibits, without requiring you to hand-tune ARIMA orders:

from prophet import Prophet

def fit_load_forecast(history: pd.DataFrame) -> Prophet:
    model = Prophet(
        daily_seasonality=True,
        weekly_seasonality=True,
        yearly_seasonality=False,  # usually not enough history to fit this reliably
        interval_width=0.90,       # 90% confidence interval on the forecast
        changepoint_prior_scale=0.05,  # conservative trend flexibility — avoid overfitting recent noise
    )
    model.fit(history)
    return model

def generate_forecast(model: Prophet, periods_hours: int = 48) -> pd.DataFrame:
    future = model.make_future_dataframe(periods=periods_hours, freq="h")
    forecast = model.predict(future)
    return forecast[["ds", "yhat", "yhat_lower", "yhat_upper"]]

changepoint_prior_scale=0.05 (Prophet’s default, deliberately kept rather than increased) matters for infrastructure data specifically — a higher value makes the model more willing to fit sudden trend changes, which sounds good until you realize it also means the model will happily interpret last Tuesday’s one-off incident-driven traffic drop as a genuine trend shift. Conservative trend flexibility, with seasonality doing most of the work, is the safer default for a signal you’re using to provision real infrastructure.

Known events as regressors, not surprises

A marketing campaign or a product launch isn’t in the historical pattern — if you know it’s coming, tell the model explicitly rather than letting it get blindsided the same way reactive autoscaling would:

def add_known_events(model: Prophet, events: list[dict]) -> Prophet:
    events_df = pd.DataFrame(events)  # columns: holiday, ds, lower_window, upper_window
    model.add_country_holidays(country_name="US")  # standard holidays as a baseline
    return model

# Custom high-traffic events beyond standard holidays
special_events = pd.DataFrame({
    "holiday": ["black_friday_campaign", "product_launch"],
    "ds": pd.to_datetime(["2026-11-27", "2026-09-15"]),
    "lower_window": [-1, 0],   # campaign traffic starts building 1 day before
    "upper_window": [2, 3],    # elevated traffic continues for days after
})

model = Prophet(daily_seasonality=True, weekly_seasonality=True, holidays=special_events)
model.fit(history)

This is the mechanism that lets a known marketing calendar directly inform infrastructure provisioning — feeding planned campaign dates into the same model that’s already forecasting organic traffic, instead of running that as a separate manual “how much capacity do we think we’ll need” conversation disconnected from the actual forecasting pipeline.

Translating a load forecast into an instance count

A forecast in requests-per-hour isn’t directly actionable — it needs to become “how many instances.” Use each instance’s known sustainable throughput (measured from load testing, not guessed) to convert:

def forecast_to_instance_count(
    forecast: pd.DataFrame,
    requests_per_instance_per_hour: float,
    safety_margin: float = 1.25,
) -> pd.DataFrame:
    plan = forecast.copy()
    # Use yhat_upper (the confidence interval's upper bound), not the point estimate —
    # under-provisioning against a point forecast means routinely being caught short
    # on the ~50% of hours where actual load exceeds the median prediction.
    plan["required_instances"] = (
        (plan["yhat_upper"] * safety_margin) / requests_per_instance_per_hour
    ).apply(lambda x: max(1, int(x) + 1))  # round up, minimum 1
    return plan[["ds", "required_instances"]]

Sizing against yhat_upper instead of yhat is the single most important decision in this function — a point forecast is a median-ish estimate, meaning by construction actual load exceeds it roughly half the time. Provisioning against the upper confidence bound, with an additional flat safety margin on top, is what keeps this a genuinely proactive plan rather than one that’s under capacity on a routine basis.

Wiring the plan into a proactive scaling schedule

The forecast needs to become a scheduled action, not just a dashboard nobody checks before the spike happens:

async def apply_proactive_scaling(plan: pd.DataFrame, k8s_client, deployment_name: str):
    now = pd.Timestamp.now(tz="UTC")
    # Look 30 minutes ahead — enough lead time for even a slow cold-start service to warm up
    upcoming = plan[(plan["ds"] >= now) & (plan["ds"] <= now + pd.Timedelta("30min"))]
    if upcoming.empty:
        return

    target_replicas = int(upcoming["required_instances"].max())
    current_replicas = await k8s_client.get_replica_count(deployment_name)

    if target_replicas > current_replicas:
        await k8s_client.scale_deployment(deployment_name, replicas=target_replicas)
        emit_metric("proactive_scale_up", tags={"deployment": deployment_name, "target": target_replicas})

Run this on a schedule (every 15 minutes is reasonable) rather than continuously — capacity planning doesn’t need second-level responsiveness, and a scheduled job is simpler to reason about and debug than a continuously running control loop.

Proactive and reactive together, not proactive instead of reactive

Don’t turn off your HPA (Horizontal Pod Autoscaler) or equivalent reactive scaler once this is running. The forecast handles the predictable baseline — the daily and weekly pattern, known campaigns — and reactive scaling remains the safety net for whatever the forecast gets wrong: a genuine surprise traffic event, an incident causing retry-storm load, a forecast that simply missed. Layering both means the forecast reduces how often and how severely the reactive scaler has to react under load, without removing the reactive scaler’s role as a backstop.

Backtesting forecast accuracy before trusting it with real infrastructure spend

Before this drives actual scaling decisions, validate it against held-out historical data — how far off was the forecast from what actually happened:

from prophet.diagnostics import cross_validation, performance_metrics

def backtest_forecast_accuracy(model: Prophet) -> pd.DataFrame:
    cv_results = cross_validation(
        model,
        initial="30 days",   # train on the first 30 days
        period="7 days",     # re-forecast every 7 days
        horizon="2 days",    # forecast 2 days ahead each time, matching real usage
    )
    metrics = performance_metrics(cv_results)
    return metrics[["horizon", "mape", "coverage"]]  # coverage: % of actuals within the CI

mape (mean absolute percentage error) tells you typical forecast error magnitude; coverage tells you whether your 90% confidence interval is actually capturing ~90% of real outcomes — if coverage is meaningfully below 90%, your safety margin needs to be larger than the interval alone suggests, because the model is more overconfident than its own stated interval implies.

The takeaway

Reactive autoscaling reacts to a threshold that’s already been crossed, which means it’s structurally incapable of preventing the overload window during a predictable spike — only forecasting ahead of the threshold can do that. The AI here isn’t replacing your existing autoscaler; it’s giving it a head start on the part of demand that was never actually unpredictable in the first place, while leaving reactive scaling to do the job it’s suited for: catching the part of demand that genuinely is.