Stop Using Frontier Models Like Employees

Your frontier model is costing you $500/month because you're using it like an employee. It answers every question. It handles every edge case. It's always on call.

That's not what frontier models are good for. And it's not what they're priced for.

Frontier models are good at imagining tasks they've never seen before; cheap models are good at executing tasks they've seen a thousand times. If you pay a consultant $500/hour to do data entry, the problem isn't the consultant—it's your job design.

The same applies here. Your bill isn't bloated because the models are expensive. It's bloated because you're having expensive models do cheap work.

The Cost Gap Is Wider Than You Think

Let me ground this in numbers. According to publicly available API pricing as of July 2026, the cost difference between a GPT-4o-class model and a 7-billion-parameter model is approximately 10 to 20 times per token.

Ten to twenty times.

A frontier model costs roughly $15 per million input tokens. A competent 7B model costs $0.30 to $0.60 per million tokens. The gap has widened, not narrowed, even as prices fell across the board.

Now consider a typical customer interaction: you need to classify a support ticket, retrieve a relevant document, run a few format checks, and summarize the findings. That's four distinct operations.

Two of them—classification and format checks—are deterministic. A cheap model handles them. One operation—retrieval—is retrieving data, not reasoning. A cheap model handles that too. One operation—summarization in a specific format—requires some nuance but no real cognitive work.

Only one operation genuinely needs frontier reasoning: figuring out which document to retrieve when the customer's request doesn't match any obvious category. That's the thinking step.

If you route all four operations to a frontier model, you're paying 10-20x for three operations that don't need it.

Since 70% to 80% of subtasks in a typical conversational turn can be handled by small models, the aggregate cost per turn drops by 80% to 95% when routed intelligently.

Eighty to ninety-five percent cost reduction. Not optimization. Restructuring.

The Consultant Pattern: Think → Execute → Verify

The mental model is simple. You hire a consultant for three things they're uniquely good at:

1. Thinking. They diagnose the problem, map the solution space, propose an approach.
2. Planning. They break the work into steps, flag risks, identify what needs execution vs. what needs more thinking.
3. Verification. They review the work after it's done, spot errors, validate the approach worked.

You don't hire them to execute every step. That's what employees are for.

In AI terms:

Thinking (frontier model): Given a customer question, what's the most useful information to retrieve? What's the actual intent here if the surface request is wrong? What edge cases does this scenario create?

Execution (cheap model): Rewrite the customer's question into a retrieval query. Classify the urgency. Format the response. Filter the results for relevance.

Verification (cheap model, then human if it fails): Check that the response matches the customer's actual intent. Spot obvious errors.

Only the first step requires frontier capability. The others need reliability and consistency, which cheap models deliver just fine once the problem is well-defined.

Here's what that looks like in practice:

# Step 1: Thinking (frontier model)
# "Given this customer request, what do we actually need to solve?"
thinking_prompt = f"""
Customer request: {customer_message}

What is the actual customer need here? What information would solve it?
Return a JSON with:
- intent: what they're really asking
- required_info: what data we need
- complexity_level: simple/medium/hard
"""

response = frontier_model.query(thinking_prompt)
intent = parse_json(response)

# Step 2: Execution (cheap model)
# Based on the intent, execute the retrieval and formatting
execution_prompt = f"""
Intent: {intent['intent']}
Need: {intent['required_info']}

Retrieve and format the response. Keep it concise.
"""

formatted_response = cheap_model.query(execution_prompt)

# Step 3: Verification (cheap model or heuristic)
# Does the output match what we said we'd deliver?
if is_relevant(formatted_response, intent):
    return formatted_response
else:
    # Escalate to frontier model if execution failed
    return frontier_model.query(f"This approach didn't work. Try again: {intent}")

The frontier model does 20% of the work. The cheap model does 70%. Heuristics and humans do 10%.

Your bill reflects that split.

The Real-World Numbers

Theory is nice. Real numbers are better. At 100,000 to 500,000 DAU, the Inworld Consumer AI Stack 2026 report estimates the routing versus non-routed architecture cost difference at $200,000 to $400,000 per month.

Two to four hundred thousand dollars. Per month. For the same output quality.

Using cheap models for cheap work. Using frontier models when failure costs more than tokens. This prevents the most common cost mistake: using a good model with the wrong reasoning budget.

That last line is key. A cheap model with the wrong task is expensive. A frontier model with the wrong task is wasted money.

The routing decision—which task goes to which model—isn't about capability matching. It's about cost-to-failure. Ask yourself: if this task fails, what's the cost? If the cost of failure is higher than the cost of tokens, use a better model. Otherwise, don't.

Where This Pattern Breaks (And Why That Matters)

This approach has a real failure mode that's worth understanding.

The 2026 conversation around AI economics has produced a consensus playbook. Route simple queries to cheap models. Keep expensive queries on capable models. Cut the bill, keep the quality. Every CFO has seen the math. Every engineering team has built it or is building it. The math is real. The Pareto trap is also real.

The Pareto trap: you route 80% of requests to cheap models, see an 80% cost reduction, and call it done. But the 20% of requests that go to the frontier model are now handling all the hard cases—the edge cases, the novel problems, the ones cheap models failed on. The frontier model is now working harder than before.

And failure creeps in: The piece below is what I told the team after we ran the post-mortem. It describes the architecture they built, the failure mode they walked into, the detection methodology that would have caught it earlier, and the architectural pattern they should have built instead.

The detection methodology matters. If you route aggressively and don't monitor, you won't know when a cheap model starts producing subtle errors. The errors accumulate. Customers notice before you do.

The fix: No model swap, no architecture change — just rewriting control logic in natural language dropped runtime by 88% and lifted benchmark scores 17 points.

Rewriting the control logic. The problem isn't the models. It's the decision tree that chooses which model to use. A bad routing decision spreads errors across all downstream tasks.

How to Implement This Without Breaking Things

Step one: don't start with routing. Start with one model, measure the cost, then identify which tasks are expensive.

Rule-based routing (a regex or keyword match) adds under 1 ms. Embedding-based routing adds about 5 ms. Semantic routing and heavier ML classifiers add 50-100 ms. The routing layer itself is fast. The problem is getting the routing decision right.

Step two: identify thinking tasks. These are tasks where the model needs to reason about novel inputs—classification of ambiguous cases, decision-making under uncertainty, creative synthesis. These need frontier models.

Step three: instrument for failure. Log every routing decision and every output. Track which model handled which task and whether it succeeded. You're not looking for cost savings yet—you're looking for the tasks that need frontier models.

Step four: implement gradual rollout. Route 10% of thinking tasks to a cheap model as a canary. Monitor error rates. If errors jump, revert. This teaches you which thinking tasks actually benefit from frontier reasoning.

Here's what that instrumentation looks like:

import logging

def route_and_log(task_type, input_data):
    # Decide which model handles this
    if task_type in ["thinking", "novel_classification"]:
        model_choice = "frontier"
    else:
        model_choice = "cheap"
    
    # Execute
    response = route_to_model(model_choice, input_data)
    
    # Log for analysis
    logging.info({
        "task_type": task_type,
        "model_choice": model_choice,
        "input_tokens": count_tokens(input_data),
        "output_tokens": count_tokens(response),
        "cost": estimate_cost(model_choice, response),
        "success": validate_output(response),
        "timestamp": now()
    })
    
    return response

# Aggregate weekly
# Group by task_type
# Find tasks where cheap model success rate drops below 95%
# Those move to frontier model in next iteration

That log becomes your routing decision tree.

The Takeaway

• Frontier models are consultants, not employees. Expensive at $500/hour. Indispensable for diagnosis and strategy. Wasted on data entry.

• Thinking vs. execution is the real division. 70-80% of AI tasks are execution—applying a well-defined decision to clear inputs. Cheap models handle these. 20-30% are thinking—defining the decision in the first place. Frontier models handle these.

• The cost gap is 10-20x. A frontier model costs ten to twenty times more per token than a competent small model. That's not a minor difference. That's the difference between a $1,000 bill and a $50,000 bill.

• Routing wrong is worse than not routing. An aggressive routing strategy with bad decision logic causes more errors than a simple "use frontier everywhere" approach. Start with monitoring, not optimization.

• Instrument before you optimize. Log every routing decision and every failure. Your routing strategy should emerge from that data, not from theory. The tasks that need frontier models aren't always obvious.

• Start with the 80/20 rule, then refine. Your first pass—route obvious execution tasks to cheap models, obvious thinking tasks to frontier models—probably captures 80% of the savings. The remaining 20% of optimization requires instrumentation and patience.

The teams winning on AI costs in 2026 aren't the ones that picked the cheapest model. They're the ones that built a routing layer that understands the difference between thinking and executing, and routed accordingly.

Your frontier model is too expensive to use like an employee. Treat it like what it is: a consultant you can afford to call for diagnosis and planning, but not for every task.

---

Tags: ai-costs, model-routing, frontier-models, architecture, llm-optimization