Your AI agency is probably AI-painted. Here's the test.

Most agencies that use AI are not AI-native. They are AI-painted. They have grafted generative tools onto a workflow built for the era when ten people were doing the work. The tools save hours. The workflow stays the same.

The test for whether your next agency is one or the other is short. Emergence Capital named it cleanly: for an AI-native services company, the proof is not revenue or logos. The proof is "AI doing a material share of the work at high gross margin." Two things in one sentence. Material share. High margin. If the agency cannot show both, the AI is decoration.

Here is how to run the test in a 60-minute conversation, before you sign anything.

Question 1 — What is AI doing in delivery?

Ask the agency to walk you through one engagement, end to end, and name the steps where AI is doing the work versus the steps where humans are.

The wrong answer sounds like: "We use AI for ideation, brainstorming, and inspiration." Translation: AI is a search engine for the team. The work is being done by humans the way it always was, with a chat window open on the side.

The right answer sounds like: "AI runs the research synthesis from these three data sources. AI generates the first draft of every asset. AI catches deviations from brand voice in QA. AI writes the weekly client report. The operator scopes the work, makes strategic calls, and approves what ships." Specific steps, owned by AI, with the operator at named decision points.

If they cannot answer the question with that level of specificity, the agency is AI-painted. They are using ChatGPT internally and billing you the old hourly rate.

Question 2 — What does the operating system look like?

Every AI-native services company has a named system. Every one. Every (every.to) ships theirs as a public Claude Code plugin called Compound Engineering. Pace publishes their work as Agent Operating Procedures. Crosby has a tiered orchestration model for legal contracts. The system is named, documented, and visible.

Ask the agency: "What is your operating system called? Can I see how it works?"

If they hesitate, or if the answer is "we have a process document somewhere," they do not have a system. They have a workflow. The difference matters because the system is what compounds across engagements; a workflow is just the way the team happens to do it this month. The system is what you are paying for. Buying access to a workflow is buying a team's habits.

Question 3 — How is it priced?

This is the question where most AI-painted agencies give themselves away.

If the pricing is hourly, retainer-by-headcount, or "we'll scope a custom engagement," the agency is structurally not AI-native. AI cuts hours. An hourly agency that uses AI is shrinking its own revenue per engagement. They cannot do that for long, so they do not do it at all — they keep the hours they bill, and the AI productivity gains stay inside the agency as margin instead of being passed on.

Crosby prices per contract — $375 for routine, more for complex, with a 58-minute median turnaround. Sequoia's framing is "deal velocity, not billable hours". The 4As is publishing the "Agency as Marketing Purveyor" thesis — productized services, licensed IP, fixed pricing per outcome.

Ask the agency: "If we do this work, what does pricing look like once it is steady-state? Do we still pay you by headcount or by hour?"

A pricing answer that scales with outcomes — per asset, per workflow, per conversion lift, per delivered project — is a sign the model is real. A pricing answer that scales with their team's hours is a sign you are buying a traditional agency with newer tools.

Question 4 — What is the team size on our account?

The AI-native services company runs lean by design, not by cost-cutting. Every operates with 15 people total across 5 products and a consulting business. Mercor turned recruiting into a $10B AI-native business with a small team. Harper rebuilt insurance brokerage at 30x revenue per rep.

Ask: "How many people will be touching our account?"

If the answer is large — eight, ten, twelve — and the pitch is that all of them are powered by AI, you are paying for a team. The AI is decorative. A real AI-native engagement has one or two named operators on your account, supported by an agent system underneath. That is not a quality compromise; that is the entire point.

Question 5 — Can we see what AI is doing in real time?

The agencies that are using AI seriously can show you what it is doing on your engagement, week by week. Audit logs. Approval gates. The actual outputs of every agent. The kill switch.

The agencies using AI as a backstage productivity hack will not let you see this. The reason is simple: showing you would expose how thin the AI layer actually is.

Ask: "How will I see what AI is doing in our engagement? What logs do you keep? What approvals do I sign off on?"

A confident, specific answer is the AI-native signal. A vague "we'll set up a Slack channel and keep you in the loop" is the AI-painted signal.

What the test actually does

Five questions. About 30 minutes of the prospect's time. Each one is hard to fake because each requires the agency to point at something concrete: a step in delivery, a system, a pricing model, a team size, a log file. AI-painted agencies fail multiple questions in a row. AI-native agencies pass all five with specifics.

We sit on the AI-native side of this test by construction — the operating system is named, the team is small, the pricing is moving toward per-outcome, the agent activity is visible to the client. The reason we wrote this post is not as a marketing exercise. It is because most prospects we meet have already met three AI-painted agencies and arrive on the call assuming everyone is the same.

The five questions are how to tell. Run them on the next agency that pitches you AI. Then run them on us.

Book a strategy call →