What to Ask When Hiring a Chief AI Officer

Most Chief AI Officer interviews test whether the candidate knows about AI. That is the wrong test. Everyone knows about AI now. The job is turning it into money, and almost nobody screens for that.

IBM's 2026 CEO Study surveyed 2,000 chief executives across 33 countries and 21 industries and found that 76 percent of their organizations now have a Chief AI Officer. A year earlier it was 26 percent. Fifty points in twelve months is a stampede, and stampedes produce a specific kind of hire: fast, senior, and vague.

I sit on both sides of this. I hold the Chief AI Officer credential and I build the systems, and I have watched companies interview for the seat like it is a technology role with a bigger title. Then eighteen months later the seat is empty again and everyone decides the role does not work. The role works fine. The interview was broken.

So here are the questions I would ask, what the good answers sound like, and the one question you have to answer yourself before you talk to a single candidate.

First, the question that is not for the candidate

Before any interview, finish this sentence out loud: the person in this seat is accountable for turning our AI into ______.

If the blank fills with revenue, margin, cost-to-serve, or a specific line in the P&L, keep going. If it fills with innovation, transformation, or enablement, stop. You are not ready to hire. You will run a great process, hire an impressive person, and hand them a job nobody defined. That is the single most common way these hires die, and I broke down the full anatomy in why Chief AI Officer hires fail.

The candidate cannot fix an undecided mandate. Neither can the search firm. Only you can.

The four questions that separate operators from narrators

There are two kinds of senior AI candidates in the market right now. Operators, who have shipped things that customers touched and finance noticed. Narrators, who have been near AI, spoken about AI, and built slides about AI. Narrators interview beautifully. They have had more practice.

These four questions tell them apart in about twenty minutes.

1. What did you ship, and who used it?

Listen for: a system in production, real users, a date, and what broke. Operators go specific fast because they were there. Red flag: pilots, proofs of concept, and pronouns that drift. If every win is "we" and every failure is "they," you are hiring a narrator.

2. What did you kill, and who was angry about it?

Listen for: a specific project they shut down, why, and the political cost. Red flag: nothing. A candidate who has never killed an AI initiative has never held real authority, because half this job is stopping expensive things that are not working. Killing things is the tell that they had the keys.

3. What number moved, and how do you know it was you?

Listen for: a business number, not a model number. Revenue, margin, cycle time, cost per case. Then attribution honesty: a good operator will tell you what they cannot cleanly claim. Red flag: accuracy scores and adoption rates offered as business outcomes, or a number so clean it clearly skipped a finance conversation.

4. Who fought you, and how did that end?

Listen for: a named function, a real conflict, and a resolution that involved the CEO or the board. Legal, security, a division head protecting a budget. Red flag: "everyone was aligned." Nobody who has actually done this job says that. Alignment that easy means the work was too small to threaten anyone.

Four questions, no whiteboard, no transformer trivia. If they have done the job, twenty minutes is enough to tell.

The technical bar, stated honestly

People ask whether a Chief AI Officer needs to be technical. My answer: technical enough to be lied to and notice.

They do not need to write production code. They do need to sit in a room while an engineer explains why something will take six months, and know whether that is true. They need to read an evaluation result and spot the benchmark that was chosen to flatter. They need to look at a demo and identify the part that was faked, because a demo always has one.

I wrote a whole piece on that gap called AI demos are easy, businesses are hard, and it is the exact skill you are hiring for. So put a real artifact in front of them. Bring your actual architecture, your actual vendor contract, your actual model evaluation, and ask what they would challenge. A narrator will compliment it. An operator will find the thing you were quietly worried about.

The governance questions, which are not about policy

Every candidate has a governance answer ready. Most of them describe a committee. Committees are where AI decisions go to get slow.

Ask these instead:

Who signs off before something risky ships, and what happens if they say no? You are testing whether they have built a real gate or a rubber stamp.
Tell me about a time your own system was wrong in production. How did you find out? The answer reveals whether they built monitoring or got a phone call from a customer.
What data would you refuse to put into a model, and who have you had to say no to? Saying no to a powerful internal stakeholder is the entire job on the worst day.

I run AI inside an export-controlled defense environment where controlled technical data never touches a public model and a human signs off before anything sensitive moves. That discipline is not a slide. It is a set of decisions someone makes under pressure, usually while a revenue team is asking why it is taking so long. Ask for the version of that story from their own career.

The reference calls, done properly

Most reference calls are a formality where you confirm the highlight reel. Waste of a good phone call.

Ask the reference about the thing that died. Which initiative got killed, who made the call, and how did the candidate take being overruled. Ask what the candidate could not get done, and why. Ask whether the candidate's budget went up or down over their tenure, because that is the cleanest available signal of whether leadership believed them.

And ask one more: would you give this person kill authority over a project you personally sponsored. The pause before the answer tells you more than the answer.

What you are actually hiring for

Strip away the title and this seat exists to close one gap: the distance between a model that works and money that shows up. That is a commercial job wearing a technical hat, which is why so many companies hire the wrong archetype, and why I keep pointing people back to what a Chief AI Officer actually owns before they write the job description.

The same IBM study found that only about a quarter of the workforce uses AI regularly, even though 86 percent of those CEOs believe their people have the skills. That gap is about ownership, not training budgets. Somebody has to decide what gets built, what gets stopped, what gets paid for, and what it has to return. If the person you hire cannot answer the four questions above with scars, they will not close that gap either. They will just describe it more articulately at the next board meeting.

Run the four questions and watch the pronouns. Then hire the one who has been in a fight and can show you the number that came out of it.

Frequently asked questions

What questions should you ask when hiring a Chief AI Officer?

Ask what they shipped, what they killed, what it earned, and who fought them. Those four questions separate operators from narrators faster than any technical screen. A strong candidate answers with systems that run in production, a decision they reversed, a number tied to revenue or margin, and a specific organizational fight they had to win. A weak one answers with frameworks, vendors, and pilots that never reached a customer.

Should a Chief AI Officer be technical?

Technical enough to be lied to and notice. They do not need to write production code, but they must be able to challenge an architecture, judge a build-versus-buy call, read an evaluation result, and know when a demo is hiding the hard part. The failure mode is a leader who cannot tell when the answer they just got was nonsense.

What is the biggest red flag in a Chief AI Officer interview?

Fluency without ownership. The candidate speaks beautifully about strategy, models, and governance, but every example is a pilot, a framework, or someone else's build. Ask who owned the number and watch the pronouns. If the wins are we and the losses are they, you are hiring a narrator. The second red flag is a candidate who never killed anything, because it means they have never had real authority.

How do you check references on a Chief AI Officer?

Skip the highlight reel and ask about the project that died. Ask the reference which initiative got killed, who made the call, and how the candidate handled being overruled. Ask what the candidate could not get done and why. The useful reference call is about friction and failure, because those are the conditions the role actually operates in.

What should you ask yourself before interviewing a Chief AI Officer?

Who is accountable for turning our AI into revenue and margin, and what levers will that person hold. If you cannot answer in one sentence, no interview will save you. Most Chief AI Officer hires fail because the company never decided what the job was, so the hire inherits the indecision. Decide the mandate, the budget, the kill authority, and the scoreboard before the first candidate call.

Do you need a Chief AI Officer in 2026?

Only if AI is close enough to how you make money that someone senior has to own the outcome. IBM's 2026 CEO Study found 76 percent of surveyed organizations now have a Chief AI Officer, up from 26 percent a year earlier, which means most of these seats were created in a single year of hiring pressure. Adoption that fast produces a lot of titles and not much accountability. If the honest answer is that AI is still a set of tools rather than a line in the P&L, you need a mandate before you need a hire.

Sources

Adoption figures, the workforce-usage gap, and the CEO skills confidence number are from the IBM Institute for Business Value 2026 CEO Study (released May 4, 2026; 2,000 CEOs and equivalent senior leaders across 33 geographies and 21 industries, surveyed February to April 2026, conducted with Oxford Economics). The interview structure, the archetype distinctions, and the governance questions are drawn from my own operating experience, not from the study.

The Chief AI Officer: the complete guide · Why Chief AI Officer hires fail · What a Chief AI Officer actually owns · The first 90 days · The fractional option

About the author

Jeff Brokaw is a sitting CMO and Certified Chief AI Officer who ships AI in production, not slideware. He built server-side visitor identification that names the companies landing on a page, a de-identified version of which runs live at unignorable.jeffbrokaw.com, and answer-engine optimization that produces measurable pipeline. He built the engine behind $185M in new business for a defense manufacturer, co-founded the AI fintech FaaStrak and took it from zero to $1M ARR in nine months, and authored the go-to-market behind a $114M institutional raise that came together in under 30 days.