Pick the Right AI Bets, Then Prove They Work

Most enterprise AI programs fail twice. They aim at the wrong problems, and they can't prove the tools they deploy actually work.

The first failure wastes money. The second creates risk, and that risk compounds, because an unvalidated tool repeats the same error at machine speed across every decision it touches. When AI shapes decisions about hiring, promotion or credit, some of that risk is legal. Anti-discrimination law applies whether a person or an algorithm makes the call, and buying the tool from a vendor doesn't transfer your accountability for the outcome.

The pattern behind the headlines

The public record is full of examples, and each one breaks a specific link between an AI output and the decision it drives:

• Amazon's résumé screener (2018). Trained on a decade of résumés submitted to the company, it learned to downgrade résumés containing the word "women's." It learned past hiring patterns, not job qualifications (Reuters, via The Irish Times).
• Epic's sepsis model (2021). An independent validation at Michigan Medicine found it missed about two-thirds of sepsis cases while flagging nearly one in five hospitalized patients. Vendor evidence didn't transfer to a new patient population (JAMA Internal Medicine).
• Commonwealth Bank of Australia (2025). The bank cut 45 call-centre roles, saying a voice bot had reduced call volumes. The union said volumes were actually rising, and the bank reversed the cuts and apologized. The claimed outcome hadn't been checked against the real one (Finance Sector Union).
• Deloitte Australia (2025). A government report produced with generative AI contained fabricated citations, and Deloitte issued a partial refund. Nobody checked the outputs against the sources they claimed to represent (The Register).
• Air Canada (2024). A support chatbot misstated the airline's bereavement fare policy, and a tribunal held the airline liable for what it said (tribunal decision).
• Workday and Eightfold (2025 and 2026). Lawsuits now ask whether AI screening disadvantaged older applicants (a court let the Workday case proceed as a nationwide age-discrimination collective in May 2025; court order) and whether hidden match scores are consumer reports that require disclosure (the Eightfold suit, filed January 2026; Epstein Becker Green summary). Neither case has reached a finding. When nobody monitors a tool, the validity question gets answered in discovery.

None of these is mainly a story about bad technology. Each tool did roughly what that kind of tool does. What failed was the step between the output and the decision: nobody gathered evidence that the output meant what people assumed it meant, for the people and situations where it was actually used.

I/O psychology has a name for that missing step. It's validity, and it isn't a feature of a tool. It belongs to a particular use of a tool's output, for a particular group of people, for a particular decision. That's why the same model can be sound in one deployment and indefensible in the next.

Question 1: Where should AI go to work?

Start from the work, not the license. "We bought a copilot, where can we use it?" is how capable tools end up aimed at the wrong problem. Two lenses fix this.

• Job analysis looks at the work. It breaks a role into tasks and the knowledge, skills and abilities they demand. It shows which tasks are routine enough to automate, which need judgment AI should support rather than replace, and what "good" looks like for each.
• Jobs to be Done looks at the person hiring the tool. A recruiter doesn't want a ranking engine. They want qualified candidates on the phone before a competitor calls them. JTBD tells you whether people will adopt the tool or quietly work around it.

The hinge is the desired outcome statement, Tony Ulwick's term: "minimize the time it takes to confirm a candidate meets the license requirement." Rate each outcome for importance and current satisfaction and you can see where the job is underserved. Those same outcome statements later become the criteria the AI is validated against. That's the thread most frameworks drop between strategy and evaluation.

Question 2: Can you trust what it tells you?

Validity theory is how employers defend a hiring assessment to a regulator or in court. It asks one thing: does the evidence support using this output to make this decision about these people? Put "AI model" in place of "assessment" and the question stays the same. So does the liability.

Take one model that scores how well a candidate's experience matches a job description. Use it to decide who a recruiter calls first, and a score that tracks reasonably with interview performance may be good enough, because a person still reviews everyone. Use the same score to auto-reject the bottom half, and it now has to support a much stronger claim: that nobody below the cut is qualified. Use it to choose internal candidates for promotion, and its evidence may not carry over at all.

The model is identical in all three. The claim changes, and so does the evidence you need. That's why a vendor's validation report or a strong demo can't settle the question for you. It shows the tool worked for someone's use, under their conditions.

So the leadership question changes too. It isn't "Is this AI accurate?" It's "Accurate for what decision, about whom, and what happens when it's wrong?"

Build the argument, weakest link first

Samuel Messick framed validity as one unified judgment about the evidence and consequences behind an interpretation. Michael Kane made it practical: write out the chain of inferences from raw output to decision, then gather evidence for the weakest link first. For AI, the chain usually looks like this:

1. Scoring. The output captures the job-relevant features and nothing else. Subject-matter experts review outputs against the job.
2. Process. It gets the right answer for the right reasons. Change the formatting but not the content of a résumé, and the ranking shouldn't move.
3. Generalization. Results hold across prompts, reruns and model versions. A single accuracy number from a single configuration tells you little.
4. Extrapolation. Performance on the evaluation predicts performance on the job, compared with what you do today.
5. Decision. Using it this way improves outcomes at acceptable risk, including adverse impact across groups.

Two threats run through every step. Construct underrepresentation means the evaluation misses part of the job, like a support bot tested only on easy tickets. Construct-irrelevant variance means scores move for reasons unrelated to the job, like an AI grader that rewards longer answers or a screener that responds to résumé polish.

Six questions to ask before you approve any AI deployment

1. Which job does this do, and how will we know it did it better?
2. Does it cover the whole job, or just the easy part?
3. Is it right for the right reasons?
4. Would we get the same answer tomorrow, from a different prompt, after the next model update?
5. Does it predict or improve the outcomes we care about, compared with what we do now?
6. Who could this harm, how would we find out, and what would we do?

If the team can't answer one of these with evidence, that's where your risk is.

Where this leaves you

Score every candidate use case on two halves: opportunity (should we?) and validity readiness (can we prove it and defend it?). High on both, pilot now under a monitoring plan. High opportunity but low readiness is usually where the biggest value sits: build the evidence first, then pilot. And any use case that shapes decisions about people needs live monitoring before launch, whatever else it scores.

None of this is new. Job analysis and validity theory have protected hiring decisions for decades. AI just raised the stakes and the speed. If your organization is funding an AI initiative right now, pick one and run it through the six questions. Our AI Hiring Validation work does exactly that, and we're glad to do the first pass with you in a working session.