AI That Works vs AI That Matters: Why Some AI Investments Don’t Pay Off (Yet)

Only 14% of Australian CEOs say they've seen revenue gains from AI so far, according to PwC's latest Global CEO Survey, less than half the global average of 30%. MIT's GenAI Divide research went further, tracking 300 real deployments and finding 95% have yet to see tangible returns.

Australian and New Zealand businesses today are pressured to keep pace, to capture the productivity upgrade. That's exactly why Agentforce adoption across the Salesforce customer base is moving this fast, with over 20,000 enterprise customers now running it in production. 

But moving fast leaves less room to stop and ask the harder questions early, and that's exactly where the gap in making sure AI investments actually pay off hides.

Two Different Scorecards 

“AI that works" and "AI that matters" get treated as the same thing, and they're not. 

“AI that works” means the agent does the job it was built for: it resolves the ticket, books the appointment, answers the question. But “mattering” to the business means someone can point to a number that moved because of it, a customer who stayed, a call that didn't come back, revenue that showed up. 

Most AI investments right now prove the first one and assume the second follows. But that’s not always a given.

At ProQuest, we've seen this exact pattern across 15 years of consulting: teams prove the technical output works, and just assume the business outcome follows.  AI is just moving faster than anything before it, which makes that gap easier to miss and more expensive when discovered later.

Three places that gap hides

In practice, it breaks down in one of 3 places:

• Data readiness: Before a single agent gets built, there's a question most companies never ask honestly enough: is the data this agent would actually pull from clean enough to trust? 
  
  Case notes scattered across old systems, inconsistent tagging, years of shortcuts and workarounds, that's the norm, not the exception. An agent trained on messy data doesn't fail loudly. It gives confidently wrong answers, and nobody notices until a customer does. 
  
  This is exactly where the right partner earns their place, not just in building the agent itself, but in the far less glamorous work underneath it. Untangling what's been duct-taped together over years, cleaning up what needs cleaning, and being honest early about how much of that has to happen before an agent should go anywhere near production.
  
• Data readiness: Before a single agent gets built, there's a question most companies never ask honestly enough: is the data this agent would actually pull from clean enough to trust? 
  
  Case notes scattered across old systems, inconsistent tagging, years of shortcuts and workarounds, that's the norm, not the exception. An agent trained on messy data doesn't fail loudly. It gives confidently wrong answers, and nobody notices until a customer does. 
  
  This is exactly where the right partner earns their place, not just in building the agent itself, but in the far less glamorous work underneath it. Untangling what's been duct-taped together over years, cleaning up what needs cleaning, and being honest early about how much of that has to happen before an agent should go anywhere near production.
  
• Small win, bigger picture: Most companies start their Agentforce journey looking inward. Faster case resolution, fewer manual tasks, lower cost to serve. That's not the wrong place to start. But it's an easy place to stop. Freed-up time doesn't automatically translate to a better customer experience; it only does if someone deliberately points it there.
  
  Before your next use case gets approved, ask the harder question: are we measuring the use case win, or the bigger effect it's meant to create? Less time on repetitive tasks is nice for the person doing it, but on its own, that's not why you invest. 
  
  Say a field service team starts using AI for pre-work briefs and faster job logging. This cuts down time on the field, but ultimately, businesses want to see that turn into more jobs closed, more cases handled.
  
  If nobody can answer that on the spot, that's not a reason to stop the project. It's a reason to define the answer with your team before it goes further, because that's exactly the number you'll need when you're standing in front of the CFO asking for the next phase of budget.

What we've learned 

Most companies plan an AI rollout like it's a single project: map the whole thing out upfront, then execute it exactly as scoped. But reality rarely cooperates. Data turns out messier than expected, adoption takes longer, scope shifts, and a rigid plan just breaks under all that instead of adapting to it. 

What does work: deciding what "worth it" looks like before the first agent goes live, then building the first phase specifically to prove that. Once it proves itself, faster resolution times customers actually notice, fewer repeat contacts, a support team with room to do better work, that phase funds the next one. This is the same discipline as the 3 gaps above, just applied instead of just diagnosed. This does three things at once.

• It builds real confidence in execution. Teams don't have to take a leap of faith on the whole rollout at once, they get to see it actually working before being asked to trust it further. That's what speeds up adoption in practice, not a mandate from leadership, but proof staff can see with their own eyes. 
• Customers also feel the value, not just the internal team. You're building backwards from what a customer will actually feel, not forwards from what the platform can technically do. That's the difference between an agent nobody outside the business notices and one that shows up in retention numbers.
• It keeps the approach flexible instead of templated. No two rollouts start from the same data, team, or goals, so a partner worth having isn't handing you a fixed playbook and hoping it fits. They're working from a proper framework, then adjusting it as your actual circumstances demand, not as a case study said they should. That's less a formula than an ongoing judgement call, and it's exactly why the first phase matters so much: it's where that framework gets tested against your reality, not someone else's. 

A business that has one genuinely good phase with a partner trusts them with the next one easily. One that gets a rushed, over-promised first phase rarely gives that partner a second chance, no matter how good the pitch sounds for phase 2. 

Stop asking whether your AI investment will pay off. Start asking what "paying off" actually looks like, specifically, for your business. If nobody in the room can answer that yet, you'll struggle to defend the spend when someone finally asks.