If you had to “interview” AI before “hiring” it, what would your screening process look like?
Many companies feel pressure to adopt the latest AI advancements. Vendors push new models, claim breakthroughs, and highlight impressive benchmark scores, fueling the idea that AI should be evaluated by how “intelligent” it is. But too often, businesses chase the newest AI rather than designing systems that make AI work in practice.
The problem with intelligence is that it’s subjective. What matters isn’t how “intelligent” an AI model seems but how well it actually fits into your business.
We don’t hire employees based on intelligence alone. We evaluate how well they solve problems, how they collaborate, and their ability to adapt to new challenges. Likewise, the best AI isn’t the “smartest” model; it’s the one that fits your workflows and improves decision-making. A successful AI initiative depends on how well it integrates into existing systems, including data infrastructure, workflows, governance, and human decision-making.
Systems thinking is the skill that helps you evaluate, implement, and oversee AI effectively. It prepares you to ask the right questions to determine where AI fits, identify what gaps still need human oversight, and recognize what changes your business must make to get real value from AI.
Systems Thinking Makes AI Models Useful.
AI systems are dynamic, interacting with workflows, data, and human operators in ways that, if poorly designed, create bottlenecks, erode trust, or disrupt operations. Benchmarks, while useful indicators of a model’s potential performance on a variety of tasks, can end up misleading businesses. A high-scoring AI system that isn’t well integrated into production can create more problems than it solves.
A client of ours attempted to automate transcript reviews, aiming for 90–95% accuracy. But accuracy wasn’t the real issue. What they needed was a structured workflow that flagged transcripts requiring human review, automated the predictable cases, scored AI confidence for triage, and built in regular audits. The problem wasn’t whether the model hit a performance threshold. It was whether the workflow itself was designed to make AI effective.
Consider an AI agent assisting customers shopping for furniture. It can answer questions and recommend products, but without live inventory data or a seamless handoff to a sales associate, the customer journey stalls. Systems thinking helps organizations avoid these pitfalls by defining where human interactions add value and where automation falls short.
Mercedes-Benz built the MO360 Data Platform to embed AI into production workflows while keeping humans in control. AI optimizes scheduling, predicts bottlenecks, and analyzes operational data, but operators make the final decisions and adjust workflows as needed. By integrating AI into a structured system that enhances human decision-making, Mercedes-Benz increased transparency, improved efficiency, and positioned its plants for a 20% productivity boost in 2025.
Mars Wrigley embedded AI into its supply chain to improve decision-making, tracking not just what AI recommended, but whether and how humans acted on it. AI suggested inventory rebalancing strategies to optimize distribution center shipments based on cost and service levels, but final execution remained with supply chain planners. Over time, by feeding execution data back into the system, Mars refined its models, reducing unnecessary stock movements and improving short-term planning. This system led to a 95% truck utilization rate, improved customer service levels, and lower transportation costs.
Designing Effective Systems Around AI
AI isn’t a single system that either works or fails. It’s a collection of interconnected components that must integrate across technology, workflows, and personnel. When evaluating an AI solution, consider these five critical elements of system design:
1. The Decision Layer (How AI Outputs Are Used in the Business)
An AI system is only as useful as the actions it enables. Insights mean nothing if they aren’t acted on, and automation fails if it doesn’t fit into existing workflows. AI must integrate into decision-making processes. If employees don’t trust or understand AI-driven recommendations, they will ignore them or find ways to work around them.
- Who is responsible for acting on AI-generated recommendations?
- What expertise do they have and what information do they need to make effective decisions?
- How are employees rewarded or compensated based on the outcomes of their decisions?
- Can employees override AI-driven decisions, and under what circumstances? What does that control look like?
- How does AI fit into existing workflows, and what needs to change for it to be effective?
2. The AI Engine (What the Model Actually Does)
Most AI solutions are not a single model but a system of models working together. One model may determine user intent, while another generates responses or takes action. You need to know when the system will perform well, what it requires to perform well, and how to detect and manage errors.
- What outputs does the model provide and how does that impact decision making?
- What elements of the model can be tuned and how are controls optimized?
- Where has this model been tested, and what conditions affect its performance?
- How does it detect and handle uncertainty?
3. The Input Layer (How AI Receives & Processes Information)
AI is only as good as the data behind it. AI systems depend on data in three key contexts: the data they were trained on, the data they rely on to operate, and the data needed to improve performance over time. Businesses adopting AI tools must assess whether the system will have access to the right information at each stage.
- What data sources fuel the AI, and how often are they updated?
- What historical data was used to train the model, and how does that impact its performance?
- What additional data is required to improve accuracy and adaptability?
- How does the system detect and handle incomplete, conflicting, or outdated data?
- Who is responsible for monitoring and maintaining data quality?
4. The Control Layer (Governance, Risk & Oversight)
AI requires continuous monitoring, human oversight, and clear accountability. Failures happen when organizations aren’t able to detect when models are degrading. AI governance is critical for quality assurance.
- What happens when AI gets something wrong?
- Who is responsible for reviewing AI-driven decisions over time? At what time scale?
- How does the system prevent security threats, compliance violations, or biased outcomes?
- How will future iterations of models be evaluated and integrated into production?
5. The Business Optimization Layer (Measuring Continuous Value & ROI)
A model can score high on statistical tests but still fail to deliver business impact. To avoid becoming just another cost center, AI must improve efficiency, reduce risk, or increase profitability in ways that justify its total cost of ownership.
- Cost Avoidance: How does AI reduce operational inefficiencies, mitigate risks, or prevent costly errors?
- Value Acceleration: Is AI shortening time-to-market, improving productivity, or increasing user adoption?
- Revenue Growth: How does AI contribute to new revenue streams, higher customer retention, or improved customer experiences?
- Sustainability & Maintenance: What are the long-term costs of maintaining, retraining, and governing the system? How do you ensure continuous improvement?
The Best AI Fits Your Business
AI transformation should be more about how well AI integrates into your systems than chasing the latest models and tools.
Toyota became a force in the auto industry through Kaizen, the practice of small, continuous improvements that compounded into long-term advantages. Businesses that succeed with AI do the same. Mars Wrigley and Mercedes-Benz built systems where AI supports human decision-making and evolves with business needs.
The companies that get AI right recognize that it operates within a larger business system with interconnected components that require oversight, adaptation, and maintenance. They shape AI to fit their processes, governance, and infrastructure. At the same time, they challenge business systems that need to evolve to get the most out of AI.
Before getting too excited about a novel solution, consider if you were to “hire” it today, would it make your business stronger? Are your systems the right environment to ensure success?
