What AI cannot see in your data

Essay

Imagine asking an AI system to help a company find its next product idea. The system is given the company’s previous successes, customer research and design archive. It produces a large set of plausible concepts. Most are recognisable variations on what the company already knows.

That result would not necessarily mean the model had failed. It may have done exactly what the brief and the available material allowed.

A system cannot recover a perspective that was excluded from the data, the question and the people judging the answer.

The field of view is set before the prompt

AI systems work with patterns available in their training, context and instructions. In an organisation, those inputs carry earlier choices: which customers were studied, which ideas were funded, whose work was documented and what the business already treats as valuable.

This is usually discussed as a fairness problem, and often it is. It is also a problem for strategy and innovation. If the available material describes only familiar markets, familiar users and familiar definitions of success, the system has less to work with when asked for something genuinely different.

More data does not automatically solve this. A larger collection can reproduce the same omissions at greater scale.

Where perspectives disappear

In the source material

Historical data reflects what an organisation chose to record. Customers who were never served, ideas that were never funded and forms of knowledge that were not captured will be hard to recover later.

In the question

A narrow brief can turn the existing business model into an invisible boundary. “What should we make next for our current customer?” is a different question from “Whose problem are we not seeing?” The model cannot correct a frame nobody has challenged.

In the review

People evaluate outputs through their own experience and incentives. An unfamiliar idea may be rejected as irrelevant when it actually points to a market, need or risk the current team does not understand.

Diversity is not a guarantee

Broader representation does not guarantee a fair decision or an original idea. Data quality, model behaviour, problem framing and organisational judgement still matter. A varied dataset can be used badly, and a diverse review group can still operate inside a narrow brief.

The practical claim is smaller: when relevant perspectives are absent, the organisation should not assume the system will invent them. Leaders need to decide where those perspectives will enter the work.

Widen the work around the model

Before using AI to explore a consequential question, ask:

• Whose experience is well represented in the material?

• Which customers, markets or forms of knowledge are missing?

• Who framed the question, and what did that frame rule out?

• Who will recognise a useful answer that does not look familiar?

Then change the process, not only the dataset. Bring people with different knowledge into the framing and review. Treat unusual outputs as questions to examine rather than novelty to celebrate or noise to remove. Record why an option was rejected so the organisation can see whether the same assumptions keep closing the field.

Use AI to extend judgement, not recycle it

The aim is not to demand that a model produce originality on command. It is to stop the organisation from mistaking fluent repetition for a wider view.

AI can help combine material, expose patterns and generate alternatives. Leadership still decides what enters the system, what counts as a useful possibility and whose perspective is needed before acting.

If the organisation wants different answers, it may need different evidence, different questions and different people in the room.