A few years back, the only people who could train computers to make predictions or recognize patterns were those who had spent years immersed in data science. We are now in the age of generative AI, and ChatGPT is the most representative tool.Â
When generative AI appeared, it promised to provide access to powerful tools with nothing more than a few well-chosen words. Nonetheless, as the tool was democratized, it brought confusion about what these models actually do and, crucially, what they do not. Â
Understanding LLMs with Dr. Harish Madhavpudi:
Dr. Harish Madhavpudi, Assistant Professor of Artificial Intelligence at the University of Bath, stands at the intersection of the AI transformation. In his work, he explores the structure and capabilities of large language models (LLMs).Â
His research questions are not aimed at understanding what LLMs are trying to do, but rather what they are capable of doing. His research is built around the growing assumption that LLMs can reason, plan, and may even innovate. As he expresses it, “we may be a little premature in being afraid of this eventuality.”Â
Why prompting is not reasoning:
What Dr. Madhavpudi and his team have found based on the behavior of models like GPT-3.5 and GPT-4.0 is that these tools are powerful, but in very specific ways.Â
They are not thinking systems. These types of tools are merely probability systems. ChatGPT can interpret a prompt and draw from patterns found in its vast training data, which primarily reflects public web content, to predict what should come next.Â
When prompts fail, hallucinations follow:
Ask the right question, and you will get something useful. Ask a vague question or assume too much, and the AI LLMs will offer answers that sound right, but unfortunately, they are not. This outcome is often called “hallucinations.”Â
Dr. Madhavpudi’s main observation is that the way users interact with these LLMs is part of the problem. Most users are unaware that prompting an LLM is not the same as asking a question to a person.Â
The best approach is to envision that every time a user types, they are constructing a miniature training set. “If we assume the system will infer intent or fill in gaps like another human would, we are setting ourselves up for failure,” he explains. The model, in essence, does not reason. It extrapolates.
Tools for evaluation, not innovation:
Of course, the expectation is that these LLMs might one day help people in the most underserved parts of the world. For instance, they can offer affordable tutoring or private health education.Â
Dr. Madhavpudi seems optimistic about these expectations, but he firmly believes that we are not quite there as yet. “Scaling up these models does not change the fundamental mechanism. They’re still doing what they were designed to do. [In other words] generate plausible continuations based on their data.” Â
Although the current mechanism is robust, it still has clear limitations. For instance, we undertook a project with a global advertising agency to investigate whether LLMs could generate compelling content to develop novel campaign ideas. We found that in an underwhelming manner, “these were creatives with decades of experience. The models couldn’t match that,” Dr. Madjavpudi says.Â
At best, the LLMs could produce variations of things that had already been done. Yet true originality still remains within human territory. So, how should we use these powerful generative AI tools? No to invent but to evaluate.
Where LLMs can add value:
Dr. Madjavpudi thinks that LLMs shine when used as filters. They can help to rank, refine, and reframe ideas generated by people. “They cannot create the next big idea, but they can help us test whether that idea resonates.”
This distinction becomes especially important in high-stakes domains. For instance, in medicine, diagnostics, and biotechnology, there are cases in which these domains require the discovery of molecules or methods. Can LLMs help there?
Possibly, but with limits. Dr. Madjavpudi explains that the key is in providing carefully curated in-context examples. This is not just asking the model to invent, but guiding it with structured prompts that include the relevant scientific details, past innovations, and target outcomes. Then, within that constraint, the model may support extensions or modifications.Â
“Instead of relying on what the model absorbed from the web, we constrain it to just the prompt, just the examples we feed it, and then ask it to generalize.”
The process does not replace innovation. It basically scaffolds it.
Where these systems fail most acutely, Dr. Madjavpudi notes, is in the kind of cognitive leap that has defined human progress. For instance, seeing planetary motion differently, redefining species, or inventing representative democracy. “Those are not things that result as a consequence of extrapolation; those are shifts,” he says. “And that’s what humans do.”Â
There are still practical ways to use LLMs in support of human creativity. One of the best aspects is that LLMs can help us navigate the space of what has already been tried. In essence, LLMs can help identify what has been done before, reveal patterns across past efforts, suggest overlooked combinations of existing ideas, and rank options on what is most likely to succeed.
This is where the future of generative AI tools might lie, not as generators of genius but as amplifiers of judgment.
When asked what advice he would give to companies trying to use LLMs for innovation, Dr. Madjavpudi’s message is clear: “Know that they’re not reasoning. They’re not doing novel systematic reasoning either. They’re generalizing from the data they were trained on. That’s incredibly useful, but it has boundaries.”
Instead of treating LLMs as idea engines, Dr. Madjavpudi recommends using them as co-pilots in the truest sense of the term. The models will not direct innovation. They will only support it.
LLMs without the cloud dependency:
Dr. Madjavpudi also emphasizes that LLMs do not need to reside in the cloud, owned by distant providers. With advances in edge computing and open-source modeling, many organizations can run these tools locally. Additionally, it is possible to safeguard intellectual property while still benefiting from the cognitive scaffolding it provides.
The takeaway: AI supports, but does not truly think
Large language models, such as ChatGPT, can support structured thinking but cannot replicate human conceptual breakthroughs — this is the key divide in AI vs human intelligence. LLMs excel at pattern matching and idea evaluation when guided with clear, bounded examples. True shifts in understanding still require the mental leaps that only humans can make.
