Why classic RAG retrieves text instead of knowledge
Classic RAG is a Raw Context Retriever: it retrieves text, not knowledge. The return of knowledge engineering with GenAI, and the move from retrieval to reasoning.
Part 4 of 4. The series starts with From raw data to usable knowledge, and the metadata in between.
The Return of Knowledge Engineering
The term "ontology" has been experiencing a resurgence over the past year. This is no coincidence: it's directly correlated with the rise of GenAI.
The first peak of interest in ontologies was correlated with the big data boom, the second one with the GenAI one.
It reminds me of my university courses, courses I found sometimes boring. Those courses are getting their revenge.
The Failure of Classic RAG
Classic RAG (Retrieval-Augmented Generation) works like this:
- Split documents into chunks
- Vectorize these chunks
- Retrieve chunks similar to the question
- Inject them into the LLM prompt
We inject raw context–pieces of text without structure. It's sufficient for simple factual questions ("What is the refund policy?"), but insufficient for complex reasoning ("Which customers are at risk of churning next month?").
Classic RAG is a Raw Context Retriever. It retrieves text, not knowledge.
From Retrieval to Reasoning
RAG Evolution: From Retrieval to Reasoning
| RAG Today | RAG Tomorrow | |
|---|---|---|
| Full name | Retrieval Augmented Generation | Reasoning Augmented Generation |
| Input | Raw text chunks | Structured knowledge |
| Method | Vector similarity | Vector similarity + Graph traversal |
| Capability | Finding facts | Inferring insights |
Raw context is interesting for facts, but it's even more impactful to be able to reason over existing knowledge in a domain.
This is where Knowledge Graphs and ontologies come into play. They allow agents to Navigate through knowledge (not just retrieve it), Infer non-explicit facts, Reason about relationships between concepts.
The Evidence for Enterprises
It has become obvious that agents need a structured way to understand reasoning processes.
- Investing in the AI layer without investing in metadata = predictable failure
- Output quality is determined by input quality (garbage in, garbage out)
- Knowledge Management is no longer a nice-to-have, it's a prerequisite
The good news: You don't need to do everything at once.
Ontology is not (only) a dusty academic concept. It's the foundation on which tomorrow's agents will be able to reason–not just retrieve text.
Classic RAG has shown its limits. GraphRAG and Knowledge Graph-based approaches point toward the future: systems that understand the structure of knowledge, not just its textual content.
For enterprises, the message is clear: before investing in the latest trendy use cases/tools, invest in your metadata. Document your tables. Define your concepts. Map your relationships.
It's less sexy than a new tool, but it's what will make the difference between an agent that hallucinates and an agent that reasons.
PA,