ProjectsBlogsResumeWork
Home/Blogs/Beyond Vector Search: Understanding Vectorless RAG
Beyond Vector Search: Understanding Vectorless RAG
VECTORLESS RAGGEN AIRETRIVAL

Beyond Vector Search: Understanding Vectorless RAG

How PageIndex, LLM-generated Wikis, and hybrid retrieval are changing the way AI systems find knowledge.

September 28, 2026·

Vector Database Is Only One Approach to Retrieval

  1. The input text is taken.
  2. The text is then converted into vector embeddings.
  3. A similarity search is performed between the input vector and the vector store.
  4. The output data is all the similar data.

This is a mathematical method where the context of the data is not present because of abrupt chunking, causing the background of the data to be lost.

Vectorless RAG

In Vectorless RAG, there is a straightforward approach of creating an index for each chunk of data so that it becomes easier to retrieve.

Keeping an index also gives context to the data chunks, making them more useful during retrieval.

This index works similarly to the index in a document, which provides information about the background and structure of the content.

In PageIndex, a tree-type data structure is formed while indexing the document so that retrieval becomes more intelligent.

It eliminates the other nodes of the tree where there is no relevant data.

Go through this PageIndex to understand how it works.

Vector less rag
Vector less rag

Going a step further, Andrej Karpathy came up with LLM Wiki.

LLM Wiki

LLM Wiki comes up with a different idea. Instead of performing a similarity search on vector embeddings, we build the wiki from scratch in the form of .md files.

These .md files are structured files that contain information such as the title, description, and complete content.

Whenever a new source or file is added to the wiki, the entire wiki gets rewired. It checks the present summary of each .md file and edits it with the required changes, removes unwanted information, and updates the existing summaries.

Go through this LLM Wiki to understand how it works.

Knowledge Graph
Knowledge Graph

Vector vs Vectorless

The retrieval process in both systems is very different.

  • Vector RAG works on similarity search, retrieving the relevant data irrespective of the context, which it is not aware of.
  • Vectorless RAG works by eliminating irrelevant indexes and making retrieval more efficient with context-aware data.
  • Practically, vector RAG works for a lot of systems.
  • Practically, vectorless RAG doesn't work in every situation due to its higher cost.
  • Cost is the biggest flaw in vectorless RAG due to the higher LLM involvement in generating the knowledge base.
Vector vs Vector less rag
Vector vs Vector less rag

Going Through All These, There Comes a Doubt: Which Is Universally Better?

The answer always depends on the problem statement you are trying to solve.

If cost is never a concern, vectorless always comes ahead. But there can be situations where a vector database is already solving the problem, making vectorless retrieval an over-implementation.

Hybrid Retrieval

  • This is a combination of both vector and vectorless retrieval.
  • This will only be useful in bigger RAG systems, which might contain some thousand documents for indexing or chunking and retrieving information from them.

For example, nearly 1,000 documents are present. We can perform a vector search first and then implement vectorless retrieval on the retrieved results to get more context about the input data.

Correct Retrieval Strategy to Select

  • Vector RAG — used when the search is needed word-to-word.
  • Vectorless RAG — used when the data has a strong structure, like books or indexed documents, and context-aware search is required.
Back to all posts