Continue to read this article if you want to experiment with Graph RAG. We used UV for python package management. Following is our setup.
| Vector DB | Observability Tool | LLM of choice | Embedding Model | Graph DB | Orchestration Framework |
|---|---|---|---|---|---|
|
Vector DB setup: (not using the latest chromadb, we want to experiment with datapipes, hence this version)
|
Open Telemetry: Phoenix (This part is optional, but good to have a dashboard for observability)
Running phoenix server: phoenix serve
Once you have phoenix observability server running and chroma db is sending telemetry data to the server.
You can view the results in the dashboard.
We will use Phoenix as LLM observability tool. We can get llama index to broadcast execution data to Phoenix |
LLM: google_gemma-3-1b-it-Q6_K.llamafile (You can use any llm, but we wanted to use open_ai_like) |
Embedding Model: We used Ollama to serve the model
|
Graph Database (Inmemory/Kuzu)
|
Orchestration Framework: Llamaindex |
+----------------------+
| Input Data Sources |
+---------+------------+
|
+-----------v-----------------+
| Data Ingestion & Processing |
+-----+------------+----------+
| |
| |
+------------------+ +------------------+
| |
+------v-------------------+ +------v-------------------+
| Knowledge Graph Const. | | Text Chunking & Embed. |
| (Extract Entities/Rels) | | (Generate Vectors) |
+------+-------------------+ +------+-------------------+
| |
+------v-------------------+ +------v-------------------+
| Graph Database | | Vector Database |
| (Kuzu) | | (Chroma DB) |
+------+-------------------+ +------+-------------------+
| |
| Structural Retrieval | Semantic Search
| (Multi-hop Traversal) | (Cosine/Euclidean)
+------------------+ +------------------+
| |
+-----v------------v-----+
| Hybrid Query Engine |
| & Retriever |
+------------+-----------+
|
+------------v-----------+
| Context Fusion & LLM |
| (Re-ranking & Synth) |
+------------+-----------+
|
+------------v-----------+
| Generated Response |
+------------------------+
Understanding some basic chunking
When a large text or a large document needs to be avilable for RAG (Retrieval Augmented Generation), we chop them into smaller chunks.
A chunk is a small piece of information. Often, a good vectorization is to optimize chunking, adjusting the sliding window and choosing the right kind of embedding model.
We will deep dive into this topic in subsequent practice sessions.
One good library that provides multiple splitters (chunkers) is langchain. Get it installed using uv.
We are using langchain-text-splitters == 1.1.2
Here is a very simple example to clearly see how it works if we used a token splitter.
from langchain_text_splitters import TokenTextSplitter
text_splitter = TokenTextSplitter(
chunk_size=20, # tokens per each chunk
chunk_overlap=5 # shared tokens in each chunk
)
raw_text = ("Choose any paragraph of your choice")
chunks = text_splitter.create_documents([raw_text]) # you may want to check the size by using len(chunks)
for i, chunk in enumerate(chunks):
print(f"\n--- Chunk {i+1} ---")
print(chunk.page_content)
Going little further, trying to understand how search would work on this
import json
from langchain_text_splitters import RecursiveJsonSplitter
# 1. Define your tabular data as a list of row dictionaries
tabular_data = [
{"Employee ID": 101, "Name": "Adi", "Department": "Engineering"},
{"Employee ID": 102, "Name": "Bhimsen", "Department": "Marketing"},
{"Employee ID": 103, "Name": "Chota", "Department": "Sales"},
{"Employee ID": 104, "Name": "No Name"},
]
# 2. We will use a RecursiveJsonSplitter
splitter = RecursiveJsonSplitter(max_chunk_size=100)
# 3. Split the data into LangChain Document objects
# This formats the JSON and converts it to search-ready text chunks
documents = splitter.create_documents(texts=[tabular_data], convert_lists=True)
# 4. Like prevous example, we can check the chunks that are created.
text_chunks = []
for i, doc in enumerate(documents):
print(f"--- Chunk {i+1} ---")
print(doc.page_content)
text_chunks.append(doc.page_content)
# 5. #now we will do some search operations on the chunks
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("all-MiniLM-L6-v2") # you may choose other embedding models
corpus_embeddings = model.encode(text_chunks, convert_to_tensor=True)
# 6. Now you can truly understand, how thise works when we do a search
# Calculate similarity strictly in-memory via tensor dot-product. There are two questions.
# Play with the top_k value for finding the matches
query_embedding = model.encode("who works in engineering deparment?", convert_to_tensor=True)
#query_embedding = model.encode("who do not work in marketing department?", convert_to_tensor=True)
#hits = util.semantic_search(query_embedding, corpus_embeddings, top_k=1)
hits = util.semantic_search(query_embedding, corpus_embeddings, top_k=5)
# 7. Check the results
for rank, hit in enumerate(hits[0]):
chunk_index = hit['corpus_id']
score = hit['score']
print(f"Rank {rank+1} (Score: {score:.4f}): {text_chunks[chunk_index]}")