Two Jev passes helped our Graph RAG pipeline outperform the LLM rerankers we tested, with higher rec...
Milvus(@milvusio) · 人工智能
Two Jev passes helped our Graph RAG pipeline outperform the LLM rerankers we tested, with higher recall and less time spent on model calls. Earlier, we released 𝗩𝗲𝗰𝘁𝗼𝗿 𝗚𝗿𝗮𝗽𝗵 𝗥𝗔𝗚, an open-source project that uses Milvus for multi-hop retrieval without a separate graph database. During retrieval, an LLM filters the relationships found through graph expansion. That LLM step could involve hundreds of candidates, slowing retrieval. To address this, we tested two Jev-based alternatives and compared both with the original setup. 1️⃣ 𝗦𝗶𝗻𝗴𝗹𝗲-𝘀𝘁𝗮𝗴𝗲 𝗝𝗲𝘃 We replaced the LLM with Jev to filter the relationships. Filtering got faster, but recall still lagged behind our stronger LLM baseline on complex multi-hop questions. A selected relationship can still lead to unhelpful text. A relationship about metformin treating diabetes could come from a drug history article that says little about side effects. 2️⃣ 𝗧𝘄𝗼-𝘀𝘁𝗮𝗴𝗲 𝗝𝗲𝘃 • 𝗦𝘁𝗮𝗴𝗲 𝟭: Jev selects graph relationships. Milvus retrieve their source passages, add results from direct vector search, and remove duplicates. • 𝗦𝘁𝗮𝗴𝗲 𝟮: Jev ranks those passages by how useful they are for answering the question. Across 2,000 questions from MuSiQue and 2Wiki, two-stage Jev 1.13 improved average Recall @5 from 80.04% to 86.07% over single-stage Jev, adding 0.85 seconds of model-call time. Jev 1.14 reached 86.61%. Check out the repo to see how we pushed Vector Graph github.com/zilliztech/vec… ier of speed and quality! 👇 https://t.co/dfYjgUmTfz 💬 0 🔄 1 ❤️ 0 👀 20 ⚡ Powered by xgo.ing