Thread

    Nate
    Nate@nate_512

    What surprised me most about AtlasKV isn't the headline number, it's the memory layout. A billion-triple knowledge graph running on an LLM with under 20GB of GPU VRAM, and the majority of that graph lives in CPU memory the whole time. Only the slices that are needed get moved over to the GPU, and yeah, shuffling data across that boundary still adds latency. The rest of the design is what gets me though. There's no separate vector stack sitting off to the side, so Pinecone, Weaviate and Qdrant can sit this one out. No external retriever module either. And nothing gets crammed into a giant prompt as a wall of retrieved docs, because the structured data goes right into the model's attention instead. That's a different angle on the whole give AI more knowledge problem. Two components carry it. KG2KV takes graph facts and turns them into key-value representations. HiKVP handles the knowledge hierarchy by filtering it progressively so the GPU doesn't get buried. And after the initial setup, fresh knowledge can be added without going through another round of training. For years the grind has been chunk sizes, retrievers and prompts, so watching that workflow get rethought from the ground up is genuinely exciting.

    Original post

    You guys are making me curious about the latency

    ago

    0 Likes0 Dislikes1 Replies
    ?

    No replies yet