Most people's vector database defaults to keeping all data in memory. 𝗧𝗵𝗮𝘁 𝗺𝗶𝗴𝗵𝘁 𝗯𝗲 ...

Two ways to stop paying for idle data (both config changes in https://t.co/0Ad3zYktCi" / X

Post

Conversation

Most people's vector database defaults to keeping all data in memory. 𝗧𝗵𝗮𝘁 𝗺𝗶𝗴𝗵𝘁 𝗯𝗲 𝗰𝗼𝘀𝘁𝗶𝗻𝗴 𝘆𝗼𝘂 𝟯-𝟭𝟬𝘅 𝗺𝗼𝗿𝗲 𝘁𝗵𝗮𝗻 𝗶𝘁 𝗻𝗲𝗲𝗱𝘀 𝘁𝗼 (once you scale to production with 100M+ vectors). Two ways to stop paying for idle data (both config changes in Milvus, not redesigns): • 𝗠𝗠𝗮𝗽 (v2.3+):data on local disk, loaded on demand. Stable latency. Disk must hold the full dataset. • 𝗧𝗶𝗲𝗿𝗲𝗱 𝘀𝘁𝗼𝗿𝗮𝗴𝗲 (v2.6+): hot data cached locally, cold data in S3. Cache misses add 50-200ms. Both need NVMe SSDs (10K+ IOPS). 𝗢𝗻 𝟭𝟬𝟬𝗠 𝘃𝗲𝗰𝘁𝗼𝗿𝘀 (𝟳𝟲𝟴-𝗱𝗶𝗺, 𝗳𝗹𝗼𝗮𝘁𝟯𝟮): 𝗣𝗶𝗰𝗸 𝗠𝗠𝗮𝗽 𝗶𝗳: • P99 < 20ms — data is local, no network fetch, no surprise spikes. ~77-230 GB memory (vs 768 GB default). • Uniform access — tiered storage's cache doesn't help if everything gets hit equally 𝗣𝗶𝗰𝗸 𝘁𝗶𝗲𝗿𝗲𝗱 𝘀𝘁𝗼𝗿𝗮𝗴𝗲 𝗶𝗳: • Cost is the priority — <77 GB memory (vs 768 GB default). Saves on both memory and disk (70-90% less) • Clear 80/20 access pattern — hot data cached, cold data stays cheap in S3 • 500M+ vectors — one node's disk can't hold it all Full config walkthrough !Image 1: 👇milvus.io/blog/how-to-cu

The media could not be played.