Prefill
KV cache
Video by Vizuara on how to speed up reply using caching and prefill
💡 Prefill saves time. Fills up the KV cache.
💡 If the system prompt is the same, just use the one that is prefilled in the KV cache
This will save response time