Technical Talk on "Memory Hierarchy and Caches: Why Memory Decides System Performance"

#architecture #system-performance #ieeetechnicaltalks

Modern processors execute billions of operations per second, yet performance is often limited by how fast data can be supplied, not how fast it can be computed. This session will examine that gap, the memory wall, and the architecture that bridges it. It covers the memory hierarchy and locality, SRAM and DRAM, building larger memories from chips, interleaving and the memory address map, and cache design: address mapping, cache types, replacement and write policies, and the three types of misses. Performance is quantified with Average Memory Access Time (AMAT), including multi-level caches. A live demonstration shows how loop order alone changes execution time. Real-world examples (LLM KV cache and prefix caching, browser, OS and database caches) and an anonymised edge AI case study connect the concepts to modern systems.