Modern embedded systems are usually implemented by integrated circuits that contain commercial microprocessor cores with internal instruction and data caches. Such systems are powerful and cost-effective, but the performance of the microprocessor cores is often limited by the performance of these internal caches. This limitation comes from the small size of the caches, the long latencies required to service a cache miss, and the high degree of cache misses that occur, especially for the instruction cache. However, the CPU cores are pre-designed, and the caches can be customized to only a limited extent, so the designers of these systems are limited in their ability to improve the cache performance. Fortunately, the software running on these systems is largely fixed and dedicated to a single application, which provides some opportunities to optimize the performance.
Experimentation with real industrial designs and software has shown that instruction cache misses often occur as a cluster of misses; each cluster results in a large amount of CPU stalling that degrades the CPU performance as the misses are serviced. Contributing to this behavior are effects of real-time processing, multi-tasking operations, and interrupts that are common to embedded systems.
In this dissertation, several techniques, involving both hardware and software enhancements, have been developed specifically to address the unique instruction cache miss problems that occur in real-time embedded systems. These techniques were tested on real industrial design and software sets, and were shown to be practical and cost effective solutions to these problems.