Try this alternative before switching to a faster SD card and/or overclocking yor pi:
- download, compile and install vmtouch
- add this line in
/opt/retropie/configs/all/autostart.sh:
sudo vmtouch -dl /opt/retropie/emulators/retroarch/bin/retroarch /opt/retropie/supplementary/emulationstation/emulationstation
(check the paths are correct in your system)
The idea is to keep the main executables of retroarch and emulationstation in main memory so switching betweeen them will be faster (no need to load the file back from the SD). You may also cache other files and emulators cores you use often.
TIP: keep the bigger files in main memory (e.g.: /opt/retropie/libretrocores/lr-genesis-plus-gx/genesis_plus_gx_libretro.so), avoid loading small files used seldom.