Well, it's not safe if any filesystem present on SD card is mounted (especially if it's root filesystem that is mounted). This is true by default in most distributions currently available for RaspberryPi. But it is not strictly needed. You can run your system from RAM (either by copying filesystem image to
tmpfs mounted filesystem or by using
initramfs) or some external harddrive or even from network. If you do so, your SD card can be swapped without problems (just ensure to unmount all filesystems mounted on it).
The only problem is that RaspberryPi has small amount of RAM so you plan on running it this way, you should have really small image (compressed filesystems like SquashFS may be handy here), memory split that gives ARM most RAM and probably 512MB version of RaspberryPi. But that's definitely possible.