I always read that swap on a Pi should be disabled because enabling it lowers SD card lifetime and is very slow anyways (no performance gain with swap). So I disabled it.
But is there a good reason why Raspbian has enabled swap by default anyways?
IMHO, the swap was enabled by default because without the swap when you run out of the memory you'll get a kernel panic and/or the application killed, that might be avoided using swap. With the swap actually active, the performance might be really bad, but the execution still continues. Since nobody could possibly know in advance how the particular RasPi would be used, enabling swap is the safest choice.
Once you figured out how much memory you need and have changed CPU/GPU memory allocation, you might disable swap.
You proabably want to use ext2 rather then ext3 or ext4 for the file system on a flash device to reduce writes (to file system logs)