Why does the RPi need a 'special' kind of Linux ? Why can't it run any distribution ?
You can run pretty much any distribution as long as it's adopted to RPI hardware. Linux adoptions to ARM architecture seem to often be a separate activity alongside their mainstream development and so is adoption to RPI. Not all distributives are announced on RPI official web site, like Slackware RPI for example.
The main issue is the ARM architecture and the userspace applications.
The Linux kernel has good support for ARM, so it won't be an issue.
But not all distributions build their applications (or packages) against ARM (or more specifically against rpi), just like a few years ago when x86_64 is not so popular and some of the distributions only support the 32bit x86.
It's not a easy task for distributions to verify their thousands of packages against a new architecture. Especially in this case where it involves two entirely different ones, x86 vs ARM. Since most of the desktop applications we use are wrote on x86, they will possibly have problems when ported to an ARM machine (which is usually the turf of the "embedded" linux distributions).
The RPi is a phone that is designed to run Android. (except you don't have battery, screen, keyboard, nor phone;)
This means that although the OS kernel is linux, the OS supported by the chip maker is not GNU/Linux.
Furthermore, the firmware is closed source, and you can't do anything other than booting
kernel.img from the 1st partition on the SD card, formated as vfat.
You can't boot from another device, so you can't bootstrap a base system on the SD easily. That's why you get a pre-installed disk image.
Apart from that, Raspbian is just a Debian Wheezy, with a package repository that is synced with the official repos, with a bunch of hardware-related programs, packaged for debian or not; notably the linux kernel and the videocore firmware are not in the database, and you get an /opt/ directory with some WIP programs that are modified to use whatever hardware specific functions that are not yet polished/upstreamed.
As the device drivers are being pushed to vanillia kernel, the differences will drop, hopefully leading to a standard distro with a hacky installation method, where hacky means: this CPU/GPU combo is meant to make phones that are sold with Android already on it, we won't help you porting syslinux and debian-installer (and as Wheezy is freezed to get a realease soon, the debian developers won't dive into that for now).