You may definitely install Eclipse on your raspberry pi (
sudo apt-get install eclipse), then you need to download and install Android SDK from Google.
There should not be any problem with Java libraries, there's a version of JDK for ARM, but you may need to recompile some of the required ADT tools for ARM architecture (source code is available from Google). Besides that, you may try to install Android Development Tools (ADT) Eclipse plug-ins directly from Eclipse.
If you don't want to use Eclipse, there are Apache ANT scripts that build android applications from the source, you'll need only Android SDK and a copy of ADT program to download the resulting .apk file to your device over USB. You won't be able to use emulator (I don't know if there's an ARM version), but working with the real Android device over USB connection might work.
All in all, it's not a very easy path to follow, and the development experience is not going to be very pleasant (compared to the development on PC), but still seems possible.