What you need is a power amplifier.
8 ohm speaker is essentially a short-circuit for most applications. By connecting it straight to a device that's not a power amplifier you're risking a damage both to the output stage of the driver (since it gets shorted and the power is used on it's internals, not the driven device) and to the speaker itself (if the driver can supply more current than the speaker can bear).
So yes, your calculations are roughly correct, you'd be trying to fry your Pi by connecting it directly. If your Pi has a current limiter (I'm no expert on Pi, mind me, but I'd assume it doesn't), you will just drive your Pi on 100% power output, so it's still a bad idea.
As to connecting it - you can either connect a speaker with a higher resistance (but that will decrease both the current and the power output of the speaker, making it quite quiet), or use a simple transistor to drive the speaker. Googling "transistor amplifier" can give you some ideas how to do it, although the simplest approach is along the lines:
or
Note that if you're feeling lucky, for testing purposes you can manage with a single transistor and two resistors: put one resistor between GPIO & transistor base to limit base current, for 3.3V & 10mA max on GPIO it'd be around 330 ohm ; plug emitter to ground ; plug your speaker in series with another resistor between Vcc & collector to limit speaker current. 0.1W / 8 ohm speaker can manage about 100mA, so a 33 ohm one should do the trick here.
It usually works nice while breadboarding, YMMV, but it's definitely NOT suited for real audio applications.