It is entirely possible to do this with a bit of careful rework (magnifier and hot air gear is required to remove the old crystal) and you do get very significant improvement in performance.
I took an RPi3 B+ with NTP and a Trimble GPS receiver, removed the stock crystal and replaced with a $10 TCXO
http://www.farnell.com/datasheets/2371462.pdf
As a 3V3 part it needed AC coupling via a 1nF capacitor, and the DC level setting with 240K & 100K resitors, but the results speak for themselves.
The stock performance is shown in these graphs, where you can see the temperature changes in my home office lead to ~6ppm variation in the crystal frequency, and during periods of faster change, this yields up to 150us wander (while NTP is good at compensating for frequency offset, it takes it time to spot and compensate for drift).
Swapping to the TCXO gives these graphs
Superficially this might look worse, until you realise that the scales for offset and frequency cover ranges are two orders of magnitude smaller! The worst case wander is reduced to less than 1us, with the RMS being around 300ns, and the frequency variation is reduced to ~30ppb. (The bottom "Wander" graph is at 0 for the TCXO system, because the ntpq data I was scraping only reports to the us resolution, and the actual wander is much less than 1us.)
All in all a very nice cheap, compact, low power, high performance NTP server / frequency reference!
If anyone wants more info on the mod, let me know.
(Pictures of the cased system, with GPS board, TCXO, and 1pps + 10MHz outputs below. TCXO is covered with foam tape to provide insulation and reduce rate of temperature change.)
Update
Several people have asked for more info.
A picture of the connection to the position of the original crystal is attached below.
This shows where I pickup the 3V3 supply, and the micro-coax I use to connect to gnd and the CPU crystal input pin. (I use coax to avoid RFI issues.)
In terms of the capacitor and resistors I mention, the output of the TCXO I used is 0.8V p-to-p, centred on 1V65 (3V3/2).
The relevant bits of the RPi run at 1V8, so we need to shift the TCXO output to a bias voltage of about 0.9V (1V8/2), ideally without significantly reducing the swing.
We achieve this by connecting the TCXO output via a 1nF chip capacitor (so it becomes AC coupled), then setting the DC bias voltage of the crystal input using a resistance divider from the 3V3 supply. (With just the capacitor, operation is unstable as the DC level of the input is floating.)
The 240K/100K divider (using two resistors I had to hand) gives a bias point of ~0.95V, and is easily overcome by the TCXO output drive current and the lowish impedance of the 1nF cap at 19.2MHz.
For space reasons, the AC coupling cap and resistive divider are all mounted on the TCXO (covered in foam), and this is connected via the length of coax to the PCB.
Sketch of change as delta to RPi3 schematic below.