The Pi Zero and A+ both have what seems like space for an ethernet port on the bottom, in the section labeled
Is this really an ethernet port, and if so, how would one connect to it (provide links to a jack that would fit)?
No it's not an ethernet port (and it doesn't look anything like one either).
It's the videocore JTAG port, that connector is used on all Pi models since the B+ (the original raspberry pi model A and B use a pin header instead). It is used by the engineers who developed the Pi for debugging.
Unfortunately it's not much use unless you have access to broadcom information and tools that are not available to the public.