Windows is always going to be an outlier; I can't help you with your Windows issues, except to offer this. I can share some general information relevant to your RPi that might help:
Wrt your RPi, there are two things to check/verify:
Under most circumastances, it should have a single line with the name of the host; e.g.:
$ cat /etc/hostname
$ cat /etc/hosts
::1 localhost ip6-localhost ip6-loopback
127.0.1.1 raspberrypi3b raspberrypi3b.local
Note the last line in particular:
127.0.1.1. At least one entry (
raspberrypi3b) must be included for
mdns to work consistently. I have added a second name (
raspberrypi3b.local) as a contingency. There's some decent documentation on this from Debian. Raspberry Pi also has some documentation, but it must be carefully parsed as it tends to lead some readers astray, and they have an odd attitude about this - at least to my way of thinking.
Just to make sure there's no confusion, the hostname I've used (
raspberry3b) is one I chose. You are free to choose your own!
Once you get your Windows hosts configured to support mDNS (aka
zero configuration networking), and verify the 2 settings above, things should fall into place.