I would recommend you to have a look at Ansible as I am already using it to update some hundred Pis that are distributed over various places and sometimes also are only connected via LTE.
To reach all those Pis, I configured an openvpn-server to which all Pis connect when they boot.
In Ansible you can write playbooks and more or less you can do everything that is needed: copy files, install/update packages, maintain users, ... and the cool thing is that things only change if they need to be changed (e.g. if openvpn is already installed, it will not be installed anymore.) this is called idempotent.
When using the Ansible solution you also might have a look at AWX which allows you to execute Ansible playbooks via a Web-UI and which is easily setup in a docker-container on your server.
Of course there are alternatives to Ansible like Puppet or Chef.
It is hard to give you a precise answer, but I hope this guides you to the right direction, if you have any more questions, feel free to ask! Also when having specific Ansible-related questions ask them on stackoverflow, as there is a very active community answering them.