Can I remove the 2 headless start files which are located in
/boot? Specifically, can I remove
wpa_supplicant.conf after I've finished the first boot and reconfigured everything with
I am assuming you are talking about Raspbian. At the first boot from a newly prepared SD card, if an ssh file and a wpa_supplicant.conf file are found in /boot then they are both moved. You would not be able to delete them, as they would be gone already. If this does not happen, something is wrong.