Answer
(moved from comments, to tidy up later)
I appreciate you experiment results very much. I also have noise problems in my projects. (1) I often use Lipo power banks (18650 x 3) 12VDC and step it down to 5V for Rpi and other circuits. (2) I even avoid using switching power supplies (LM2596/LM2941) and use series regulators (7805) and LDOs, (3) I use huge bypass capacitor (electrolytelyte) 10,00 uF), 0.1~1uF tanlum/cermic caps. (4) I also use dedicated/separate PSUs for Rpi and power amps,
(5) Grounding is also very important, I separate my grounds into (a) power ground, (b) logic ground, and (c) audio signal ground, and of course shielded or double shielded AV cableseparate my grounds into (a) power ground, (b) logic ground, and (c) audio signal ground, and of course shielded or double shielded AV cables. I even dedicate the Rpi's several ground pins into groups, for I2C/SPI/UART/GPIO (audio) etc. Of course I need to use "ground stubs" later. For also signals Audio or I2C etc, I have separate ground lines going with them,
(6) For power amps, I sometimes use differential input D type power amp. (7) I also have digital low/band pass filters (for my EEG/ECG/GPS etc) to try to reduce noise. (8) I use UGreen USB sound cards (OK, they are not very HiFi, but cheap, with button selection of raw/music/gaming band widths. (9) I also try dynamic/moving coil (Sony grade) and cheap pizo microphones.
References
/ to continue, ...
Appendices
Appendix A -
Appendix B -
Appendix C -
Appendix D -
Appendix E -
End of Answer