Answer
(moved from comments, to tidy up later)
I appreciate you experiment results very much. I also have noise problems in my projects. (1) I often use Lipo power banks (18650 x 3) 12VDC and step it down to 5V for Rpi and other circuits. (2) I even avoid using switching power supplies (LM2596/LM2941) and use series regulators (7805) and LDOs, (3) I use huge bypass capacitor (electrolytelyte) 10,00 uF), 0.1~1uF tanlum/cermic caps. (4) I also use dedicated/separate PSUs for Rpi and power amps,
(5) Grounding is also very important, I separate my grounds into (a) power ground, (b) logic ground, and (c) audio signal ground, and of course shielded or double shielded AV cables. I even dedicate the Rpi's several ground pins into groups, for I2C/SPI/UART/GPIO (audio) etc. Of course I need to use "ground stubs" later. For also signals Audio or I2C etc, I have separate ground lines going with them,
(6) For power amps, I sometimes use differential input D type power amp. (7) I also have digital low/band pass filters (for my EEG/ECG/GPS etc) to try to reduce noise. (8) I use UGreen USB sound cards (OK, they are not very HiFi, but cheap, with button selection of raw/music/gaming band widths. (9) I also try dynamic/moving coil (Sony grade) and cheap pizo microphones.
References
(1) UGreen USB Sound Card Discussion 1/2
(2) UGreen USB Sound Card Discussion 2/2
(3) Rpi GPIO Push Button Circuit Grounding Problem
(4) Ground loop problems and how to get rid of them Written and copyright by Tomi Engdahl 1997-2013
(5) RpiZero Power Supply through 40 pin Power/GPIO Header or microUSB connector
(6) Why Grounding is used - Epanorama
(7) Using bypass capacitors to reduce voltage spikes and noise causing I2C read/write errors
(8) Clean Power for Every IC, Part 1: Understanding Bypass Capacitors - R Keim 2015sep21
(9) Clean Power for Every IC, Part 2: Choosing and Using Your Bypass Capacitors - R Keim 2015sep27
/ to continue, ...
Appendices
Appendix A -
Appendix B -
Appendix C -
Appendix D -
Appendix E -
Appendix F - MAX4466 Microphone Amplifier and 24 but HiFi Stero ADC
MAX4466 Microphone Amplifier and 24 but HiFi Stero ADC
End of Answer