Raspbian Lite is, not surprisingly, very light on startup. Memory usage is minimal on a fresh boot:
$ free -m
total used free shared buff/cache available
Mem: 434 23 356 3 54 361
Swap: 99 0 99
If you are overly concerned about it, you might try using DietPi, although in my experience, by the time I configure the system the way I want, there is very little actually memory gain. According to the DietPi pages, the actual difference in memory between their optimized configuration and a basic raspbian lite install is 4 MB. They claim 11 versus 18 processes running, and a 1.3 second faster boot time. I suspect much of those gains are due to running dropbear in place of "regular" services such as openssh.
There are specialized installs that are even lighter, but you sacrifice a lot (e.g. regularly updated and patched repository) for little actual memory gain.
There is something to be said for minimizing the packages on disk in order to speed up updates, but you're not going to see a lot of memory, performance or boot time gains by doing so.
Here is the output of
$ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
└─ssh.service @35.298s +811ms
└─dhcpcd.service @10.641s +24.627s
└─systemd-timesyncd.service @9.500s +882ms
└─systemd-tmpfiles-setup.service @8.589s +620ms
└─boot.mount @8.152s +196ms