Is it possible to monitor bandwidth usage through home router, by making RPi DHCP server, and track usage of each IP address through it? I have already seen some third party firmware for routers that will do the job, but I want to see if there is any more general solution than that.
DHCP servers just listen for dhcp broadcast messages from devices requesting ip addresses and responding with an ip address (as well as some other networking related information). It does not cause traffic to pass through the DHCP server.
In order to monitor network usage you need to see the network traffic to monitor it, which means you need the network traffic to pass through the device that monitors it. Devices attached to routers and switches do not receive all the traffic that passes through the router or switch as they are smart enough to learn where to send most traffic and avoid it being sent to all devices.
Network hubs do not have this problem, they forward all traffic received on one port to all other ports. So if you attached a pi to a hub it would be able to see all the traffic sent through the hub and thus track/monitor it.
But you already have a device that can monitor all traffic through the router - the router. Most decent routers can already expose this information and if you are that interested in networking it pays to invest in a decent router that does what you require.