This is most always a case of either a bad power supply or software that is too "heavy" on the pi.
Hardware
Some times adapters output a different voltage/current then the label says they do. If you have a multimeter you can check to see the current and voltage output of your charger. In order for you Pi to run properly in an idle state for most boards it needs at least 700 mA and between 4.75 V (absolute minimum) and 5.25 volts. Depending on peripherals such as, high current Wifi adapters, wireless keyboard adapters, and external hard drives, you will need more current input.
Software
The issue could also be a problem with the software overloading the Pi. You could write a simple script to monitor the cpu usage to check this. Sort of something like this or this. If you are running your pi headless as a server if you have not already done so I suggest you run sudo raspi-config
and split the ram to give the most to the cpu since you are not using the gpu.