The X session that you see is running in user space. You will need privileges to shutdown. If you start an X session using
startx, it will not allow you to shut down either. You may only logout (xfce). If you start an x session from a display manager like gdm, lightdm, xdm, mdm etc, the X session is running with some privileges. command line is the best you can do.
Alternatively, you can write a script with
gksudo poweroff It shall ask you for your password and shall proceed with the shutdown. You can place a shortcut on the desktop or as an applet in your taskbar.