The simplest "device" that you can write a hardware driver for (if hardware driver development is your thing) can also be as simple as a LED (I added the quotation marks because technically a LED is not a device but it's still a piece of hardware) as @PiBorg has suggested.
Other choices would be some easy-to-interface device/components like a photoresistors, passive infrared sensors (short: PIR), temperature sensors, PC fans (preferably a 4-wire fan that allows you not only to monitor but also to control the RPM), LED dot matrices and so on. Basically such simple devices will allow you to have the minimum on hardware (so that you can see and touch what you've actually accomplished) and at the same time you can learn about a lot of topics that are used for much more complex devices where the complexity comes mostly from the protocol they use.
Mind also that you don't need to go the extra mile digging into the kernel modules. Of course if you want to do that nobody is stopping you. :)
Here is an example of interfacing a passive infrared sensor (I'm going to test it soon when my PIR gets delivered :3). You can take this and start digging deeper into the world of the Linux kernel in order to see how you can for example create a kernel driver, which works with the PIR.