You could achieve this with an "exclusive OR" (or XOR) gate and some feedback from the manual switches to the Pi (so it can adjust it's output when you manually change the light state) but if this is the case then you might as well drive the lights from the Pi all the time and treat the manual switch as an additional input to the Pi that should be combined with the external request by the software (via WiFi or whatever):
simulate this circuit – Schematic created using CircuitLab
As we're dealing with mains voltages you will need to ensure that you, your house and your Pi are safe and legal -- if you are not sure how to do this then pick a different project or educate yourself first. Note that this is just a rough outline, you'd need all sorts of isolation and protection here to prevent damage or death. I repeat for emphasis: dealing with mains voltages should be done with care, it's up to you to know what you are doing.
Given the above you might be better off looking at some of the commercially available modules available, such as Sonoff or Crownstone.