0

Hello all I was wondering how I can monitor all my networking traffic with a raspberry pi. What I want to do is record all websites visted so I can create a app so I can see what sites I frequent the most.

3

This is not a pi specific question. And the answer is there is no guarantee such a thing will work (it won't work if encryption like ssl, proxy, vpn or caching of dns is used). A better solution for your goal would be to use the same brand of browser on all your devices and enable history sync. Then you can just query that from any of your computers. If you still want to try this than get a second Ethernet port for the rpi and put it between your modem and router (won't log local traffic)

  • 2
    The IP addresses are never encrypted (except on local wifi) -- they cannot be for hopefully obvious reasons. I agree there are probably better solutions to this if the goal is just WWW statistics. – goldilocks May 8 '15 at 12:06
  • @goldilocks - IP addresses aren't encrypted, but encrypted traffic to a proxy would conceal the real IP address and show only the proxy's. Also, many smaller sites are hosted by shared serving infrastructure such that the IP address does not uniquely identify the site. – Chris Stratton May 11 '15 at 22:47
  • @ChrisStratton Yes, monitoring anything beyond the basic addressing (MAC, IP, port) is totally impossible -- that's the whole point of point-to-point encryption, which includes your dialog w/ the server (what website data I'm looking for, etc). – goldilocks May 12 '15 at 8:30
0

If there is encrypted wireless involved this is not so easy; see my question here about looking for an alternative to tshark for a similar purpose.

Note that if you don't use a filter, tshark may not grow the same way, but then you either need something else to filter the packets, or save them all (!) and process them subsequently. No matter what though, there is the complication of having to stop and restart the LAN's wifi after you begin monitoring.

That said, it did work, and once I got the traffic down to < 1 GB a day (instead of > 1 GB/hour) tshark would last a day or so on a Pi 2.

Unfortunately, if your router is like mine I do not think there is a way to deal with wifi and ethernet traffic together; you need to deal with them separately. The eth is pretty easy as it does not require decryption to get the TCP/UDP packet headers.

  • The logical place to do monitoring of course is on the router itself. Encrypted traffic will still be opaque, but at least you get a chance to see everything that is happening. – Chris Stratton May 11 '15 at 22:48
  • @ChrisStratton The router is definitely the ideal place. The encryption issue has to do specifically w/ how WPA works. Encryption a la SSL/TLS is not a problem, because that's inside the onion layers of a packet -- only the content and whatever protocols are inside SSL/TLS are encrypted. That doesn't include the IP and TCP/UDP layers; so e.g. SSL/TLS does not encrypt your IP address, MAC, relevant ports, etc. It's just the payload. But WPA is pretty much the physical layer of the OSI model.... – goldilocks May 12 '15 at 8:14
  • ...It does not have any correlate ethernet wise. WPA encrypts everything from the ground up. A WPA encrypted packet is just gobbeldy gook until decrypted -- it does not have any discernible source or destination address. It is just static in the air. SSL/TLS encryption is between individual parties all the way across the WWW. But WPA encryption is done by the router just for the entire local subnet -- if you are on it, then it doesn't matter, it is completely transparent. But (unlike ethernet) the only point at which you can monitor a wifi subnet that way is the router. – goldilocks May 12 '15 at 8:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.