How to retrieve data from Arduino and ESP via local wifi network on RPI 3

Question

I have a DHT22 connected to an Arduino Uno with a simple DHT sketch. Also connected is an ESP8266-01 that has been flashed and an AP +STA mode program uploaded to it. I'm wanting to use a RPI 3 to retrieve temp and humidity data from the Arduino and display it in a spreadsheet.

I have successfully pinged my ESP from my RPI.

I know similar projects have been done but they either use a webpage on the internet (I don't want to do) or a USB connection (also don't want to do), I want to stay on my local network without having to use the internet. Also, my Arduino, DHT, & ESP are about 30 feet from my wifi router and my RPI is another 15 feet the other direction.

I have installed MQTT on my RPI but I don't understand it. It has been years since I did anything with python scripts and honestly I barely remember anything. With that said, I know that python scripts would be the best and easiest way for me on the RPI.

Arduino code :

#include Adafruit_Sensor.h
#include DHT.h

//Constants
#define DHTPIN 2     // what pin we're connected to
#define DHTTYPE DHT11   // DHT 11  (AM2302)
//#define DHTTYPE DHT22   // DHT 22  (AM2302)
DHT dht(DHTPIN, DHTTYPE); //// Initialize DHT sensor for normal 16mhz Arduino

float hum;  //Stores humidity value
float temp; //Stores temperature value

void setup() {
  Serial.begin(115200);
  dht.begin();
  }

void loop() {

  /////////////////////////////////DHT22///////////////////////////////
  {
    Serial.println("     Temperature and Humidity (DHT22)");
    //Serial.println("");
    delay(1000); //Delay 1 sec.
    //Read data and store it to variables hum and temp
    hum = dht.readHumidity();
    temp = dht.readTemperature() * 1.8 + 32.1;
    //Fahrenheit  = (Celsius * 9 + 2) / 5 + 32; // The +2 takes care of the better rounding
    // or you can use->   Celsius * 1.8 + 32
    //Print temp and humidity values to serial monitor
    Serial.print("Humidity = ");
    Serial.print(hum);
    Serial.println(" %");
    Serial.print("Temp = ");
    Serial.print(temp);
    Serial.println(" degrees Fahrenheit");
    Serial.println("");
    Serial.println("");
    delay(2000);       //  waits 2000 milliseconds (2 sec).
  }

  }

ESP code:

    #include <ESP8266WiFi.h>        // Include the Wi-Fi library

const char* ssid     = "ssid";         // The SSID (name) of the Wi-Fi network you want to connect to
const char* password = "passwrd";     // The password of the Wi-Fi network
const char *ssid2 = "ssid"; // The name of the Wi-Fi network that will be created
const char *password2 = "passwrd";   // The password required to connect to it, leave blank for an open network

void setup() {
  Serial.begin(115200);         // Start the Serial communication to send messages to the computer
  delay(10);
  Serial.println('\n');

 ////////////////////////////****  Station mode  ****///////////////////////////////////////////
 WiFi.begin(ssid, password);             // Connect to the network
  Serial.print("Connecting to ");
  Serial.print(ssid); Serial.println(" ...");

  int i = 0;
  while (WiFi.status() != WL_CONNECTED) { // Wait for the Wi-Fi to connect
    delay(1000);
    Serial.print(++i); Serial.print(' ');
  }

  Serial.println('\n');
  Serial.println("Connection established!");  
  Serial.print("IP address:\t");
  Serial.println(WiFi.localIP());         // Send the IP address of the ESP8266 to the computer

/////////////////////////////****  Access Point mode  ****/////////////////////////////////////
  WiFi.softAP(ssid2, password2);             // Start the access point
  Serial.print("Access Point \"");
  Serial.print(ssid2);
  Serial.println("\" started");

  Serial.print("IP address:\t");
  Serial.println(WiFi.softAPIP());         // Send the IP address of the ESP8266 to the computer
}

void loop() { }

Answer 1

MQTT is worthy of further investigation for your use.

MQTT is a messaging protocol used for machine to machine communication. It uses a publish and subscribe model. Topics are created on the MQTT server (which is called a broker) and clients can either publish data to the topic or subscribe to the topic waiting for data. There is no complicated formatting of the data, it’s just a string of text. Perhaps, in csv format.

The RPI would be the MQTT server/broker and the Arduino/ESP would be the client publishing data. You now need a client on either your RPI or PC to subscribe to the incoming data and save it to a file that could be imported to a spreadsheet. If you’d rather not use Python have you looked at Processing? I’ve never used it but from what I understand it’s pretty similar to the Arduino IDE/language and there’s a MQTT library.

Answer 2

Your problem is you do not have enough (or perhaps, any) experience in network programming. The fundamental entity of network programming is the socket. If you have done much of anything at all with the ESP, you probably have used these already.

A fundamental concept of network programming is the client - server relationship. A server is an application that waits on an internet port¹ and answers requests from clients, a client being an application that connects to a server and initiates two-way communication. An obvious example of this is web browers and web servers. The central point is that servers do not connect to clients, clients connect to them. It is possible for one application to act as both a client and a server. For example, a web server may connect as a client to a database server to retrieve information, but in that context the program is acting as a client, likely in parallel to its role as a server to web clients.

If all this is new to you, you could do some tutorials and experiments with the ESP, although working with a regular OS based computer (such as a Raspberry Pi) is probably easier.

You have to decide whether you want the ESP to act as a server or a client. If the latter, then it will connect to a server on the pi. If the former, a client on the pi will have to connect to it. It probably makes the most sense for the ESP to be the server (unless you want to use SSL/TLS for privacy, since I believe there are some issues with running a TLS server on the ESP).

You also need to decide whether to use UDP or TCP; if you do not know the difference, read up on that. Note that UDP obscures the server/client distinction a bit since for every message back and forth the parties switch roles (i.e., a UDP message is always from a client to a server, whereas with TCP once the connection is established, each party maintains the role it had to start with but either one may send or receive a message, hence TCP is considered persistent).

The last major piece would be deciding on a protocol to use. In this case, if the ESP passes text messages straight on from the Arduino, you could use a terminator based protocol, that is, one in which a single message is terminated by a special character or sequence of characters. \n or \r\n are commonly used with text.

---

^{1. Note "internet" refers to the Internet Protocol (IP, see the wikipedia bit on ports) and does not necessarily mean that you need either party to be connected to the internet at large. The methodology is the same regardless.}