Microcontroller projects

Programmimg ESP's by using the ESPToolbox library

last updated: 2022-11-11

esptoolbox

Quick links

Intro

Why this? There are cool books and tutorials on the net with code snippets to program ESP's. You can find e.g. a really useful guide from Pieter P on github.io.

I work often with both microcontroller from Espressif, the ESP8266 and the ESP32. I'm a forgetful man, and often search quite a long time to find back pieces of code already written and used in my projects. So I wrote an Arduino library to hold all those pieces of code and named it ESPToolbox (first version was called ESPBacker). The code is intended to work on ESP8266 and ESP32, and to help coding quicker with shorter and clearer code.

This toolbox contains methods to:

And an example to use everything together with MQTT and a temperature, humidity and pressure sensor (BME280).

The Toolbox has already proved its usefulness and reliability in my last projects like:

Boards used

I often use the LOLIN (WEMOS) D1 mini pro. The MHEtLive ESP32MiniKit is small and nearly (RxD and TxD are interchanged!!) pin compatible with the LOLIN board and comes handy if we need a more powerful controller.

ESP boards

In the middle of the picture we see the prior version of the LOLIN board named Wemos D1 mini pro.

Other ESP8266 and ESP32 boards should also work with the following software.

Naming conventions

I was often confused and did not know what convention for naming identifiers (names given to a program element) to use in different languages, and so was sometimes not consistent in my own programs.

In last years I often use Python with a quite clear convention (PEP8) and as this convention does not conflict with C++ (Arduino) I will try to stick to that convention beginning today ;). Here my rules:

Installing the library

You find the library on github: https://github.com/weigu1/esptoolbox. Click on the green Code Icon and then on Download ZIP.

Now in your Arduino IDE click on Sketch > Include Library > Add .ZIP Library.... An choose the file esptoolbox-main.zip in your Download folder.

Now all the libraries files are in your libraries folder inside your Arduino Sketchbook folder.

In File > Examples > ESPToolbox you now find all the example sketches from this library.

The library relies only on core libraries, so no other libraries have to be installed.

Logging and debugging

Let's start with the most important chapter :).

Debug with BUILTIN_LED

Most boards have one or more built in LEDs. When no serial interface or UDP connection is available, it's good to use the LED's to pass debug infos. In the library we have little helper methods to use the built-in LED.

For many boards LED_BUILTIN is already defined. If this is not the case or if you want to use another LED use the overloaded method set_led_log(bool flag, byte led_pin) to change the pin number. The LED's on ESP8266 and ESP32 often use negative logic! If the logic on your board is negative use the overloaded method: `set_led_log(bool flag, bool pos_logic).

Here the example code using the library:

    /* esp_debug_with_builtin_LED.ino
       www.weigu.lu
       Led is on after enabling log!  */

    #include <ESPToolbox.h>

    ESPToolbox Tb;                        // create an ESPToolbox object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_led_log(true);               // LED logging. (pos logic, LED on)
      //Tb.set_led_log(true,false);       // use neg. logic: 2 par = false)
      //Tb.set_led_log(true,5);           // other pin if LED_BUILTIN not def.
      //Tb.set_led_log(true,5,false);     // alt. pin + neg. logic
      delay(2000);
    }
    /****** LOOP **************************************************************/

    void loop() {
      Tb.blink_led_x_times(3);            // default blink with 100ms (f=5Hz)
      delay(2000);                        // (LED was on and is on after blink)
      Tb.blink_led_x_times(3,500);        // blink with 500ms delay (f=1s)
      delay(2000);
      Tb.led_off();
      delay(2000);
      Tb.led_on();
    }

There exists a bunch of other LED helper functions not connected to logging to work with LED's:

Debug/Log with Serial

Logging per serial monitor helps a lot when programming and debugging. Unfortunately TxD and/or RxD from Serial0 (Serial) are also needed for programming most ESP boards or are needed for communication to a sensor or a device.

ESP8266 has fortunately a second serial interface (Pin D4 (GPIO2) for WEMOS/LOLIN D1 mini pro board, look here). Don't use the LED logging simultanously with Serial1 on the Wemos D1 mini boards, because the builtin LED uses the same pin (D4, GPIO2)!

ESP32 has even three serial interfaces. The second and the third serial interface are accessed through Serial1 ans Serial2.

With a serial to USB adapter cable (3.3V!! for ESP boards) and a terminal program like Cutecom we can log over Serial1 or Serial2.

The library contains the following methods for serial logging:

To debug or log messages we use the log() or log_ln()functions.

    /* esp_log_serial.ino
      www.weigu.lu
      Serial1 on Wemos D1 Mini: pin D4 (only Tx)
      Serial1 on MH ET Live ESP32 MiniKit (Tx): SD3  (Pin can be changed!)
      Serial2 on MH ET Live ESP32 MiniKit (Tx): IO17 (Pin can be changed!)  */

    #include <ESPToolbox.h>

    ESPToolbox Tb;                        // create an ESPToolbox object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_serial_log(true);            // enable LED  serial logging on Serial
      Tb.set_led_log(true);               // enable LED logging (pos logic)
      // overloaded method to choose Serial1 (1) or Serial2 (2, only ESP32)
      //Tb.set_serial_log(true,1);
    }
    /****** LOOP **************************************************************/

    void loop() {
      Tb.blink_led_x_times(3);            // default blink with 100 ms
      Tb.log("Hel");                      // no new line
      Tb.log_ln("lo");                    // add newline (\n) at the end
      delay(2000);
    }

Debug/Log over UDP

Serial is cool over the Arduino serial monitor. But if this is not possible, it gets cumbersome because additional hardware is needed. The ESP's have WiFi on board and allow us to use UDP to send messages to a computer. If we have a Linux computer (e.g. raspberry pi) the simple netcat program allows us to listen to our log information.

So we will see all the logging information in a terminal window of the Linux computer (e.g. raspberry pi)!

To do so we open a terminal window and type:

    nc -kulw 0 6464

The port (here 6464) we want to use can naturally be changed at will.

The library contains the following methods for UDP logging:

As we use WiFi and we need to use a WiFi method explained in the next chapter.

Config.h

We also have to to pass our user name and password. For UDP we need to have a fixed IP address for our Linux computer and a port. To make these settings more accessible, they are contained in a config file named config.h. This file must be in the same folder as our sketch (.ino file).

When we want to distribute the software, we don't want to share the password in the config file. So here is another way:
We create a new folder called e.g. Secrets in the libraries folder in our Arduino sketchbook folder (the path to the sketchbook folder can be found in the Preferences Window (File Menu in Arduino IDE)). Now we create a new text file called secrets_xxx.h (xxx will be replaced with a meaningful extension) in the Secrets folder. Then we copy the content of the config file config.h to the secrets_xxx.h file.

Last we uncomment the line #define USE_SECRETS in the Sketch. Here an example with the config data in a file named secrets_udp_logging.h.

    /* esp_log_udp.ino
       www.weigu.lu
       for UDP, listen on Linux PC (UDP_LOG_PC_IP) with netcat command:
       nc -kulw 0 6464                                                   */

    /*!!!!!!       Make your changes in config.h (or secrets_xxx.h)      !!!!!!*/

    /*------ Comment or uncomment the following line suiting your needs -------*/
    #define USE_SECRETS

    /****** Arduino libraries needed ******/
    #include "ESPToolbox.h"            // ESP helper lib (more on weigu.lu)
    #ifdef USE_SECRETS
      // The file "secrets_xxx.h" has to be placed in a sketchbook libraries
      // folder. Create a folder named "Secrets" in sketchbook/libraries and copy
      // the config.h file there. Rename it to secrets_xxx.h
      #include <secrets_udp_logging.h> // things you need to change are here or
    #else
      #include "config.h"              // things you need to change are here
    #endif // USE_SECRETS

    /****** WiFi and network settings ******/
    const char *WIFI_SSID = MY_WIFI_SSID;         // in secrets_xxx.h or config.h
    const char *WIFI_PASSWORD = MY_WIFI_PASSWORD; // in secrets_xxx.h or config.h
    IPAddress UDP_LOG_PC_IP(UDP_LOG_PC_IP_BYTES); // in secrets_xxx.h or config.h

    ESPToolbox Tb;                     // Create an ESPToolbox Object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_udp_log(true, UDP_LOG_PC_IP, UDP_LOG_PORT);
      Tb.set_led_log(true); // enable LED logging (pos logic)
      Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD);
    }
    /****** LOOP **************************************************************/

    void loop() {
      Tb.log_ln("Hi there :)");
      delay(5000);
      Tb.blink_led_x_times(3);
      if (WiFi.status() != WL_CONNECTED) {   // if WiFi disconnected, reconnect
        Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD);
      }
    }

Here the output of the program:

    Connected to SSID mywifi with IP 192.168.178.144
    Signal strength is -50 dBm
    Hi there :)
    Hi there :)

The other way around if we want to use the file config.h from the .ino folder we comment the line #define USE_SECRETS:

    //#define USE_SECRETS

Here the content of a minimal config.h (or secrets_xxx.h) file:

    /****** WiFi SSID and PASSWORD ******/
    const char *MY_WIFI_SSID = "your_ssid";
    const char *MY_WIFI_PASSWORD = "your_password";

    /****** WiFi and network settings ******/
    // UDP logging settings if enabled in setup(); Port used for UDP logging
    const word UDP_LOG_PORT = 6464;
    // IP address of the computer receiving UDP log messages
    const byte UDP_LOG_PC_IP_BYTES[4] = {192, 168, 178, 100};

WiFi methods and a static IP address

Above we used the simplest WiFi method to connect an ESP to a router (Wifi station mode STA). There exist an overloaded method to add mDNS and an own hostname:

If we want to use a static IP address we have a setter method to set a flag and pass the needed IP addresses and a getter method to check the flag:

(One problem that is not yet solved: With a static IP we get no own hostname (why?).)

Let's look at an example with fixed IP, mDNS and hostname.

First we need to add things to our config e.g. secrets file (secrets_static_ip.h):

    /****** WiFi SSID and PASSWORD ******/
    const char *MY_WIFI_SSID = "your_ssid";
    const char *MY_WIFI_PASSWORD = "your_password";

    /*+++++++ Things you can change: +++++++*/

    /****** WiFi and network settings ******/
    // UDP logging settings if enabled in setup(); Port used for UDP logging
    const word UDP_LOG_PORT = 6464;
    // IP address of the computer receiving UDP log messages
    const byte UDP_LOG_PC_IP_BYTES[4] = {192, 168, 178, 100};
    // optional (access with UDP_logger.local)
    const char *NET_MDNSNAME = "ESP_static_IP";
    // optional hostname
    const char *NET_HOSTNAME = "ESP_static_IP";
    // only if you use a static address (uncomment //#define STATIC in ino file)
    const byte NET_LOCAL_IP_BYTES[4] = {192, 168, 178, 155};
    const byte NET_GATEWAY_BYTES[4] = {192, 168, 178, 1};
    const byte NET_MASK_BYTES[4] = {255,255,255,0};
    const byte NET_DNS_BYTES[4] = {8,8,8,8}; //  second dns (first = gateway), 8.8.8.8 = google

A new line lets you choose if a static IP address is used or not (comment (//#define STATIC) or uncomment the line).

    #define STATIC       // if static IP needed (no DHCP)
    /*
      esp_static_ip.ino
      www.weigu.lu
      for UDP, listen on Linux PC (UDP_LOG_PC_IP) with netcat command:
      nc -kulw 0 6464*/

    /*!!!!!!       Make your changes in config.h (or secrets_xxx.h)      !!!!!!*/

    /*------ Comment or uncomment the following line suiting your needs -------*/
    #define USE_SECRETS  // if we use secrets file instead of config.h
    //#define STATIC       // if static IP needed (no DHCP)

    /****** Arduino libraries needed ******/
    #include "ESPToolbox.h"            // ESP helper lib (more on weigu.lu)
    #ifdef USE_SECRETS
      // The file "secrets_xxx.h" has to be placed in a sketchbook libraries
      // folder. Create a folder named "Secrets" in sketchbook/libraries and copy
      // the config.h file there. Rename it to secrets_xxx.h
      #include <secrets_static_ip.h> // things you need to change are here or
    #else
      #include "config.h"              // things you need to change are here
    #endif // USE_SECRETS

    /****** WiFi and network settings ******/
    const char *WIFI_SSID = MY_WIFI_SSID;         // in secrets_xxx.h or config.h
    const char *WIFI_PASSWORD = MY_WIFI_PASSWORD; // in secrets_xxx.h or config.h
    IPAddress UDP_LOG_PC_IP(UDP_LOG_PC_IP_BYTES); // in secrets_xxx.h or config.h
    #ifdef STATIC
      IPAddress NET_LOCAL_IP (NET_LOCAL_IP_BYTES);// 4x optional for static IP
      IPAddress NET_GATEWAY (NET_GATEWAY_BYTES);  // look in config.h
      IPAddress NET_MASK (NET_MASK_BYTES);
      IPAddress NET_DNS (NET_DNS_BYTES);
    #endif // ifdef STATIC

    ESPToolbox Tb;                                // Create an ESPToolbox Object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_udp_log(true, UDP_LOG_PC_IP, UDP_LOG_PORT);
      Tb.set_led_log(true); // enable LED logging (pos logic)
      #ifdef STATIC
        Tb.set_static_ip(true,NET_LOCAL_IP, NET_GATEWAY, NET_MASK, NET_DNS);
      #endif // ifdef STATIC
      Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD, NET_MDNSNAME, NET_HOSTNAME);
    }

    /****** LOOP **************************************************************/

    void loop() {
      Tb.log_ln("Hi there again with static IP :)");
      delay(5000);
      Tb.blink_led_x_times(3);
      if (WiFi.status() != WL_CONNECTED) {   // if WiFi disconnected, reconnect
        Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD);
      }
    }

Using Over The Air programming (OTA)

ESPs have WiFi and so Over The Air programming is very convenient if your project is in a closed housing in the garden or cellar.

Remember that logging with Serial (Serial0) is not possible while using OTA. But naturally Serial1 and Serial2 will work, but UDP is more comfortable :).

After activating OTA you have to restart the Arduino IDE to see the new port (Tool > Port).

As for the other features a define line lets you choose if OTA is used or not (comment (//#define OTA) or uncomment the line).

    #define OTA       // if Over The Air update needed (security risk!)

We only need one method in setup():

Inside the loop we use the method from the OTA lib: ArduinoOTA.handle().

But first we need to add two lines our config e.g. secrets file (secrets_use_ota.h):

    /****** WiFi SSID and PASSWORD ******/
    const char *MY_WIFI_SSID = "your_ssid";
    const char *MY_WIFI_PASSWORD = "your_password";

    /****** WiFi and network settings ******/
    // UDP logging settings if enabled in setup(); Port used for UDP logging
    const word UDP_LOG_PORT = 6464;
    // IP address of the computer receiving UDP log messages
    const byte UDP_LOG_PC_IP_BYTES[4] = {192, 168, 178, 100};
    // optional (access with UDP_logger.local)
    const char *NET_MDNSNAME = "UDP_logger";
    // optional hostname
    const char *NET_HOSTNAME = "UDP_logger";
    // only if you use OTA (uncomment //#define OTA in ino file)
    const char *MY_OTA_NAME = "esp_with_ota"; // optional (access with esp_with_ota.local)
    // Linux Create Hasgh with: echo -n 'P@ssword1' | md5sum
    const char *MY_OTA_PASS_HASH = "myHash";     // Hash for password

Here the basic code:

    /* esp_use_ota.ino
      www.weigu.lu
      for UDP, listen on Linux PC (UDP_LOG_PC_IP) with netcat command:
      nc -kulw 0 6464  */

    /*!!!!!!       Make your changes in config.h (or secrets_xxx.h)      !!!!!!*/

    /*------ Comment or uncomment the following line suiting your needs -------*/
    //#define USE_SECRETS  // if we use secrets file instead of config.h
    #define OTA          // if Over The Air update needed (security risk!)

    /****** Arduino libraries needed ******/
    #include "ESPToolbox.h"            // ESP helper lib (more on weigu.lu)
    #ifdef USE_SECRETS
      // The file "secrets_xxx.h" has to be placed in a sketchbook libraries
      // folder. Create a folder named "Secrets" in sketchbook/libraries and copy
      // the config.h file there. Rename it to secrets_xxx.h
      #include <secrets_use_ota.h> // things you need to change are here or
    #else
      #include "config.h"              // things you need to change are here
    #endif // USE_SECRETS

    /****** WiFi and network settings ******/
    const char *WIFI_SSID = MY_WIFI_SSID;         // in secrets_xxx.h or config.h
    const char *WIFI_PASSWORD = MY_WIFI_PASSWORD; // in secrets_xxx.h or config.h
    IPAddress UDP_LOG_PC_IP(UDP_LOG_PC_IP_BYTES); // in secrets_xxx.h or config.h
    #ifdef OTA                                    // optional OTA settings
      const char *OTA_NAME = MY_OTA_NAME;         // look in config.h
      const char *OTA_PASS_HASH = MY_OTA_PASS_HASH;
    #endif // ifdef OTA

    ESPToolbox Tb;                                // Create an ESPToolbox Object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_udp_log(true, UDP_LOG_PC_IP, UDP_LOG_PORT);
      Tb.set_led_log(true); // enable LED logging (pos logic)
      Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD);
      #ifdef OTA
        Tb.init_ota(OTA_NAME, OTA_PASS_HASH);
      #endif // ifdef OTA
    }
    /****** LOOP **************************************************************/

    void loop() {
      #ifdef OTA
        ArduinoOTA.handle();
      #endif // ifdef OTA
      Tb.log_ln("Hi there; program me over the air :)");
      delay(5000);
      Tb.blink_led_x_times(3);
      if (WiFi.status() != WL_CONNECTED) {   // if WiFi disconnected, reconnect
        Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD);
      }
    }

Get the time from an Network Time Protocol (NTP) server

Most projects working with sensor data need an exact time. As we are connected to WiFi we can get the time from an NTP server in the Internet.

WE get the following methods:

The following program gives us the whole time structure (you can choose what you need :)) and the time.

    /* esp_use_ntp.ino
      www.weigu.lu
      for UDP, listen on Linux PC (UDP_LOG_PC_IP) with netcat command:
      nc -kulw 0 6464 */

    /*!!!!!!       Make your changes in config.h (or secrets_xxx.h)      !!!!!!*/

    /*------ Comment or uncomment the following line suiting your needs -------*/
    //#define USE_SECRETS  // if we use secrets file instead of config.h

    /****** Arduino libraries needed ******/
    #include "ESPToolbox.h"            // ESP helper lib (more on weigu.lu)
    #ifdef USE_SECRETS
      // The file "secrets_xxx.h" has to be placed in a sketchbook libraries
      // folder. Create a folder named "Secrets" in sketchbook/libraries and copy
      // the config.h file there. Rename it to secrets_xxx.h
      #include <secrets_use_ntp.h> // things you need to change are here or
    #else
      #include "config.h"              // things you need to change are here
    #endif // USE_SECRETS

    /****** WiFi and network settings ******/
    const char *WIFI_SSID = MY_WIFI_SSID;         // in secrets_xxx.h or config.h
    const char *WIFI_PASSWORD = MY_WIFI_PASSWORD; // in secrets_xxx.h or config.h
    IPAddress UDP_LOG_PC_IP(UDP_LOG_PC_IP_BYTES); // in secrets_xxx.h or config.h

    ESPToolbox Tb;                                // Create an ESPToolbox Object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_udp_log(true, UDP_LOG_PC_IP, UDP_LOG_PORT);
      Tb.set_led_log(true); // enable LED logging (pos logic)
      Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD);
      Tb.init_ntp_time();
    }

    /****** LOOP **************************************************************/

    void loop() {
      Tb.get_time();
      Tb.log("\nHere is the time structure: ");
      Tb.log_time_struct();
      Tb.log("\nTo get e.g. the time use Tb.log_ln(Tb.t.time); and you get: ");
      Tb.log_ln(Tb.t.time);
      delay(2000);
      Tb.blink_led_x_times(3);
      if (WiFi.status() != WL_CONNECTED) {   // if WiFi disconnected, reconnect
        Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD);
      }
    }

The program uses the NTP server address and the time zone configured in the library:

    char *NTP_SERVER = "lu.pool.ntp.org"; // NTP server
    // Time zone for Luxembourg (https://remotemonitoringsystems.ca/time-zone-abbreviations.php)
    char *TZ_INFO    = "CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00";

If you want to use another time zone or server you can overload the class constructor and add your values to the config e.g. secrets file (secrets_use_ntp.h):

    /****** WiFi SSID and PASSWORD ******/
    const char *MY_WIFI_SSID = "your_ssid";
    const char *MY_WIFI_PASSWORD = "your_password";

    /****** WiFi and network settings ******/
    // UDP logging settings if enabled in setup(); Port used for UDP logging
    const word UDP_LOG_PORT = 6464;
    // IP address of the computer receiving UDP log messages
    const byte UDP_LOG_PC_IP_BYTES[4] = {192, 168, 178, 100};
    // optional (access with UDP_logger.local)
    const char *NET_MDNSNAME = "ESP_NTP";
    // optional hostname
    const char *NET_HOSTNAME = "ESP_NTP";
    // if you want to use an NTP server
    const char *NTP_SERVER = "de.pool.ntp.org"; // NTP settings
    // your time zone (https://remotemonitoringsystems.ca/time-zone-abbreviations.php)
    const char *TZ_INFO    = "CET-1CEST-4,M3.5.0/02:00:00,M10.5.0/03:00:00";

In your main program exchange ESPToolbox Tb; with:

    ESPToolbox Tb(NTP_SERVER, TZ_INFO); // Create an ESPToolbox Object

The output of the sketch:

    Here is the time structure: 
    t.second: 2
    t.minute: 11
    t.hour: 11
    t.day: 27
    t.month: 8
    t.year: 2022
    t.weekday: 6
    t.yearday: 239
    t.daylight_saving_flag: 1
    t.name_of_day: Saturday
    t.name_of_month: August
    t.date: 2022-08-27
    t.time: 11:11:02
    t.datetime: 2022-08-27T11:11:02

    To get e.g. the time use Tb.log_ln(Tb.t.time); and you get: 11:11:02

Method for using millis() instead of delay()

As I often need millis() do do something non-blocking in loop() I wrote three helper methods for this.

They allow to control from 1 to 3 different events, and return true e.g. a number if time is up and the event should be executed.

Here a little example with 3 different events.

/* esp_non_blocking_delay.ino
   www.weigu.lu  */

    #include "ESPToolbox.h"            // ESP helper lib (more on weigu.lu)

    ESPToolbox Tb;                     // Create an ESPToolbox Object

    const byte PIN_LED2 = D1;          // Wemos D1 mini pro
    const byte PIN_LED3 = D2;          // Wemos D1 mini pro

    void setup() {
      Tb.init_led(false);              // Wemos D1 mini pro has neg. logic!
      Tb.init_led(PIN_LED2,true);
      Tb.init_led(PIN_LED3,true);
    }

    void loop() {
      switch (Tb.non_blocking_delay_x3(100, 200, 400)) {
        case 1:
          Tb.led_toggle();
          break;
        case 2:
          Tb.led_toggle(PIN_LED2);
          break;
        case 3:
          Tb.led_toggle(PIN_LED3);
          break;
        case 0:
          break;
      }
      yield();                         // feed the (watch) dog
      // do whatever you want here
    }

Using everything together with MQTT

So here a last example where we put everything together: Debugging over UDP, static IP, programming over OTA, getting the time per NTP server, and publishing a message at a fix time interval using the non blocking delay.

To make it more complete we add the temp/hum/press sensor BME280 over I²C (Connect 3.3 V, GND, SDA (D2,21) and SCL(D1,22)) and publish the values and the time in JSON format over MQTT.

esptb mqtt circuit

The software MQTT_Explorer (or MQTT.fx) shows us the following output:

mqtt explorer output 1

mqtt explorer output 2

udp output

Here is the config.h (secrets_xxx.h) file, with the added MQTT infos:

    /****** WiFi SSID and PASSWORD ******/
    const char *MY_WIFI_SSID = "your_ssid";
    const char *MY_WIFI_PASSWORD = "your_password";

    /****** WiFi and network settings ******/
    // UDP logging settings if enabled in setup(); Port used for UDP logging
    const word UDP_LOG_PORT = 6464;
    // IP address of the computer receiving UDP log messages
    const byte UDP_LOG_PC_IP_BYTES[4] = {192, 168, 178, 100};
    // optional (access with UDP_logger.local)
    const char *NET_MDNSNAME = "ESP_MQTT";
    // optional hostname
    const char *NET_HOSTNAME = "ESP_MQTT";
    // only if you use a static address (uncomment //#define STATIC in ino file)
    const byte NET_LOCAL_IP_BYTES[4] = {192, 168, 178, 155};
    const byte NET_GATEWAY_BYTES[4] = {192, 168, 178, 1};
    const byte NET_MASK_BYTES[4] = {255,255,255,0};
    const byte NET_DNS_BYTES[4] = {8,8,8,8}; //  second dns (first = gateway), 8.8.8.8 = google
    // only if you use OTA (uncomment //#define OTA in ino file)
    const char *MY_OTA_NAME = "esp_mqtt"; // optional (access with esp_with_ota.local)
    // Linux Create Hasgh with: echo -n 'P@ssword1' | md5sum
    const char *MY_OTA_PASS_HASH = "myHash";     // Hash for password

    /****** MQTT settings ******/
    const char *MQTT_SERVER = "192.168.178.222";
    const long PUBLISH_TIME = 10000; //Publishes every in milliseconds
    const int MQTT_MAXIMUM_PACKET_SIZE = 1024; // look in setup()
    const char *MQTT_CLIENT_ID = "ntp_time"; // this must be unique!!!
    String MQTT_TOPIC_OUT = "mqtt_test/data";
    String MQTT_TOPIC_IN = "mqtt_test/command";
    const short MY_MQTT_PORT = 1883; // or 8883
    // only if you use MQTTPASSWORD (uncomment //#define MQTTPASSWORD in ino file)
    const char *MY_MQTT_USER = "me";
    const char *MY_MQTT_PASS = "meagain";

For our program we need the following supplementary libraries (Wire is a core library):

The main program stays with the help of the library and 4 functions for MQTT and the sensor quite short.

    ESPToolbox Tb;                                // Create an ESPToolbox Object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_udp_log(true, UDP_LOG_PC_IP, UDP_LOG_PORT);
      Tb.set_led_log(true); // enable LED logging (pos logic)
      #ifdef STATIC
        Tb.set_static_ip(true,NET_LOCAL_IP, NET_GATEWAY, NET_MASK, NET_DNS);
      #endif // ifdef STATIC
      Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD, NET_MDNSNAME, NET_HOSTNAME);
      Tb.init_ntp_time();
      MQTT_Client.setBufferSize(MQTT_MAXIMUM_PACKET_SIZE);
      MQTT_Client.setServer(MQTT_SERVER,MQTT_PORT); //open connection MQTT server
      #ifdef OTA
        Tb.init_ota(OTA_NAME, OTA_PASS_HASH);
      #endif // ifdef OTA
      #ifdef BME280_I2C
        init_bme280();
      #endif // ifdef BME280_I2C
      Tb.blink_led_x_times(3);
      Tb.log_ln("Setup done!");
    }

    /****** LOOP **************************************************************/

    void loop() {
      #ifdef OTA
        ArduinoOTA.handle();
      #endif // ifdef OTA
      if (Tb.non_blocking_delay(PUBLISH_TIME)) { // PUBLISH_TIME in config.h
        mqtt_get_temp_and_publish();
        Tb.blink_led_x_times(3);
      }
      if (WiFi.status() != WL_CONNECTED) {   // if WiFi disconnected, reconnect
        Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD, NET_MDNSNAME, NET_HOSTNAME);
      }
      if (!MQTT_Client.connected()) {        // reconnect mqtt client, if needed
        mqtt_connect();
      }
      MQTT_Client.loop();                    // make the MQTT live
      delay(10); // needed for the watchdog! (alt. yield())
    }

And here the complete code:

    /* esp_use_mqtt.ino
      www.weigu.lu
      for UDP, listen on Linux PC (UDP_LOG_PC_IP) with netcat command:
      nc -kulw 0 6464 */

    /*!!!!!!       Make your changes in config.h (or secrets_xxx.h)      !!!!!!*/

    /*------ Comment or uncomment the following line suiting your needs -------*/
    //#define USE_SECRETS
    #define OTA               // if Over The Air update needed (security risk!)
    //#define MQTTPASSWORD    // if you want an MQTT connection with password (recommended!!)
    #define STATIC            // if static IP needed (no DHCP)
    #define BME280_I2C

    /****** Arduino libraries needed ******/
    #include "ESPToolbox.h"            // ESP helper lib (more on weigu.lu)
    #ifdef USE_SECRETS
      // The file "secrets_xxx.h" has to be placed in a sketchbook libraries
      // folder. Create a folder named "Secrets" in sketchbook/libraries and copy
      // the config.h file there. Rename it to secrets_xxx.h
      #include <secrets_use_mqtt.h> // things you need to change are here or
    #else
      #include "config.h"              // things you need to change are here
    #endif // USE_SECRETS
    #include <PubSubClient.h>          // for MQTT
    #include <ArduinoJson.h>           // convert MQTT messages to JSON
    #ifdef BME280_I2C
      #include <Wire.h>                // BME280 on I2C (PU-resistors!)
      #include <BME280I2C.h>
    #endif // ifdef BME280_I2C

    /****** WiFi and network settings ******/
    const char *WIFI_SSID = MY_WIFI_SSID;           // if no secrets, use config.h
    const char *WIFI_PASSWORD = MY_WIFI_PASSWORD;   // if no secrets, use config.h
    #ifdef STATIC
      IPAddress NET_LOCAL_IP (NET_LOCAL_IP_BYTES);  // 3x optional for static IP
      IPAddress NET_GATEWAY (NET_GATEWAY_BYTES);    // look in config.h
      IPAddress NET_MASK (NET_MASK_BYTES);
      IPAddress NET_DNS (NET_DNS_BYTES);
    #endif // ifdef STATIC
    #ifdef OTA                                      // Over The Air update settings
      const char *OTA_NAME = MY_OTA_NAME;
      const char *OTA_PASS_HASH = MY_OTA_PASS_HASH; // use the config.h file
    #endif // ifdef OTA

    IPAddress UDP_LOG_PC_IP(UDP_LOG_PC_IP_BYTES);   // UDP log if enabled in setup

    /****** MQTT settings ******/
    const short MQTT_PORT = MY_MQTT_PORT;
    WiFiClient espClient;
    PubSubClient MQTT_Client(espClient);
    #ifdef MQTTPASSWORD
      const char *MQTT_USER = MY_MQTT_USER;
      const char *MQTT_PASS = MY_MQTT_PASS;
    #endif // MQTTPASSWORD

    /******* BME280 ******/
    float temp(NAN), hum(NAN), pres(NAN);
    #ifdef BME280_I2C
      BME280I2C bme;    // Default : forced mode, standby time = 1000 ms
                        // Oversampling = press. ×1, temp. ×1, hum. ×1, filter off
    #endif // ifdef BME280_I2C

    ESPToolbox Tb;                                // Create an ESPToolbox Object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_udp_log(true, UDP_LOG_PC_IP, UDP_LOG_PORT);
      Tb.set_led_log(true); // enable LED logging (pos logic)
      #ifdef STATIC
        Tb.set_static_ip(true,NET_LOCAL_IP, NET_GATEWAY, NET_MASK, NET_DNS);
      #endif // ifdef STATIC
      Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD, NET_MDNSNAME, NET_HOSTNAME);
      Tb.init_ntp_time();
      MQTT_Client.setBufferSize(MQTT_MAXIMUM_PACKET_SIZE);
      MQTT_Client.setServer(MQTT_SERVER,MQTT_PORT); //open connection MQTT server
      #ifdef OTA
        Tb.init_ota(OTA_NAME, OTA_PASS_HASH);
      #endif // ifdef OTA
      #ifdef BME280_I2C
        init_bme280();
      #endif // ifdef BME280_I2C
      Tb.blink_led_x_times(3);
      Tb.log_ln("Setup done!");
    }

    /****** LOOP **************************************************************/

    void loop() {
      #ifdef OTA
        ArduinoOTA.handle();
      #endif // ifdef OTA
      if (Tb.non_blocking_delay(PUBLISH_TIME)) { // PUBLISH_TIME in config.h
        mqtt_get_temp_and_publish();
        Tb.blink_led_x_times(3);
      }
      if (WiFi.status() != WL_CONNECTED) {   // if WiFi disconnected, reconnect
        Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD, NET_MDNSNAME, NET_HOSTNAME);
      }
      if (!MQTT_Client.connected()) {        // reconnect mqtt client, if needed
        mqtt_connect();
      }
      MQTT_Client.loop();                    // make the MQTT live
      delay(10); // needed for the watchdog! (alt. yield())
    }

    /********** MQTT functions ***************************************************/

    // connect to MQTT server
    void mqtt_connect() {
      while (!MQTT_Client.connected()) { // Loop until we're reconnected
        Tb.log("Attempting MQTT connection...");
        #ifdef MQTTPASSWORD
          if (MQTT_Client.connect(MQTT_CLIENT_ID, MQTT_USER, MQTT_PASS)) {
        #else
          if (MQTT_Client.connect(MQTT_CLIENT_ID)) { // Attempt to connect
        #endif // ifdef MQTTPASSWORD
          Tb.log_ln("MQTT connected");
          MQTT_Client.subscribe(MQTT_TOPIC_IN.c_str());
        }
        else {
          Tb.log("MQTT connection failed, rc=");
          Tb.log(String(MQTT_Client.state()));
          Tb.log_ln(" try again in 5 seconds");
          delay(5000);  // Wait 5 seconds before retrying
        }
      }
    }

    // MQTT get the time, relay flags ant temperature an publish the data
    void mqtt_get_temp_and_publish() {
      DynamicJsonDocument doc_out(1024);
      String mqtt_msg, we_msg;
      Tb.get_time();
      doc_out["datetime"] = Tb.t.datetime;
      #ifdef BME280_I2C
        get_data_bme280();
        doc_out["temperature_C"] = (int)(temp*10.0 + 0.5)/10.0;
        doc_out["humidity_%"] = (int)(hum*10.0 + 0.5)/10.0;
        doc_out["pressure_hPa"] = (int)((pres + 5)/10)/10.0;
      #endif // ifdef BME280_I2C
      mqtt_msg = "";
      serializeJson(doc_out, mqtt_msg);
      MQTT_Client.publish(MQTT_TOPIC_OUT.c_str(),mqtt_msg.c_str());
      Tb.log("MQTT published at ");
      Tb.log_ln(Tb.t.time);
    }

    /********** BME280 functions *************************************************/

    // initialise the BME280 sensor
    #ifdef BME280_I2C
      void init_bme280() {
        Wire.begin();
        while(!bme.begin()) {
          Tb.log_ln("Could not find BME280 sensor!");
          delay(1000);
        }
        switch(bme.chipModel())  {
          case BME280::ChipModel_BME280:
            Tb.log_ln("Found BME280 sensor! Success.");
            break;
          case BME280::ChipModel_BMP280:
            Tb.log_ln("Found BMP280 sensor! No Humidity available.");
            break;
          default:
            Tb.log_ln("Found UNKNOWN sensor! Error!");
        }
      }

    // get BME280 data and log it
    void get_data_bme280() {
      BME280::TempUnit tempUnit(BME280::TempUnit_Celsius);
      BME280::PresUnit presUnit(BME280::PresUnit_Pa);
      bme.read(pres, temp, hum, tempUnit, presUnit);
      Tb.log_ln("Temp: " + (String(temp)) + " Hum: " + (String(hum)) +
              " Pres: " + (String(pres)));
    }
    #endif // ifdef BME280_I2C

Use Ethernet instead of WiFi

We can connect an Ethernet breakout board to our ESP and use Ethernet instead of WiFi with the core Arduino Ethernet library. Es UDP uses other methods for Ethernet as WiFi we need a flag to tell the library that we use Ethernet.

We get the following new methods:

A setter method to set a flag and a getter method to check the flag:

and the important method ti initialise Ethernet:

We can also use Ethernet with a static IP by setting this flag (see above). It is not possible to set an hostname or mDNS.

This is tested with a Funduino board with W5100 chip and an ESP8266:

smartyreader circuit

In the config (secrets) file we only have to add a MAC address.

    /****** WiFi SSID and PASSWORD ******/
    const char *MY_WIFI_SSID = "your_ssid";
    const char *MY_WIFI_PASSWORD = "your_password";

    /****** WiFi and network settings ******/
    // UDP logging settings if enabled in setup(); Port used for UDP logging
    const word UDP_LOG_PORT = 6464;
    // IP address of the computer receiving UDP log messages
    const byte UDP_LOG_PC_IP_BYTES[4] = {192, 168, 178, 100};
    // optional (access with UDP_logger.local)
    const char *NET_MDNSNAME = "ESP_Ethernet";
    // optional hostname
    const char *NET_HOSTNAME = "ESP_Ethernet";
    // only if you use a static address (uncomment //#define STATIC in ino file)
    const byte NET_LOCAL_IP_BYTES[4] = {192, 168, 178, 155};
    const byte NET_GATEWAY_BYTES[4] = {192, 168, 178, 1};
    const byte NET_MASK_BYTES[4] = {255,255,255,0};
    const byte NET_DNS[4] = {8, 8, 8, 8}; // optional (your gateway or 8.8.8.8 (google))
    byte NET_MAC[6] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0x00};  // for ethernet (e.g. Funduino board with W5100)

    /****** MQTT settings ******/
    const char *MQTT_SERVER = "192.168.178.222";
    const long PUBLISH_TIME = 10000; //Publishes every in milliseconds
    const int MQTT_MAXIMUM_PACKET_SIZE = 1024; // look in setup()
    const char *MQTT_CLIENT_ID = "ntp_time"; // this must be unique!!!
    String MQTT_TOPIC_OUT = "mqtt_test/data";
    String MQTT_TOPIC_IN = "mqtt_test/command";
    const short MY_MQTT_PORT = 1883; // or 8883
    // only if you use MQTTPASSWORD (uncomment //#define MQTTPASSWORD in ino file)
    const char *MY_MQTT_USER = "me";
    const char *MY_MQTT_PASS = "meagain";

Now here is the same MQTT example as above with Ethernet:

    /* esp_use_mqtt.ino
      www.weigu.lu
      for UDP, listen on Linux PC (UDP_LOG_PC_IP) with netcat command:
      nc -kulw 0 6464 */

    /*!!!!!!       Make your changes in config.h (or secrets_xxx.h)      !!!!!!*/

    /*------ Comment or uncomment the following line suiting your needs -------*/
    #define USE_SECRETS
    //#define MQTTPASSWORD    // if you want an MQTT connection with password (recommended!!)
    #define ETHERNET          // with an Ethernet board ((e.g. Funduino board with W5100))
    #define STATIC            // if static IP needed (no DHCP)
    //#define BME280_I2C


    /****** Arduino libraries needed ******/
    #include "ESPToolbox.h"            // ESP helper lib (more on weigu.lu)
    #ifdef USE_SECRETS
      // The file "secrets_xxx.h" has to be placed in a sketchbook libraries
      // folder. Create a folder named "Secrets" in sketchbook/libraries and copy
      // the config.h file there. Rename it to secrets_xxx.h
      #include <secrets_use_ethernet.h> // things you need to change are here or
    #else
      #include "config.h"              // things you need to change are here
    #endif // USE_SECRETS
    #include <PubSubClient.h>          // for MQTT
    #include <ArduinoJson.h>           // convert MQTT messages to JSON
    #ifdef BME280_I2C
      #include <Wire.h>                // BME280 on I2C (PU-resistors!)
      #include <BME280I2C.h>
    #endif

    /****** WiFi and network settings ******/
    const char *WIFI_SSID = MY_WIFI_SSID;           // if no secrets, use config.h
    const char *WIFI_PASSWORD = MY_WIFI_PASSWORD;   // if no secrets, use config.h
    #if defined(STATIC) || defined(ETHERNET)
      IPAddress NET_LOCAL_IP (NET_LOCAL_IP_BYTES);  // 3x optional for static IP
      IPAddress NET_GATEWAY (NET_GATEWAY_BYTES);    // look in config.h
      IPAddress NET_MASK (NET_MASK_BYTES);
      IPAddress NET_DNS (NET_MASK_BYTES);
    #endif // #if defined(STATIC) || defined(ETHERNET)

    IPAddress UDP_LOG_PC_IP(UDP_LOG_PC_IP_BYTES);   // UDP log if enabled in setup

    /****** MQTT settings ******/
    const short MQTT_PORT = MY_MQTT_PORT;
    #ifdef ETHERNET
      EthernetClient espClient;
    #else
      WiFiClient espClient;
    #endif // ifdef ETHERNET
    PubSubClient MQTT_Client(espClient);
    #ifdef MQTTPASSWORD
      const char *MQTT_USER = MY_MQTT_USER;
      const char *MQTT_PASS = MY_MQTT_PASS;
    #endif // MQTTPASSWORD

    /******* BME280 ******/
    float temp(NAN), hum(NAN), pres(NAN);
    #ifdef BME280_I2C
      BME280I2C bme;    // Default : forced mode, standby time = 1000 ms
                        // Oversampling = press. ×1, temp. ×1, hum. ×1, filter off
    #endif // BME280_I2C

    ESPToolbox Tb;                                // Create an ESPToolbox Object

    /****** SETUP *************************************************************/

    void setup() {
      Tb.set_serial_log(true);
      Tb.set_udp_log(true, UDP_LOG_PC_IP, UDP_LOG_PORT);
      Tb.set_led_log(true); // enable LED logging (pos logic)
      #ifdef STATIC
        Tb.set_static_ip(true,NET_LOCAL_IP, NET_GATEWAY, NET_MASK, NET_DNS);
      #endif // ifdef STATIC
      #ifdef ETHERNET
        Tb.set_ethernet(true);
        Tb.init_eth(NET_MAC);
      #else
        Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD, NET_MDNSNAME, NET_HOSTNAME);
      #endif // ifdef ETHERNET
      Tb.init_ntp_time();
      MQTT_Client.setBufferSize(MQTT_MAXIMUM_PACKET_SIZE);
      MQTT_Client.setServer(MQTT_SERVER,MQTT_PORT); //open connection MQTT server
      #ifdef BME280_I2C
        init_bme280();
      #endif // BME280_I2C
      Tb.blink_led_x_times(3);
      Tb.log_ln("Setup done!");
    }

    /****** LOOP **************************************************************/

    void loop() {
      if (Tb.non_blocking_delay(PUBLISH_TIME)) { // PUBLISH_TIME in config.h
        mqtt_get_temp_and_publish();
        Tb.blink_led_x_times(3);
      }
      #ifdef ETHERNET
      if (!espClient.connected()) {
        Tb.init_eth(NET_MAC);
      }
      #else
        if (WiFi.status() != WL_CONNECTED) {   // if WiFi disconnected, reconnect
          Tb.init_wifi_sta(WIFI_SSID, WIFI_PASSWORD, NET_MDNSNAME, NET_HOSTNAME);
        }
      #endif // ifdef ETHERNET
      if (!MQTT_Client.connected()) {        // reconnect mqtt client, if needed
        mqtt_connect();
      }
      MQTT_Client.loop();                    // make the MQTT live
      delay(10); // needed for the watchdog! (alt. yield())
    }

    /********** MQTT functions ***************************************************/

    // connect to MQTT server
    void mqtt_connect() {
      while (!MQTT_Client.connected()) { // Loop until we're reconnected
        Tb.log("Attempting MQTT connection...");
        #ifdef MQTTPASSWORD
          if (MQTT_Client.connect(MQTT_CLIENT_ID, MQTT_USER, MQTT_PASS)) {
        #else
          if (MQTT_Client.connect(MQTT_CLIENT_ID)) { // Attempt to connect
        #endif // ifdef UNMQTTSECURE
          Tb.log_ln("MQTT connected");
          MQTT_Client.subscribe(MQTT_TOPIC_IN.c_str());
        }
        else {
          Tb.log("MQTT connection failed, rc=");
          Tb.log(String(MQTT_Client.state()));
          Tb.log_ln(" try again in 5 seconds");
          delay(5000);  // Wait 5 seconds before retrying
        }
      }
    }

    // MQTT get the time, relay flags ant temperature an publish the data
    void mqtt_get_temp_and_publish() {
      DynamicJsonDocument doc_out(1024);
      String mqtt_msg, we_msg;
      Tb.get_time();
      doc_out["datetime"] = Tb.t.datetime;
      #ifdef BME280_I2C
        get_data_bme280();
        doc_out["temperature_C"] = (int)(temp*10.0 + 0.5)/10.0;
        doc_out["humidity_%"] = (int)(hum*10.0 + 0.5)/10.0;
        doc_out["pressure_hPa"] = (int)((pres + 5)/10)/10.0;
      #endif
      mqtt_msg = "";
      serializeJson(doc_out, mqtt_msg);
      MQTT_Client.publish(MQTT_TOPIC_OUT.c_str(),mqtt_msg.c_str());
      Tb.log("MQTT published at ");
      Tb.log_ln(Tb.t.time);
    }

    /********** BME280 functions *************************************************/

    // initialise the BME280 sensor
    #ifdef BME280_I2C
      void init_bme280() {
        Wire.begin();
        while(!bme.begin()) {
          Tb.log_ln("Could not find BME280 sensor!");
          delay(1000);
        }
        switch(bme.chipModel())  {
          case BME280::ChipModel_BME280:
            Tb.log_ln("Found BME280 sensor! Success.");
            break;
          case BME280::ChipModel_BMP280:
            Tb.log_ln("Found BMP280 sensor! No Humidity available.");
            break;
          default:
            Tb.log_ln("Found UNKNOWN sensor! Error!");
        }
      }

    // get BME280 data and log it
    void get_data_bme280() {
      BME280::TempUnit tempUnit(BME280::TempUnit_Celsius);
      BME280::PresUnit presUnit(BME280::PresUnit_Pa);
      bme.read(pres, temp, hum, tempUnit, presUnit);
      Tb.log_ln("Temp: " + (String(temp)) + " Hum: " + (String(hum)) +
              " Pres: " + (String(pres)));
    }
    #endif  // BME280_I2C

Next steps

I wanted to integrate MQTT and a Webserver into the library. But I had problems with the callback functions. Perhaps I could also integrate DS18B20 and BME280 functions in a later version.

Downloads