Page 1 of 1

Connectifity issues

Posted: Tue Feb 05, 2019 8:58 am
by Thomseeen
Hey! :)

so I have been playing around with an ESP32 for a little now and my project should be finished but I have some weird connectivity issues which I just can't manage to resolve.
The latest version of my project can be found here https://github.com/Thomseeen/Summerscho ... /Debugging.
I am using the PlatformIO IDE in VSCode, that's why makefiles and such are missing.
My project uses an OV2640 camera with the camera library provided by espressif. To be able to send MQTT messages I'm using this library https://github.com/256dpi/esp-mqtt.
My problem: A lot of TCP packages seem to not reach their destination.
The program sends a timestamp and a picture (up to 37kB) via MQTT.
The ESP always connects successfully to the WiFi.
The lwip SNTP app is always able to get the current time.
Then the library tries to connect to an MQTT-broker on my computer (mosquitto) and this is the first step where it (sometimes) fails. I enabled lwip debugging in lwipopts.h. Looking at Wireshark at my computer I can see that the TCP-SYN package reaches the broker and the broker ACK successful. Then the MQTT-library sends a connection request but that never reaches my computer (can't see it in Wireshark) but lwip got it from the library tcp_write: queueing 6510:6540 and sends it tcp_output_segment: 6510:6540 . I can see the tcp_output_segment: 6510:6540 multiple times (retries as no ACK is received I presume...) and sometimes it reaches my computer after a few tries but more often than not it doesn't and the connection attempt times out.
If eventually the MQTT-connection gets established and I try to publish the timestamp (6bytes payload) it almost always reaches the broker. Then the picture follows, gets split up into 1436byte TCP-packages by lwip and gets sent. As with the MQTT-connection attempt I can see the packages getting queued and sent (multiple times if no ACK is recieved) and eventually the sending times out because one or more packages never reach my computer and never get acknowledged.
Sometime it works just fine for a long time... picture and timestamp reach my computer without any problem within a second. Sometimes it takes about half a minute (timeout is 60s) to sent the picture. Sometimes it doesn't sent the picture, sometimes I can't even connect to the MQTT-broker.
I have multiple ESP32 devkit 1 boards I tried. I also tried different WiFi access points. Neither changes the issue at all.

I know this is probably a very very long shot but at this point I'm somewhat desperate...
Maybe someone has the motivation to help me look into this issue... I really would appreciate any help.

Re: Connectifity issues

Posted: Thu Feb 07, 2019 4:05 pm
by Thomseeen
After a few days I found a quiet simple solution altrough I still have no idea what was/is going on.
As stated here https://github.com/espressif/esp-mqtt/issues/48 for larger buffers sent via lwip

Code: Select all

#define CONFIG_USE_ONLY_LWIP_SELECT 1
needs to be set. In menuconfig it can be found at "Component Setting" -> "LWIP" -> "Support LWIP socket select() only".

Re: Connectifity issues

Posted: Sat Feb 09, 2019 4:39 pm
by ESP_cermak
Hi Thomas,

Have you tried the espressif/esp-mqtt from latest idf branch? https://github.com/espressif/esp-mqtt/tree/idf, recent commits actually solve publishing large data (>16k)

If the option `CONFIG_USE_ONLY_LWIP_SELECT` is set to true, then the vfs component is bypassed, so select goes directly to network stack. There's no difference in logic, just a performance impact, however I can imagine some mqtt issues might be theoretically caused by delays on network side, as multiple mqtt messages can be packed to a single tcp packet (this caused some trouble also in esp-mqtt, but also should be resolved by recent updates).

Re: Connectifity issues

Posted: Sun Feb 10, 2019 9:47 am
by Ritesh
Thomseeen wrote:
Thu Feb 07, 2019 4:05 pm
After a few days I found a quiet simple solution altrough I still have no idea what was/is going on.
As stated here https://github.com/espressif/esp-mqtt/issues/48 for larger buffers sent via lwip

Code: Select all

#define CONFIG_USE_ONLY_LWIP_SELECT 1
needs to be set. In menuconfig it can be found at "Component Setting" -> "LWIP" -> "Support LWIP socket select() only".
Hi,

Did you try with MATT Paho eclipse based component which also can be used in replacement of MATT library? As we have used MATT paho library into our project and didn't face any issue like your issue?

Or would you please try with latest stable IDF with ESP-MQTT component?

Still let me know if issue remains same with above option.

Re: Connectifity issues

Posted: Mon Feb 11, 2019 9:02 am
by Thomseeen
ESP_cermak wrote:
Sat Feb 09, 2019 4:39 pm
There's no difference in logic, just a performance impact, however I can imagine some mqtt issues might be theoretically caused by delays on network side, as multiple mqtt messages can be packed to a single tcp packet (this caused some trouble also in esp-mqtt, but also should be resolved by recent updates).
Yes, thats what I have observed as some TCP-packages seem to come through while others don't. With lwip trying to resent the packages the whole MQTT-publish evantually gets recieved correctly or the send-try times out (one could say about 80% of the tries fail with the send timeout set to !!!60s!!!). With `CONFIG_USE_ONLY_LWIP_SELECT` set to true above 95% get recieved properly with the timeout set to just 5s.
Ritesh wrote:
Sun Feb 10, 2019 9:47 am
Hi,

Did you try with MATT Paho eclipse based component which also can be used in replacement of MATT library? As we have used MATT paho library into our project and didn't face any issue like your issue?

Or would you please try with latest stable IDF with ESP-MQTT component?

Still let me know if issue remains same with above option.
I tried the ESP-MQTT library (after managing to include it into my PlatformIO build environment that's still based on IDF 3.1). That library also fails to send larger MQTT-publishes even with `CONFIG_USE_ONLY_LWIP_SELECT` set to true. I observed that my mosquitto broker (or whatever it uses as TCP/IP-stack) almost always responds to the first TCP-package of a larger publish with a RST-flagged TCP-package. While investigating I noticed an error in the way the ESP-MQTT library calculates the `remaining_length` field of the MQTT header. As it's the layer above that shouldn't cause the TCP-RST (to my understanding)... but I have posted the issue on github.
https://github.com/espressif/esp-mqtt/issues/96
https://github.com/espressif/esp-mqtt/issues/48

Re: Connectifity issues

Posted: Tue Feb 12, 2019 10:39 am
by Ritesh
Thomseeen wrote:
Mon Feb 11, 2019 9:02 am
ESP_cermak wrote:
Sat Feb 09, 2019 4:39 pm
There's no difference in logic, just a performance impact, however I can imagine some mqtt issues might be theoretically caused by delays on network side, as multiple mqtt messages can be packed to a single tcp packet (this caused some trouble also in esp-mqtt, but also should be resolved by recent updates).
Yes, thats what I have observed as some TCP-packages seem to come through while others don't. With lwip trying to resent the packages the whole MQTT-publish evantually gets recieved correctly or the send-try times out (one could say about 80% of the tries fail with the send timeout set to !!!60s!!!). With `CONFIG_USE_ONLY_LWIP_SELECT` set to true above 95% get recieved properly with the timeout set to just 5s.
Ritesh wrote:
Sun Feb 10, 2019 9:47 am
Hi,

Did you try with MATT Paho eclipse based component which also can be used in replacement of MATT library? As we have used MATT paho library into our project and didn't face any issue like your issue?

Or would you please try with latest stable IDF with ESP-MQTT component?

Still let me know if issue remains same with above option.
I tried the ESP-MQTT library (after managing to include it into my PlatformIO build environment that's still based on IDF 3.1). That library also fails to send larger MQTT-publishes even with `CONFIG_USE_ONLY_LWIP_SELECT` set to true. I observed that my mosquitto broker (or whatever it uses as TCP/IP-stack) almost always responds to the first TCP-package of a larger publish with a RST-flagged TCP-package. While investigating I noticed an error in the way the ESP-MQTT library calculates the `remaining_length` field of the MQTT header. As it's the layer above that shouldn't cause the TCP-RST (to my understanding)... but I have posted the issue on github.
https://github.com/espressif/esp-mqtt/issues/96
https://github.com/espressif/esp-mqtt/issues/48
Would you please try with MQTT paho Library provided by Eclipse? As we are using that same library into our project and didn't face any issue like you are facing into ESP32-MQTT Library