MQTT Disconnect socket write then close
When there are network issues and I want to disconnect from mqtt client, the socket writes are blocking and cannot be changed (I havent been successful) so it will just hang on the self.sock.write(b"\xe0\0") line in the disconnect function of simple.py. Is this a necessary action, or can I not write that byte array and just close the socket so that I do not have to worry about hanging on the write indefinitely?
UMQTT: Library stops receiving broker responses after a subscribed message arrives
MicroPython v1.24.1
Hardware: Raspberry Pi Pico W
Attempted using umqtt.simple v1.5.0 installed via Thonny package manager, as well as manually installing most recent simple.py and robust.py files from GitHub.
I have an IoT application I'm creating with a Raspberry Pi Pico W, and have been trying to get either the umqtt.simple or .robust tools to work for several days now, but am running into a wall where upon receipt of certain messages, the library just freezes and will no longer receive ANY messages until the client connection is completely torn-down and reconnected. I initially thought that this might be some kind of networking issue or socket issue, but it appears to be a function of inbound messages arriving on a subscribed topic.
I've modified simple.py to add a debug print() inside of wait_msg() (right under the "res = self.sock.read(1)") so I can see the raw byte(s) coming back from the socket read, and try and determine where the problem is coming in. I've also implemented a standard ICMP Ping to be able to test network connectivity at the same time as MQTT connectivity, to rule out network-related issues. It's resulted in useful debug output, but I'm no closer to a solution.
Simplified code example:
import network
import ubinascii
import time
from icmp import ping
from umqtt.simple import MQTTClient
from config import *
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect(WIFI_SSID, WIFI_PASSWORD)
time.sleep(5)
def mqtt_callback(topic, msg, retained=False, properties=None):
message = msg.decode()
topic = topic.decode()
print(f"Received message: {message} on topic: {topic}, Retained: {retained}, Properties: {properties}")
client_id = ubinascii.hexlify(machine.unique_id()).decode()
client = MQTTClient(client_id, MQTT_BROKER, MQTT_PORT, MQTT_USER, MQTT_PASSWORD)
client.set_callback(mqtt_callback)
client.connect(clean_session=False)
client.subscribe(MQTT_TOPIC_FAN_CONTROL)
client.subscribe(MQTT_DISCOVERY_REQUEST_TOPIC)
client.subscribe(MQTT_TOPIC_FURNACE_STATE)
ping_counter = 0
while True:
if ping_counter >= 10:
print("Pinging ICMP and MQTT")
ping()
client.ping()
ping_counter = 0
else:
ping_counter += 1
client.check_msg()
time.sleep(1)
So it checks for a new message every second, and every 10 seconds it both ICMP pings the MQTT broker host, and MQTT pings the broker service.
Initial output:
Received byte from MQTT server: b'\x90'
Received byte from MQTT server: b'\x90'
Received byte from MQTT server: b'\x90'
Received byte from MQTT server: b'1'
Received message: ON on topic: testtopic/result, Retained: True, Properties: False
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
...
Pinging ICMP and MQTT
ICMP TX(18) : 08 00 16 f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46
ICMP RX(18) : 00 00 1e f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46
IP Version = v4
IP Header = 20
IP Length = 44
Protocol = 0x01
Source = 192.168.1.237
Destination = 192.168.1.131
ICMP Length = 24
ICMP Type = 00
ICMP Code = 00
Checksum = Passed
Identifier = 1234
Sequence N = 0001
ICMP Data = 0123456789ABCDEF
Received byte from MQTT server: None
Received byte from MQTT server: b'\xd0'
Received byte from MQTT server: None
Received byte from MQTT server: None
So, setup proceeds as expected. I subscribe to 3 topics, and get back 3x "\x90" acknowledgement messages. I've configured the messages I'm sending to the broker to be retained, so it then immediately receives a "1" and hits the callback for a received message. A few seconds go by, and we hit our first Ping. The ICMP ping succeeds, so the network is up. A second later, the MQTT server replies with a "\xd0" ping acknowledgement byte. All good so far!
This will run for some arbitrary length of time just fine. And, as far as I can tell, if no messages are sent to subscribed topics, it will run forever. But eventually, after some arbitrary number of messages received, the library just stops responding, even though the network is verifiably up, as is the MQTT broker:
Pinging MQTT Server.
ICMP TX(18) : 08 00 16 f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46
ICMP RX(18) : 00 00 1e f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46
IP Version = v4
IP Header = 20
IP Length = 44
Protocol = 0x01
Source = 192.168.1.237
Destination = 192.168.1.131
ICMP Length = 24
ICMP Type = 00
ICMP Code = 00
Checksum = Passed
Identifier = 1234
Sequence N = 0001
ICMP Data = 0123456789ABCDEF
Received byte from MQTT server: None
Received byte from MQTT server: b'\xd0'
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None **<==== MESSAGE RECEIVED APPROXIMATELY HERE**
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Pinging MQTT Server.
ICMP TX(18) : 08 00 16 f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46
ICMP RX(18) : 00 00 1e f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46
IP Version = v4
IP Header = 20
IP Length = 44
Protocol = 0x01
Source = 192.168.1.237
Destination = 192.168.1.131
ICMP Length = 24
ICMP Type = 00
ICMP Code = 00
Checksum = Passed
Identifier = 1234
Sequence N = 0001
ICMP Data = 0123456789ABCDEF
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Pinging MQTT Server.
ICMP TX(18) : 08 00 16 f3 12 34 00 02 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46
ICMP RX(18) : 00 00 1e f3 12 34 00 02 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46
IP Version = v4
IP Header = 20
IP Length = 44
Protocol = 0x01
Source = 192.168.1.237
Destination = 192.168.1.131
ICMP Length = 24
ICMP Type = 00
ICMP Code = 00
Checksum = Passed
Identifier = 1234
Sequence N = 0002
ICMP Data = 0123456789ABCDEF
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
The above output snip shows the point where a message is received. We have a good ping at the beginning. Approximately 5 seconds later, a message is sent on a subscribed topic. The message never arrives, but more importantly neither does anything else, ever again. Even though pings are still being sent, the acknowledgement from the server is never received. ICMP pings to the MQTT broker continue to succeed. The only way to restore operation is to tear down and reconnect the entire MQTT client.
I'm completely baffled and have been unable to determine the cause. Any assistance would be appreciated.
Did you ever find out a solution to this? I think my code is behaving the same way...
Regrettably, no. I tried multiple solutions but was never able to even figure out if it was a bug in the library, or something unique to the RPi Pico. Either way, I ended up swapping to a different MQTT library and the app is working. But I'll have to reply again later to let you know which library that was. I dead-ended here. Sorry.
Instead of the method
ping(), usecheck_msg(). AFIK,wait_mgs()andcheck_msg()handle the callbacks and then usesping()internally. I guess if you use only ping, incoming messages are received, but the callbacks aren't called, so you never see the message of your subscribed topics.https://mpython.readthedocs.io/en/v2.2.1/library/mPython/umqtt.simple.html#umqtt.simple.MQTTClient.check_msg
Yea I have it subscribe to MQTT and only use check_msg(). After a while it dies, so I started publishing messages within the keep alive period, and that would go on better, but after a while it still randomly hangs on subscribe (I checked the broker and after a while it stops receiving messages from the pico). I also don't know if it's MQTT, micropython, or the pico that is causing it. I even have the bug where it won't connect to MQTT unless I power cycle it first. Strange.
@darloxflyer if you don't mind letting me know the library that would be great.
Otherwise, I've set up using the C SDK to try to make this work reliably, but haven't gotten to work on it yet.
Because I am a terrible person who doesn't follow the advice he imposes on his own staff... I apparently never checked the latest version of that code into Github. So I have to get it off of my Dev box at home, and I'm away for the holidays until Jan 5. Sorry for the delay but will reply ASAP.
No rush, Happy Holidays!
Hi Folks, found this issue because it seems I have an equal behavior on esp8266.
Difference: I use mqtt.robust. simple crashed once, and I changed quickly.
One additonal information: It seems to run more stable when I monitor with thonny. After starting without the device seem to stop receiving within a couple of minutes. If it happens while thonny debugging the received callback seems not to be called at all - explice prints not shown.
On my device the problem dies not occur after the first received message, but somehow later.
Mqtt is quite sure up and working - I receive published messages from the same device and monitored with the sending device that the message is distributed. Next days I'll check my server log about messages.
I tried a lot of things today with shorter/longer/no sleep - without success. Exception handling is increased up to machine.reset including reconnect - but it seems this does not occure.
I would be interested on further results as well and can check in the next days.
At the moment I try with my full code what is just measure and publish a couple of DS1820 values every 10s. No sleep, nothing special, no timers, just a simple state machine based on tickcounts. AFAIK no exceptions/errors beside the topic with the subscribed topic.
BR hufnala
I ended up using: https://github.com/peterhinch/micropython-mqtt
It's a LOT heavier weight than this implementation, and so for send-only MQTT messages, I continued to use this library. But for anything that needed to receive messages, I swapped to that other package. Leaves very little room for much else on a Pico W, however, so your mileage may vary in terms of usefulness.
Thanks, this worked great (once I figured out some python async things I forgot about). Running for over a day without issues.