QUERY · ISSUE

umqtt.simple got OSError: -1 after publishing 65535 packets message on esp32

openby mzhboyopened 2023-10-21updated 2023-10-21

platform: MicroPython v1.21.0 on 2023-10-06; Generic ESP32 module
broker: mosquitto
description:
got OSError: -1 after sending 65535 packets

Result:

pkcnt 65000
Traceback (most recent call last):
File "<stdin>", line 47, in <module>
File "<stdin>", line 40, in publish_test
File "umqtt/simple.py", line 144, in publish
File "umqtt/simple.py", line 184, in wait_msg
OSError: -1

code:

from umqtt.simple import MQTTClient
import time
import ujson as json
from machine import unique_id
import sys, os

def str_current_time():
    tm_ns = time.time_ns()
    tm_rem_ms = (tm_ns % 1_000_000_000) // 1_000_000
    tm = time.localtime(tm_ns//1_000_000_000)
    str_tm = str(tm[3])+':'+str(tm[4])+':'+str(tm[5])+'.'+ str(tm_rem_ms)
    return str_tm
   

id = unique_id() #machine.unique_id()
id_str = '{:02x}{:02x}{:02x}{:02x}'.format(id[0], id[1], id[2], id[3])

mq_id = id_str
mq_machine = os.uname().machine.split()[0]
mq_server = '192.168.12.25'
mq_user='esp32-srv'
mq_pass='123456'
mq_topic= b'esp/test'
mq_message = {'id':mq_id, 'machine': mq_machine, 'time': str_current_time(), 'hello': 'hello'}
mqClient = MQTTClient(mq_id, mq_server, port=1883, user=mq_user, password=mq_pass, keepalive=60*60*12)
mqClient.connect(clean_session=True) # only support clean_sesson=True
mqClient.publish(b'esp/hello', json.dumps(mq_message), qos=1)

pkcnt = 0x00_00_00_01
pkcnt_max = 999_999

def publish_test():
    global pkcnt, pkcnt_max
    pkcnt += 1
    if pkcnt >= pkcnt_max :
        pkcnt = 1
    tm = time.time()
    message = {'id':mq_id, 'machine': mq_machine, 'time': tm, 'current': 1, 'ctrl': 1, 'pkcnt':pkcnt}
    mq_message = json.dumps(message)
    mqClient.publish(mq_topic, mq_message, qos=1)
    if pkcnt % 1000 == 0:
        print('pkcnt',pkcnt, 'time', tm)

if __name__ == '__main__' :
    print('test start')
    while True :
        publish_test()

2 comments

mzhboy · 2023-10-21

also tried reconnect after exception, didn't work

    try :
        mqClient.publish(mq_topic, mq_message, qos=1)
    except OSError as err:
        print('error: when publish,', err)
        try :
            mqClient.connect()
        except Exception as err2 :
            print('error: when reconnect,', err2)

mzhboy · 2023-10-21

how ever mqtt_as has no issue

"pkcnt": 147481,

CANDIDATE · ISSUE

"OSError: -1" if use keepalive>0

openby leech001opened 2017-10-01updated 2023-05-26

Traceback (most recent call last):
File "main.py", line 55, in <module>
File "umqtt/simple.py", line 202, in check_msg
File "umqtt/simple.py", line 171, in wait_msg
OSError: -1

10 comments

dpgeorge · 2017-10-03

Please try using the latest umqtt library and then provide a minimal reproducible test case that shows the error.

kevinkk525 · 2017-10-30

guess i'll just take over this issue thread ;)

when using keepalive>0, server disconnects from client and client raises an exception in given line 171 in wait_msg. So the socket disconnection is being correctly detected.

disregard my error description below! (only if you have the same problem and need a solution)
the solution was to connect to the mqtt server with mqtt.connect(False) as only then a persistent session is started and the subscriptions are stored between connection problems. see #186
I forgot that in the connect method at the beginning and did it only in the reconnect function..
the only problem i have left is that with a restart of mosquitto, mosquitto loses all subscriptions although persistence is on. but this is not an issue of this library.

now to the original issue:
i did set a last will message that is correctly send by the server.
After this exception my client tries to reconnect and does it correctly (mqtt.connect(False) or True, does not make a difference here).
I confirm the connection by sending out an online status from the client.
Sending a message to the client now does nothing. the client just does not appear to get the message.

my thoughts on this so far:

connection is there as the client sends a message to the server
function check_msg is called without an exception as i'm monitoring it with an output
the set callback is not called

Steps to reproduce:

set last will
catch exception in wait_msg, reconnect and let it send a message for proof
connect to server with keepalive>0 without sending any messages or pings
subscribe to a topic and have a callback printing new messages
wait until connection breaks
last will is being sent and shortly after that client sends the reconnection message proofing the connection
send something to the subscribed topic
that message should show up but it does not

maybe i have a misconfigured server, won't exclude that possibility although it would be rather strange if that is the problem.
I'm using a recent version of the mosquitto server.

Actpohomoc · 2017-11-22

I had same problem with continue disconnecting from the mqtt server with keepalive > 0.
But I figure it out that disconnection happens because there was no any activity from my board with mqtt server during the period of time like keepalive value of seconds.
So if the board will publish any mqtt message ("i'm alive") - the connection will not drop.
I program to send the current board time to the debug topic during the half of the keepalive value of seconds. And it is working like a charm!

If I correct this issue is not an issue it is a feature.

If you set keepalive value - you have to send something to mqtt server to inform that you are alive in less time than keepalive value of seconds.

Actpohomoc · 2017-11-22

I'm not sure is it task for mqtt client to keep connection alive in time.
The umqtt/simple.py lib is only sending information to the server about keepalive time.
Nothing more. It looks that this task to user who use this lib.

MQTT v3.1.1 specification

The Keep Alive is a time interval measured in seconds. Expressed as a 16-bit word, it is the maximum time interval that is permitted to elapse between the point at which the Client finishes transmitting one Control Packet and the point it starts sending the next. It is the responsibility of the Client to ensure that the interval between Control Packets being sent does not exceed the Keep Alive value. In the absence of sending any other Control Packets, the Client MUST send a PINGREQ Packet [MQTT-3.1.2-23].

The Client can send PINGREQ at any time, irrespective of the Keep Alive value, and use the PINGRESP to determine that the network and the Server are working.

If the Keep Alive value is non-zero and the Server does not receive a Control Packet from the Client within one and a half times the Keep Alive time period, it MUST disconnect the Network Connection to the Client as if the network had failed [MQTT-3.1.2-24].

If a Client does not receive a PINGRESP Packet within a reasonable amount of time after it has sent a PINGREQ, it SHOULD close the Network Connection to the Server.

A Keep Alive value of zero (0) has the effect of turning off the keep alive mechanism. This means that, in this case, the Server is not required to disconnect the Client on the grounds of inactivity.
Note that a Server is permitted to disconnect a Client that it determines to be inactive or non-responsive at any time, regardless of the Keep Alive value provided by that Client.

Non normative comment
The actual value of the Keep Alive is application specific; typically this is a few minutes. The maximum value is 18 hours 12 minutes and 15 seconds.

kevinkk525 · 2017-11-22

that's why I call mqtt.ping() regularly. Together with last will this shows the device state in home-assistant.
But I do not check if the server actually answers the request. If the server is down, the socket breaks anyways, throwing an exception, so the only problem left would be a mqtt server that is stuck, but this should be really rare.
I'm using about a minute as keepalive.

kiscore · 2018-12-05

The problem remains unsolved.
I also have this problem.

leech001 · 2018-12-08

Hi! look this project https://github.com/leech001/Security-fire-alarm-ESP8266
I think I solved the problem there

xhlinux · 2021-04-20

keepLiveTime = 120
c = MQTTClient(CLIENT_ID, server, 1883, username, password, keepLiveTime)

tim = Timer(-1)
tim.init(period=keepLiveTime*1000, mode=Timer.PERIODIC, callback=lambda t: keepLiveTimeCb(c))

def keepLiveTimeCb(c):
c.ping()

shell use ping() func

ian-llewellyn · 2023-05-26

I've explored this issue and its comments and believe it can be closed:

In the OPs example, the behaviour is expected - the client must periodically ping() the broker, or otherwise send the broker some traffic to communicate that it's still alive.
The "original issue" outlined by @kevinkk525 seems to be a different problem: messages not received after reconnecting - There are many reasons why this can occur: broker not set to support sessions, connect() called without clean_session=False, etc. If it is reproducible, some code, and a new issue would be great. However, it looks like this was resolved using a periodic ping()
Other comments contain helpful tips that could be considered for inclusion as examples.

kiscore · 2023-05-26

您好，您的邮件已收到。

umqtt.simple got OSError: -1 after publishing 65535 packets message on esp32

"OSError: -1" if use keepalive>0

Keyboard