← index #686Issue #950
Off-topic · high · value 0.190
QUERY · ISSUE

aioble: Connecting to the Apple Media Service (Pico-W)

openby neil-morrison44opened 2023-06-18updated 2024-02-11

Hi,

I'm trying to use a Pi Pico W to connect to the Apple Media Service, but I'm running into a few issues.

async def connectBLE():
    connection = await aioble.advertise(
        _ADV_INTERVAL_MS,
        name="AMS",
        services=[_AMS_UUID],
        appearance=0,
        manufacturer=(0xABCD, b"1234"),
    )
    print("Connection from", connection.device)
    return connection

_AMS_UUID = bluetooth.UUID("89D3502B-0F36-433A-8EF4-C502AD55F8DC")
_AMS_ENTITY_UPDATE = bluetooth.UUID("2F7CABCE-808D-411F-9A0C-BB92BA96C102")
_ADV_INTERVAL_MS = const(250000)

async def main():
    ams_service = aioble.Service(_AMS_UUID)
    aioble.register_services(ams_service)

    connection = await connectBLE()
    ams_service = await connection.service(_AMS_UUID)
    ams_entity_update_characteristic = await ams_service.characteristic(
        _AMS_ENTITY_UPDATE
    )
    await ams_entity_update_characteristic.subscribe(notify=True)

This'll connect, find the services, characteristics but then throws:

Traceback (most recent call last):
  File "<stdin>", line 160, in <module>
  File "uasyncio/core.py", line 1, in run
  File "uasyncio/core.py", line 1, in run_until_complete
  File "uasyncio/core.py", line 1, in run_until_complete
  File "<stdin>", line 139, in main
  File "aioble/client.py", line 430, in subscribe
  File "aioble/client.py", line 295, in write
GattError: 5

On the .subscribe(notify=True) bit, not sure how / if I can look up the GattError: 5 bit for more info?

There is a CircuitPython implementation here, https://docs.circuitpython.org/projects/ble_apple_media/en/latest/index.html

Which uses a "Service Solicitation" advertisement - what's the method for constructing that in aioble? I'm guessing custom adv_data & resp_data but I'm unsure of the format, etc.

That implementation also pairs to the connecting device, but when I insert a await connection.pair() I get

Traceback (most recent call last):
  File "<stdin>", line 161, in <module>
  File "uasyncio/core.py", line 1, in run
  File "uasyncio/core.py", line 1, in run_until_complete
  File "uasyncio/core.py", line 1, in run_until_complete
  File "<stdin>", line 124, in main
  File "<stdin>", line 99, in connectBLE
  File "aioble/device.py", line 256, in pair
  File "aioble/security.py", line 171, in pair
ValueError: unknown config param

Which I guess is due to the Pico-W implementation not (currently?) supporting something in ble.config(bond=bond, le_secure=le_secure, mitm=mitm, io=io)

Thanks

16 comments
sebastian-blohm · 2023-08-12

Hi!
Were you able to find out what GattError: 5 means? I'm seeing the same in a similar context.

neil-morrison44 · 2023-08-12

No I couldn’t work it out, dug into the code a bit to see if I could find it’s origin but even that didn’t help much (though I might not have gone to a deep enough layer)

puppet13th · 2023-09-20

@neil-morrison44 if you want to pico-w to connect to anything you should use central role( scan and connect ) like this aioble example. the code that you're using now is for peripheral mode( aioble.advertise )

neil-morrison44 · 2023-09-20

@puppet13th peripheral mode is correct for the Apple Media Service, the phone is the central device.

puppet13th · 2023-09-20

from this picture there is MS and MR. what is pico role there?

puppet13th · 2023-09-20

@neil-morrison44 after connection try add these line:

await connection.exchange_mtu()
await connection.pair( bond=False, le_secure=False, mitm=False )

if those config not implemented for pico-w yet you can edit security.py comment out 171 line:
ble.config(bond=bond, le_secure=le_secure, mitm=mitm, io=io)

neil-morrison44 · 2023-09-20

@puppet13th puppet13th it'd be the MR device - I believe that's roughly what I tried back in June which threw the error, but it may well be supported now if there's been updates so I'll have another go on the latest MicroPython at some point & update this

puppet13th · 2023-09-21

@neil-morrison44 i have not use any pico-w board yet, have you check if ble pair implemented in pico-w port? on esp32 port pair only implemented since firmware version 1.20

foxt · 2023-11-13

Yeah - it seems like BLE.gap_pair is not present on the Pico W, so aioble.pair will fail with AttributeError: 'BLE' object has no attribute 'gap_pair'

hoihu · 2024-02-04

I'm facing the same issue with my raspberry pico w. I'm trying to read the current time service characteristic from an iPhone.

I guess the gatt error 5 is the GATTS_ERROR_INSUFFICIENT_AUTHENTICATION as specified in https://docs.micropython.org/en/latest/library/bluetooth.html#event-handling

That makes somewhat sense when the peripheral is not paired / bonded to the iPhone. However, when adding the await connection.pair() to the code, it fails with ValueError: unknown config param .

As I interepret the code in modbluetooth.c this could only be case when the bonding support is not compiled in. But MICROPY_PY_BLUETOOTH_ENABLE_PAIRING_BONDING seems to be defined in the the make file (?)

Also note that the example from the official repo https://github.com/micropython/micropython/blob/master/examples/bluetooth/ble_bonding_peripheral.py does not seem to work anymore, giving the same fault at https://github.com/micropython/micropython/blob/master/examples/bluetooth/ble_bonding_peripheral.py#L70

(using the nightly build version as of 3.2.2024)

hoihu · 2024-02-04

But MICROPY_PY_BLUETOOTH_ENABLE_PAIRING_BONDING seems to be defined in the the make file (?)

huh looking into it it may not be... since it uses btstack not nimble (at least from the build output)? Means bonding support is not enabled on the rp2 port?

andrewleech · 2024-02-04

Yeah I'm pretty sure bonding is currently only supported on stm32

hoihu · 2024-02-04

Yeah I'm pretty sure bonding is currently only supported on stm32

thanks for the feedback. Yeah I feared that a bit... pitty since bonding is sometimes required to readout characteristics.

studying the release notes: https://micropython.org/resources/micropython-ChangeLog.txt it was mentioned that the esp32 port is also supporting bonding from 1.20 onward... but haven't tested that. I'll give it a shot with a pyboard d.

In general, it would be good to have an overview with a list of ports: <ble_features>.

hoihu · 2024-02-05

Ok so the bonding support is not enabled on the pico w, but it can be done via

diff --git a/ports/rp2/CMakeLists.txt b/ports/rp2/CMakeLists.txt
index 3a46fdaa7..2ced4e86e 100644
--- a/ports/rp2/CMakeLists.txt
+++ b/ports/rp2/CMakeLists.txt
@@ -284,6 +284,7 @@ if (MICROPY_BLUETOOTH_BTSTACK)
 
     target_compile_definitions(${MICROPY_TARGET} PRIVATE
         MICROPY_BLUETOOTH_BTSTACK=1
+        MICROPY_PY_BLUETOOTH_ENABLE_PAIRING_BONDING=1
         MICROPY_BLUETOOTH_BTSTACK_CONFIG_FILE=\"btstack_inc/btstack_config.h\"
     )

recompiling and then the ValueError: unknown config param faults are disappearing. On the mobile device, a pairing request pops up (depending on the selected pairing option). So far so good.

However it doesn't continue any further from there. If I confirm it on the mobile it doesn't seem to continue the bonding process.

I then enabled the logs via aioble.core.log_level = 3 and checked if there are any incoming events that are not handled, but nope.

andrewleech · 2024-02-05

Yeah the problem is a bunch of C functions would need to be written to recieve any pairing and bonding events from btstack and convert those events into the form that aioble expects. I wrote some of the initial stm versions of these a few years ago for nimble, but btstack was not supported on my hardware (stm32wb55) so I wasn't able to prepare the code for that stack.

If you take a look in https://github.com/micropython/micropython/blob/master/extmod/nimble/modbluetooth_nimble.c at everywhere enabled by MICROPY_PY_BLUETOOTH_ENABLE_PAIRING_BONDING you'll see the bulk of the code supporting this feature, there probably needs to be more of this for btstack in https://github.com/micropython/micropython/blob/master/extmod/btstack/modbluetooth_btstack.c (though some support is already there from https://github.com/micropython/micropython/commit/a1fcf301217b34ae74fbb937a9488be5bc6e61a5 )

hoihu · 2024-02-11

I'll give it a shot with a pyboard d.

so I tried - but faced isses... see. https://github.com/micropython/micropython/issues/13639

@andrewleech thanks for the pointers! I had a look around those files and I also think it should be doable. But what I really need is a working (uPy-) example of a platform where bonding is known to work, to get a feeling of how the interactions works during the bonding process. For example, it's still unclear to me if addr_mode set to 2 is a requirement for bonding.

CANDIDATE · ISSUE

[aioble] After timeout, reconnection is successful. (Peripherals are always powered off)

openby 63rabbitsopened 2024-12-22updated 2025-05-27

The other device is powered off. However, after a connection timeout, it appears to be connected when retried. In fact, it is failing.

I'm not a native speaker, so sorry if my writing is wrong.

  • Environment

    • Raspberry Pi Pico W with RP2040
    • MicroPython v1.24.0 on 2024-10-25
    • IDE:Thonny
  • Steps to reproduce

    1. Power off the peripheral device.
    2. Timeout occurs when trying to connect.
    3. Retry and the connection succeeds. (It actually fails.)
  • Code used for testing

from micropython import const
import asyncio
import aioble
# import test_aioble as aioble

import sys


_TARGET_DEVICE_ADDR = const("d7:de:4c:f4:56:e5")


async def gather(ai):
    ret=[]
    async for x in ai: ret.append(x)
    return ret


async def device_details(connection):

    if connection is None:
        return

    try:
        services = await gather(connection.services())
    except asyncio.TimeoutError as e:
        print("Timeout. (discovering services)")
        sys.print_exception(e)
    except Exception as e:
        print("Error (discovering services): {}".format(e))
        sys.print_exception(e)
    else:
        if services is None or len(services) <= 0:
            print("\n\"Service\" not found.")
            return
        
        for s in services:
            print("\t", s)


async def print_details(device):

    connection = None

    # connecting to device
    try:
        print("\nConnecting to {} ... ".format(device), end="")
        connection = await device.connect(timeout_ms=2000)
    except asyncio.TimeoutError as e:
        print("Timeout.")
        sys.print_exception(e)
    except Exception as e:
        print("Error: {}".format(e))
        sys.print_exception(e)
    else:
        print("Connected.")
        await device_details(connection)
        await connection.disconnect()


async def main():

    device = aioble.Device(aioble.ADDR_RANDOM, _TARGET_DEVICE_ADDR)

    while True:
        await print_details(device)
        r = input("\nenter return (to retry) / q(uit). > ").strip()
        if r.upper() == "Q":
            sys.exit(0)


asyncio.run(main())

  • Output
MPY: soft reboot
MicroPython v1.24.0 on 2024-10-25; Raspberry Pi Pico W with RP2040

Type "help()" for more information.

>>> 

>>> %Run -c $EDITOR_CONTENT

MPY: soft reboot

Connecting to Device(ADDR_RANDOM, d7:de:4c:f4:56:e5) ... Timeout.
Traceback (most recent call last):
  File "<stdin>", line 47, in print_details
  File "aioble/device.py", line 149, in connect
  File "aioble/central.py", line 140, in _connect
  File "aioble/device.py", line 94, in __exit__
TimeoutError: 

enter return (to retry) / q(uit). > 

Connecting to Device(ADDR_RANDOM, d7:de:4c:f4:56:e5, CONNECTED) ... Connected.
Error (discovering services): can't convert NoneType to int
Traceback (most recent call last):
  File "<stdin>", line 24, in device_details
  File "<stdin>", line 14, in gather
  File "aioble/client.py", line 128, in __anext__
  File "aioble/client.py", line 120, in _start
  File "aioble/client.py", line 193, in _start_discovery
TypeError: can't convert NoneType to int

enter return (to retry) / q(uit). > 

  • How to fix it
    I added exception handling because of the following problem. (It is not a good idea to fix it in this place.)
    • When the timeout occurs, the variable "_connection" in aioble is not in the correct state.
    • When the timeout occurs, the internal state of the module "bluetooth" seems to remain connected.
# https://github.com/micropython/micropython-lib/blob/master/micropython/bluetooth/aioble/aioble/central.py#L107

async def _connect(
    connection, timeout_ms, scan_duration_ms, min_conn_interval_us, max_conn_interval_us
):

                << Omitted >>

    try:
        with DeviceTimeout(None, timeout_ms):
            ble.gap_connect(
                device.addr_type,
                device.addr,
                scan_duration_ms,
                min_conn_interval_us,
                max_conn_interval_us,
            )

            # Wait for the connected IRQ.
            await connection._event.wait()
            assert connection._conn_handle is not None

            # Register connection handle -> device.
            DeviceConnection._connected[connection._conn_handle] = connection
    except:
        device._connection = None
        ble.gap_connect(None)
        raise
    finally:
        # After timeout, don't hold a reference and ignore future events.
        _connecting.remove(device)

9 comments
brianreinhold · 2025-01-21

I am facing a similar issue which appears to be related. I have a crappy device that after it disconnects it continues to send trailing connectable advertisements but when a connection attempt is made, the device simply does not respond. Its advertisements also cease. Eventually the connection attempt times out, but the system remains 'connected' in some sense and the system is corrupted. If I take a measurement with another device that is known (a bonded reconnect) I get EINVAL 22 errors all over the place. If, instead, I try and connect to the original device that had the trailing advertisements I get a connection event in the aioble event handler but nothing further happens. The only recovery is to reboot that application.

brianreinhold · 2025-01-22

I think I have found the reason for the issue and the solution. When the connection attempt times out, aioble fails to cancel the ongoing connection. If you then make tour device discoverable again, the connection will complete (the IRQ gets handled) but there is no longer any support for it so it goes nowhere. If you try to connect with another device, when aioble calls the ble.gap_connect() method it is already established for a different device and you will get EINVAL 22 errors like crazy.

I was able to solve the issue by calling ble.gap_connect(None) which does a cancel at the low levels in the device.py file

        try:
            if exc_type == asyncio.CancelledError:
                # Case 2, we started a timeout and it's completed.
                if self._timeout_ms and self._timeout_task is None:
                    ble.gap_connect(None)  # Cancel ongoing connection
                    raise asyncio.TimeoutError

I am assuming this timeout get signaled ONLY when the connection attempt times out.

63rabbits · 2025-01-25
  • About ble.gap_connect(None):

    Since DeviceTimeout is used in read/write/notify/indicate for timeout, it is better not to call ble.gap_connect(None) in device.py. It may not have a negative effect, as it only cancels the connection attempt. However, it is better not to run it.
    As ble.gap_connect() is only called in one place in central.py, you can limit the extent of its effect by running ble.gap_connect(None) nearby.

# https://github.com/micropython/micropython-lib/blob/master/micropython/bluetooth/aioble/aioble/client.py#L253

        with self._connection().timeout(timeout_ms):

  • About "device._connection = None":

    At the moment I am not sure that it is correct to clear _connection directly in the exception handler as described above. Do you have a better idea?

brianreinhold · 2025-01-31

I would like to be able to call the cancel connect on the timeout of the connect attempt, but it was not clear how to do that. I could not find a way to identify the cause of the timeout so I handled it in the timeout. So far it solves the problem I have and has not introduced a problem.

How would I do this?
you can limit the extent of its effect by running ble.gap_connect(None) nearby.

I could not see how to do it in this block of code

with self._connection().timeout(timeout_ms):

which is where I would really like to do it.

63rabbits · 2025-01-31

Sorry for the poor explanation. (I use a translator.)

Call ble.gap_connect(None) in the exception handler as follows.
# ble.gap_connect() is only executed at this location, so it is better to execute ble.gap_connect(None) near here.

# https://github.com/micropython/micropython-lib/blob/master/micropython/bluetooth/aioble/aioble/central.py#L107

async def _connect(
    connection, timeout_ms, scan_duration_ms, min_conn_interval_us, max_conn_interval_us
):

                << Omitted >>

    try:
        with DeviceTimeout(None, timeout_ms):
            ble.gap_connect(
                device.addr_type,
                device.addr,
                scan_duration_ms,
                min_conn_interval_us,
                max_conn_interval_us,
            )

            # Wait for the connected IRQ.
            await connection._event.wait()
            assert connection._conn_handle is not None

            # Register connection handle -> device.
            DeviceConnection._connected[connection._conn_handle] = connection
#     except:
    except asyncio.TimeoutError:
        device._connection = None   # Clear connection.
        ble.gap_connect(None)       # Cancel ongoing connection.
        raise
    finally:
        # After timeout, don't hold a reference and ignore future events.
        _connecting.remove(device)

In the following, the DeviceTimeout class is used to show where ‘read’ starts the timeout.

with self._connection().timeout(timeout_ms):

In the code below, gap_connect(None) is called on a 'read' timeout.

        try:
            if exc_type == asyncio.CancelledError:
                # Case 2, we started a timeout and it's completed.
                if self._timeout_ms and self._timeout_task is None:
                    ble.gap_connect(None)  # Cancel ongoing connection
                    raise asyncio.TimeoutError
anacrolix · 2025-05-23

I statically analyzed the code and found the same issue. There's a missing gap_connect(None) after timeouts. I can make a PR?

63rabbits · 2025-05-23

Yes, please create a PR.
I've never done PR.
I was wondering what to do.
Thanks.

brianreinhold · 2025-05-23

The company I worked for folded so I am not doing this anymore and have gone back to meteorological research vs Bluetooth health devices. I will say that the fix I used has worked and has not caused any unwanted side effects in operation.

Some other issues you might want to look at in this PR is better handling of disconnects from the remote peer. There needs to be a more direct connection between aioble and the application to assure that the application does not try to do something as if connected when in fact it is disconnected. As it is know, by the time the application gets signaled of the disconnect the application can try and do something that it should not do in the disconnected state. That will cause catastrophic failures in aioble. I had to create an application 'global' that was set in aioble in the disconnect callback. The application would use this variable to check the connection state instead of the aioble APIs.

The other big ticket item is pairing/bonding support. It does not exist for the PICO-W. We had to do a lot of low level work to support bonding and bonded reconnects using the BlueKitchen btStack. We could not operate with many medical health devices without it. In this task one needs to consider

  • peripheral security requests
  • insufficient authentication errors

which are handled differently in the btStack (unfortunately). The btStack handles the security requests internally (but the application does not know it so you have to add code to tell the application that pairing/encryption has happened). The btStack does not handle the insufficient authentication error and that needs to be passed up to the application so it can invoke pairing/encryption

  • pairing passkeys also need to be supported.
anacrolix · 2025-05-27

The question was more for the maintainers I don't want to make a PR if they're overwhelmed already.

Keyboard

j / / n
next pair
k / / p
previous pair
1 / / h
show query pane
2 / / l
show candidate pane
c
copy suggested comment
r
toggle reasoning
g i
go to index
?
show this help
esc
close overlays

press ? or esc to close

copied