BLESession.setNotifications: Lookup CCCD to start or stop notifications

Per Bluetooth Core Specification Vol.3, Part G, Section 3.3.1.1, if
Notify property is set for a Characteristic, the Client Characteristic
Configuration Descriptor, or CCCD, shall exist. As noted in section
3.3.3.3 of the specification, notification of the Characteristic is
enabled or disabled by setting bit 0 of the CCCD.

This CCCD exists within the characteristic definition following the
Characteristic Value declaration, and section 3.3.3 of the specification
warns that the order of characteristic declaration shall not be assumed.
However, current pyscrlink code assumes that the CCCD comes just after
the Characteristic Value and CCCD handle is obtained just adding 1 to
the handle of the Characteristic Value. This works many of the BLE
devices but does not work all of them.

To get correct CCCD handle, look up descriptors of the Characteristic to
find out the CCCD.

Signed-off-by: Shin'ichiro Kawasaki <kawasaki@juno.dti.ne.jp>

README.md

@@ -42,7 +42,7 @@ Linux distros:
 * elementary OS 5.1 Hera
 
 Browsers:
-* FireFox
+* Firefox
 * Chromium
 
 It was reported that pyscrlink (former bluepy-scratch-link) working with
@@ -51,7 +51,7 @@ following devices and Linux distros.
 Devices:
 * LEGO Mindstorm EV3 by @chrisglencross
 * LEGO WeDo by @zhaowe, @KingBBQ
-* LEGO Boost by @laurentchar, @miguev, @jacquesdt
+* LEGO Boost and compatible devices by @laurentchar, @miguev, @jacquesdt, @n3storm
 * Intelino Smart Train by @ErrorJan
 * toio by @shimodash
 
@@ -59,6 +59,7 @@ Linux distros:
 * Raspbian by @chrisglencross
 * Ubuntu 16.04 @jacquesdt
 * Ubuntu Studio 20.04 @miguev
+* Debian 11 @n3storm
 
 Installation
 ------------
@@ -87,7 +88,8 @@ Installation
     $ pip3 install pyscrlink
     ```
 
-4. For Bluetooth Low Energy (BLE) devices, set bluepy-helper capability.
+4. Set bluepy-helper capability.
+   This step is required for most of devices except LEGO Mindstorm EV3.
 
     ```
     $ bluepy_helper_cap
@@ -159,6 +161,8 @@ Usage
     ```sh
     $ scratch_link
     ```
+    If your device is toio, add "-s 1" option to the scratch_link command. It
+    allows the toio Do Visual Programming to connect to toio automatically.
 
 3. Connect scratch to the target device such as micro:bit or LEGO Mindstorms:
     * Open FireFox or Chrome. (Make sure to run as the same user for scratch-link python script.)
@@ -202,6 +206,15 @@ Please file issues to [GitHub issue tracker](https://github.com/kawasaki/pyscrli
 Releases
 --------
 
+Release 0.2.4
+
+* Added -s option to specify BLE scan duration
+* Improved README.md
+
+Release 0.2.3
+
+* Fixed eternal loop caused by hostname resolve failure
+
 Release 0.2.2
 
 * Supported multiple device connections for toio

pyscrlink/scratch_link.py

@@ -8,12 +8,14 @@ import asyncio
 import pathlib
 import ssl
 import websockets
+import socket
 import json
 import base64
 import logging
 import sys
 import signal
 import traceback
+import argparse
 
 # for Bluetooth (e.g. Lego EV3)
 import bluetooth
@@ -42,6 +44,11 @@ logger.setLevel(logLevel)
 logger.addHandler(handler)
 logger.propagate = False
 
+HOSTNAME="device-manager.scratch.mit.edu"
+scan_seconds=10.0
+
+CCCD_UUID = 0x2902
+
 class Session():
     """Base class for BTSession and BLESession"""
     def __init__(self, websocket, loop):
@@ -165,6 +172,9 @@ class BTSession(Session):
                 logger.debug(f"Found device {name} addr={address} class={device_class} rssi={rssi}")
                 major_class = (device_class & 0x1F00) >> 8
                 minor_class = (device_class & 0xFF) >> 2
+                if "LEGO Hub" in name:
+                    minor_class = 1
+                    logger.info(f"Pretend to be LEGO EV3 with LEGO Hub: class={major_class/minor_class}")
                 if major_class == self.major_class and minor_class == self.minor_class:
                     self.found_devices[address] = (name, device_class, rssi)
 
@@ -180,7 +190,13 @@ class BTSession(Session):
             self.ping_time = None
 
         def discover(self):
-            discoverer = self.BTDiscoverer(self.major_device_class, self.minor_device_class)
+            major = self.major_device_class
+            minor = self.minor_device_class
+            if major == 8 and minor == 1:
+                logger.info(f"Search LEGO Hub instead of LEGO EV3")
+                minor = 4
+            logger.debug(f"BT discover: class={major}/{minor}")
+            discoverer = self.BTDiscoverer(major, minor)
             discoverer.find_devices(lookup_names=True)
             while self.session.status == self.session.DISCOVERY and not discoverer.done and not self.cancel_discovery:
                 readable = select.select([discoverer], [], [], 0.5)[0]
@@ -198,7 +214,7 @@ class BTSession(Session):
         def run(self):
             while self.session.status != self.session.DONE:
 
-                logger.debug("loop in BT thread")
+                logger.debug(f"loop in BT thread: session status={self.session.status}")
                 current_time = int(round(time.time()))
 
                 if self.session.status == self.session.DISCOVERY and not self.cancel_discovery:
@@ -348,6 +364,14 @@ class BLESession(Session):
     scan_lock = threading.RLock()
     scan_started = False
 
+    @staticmethod
+    def _getDevName(dev):
+        """
+        Get AdType 0x09 (Completed local name). If it is not available,
+        get AdType 0x08 (Shortened local name).
+        """
+        return dev.getValueText(0x9) or dev.getValueText(0x8)
+
     class BLEThread(threading.Thread):
         """
         Separated thread to control notifications to Scratch.
@@ -367,7 +391,7 @@ class BLESession(Session):
                     for d in devices:
                         params = { 'rssi': d.rssi }
                         params['peripheralId'] = devices.index(d)
-                        params['name'] = d.getValueText(0x9) or d.getValueText(0x8)
+                        params['name'] = BLESession._getDevName(d)
                         self.session.notify('didDiscoverPeripheral', params)
                     time.sleep(1)
                 elif self.session.status == self.session.CONNECTED:
@@ -384,7 +408,8 @@ class BLESession(Session):
                                 logger.debug("after waitForNotification")
                             logger.debug("released lock for waitForNotification")
                         except Exception as e:
-                            logger.error(e)
+                            logger.error(f"Exception in waitForNotifications: "
+                                         f"{type(e).__name__}: {e}")
                             self.session.close()
                             break
                     else:
@@ -408,7 +433,8 @@ class BLESession(Session):
             self.restart_notification_event.set()
 
         def add_handle(self, serviceId, charId, handle):
-            logger.debug(f"add handle for notification: {handle}")
+            logger.debug(f"add handle for notification: "
+                         f"{serviceId} {charId} {handle}")
             params = { 'serviceId': UUID(serviceId).getCommonName(),
                        'characteristicId': charId,
                        'encoding': 'base64' }
@@ -416,6 +442,14 @@ class BLESession(Session):
 
         def handleNotification(self, handle, data):
             logger.debug(f"BLE notification: {handle} {data}")
+            if handle not in self.handles:
+                logger.error(f"Notification with unknown handle: {handle}")
+                keys = list(self.handles.keys())
+                if keys and len(keys) == 1:
+                    logger.debug(f"Debug: override {handle} with {keys[0]}")
+                    handle = keys[0]
+                else:
+                    return
             params = self.handles[handle].copy()
             params['message'] = base64.standard_b64encode(data).decode('ascii')
             self.session.notify('characteristicDidChange', params)
@@ -448,12 +482,14 @@ class BLESession(Session):
     def __del__(self):
         self.close()
 
-    def _get_dev_uuid(self, dev):
+    def _get_dev_uuids(self, dev):
         for adtype in self.SERVICE_CLASS_UUID_ADTYPES:
-            service_class_uuid = dev.getValueText(adtype)
-            if service_class_uuid:
-                logger.debug(self.SERVICE_CLASS_UUID_ADTYPES[adtype])
-                return UUID(service_class_uuid)
+            service_class_uuids = dev.getValue(adtype)
+            if service_class_uuids:
+                for u in service_class_uuids:
+                    a = self.SERVICE_CLASS_UUID_ADTYPES[adtype]
+                    logger.debug(f"service class uuid for {a}/{adtype}: {u}")
+                return service_class_uuids
         return None
 
     def matches(self, dev, filters):
@@ -466,18 +502,19 @@ class BLESession(Session):
                 for s in f['services']:
                     logger.debug(f"service to check: {s}")
                     given_uuid = s
-                    logger.debug(f"given: {given_uuid}")
-                    dev_uuid = self._get_dev_uuid(dev)
-                    if not dev_uuid:
+                    logger.debug(f"given UUID: {given_uuid} hash={UUID(given_uuid).__hash__()}")
+                    dev_uuids = self._get_dev_uuids(dev)
+                    if not dev_uuids:
                         continue
-                    logger.debug(f"dev: {dev_uuid}")
-                    logger.debug(given_uuid == dev_uuid)
-                    if given_uuid == dev_uuid:
-                        logger.debug("match...")
-                        return True
+                    for u in dev_uuids:
+                        logger.debug(f"dev UUID: {u} hash={u.__hash__()}")
+                        logger.debug(given_uuid == u)
+                        if given_uuid == u:
+                            logger.debug("match...")
+                            return True
             if 'namePrefix' in f:
                 # 0x08: Shortened Local Name
-                deviceName = dev.getValueText(0x08)
+                deviceName = self._getDevName(dev)
                 if not deviceName:
                     continue
                 logger.debug(f"Name of \"{deviceName}\" begins with: \"{f['namePrefix']}\"?")
@@ -492,6 +529,7 @@ class BLESession(Session):
         return False
 
     def _scan_devices(self, params):
+        global scan_seconds
         if BLESession.nr_connected > 0:
             return len(BLESession.found_devices) > 0
         found = False
@@ -502,7 +540,8 @@ class BLESession(Session):
                 for i in range(self.MAX_SCANNER_IF):
                     scanner = Scanner(iface=i)
                     try:
-                        devices = scanner.scan(10.0)
+                        logger.debug(f"start BLE scan: {scan_seconds} seconds")
+                        devices = scanner.scan(scan_seconds)
                         for dev in devices:
                             if self.matches(dev, params['filters']):
                                 BLESession.found_devices.append(dev)
@@ -591,7 +630,7 @@ class BLESession(Session):
         elif self.status == self.DISCOVERY and method == 'connect':
             logger.debug("connecting to the BLE device")
             self.device = BLESession.found_devices[params['peripheralId']]
-            self.deviceName = self.device.getValueText(0x9) or self.device.getValueText(0x8)
+            self.deviceName = self._getDevName(self.device)
             try:
                 self.perip = Peripheral(self.device)
                 logger.info(f"connected to the BLE peripheral: {self.deviceName}")
@@ -668,11 +707,19 @@ class BLESession(Session):
         service = self._get_service(service_id)
         c = self._get_characteristic(chara_id)
         handle = c.getHandle()
+        # get CCCD or Client Characterstic Configuration Descriptor
+        cccd = None
+        for d in c.getDescriptors():
+            if d.uuid == UUID(CCCD_UUID):
+                cccd = d
+        if not cccd:
+            logger.error("Characteristic {char_id} does not have CCCD")
+            return
         # prepare notification handler
         self.delegate.add_handle(service_id, chara_id, handle)
         # request notification to the BLE device
         with self.lock:
-            self.perip.writeCharacteristic(handle + 1, value, True)
+            self.perip.writeCharacteristic(cccd.handle, value, True)
 
     def startNotifications(self, service_id, chara_id):
         logger.debug(f"start notification for {chara_id}")
@@ -715,18 +762,20 @@ def stack_trace():
          print(line)
 
 def main():
-    opts = [opt for opt in sys.argv[1:] if opt.startswith("-")]
-    if "-h" in opts:
-        print((f"Usage: {sys.argv[0]} [OPTS]\n"
-               "OPTS:\t-h Show this help.\n"
-               "\t-d Print debug messages."
-        ))
-        sys.exit(1)
-    elif "-d" in opts:
+    global scan_seconds
+    parser = argparse.ArgumentParser(description='start Scratch-link')
+    parser.add_argument('-d', '--debug', action='store_true',
+                        help='print debug messages')
+    parser.add_argument('-s', '--scan_seconds', type=float, default=10.0,
+                        help='specifiy duration to scan BLE devices in seconds')
+    args = parser.parse_args()
+    if args.debug:
         print("Print debug messages")
         logLevel = logging.DEBUG
         handler.setLevel(logLevel)
         logger.setLevel(logLevel)
+    scan_seconds = args.scan_seconds
+    logger.debug(f"set scan_seconds: {scan_seconds}")
 
     # Prepare certificate of the WSS server
     gencert.prep_cert()
@@ -738,7 +787,7 @@ def main():
     ssl_context.load_cert_chain(localhost_cer, localhost_key)
 
     start_server = websockets.serve(
-         ws_handler, "device-manager.scratch.mit.edu", 20110, ssl=ssl_context
+         ws_handler, HOSTNAME, 20110, ssl=ssl_context
     )
 
     while True:
@@ -749,7 +798,13 @@ def main():
         except KeyboardInterrupt as e:
             stack_trace()
             break
+        except socket.gaierror as e:
+            logger.error(f"{type(e).__name__}: {e}")
+            logger.info(f"Check internet connection to {HOSTNAME}. If not "
+                        f"available, add '127.0.0.1 {HOSTNAME}' to /etc/hosts.")
+            break
         except Exception as e:
+            logger.error(f"{type(e).__name__}: {e}")
             logger.info("Restarting scratch-link...")
 
 if __name__ == "__main__":

setup.py

@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
 
 setuptools.setup(
     name="pyscrlink",
-    version="0.2.2",
+    version="0.2.4",
     author="Shin'ichiro Kawasaki",
     author_email='kawasaki@juno.dti.ne.jp',
     description='Scratch-link for Linux with Python',