ble: Separate BLEDBusSession class to a separated new file ble.py

Signed-off-by: Shin'ichiro Kawasaki <kawasaki@juno.dti.ne.jp>

.gitignore

@@ -1 +1,5 @@
-/__pycache__
+__pycache__
+build
+dist
+pyscrlink/__pycache__
+pyscrlink.egg-info

README.md

@@ -9,11 +9,16 @@ micro:bit.
 
 Pyscrlink allows you to connect Scratch and bluetooth devices with the Linux
 OSes. It uses the Linux Bluetooth protocol stack [Bluez](http://www.bluez.org/)
-and its python interfaces [pybluez](https://github.com/pybluez/pybluez) to
+and [bluepy](https://github.com/IanHarvey/bluepy) to handle Bluetooth Low Energy
-handle Bluetooth, and [bluepy](https://github.com/IanHarvey/bluepy) to handle
+(BLE) connections. It has been reported that pyscrlink connects Scratch 3.0 with
-Bluetooth Low Energy (BLE) connections. It has been reported that pyscrlink
+micro:bit, LEGO WeDo, LEGO Boost and toio.
-connects Scratch 3.0 with micro:bit, LEGO Mindstorms EV3, LEGO WeDo, LEGO
+
-Boost, Intelino Smart Train and toio.
+Until version v0.2.5, pyscrlink supported Bluetooth Classic protocol using
+[pybluez](https://github.com/pybluez/pybluez). Unfortunately, pybluez is not
+well maintained and caused technical troubles. Then Bluetooth Classic protocol
+support is dropped from pyscrlink. This means that LEGO Mindstorm EV3 can not
+be connected with pyscrlink. Bluetooth Classic support is the improvement
+opportunity of pyscrlink.
 
 To use websockets, pyscrlink requires python version 3.6 or later. If your
 system has python older than version 3.6, install newer version. If your
@@ -39,26 +44,24 @@ Devices:
 
 Linux distros:
 * Arch Linux
-* elementary OS 5.1 Hera
 
 Browsers:
-* FireFox
+* Firefox
 * Chromium
 
 It was reported that pyscrlink (former bluepy-scratch-link) working with
 following devices and Linux distros.
 
 Devices:
-* LEGO Mindstorm EV3 by @chrisglencross
 * LEGO WeDo by @zhaowe, @KingBBQ
-* LEGO Boost by @laurentchar, @miguev, @jacquesdt
+* LEGO Boost and compatible devices by @laurentchar, @miguev, @jacquesdt, @n3storm
-* Intelino Smart Train by @ErrorJan
 * toio by @shimodash
 
 Linux distros:
 * Raspbian by @chrisglencross
 * Ubuntu 16.04 @jacquesdt
 * Ubuntu Studio 20.04 @miguev
+* Debian 11 @n3storm
 
 Installation
 ------------
@@ -87,7 +90,7 @@ Installation
     $ pip3 install pyscrlink
     ```
 
-4. For Bluetooth Low Energy (BLE) devices, set bluepy-helper capability.
+4. Set bluepy-helper capability.
 
     ```
     $ bluepy_helper_cap
@@ -106,65 +109,18 @@ Installation
 
 Usage
 -----
-1. For LEGO Mindstorms EV3, pair your Linux PC to the EV3 brick.
+1. Start scratch-link python script.
-
-   First, turn on the EV3 and ensure Bluetooth is enabled.
-
-   Then, pair using your Linux desktop's the Bluetooth settings.
-
-   If using Gnome:
-      * Settings -> Bluetooth
-      * Click on the EV3 device name
-      * Accept the connection on EV3 brick
-      * Enter a matching PIN on EV3 brick and Linux PC. '1234' is the value Scratch suggests.
-      * Confirm EV3 status is "Disconnected" in Bluetooth settings
-
-   With a Raspberry Pi default Raspbian desktop, click the Bluetooth logo in the top right of the screen and
-   Add Device. Then follow the Gnome instructions. You will be warned that the Raspberry Pi
-   does not know how to talk to this device; that is not a problem.
-
-   Alternatively you can perform pairing from the command-line:
-   ```shell script
-   $ bluetoothctl
-
-   [bluetooth]# power on
-   Changing power on succeeded
-
-   [bluetooth]# pairable on
-   Changing pairable on succeeded
-
-   [bluetooth]# agent KeyboardOnly
-   Agent registered
-
-   [bluetooth]# devices
-   ...
-   Device 00:16:53:53:D3:19 EV3
-   ...
-
-   [bluetooth]# pair 00:16:53:53:D3:19
-   Attempting to pair with 00:16:53:53:D3:19
-
-   # Confirm pairing on the EV3 display, set PIN to 1234
-
-   Request PIN code
-   [agent] Enter PIN code: 1234
-   [CHG] Device 00:16:53:53:D3:19 Connected: yes
-   [CHG] Device 00:16:53:53:D3:19 Paired: yes
-   Pairing successful
-
-   [bluetooth]# quit
-   ```
-
-2. Start scratch-link python script.
     ```sh
     $ scratch_link
     ```
+    If your device is toio, add "-s 1" option to the scratch_link command. It
+    allows the toio Do Visual Programming to connect to toio automatically.
 
-3. Connect scratch to the target device such as micro:bit or LEGO Mindstorms:
+2. Connect scratch to the target device such as micro:bit:
     * Open FireFox or Chrome. (Make sure to run as the same user for scratch-link python script.)
     * Access [Scratch 3.0](https://scratch.mit.edu/) and create your project.
     * Select the "Add Extension" button.
-    * Select the extension for your device (e.g., micro:bit or Lego Mindstorms EV3 extension) and follow the prompts to connect.
+    * Select the extension for your device (e.g., micro:bit) and follow the prompts to connect.
     * Build your project with the extension blocks.
 
 In Case You Fail to Connect
@@ -185,10 +141,22 @@ In Case You Fail to Connect
     * If the service is not working, refer guide of your distro to set it up.
     * If the service is working, also check that /etc/bluetooth/main.conf sets AutoEnable=true.
 
-3. If scratch_link.py says "failed to connect to BT device: [Errno 13] Permission denied",
+3. If device scan still fails, use -r option to retry device scan.
+    The command line below does device scan twice. Each scan takes 10 seconds.
+    ```
+    $ scratch_link -r 2
+    ```
+    It would be good to use -s option together to reduce each scan duration.
+    The command line below does 3 seconds device scan twice.
+
+    ```
+    $ scratch_link -r 2 -s 3
+    ```
+
+4. If scratch_link.py says "failed to connect to BT device: [Errno 13] Permission denied",
    make sure to pair the bluetooth device to your PC before connecting to Scratch.
 
-4. To connect to multiple devices at the same time, make all the target devices
+5. To connect to multiple devices at the same time, make all the target devices
    ready for scan at the first device scan. This is important for toio. The toio
    allows a single project to connect to two toio devices.
    * When the second device was prepared after the first device was connected, device scan can not find the second device.
@@ -202,6 +170,32 @@ Please file issues to [GitHub issue tracker](https://github.com/kawasaki/pyscrli
 Releases
 --------
 
+Release 0.2.8
+
+* Supported Microbit More v2
+
+Release 0.2.7
+
+* Supported Snap Firefox and Chromium
+* Added -r option to retry BLE scan
+
+Release 0.2.6
+
+* Removed Bluetooth Classic and LEGO Mindstorm EV3 support
+
+Release 0.2.5
+
+* Fixed handling of multiple UUIDs for LEGO Boost
+
+Release 0.2.4
+
+* Added -s option to specify BLE scan duration
+* Improved README.md
+
+Release 0.2.3
+
+* Fixed eternal loop caused by hostname resolve failure
+
 Release 0.2.2
 
 * Supported multiple device connections for toio

pyscrlink/ble.py

@@ -0,0 +1,374 @@
+from sdbus import DbusInterfaceCommonAsync, SdBus, sd_bus_open_system
+from sdbus.dbus_proxy_async_interfaces import DbusIntrospectableAsync
+import xml.etree.ElementTree as ET
+from sdbus_async.bluez.adapter_api import AdapterInterfaceAsync
+from sdbus_async.bluez.device_api import DeviceInterfaceAsync
+from sdbus_async.bluez.gatt_api import (
+    GattCharacteristicInterfaceAsync,
+    GattServiceInterfaceAsync,
+)
+import asyncio
+import base64
+from asyncio import sleep
+from os import dup, fdopen, close
+
+import pyscrlink.scratch_link
+from pyscrlink.scratch_link import BTUUID
+
+import logging
+logger = logging.getLogger('pyscrlink.scratch_link')
+
+class BLEDBusSession(pyscrlink.scratch_link.Session):
+    """
+    Manage a session for Bluetooth Low Energy device such as micro:bit using
+    DBus as backend.
+    """
+
+    INITIAL = 1
+    DISCOVERY = 2
+    CONNECTED = 3
+    DONE = 4
+
+    MAX_SCANNER_IF = 3
+
+    connected_devices = {}
+
+    class Device():
+        def __init__(self, interface, path, node_name, name, address):
+            self.interface = interface
+            self.path = path
+            self.node_name = node_name
+            self.name = name
+            self.address = address
+
+    class Notification():
+        def __init__(self, loop, acquired_fd, fd, fp, params):
+            self.loop = loop
+            self.acquired_fd = acquired_fd
+            self.fd = fd
+            self.fp = fp
+            self.params = params
+
+        def close(self):
+            self.loop.remove_reader(self.fd)
+            self.fp.close()
+
+    def _connect_to_adapters(self):
+        self.iface = None
+        self.adapter = None
+        self.adapter_introspect = None
+        adapter = AdapterInterfaceAsync()
+        for i in range(self.MAX_SCANNER_IF):
+            iface = '/org/bluez/hci' + str(i)
+            logger.debug(f"try connect to {iface}")
+            try:
+                adapter._connect('org.bluez', iface, bus=self.dbus)
+                logger.debug(f"connected to {iface}")
+                adapter_introspect = DbusIntrospectableAsync()
+                adapter_introspect._connect('org.bluez', iface, bus=self.dbus)
+                self.iface = iface
+                self.adapter = adapter
+                self.adapter_introspect = adapter_introspect
+                return
+            except Exception as e:
+                logger.error(e)
+        raise Exception("no adapter is available")
+
+    async def _start_discovery(self):
+        logger.debug(f"Starting discovery... {self.adapter}")
+        assert not self.discovery_running
+        await self.adapter.start_discovery()
+        self.discovery_running = True
+
+        asyncio.create_task(self._find_devices())
+        asyncio.create_task(self._stop_discovery())
+        logger.debug(f"Task to stop discovery has got created.")
+
+    async def _matches(self, dev, filters):
+        """
+        Check if the found BLE device matches the filters Scratch specifies.
+        """
+        logger.debug(f"in matches {dev} {filters}")
+        for f in filters:
+            if 'services' in f:
+                for s in f['services']:
+                    logger.debug(f"service to check: {s}")
+                    given_uuid = BTUUID(s)
+                    logger.debug(f"given UUID: {given_uuid} hash={given_uuid.__hash__()}")
+                    dev_uuids = await dev.interface.uuids
+                    if not dev_uuids:
+                        logger.debug(f"dev UUID not available")
+                        continue
+                    for uuid in dev_uuids:
+                        u = BTUUID(uuid)
+                        logger.debug(f"dev UUID: {u} hash={u.__hash__()}")
+                        logger.debug(given_uuid == u)
+                        if given_uuid == u:
+                            logger.debug("match...")
+                            return True
+            if 'namePrefix' in f:
+                logger.debug(f"given namePrefix: {f['namePrefix']}")
+                if dev.name:
+                    logger.debug(f"name: {dev. name}")
+                    if dev.name.startswith(f['namePrefix']):
+                        logger.debug(f"match...")
+                        return True
+            if 'name' in f or 'manufactureData' in f:
+                logger.error("name/manufactureData filters not implemented")
+                # TODO: implement other filters defined:
+                # ref: https://github.com/LLK/scratch-link/blob/develop/Documentation/BluetoothLE.md
+        return False
+
+    async def _notify_device(self, device) -> None:
+        params = { 'rssi': -80, 'name': 'Unknown' }
+        try:
+            params['rssi'] = await device.interface.rssi
+        except Exception:
+            None
+        if device.name:
+            params['name'] = device.name
+        params['peripheralId'] = device.node_name
+        await self._send_notification('didDiscoverPeripheral', params)
+
+    async def _find_devices(self) -> None:
+        assert self.discovery_running
+        while self.discovery_running:
+            await sleep(1)
+            s = await self.adapter_introspect.dbus_introspect()
+            parser = ET.fromstring(s)
+            nodes = parser.findall("./node")
+            if not nodes:
+                logger.info("device not found")
+                continue
+            logger.debug(f"{len(nodes)} device(s) found")
+            for node in nodes:
+                node_name = node.attrib['name']
+                logger.debug(f"  {node_name}")
+                if self.found_devices.get(node_name):
+                    continue
+                devpath = self.iface + "/" + node_name
+                if BLEDBusSession.connected_devices.get(devpath):
+                    continue
+                interface = DeviceInterfaceAsync()
+                interface._connect('org.bluez', devpath, bus=self.dbus)
+                try:
+                    devname = await interface.name
+                except Exception as e:
+                    logger.debug(f"device {node_name} does not have name: {e}")
+                devaddr = await interface.address
+                device = self.Device(interface, devpath, node_name, devname,
+                                     devaddr)
+                if not await self._matches(device, self.discover_filters):
+                    await interface.disconnect()
+                    continue
+                self.found_devices[node_name] = device
+                await self._notify_device(device)
+
+        logger.debug("end _find_device.")
+
+    async def _stop_discovery(self) -> None:
+        assert self.discovery_running
+        logger.debug(f"Wait discovery for {self.scan_seconds} seconds")
+        await sleep(self.scan_seconds)
+        logger.debug(f"Stopping discovery... {self.adapter}")
+        self.discovery_running = False
+        await self.adapter.stop_discovery()
+
+    def __init__(self, websocket, loop, scan_seconds):
+        super().__init__(websocket, loop, scan_seconds)
+        logger.debug("dbus init")
+        self.status = self.INITIAL
+        self.dbus = sd_bus_open_system()
+        self.discovery_running = False
+        self.iface = None
+        self.services = {}
+        self.chars = {}
+        self.chars_cache = {}
+        self.notifications = {}
+        self._connect_to_adapters()
+        self.found_devices = {}
+
+    async def _get_characteristics(self, service_path):
+        service_introspect = DbusInterfaceCommonAsync()
+        service_introspect._connect('org.bluez', service_path, bus=self.dbus)
+        s = await service_introspect.dbus_introspect()
+        parser = ET.fromstring(s)
+        nodes = parser.findall("./node")
+        if not nodes:
+            logger.error(f"characteristic not found at {service_path}")
+            return
+        for node in nodes:
+            path = service_path + '/' + node.attrib['name']
+            if self.chars.get(path):
+                continue
+            logger.debug(f"getting GATT characteristic at {path}")
+            char = GattCharacteristicInterfaceAsync()
+            char._connect('org.bluez', path, bus=self.dbus)
+            self.chars[path] = char
+            cid = await char.uuid
+            logger.debug(f"found char {cid}")
+
+    async def _get_services(self):
+        # do D-Bus introspect to the device path and get service paths under it
+        for i in range(5):
+            dev_introspect = DbusInterfaceCommonAsync()
+            dev_introspect._connect('org.bluez', self.device.path,
+                                    bus=self.dbus)
+            s = await dev_introspect.dbus_introspect()
+            parser = ET.fromstring(s)
+            nodes = parser.findall("./node")
+            if nodes:
+                break
+            else:
+                logger.error("Service not found. Try again.")
+                await sleep(1)
+        if not nodes:
+            return []
+        for node in nodes:
+            path = self.device.path + '/' + node.attrib['name']
+            if self.services.get(path):
+                continue
+            logger.debug(f"getting GATT service at {path}")
+            service = GattServiceInterfaceAsync()
+            service._connect('org.bluez', path, bus=self.dbus)
+            self.services[path] = service
+            sid = await service.uuid
+            logger.debug(f"found service {sid}")
+            await self._get_characteristics(path)
+
+    async def _get_char(self, id):
+        char = self.chars_cache.get(id)
+        if char:
+            return char
+        for i in range(5):
+            await self._get_services()
+            btuuid = BTUUID(id)
+            for char in self.chars.values():
+                raw_uuid = await char.uuid
+                char_uuid = BTUUID(raw_uuid)
+                if char_uuid == btuuid:
+                    self.chars_cache[id] = char
+                    return char
+            logger.error(f"Can not get characteristic: {id}. Retry.")
+        logger.error(f"Abandoned to get characteristic: {id}.")
+        return None
+
+    async def _start_notification(self, sid, cid, char):
+        logger.debug('startNotification')
+        (acquired_fd, mtu) = await char.acquire_notify({})
+        fd = dup(acquired_fd)
+        fp = fdopen(fd, mode='rb', buffering=0, newline=None)
+        self.loop.add_reader(fd, self._read_notification, fd)
+        notification = self.Notification(self.loop, acquired_fd, fd, fp, {
+            'serviceId': sid,
+            'characteristicId': cid,
+            'encoding': 'base64'
+        })
+        self.notifications[fd] = notification
+        logger.debug(f'added notification reader: {notification}')
+
+    def _stop_notifications(self):
+        for n in self.notifications.values():
+            n.close()
+
+    def _read_notification(self, *args):
+        fd = args[0]
+        notification = self.notifications[fd]
+        data = notification.fp.read()
+        if len(data) == 0:
+            logger.debug(f'empty notification data')
+            asyncio.create_task(self.async_close())
+            return
+        params = notification.params.copy()
+        params['message'] = base64.standard_b64encode(data).decode('ascii')
+        self.loop.create_task(self._send_notification('characteristicDidChange', params))
+
+    def handle_request(self, method, params):
+        logger.debug("handle request")
+
+    async def async_handle_request(self, method, params):
+        logger.debug(f"async handle request: {method} {params}")
+
+        res = { "jsonrpc": "2.0" }
+        err_msg = None
+
+        if self.status == self.INITIAL and method == 'discover':
+            self.discover_filters = params['filters']
+            logger.debug(f"discover: {self.discover_filters}")
+            try:
+                await self._start_discovery()
+                logger.debug(f"discover started: {self.discover_filters}")
+                res["result"] = None
+                self.status = self.DISCOVERY
+            except Exception as e:
+                res["error"] = { "message": "Failed to start device discovery" }
+                self.status = self.DONE
+
+        elif self.status == self.DISCOVERY and method == 'connect':
+            logger.debug("connecting to the BLE device")
+            dev = self.found_devices[params['peripheralId']]
+            try:
+                logger.debug(f"  {dev}")
+                await dev.interface.connect()
+                res["result"] = None
+                self.device = dev
+                self.status = self.CONNECTED
+                logger.info(f"Connected: '{dev.name}'@{dev.address}")
+                BLEDBusSession.connected_devices[dev.path] = dev
+            except NotImplementedError as e:
+                logger.error(e)
+                res["error"] = { "message": "Failed to connect to device" }
+                self.status = self.DONE
+            except Exception as e:
+                logger.error(f"failed to connect: {e}")
+                res["error"] = { "message": "Failed to connect to device" }
+                self.status = self.DONE
+
+        elif self.status == self.CONNECTED and method == 'read':
+            logger.debug("handle read request")
+            service_id = params['serviceId']
+            chara_id = params['characteristicId']
+            c = await self._get_char(chara_id)
+            value = await c.read_value({})
+            message = base64.standard_b64encode(value).decode('ascii')
+            res['result'] = { 'message': message, 'encode': 'base64' }
+            if params.get('startNotifications') == True:
+                await self._start_notification(service_id, chara_id, c)
+
+        elif self.status == self.CONNECTED and method == 'write':
+            logger.debug(f"handle write request {params}")
+            service_id = params['serviceId']
+            chara_id = params['characteristicId']
+            c = await self._get_char(chara_id)
+            if params['encoding'] != 'base64':
+                logger.error("encoding other than base 64 is not "
+                             "yet supported: ", params['encoding'])
+            else:
+                msg_bstr = params['message'].encode('ascii')
+                data = base64.standard_b64decode(msg_bstr)
+                await c.write_value(data, {})
+                res['result'] = len(data)
+
+        logger.debug(res)
+        return res
+
+    def end_request(self):
+        logger.debug("end request")
+        return False
+
+    async def async_close(self):
+        if not self.device:
+            return
+        dev = self.device
+        logger.info(f"Disconnecting from '{dev.name}'@{dev.address}")
+        self._stop_notifications()
+        await dev.interface.disconnect()
+        BLEDBusSession.connected_devices.pop(dev.path)
+        logger.info(f"Disconnected from '{dev.name}'@{dev.address}")
+        self.device = None
+        await self.websocket.close()
+        return
+
+    def close(self):
+        logger.debug("close")
+        return

pyscrlink/gencert.py

@@ -152,36 +152,37 @@ def prep_nss_cert(dir, cert, nickname):
     remove_cert(dir, nickname)
     add_cert(dir, cert, nickname)
 
-def prep_cert():
+def prep_cert_for_app(cert, app, search_path):
-    # Generate certification and key
+    """
-    gen_cert(cert_file_path, key_file_path)
+    Find a NSS DB in the search_path for the app and prepare the cert in the DB.
-
+    """
-    # Add certificate to FireFox
     nssdb = None
-    firefox_nss_path = os.path.join(homedir, ".mozilla/firefox/")
+    for root, dirs, files in os.walk(os.path.join(homedir, search_path)):
-    for root, dirs, files in os.walk(firefox_nss_path):
         for name in files:
             if not re.match("key.*\.db", name):
                 continue
             nssdb = root
-            if prep_nss_cert(nssdb, cert_file_path, SCRATCH_CERT_NICKNAME):
+            if prep_nss_cert(nssdb, cert, SCRATCH_CERT_NICKNAME):
-                logger.error(f"Failed to add certificate to FireFox NSS DB: {nssdb}")
+                logger.error(f"Failed to add certificate to {app}: {nssdb}")
                 sys.exit(3)
             else:
-                logger.info(f"Certificate is ready in FireFox NSS DB: {nssdb}")
+                logger.info(f"Certificate is ready in {app} NSS DB: {nssdb}")
     if not nssdb:
-        logger.info("FireFox NSS DB not found. Do not add certificate.")
+        logger.debug(f"NSS DB for {app} not found. Do not add certificate.")
 
-    # Add certificate to Chrome
+
-    nssdb = os.path.join(homedir, ".pki/nssdb")
+def prep_cert():
-    if os.path.isdir(nssdb):
+    # Generate certification and key
-        if prep_nss_cert(nssdb, cert_file_path, SCRATCH_CERT_NICKNAME):
+    gen_cert(cert_file_path, key_file_path)
-            logger.error(f"Failed to add certificate to Chrome")
+
-            sys.exit(4)
+    nss_dbs = {
-        else:
+        "FireFox": ".mozilla/firefox/",
-            logger.info("Certificate is ready for Chrome")
+        "FireFox(Snap)": "snap/firefox/common/.mozilla/firefox/",
-    else:
+        "Chrome": ".pki",
-        logger.info("Chrome NSS DB not found. Do not add certificate.")
+        "Chromium(Snap)": "snap/chromium",
+        }
+
+    [ prep_cert_for_app(cert_file_path, k, nss_dbs[k]) for k in nss_dbs ]
 
 if __name__ == "__main__":
     prep_cert()

pyscrlink/scratch_link.py

@@ -8,32 +8,35 @@ import asyncio
 import pathlib
 import ssl
 import websockets
+import socket
 import json
+import uuid
 import base64
 import logging
 import sys
 import signal
 import traceback
-
+import argparse
-# for Bluetooth (e.g. Lego EV3)
-import bluetooth
 
 # for BLESession (e.g. BBC micro:bit)
-from bluepy.btle import Scanner, UUID, Peripheral, DefaultDelegate
+from bluepy.btle import Scanner, UUID, Peripheral, DefaultDelegate, ScanEntry
 from bluepy.btle import BTLEDisconnectError, BTLEManagementError
 from pyscrlink import bluepy_helper_cap
 
 import threading
 import time
 import queue
+from asyncio import sleep
 
 # for websockets certificate
 from pyscrlink import gencert
 
+from pyscrlink import ble
+
 logLevel = logging.INFO
 
 # for logging
-logger = logging.getLogger(__name__)
+logger = logging.getLogger('pyscrlink.scratch_link')
 formatter = logging.Formatter(fmt='%(asctime)s %(message)s')
 handler = logging.StreamHandler()
 handler.setLevel(logLevel)
@@ -42,11 +45,33 @@ logger.setLevel(logLevel)
 logger.addHandler(handler)
 logger.propagate = False
 
+HOSTNAME="device-manager.scratch.mit.edu"
+scan_seconds=10.0
+
+class BTUUID(uuid.UUID):
+    BLUETOOTH_BASE_UUID = "00001000800000805F9B34FB"
+
+    def __init__(self, val):
+        if isinstance(val, int):
+            if (val < 0) or (val > 0xFFFFFFFF):
+                raise ValueError(
+                    "Short form UUIDs must be in range 0..0xFFFFFFFF")
+            val = "%04X" % val
+        else:
+            val = str(val)
+
+        val = val.replace("-", "")
+        if len(val) <= 8:  # Short form
+            val = ("0" * (8 - len(val))) + val + self.BLUETOOTH_BASE_UUID
+
+        uuid.UUID.__init__(self, val)
+
 class Session():
     """Base class for BTSession and BLESession"""
-    def __init__(self, websocket, loop):
+    def __init__(self, websocket, loop, scan_seconds):
         self.websocket = websocket
         self.loop = loop
+        self.scan_seconds = scan_seconds
         self.lock = threading.RLock()
         self.notification_queue = queue.Queue()
 
@@ -65,6 +90,10 @@ class Session():
         if jsonreq['jsonrpc'] != '2.0':
             logger.error("error: jsonrpc version is not 2.0")
             return True
+        if type(self) is ble.BLEDBusSession:
+            jsonres = await self.async_handle_request(jsonreq['method'],
+                                                      jsonreq['params'])
+        else:
             jsonres = self.handle_request(jsonreq['method'], jsonreq['params'])
         if 'id' in jsonreq:
             jsonres['id'] = jsonreq['id']
@@ -79,6 +108,10 @@ class Session():
         """Default request handler"""
         logger.debug(f"default handle_request: {method}, {params}")
 
+    async def async_handle_request(self, method, params):
+        """Default async request handler"""
+        logger.debug(f"default async handle_request: {method}, {params}")
+
     def end_request(self):
         """
         Default callback at request end. This callback is required to
@@ -118,11 +151,17 @@ class Session():
                     break
                 await self._send_notifications()
                 logger.debug("in handle loop")
-            except websockets.ConnectionClosedError as e:
+            except (websockets.ConnectionClosedOK, websockets.ConnectionClosedError) as e:
                 logger.info("scratch closed session")
                 logger.debug(e)
+                if type(self) is ble.BLEDBusSession:
+                    await self.async_close()
+                else:
                     self.close()
                 break
+            except Exception as e:
+                t = type(e)
+                logger.info(f"scratch closed with unkown exception: {e}: {t}")
 
     def close(self):
         """
@@ -133,198 +172,10 @@ class Session():
 class BTSession(Session):
     """Manage a session for Bluetooth device"""
 
-    INITIAL = 1
-    DISCOVERY = 2
-    DISCOVERY_COMPLETE = 3
-    CONNECTED = 4
-    DONE = 5
-
-    # Split this into discovery thread and communication thread
-    # discovery thread should auto-terminate
-
-    class BTThread(threading.Thread):
-        """
-        Separated thread to control notifications to Scratch.
-        It handles device discovery notification in DISCOVERY status
-        and notifications from bluetooth devices in CONNECTED status.
-        """
-
-        class BTDiscoverer(bluetooth.DeviceDiscoverer):
-
-            def __init__(self, major_class, minor_class):
-                super().__init__()
-                self.major_class = major_class
-                self.minor_class = minor_class
-                self.found_devices = {}
-                self.done = False
-
-            def pre_inquiry(self):
-                self.done = False
-
-            def device_discovered(self, address, device_class, rssi, name):
-                logger.debug(f"Found device {name} addr={address} class={device_class} rssi={rssi}")
-                major_class = (device_class & 0x1F00) >> 8
-                minor_class = (device_class & 0xFF) >> 2
-                if major_class == self.major_class and minor_class == self.minor_class:
-                    self.found_devices[address] = (name, device_class, rssi)
-
-            def inquiry_complete(self):
-                self.done = True
-
-        def __init__(self, session, major_device_class, minor_device_class):
-            threading.Thread.__init__(self)
-            self.session = session
-            self.major_device_class = major_device_class
-            self.minor_device_class = minor_device_class
-            self.cancel_discovery = False
-            self.ping_time = None
-
-        def discover(self):
-            discoverer = self.BTDiscoverer(self.major_device_class, self.minor_device_class)
-            discoverer.find_devices(lookup_names=True)
-            while self.session.status == self.session.DISCOVERY and not discoverer.done and not self.cancel_discovery:
-                readable = select.select([discoverer], [], [], 0.5)[0]
-                if discoverer in readable:
-                    discoverer.process_event()
-                    for addr, (device_name, device_class, rssi) in discoverer.found_devices.items():
-                        logger.debug(f"notifying discovered {addr}: {device_name}")
-                        params = {"rssi": rssi, 'peripheralId': addr, 'name': device_name.decode("utf-8")}
-                        self.session.notify('didDiscoverPeripheral', params)
-                    discoverer.found_devices.clear()
-
-            if not discoverer.done:
-                discoverer.cancel_inquiry()
-
-        def run(self):
-            while self.session.status != self.session.DONE:
-
-                logger.debug("loop in BT thread")
-                current_time = int(round(time.time()))
-
-                if self.session.status == self.session.DISCOVERY and not self.cancel_discovery:
-                    logger.debug("in discovery status:")
-                    try:
-                        self.discover()
-                        self.ping_time = current_time + 5
-                    finally:
-                        self.session.status = self.session.DISCOVERY_COMPLETE
-
-                elif self.session.status == self.session.CONNECTED:
-                    logger.debug("in connected status:")
-                    sock = self.session.sock
-                    try:
-                        ready = select.select([sock], [], [], 1)
-                        if ready[0]:
-                            header = sock.recv(2)
-                            [msg_len] = struct.unpack("<H", header)
-                            msg_data = sock.recv(msg_len)
-                            data = header + msg_data
-                            params = {'message': base64.standard_b64encode(data).decode('utf-8'), "encoding": "base64"}
-                            self.session.notify('didReceiveMessage', params)
-                            self.ping_time = current_time + 5
-
-                    except Exception as e:
-                            logger.error(e)
-                            self.session.close()
-                            break
-
-                    # To avoid repeated lock by this single thread,
-                    # yield CPU to other lock waiting threads.
-                    time.sleep(0)
-                else:
-                    # Nothing to do:
-                    time.sleep(1)
-
-                # Terminate if we have lost websocket connection to Scratch (e.g. browser closed)
-                if self.ping_time is None or self.ping_time <= current_time:
-                    try:
-                        self.session.notify('ping', {})
-                        self.ping_time = current_time + 5
-                    except Exception as e:
-                        logger.error(e)
-                        self.session.close()
-                        break
-
-    def __init__(self, websocket, loop):
-        super().__init__(websocket, loop)
-        self.status = self.INITIAL
-        self.sock = None
-        self.bt_thread = None
-
-    def close(self):
-        self.status = self.DONE
-        if self.sock:
-            logger.info(f"disconnect to BT socket: {self.sock}")
-            self.sock.close()
-            self.sock = None
-
-    def __del__(self):
-        self.close()
-
-    def handle_request(self, method, params):
-        """Handle requests from Scratch"""
-        logger.debug("handle request to BT device")
-        logger.debug(method)
-        if len(params) > 0:
-            logger.debug(params)
-
-        res = { "jsonrpc": "2.0" }
-
-        if self.status == self.INITIAL and method == 'discover':
-            logger.debug("Starting async discovery")
-            self.status = self.DISCOVERY
-            self.bt_thread = self.BTThread(self, params["majorDeviceClass"], params["minorDeviceClass"])
-            self.bt_thread.start()
-            res["result"] = None
-
-        elif self.status in [self.DISCOVERY, self.DISCOVERY_COMPLETE] and method == 'connect':
-
-            # Cancel discovery
-            while self.status == self.DISCOVERY:
-                logger.debug("Cancelling discovery")
-                self.bt_thread.cancel_discovery = True
-                time.sleep(1)
-
-            addr = params['peripheralId']
-            logger.debug(f"connecting to the BT device {addr}")
-            try:
-                self.sock = bluetooth.BluetoothSocket(bluetooth.RFCOMM)
-                self.sock.connect((addr, 1))
-                logger.info(f"connected to BT device: {addr}")
-            except bluetooth.BluetoothError as e:
-                logger.error(f"failed to connect to BT device: {e}", exc_info=e)
-                self.status = self.DONE
-                self.sock = None
-
-            if self.sock:
-                res["result"] = None
-                self.status = self.CONNECTED
-            else:
-                err_msg = f"BT connect failed: {addr}"
-                res["error"] = { "message": err_msg }
-                self.status = self.DONE
-
-        elif self.status == self.CONNECTED and method == 'send':
-            logger.debug("handle send request")
-            if params['encoding'] != 'base64':
-                logger.error("encoding other than base 64 is not "
-                                 "yet supported: ", params['encoding'])
-            msg_bstr = params['message'].encode('ascii')
-            data = base64.standard_b64decode(msg_bstr)
-            self.sock.send(data)
-            res['result'] = len(data)
-
-        logger.debug(res)
-        return res
-
-    def end_request(self):
-        logger.debug(f"end_request of BTSession {self}")
-        return self.status == self.DONE
-
-
 class BLESession(Session):
     """
-    Manage a session for Bluetooth Low Energy device such as micro:bit
+    Manage a session for Bluetooth Low Energy device such as micro:bit using
+    bluepy as backend.
     """
 
     INITIAL = 1
@@ -372,6 +223,7 @@ class BLESession(Session):
                     time.sleep(1)
                 elif self.session.status == self.session.CONNECTED:
                     logger.debug("in connected status:")
+
                     delegate = self.session.delegate
                     if delegate and len(delegate.handles) > 0:
                         if not delegate.restart_notification_event.is_set():
@@ -420,8 +272,8 @@ class BLESession(Session):
             params['message'] = base64.standard_b64encode(data).decode('ascii')
             self.session.notify('characteristicDidChange', params)
 
-    def __init__(self, websocket, loop):
+    def __init__(self, websocket, loop, scan_seconds):
-        super().__init__(websocket, loop)
+        super().__init__(websocket, loop, scan_seconds)
         self.status = self.INITIAL
         self.device = None
         self.deviceName = None
@@ -448,43 +300,50 @@ class BLESession(Session):
     def __del__(self):
         self.close()
 
-    def _get_dev_uuid(self, dev):
+    def _get_dev_uuids(self, dev):
         for adtype in self.SERVICE_CLASS_UUID_ADTYPES:
-            service_class_uuid = dev.getValueText(adtype)
+            service_class_uuids = dev.getValue(adtype)
-            if service_class_uuid:
+            if service_class_uuids:
-                logger.debug(self.SERVICE_CLASS_UUID_ADTYPES[adtype])
+                for u in service_class_uuids:
-                return UUID(service_class_uuid)
+                    a = self.SERVICE_CLASS_UUID_ADTYPES[adtype]
+                    logger.debug(f"service class uuid for {a}/{adtype}: {u}")
+                return service_class_uuids
         return None
 
     def matches(self, dev, filters):
         """
         Check if the found BLE device matches the filters Scratch specifies.
         """
-        logger.debug(f"in matches {dev.addr} {filters}")
+        logger.debug(f"in matches {dev.address} {filters}")
         for f in filters:
             if 'services' in f:
                 for s in f['services']:
                     logger.debug(f"service to check: {s}")
                     given_uuid = s
-                    logger.debug(f"given: {given_uuid}")
+                    logger.debug(f"given UUID: {given_uuid} hash={UUID(given_uuid).__hash__()}")
-                    dev_uuid = self._get_dev_uuid(dev)
+                    dev_uuids = self._get_dev_uuids(dev)
-                    if not dev_uuid:
+                    if not dev_uuids:
                         continue
-                    logger.debug(f"dev: {dev_uuid}")
+                    for u in dev_uuids:
-                    logger.debug(given_uuid == dev_uuid)
+                        logger.debug(f"dev UUID: {u} hash={u.__hash__()}")
-                    if given_uuid == dev_uuid:
+                        logger.debug(given_uuid == u)
+                        if given_uuid == u:
                             logger.debug("match...")
                             return True
             if 'namePrefix' in f:
-                # 0x08: Shortened Local Name
+                logger.debug(f"given namePrefix: {f['namePrefix']}")
-                deviceName = dev.getValueText(0x08)
+                deviceName = dev.getValueText(ScanEntry.SHORT_LOCAL_NAME)
-                if not deviceName:
+                if deviceName:
-                    continue
+                    logger.debug(f"SHORT_LOCAL_NAME: {deviceName}")
-                logger.debug(f"Name of \"{deviceName}\" begins with: \"{f['namePrefix']}\"?")
+                    if deviceName.startswith(f['namePrefix']):
-                if(deviceName.startswith(f['namePrefix'])):
+                        logger.debug(f"match...")
-                    logger.debug("Yes")
+                        return True
+                deviceName = dev.getValueText(ScanEntry.COMPLETE_LOCAL_NAME)
+                if deviceName:
+                    logger.debug(f"COMPLETE_LOCAL_NAME: {deviceName}")
+                    if deviceName.startswith(f['namePrefix']):
+                        logger.debug(f"match...")
                         return True
-                logger.debug("No")
             if 'name' in f or 'manufactureData' in f:
                 logger.error("name/manufactureData filters not implemented")
                 # TODO: implement other filters defined:
@@ -501,15 +360,23 @@ class BLESession(Session):
                 BLESession.found_devices.clear()
                 for i in range(self.MAX_SCANNER_IF):
                     scanner = Scanner(iface=i)
+                    for j in range(scan_retry):
                         try:
-                        devices = scanner.scan(10.0)
+                            logger.debug(f"start BLE scan: {self.scan_seconds} seconds")
+                            devices = scanner.scan(self.scan_seconds)
                             for dev in devices:
                                 if self.matches(dev, params['filters']):
                                     BLESession.found_devices.append(dev)
                                     found = True
                                     logger.debug(f"BLE device found with iface #{i}");
-                    except BTLEManagementError as e:
+                            if found:
-                        logger.debug(f"BLE iface #{i}: {e}");
+                                break
+                        except BTLEDisconnectError as de:
+                            logger.debug(f"BLE iface #{i}: {de}");
+                        except BTLEManagementError as me:
+                            logger.debug(f"BLE iface #{i}: {me}");
+                    if found:
+                        break
             else:
                 found = len(BLESession.found_devices) > 0
         return found
@@ -689,11 +556,13 @@ class BLESession(Session):
         return self.status == self.DONE
 
 async def ws_handler(websocket, path):
-    sessionTypes = { '/scratch/ble': BLESession, '/scratch/bt': BTSession }
+    global scan_seconds
+    sessionTypes = { '/scratch/bt': BTSession }
+    sessionTypes['/scratch/ble'] = ble.BLEDBusSession if dbus else BLESession
     try:
         logger.info(f"Start session for web socket path: {path}")
         loop = asyncio.get_event_loop()
-        session = sessionTypes[path](websocket, loop)
+        session = sessionTypes[path](websocket, loop, scan_seconds)
         await session.handle()
     except Exception as e:
         logger.error(f"Failure in session for web socket path: {path}")
@@ -715,18 +584,29 @@ def stack_trace():
          print(line)
 
 def main():
-    opts = [opt for opt in sys.argv[1:] if opt.startswith("-")]
+    global scan_seconds
-    if "-h" in opts:
+    global scan_retry
-        print((f"Usage: {sys.argv[0]} [OPTS]\n"
+    global dbus
-               "OPTS:\t-h Show this help.\n"
+    parser = argparse.ArgumentParser(description='start Scratch-link')
-               "\t-d Print debug messages."
+    parser.add_argument('-d', '--debug', action='store_true',
-        ))
+                        help='print debug messages')
-        sys.exit(1)
+    parser.add_argument('-s', '--scan_seconds', type=float, default=10.0,
-    elif "-d" in opts:
+                        help='specifiy duration to scan BLE devices in seconds')
+    parser.add_argument('-r', '--scan_retry', type=int, default=1,
+                        help='specifiy retry times to scan BLE devices')
+    parser.add_argument('-b', '--dbus', action='store_true',
+                        help='use DBus backend for BLE devices')
+    args = parser.parse_args()
+    if args.debug:
         print("Print debug messages")
         logLevel = logging.DEBUG
         handler.setLevel(logLevel)
         logger.setLevel(logLevel)
+    scan_seconds = args.scan_seconds
+    scan_retry = args.scan_retry
+    dbus = args.dbus
+    logger.debug(f"set scan_seconds: {scan_seconds}")
+    logger.debug(f"set scan_retry: {scan_retry}")
 
     # Prepare certificate of the WSS server
     gencert.prep_cert()
@@ -738,7 +618,7 @@ def main():
     ssl_context.load_cert_chain(localhost_cer, localhost_key)
 
     start_server = websockets.serve(
-         ws_handler, "device-manager.scratch.mit.edu", 20110, ssl=ssl_context
+         ws_handler, HOSTNAME, 20110, ssl=ssl_context
     )
 
     while True:
@@ -749,7 +629,13 @@ def main():
         except KeyboardInterrupt as e:
             stack_trace()
             break
+        except socket.gaierror as e:
+            logger.error(f"{type(e).__name__}: {e}")
+            logger.info(f"Check internet connection to {HOSTNAME}. If not "
+                        f"available, add '127.0.0.1 {HOSTNAME}' to /etc/hosts.")
+            break
         except Exception as e:
+            logger.error(f"{type(e).__name__}: {e}")
             logger.info("Restarting scratch-link...")
 
 if __name__ == "__main__":

release.sh

@@ -4,13 +4,17 @@ usage() {
 	echo "Usage: ${0} COMMAND"
 	echo -e "COMMAND:"
 	echo -e "\tbuild"
+	echo -e "\tupload FILES"
 	echo -e "\tupload-testpypi FILES"
+	echo -e "examples:"
+	echo -e "\t${0} build"
+	echo -e "\t${0} upload dist/*0.2.6*"
 	exit 1
 }
 
 case ${1} in
 	build)
-		python setup.py sdist bdist_wheel
+		python -m build
 		;;
 	upload)
 		shift

setup.py

@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
 
 setuptools.setup(
     name="pyscrlink",
-    version="0.2.2",
+    version="0.2.8",
     author="Shin'ichiro Kawasaki",
     author_email='kawasaki@juno.dti.ne.jp',
     description='Scratch-link for Linux with Python',
@@ -22,7 +22,6 @@ setuptools.setup(
     install_requires=[
         'websockets',
         'bluepy',
-        'pybluez',
         'pyOpenSSL',
         ],
     entry_points={