BlueZ Profile RequireAuthentication=False not working when trying to initiate connection without pairing

Question

Following up from this question: How can I open a connection to my Raspberry Pi over Bluetooth without having to pair?

I am trying to connect from my Windows Laptop to my Raspberry Pi over Bluetooth without having to pair and I'm finding it very difficult to do. I've followed ukBaz's Answer however, even after creating a profile using BlueZ that has RequireAuthentication and RequireAuthorization both set to False, I am not able to connect unless I first pair the two devices.

I have also run the example provided in the answer with the same exact result

At this point, I'm sure this is some configuration issue with my Bluetooth settings or something.

Here's my Python code that is run on the Raspberry Pi:

# register bluetooth profile that doesn't require pairing before connection:
import os
import socket
from bluetooth.btcommon import RFCOMM

import dbus

from dbus.service import method
from dbus.mainloop.glib import DBusGMainLoop
from gi.repository import GLib



class NoAuthProfile(dbus.service.Object):
    fd = -1
    

    @method( dbus_interface='org.bluez.Profile1')
    def Release(self):
        print("Release")
        loop.quit()
    
    @method(dbus_interface='org.bluez.Profile1', in_signature="oha{sv}", out_signature="")
    def NewConnection(self, device, fd, fd_properties):
        print("New connection")
        self.fd = fd.take()
        self.client_socket = socket.socket(fileno=self.fd)

        self.client_socket.setblocking(True)

        #read input:
        while True:
            r = self.client_socket.recv(1024)
            print(r.decode())

    @method(dbus_interface='org.bluez.Profile1', in_signature="o", out_signature="")
    def RequestDisconnection(self, path):
        print("Disconnected")
        if self.fd > 0:
            self.client_socket.close()
            self.fd = -1


# connect to system daemon
dbus_loop = DBusGMainLoop()
bus = dbus.SystemBus(mainloop=dbus_loop)

# get profile_manager interface
profile_manager = dbus.Interface(bus.get_object('org.bluez', '/org/bluez'), 'org.bluez.ProfileManager1')

# initialize new profile
new_profile = NoAuthProfile(bus, '/org/bluez/test/no_auth_profile')

UUID = '54863f37-422f-4f04-9281-03e90a770aa0'

options = {
    'Name': dbus.types.String("NoAuthProfile"),
    "Channel": dbus.types.UInt16(1),
    "RequireAuthentication": dbus.types.Boolean(False),
    'RequireAuthorization': dbus.types.Boolean(False),
    "Service": dbus.types.String('54863f37-422f-4f04-9281-03e90a770aa0'),
    "AutoConnect": dbus.types.Boolean(True)
}
loop = GLib.MainLoop()


profile_manager.RegisterProfile('/org/bluez/test/no_auth_profile', UUID, options)
print("Listening on port 1...")

loop.run()

When paired it runs perfectly:

However, when not paired, the client application crashes:

Unhandled Exception: System.Net.Sockets.SocketException: A socket operation was attempted to an unreachable network B827EB131622:1 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) at System.Net.Sockets.Socket.Connect(EndPoint remoteEP) at InTheHand.Net.Bluetooth.Msft.SocketBluetoothClient.Connect(BluetoothEndPoint remoteEP) at InTheHand.Net.Sockets.BluetoothClient.Connect(BluetoothEndPoint remoteEP) at BluetoothTest2.Module1.Main() in C:\Users\..\BluetoothTest2\Module1.vb:line 51

General idea of VB.Net Code for client:

Dim cli As New BluetoothClient
Dim found() As BluetoothDeviceInfo = cli.DiscoverDevices()

Dim dx As BluetoothDeviceInfo = found(i) ' i is some device in list of found devices
Dim endp As New BluetoothEndPoint(dx.DeviceAddress, New Guid("54863f37-422f-4f04-9281-03e90a770aa0"), 1) ' some random UUID matching server

cli.Connect(endp) ' line 51

After reading the hcidump, I can see that the connection seems to initiate correctly and I think it even finishes but then I get a Disconnect event "Reason: Connection Terminated by Local Host"

> HCI Event: Connect Request (0x04) plen 10
    bdaddr 9C:B6:D0:FE:55:B4 class 0x2a010c type ACL

> HCI Event: Command Status (0x0f) plen 4
    Accept Connection Request (0x01|0x0009) status 0x00 ncmd 1

> HCI Event: Connect Complete (0x03) plen 11
    status 0x00 handle 11 bdaddr 9C:B6:D0:FE:55:B4 type ACL encrypt 0x00

> HCI Event: Command Status (0x0f) plen 4
    Read Remote Supported Features (0x01|0x001b) status 0x00 ncmd 1

> HCI Event: Read Remote Supported Features (0x0b) plen 11
    status 0x00 handle 11
    Features: 0xff 0xfe 0x8f 0xfe 0xd8 0x3f 0x5b 0x87

> HCI Event: Command Status (0x0f) plen 4
    Read Remote Extended Features (0x01|0x001c) status 0x00 ncmd 1

> HCI Event: Read Remote Extended Features (0x23) plen 13
    status 0x00 handle 11 page 1 max 2
    Features: 0x0f 0x00 0x00 0x00 0x00 0x00 0x00 0x00

> HCI Event: Command Status (0x0f) plen 4
    Remote Name Request (0x01|0x0019) status 0x00 ncmd 1

> HCI Event: Remote Name Req Complete (0x07) plen 255
    status 0x00 bdaddr 9C:B6:D0:FE:55:B4 name 'MY-LAPTOP'

> HCI Event: Command Complete (0x0e) plen 10
    IO Capability Request Reply (0x01|0x002b) ncmd 1
    status 0x00 bdaddr 9C:B6:D0:FE:55:B4

> HCI Event: Command Complete (0x0e) plen 10
    User Confirmation Request Reply (0x01|0x002c) ncmd 1
    status 0x00 bdaddr 9C:B6:D0:FE:55:B4

> HCI Event: Encrypt Change (0x08) plen 4
    status 0x00 handle 11 encrypt 0x02

> HCI Event: Command Complete (0x0e) plen 7
    Read Encryption Key Size (0x05|0x0008) ncmd 1

> HCI Event: Command Status (0x0f) plen 4
    Read Clock Offset (0x01|0x001f) status 0x00 ncmd 1

> HCI Event: Command Status (0x0f) plen 4
    Disconnect (0x01|0x0006) status 0x00 ncmd 1

> HCI Event: Disconn Complete (0x05) plen 4
    status 0x00 handle 11 reason 0x16
    Reason: Connection Terminated by Local Host

Looking at the Wireshark output (btvs) on the Windows Client end, I see something similar:

But this time I get a reason: "Authentication Failure", but unfortunately I'm not sure how to fix this.

How come, even after setting "RequireAuthentication" and "RequireAuthorization" to False in the BlueZ profile, is the non-paired device being rejected?

At this point I think I might give up and just go through the painless process of pairing but I would love to understand and gain more insight as to what this is and why it's happening.

---

Update 1/4/2022

After running sudo btmon:

> HCI Event: Connect Request (0x04) plen 10                                                         #1 [hci0] 29.398908
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Class: 0x2a010c
          Major class: Computer (desktop, notebook, PDA, organizers)
          Minor class: Laptop
          Networking (LAN, Ad hoc)
          Capturing (Scanner, Microphone)
          Audio (Speaker, Microphone, Headset)
        Link type: ACL (0x01)
< HCI Command: Accept Connection Request (0x01|0x0009) plen 7                                       #2 [hci0] 29.399054
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Role: Master (0x00)
> HCI Event: Command Status (0x0f) plen 4                                                           #3 [hci0] 29.402899
      Accept Connection Request (0x01|0x0009) ncmd 1
        Status: Success (0x00)
> HCI Event: Role Change (0x12) plen 8                                                              #4 [hci0] 29.617052
        Status: Success (0x00)
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Role: Master (0x00)
> HCI Event: Connect Complete (0x03) plen 11                                                        #5 [hci0] 29.644398
        Status: Success (0x00)
        Handle: 11
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Link type: ACL (0x01)
        Encryption: Disabled (0x00)
< HCI Command: Read Remote Supported Features (0x01|0x001b) plen 2                                  #6 [hci0] 29.644917
        Handle: 11
> HCI Event: Max Slots Change (0x1b) plen 3                                                         #7 [hci0] 29.650021
        Handle: 11
        Max slots: 5
> HCI Event: Command Status (0x0f) plen 4                                                           #8 [hci0] 29.650049
      Read Remote Supported Features (0x01|0x001b) ncmd 1
        Status: Success (0x00)
> HCI Event: Read Remote Supported Features (0x0b) plen 11                                          #9 [hci0] 29.650056
        Status: Success (0x00)
        Handle: 11
        Features: 0xff 0xfe 0x8f 0xfe 0xd8 0x3f 0x5b 0x87
          3 slot packets
          5 slot packets
          Encryption
          Slot offset
          Timing accuracy
          Role switch
          Hold mode
          Sniff mode
          Power control requests
          Channel quality driven data rate (CQDDR)
          SCO link
          HV2 packets
          HV3 packets
          u-law log synchronous data
          A-law log synchronous data
          CVSD synchronous data
          Paging parameter negotiation
          Power control
          Transparent synchronous data
          Broadcast Encryption
          Enhanced Data Rate ACL 2 Mbps mode
          Enhanced Data Rate ACL 3 Mbps mode
          Enhanced inquiry scan
          Interlaced inquiry scan
          Interlaced page scan
          RSSI with inquiry results
          Extended SCO link (EV3 packets)
          AFH capable slave
          AFH classification slave
          LE Supported (Controller)
          3-slot Enhanced Data Rate ACL packets
          5-slot Enhanced Data Rate ACL packets
          Sniff subrating
          Pause encryption
          AFH capable master
          AFH classification master
          Enhanced Data Rate eSCO 2 Mbps mode
          Extended Inquiry Response
          Simultaneous LE and BR/EDR (Controller)
          Secure Simple Pairing
          Encapsulated PDU
          Non-flushable Packet Boundary Flag
          Link Supervision Timeout Changed Event
          Inquiry TX Power Level
          Enhanced Power Control
          Extended features
< HCI Command: Read Remote Extended Features (0x01|0x001c) plen 3                                  #10 [hci0] 29.650236
        Handle: 11
        Page: 1
> HCI Event: Command Status (0x0f) plen 4                                                          #11 [hci0] 29.655418
      Read Remote Extended Features (0x01|0x001c) ncmd 1
        Status: Success (0x00)
> HCI Event: Read Remote Extended Features (0x23) plen 13                                          #12 [hci0] 29.655448
        Status: Success (0x00)
        Handle: 11
        Page: 1/2
        Features: 0x0f 0x00 0x00 0x00 0x00 0x00 0x00 0x00
          Secure Simple Pairing (Host Support)
          LE Supported (Host)
          Simultaneous LE and BR/EDR (Host)
          Secure Connections (Host Support)
> HCI Event: Page Scan Repetition Mode Change (0x20) plen 7                                        #13 [hci0] 29.655455
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Page scan repetition mode: R1 (0x01)
< HCI Command: Remote Name Request (0x01|0x0019) plen 10                                           #14 [hci0] 29.655733
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Page scan repetition mode: R2 (0x02)
        Page scan mode: Mandatory (0x00)
        Clock offset: 0x0000
< ACL Data TX: Handle 11 flags 0x00 dlen 10                                                        #15 [hci0] 29.655790
      L2CAP: Information Request (0x0a) ident 1 len 2
        Type: Extended features supported (0x0002)
> HCI Event: Command Status (0x0f) plen 4                                                          #16 [hci0] 29.657653
      Remote Name Request (0x01|0x0019) ncmd 1
        Status: Success (0x00)
> HCI Event: Remote Name Req Complete (0x07) plen 255                                              #17 [hci0] 29.666808
        Status: Success (0x00)
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Name: MY-LAPTOP
@ MGMT Event: Device Connected (0x000b) plen 33                                               {0x0002} [hci0] 29.666920
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Flags: 0x00000000
        Data length: 20
        Name (complete): MY-LAPTOP
        Class: 0x2a010c
          Major class: Computer (desktop, notebook, PDA, organizers)
          Minor class: Laptop
          Networking (LAN, Ad hoc)
          Capturing (Scanner, Microphone)
          Audio (Speaker, Microphone, Headset)
@ MGMT Event: Device Connected (0x000b) plen 33                                               {0x0001} [hci0] 29.666920
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Flags: 0x00000000
        Data length: 20
        Name (complete): MY-LAPTOP
        Class: 0x2a010c
          Major class: Computer (desktop, notebook, PDA, organizers)
          Minor class: Laptop
          Networking (LAN, Ad hoc)
          Capturing (Scanner, Microphone)
          Audio (Speaker, Microphone, Headset)
> HCI Event: IO Capability Response (0x32) plen 9                                                  #18 [hci0] 29.693460
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        IO capability: DisplayYesNo (0x01)
        OOB data: Authentication data not present (0x00)
        Authentication: No Bonding - MITM not required (0x00)
> HCI Event: IO Capability Request (0x31) plen 6                                                   #19 [hci0] 29.693554
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
< HCI Command: IO Capability Request Reply (0x01|0x002b) plen 9                                    #20 [hci0] 29.693724
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        IO capability: NoInputNoOutput (0x03)
        OOB data: Authentication data not present (0x00)
        Authentication: No Bonding - MITM not required (0x00)
> HCI Event: Command Complete (0x0e) plen 10                                                       #21 [hci0] 29.695453
      IO Capability Request Reply (0x01|0x002b) ncmd 1
        Status: Success (0x00)
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
> ACL Data RX: Handle 11 flags 0x02 dlen 16                                                        #22 [hci0] 29.695482
      L2CAP: Information Response (0x0b) ident 1 len 8
        Type: Extended features supported (0x0002)
        Result: Success (0x0000)
        Features: 0x000000b8
          Enhanced Retransmission Mode
          Streaming Mode
          FCS Option
          Fixed Channels
< ACL Data TX: Handle 11 flags 0x00 dlen 10                                                        #23 [hci0] 29.695666
      L2CAP: Information Request (0x0a) ident 2 len 2
        Type: Fixed channels supported (0x0003)
> HCI Event: Number of Completed Packets (0x13) plen 5                                             #24 [hci0] 29.700247
        Num handles: 1
        Handle: 11
        Count: 2
> ACL Data RX: Handle 11 flags 0x02 dlen 20                                                        #25 [hci0] 29.700284
      L2CAP: Information Response (0x0b) ident 2 len 12
        Type: Fixed channels supported (0x0003)
        Result: Success (0x0000)
        Channels: 0x0000000000000082
          L2CAP Signaling (BR/EDR)
          Security Manager (BR/EDR)
> HCI Event: User Confirmation Request (0x33) plen 10                                              #26 [hci0] 30.382842
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Passkey: 361135
< HCI Command: User Confirmation Request Reply (0x01|0x002c) plen 6                                #27 [hci0] 30.382970
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
> HCI Event: Command Complete (0x0e) plen 10                                                       #28 [hci0] 30.386478
      User Confirmation Request Reply (0x01|0x002c) ncmd 1
        Status: Success (0x00)
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
> HCI Event: Simple Pairing Complete (0x36) plen 7                                                 #29 [hci0] 30.958886
        Status: Success (0x00)
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
> HCI Event: Link Key Notification (0x18) plen 23                                                  #30 [hci0] 31.003083
        Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Link key: aa182c9e3106b28fdb98bda8c6bb5dc2
        Key type: Debug Combination key (0x03)
@ MGMT Event: New Link Key (0x0009) plen 26                                                   {0x0002} [hci0] 31.003235
        Store hint: No (0x00)
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Key type: Debug Combination key (0x03)
        Link key: aa182c9e3106b28fdb98bda8c6bb5dc2
        PIN length: 0
@ MGMT Event: New Link Key (0x0009) plen 26                                                   {0x0001} [hci0] 31.003235
        Store hint: No (0x00)
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Key type: Debug Combination key (0x03)
        Link key: aa182c9e3106b28fdb98bda8c6bb5dc2
        PIN length: 0
> HCI Event: Encryption Change (0x08) plen 4                                                       #31 [hci0] 31.047965
        Status: Success (0x00)
        Handle: 11
        Encryption: Enabled with AES-CCM (0x02)
< HCI Command: Read Encryption Key Size (0x05|0x0008) plen 2                                       #32 [hci0] 31.048139
        Handle: 11
> HCI Event: Command Complete (0x0e) plen 7                                                        #33 [hci0] 31.048609
      Read Encryption Key Size (0x05|0x0008) ncmd 1
        Status: Success (0x00)
        Handle: 11
        Key size: 16
< ACL Data TX: Handle 11 flags 0x00 dlen 11                                                        #34 [hci0] 31.048845
      BR/EDR SMP: Pairing Request (0x01) len 6
        IO capability: DisplayOnly (0x00)
        OOB data: Authentication data not present (0x00)
        Authentication requirement: No bonding, No MITM, Legacy, No Keypresses, CT2 (0x20)
        Max encryption key size: 16
        Initiator key distribution: EncKey Sign (0x05)
        Responder key distribution: EncKey IdKey Sign (0x07)
> ACL Data RX: Handle 11 flags 0x02 dlen 10                                                        #35 [hci0] 31.051043
      L2CAP: Information Request (0x0a) ident 193 len 2
        Type: Extended features supported (0x0002)
< ACL Data TX: Handle 11 flags 0x00 dlen 16                                                        #36 [hci0] 31.051196
      L2CAP: Information Response (0x0b) ident 193 len 8
        Type: Extended features supported (0x0002)
        Result: Success (0x0000)
        Features: 0x000002b8
          Enhanced Retransmission Mode
          Streaming Mode
          FCS Option
          Fixed Channels
          Unicast Connectionless Data Reception
> HCI Event: Number of Completed Packets (0x13) plen 5                                             #37 [hci0] 31.054684
        Num handles: 1
        Handle: 11
        Count: 2
> ACL Data RX: Handle 11 flags 0x02 dlen 10                                                        #38 [hci0] 31.055839
      L2CAP: Information Request (0x0a) ident 194 len 2
        Type: Fixed channels supported (0x0003)
< ACL Data TX: Handle 11 flags 0x00 dlen 20                                                        #39 [hci0] 31.055980
      L2CAP: Information Response (0x0b) ident 194 len 12
        Type: Fixed channels supported (0x0003)
        Result: Success (0x0000)
        Channels: 0x0000000000000086
          L2CAP Signaling (BR/EDR)
          Connectionless reception
          Security Manager (BR/EDR)
> ACL Data RX: Handle 11 flags 0x02 dlen 12                                                        #40 [hci0] 31.057115
      L2CAP: Connection Request (0x02) ident 195 len 4
        PSM: 3 (0x0003)
        Source CID: 64
< ACL Data TX: Handle 11 flags 0x00 dlen 16                                                        #41 [hci0] 31.057362
      L2CAP: Connection Response (0x03) ident 195 len 8
        Destination CID: 64
        Source CID: 64
        Result: Connection successful (0x0000)
        Status: No further information available (0x0000)
< ACL Data TX: Handle 11 flags 0x00 dlen 27                                                        #42 [hci0] 31.057412
      L2CAP: Configure Request (0x04) ident 3 len 19
        Destination CID: 64
        Flags: 0x0000
        Option: Maximum Transmission Unit (0x01) [mandatory]
          MTU: 1021
        Option: Retransmission and Flow Control (0x04) [mandatory]
          Mode: Basic (0x00)
          TX window size: 0
          Max transmit: 0
          Retransmission timeout: 0
          Monitor timeout: 0
          Maximum PDU size: 0
> ACL Data RX: Handle 11 flags 0x02 dlen 6                                                         #43 [hci0] 31.062609
      BR/EDR SMP: Pairing Failed (0x05) len 1
        Reason: Cross-transport Key Derivation/Generation not allowed (0x0e)
@ MGMT Event: Authentication Failed (0x0011) plen 8                                           {0x0002} [hci0] 31.062746
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Status: Authentication Failed (0x05)
@ MGMT Event: Authentication Failed (0x0011) plen 8                                           {0x0001} [hci0] 31.062746
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Status: Authentication Failed (0x05)
> HCI Event: Number of Completed Packets (0x13) plen 5                                             #44 [hci0] 31.062642
        Num handles: 1
        Handle: 11
        Count: 2
> ACL Data RX: Handle 11 flags 0x02 dlen 16                                                        #45 [hci0] 31.062650
      L2CAP: Configure Request (0x04) ident 196 len 8
        Destination CID: 64
        Flags: 0x0000
        Option: Maximum Transmission Unit (0x01) [mandatory]
          MTU: 1017
< HCI Command: Read Clock Offset (0x01|0x001f) plen 2                                              #46 [hci0] 31.062968
        Handle: 11
> ACL Data RX: Handle 11 flags 0x02 dlen 29                                                        #47 [hci0] 31.063782
      L2CAP: Configure Response (0x05) ident 3 len 21
        Source CID: 64
        Flags: 0x0000
        Result: Success (0x0000)
        Option: Maximum Transmission Unit (0x01) [mandatory]
          MTU: 1017
        Option: Retransmission and Flow Control (0x04) [mandatory]
          Mode: Basic (0x00)
          TX window size: 0
          Max transmit: 0
          Retransmission timeout: 0
          Monitor timeout: 0
          Maximum PDU size: 0
> HCI Event: Command Status (0x0f) plen 4                                                          #48 [hci0] 31.064181
      Read Clock Offset (0x01|0x001f) ncmd 1
        Status: Success (0x00)
< HCI Command: Disconnect (0x01|0x0006) plen 3                                                     #49 [hci0] 31.064306
        Handle: 11
        Reason: Authentication Failure (0x05)
> HCI Event: Command Status (0x0f) plen 4                                                          #50 [hci0] 31.064846
      Disconnect (0x01|0x0006) ncmd 1
        Status: Success (0x00)
> HCI Event: Number of Completed Packets (0x13) plen 5                                             #51 [hci0] 31.142013
        Num handles: 1
        Handle: 11
        Count: 1
> HCI Event: Disconnect Complete (0x05) plen 4                                                     #52 [hci0] 31.142038
        Status: Success (0x00)
        Handle: 11
        Reason: Connection Terminated By Local Host (0x16)
@ MGMT Event: Device Disconnected (0x000c) plen 8                                             {0x0002} [hci0] 31.142162
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Reason: Connection terminated by local host (0x02)
@ MGMT Event: Device Disconnected (0x000c) plen 8                                             {0x0001} [hci0] 31.142162
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Reason: Connection terminated by local host (0x02)
@ MGMT Command: Unpair Device (0x001b) plen 8                                                 {0x0001} [hci0] 31.162623
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Disconnect: Enabled (0x01)
@ MGMT Event: Command Complete (0x0001) plen 10                                               {0x0001} [hci0] 31.162673
      Unpair Device (0x001b) plen 7
        Status: Not Paired (0x06)
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)

I think some important lines to note are:

> ACL Data RX: Handle 11 flags 0x02 dlen 6                                                         #43 [hci0] 31.062609
      BR/EDR SMP: Pairing Failed (0x05) len 1
        Reason: Cross-transport Key Derivation/Generation not allowed (0x0e)
@ MGMT Event: Authentication Failed (0x0011) plen 8                                           {0x0002} [hci0] 31.062746
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Status: Authentication Failed (0x05)
@ MGMT Event: Authentication Failed (0x0011) plen 8                                           {0x0001} [hci0] 31.062746
        BR/EDR Address: 9C:B6:D0:FE:55:B4 (Rivet Networks)
        Status: Authentication Failed (0x05)

< HCI Command: Disconnect (0x01|0x0006) plen 3                                                     #49 [hci0] 31.064306
        Handle: 11
        Reason: Authentication Failure (0x05)
> HCI Event: Command Status (0x0f) plen 4                                                          #50 [hci0] 31.064846
      Disconnect (0x01|0x0006) ncmd 1
        Status: Success (0x00)
> HCI Event: Number of Completed Packets (0x13) plen 5                                             #51 [hci0] 31.142013
        Num handles: 1
        Handle: 11
        Count: 1
> HCI Event: Disconnect Complete (0x05) plen 4                                                     #52 [hci0] 31.142038
        Status: Success (0x00)
        Handle: 11
        Reason: Connection Terminated By Local Host (0x16)

Update 1/4/2022 - Solution!?

So after trying random stuff on the internet, I found this post:

Having an issue with Bluetooth Manager/Pairing on the Pi

Although I was skeptical since I wasn't making an audio device I decided to follow the top answer:

sudo apt-get install pulseaudio-module-bluetooth
pulseaudio --start

And somehow, for some reason, it magically connects and works exactly as it's supposed to! In fact, I was having some issues pairing where randomly the Pi would keep unpairing and disconnecting from my laptop when trying to pair - but that got fixed too! However, Windows for some reason thinks it's an audio device...

However, I'm still confused as hell why this works, and why nothing was indicating in the logs that I was missing a required audio protocol.

In fact, now when I stop the pulseaudio daemon, it doesn't pair or make connection at all.

If anyone could point me to a log file on either Windows or Linux that would show an error like "Missing pulseaudio... etc..." and tell me why it's happening and maybe provide a solution I'll mark it as an answer.

Answer 1

I have tested the Python code you posted in your question on my system and it also didn't work.

Looking at it more closely, and running some experiments, I noticed that you were not making the server discoverable so I added the following lines to make it discoverable if it was not already:

adapter = dbus.Interface(bus.get_object('org.bluez',
                                        '/org/bluez/hci0'),
                         dbus.PROPERTIES_IFACE)
discoverable = adapter.Get('org.bluez.Adapter1', 'Discoverable')
if not discoverable:
    print('Making discoverable...')
    adapter.Set('org.bluez.Adapter1', 'Discoverable', True)

After this change I was able to connect from my Python client on a Windows machine.

For clarity here is the full code that I used as the server:

import socket

import dbus

from dbus.service import method
from dbus.mainloop.glib import DBusGMainLoop
from gi.repository import GLib


class NoAuthProfile(dbus.service.Object):
    fd = -1

    @method(dbus_interface='org.bluez.Profile1')
    def Release(self):
        print("Release")
        loop.quit()

    @method(dbus_interface='org.bluez.Profile1', in_signature="oha{sv}", out_signature="")
    def NewConnection(self, device, fd, fd_properties):
        print("New connection")
        self.fd = fd.take()
        self.client_socket = socket.socket(fileno=self.fd)

        self.client_socket.setblocking(True)

        # read input:
        while True:
            r = self.client_socket.recv(1024)
            print(r.decode())
            if r:
                self.client_socket.send(bytes(list(reversed(r.rstrip()))) + b'\n')

    @method(dbus_interface='org.bluez.Profile1', in_signature="o", out_signature="")
    def RequestDisconnection(self, path):
        print("Disconnected")
        if self.fd > 0:
            self.client_socket.close()
            self.fd = -1


# connect to system daemon
dbus_loop = DBusGMainLoop()
bus = dbus.SystemBus(mainloop=dbus_loop)

# get profile_manager interface
profile_manager = dbus.Interface(bus.get_object('org.bluez', '/org/bluez'), 'org.bluez.ProfileManager1')

# Make server discoverable
adapter = dbus.Interface(bus.get_object('org.bluez',
                                        '/org/bluez/hci0'),
                         dbus.PROPERTIES_IFACE)
discoverable = adapter.Get('org.bluez.Adapter1', 'Discoverable')
if not discoverable:
    print('Making discoverable...')
    adapter.Set('org.bluez.Adapter1', 'Discoverable', True)

# initialize new profile
new_profile = NoAuthProfile(bus, '/org/bluez/test/no_auth_profile')

UUID = '54863f37-422f-4f04-9281-03e90a770aa0'

options = {
    'Name': dbus.types.String("NoAuthProfile"),
    "Channel": dbus.types.UInt16(1),
    "RequireAuthentication": dbus.types.Boolean(False),
    'RequireAuthorization': dbus.types.Boolean(False),
    "Service": dbus.types.String('54863f37-422f-4f04-9281-03e90a770aa0'),
    "AutoConnect": dbus.types.Boolean(True)
}
loop = GLib.MainLoop()

profile_manager.RegisterProfile('/org/bluez/test/no_auth_profile', UUID, options)
print("Listening on port 1...")

loop.run()

The other changes I made was to remove the import of bluetooth and os as they are not used.

On the Windows machine, Python 3.9 or newer is required for the following client:

import socket

server_address = "xx:xx:xx:xx:xx:xx"
server_port = 1
with socket.socket(socket.AF_BLUETOOTH,
                   socket.SOCK_STREAM,
                   socket.BTPROTO_RFCOMM) as c:

    c.connect((server_address, server_port))
    c.send(b"desserts")
    print(c.recv(1024).decode())

Answer 2

This answer only relates to the UUID question in the comments. See section 5.1 in https://github.com/petzval/btferret documentation for a discussion of serial UUIDs. The specification of a service as a serial RFCOMM channel is in the aid=0004 SDP entry rather than the UUID, so any number can be used as the UUID. However, by convention serial services have 1101 as the 3rd/4th bytes or just a 16-bit 1101 UUID, and some clients (such as the Windows COM connection) are fussy and only recognise 1101 UUIDs. But this does not seem to be the case here, and the specified UUID should work. It's not an answer, but I've seen bluez disconnect in this way without explanation also, and it may be that the client is insisting on authentication. In case you want to go to the trouble of trying it as the Pi server, btferret does not require authentication and will never initiate a disconnection as here, and the verbose print mode might reveal what is happening.