Thunderbolt Networking

Earlier this year I upgraded my MacBook Pro to one of the new ones with USB-C / Thunderbolt 3, and rebuilt my PC with a Thunderbolt 3-capable motherboard and a Thunderbolt 3 add-in card^[1].

Although I couldn't find much information online at the time, I can confirm that the Thunderbolt Bridge network adapter works between Windows and macOS hosts - by plugging two Thunderbolt-capable machines together, you automatically get a 10Gbps ethernet link.

I tried some simple artificial benchmarks and couldn't get anywhere close to the 10Gbps theoretical max throughput, but in real-world usage of copying large files over CIFS/SMB, the transfer speed appears to be limited by the target hard disk write speeds as I get insanely fast transfer when copying from Windows to my Mac (500-700MB/s) , but slower speeds (yet still quite fast) the other way around.

The problem with network file sharing over Thunderbolt is that by default, neither host is running a DHCP server on the Thunderbolt adapter. Windows and macOS traditionally aren't very good at finding each other by name, so I figured that either I can keep looking up the link-local address (169.254.0.0/16) that each machine keeps generating, or I can run a DHCP server that creates a /30 network (which has two available IP addresses). That way, the host running the server will have a fixed address, and any Thunderbolt-capable machine which connects will get the other address.

Due to my setup I wanted to run the DHCP server on the Windows box, however this is nearly impossible. On macOS, the Thunderbolt adapter is always around, and goes 'up' when connected and 'down' when disconnected.

On Windows, however, the Thunderbolt adapter gets created when the link is connected and gets destroyed when unplugged. This makes it impossible to bind a traditional DHCP server to the socket, as it all disappears after unplugging. The DHCP servers I tried had to be manually restarted after plugging in the Thunderbolt cable in order to re-bind to the newly-created adapter.

I played around for a few hours reading through pages upon pages of protocol specifications and managed to build my own tiny DHCP server which can dynamically handle network adapters being added and removed.

Four hours later, it "works". 😊 pic.twitter.com/HIPNeKNlKD

— Yaakov (@yaakov_h) April 29, 2017

After going through that, I thought maybe I can do something simpler. Windows and macOS both have a way to say "use DHCP for this adapter, but if that fails then use the IP settings I have specified" - or so I thought.

Windows has this capability, as shown below.

It turns out, however, that macOS does not. The options on a Mac are either:

• DHCP
• Static IP
• DHCP with Manual Address

Whilst DHCP with Manual Address sounds like what Windows has above, what this really does is use DHCP for subnet mask, default gateway, DNS and so on, but use the preconfigured IP address instead of the one offered up by the DHCP server.

This makes it impossible to use on the Thunderbolt Bridge, or any other adapter without a DHCP server, as the default subnet mask it picks is 255.255.255.255.

After all of this I then discovered an even simpler way to get the two machines to find each other.

It turns out that although my two PCs - named Eadu and Atollon - cannot reliably find each other by name, both Windows and macOS can reliably find the other machine if I search for Eadu.local and Atollon.local respectively.

From macOS:

[08:48pm yaakov@Eadu:~] ping atollon
ping: cannot resolve atollon: Unknown host
[08:48pm yaakov@Eadu:~] ping atollon.local
PING atollon.local (169.254.148.196): 56 data bytes
64 bytes from 169.254.148.196: icmp_seq=0 ttl=128 time=0.282 ms
^C
--- atollon.local ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.282/0.282/0.282/0.000 ms
[08:48pm yaakov@Eadu:~] _

From Windows:

C:\Users\yaakov>ping eadu

Pinging eadu [192.168.0.20] with 32 bytes of data:
Reply from 192.168.0.20: bytes=32 time=6ms TTL=64

Ping statistics for 192.168.0.20:
    Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 6ms, Maximum = 6ms, Average = 6ms
Control-C
^C
C:\Users\yaakov>ping eadu.local

Pinging eadu.local [fe80::1410:6b6b:XXXX:XXXX%17] with 32 bytes of data:
Reply from fe80::1410:6b6b:XXXX:XXXX%17: time<1ms

Ping statistics for fe80::1410:6b6b:XXXX:XXXX%17:
    Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
Control-C
^C
C:\Users\yaakov> _

I have no idea why, but it does work, so there you go.