Travelling with an iPhone as a hotspot is good enough in a pinch, but the signal is weak and will cause disconnections if used as the main Access Point in a home.

This post explains how to connect an iPhone to a Ubuntu machine via usb and then share the internet connection to a wifi router via the ethernet port of that machine.

iPhone + Ubuntu box + Router network

I am using a TP-Link WR1502X Travel Router. The router is supposed to have built-in USB tethering, but it did not work with my iPhone 15 Pro running iOS 17. An older iPhone SE 2020 with iOS 16 works, so it goes to show how reliable these USB tethering things are. Putting an HP T630 Thin Client to act as our Gateway via ethernet will fix this.

Configuring the Linux PC

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# see these values with `sudo lshw -class network -short`
IPHONE_NIC='enxd26b7855052c' # the iphone's Ethernet "virtual connection"
ETHERNET_NIC='enp1s0' # your network card where you'll connect the router via cable


# Enable IP Forwarding sysctl -w net.ipv4.ip_forward=1. This will enable the kernel to forward packets, which are arriving to this machine.
sudo sysctl -w net.ipv4.ip_forward=1

# Assign a static IP within the same mask as the router's DHCP pool, but that you will later assign as a reserved IP in the DHCP pool
sudo ifconfig $ETHERNET_NIC 192.168.0.253 netmask 255.255.255.0 up

# install persistent iptables to make the forwarding -> masquerading permanent
sudo apt install -y iptables-persistent
sudo systemctl enable netfilter-persistent.service
sudo systemctl status netfilter-persistent.service

# Enable masquerading on the interface which is connected to the internet. sudo iptables -t nat -A POSTROUTING -o $IPHONE_NIC -j MASQUERADE. This will masquerade (replace the src ip on the packet with the $IPHONE_NIC ip) all traffic arriving from other interfaces, to the $IPHONE_NIC interface.
sudo iptables -t nat -A POSTROUTING -o $IPHONE_NIC -j MASQUERADE

# Add iptable rules to ACCEPT and FORWARD traffic from the subnet
sudo iptables -I FORWARD -o $IPHONE_NIC -s 192.168.0.0/16 -j ACCEPT
sudo iptables -I INPUT -s 192.168.0.0/16 -j ACCEPT

# Make changes persistent across boots
sudo iptables-save | sudo tee /etc/iptables/rules.v4

Router configuration

Router will be at http://192.168.0.1 when you are connected to its Wifi network.

1. Put router in router mode (physical slider)
2. Put ethernet cable in LAN port and connect to linux host ethernet port
3. Access http://192.168.0.1
4. Network tab will show error, never mind that
5. Internet tab - Type: Router (Current) - Internet connection type -> Dynamic IP - Use Default MAC address
6. Wifi configure how you want (SSID + PASS)

7. Advanced - Internet Connection Type -> Click Renew lease - LAN - IP Address -> 192.168.0.1

8. DHCP Server - Enable - IP Address pool - 192.168.0.2 - 192.168.0.100 - Default Gateway - 192.168.0.253 (Static IP in Linux Machine’s ethernet port) - Primary DNS: 8.8.8.8 - Secondary DNS: 8.8.4.4 - Address Reservation -> hotspot (linux machine) - MAC ADDRESS: Computer’s Ethernet MAC address (use ifconfig to see it) - IP Address: 192.168.0.253

References

Sharing internet connection from a linux machine over Ethernet.

Replace libinput touchpad driver with the Synaptics driver

I find that the Synaptics touchpad driver for the P1 performs better than the default one. If you are happy with it, skip everything regarding the Trackpad.

First, go here and install the synaptics driver for the touchpad. You will be prompted for a password that you need to enter again on a blue screen that will show up after the first reboot, since this driver lives in the UEFI.

Install Event Device driver for the TrackPoint

We will be replacing libinput with evdev driver for the trackpoint. It has been around since the IBM laptop days, so this guarantees an even more Vintage experience:

1
sudo apt-get install evdev

Configuration files

You now need to place a few files in certain locations.

sudo vim /etc/X11/xorg.conf.d/70-synaptics.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
# Example xorg.conf.d snippet that assigns the touchpad driver
# to all touchpads. See xorg.conf.d(5) for more information on
# InputClass.
# DO NOT EDIT THIS FILE, your distribution will likely overwrite
# it when updating. Copy (and rename) this file into
# /etc/X11/xorg.conf.d first.
# Additional options may be added in the form of
#   Option "OptionName" "value"
#
Section "InputClass"
        Identifier "touchpad catchall"
        MatchIsTouchpad "on"
# This option is recommend on all Linux systems using evdev, but cannot be
# enabled by default. See the following link for details:
# http://who-t.blogspot.com/2010/11/how-to-ignore-configuration-errors.html
       	MatchDevicePath "/dev/input/event*"
       	Option "PalmDetect" "1"
       	Option "PalmMinWidth" "10"
       	Option "PalmMinZ" "50"
       	Option "HorizTwoFingerScroll" "1"
	Option "MinSpeed"              "0.6"
        Option "MaxSpeed"              "2.3"
        Option "AccelFactor"           "0.11"
        Option "TapButton1"            "1"
        Option "TapButton2"            "2"     # multitouch
        Option "TapButton3"            "3"     # multitouch
        Option "VertTwoFingerScroll"   "1"     # multitouch
        Option "HorizTwoFingerScroll"  "1"     # multitouch
        Option "VertEdgeScroll"        "1"
        Option "CoastingSpeed"         "0"
        Option "CornerCoasting"        "1"
        Option "CircularScrolling"     "1"
        Option "CircScrollTrigger"     "7"
        Option "EdgeMotionUseAlways"   "1"
        Option "LBCornerButton"        "8"     # browser "back" btn
        Option "RBCornerButton"        "9"     # browser "forward" btn
EndSection

Section "InputClass"
        Identifier "touchpad ignore duplicates"
        MatchIsTouchpad "on"
        MatchOS "Linux"
        MatchDevicePath "/dev/input/mouse*"
        Option "Ignore" "on"
EndSection

# This option enables the bottom right corner to be a right button on clickpads
# and the right and middle top areas to be right / middle buttons on clickpads
# with a top button area.
# This option is only interpreted by clickpads.
Section "InputClass"
        Identifier "Default clickpad buttons"
        MatchDriver "synaptics"
        Option "SoftButtonAreas" "50% 0 82% 0 0 0 0 0"
        Option "SecondarySoftButtonAreas" "58% 0 0 15% 42% 58% 0 15%"
EndSection

# This option disables software buttons on Apple touchpads.
# This option is only interpreted by clickpads.
Section "InputClass"
        Identifier "Disable clickpad buttons on Apple touchpads"
        MatchProduct "Apple|bcm5974"
        MatchDriver "synaptics"
        Option "SoftButtonAreas" "0 0 0 0 0 0 0 0"
EndSection

sudo vim /etc/X11/xorg.conf.d/90-trackpoint.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# Section "InputClass"  
#    Identifier "libinput pointer catchall"  
#    MatchIsPointer "on"  
#    MatchDevicePath "/dev/input/event*"  
#    Driver "libinput"  
#    Option "AccelSpeed" "-1.25"  
# EndSection  

Section "InputClass"
    Identifier "Trackpoint Settings"
    MatchProduct "TPPS/2 Elan TrackPoint"
    MatchDevicePath "/dev/input/event*"
    Driver "evdev"
    Option "EmulateWheel" "true"
    Option "EmulateWheelButton" "2"
    # Option "Emulate3Buttons" "true"
    Option "EmulateWheelInertia"  "10'
    Option "EmulateWheelTimeOut" "200"
    Option "Emulate3Timeout" "50"
    Option "XAxisMapping" "6 7"
    Option "YAxisMapping" "4 5"
    Option "ButtonMapping" "1 0 3 4 5 6 7"


# Set up an acceleration config
    Option   "VelocityScale"           "5"
    Option   "AccelerationProfile"     "7"
    Option   "AccelerationNumerator"   "14"
    Option   "AccelerationDenominator" "3"
    Option   "ConstantDeceleration"  "2"
    option   "AdaptiveDeceleration"  "3"
    Option   "AccelerationScheme" "predictable"
    Option   "AccelerationThreshold" "6"
EndSection
 

Gnome settings

• In the Gnome Extensions app, go to “Keyboard and Mouse” and make sure that the mouse acceleration profile is “Flat”.
• In the Gnome Setting app, make sure that the trackpad and mouse speed sliders are in the default (middle) position.

References

How do I set acceleration on “TPPS/2 IBM TrackPoint” ?.

How to get the perfect TrackPoint experience on Linux.

Unfortunately, most examples I found on the web using the iTerm2 Python API started everything in parallel. I needed to actually wait for each server to print its “ready/listening” message before moving on to the next one. This rules out port clash problems, synchronisation conflicts, etc. Also, when running on a laptop, sometimes there is not enough CPU power to boot up everything simultaneously.

Here is my example script for starting multiple services in iTerm2, each on their separate pane. Each one will wait for a completion message before opening the next pane. I separated the logic into a neat open_pane_and_start_service function that you can call multiple times for starting up all the services you need.

Where to put the script

To use it, save it with a .py extension in the iTerm2 scripts folder.

Opening the iTerm2 Python Scripts folder

The script will then appear in the list of available scripts:

Script visible in the list of iTerm scripts

What it does, in detail

The script will:

1. Open a new pane at a folder that you specify
   1. Play with Vertical=True or Vertical=False to achieve the layout you need
   2. Provide an optional pane_to_split value to specify a previous pane to split from
2. Run a command
3. Wait for a certain line to appear on the screen (typically Listening at... or similar)
4. Move on to the next service.

Personally, I use this with multiple mutagen sessions, which sometimes can take 10+ minutes to perform a full file synchronization first thing in the morning.

The code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
#!/usr/bin/env python3.7
import asyncio
import re
import iterm2


async def open_pane_and_start_service(
        window,
        fs_path,
        command,
        string_to_detect_to_finish,
        pane_to_split=None,
        vertical=False,
):
    tab = window.current_tab
    session = tab.current_session
    pane = session

    if pane_to_split is not None:
        pane = await session.async_split_pane(vertical)
        await pane.async_activate()

    async def run_fs():
        await pane.async_send_text('cd ' + fs_path + '\n')
        await pane.async_send_text(command + '\n')

        finished = False
        async with pane.get_screen_streamer() as streamer:
            while not finished:
                stringified_string_contents = ""
                screen_output = await streamer.async_get()
                for line_no in range(screen_output.number_of_lines + screen_output.number_of_lines_above_screen):
                    stringified_string_contents = stringified_string_contents + screen_output.line(line_no).string
                if len(stringified_string_contents) > 0:
                    finished = re.search(pattern=string_to_detect_to_finish,
                                         string=stringified_string_contents,
                                         flags=re.IGNORECASE) is not None
            print("Found string [ " + string_to_detect_to_finish + " ] , service is started.")
        return pane

    loop = asyncio.get_event_loop()
    f_task = loop.create_task(run_fs())

    await f_task
    return pane


async def main(connection):
    app = await iterm2.async_get_app(connection)
    window = app.current_terminal_window

    if window is not None:
        # Service 1
        service_1 = await open_pane_and_start_service(
            window,
            "~/GitHub/service_1",
            "make start-dev",
            "Watching for changes...",
            vertical=False,
        )

        # Service 2
        service_2 = await open_pane_and_start_service(
            window,
            "~/GitHub/service_2",
            "make start-dev",
            "compiled successfully in",
            pane_to_split=service_1,
            vertical=True
        )

        # Service 3
        service_2 = await open_pane_and_start_service(
            window,
            "~/GitHub/service_3",
            "make build && make start",
            "Listening on",
            vertical=False,
            pane_to_split = service_2
        )

        # Service 4
        service_4 = await open_pane_and_start_service(
            window,
            "~/GitHub/service_4",
            "make start",
            "Server started at",
            vertical=True,
            pane_to_split = service_3
        )
    else:
        print("No current window")


iterm2.run_until_complete(main)

Now you can start all your microservices while you brew your morning coffee!

1. After installing macports, install the following 2 packages ¹:

   1
   2
    sudo port install clamav
    sudo port install clamav-server
   

2. By default, the freshclam.conf file comes empty ². Let’s fix it. Edit the freshclam (process that regularly updates the virus definitions database)

   1
    sudo vim /opt/local/etc/freshclam.conf
   

   Add this if the file does not have the DatabaseMirror line ³:

   1
    DatabaseMirror database.clamav.net
   

3. Update antivirus definitions database

   1
    sudo freshclam
   

   It will download a lot of data (> 250MB). Please wait until it finishes.

4. Enable clamav daemons for continuous database upgrades and background virus scanning

   1
    sudo port load clamav-server
   

That being said, I do not recommend using a completely outdated and insecure OS as your main machine on a day-to-day basis. If my 2008 MacBook Pro were my only machine I would be running an up-to-date Linux distro.

Preferences dialog in XCode 14.0 - Setting up the Command Line Tools home

1. Download the homebrew install script, disable check for command line tools

1
curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh > install.sh

Open install.sh, and replace

1
2
3
should_install_command_line_tools() {
	-----
}

with

1
2
3
should_install_command_line_tools() {
	return false;
}

1. Homebrew will stop complaining and casks will install now:

Modifying the homebrew install script not to install Commandline Tools

iCloud always full! Filming in 4k fills up my iCloud drive in minutes!

Is the image above a familiar sight? To me it is, and I today I am showing you how to implement an automated workflow for organising my media. It goes along these lines:

1. Send my pictures and video to my Mac Mini using AirDrop (Apple’s wireless file transfer that works pretty well).
2. The files are saved by AirDrop in a folder in my Mac Mini.
3. I should not need to log into the machine to AirDrop files into it. Ideally, this workflow should work with only a power and network cable connected to the Mini.
4. A script detects these new files and, after looking at each file’s EXIF metadata (date taken), sends the files to my external hard drive in an organised structure.
5. The hard drive is organised in a simple folder structure: <year>/<month>/<day>/<filename>.
   • Previous experience has taught me that relying on apps like Aperture (deprecated), Apple Photos (more incompatible versions than I remember) or Adobe Lightroom is not a good long-term choice. These apps are nice and beautiful but ultimately make you hostage by holding your photos while they force you to pay/upgrade again and again when and the old one ceases to work on new OS versions or Apple drops support for your machine.
6. Duplicates will be detected and deleted from the target folder to save space.
7. The script shall run continuously in the background, monitoring the folder for new media files, so that I do not need to perform any manual actions.
8. The script shall use efficient OS mechanisms like fswatch (which I previously combined with rsync here to avoid having to scan the whole folder continuously. Event handling instead of polling, to minimise resource usage.
9. All this has to use free software, because I need money to pay for petrol (€2.30/l in my country at time of writing).

Selecting a media organisation software

Like I said, I want to avoid closed-source, large applications by big names like Apple and Adobe, since they tend to need periodic upgrades. In the case of Adobe, such upgrades will ultimately force you into a subscription. In the case of Apple, you may be forced to spend money on a more modern computer as your current one is removed from the list of supported machines. Recently, with the presentation of macOS Ventura we saw an extreme example of this, with my 2018 Mac Mini barely escaping the axe.

What one needs to understand is that pictures and videos are long-lasting digital assets. The iPhoto library where you archived your photos 10 years ago will need to be migrated into the most recent Apple Photos version. For that, you may need to buy an old MacBook, just to pry out the pictures from the library file. And with it, there are no assurances that the metadata (dates, locations, etc.) will be correctly migrated! We should only rely on EXIF metadata and a simple structure of folders and files for our long-term preservation.

After a search on GitHub for media organization, I selected phockup for this job. It is written in Python and uses exiftool, an open-source¹ and platform-independent tool, to extract EXIF data from the pictures and videos. It is also quite easy to install on a Mac using Homebrew.

Setting up your Mac

1. Install Homebrew.
2. For macOS Ventura, you may need to install Python from source.

   1
   brew install --build-from-source [email protected]
   

3. Install phockup’s Python tqdm dependency:

   1
    pip3 install tqdm
   

4. Install fswatch :

   1
    brew install fswatch
   

5. Install phockup :

   1
   2
    brew tap ivandokov/homebrew-contrib
    brew install phockup
   

Try running phockup. You should see this:

Starting up phockup

The synchronisation script

The basic requirements are:

• When new files are detected, call phockup to organise them to a target folder.
• Only one synchronisation process can execute at any given point in time, to avoid synchronisation conflicts.
  • We will be using a simple file .pid file in the temporary file directory /tmp for this.
• If a synchronisation process crashes, the next run should know that the original process crashed and try again.

@see https://superuser.com/questions/28384/what-should-i-do-about-com-apple-quarantine

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
#!/usr/bin/env bash

# NOT WORKING YET!

# Set up two folders to monitor. 
# 
# - Downloads folder will be monitored for files sent via AirDrop only
# - DropFolder will be monitored for any files 
AIRDROP_FOLDER="$HOME/Downloads"
SOURCE_FOLDER="$HOME/Pictures/DropFolder"
TARGET_FOLDER="/Volumes/Backups/Pictures"
PIDFILE="/tmp/photo_organiser.pid"

# Print lines to Mac's system log Console app
function print_lines_to_system_log() {
  while IFS= read -r line;
  do
	  echo "$line"
	  syslog -s -k Facility com.apple.console \
             Level Notice \
             Sender "Photo Sync Script by @silvae86"\
             Message "$line"
  done 
}

echo "Monitoring files AirDropped into $AIRDROP_FOLDER and copied into $SOURCE_FOLDER...." | print_lines_to_system_log
echo "Media files will be organised into $TARGET_FOLDER." | print_lines_to_system_log

# Run phockup instance on a folder. Will print logs to system log
# Organise all files in the DropFolder. 
# Allows manual copying into it, but is useful for organisation of 
# old hard drives, for example.
# move_to_source_folder_if_airdropped will move airdropped 
function organise_folder() {
  local FOLDER_TO_ORGANISE=$1
  local PIDFILE=$2
  echo "Starting organisation of folder $FOLDER_TO_ORGANISE."
  # phockup "$FOLDER_TO_ORGANISE" "$OUTPUT_FOLDER" --verbose | print_lines_to_system_log - &
  yes > /dev/null &
  echo $! > "$PIDFILE"
  echo "yes running"
  echo "$(cat $PIDFILE)"
  sleep 5
  cat "$PIDFILE" | xargs -I{} kill -9 {}
} 

# Moves detected files, only if they were AirDropped but not downloads
function move_to_source_folder_if_airdropped() {
  local FILE="$1"
  
  	# Filter files based on their macos extended attributes, including only those sent via AirDrop. 
   # This will ignore files downloaded from the internet
   
   # Need to detect a file with a WhereFroms metadata attribute, but the attribute does not include a web address (http://....)
   xattr -p com.apple.metadata:kMDItemWhereFroms "$FILE" > /dev/null 2>&1
   AIRDROPPED_OR_DOWNLOADED=$?
   if (( $AIRDROPPED_OR_DOWNLOADED == 1 )); then
	   echo "File $FILE does not have WhereFrom metadata at all. Skipping."
	   return 0
   fi
   
   xattr -px com.apple.metadata:kMDItemWhereFroms "$FILE" | xxd -r -p | plutil -convert xml1 - -o - | sed -n -E 's/^.*<string>(.*)<\/string>$/\1/p' | awk '/http/{print}' | grep . > /dev/null
   DOWNLOADED=$?
      
   if (( $DOWNLOADED == 1 )); then
      echo "Airdropped file $FILE detected. Moving to $SOURCE_FOLDER to be organised." | print_lines_to_system_log -
      # mv "$FILE" $SOURCE_FOLDER
   fi
}

# Monitor folder for new files using OS-fired events instead of polling (e.g. macOS's `fswatch`)
function monitor_folder() {
  local FOLDER_TO_MONITOR=$1
  echo "Monitoring $FOLDER_TO_MONITOR...to move any Airdropped files into $SOURCE_FOLDER" | print_lines_to_system_log -
  
  fswatch --recursive "$FOLDER_TO_MONITOR" | move_to_source_folder_if_airdropped - &
}

# Try to move any folders from the source folder
function scan_and_move() {
	local FOLDER_TO_SCAN="$1"
	# find all files recursively, run 'file0 command on each. 
	# Run 'awk' to match for mimetypes with video or image
	# Then, pipe only those filenames to a while loop
	# to move each file into the $SOURCE_FOLDER
	
	find "$FOLDER_TO_SCAN" -type f -print0 -exec file --mime-type {} \+ | awk -F: '{if ($2 ~/image|video\//) print $1}' | while read file;
	do
		move_to_source_folder_if_airdropped "$file"
	done
}

# Remove PID file if phockup process died
function try_to_unlock() {
  local PIDFILE="$1"
  if [ -f "$PIDFILE" ]; 
  then
    PID=$(cat "$PIDFILE")
	ps -p "$PID" | awk "/$PID/{print}" | grep . > /dev/null 2&>1
	PROCESS_RUNNING=$?
	echo "Process running? $PROCESS_RUNNING"
	
    if (( $PROCESS_RUNNING != 0 )); 
	then
		echo "Process $PID died. Unlocking so a new session can start..." | print_lines_to_system_log -
        rm -f "$PIDFILE" && echo "Deleted pidfile at $PIDFILE. New session can start " | print_lines_to_system_log -
		return 0
    else
		echo "Organiser process still running (PID $PID)..."
      	return 1
    fi
  else
	  echo "no pid file found"
  fi
  return 0
}

# Perform initial scan
echo "Photo organiser performing initial scan...." | print_lines_to_system_log -
scan_and_move "$AIRDROP_FOLDER"

# After initial scan, monitor folder for new files
monitor_folder "$AIRDROP_FOLDER"

# Call organisation script if an organisation script is not running already
while :
do
  try_to_unlock "$PIDFILE" && organise_folder "$SOURCE_FOLDER" "$PIDFILE" &
  sleep 1
done &

wait 

Making the synchronisation script run in the background and on startup

We will use the init system of macOS for starting our synchronisation script in the background whenever the machine starts.

Building a .plist file to configure the daemon

Adding the LaunchDaemon to initctl

Testing it out

Kięczkowska” Link

how?” Link

Metadata” Link

[^sed-ignore-non-matching-lines] “Have sed ignore non-matching lines ( StackOverflow)” Link

[^loop-filenames-spaces] “BASH Shell: For Loop File Names With Spaces” Link

[^list-graphic-images] “BASH Shell: For Loop File Names With Spaces” Link

[^awk-grep-dot] “AWK Return Codes - Grep dot trick” Link

month and day.” Link

The OWC Thunderbay 4 Mini is a pretty unique value proposition due to its dimensions and flexibility. It sports a tiny footprint, fitting perfectly in a minimalistic desktop, and achieves it by using 2.5’’ hard drives instead of 3.5’’.

Despite being quite expensive for an enclosure at around €350, its Thunderbolt 4 interface makes it very fast, and the 4 SATA bays make it very flexible: you can use SSDs for maximum speed, allowing you to edit video straight off the Thunderbay, or HDDs to cope with some big storage needs. When combined with some hard drive shucking it can become an attractive proposal from a cost/benefit perspective.

Design and dimensions

Thunderbay 4 Mini Design.

Several objects around for size comparison.

The Thunderbay is an all-aluminium enclosure design. It is appealing and matches the aesthetic that people come to associate with Apple computers. It is minimal and looks great on the desk, with a grill on the front that is somewhat reminiscent of the Classic Mac Pro (“Cheese Grater”). It makes for a timeless industrial look and lets in a sufficient amount of air for cooling the hard drives. The front has a lock, which you need to place in the open position to pull out the front door that covers the hard drives. The key does not come off in the unlocked position, as the front cover would just fall to the front if they did.

The fan noise (and several solutions)

The enclosure has a 60mm fan at the back that cools the hard drives inside by pulling air through the front air intake, through the whole enclosure and exhausting it out the back. My unit came with a Noctua fan, which runs at 100% all the time, making it quite noisy at idle. After cracking open the unit, I saw that there are only two wires (one black and one red) that connect the fan to the PCB. I guess there is no RPM reporting, so the PCB just runs it at 12 volts all the time.

It seems not all units come with a Noctua NF-A6x25 FLX fan. Some carry a Sunon one, which seems to be even more noisy. It is disappointing that OWC does not provide any way to regulate the fan speed, as Mac desktops like my Mac Mini tend to be quite silent at idle.

There are several options to solve this:

1. Unplugging the fan altogether since I am not using SSDs but instead external rotational hard drives, which by design do not need any active cooling.
2. Solder a 2 kΩ potentiometer to the red wire that runs from the fan to the PCB. I could then drill a tiny hole in the back of the case for the shaft of the potentiometer and fine tune the speed of the fan as I see fit.
3. Replace the fan with a Noiseblocker XL1, which runs at a much lower speed. I would lose overall cooling capacity but would require fewer modifications than the potentiometer option.

Cracking open the rear hatch to unplug the fan at the back

Unplugging the fan as easy as removing the two small Torx screws in the back and (gently) pulling the connector from its socket. The enclosure only gets slightly warm to the touch, even after performing several copies of hundreds of thousands of small files—the 2018 Mac Mini i7 by its side is always toasty by comparison! The Thunderbay is also silent when the RAID volume is unmounted (disks spin down), and the seek/write noise level when operating the unit is very reasonable.

Shucking 2.5’’ hard drives for the RAID array

Hard drive shucking is the process of buying an external (USB) hard drive and cracking open the enclosure to extract the hard drive inside and use it as an internal hard drive. The main advantage is a much lesser cost per drive (internals are approximately 37% more expensive for the same capacity in my particular case) and you get a free enclosure + USB 3 cable. This can be useful for when you finally decide to retire the drives from active use by putting them back in their case.

A very important aspect to take into consideration is that 2.5’’ drives used in external hard drives are 15mm thick, so they will not fit into a laptop, for example. However, they are perfect for the Thunderbay Mini, since the enclosure supports hard drives up to 15mm thick. I initially found this detail in the Thunderbay 4 Mini with Thunderbolt 2 manual, and it remains the same for the newer TB3 model.

Drives that can be shucked are rarer and rarer though; you need to do some research before buying your drives.

Selecting a shuckable hard drive

Manufacturers are wise to the fact that people shuck their drives. Practically all external drives in the market are now built with a direct connection between the USB port and the hard drive controller, without a SATA port in between. Western Digital and Toshiba now adopt such a practice, with a notable exception being Seagate. Many Seagate drives still sport a USB-to-SATA adapter board that you can detach, converting the drive back to its ‘internal’ form.

I chose the Seagate Basic Portable 5TB external hard drive (STJL5000400), which inside holds a standard Seagate Barracuda hard drive at 5400rpm. No fancy finishing, simple plastic case (easy to crack open) and, most importantly, uses a SATA-to-USB3 adapter board. I bought 5 of them to have one in reserve, while 4 are in use inside the Thunderbay. While RAID does not require identical disks, I like to have them. It is better to buy them now than be looking for one some years down the road when one drive fails.

Shucking the Seagate 5TB Basic Portable

Seagate Basic Portable 5TB external drives

Stick a spudger in the edge to crack open the enclosure

Enclosure open

Hard drive revealed

Hard drive model and part number

Peeling off the sticker

Removing SATA to USB3 adapter board

Removing anti-vibration feet

Assembling the hard drives inside the Thunderbay enclosure

Open up the Thunderbay to expose the 4 hard drive trays...

Attach a sliding tray to the drive using the 4 screws already on the drive

Slide the drive into the slot

Repeat for all 4 drives

Connect and start up the drive. All status LEDs should light up

Choosing between RAID 10 (1+0) or 01 (0+1)

Given that this is a 4-disk configuration, we can go for a number of RAID configurations. The most simple of the nested configurations that do not rely on splitting parity information among different disks are RAID 10 and RAID 01.

From these there is really only one choice (RAID 10) since the probability of failure of a RAID 10 array with 4 disks is 33%, while for a RAID 01 that is 66%. Moreover, the probability of failure for a RAID 10 array will tend to 0 as more disks are added, while it will always be at least 50% in a RAID 01 array ¹.

In a 4-drive RAID 10 array you first create a RAID 1 array with 2 of the 4 hard drives. You then do the same for the other pair. Then, you build a RAID 0 array from the two RAID 0 arrays. This is also known as RAID 1+0, mnemonic that tells you the order of the building of the array (first RAID 1 arrays and then you RAID 0 those RAID 1 arrays) ².

Representation of a `RAID 10` array

Why I chose Apple Disk Utility instead of SoftRAID

The enclosure includes a license of OWC’s software RAID solution, SoftRAID XT. After some investigation, and even though I paid for the software as a part of the bundle, I decided against using it even before I bought the unit. Instead, I had already opted for Apple’s built-in software RAID instead. Here are the pros and cons of SoftRAID:

✅ Pros

1. Windows + Mac compatibility. You can use your drives on both operating systems, provided SoftRAID is installed.
2. Nice UI that is easy to use for everyone
3. Drive status monitoring + alerts

❌ Cons

1. Should you opt for using SoftRAID, you are stuck with it forever. All data in the Thunderbay will only be readable and writeable with the software installed. It works as a macOS Kernel Extension, meaning that it is flaky as they come, and it working or not will depend on Apple’s whim. Should Apple decide to implement changes to the OS that break SoftRAID’s Kernel Extension, you are left stuck on an older operating system while you wait for OWC to update the software, if at all possible. This has happened when Apple released macOS Big Sur. For Catalina it was even worse, with similar issues and a nasty fix provided at the time requiring you to disable System Integrity Protection. At least OWC seem to be committed to good customer service, providing not only a list of known issues with SoftRAID, but also continuing to support the software. Long story short, I worry about using such software to keep valuable data, even if OWC are not to blame.
2. Although touted as a selling point of SoftRAID, support for Windows seems flaky and buggy. For me in particular, this is a non-issue since I only use macOS, but be aware if you plan to use Windows with your Thunderbay.
3. The SoftRAID license is single-machine only, meaning that if you own several Macs and need to alternate your Thunderbay between more than one machine, you have to deactivate SoftRAID on the first and activate it on the second machine. Every. Single. Time. Not for me, sorry.
4. SoftRAID comes in several flavours (Lite, XT, Pro). The XT version (bundled with the Thunderbay 4 Mini) only allows you to use the software with OWC enclosures. In the event that the enclosure itself fails, I have to buy another one from OWC to get my data out of the hard drives, or spend $ 249 on the Pro version of SoftRAID. By using Apple’s software RAID, it is possible to take out the drives from the dead enclosure, connect them all to a another Mac using USB docks, or to a Hackintosh (which has enough SATA ports, unlike Apple’s modern computers), and get my data out.
5. I am not interested in RAID 5 (offered by SoftRAID but not by Apple’s Disk Utility). When compared to RAID 5, RAID 10 offers you more usable storage, but its working principle seem less robust to me. Requiring a minimum of N hard drives, RAID 5 reserves, 1/N of the storage of each drive for parity information. More importantly, it allows a single drive to fail. The problems come when a single drive does fail, since the volume rebuild process can take a long time and will put high strain on all the remaining drives. As a rule of thumb, if you have a drive fail, it is reasonable to assume that at least one of the others could be close to failure. Stressing them all for tens of hours to rebuild the volume through SoftRAID does not seem healthy, since if any of the other drives fail, you lose all data in all the drives.
6. You need SoftRAID to rebuild the volumes in case a drive fails. Using Apple’s software RAID, you can use the command line or the Disk Utility. I like command-line stuff because although ugly, you know it’s going to work and come OOB with every single Mac for years to come.

Setting up RAID 10 array using Apple Disk Utility

I will be using Apple Disk Utility. Follow along:

Set up first RAID 1 array with a pair of disks

Select two of the drives:

The operation will seem to fail, but if you wait some time you will see the array first appear as grayed out in the list of disks and then go online.

We repeat the process for the remaining two drives:

Again we get the same error message, ignore it. After the lights blink in alternating fashion in the Thunderbay, you will see the RAID array come online:

Notice the letters VaultRAID1Slice1 in the name of the go from grayed out to black (array online):

Set up the RAID 0 with the two nested RAID 1 arrays

This time, select RAID 0 instead of RAID 1:

Choose the final name of your complete array. Here I will be using Vault, which will represent the complete set of 4 drives in our RAID 10 form.

Final Result - 10TB in RAID 10!

Performance

With this configuration (RAID 10) I got these numbers in BlackMagic Disk Speed Test. Not anywhere near SSD levels, but more than enough for regular media backup. However, it is good to know that I can upgrade to SSDs in the future if I ever need it and prices allow, as the Thunderbolt connection will always allow me to take advantage of their speed.

This script uses fswatch to monitor for changes in the local folder and rsync to perform the actual synchronization to the folder on the remote machine.

It is highly advisable to configure your SSH keys and hosts to have a completely seamless synchronization experience (i.e. no constant nagging for passwords).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#!/bin/bash

# Configure these variables to your reality
REMOTE_HOST=macpro
REMOTE_FOLDER=/tmp/silvae86.github.io
LOCAL_FOLDER=$(pwd)

# initial sync step
rsync -af --progress . "$REMOTE_HOST:$REMOTE_FOLDER"

# initial check may take some time, later changes may be instant
fswatch \
--one-per-batch \ 
--recursive \
--latency 1 \
--verbose \
"$LOCAL_FOLDER" | xargs -I{} rsync -a --progress "$LOCAL_FOLDER" "$REMOTE_HOST:$REMOTE_FOLDER"

Explained:

• --one-per-batch → bubble/combine events so we do not ask for a sync every time a single file is changed
• --recursive → scan directories and subdirectories
• --latency 1 → wait 1 second before triggering sync
• "$LOCAL_FOLDER" →source folder

Next, a piped command calls rsync to sync the local dir to the remote one:

• rsync -a --progress "$LOCAL_FOLDER" "$REMOTE_HOST:$REMOTE_FOLDER" → call rsync to synchronize $LOCAL_FOLDER to $REMOTE_FOLDER on $REMOTE_HOST, via SSH.

After installing pyenv via homebrew:

1
brew install pyenv

I tried to set my Python versions:

1
2
$ pyenv global 3.7.0 2.7.15
pyenv: version `3.7.0' not installed

Running the installation command produced the following error:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ pyenv install 3.7.0       
python-build: use [email protected] from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.0.tar.xz...
-> https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz
Installing Python-3.7.0...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 12.3.1 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/d1/k0pk6j150dd7xw7qh55n_0bc0000gn/T/python-build.XXXXXXXXX
Results logged to /var/folders/d1/k0pk6j150dd7xw7qh55n_0bc0000gn/T/python-build.XXXXXXXXXX.log

Last 10 log lines:
checking for --with-cxx-main=<compiler>... no
checking for clang++... no
configure:

  By default, distutils will build C++ extension modules with "clang++".
  If this is not intended, then set CXX on the configure command line.
  
checking for the platform triplet based on compiler characteristics... darwin
configure: error: internal configure error for the platform triplet, please file a bug report
make: *** No targets specified and no makefile found.  Stop.

To fix it, I first selected the version I wanted to install with pyenv install --list. Then, I installed GCC from homebrew, and tried installing again, this time specifying the path to the C++ compiler.

I the following script, replace gcc-12 with a newer version if necessary. Type cd /opt/homebrew/bin/gcc<Press Tab key> in the Terminal and see what version you have installed.

1
2
3
brew install gcc
CC=/opt/homebrew/bin/gcc-12 pyenv install 3.8.12 
CC=/opt/homebrew/bin/gcc-12 pyenv install 2.7.18

Note for macOS Ventura Beta!

When running xcode-select --install you may face this error:

1
2
3
Can't install the software because it is not
currently available from the Software
Update server

To fix it, download command line tools from Apple’s More Downloads page on Apple Developer before running brew install gcc.

I then set them as the default python versions globally:

1
pyenv global 3.9.12 2.7.18

Done! Bash Beautify works in Visual Studio Code now.