There are so many different things that can go wrong when connecting drives, my buddy Dean had a huge problem when he tried to plug a USB Drive into his M1 Mac, it caused kernel panics. The problem I’m diagnosing now is similar, I’m having startup and speed problems. The machine configuration is very odd:

1. MacBook Pro 2017 running Ventura
2. Razer Thunderbolt 4 Docking station
3. Benq 32-inch monitor with DisplayPort to USB C adapter directly to the laptop
4. RocketStor 5212 which is a dual drive enclosure with Thunderbolt 2 through a Thunderbolt 3 Adapter with RAID 1
5. Either drive the ThunderBay 8 enclosure with Thunderbolt 4 directly to the laptop
6. USB 2.0 connection to DroboPro DAS connected through the Razer
7. USB C Flash Drive connected directly to the laptop

So let the debugging begin, note that I have a sea of background processes:

1. Backblaze is running in the background backing all the DAS systems up (that was the main point of this, to use the Backblaze system to do a big backup from the DAS).
2. Synology Drive Client downloads the content of my Synology NAS to the ThunderBay 8
3. There are about 18 TB of content on the DroboPro that needs to be uploaded and copied to the new ThunderBay 8
4. There is 15TB of content on the RAID

Booting up with lots of peripherals problems

The first symptom is that I don’t reliably get my DisplayPort monitor to work. When I reboot the machine with all this stuff connected, the laptop just hangs. The touchpad is unresponsive and the cursor is frozen. But if I just boot it with everything disconnected it seems to work fine. I suspect that in the boot sequence, all of these hard drives are getting in the way.

But I have found that if I wait long enough like 30 minutes, then bootup works fine

Disk Access is very slow until a reboot

OK, in the theoretic world, there is plenty of bandwidth for everything. Even USB 2.0 is a 480Mbps link so should be about to sustain 40MBps downloads. A Synology NAS with six drives in RAID10 lover 2x1Gbps Ethernet should easily be able to sustain 100MBps data rates.

But the reality is quite different, some of the big issues are that USB 2.0 was not designed to be a sustained bandwidth system and there are many more things that bandwidth that determine how well a system performs. The biggest thing is some IO Operations and also the seek time of these physical drives especially when they are fragmented. The second issue is that USB 2.0 is not optimized for these large file transfers. As an example

1. USB 2.0 Direct-attached-Storage Drobo to Thunderbolt 3 attached OWC ThunderBay 8. This should sustain incredible data rates. The individual tests show very good data rates at 100MBps, but copying 990GB takes over 8 days and only does about 123GB per day which is way way way slower. To do the rough math 5GB/hour is just 1.7MBps or about 17Mbps or about 4% of what it should be. The brought math of this is that USB 2.0 has 1ms frames with 8 micro frames but most controllers can’t handle 13 packets per Microframe so the real-world limit is 33 or so MBps but even then it’s pretty far off. Note also that USB 2.0 is half-duplex, so if there is traffic going the other way, bandwidth falls. It’s interesting that after two days, the transfer actually failed. Personally, I suspect the old firmware in this obsolete DAS 🙂 and since it is twice as fast on reboot and the download eventually failed, you have to wonder.
2. Synology NAS with 6 drive RAID10 to Thunderbolt 3 OWC ThunderBay 8. Surprisingly, this is working at about 20Mbps, but because the drives are nearly full, they are highly fragmented, so I try to stream from another machine at 3MBps, the whole thing freezes.

Post Reboot is better but not great: 3MBps from Drobo to Synology, 1.4-5MBps to ThunderBay

However, after a reboot, everything seems much faster. I tried to turn on the console to see what was going on, but a copy from the USB 2 DroboPro to the Ethernet-connected Synology is running 24GB in 2 hours or 3.3MBps still not super fast but twice the speed before.

The speed on the local machine from USB 2 DroboPro to a ThunderBolt 3 ThunderBay was not much faster, so it seems like it has something to do with the USB 2.0 connection or the DroboPro itself and how it is handling it. But in a test where I also had a full copy running from a NAS and another copy running, I got 4.5GB in 51 minutes or 1.4MBps. So to test the load theory, I then turned off:

1. The Synology Drive Client. It was in the middle of copying from the NAS, this doubled the speed of the USB transfer to 3MBps (still way under the other figure). I also tried to run the BlackMagic Speed Test of the ThunderBay under load and it was 182MBps write/175MBps read so definitely much slower than without a loaded system.
2. When I paused the BackBlaze backup, the speed shot up to 591.4 MBps write and 348MBps read, so it seems that there is some loading going on. BackBlaze should only be uploading, so I’m confused as to why write times went up so much. One notable thing is the copy times do not change much even as the speed of copy changes, for instance, I see the same 27 minutes to copy 4.6GB, but in five minutes it has already copied 1GB which is now 4MBps. And just observing the progress bar, it seems a bit uneven sometimes it moves fast and the other 10MB/2 seconds observed or 5MBps, is still way under the 30MBps I would have expected. And I can’t actually run the Disk Speed Test on the USB 2.0 drive right now, the test hangs.
3. Now when I also pause a copy that is also being done in the background from Transmission

Other reasons for slow USB 2.0: Spotlight

Well, in looking through other causes, one that jumped out was Spotlight, I do see that there is a lot of indexing going on, so just go to System Settings > Siri & Spotlight > Spotlight Privacy and select those drives and see if that helps. Certainly indexing is not going to be good for these slow USB 2.0 drives. Note that at least on Ventura, the user interface is really buggy, I keep adding drives to Privacy and the file user interface hangs and forgets drives.

The mystery of Searchpartyd running

I’ve noticed that I/O gets very slow after a while on the system and there is a searchpartyd daemon running that is gobbling up CPU, so I wonder if there is something long-running. Searchpartyd is strangely something that runs for Find My, so I’m not sure why it is spinning away but it has happened twice now.

In looking at the logs while Searchpartyd is taking up 100% of a core, the message is “TODO: Need to figure how to retrieve lastObservedIndex on macOS” and this repeats over and over. This is a pretty strange error message and I can’t find anything like this on the Internet. It definitely happens and can happen for hours and then go away on reboot or after a while.

Finder is unbearably slow (as is Safari)

This is another problem that I have not quite solved, but sometimes Finder just becomes super slow and unresponsive, it can take minutes for it to show and we talked about the copy problems above. The same with Safari, but a reboot seems to solve this. Sigh.

The symptom is that when you try to do a copy, it will hang at zero bytes for hours. It is as if there is a lock held. Then when copy does begin it can take days and then finally fail. The error logs are not really helpful here I don’t see anything coming out of the console.

The same with Safari it is just stuck. It sure feels like something is grabbing lots of interrupts but hard to tell with so many devices attached.

Looking at Logs and Activity Monitor

Well, you can turn on the console logs and try to figure out what is happening, but it’s a real blizzard of errors and nothing obvious.

Synology Drive Client Saturates Synology NAS

This was a little bit of a surprise, but when I’m running the Synology Drive Client to download files for copying, it seems to saturate the Synology itself, so it can’t stream say an 4K movie at 3MBps. This is sort of a surprise as the BlackMagic tests indicate there is over 200MBps of transfer capability. It could be because the Drive Client is slow, but I’m not sure. But, others are complaining the file transfer is slow and also it takes a lot of CPU and maybe what is happening is my old Synology 1812+ just can’t keep up, although CPU utilization looks like 50% and the 4GB of RAM shows 2GB are free so hard to say.