Suppose I have an IoT device, controlled by firm A, that buys a bunch of services from firm B. I will be responsible for all the charges, even if I had no idea that firm A was going to make these purchases on my behalf.

Over the weekend, I discovered that I had such a device and that the money at risk is not trivial. A background process called nsurlsessiond that Apple uses to manage iCloud services downloaded 400 GB (yes, GIGA and yes BYTES) of data to my iMac over the course of 31 hours. Apple makes it very difficult to see or understand what this process is doing. So in effect, it turned my iMac into an IoT appliance that burned up bandwidth.

I learned that there was a problem only because I spent some time working remotely on my Macbook, using my iPad’s 4G connection to the internet. It had also been turned into one of these IoT appliances, so I soon received a warning from my ISP saying that I had already exceeded my entire monthly allocation of 20 GB of downloaded data via wireless. I stopped connecting over 4G as soon as I could, but still ended up with $75 in excess data charges.

When I got home, I checked my iMac. I couldn’t find a way to get the information I needed to figure out what was going on from the built-in Activity Monitor. Fortunately, I also run a program called Little Snitch that monitors network traffic. Its report on nsurlsessiond showed this:

To quantify the liabilities here, if this download had been over my 4G connection, my ISP’s overage charge of $15 per GB, multiplied by 400 GB of data, would imply $6000 in data charges. My connection at home does not have a data cap, but not everyone has this protection. The tech at Apple said that if I did face big charges, sometimes it works to call the ISP and beg for mercy.

Apple and my ISP have a long-term relationship. Together, they have a joint interest in getting me to use a lot more data that the ISP can charge for. My ISP can reward Apple by encouraging sales of Apple mobile devices.

Presumably, the two of them would like to avoid an attention getting spike of usage like the one I faced. It was surely caused by a bug. But it is telling that Apple has configured things so that by default I have no visibility into background data usage and without any throttle or cap that could limits the potential damage that when things spiral out of control.

If you have to pay for data, be very careful about using iCloud. And get ready for lots of surprises from the IoT.

For Anyone Facing a Similar Problem with nsurlsessiond

Summary

1. If you need a temporary fix that stops the massive data downloads, one option is to install Little Snitch and add a rule that you can turn on and off that denies any outgoing connections to nsurlsessiond. Although this only stops the outgoing requests, but this seem to be required to sustain the incoming downloads.

2. To get a permanent fix, I had to stop using iCloud entirely. I deleted all data from iDrive and I deleted all data from the one application (Notability) that had permission to store data in iCloud.

3. If, like me, you have a problem that is a true bug, not a built in feature of Photos–huge data transfers if you have a large photo library and agree to use Photos with iCloud–you will have to spend a lot of time and effort with Apple Tech Support persuading them that Photos is not the source of your problem.

Details

1. nsurlsessiond There are many reports of users who ended up with huge downloads that they did not anticipate. Many of the reports from more technical users identify nsurlsessiond as the process doing the download. Many of these are apparently the result of defaults and a misleading interface that gets people to agree to store large quantities of photo data in iCloud without realizing the consequences of this decision. I was careful not to let any of Apple’s applications use iCloud. To do this, I had to change the defaults for iCloud, which turn the Apple apps on.

2. iCloud Drive As part of iCloud, I have been using iDrive, which lets me have part of my file system in the cloud. I can see these files in Finder under iCloud Drive. This behaves like Dropbox or Google Drive, but with one crucial difference. I have no control over the sync process with the local store of data. If iDrive is turned on, a background process (probably nsurlsessiond but there may be others) syncs the files on iDrive with a cache on my machine. I had about 30 GB of data on iDrive.

3. Letting an app use iCloud I let just one application (Notability) store its data in iCloud. This app runs on both IOS devices and OSX. Under OSX, this too involves a sync between the data stored in the cloud and a local cache. The difference is that as you might expect from the IOS world, files are not visible via Finder. When I checked iCloud under System Preferences, I could see that Notability was storing about 3 GB on iCloud. (This is far more than Notability seems designed to handle. This large quantity of data arose partly because on an IOS device, Notability duplicates every note if you turn iCloud access off and then back on. Apparently, turning it off turns the local cache into a permanent local data store. Then if you turn iCloud back on, Notability downloads another copy of every note from iCloud and puts them in a new cache. As I tried to debug problems with Notability, I ended up with 4 copies of every note.)

4. The source of my problem I saw indications of bugs in both Notability and iDrive so I am not sure what caused the massive downloads.

The cache files are under the user Library, which you can access via Finder by holding down the option key when you click on Go. To see what is going on in the Library, it helps to select Show View Options for Finder (from the View menu) and check Calculate All Sizes so that Finder can display the size of all folders in a directory.

In Library, the Mobile Documents folder seems to be the cache for iDrive. It had the same amount of data, 30 GB, that I originally uploaded to iDrive.

I suspect that Notability was caching information in Library/Caches/CloudKit. At different times, this folder ended up with something between 5 and 20 GB, which of course is far too much relative to the 3 GB that Notability was supposed to be managing. Taking the two together, there is no reason why syncing a total of 30+3 GB of data should require 400 GB of downloads to a single machine. Something must have kept trying, over and over, to accomplish something that wasn’t working.

The large cache in CloudKit suggested that Notability was the problem. I tried deleting the CloudKit folder and even the entire Caches folder. When these were in Trash, they were still being used by various system processes. After a reboot, I sometimes saw a large cache remerge under CloudKit too quickly for this to be the result of downloads, so I suspect that the same processes that had been using the files while they were in Trash recovered them to CloudKit after a reboot. The only way I found to prevent this was to i) turn of my internet connection, ii) delete the CloudKit folder, iii) reboot, iv) empty the trash, v) turn my internet connection back on. But then the CloudKit cache still seemed to keep rebuilding itself via a download. This seemed to continue even after deleting all of my data from inside Notability. (In retrospect, I should have tried deleting this cache from both my iMac and my MacBook to rule out the possibility that the cache on one machine was trying to rebuild the cache on the other.)

My problems resolved only after I also deleted all of my data from iDrive, doing this on each machine and for each user with access to iDrive. Along the way, I also tried several more times to delete the CloudKit cache folder. (It now has very little data.)

My hunch is that Notability was the culprit, but it is possible that iDrive was also using the CloudKit cache in some way and that it was the true source of the problem; or that there is some interaction between Notability and iDrive; or that there was a delay or an interaction between the caches on the two machines so that getting rid of the Notability data was did solve the problem but this did not show up until after I deleted all the data from iDrive; or …