Downloading iCloud photos

I take a lot of photos and try to be very careful with their management and storage. Everything gets included in Lightroom, and backed up in a variety of ways.

The photos on my and my wife’s iPhone live in the cloud, outside the rest of my archive. This is something that’s always irked me.

Downloading directly from icloud.com is not the easiest task. Although possible to download using a browser, it does so as multiple files which have to be manually selected. This is fine for a handful of images, but not great when you want to download several years.

I came across a great Python tool that automates this:

https://github.com/ndbroadbent/icloud_photos_downloader

It’s installed simply with:

pip install icloudpd

Supply it with appropriate login arguments (it supports 2FA accounts) and it will dutifully download every file from your iCloud account, including Live Photos (which are downloaded as video files).

You can configure it for specific date ranges, making it suitable for automation, and it can use your keychain for storing login details.

It does tend to freeze occasionally, although I suspect this is more an unofficial-iCloud-API kind of fault. Fortunately it figures out what has already downloaded, and you can restart from your last position.

Downloading the last three years worth of photos and videos took several days. This was a combination of size (there’s about 200GB of stuff), and having to restart the process every few hours.

‘Fixing’ a Commodore 64

A while back I made a nostalgic impulse purchase on eBay and ended up with a Commodore 64 – a real breadbin style one, with no emulation in sight.

This was my first ever computer and I have fond memories of playing countless games, waiting patiently for things to load from tape, and borrowing library books to learn how to code. It literally was my gateway into the computer world.

The machine I bought was a fairly simple bundle – just the computer itself, boxed, with power supply. It arrived a few days later and I was amazed at how small it is.

I spent time cleaning 35 years of accumulated gunk. Everything seemed in pretty good shape, with labels in place, no significant plastic yellowing, and just a few scuffs underneath.

My first problem was that the Commodore outputs an analogue video display, and all my displays are digital. Without a display I can’t see if the machine works.

Speaking of which, I didn’t even know if it worked. The computer was advertised as being untested – a common term for broken – but I’d assumed it just hadn’t been used in 30 years.

Should I ever get it to work and displayed on a screen I then faced the problem of having no media, and no joystick. A problem for another time.

C64 Video Display

As expected, there are devices that convert the analogue signal to digital, and I went with an s-video/composite to HDMI convertor.

The C64 outputs a composite signal, but through a special DIN connector. I found a wiring diagram online, ordered some parts, and built a shoddy adaptor.

Power on – first attempt

Now that I had a compatible display it was time to try the machine out.

First I just plugged it in and switched it on. Nothing happened.

A visit to YouTube showed me that pretty much everything inside the machine can fail. None of it sounded fun. The power supply was the first place to look at as these are notoriously poor quality (I remember warming my feet on mine, back in the day).

Using a multimeter I attempted to read the power coming out of the supply. At first I think I got some values, but on checking again I got nothing so I either fried the supply in reading it, or it was on the way out and died.

I didn’t want to buy an old power supply. I read that the supply is unusual in that it outputs both AC and DC power., This got me to wondering if I could get separate AC and DC power supplies and feed them into the computer’s power socket.

Building a power supply

I found a 5V/1.5A DC supply from a USB hub I had lying around the house. This matches the DC requirements.

I ordered a 9V/1A AC adaptor from the internet – this was much harder to source. Mine ended up having a euro connector, so I also needed an adaptor on top. It’s quite a frankenpower supply.

Both of the power supplies had a 2.5cm barrel plug, so I ordered two barrel sockets as I didn’t want to butcher the cables.

The C64 power is supplied through a 9 pin DIN plug which are still easy to get hold of.

Looking at socket diagrams it seemed simple enough – two pins for DC and two pins for AC. That seemed very manageable.

Here’s the result:

Not bad, although another soldering horror job inside. The cheap component parts didn’t help, melting somewhat.

Power on – second attempt

Now I’ll state that I have no knowledge or experience of electronics, and the double-supply I built may be extremely dangerous, may instantly catch fire, or may somehow create a small black hole.

But it worked!

I plugged everything in – the C64 now sporting 3 power supplies (the video convertor needs one too) – and switched it on. The red LED came on, the monitor went black, and then the pale blue C64 screen appeared with a full complement of memory. Hurrah!

It’s a bit fuzzy and that may be due to the video cable I made. I’ll try another soon.

I don’t know if the SID chip works, and I don’t know if it can play a game. There are devices you can buy that take an SD card full of files and emulate a disk drive to the C64. That’s something for another day.

My joy at making something that worked was a little tempered when I reassembled the C64 and managed to crack the brittle plastic with the screws. Also, I have a spare screw I can’t find a home for. Annoying.

Speeding up a Synology rebuild

After purchasing a Synology NAS I inserted a bunch of drives and left ‘expanding’ overnight.

When I came back in the morning it had gone from 2% to 5% complete – about 95% less than what I was hoping.

I discovered that you can SSH into the box and check the status of the rebuild with:

cat /proc/mdstat

This showed that I had another 14 days left. Hmm, I don’t think so.

After some poking about I discovered it defaults to a strangely small 256 byte cache size.

There were a lot of posts on the internet about changing this value in /sys/block/md3/md/stripe_cache_size, but this failed with a permission problem. Maybe this applied to older versions.

Instead you can edit:

/etc.defaults/synoinfo.conf

And add the line:

md_resync_cache_size="16384"

It took a few minutes for this to be picked up and then the rebuild time decreased to 14 hours. Much more acceptable.

I tried some larger cache size values but they didn’t make any difference.

Clearing Chrome’s DNS cache

If you ever need to point a domain name at an IP address, for sandboxing during development, or for whatever reasons, then Chrome can be a real nuisance. It seems to cache the domain name and refuses to use your new IP address.

Recent versions supposedly monitor /etc/hosts, and if you view the DNS cache this seems to be true.

chrome://net-internals/#dns

However it can still be stubborn about actually using the new IP. After some digging I found that clearing the socket pool usually does the trick.

chrome://net-internals/#sockets

Certainly beats waiting until Chrome finally gives up the old values.

MacOS terminal cursor navigation with a mouse

File this under ‘I can’t believe I didn’t know this sooner’.

Ever had to edit a long command in a terminal window? Certainly you can set things up so you can jump back and forward using the keyboard. But you can also option+click to move the cursor instantly – really handy for some commands.

Also, in iTerm at least, you can apple+click a URL to open it in your browser.

Using pv to view the progress of dd

If you use dd for any kind of copying you’ll know how it’s not exactly forthcoming with information and it’s hard to tell if it’s doing anything.

Some versions of dd have a status=progress parameter that will output progress information. Sadly this doesn’t exist on the native MacOS version.

An alternative way of showing progress information is to use pv (or pipe viewer).

First install it with brew:

brew install pv

Then wedge pv between reading from the dd source and writing to the dd output.

For example, you might write a Raspberry Pi image to an SD card with:

dd if=retropie.img of=/dev/rdisk4

Using pv you break this into the input and output parts and put pv in the middle:

dd if=retropie.img | pv | dd of=/dev/rdisk4

Sudo needs to be added to both calls to dd:

sudo dd if=retropie.img | pv | sudo dd of=/dev/rdisk4

Telling pv how much data you are transferring allows it to provide better information, and you can do this with the -s flag. In this instance the amount of data is the size of the input image, so:

sudo dd if=retropie.img | pv -s `stat -f%s retropie.img` | sudo dd of=/dev/rdisk4

And what is the result of this? You get to see the progress! Useful if you’re transferring a large amount of data.

Yarn

There’s another new toy from Facebook, this time called Yarn. Here’s how it went for me:

Installed yarn with brew install yarn
Typed yarn in my project directory
Waited a while
Committed changes to git

All very easy. I’ve not yet experienced any of the benefits of yarn, but that’s probably just a question of time.

John's Blog

Category: Beep