Like oil and water, or how I finally started using Syncthing

https://xpil.eu/akps 📋

I had several attempts at using Syncthing in my life, and it always ended with me giving up because it was too complicated. Or I couldn't be bothered. Or it didn't work... but mainly because it was too complicated.

However, recently I told myself: it's now or never. Let's try to see this thing through to the end.

And it worked!

So what is this Syncthing thing? It is a piece of software whose sole task is to make sure that files in a folder on computer A and files in a folder on computer B are synchronized with each other, regardless of what kind of files they are, how many there are, or how the computers "see" each other over the network.

Syncthing works on local networks, public networks, behind firewalls, behind NAT, basically everywhere. Generally, if both computers are connected to the Internet, they will "see" each other in Syncthing. The synchronization is secure (data in transit is well encrypted), quite fast, and logical; the only difficulty is - starting. After that, it's downhill.

Usually, we do something for a reason ^{[citation needed]}. It's hard for me to imagine Paddy from the local vegetable store, who between one celery and another suddenly decides, out of the blue: "oh, I'm feeling like installing Syncthing!" In my case, the main motivation were the limitations of "big" cloud clients, such as Dropbox or OneDrive, when it comes to sharing entire folders, combined with the Obsidian application on mobile devices, which needs access to the entire folder, not just individual files. And yes, big cloud services can synchronize entire folders, but only within the client application, so you can see the folder in the Dropbox client, but not in the Files app (the Android equivalent of Windows Explorer). A folder synchronized through OneDrive is also not available externally on the mobile OneDrive client. This is quite frustrating, but whatcanyoudo.

Or... just do something about it!

We start with downloading the Syncthing installer and installing it. If everything goes well, after opening http://127.0.0.1:8384 with Firefox (because feck-off other browsers), we should see the user interface, where we can manage synchronization.

So, first, we add a folder - or folders - to ST (I will use the "ST" acronym instead of "Syncthing" because it's shorter and I'm lazy) for further synchronization. Then we add other devices (on which we should also first install ST - like a phone, tablet, or another computer). The latter will require scanning a QR code, or possibly sending a long (56 characters plus hyphens) identifier by some other means.

Here - note - a big advantage of ST compared to most of the competition: adding a new device must be done twice: first from computer X we add computer Y, then from computer Y we add computer X. This prevents the unpleasant situation where someone somehow intercepts the identification codes of one of the devices and tries to add their own device to steal our data.

Finally, in the folder sharing settings, we tick the devices with which we want to share that folder. At this point, on the other device, a notification pops up asking for confirmation, we then indicate the location of the folder in the target file system, and a moment later the synchronization process begins automatically. In my case, I synchronize a folder with Obsidian vaults (currently I have four of them: three of my own and one sample from some fancy-shmancy online creator), around 200MB in total - and everything works perfectly well. My home PC is at the center of this mess, and connected to it through ST are my smartphone and tablet.

Delays? Of course, there have to be some. I did an experiment and measured the time between making a change in a file on the smartphone and the appearance of this change on the tablet (think of it, the file had to be synchronized with the PC first, and then with the tablet) - the delay over the local network (via WiFi) was around 8-10 seconds. Not bad. I haven't measured delays over WAN yet.

The only - fortunately tiny - fly in the ointment is the way ST deals with synchronization conflicts, which are unavoidable in such situations. They occur when I change the same note in Obsidian on two different devices in a very short space of time. Although you have to really try to generate such a conflict, so in everyday use, they will appear very rarely (some ST users mention that with hundreds of thousands of synchronized files, they average between 1 and 10 conflicts a year, but of course, it all depends on your usage patterns).

But... let's assume that such a conflict arises. For example, computer A changes the first line in the main.py file and before ST can propagate this change, computer B changes the same file but in the second line. Bam - we have a conflict. What does ST do at this moment to resolve it?

Well - nothing. Abso-fricking-lutely nothing. It simply saves both versions of the file under two different names: main.sync-conflict-<date>-<time>-<computer-name>.py - and that's it. It won't even notify us that such a conflict has occurred (although the conflict information will appear in the UI of the app, but who would look there, right?). From now on, both versions of the file will synchronize between computers until death do us part, or until we resolve the problem manually. In addition to this, of course, both computers will retain files with the original name (so in our example main.py) and with different contents.

A synchronization conflict also occurs when one computer deletes a file and the other changes it at the same time. In this case, ST assumes that the second computer "is right" and the deleted file will soon reappear everywhere, with new content. This is a conservative approach - I personally like it, although I can imagine scenarios where it may be bothersome.

Finally, a quick thought on battery, CPU, and network usage.

If we add a large folder to the system (gigabytes, terabytes...), it will turn all involved devices into radiators before it synchronizes, because cryptographic hashes of each file aren't going to calculate themselves (which also means - more power, ergo more battery eaten up). Everything also needs to be sent over the network, so if you have limited data plan, you might find yourself up the creek without a paddle.

But once the initial synchronization is completed, it turns out, ST uses exceptionally few resources to keep things in sync. Wherever possible, it compresses data to save bandwidth. It also consumes processor cycles as if on a hunger strike. In short - just peachy.

And all this for free! Unless you have a surplus in your account, then you can support ST with a donation.

I'm writing this just a few days after setting up ST on my toys, so I am only at the very begging of my Syncthing journey. Maybe some bugs will come out in the longer use (and then there will be something for me to write about), but for now, I am delighted. If the system proves itself, I will consider cancelling the bill for at least one cloud provider, to save a few pennies.

https://xpil.eu/akps 📋