[LINUX] I tried using "Syncthing" to synchronize files on multiple PCs

I wanted to share files on multiple PCs without git push, so I tried Syncthing introduced by the company!

What is Syncthing?

Official repository https://github.com/syncthing/syncthing/

Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. (From the official repository)

The literal translation is "Share files between two or more PCs."

It's similar to DropBox and OneDrive, but unlike these, the files aren't uploaded to the server.

Compatible OS, etc.

As of March 2020, the following are supported.

Installation

I have a Mac and Ubuntu at hand, so I'll try them with these two. First, I will describe the setting procedure with GUI, but since it seems that it can be done only with CLI, I will describe it later.

Ubuntu

reference https://apt.syncthing.net/

curl -s https://syncthing.net/release-key.txt | sudo apt-key add -
echo "deb https://apt.syncthing.net/ syncthing stable" | sudo tee /etc/apt/sources.list.d/syncthing.list
sudo apt update && sudo apt install syncthing
#Execute → Browser starts up
syncthing

Mac

brew install syncthing
#Execute → Browser starts up
syncthing

CentOS / Other Linux

I have not confirmed it because there is no environment, but it seems that binary installation can also be done like the following site. https://computingforgeeks.com/install-and-configure-syncthing-on-centos-linux/

Android

I haven't confirmed this because there is no environment, but there is an app in the store. https://play.google.com/store/apps/details?id=com.nutomic.syncthingandroid&hl=ja


When you execute the syncthing command, the browser will be launched and the following screen will be displayed. image.png

Register / execute as a service

If you execute the syncthing command according to the above procedure, synchronization will not be possible unless you leave the console running, so register it as a service.

Ubuntu

sudo systemctl enable [email protected]${USER}.service
sudo systemctl start [email protected]${USER}.service

Mac

brew services start syncthing

On Mac, Syncthing starts up, but I can't see the web GUI ... If it is already set, synchronization will be performed properly.

If you want to launch the Web GUI on Mac, use the separate syncthing -browser-only command.

Sync settings

There are the following steps to synchronize.

  1. Launch syncthing on all the devices you want to sync
  2. Register the "destination device"
  3. Add a folder to sync
  4. Shared folder settings
  5. Wait for the synchronization to complete

Since 1. is completed, we will explain from 2.

Register "destination device"

There is an "Add connected device" button at the bottom right of the screen displayed on the browser, so press it. Then, the following pop-up will be displayed. image.png

If the devices to be synchronized are on the same network with syncsync running up The device ID of the other device will be displayed automatically, so you can just select it.

If they do not exist on the same network, you need to find out and enter the device ID.

How to find the device ID

You can check the device ID by selecting Menu → Show ID at the top right of the screen. image.png

Set the above settings for 2 units.

Add a folder to sync

This time I will try to synchronize the ~ / github folder of Ubuntu.

Sync source (Ubuntu) settings

There is an "Add Folder" button on the left side of the browser of the synchronization source, so press it. image.png

Then, the following pop-up will be displayed. Specify the folder you want to share in the folder name. I happened to be a folder in my home directory, so it's okay as it is, but in other cases, set the folder path as well.

image.png

Make a note of this folder ID as it is important.

There is a ignore filename tab on the pop-up, so you can exclude files you don't want to share here. I did it directly in the config file.

https://docs.syncthing.net/users/ignoring.html

Sync destination (Mac) settings

Make the same settings at the synchronization destination. The point is to make the folder ID the same. image.png

Shared folder settings

Click the edit button of the connected device. image.png

Press Share in the displayed popup, check the folder you want to share in Folder to share with this device, and save. image.png

Wait for the synchronization to complete

If the synchronization setting is successful, you just have to wait for the synchronization to start as shown below! image.png

Synchronize in a CLI-only environment

With the default settings, you can only open the web GUI screen from localhost. Therefore, if you change the settings so that the Web GUI can be displayed from the outside, you can easily set the settings even in a CLI-only environment.

However, in a public environment such as VPS or EC2, anyone can change the setting, so be sure to set a password.

Edit configuration file

The location of the configuration file differs depending on the environment.

--For Linux

~/.config/syncthing/config.xml

--For Mac

~/Library/ApplicationSupport/Syncthing/config.xml

Since there is a GUI setting, modify it as follows.

before


<gui enabled="true" tls="false" debugging="false">
    <address>127.0.0.1:8384</address>
    <apikey>aNw6DrLYWukHvLK22Ed7M5hoR5tixPJG</apikey>
    <theme>dark</theme>
</gui>

after


<gui enabled="true" tls="false" debugging="false">
    <!--0 here.0.0.Fixed to 0-->
    <address>0.0.0.0:8384</address>
    <apikey>aNw6DrLYWukHvLK22Ed7M5hoR5tixPJG</apikey>
    <theme>dark</theme>
</gui>

If syncthing is running, restart syncthing.

Now you can see it from the outside, but as mentioned above, it's in a public state, so you'll get a warning. image.png

Click the Settings button at the bottom right and set the GUI user and password. image.png

I want to set synchronization completely with CLI only

It was also possible to set it completely on the command line just by setting config.xml. However, it was a bit annoying because I need to edit the XML directly & it needs to be described on all multiple terminals.

Although it is light, I will describe the setting method.

Installation

The installation procedure is exactly the same as above.

Device registration

I think there is a device tag, so copy this content and set it directly below. Set the id of the device you want to synchronize with the id of the device.

<device id="XXXXXXXXX" name="xxx-ubuntu" compression="metadata" introducer="false" skipIntroductionRemovals="false" introducedBy="">
     <address>dynamic</address>
     <paused>false</paused>
     <autoAcceptFolders>false</autoAcceptFolders>
     <maxSendKbps>0</maxSendKbps>
     <maxRecvKbps>0</maxRecvKbps>
     <maxRequestKiB>0</maxRequestKiB>
</device>

The device ID is the log displayed when the syncthing command is executed, the WEB screen in the case of GUI, and the service (daemon) in the case of execution. Check from the following command.

syncthing -device-id

Folder registration

There is a tag of folder id =" default " at the top of config.xml, so copy all of this and set it directly below.

<folder id="default" label="Default Folder" path="/Users/xxxx/Sync" type="sendreceive" rescanIntervalS="3600" fsWatcherEnabled="true" fsWatcherDelayS="10" ignorePerms="false" autoNormalize="true">
    <filesystemType>basic</filesystemType>
    <device id="XXXXXXXXX" introducedBy=""></device>
    <minDiskFree unit="%">1</minDiskFree>
    <versioning></versioning>
    <copiers>0</copiers>
    <pullerMaxPendingKiB>0</pullerMaxPendingKiB>
    <hashers>0</hashers>
    <order>random</order>
    <ignoreDelete>false</ignoreDelete>
    <scanProgressIntervalS>0</scanProgressIntervalS>
    <pullerPauseS>0</pullerPauseS>
    <maxConflicts>-1</maxConflicts>
    <disableSparseFiles>false</disableSparseFiles>
    <disableTempIndexes>false</disableTempIndexes>
    <paused>false</paused>
    <weakHashThresholdPct>25</weakHashThresholdPct>
    <markerName>.stfolder</markerName>
    <copyOwnershipFromParent>false</copyOwnershipFromParent>
    <modTimeWindowS>0</modTimeWindowS>
</folder>

Rewrite the following part of the copied settings.

After changing the settings, restart syncthing.

Setting Example

example


<folder id="github-folder-id" label="github" path="/Users/xxxxx/github" type="sendreceive" rescanIntervalS="3600" fsWatcherEnabled="true" fsWatcherDelayS="10" ignorePerms="false" autoNormalize="true">
    <filesystemType>basic</filesystemType>
    <device id="Device ID 1" introducedBy=""></device>
    <device id="Device ID 2" introducedBy=""></device>
    <device id="Device ID 3" introducedBy=""></device>
    <minDiskFree unit="%">1</minDiskFree>
    <versioning></versioning>
    <copiers>0</copiers>
    <pullerMaxPendingKiB>0</pullerMaxPendingKiB>
    <hashers>0</hashers>
    <order>random</order>
    <ignoreDelete>false</ignoreDelete>
    <scanProgressIntervalS>0</scanProgressIntervalS>
    <pullerPauseS>0</pullerPauseS>
    <maxConflicts>-1</maxConflicts>
    <disableSparseFiles>false</disableSparseFiles>
    <disableTempIndexes>false</disableTempIndexes>
    <paused>false</paused>
    <weakHashThresholdPct>25</weakHashThresholdPct>
    <markerName>.stfolder</markerName>
    <copyOwnershipFromParent>false</copyOwnershipFromParent>
    <modTimeWindowS>0</modTimeWindowS>
</folder>

that's all. Let's use it for a while and check its usability.

Recommended Posts

I tried using "Syncthing" to synchronize files on multiple PCs
I tried to synthesize WAV files using Pydub.
I tried to digitize the stamp stamped on paper using OpenCV
I tried to visualize BigQuery data using Jupyter Lab on GCP
I tried using Azure Speech to Text.
I tried to classify text using TensorFlow
I tried to predict Covid-19 using Darts
Using Keras's stateful RNN, I tried automatic composition based on wav files.
I tried using jpholidayp over proxy to execute cron only on weekdays
I tried to control multiple servo motors MG996R using the servo driver PCA9685.
I tried using Remote API on GAE / J
I want to visualize csv files using Vega-Lite!
[Pythonocc] I tried using CAD on jupyter notebook
I tried to make a ○ ✕ game using TensorFlow
I tried to explain multiple regression analysis as easily as possible using concrete examples.
I tried to make a translation BOT that works on Discord using googletrans
I tried using argparse
I tried using anytree
I tried using aiomysql
I tried using Summpy
I tried using coturn
I tried using Pipenv
I tried using matplotlib
I tried using "Anvil".
I tried using Hubot
I tried using ESPCN
I tried using openpyxl
I tried using Ipython
I tried to debug.
I tried using PyCaret
I tried using cron
I tried using ngrok
I tried using face_recognition
I tried to paste
I tried using Jupyter
I tried using PyCaret
I tried using Heapq
I tried using doctest
I tried using folium
I tried using jinja2
I tried using folium
I tried using time-window
I tried to implement Minesweeper on terminal with python
I tried to get an AMI using AWS Lambda
I tried to approximate the sin function using chainer
I tried to become an Ann Man using OpenCV
I tried using PySpark from Jupyter 4.x on EMR
I tried to identify the language using CNN + Melspectogram
I tried to access Google Spread Sheets using Python
I tried to notify the honeypot report on LINE
I tried to install scrapy on Anaconda and couldn't
I tried to complement the knowledge graph using OpenKE
I tried to draw a configuration diagram using Diagrams
I tried to compress the image using machine learning
I tried to search videos using Youtube Data API (beginner)
[Images available] I tried using neofetch on various operating systems!
[I tried using Pythonista 3] Introduction
I tried two ways to combine multiple commits in Git
I tried using easydict (memo).
I tried to simulate ad optimization using the bandit algorithm.
I tried to organize SVM.