I have a Nextcloud instance that various things automatically upload photos to. These automatic folders sync to a directory on my desktop. I wanted to pull things out of that directory without deleting them, and only once. (My wife might move them out of the directory on her computer, and I might arrange them into targets on my end.)
In other words, I wanted to copy a file from a source to a destination, but remember what had been copied before so it only ever copies once.
rsync doesn’t quite do this. But it turns out that tar’s listed-incremental feature can do exactly that. Ordinarily, it would delete files that were deleted on the source. But if we make the tar file with the incremental option, but extract it without, it doesn’t try to delete anything at extract time.
Here’s my synconce script:
#!/bin/bash
set -e
if [ -z "$3" ]; then
echo "Syntax: $0 snapshotfile sourcedir destdir"
exit 5
fi
SNAPFILE="$(realpath "$1")"
SRCDIR="$2"
DESTDIR="$(realpath "$3")"
cd "$SRCDIR"
if [ -e "$SNAPFILE" ]; then
cp "$SNAPFILE" "${SNAPFILE}.new"
fi
tar "--listed-incremental=${SNAPFILE}.new" -cpf - . | \
tar -xf - -C "$DESTDIR"
mv "${SNAPFILE}.new" "${SNAPFILE}"
Just have the snapshotfile be outside both the sourcedir and destdir and you’re good to go!
Can you discuss how your devices are set up to automatically upload photos?
I run the Nextcloud server, and either the Nextcloud or Owncloud clients on every mobile device except one, which uses FolderSync Pro.
Can you explain why rsync doesn’t achieve this? It would only copy if things once, and only if it exists? I’m probably missing something. I have my own nextcloud instance and sync photos just like you do, and I also copy images away to a media server manually once in a while. If there is some automation possible, I’d like to learn it :)
So let’s say that there are photos that person A took. Both person A and person B maintain some photo collection. Person B doesn’t want to delete from the shared folder the photos that person A will later process also, but also doesn’t want to process the same photos over and over. So with this approach, things from the shared folder are copied — once — to an “incoming” folder. When person B files them away, they are deleted from this incoming folder but not the shared folder, and are never copied back again. Person A can then come by and move them out of the shared area later.
Does that help?