Checksums on files / reliable up/download

[tobiasKaminsky]

To have reliable up- and downloads, generating a checksum on server is needed.

Upload

• client generates checksum
• client uploads file with checksum
• server checks checksum and stores it after successfull upload

Download

• client retrieves file and checksum
• client checks checksum

On backward compatibility:

Upload:

• if no checksum is provided, e.g. legacy apps / 3rd party apps -> generate checksum on server

Download:

• legacy clients can simply ignore checksum

Additional for Android Files we can check if files are already uploaded (despite on relying on file names).

(this was shortly discussed in NC15 planning meeting)

---

Additionally, the ability to request checksum of file via endpoint from #25949:

• client detects that file with same name already exists
• we do not know if it is the same file
• client requests only for this file via new(?) endpoint a checksum
• client compares it
• only if this differs, it presents conflict resolution dialog

[nextcloud-bot]

GitMate.io thinks possibly related issues are #56 (File Drop: Create confirmation / show file checksum), #3475 (question about file count), #6129 (When using the REST API to find out checksums of files, a 404 is returned.), #7629 (Create Folders when moving files), and #1371 (No file-uploading possible at all).

[jospoortvliet]

I believe the clients checksums files, not sure if it syncs the checksums and/or if the server verifies them. But that's more meant to check if files are transferred correctly.

[tobiasKaminsky]

This should be done on server side, as even if all clients would compute & upload a checksum, the webUI would be missing and then we cannot use this at all.

[tobiasKaminsky]

@icewind1991 we talked about this on NC15 meeting.
Do you think this is feasible to get it still in NC15?

[MorrisJobke]

This should be done on server side, as even if all clients would compute & upload a checksum, the webUI would be missing and then we cannot use this at all.

The big problem with this is that it is nearly impossible to generate the checksums for external storages. We can't do anything there and also updating them in a timely manner is nearly out of scope. For a pure "all data goes through NC" this would be no problem, but the external storages are a real problem.

[MorrisJobke]

cc @rullzer @icewind1991 because we discussed quite a lot about this.

[tribut]

FYI the fact that NC does not have this apparently causes owncloud-client to warn when connecting to a nextcloud instance, which is a problem in Debian because there is no nextcloud-client (yet):

https://alioth-lists.debian.net/pipermail/pkg-owncloud-maintainers/2019-January/003498.html