files are vanishing

[vanbernaert]

There are multiple reports files are vanishing since update NC to 32.0.1.

• Mostly Excel files
• Always files under __groupfolders

Active thread is here: https://help.nextcloud.com/t/files-vanish-not-a-single-trace/235588

[DJeremyy]

I’m experiencing the same problem since the update to Nextcloud server version 32.0.0 with groupfolders v20.1.2, which is still present in Nextcloud 32.0.1 with groupfolders v20.1.3.

I haven’t been able to find any trace of deletion or activity in the system logs. Nothing is recorded in the database’s oc_activity table when I search using the object_id of one of the affected files.

[come-nc]

Can someone with the issue run an occ info:file on a vanished file? (the idea is to check if it disappears from FS or also from DB. Ideally this should be done using the fileid but I’m not sure whether it’s possible to know the fileid once the file is gone, and from what I understood we have no reproduction steps yet?)

[DJeremyy]

Hello Come-nc,
I found the object_id in the oc_activity table of two files that were vanished by searching using the path where the file was stored before it disappeared. In both cases, I get nothing with occ info:file

occ info:file 7511080
file 7511080 not found

occ info:file 4145866
file 4145866 not found

[come-nc]

Are you able to double-check directly in database if these ids appear in oc_filecache fileid column?

Also, does that ONLY happens in groupfolders? (I’ve seen people say «mostly», that’s not helpful 🙃 )
And is it ONLY office files?

Is this with a standard local storage or is there s3 or anything special setup on that side?

[DJeremyy]

No unfortunately, select * from oc_filecache where fileid = 7511080; and select * from oc_filecache where fileid = 4145866; returns 0 rows.

I know the clues are slim but it is hard to make assertions without proof and without logs.

For my part, this always happened on files stored in groupfolders. The files that have vanished have the common point of being XLS. I have never observed this on XLSX or other formats.

The installation is directly on the file system, it is not S3.

[sgiunchi]

Hi @come-nc,

occ info:file 577943
file 577943 not found

MariaDB [nextcloud]> select * from oc_filecache where fileid = 577943;`
Empty set (0.000 sec)

I confirm both that missing files are in groupfolders and XLS files.
Local filesystem.

[SystemKeeper]

Can you enable audit logging as described at https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/logging_configuration.html#admin-audit-log-optional and see if there's anything logged for those files? Please double check it's correctly enabled.

Since XLS seems some common point, all of you using Collabora?

[sgiunchi]

I already did activate admin audit, you can see some details in this post.
I confirm I had Collabora enabled.
Unfortunately, today I transformed all group folders in normal shares, so I will not be able to replicate the problem.
Or I hope so!

[ebb-dwandel]

For the statistics: We also have about 10 lost files. All XLS, all group folders. So far, I have not received any reports about other files.
(We also use Groupfolders and Collabora)

[DJeremyy]

Similarly, the XLS files were in a groupfolders. Oddly enough, not all of these files with this format vanished. I restored the files from a backup for which I received user warnings. The restored files disappeared again. There's no information in the logs, even in debug mode.

To prevent this from happening again, I converted the affected files to XLSX.

Me too, Collabora is enabled.

[vanbernaert]

indeed all .xls files and never an .xlsx file.
That's a serieus helpfull hint to avoid temporarily this disaster-bug

Does that mean the problem doesn't occur with Colabora disabled? We have it enabled, but users hardly use it. So we can disable this immediatly.

[vanbernaert]

For those it might help, to bypass instantly this serious bug:

1/ the files that are disappearing are only Excel files with the old extension XLS
2/ we installed on our nextcloud server “ssconvert“ to batch convert all the xls-files to xlsx

we did this straight in the __groupfolders/
check the file permission afterwards! mostly you need to "chown -R www-data: ./" in each groupfolder
then rescan on new files: sudo -u www-data php /var/www/html/occ groupfolders:scan XX (XX = folder ID)

be aware of the SSCONVERTT specs!! it’s converting very well regular excel, but not complex spreadsheets with VBS scripts and so on.