Skip to content

Conversation

@ChristophWurst
Copy link
Member

@ChristophWurst ChristophWurst commented Jan 24, 2025

  • Resolves: Nextcloud upgrade taking hours on large installations. This is because the oc_properties table has indices for userid+propertypath and propertypath. The DELETE query filters on propertyname and propertyvalue. There are no other queries during normal operation that would benefit from an index, so it's not a valuable alternative.

Summary

Moves the repair step from a synchronour repair step to an expensive ones. These are executed via occ.

TODO

  • Change registration
  • Test repair without expensive steps ✔️ step doesn't run anymore
  • Test repair with expensive steps ✔️ step runs

Checklist

@ChristophWurst
Copy link
Member Author

@tcitworld correct me if my assumption is wrong but we have had this step since #30368 so at late upgrades the repair step will not remove a lot of rows.

@ChristophWurst
Copy link
Member Author

/backport to stable31

@ChristophWurst
Copy link
Member Author

/backport to stable30

),
\OC::$server->get(ValidatePhoneNumber::class),
\OC::$server->get(DeleteSchedulingObjects::class),
\OC::$server->get(RemoveObjectProperties::class),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a huge fan of using dav app classes in server but there is no other way right now. #50390 tracks the possibility to declare expensiveness directly in info.xml.

@juliusknorr
Copy link
Member

Make sense, though the query still does not seem to use an index. Can we be sure that the delete does not cause locks on the table for too long or may cause other problems?

@ChristophWurst
Copy link
Member Author

Correct, the query does not use an index.

This is because the oc_properties table has indices for userid+propertypath and propertypath. The DELETE query filters on propertyname and propertyvalue. There are no other queries during normal operation that would benefit from an index, so it's not a valuable alternative.

Adding an index would probably also take a couple of hours, and be counterproductive for normal operations outside of the repair step. Let me double check for any queries that could benefit from the query nevertheless.

@ChristophWurst
Copy link
Member Author

The most fitting index would be one on propertyname, propertypath and userid:

\OCA\DAV\DAV\CustomPropertiesBackend::getUserProperties -> either userid+propertypath OR propertyname+propertypath+userid
\OCA\DAV\DAV\CustomPropertiesBackend::createDeleteQuery -> propertyname+propertypath+userid
\OCA\DAV\DAV\CustomPropertiesBackend::createUpdateQuery -> propertyname+propertypath+uderid
\OCA\DAV\BackgroundJob\UserStatusAutomation::getAvailabilityFromPropertiesTable -> propertyname+propertypath+userid
\OCA\DAV\Db\PropertyMapper::findPropertyByPathAndName -> propertyname+propertypath+userid
\OCA\DAV\Migration\RemoveObjectProperties::run -> propertyname

Other queries that only filter by propertypath and userid would still use properties_path_index

@tcitworld
Copy link
Member

correct me if my assumption is wrong but we have had this step since #30368 so at late upgrades the repair step will not remove a lot of rows.

That's right.

Btw, a repair step that we should skip on upgrades is #13430, which should be a one time job and not run on every upgrade.

@susnux susnux added this to the Nextcloud 32 milestone Mar 2, 2025
This was referenced Aug 22, 2025
This was referenced Aug 28, 2025
@ChristophWurst ChristophWurst force-pushed the fix/dav/remove-object-properties-expensive branch from 7100725 to ffd2b3b Compare September 2, 2025 08:49
@ChristophWurst ChristophWurst requested a review from a team as a code owner September 2, 2025 08:49
@ChristophWurst ChristophWurst requested review from icewind1991, salmart-dev and yemkareems and removed request for a team September 2, 2025 08:49
@ChristophWurst ChristophWurst removed the 3. to review Waiting for reviews label Sep 2, 2025
@ChristophWurst ChristophWurst added the 4. to release Ready to be released and/or waiting for tests to finish label Sep 2, 2025
@kesselb kesselb merged commit 478b012 into master Sep 2, 2025
217 of 223 checks passed
@kesselb kesselb deleted the fix/dav/remove-object-properties-expensive branch September 2, 2025 19:36
@github-project-automation github-project-automation bot moved this from 🏗️ In progress to ☑️ Done in 💌 📅 👥 Groupware team Sep 2, 2025
@skjnldsv skjnldsv modified the milestones: Nextcloud 32, Nextcloud 33 Sep 28, 2025
@blizzz blizzz mentioned this pull request Oct 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4. to release Ready to be released and/or waiting for tests to finish feature: install and update

Projects

Status: ☑️ Done

Development

Successfully merging this pull request may close these issues.

8 participants