[Bug]: `occ db:convert-type` fails to migrate `app_api` tables and has issues with `groupfolders` and `spreed`

[kyteinsky]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

Some apps fail to migrate from the old to the new db.

1. groupfolders and spreed (talk) throw errors about the column being inexistent. The columns were present in the old db and not present in the new db.
2. app_api is silently ignored and the tables are just listed as not to be migrated at the end.
3. oc_context_chat_content_queue is not migrated for no apparent reason, other tables from the app have been migrated.

Steps to reproduce

1. Setup nextcloud with sqlite as dbtype
2. Setup an empty postgresql db with just the database named "nextcloud" created, with no tables
3. Run php -d memory_limit=2048M occ db:convert-type --password postgres --all-apps pgsql postgres database-pgsql --port 5432 nextcloud
4. Remove/Rename the apps that have errors.
   I have the old db data still.

Expected behavior

All database data is migrated successfully.

Nextcloud Server version

master

Operating system

None

PHP engine version

None

Web server

None

Database engine version

(to) postgresql version: Debian 17.0-1.pgdg120+1

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "debug": true,
        "profiler": true,
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/apps-extra",
                "url": "\/apps-extra",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/apps-shared",
                "url": "\/apps-shared",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/apps-writable",
                "url": "\/apps-writable",
                "writable": true
            }
        ],
        "allow_local_remote_servers": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "1025",
        "skeletondirectory": "\/skeleton",
        "setup_create_db_user": false,
        "loglevel": 2,
        "log_query": false,
        "query_log_file": "\/shared\/log\/querylog-nextcloud.log",
        "query_log_file_requestid": "yes",
        "diagnostics.logging": false,
        "diagnostics.logging.threshold": 0,
        "log.condition": {
            "apps": [
                "diagnostics",
                "admin_audit"
            ]
        },
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "nextcloud.local"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "32.0.0.0",
        "overwrite.cli.url": "http:\/\/nextcloud.local",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "git",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "theme": "",
        "maintenance": false
    }
}

List of activated Apps

Enabled:
  - app_api: 32.0.0-dev.2
  - assistant: 2.3.0
  - bruteforcesettings: 5.0.0-dev.0
  - circles: 32.0.0-dev.0
  - cloud_federation_api: 1.15.0
  - comments: 1.22.0
  - contacts: 7.1.0-dev.0
  - contactsinteraction: 1.13.0
  - context_chat: 4.0.3
  - dashboard: 7.12.0
  - dav: 1.34.0
  - federatedfilesharing: 1.22.0
  - federation: 1.22.0
  - files: 2.4.0
  - files_external: 1.24.0
  - files_pdfviewer: 5.0.0-dev.0
  - files_sharing: 1.24.0
  - files_trashbin: 1.22.0
  - files_versions: 1.25.0
  - groupfolders: 20.0.0-dev.0
  - logreader: 5.0.0-dev.0
  - lookup_server_connector: 1.20.0
  - notifications: 5.0.0-dev.0
  - oauth2: 1.20.0
  - photos: 5.0.0-dev.0
  - profile: 1.1.0
  - provisioning_api: 1.22.0
  - serverinfo: 4.0.0-dev.0
  - settings: 1.15.0
  - sharebymail: 1.22.0
  - systemtags: 1.22.0
  - text: 6.0.0-dev.0
  - theming: 2.7.0
  - twofactor_backupcodes: 1.21.0
  - updatenotification: 1.22.0
  - user_status: 1.12.0
  - viewer: 5.0.0-dev.0
  - weather_status: 1.12.0
  - webhook_listeners: 1.3.0
  - workflowengine: 2.14.0

Nextcloud Signing status

Nextcloud Logs

{"reqId":"V8Uhu4ZpPdkQVjv53hAi","level":3,"time":"2025-02-05T09:19:12+00:00","remoteAddr":"","user":false,"app":"no app in context","method":"","url":"--","message":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","userAgent":"--","version":"32.0.0.0","exception":{"Exception":"OCP\\AppFramework\\QueryException","Message":"Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","Code":0,"Trace":[{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":124,"function":"resolve","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php","line":439,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php","line":411,"function":"queryNoFallback","class":"OC\\AppFramework\\DependencyInjection\\DIContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":42,"function":"query","class":"OC\\AppFramework\\DependencyInjection\\DIContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/apps/admin_audit/lib/AppInfo/Application.php","line":127,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/Bootstrap/Coordinator.php","line":157,"function":"boot","class":"OCA\\AdminAudit\\AppInfo\\Application","type":"->","args":[{"__class__":"OC\\AppFramework\\Bootstrap\\BootContext"}]},{"file":"/var/www/html/lib/private/App/AppManager.php","line":479,"function":"bootApp","class":"OC\\AppFramework\\Bootstrap\\Coordinator","type":"->","args":["admin_audit"]},{"file":"/var/www/html/lib/private/legacy/OC_App.php","line":93,"function":"loadApp","class":"OC\\App\\AppManager","type":"->","args":["admin_audit"]},{"file":"/var/www/html/core/Command/Db/ConvertType.php","line":213,"function":"loadApp","class":"OC_App","type":"::","args":["admin_audit"]},{"file":"/var/www/html/core/Command/Db/ConvertType.php","line":169,"function":"createSchema","class":"OC\\Core\\Command\\Db\\ConvertType","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Command/Command.php","line":298,"function":"execute","class":"OC\\Core\\Command\\Db\\ConvertType","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":1040,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":301,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->","args":[{"__class__":"OC\\Core\\Command\\Db\\ConvertType"},"*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":171,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Console/Application.php","line":187,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/console.php","line":87,"function":"run","class":"OC\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/occ","line":11,"args":["/var/www/html/console.php"],"function":"require_once"}],"File":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","Line":108,"message":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","exception":{},"CustomMessage":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated"}}
{"reqId":"V8Uhu4ZpPdkQVjv53hAi","level":3,"time":"2025-02-05T09:19:12+00:00","remoteAddr":"","user":false,"app":"PHP","method":"","url":"--","message":"Undefined array key \"SERVER_PROTOCOL\" at /var/www/html/lib/private/AppFramework/Http/Request.php#654","userAgent":"--","version":"32.0.0.0","data":{"app":"PHP"}}
{"reqId":"V8Uhu4ZpPdkQVjv53hAi","level":3,"time":"2025-02-05T09:19:12+00:00","remoteAddr":"","user":false,"app":"PHP","method":"","url":"--","message":"Undefined array key \"HTTP_USER_AGENT\" at /var/www/html/apps/profiler/lib/DataCollector/HttpDataCollector.php#31","userAgent":"--","version":"32.0.0.0","data":{"app":"PHP"}}
{"reqId":"ZRgClME0hHcX2Sx0oEhq","level":3,"time":"2025-02-05T09:25:56+00:00","remoteAddr":"","user":false,"app":"no app in context","method":"","url":"--","message":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","userAgent":"--","version":"32.0.0.0","exception":{"Exception":"OCP\\AppFramework\\QueryException","Message":"Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","Code":0,"Trace":[{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":124,"function":"resolve","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php","line":439,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php","line":411,"function":"queryNoFallback","class":"OC\\AppFramework\\DependencyInjection\\DIContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":42,"function":"query","class":"OC\\AppFramework\\DependencyInjection\\DIContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/apps/admin_audit/lib/AppInfo/Application.php","line":127,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/Bootstrap/Coordinator.php","line":157,"function":"boot","class":"OCA\\AdminAudit\\AppInfo\\Application","type":"->","args":[{"__class__":"OC\\AppFramework\\Bootstrap\\BootContext"}]},{"file":"/var/www/html/lib/private/App/AppManager.php","line":479,"function":"bootApp","class":"OC\\AppFramework\\Bootstrap\\Coordinator","type":"->","args":["admin_audit"]},{"file":"/var/www/html/lib/private/legacy/OC_App.php","line":93,"function":"loadApp","class":"OC\\App\\AppManager","type":"->","args":["admin_audit"]},{"file":"/var/www/html/core/Command/Db/ConvertType.php","line":213,"function":"loadApp","class":"OC_App","type":"::","args":["admin_audit"]},{"file":"/var/www/html/core/Command/Db/ConvertType.php","line":169,"function":"createSchema","class":"OC\\Core\\Command\\Db\\ConvertType","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Command/Command.php","line":298,"function":"execute","class":"OC\\Core\\Command\\Db\\ConvertType","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":1040,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":301,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->","args":[{"__class__":"OC\\Core\\Command\\Db\\ConvertType"},"*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":171,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Console/Application.php","line":187,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/console.php","line":87,"function":"run","class":"OC\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/occ","line":11,"args":["/var/www/html/console.php"],"function":"require_once"}],"File":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","Line":108,"message":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","exception":{},"CustomMessage":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated"}}
{"reqId":"ZRgClME0hHcX2Sx0oEhq","level":3,"time":"2025-02-05T09:26:00+00:00","remoteAddr":"","user":false,"app":"PHP","method":"","url":"--","message":"Undefined array key \"SERVER_PROTOCOL\" at /var/www/html/lib/private/AppFramework/Http/Request.php#654","userAgent":"--","version":"32.0.0.0","data":{"app":"PHP"}}
{"reqId":"ZRgClME0hHcX2Sx0oEhq","level":3,"time":"2025-02-05T09:26:00+00:00","remoteAddr":"","user":false,"app":"PHP","method":"","url":"--","message":"Undefined array key \"HTTP_USER_AGENT\" at /var/www/html/apps/profiler/lib/DataCollector/HttpDataCollector.php#31","userAgent":"--","version":"32.0.0.0","data":{"app":"PHP"}}
{"reqId":"8INNP77VR5U8G8wBw1t4","level":3,"time":"2025-02-05T09:26:31+00:00","remoteAddr":"","user":false,"app":"no app in context","method":"","url":"--","message":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","userAgent":"--","version":"32.0.0.0","exception":{"Exception":"OCP\\AppFramework\\QueryException","Message":"Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","Code":0,"Trace":[{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":124,"function":"resolve","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php","line":439,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php","line":411,"function":"queryNoFallback","class":"OC\\AppFramework\\DependencyInjection\\DIContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":42,"function":"query","class":"OC\\AppFramework\\DependencyInjection\\DIContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/apps/admin_audit/lib/AppInfo/Application.php","line":127,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\AdminAudit\\IAuditLogger"]},{"file":"/var/www/html/lib/private/AppFramework/Bootstrap/Coordinator.php","line":157,"function":"boot","class":"OCA\\AdminAudit\\AppInfo\\Application","type":"->","args":[{"__class__":"OC\\AppFramework\\Bootstrap\\BootContext"}]},{"file":"/var/www/html/lib/private/App/AppManager.php","line":479,"function":"bootApp","class":"OC\\AppFramework\\Bootstrap\\Coordinator","type":"->","args":["admin_audit"]},{"file":"/var/www/html/lib/private/legacy/OC_App.php","line":93,"function":"loadApp","class":"OC\\App\\AppManager","type":"->","args":["admin_audit"]},{"file":"/var/www/html/core/Command/Db/ConvertType.php","line":213,"function":"loadApp","class":"OC_App","type":"::","args":["admin_audit"]},{"file":"/var/www/html/core/Command/Db/ConvertType.php","line":169,"function":"createSchema","class":"OC\\Core\\Command\\Db\\ConvertType","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Command/Command.php","line":298,"function":"execute","class":"OC\\Core\\Command\\Db\\ConvertType","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":1040,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":301,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->","args":[{"__class__":"OC\\Core\\Command\\Db\\ConvertType"},"*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":171,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Console/Application.php","line":187,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/console.php","line":87,"function":"run","class":"OC\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/occ","line":11,"args":["/var/www/html/console.php"],"function":"require_once"}],"File":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","Line":108,"message":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated","exception":{},"CustomMessage":"Could not boot admin_audit: Could not resolve OCA\\AdminAudit\\IAuditLogger! Class can not be instantiated"}}
{"reqId":"8INNP77VR5U8G8wBw1t4","level":3,"time":"2025-02-05T09:26:31+00:00","remoteAddr":"","user":false,"app":"no app in context","method":"","url":"--","message":"Could not boot gf: Could not resolve cacheListener! Class \"cacheListener\" does not exist","userAgent":"--","version":"32.0.0.0","exception":{"Exception":"OC\\AppFramework\\Utility\\QueryNotFoundException","Message":"Could not resolve cacheListener! Class \"cacheListener\" does not exist","Code":0,"Trace":[{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":124,"function":"resolve","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["cacheListener"]},{"file":"/var/www/html/lib/private/ServerContainer.php","line":155,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["cacheListener",true]},{"file":"/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php","line":414,"function":"query","class":"OC\\ServerContainer","type":"->","args":["cacheListener",true]},{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":42,"function":"query","class":"OC\\AppFramework\\DependencyInjection\\DIContainer","type":"->","args":["cacheListener"]},{"file":"/var/www/html/lib/private/AppFramework/Bootstrap/FunctionInjector.php","line":39,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["cacheListener"]},{"function":"OC\\AppFramework\\Bootstrap\\{closure}","class":"OC\\AppFramework\\Bootstrap\\FunctionInjector","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/AppFramework/Bootstrap/FunctionInjector.php","line":49,"function":"array_map","args":[{"__class__":"Closure"},[{"__class__":"ReflectionParameter","name":"mountProviderCollection"},"*** sensitive parameters replaced ***",{"__class__":"ReflectionParameter","name":"eventDispatcher"}]]},{"file":"/var/www/html/lib/private/AppFramework/Bootstrap/BootContext.php","line":32,"function":"injectFn","class":"OC\\AppFramework\\Bootstrap\\FunctionInjector","type":"->","args":[{"__class__":"Closure"}]},{"file":"/var/www/html/apps-extra/gf/lib/AppInfo/Application.php","line":237,"function":"injectFn","class":"OC\\AppFramework\\Bootstrap\\BootContext","type":"->","args":[{"__class__":"Closure"}]},{"file":"/var/www/html/lib/private/AppFramework/Bootstrap/Coordinator.php","line":157,"function":"boot","class":"OCA\\GroupFolders\\AppInfo\\Application","type":"->","args":[{"__class__":"OC\\AppFramework\\Bootstrap\\BootContext"}]},{"file":"/var/www/html/lib/private/App/AppManager.php","line":479,"function":"bootApp","class":"OC\\AppFramework\\Bootstrap\\Coordinator","type":"->","args":["gf"]},{"file":"/var/www/html/lib/private/legacy/OC_App.php","line":93,"function":"loadApp","class":"OC\\App\\AppManager","type":"->","args":["gf"]},{"file":"/var/www/html/core/Command/Db/ConvertType.php","line":213,"function":"loadApp","class":"OC_App","type":"::","args":["gf"]},{"file":"/var/www/html/core/Command/Db/ConvertType.php","line":169,"function":"createSchema","class":"OC\\Core\\Command\\Db\\ConvertType","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Command/Command.php","line":298,"function":"execute","class":"OC\\Core\\Command\\Db\\ConvertType","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":1040,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":301,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->","args":[{"__class__":"OC\\Core\\Command\\Db\\ConvertType"},"*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/console/Application.php","line":171,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/lib/private/Console/Application.php","line":187,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"]},{"file":"/var/www/html/console.php","line":87,"function":"run","class":"OC\\Console\\Application","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/occ","line":11,"args":["/var/www/html/console.php"],"function":"require_once"}],"File":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","Line":113,"message":"Could not boot gf: Could not resolve cacheListener! Class \"cacheListener\" does not exist","exception":{},"CustomMessage":"Could not boot gf: Could not resolve cacheListener! Class \"cacheListener\" does not exist"}}
{"reqId":"kyfh5LNzomnR4aQEjUfr","level":2,"time":"2025-02-05T09:30:03+00:00","remoteAddr":"","user":false,"app":"cron","method":"","url":"--","message":"failed to create instance of background job: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions","userAgent":"--","version":"32.0.0.0","exception":{"Exception":"OC\\AppFramework\\Utility\\QueryNotFoundException","Message":"Could not resolve OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions! Class \"OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions\" does not exist","Code":0,"Trace":[{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":124,"function":"resolve","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions"]},{"file":"/var/www/html/lib/private/ServerContainer.php","line":155,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions",true]},{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":42,"function":"query","class":"OC\\ServerContainer","type":"->","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions"]},{"file":"/var/www/html/lib/public/Server.php","line":39,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions"]},{"file":"/var/www/html/lib/private/BackgroundJob/JobList.php","line":288,"function":"get","class":"OCP\\Server","type":"::","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions"]},{"file":"/var/www/html/lib/private/BackgroundJob/JobList.php","line":197,"function":"buildJob","class":"OC\\BackgroundJob\\JobList","type":"->","args":[{"id":907,"class":"OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions","argument":"null","last_run":1738746905,"last_checked":1738747503,"0":"And 4 more entries, set log level to debug to see all entries"}]},{"file":"/var/www/html/cron.php","line":150,"function":"getNext","class":"OC\\BackgroundJob\\JobList","type":"->","args":[false,null]}],"File":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","Line":113,"message":"failed to create instance of background job: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions","exception":{},"CustomMessage":"failed to create instance of background job: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions"}}
{"reqId":"kyfh5LNzomnR4aQEjUfr","level":2,"time":"2025-02-05T09:30:04+00:00","remoteAddr":"","user":false,"app":"cron","method":"","url":"--","message":"failed to create instance of background job: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash","userAgent":"--","version":"32.0.0.0","exception":{"Exception":"OC\\AppFramework\\Utility\\QueryNotFoundException","Message":"Could not resolve OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash! Class \"OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash\" does not exist","Code":0,"Trace":[{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":124,"function":"resolve","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash"]},{"file":"/var/www/html/lib/private/ServerContainer.php","line":155,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash",true]},{"file":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","line":42,"function":"query","class":"OC\\ServerContainer","type":"->","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash"]},{"file":"/var/www/html/lib/public/Server.php","line":39,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash"]},{"file":"/var/www/html/lib/private/BackgroundJob/JobList.php","line":288,"function":"get","class":"OCP\\Server","type":"::","args":["OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash"]},{"file":"/var/www/html/lib/private/BackgroundJob/JobList.php","line":197,"function":"buildJob","class":"OC\\BackgroundJob\\JobList","type":"->","args":[{"id":908,"class":"OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash","argument":"null","last_run":1738746905,"last_checked":1738747503,"0":"And 4 more entries, set log level to debug to see all entries"}]},{"file":"/var/www/html/lib/private/BackgroundJob/JobList.php","line":226,"function":"getNext","class":"OC\\BackgroundJob\\JobList","type":"->","args":[false,null]},{"file":"/var/www/html/cron.php","line":150,"function":"getNext","class":"OC\\BackgroundJob\\JobList","type":"->","args":[false,null]}],"File":"/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php","Line":113,"message":"failed to create instance of background job: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash","exception":{},"CustomMessage":"failed to create instance of background job: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupTrash"}}

Additional info

php -d memory_limit=2048M occ db:convert-type --password postgres --all-apps pgsql postgres database-pgsql --port 5432 nextcloud

The command output logs:

In TableDoesNotExist.php line 16:
                                                                       
  There is no table with name "postgres.oc_talk_rooms" in the schema.  

In TableDoesNotExist.php line 16:
                                                                              
  There is no table with name "postgres.oc_group_folders_acl" in the schema.  

These errors were skipped by just removing the respective apps.

On a successful run, this was printed for the skipped tables. oc_ex_* tables and *_ex tables belong to app_api and there was no error about it.
There is nothing special about oc_context_chat_content_queue for it to be not migrated, apart from the fact that it is empty (did not have any rows).

The following tables will not be converted:
oc_appconfig_ex
oc_context_chat_content_queue
oc_ex_apps
oc_ex_apps_daemons
oc_ex_apps_routes
oc_ex_event_handlers
oc_ex_occ_commands
oc_ex_settings_forms
oc_ex_speech_to_text
oc_ex_speech_to_text_q
oc_ex_task_processing
oc_ex_text_processing
oc_ex_text_processing_q
oc_ex_translation
oc_ex_translation_q
oc_ex_ui_files_actions
oc_ex_ui_scripts
oc_ex_ui_states
oc_ex_ui_styles
oc_ex_ui_top_menu
oc_group_folders
oc_group_folders_acl
oc_group_folders_groups
oc_group_folders_manage
oc_group_folders_trash
oc_group_folders_versions
oc_preferences_ex
oc_talk_attachments
oc_talk_attendees
oc_talk_bans
oc_talk_bots_conversation
oc_talk_bots_server
oc_talk_bridges
oc_talk_commands
oc_talk_consent
oc_talk_internalsignaling
oc_talk_invitations
oc_talk_poll_votes
oc_talk_polls
oc_talk_proxy_messages
oc_talk_reminders
oc_talk_retry_ocm
oc_talk_rooms
oc_talk_sessions

[solracsf]

As a workaround in the meanwhile, you could try pgLoader: https://pgloader.readthedocs.io/en/latest/intro.html

[joshtrichards]

DB conversion can't function for apps that aren't functioning / can't load.

Does the same thing happen if you fix the admin_audit configuration? (looks like you have it configured for usage but don't have the admin_audit app itself installed and enabled). That's what a lot of the log entries look like they're for.

Also, since you ran with --all-apps, any disabled apps have to be loadable w/o errors too.

Re: app_api tables - is app_api fully functioning?

Happen to have the full output of the run?

Sounds like our error handling could use some clean-up here in the convertor.

[kyteinsky]

hello, thanks for the solutions. Sorry I forgot to mention this was on a test setup and I have not lost any data, nor stuck with it.

Does the same thing happen if you fix the admin_audit configuration? (looks like you have it configured for usage but don't have the admin_audit app itself installed and enabled). That's what a lot of the log entries look like they're for.

The log_condition is set to use admin_audit but the app itself was disabled and loadable.

Also, since you ran with --all-apps, any disabled apps have to be loadable w/o errors too.

No disabled app had its own custom table.

Re: app_api tables - is app_api fully functioning?

Yes, it was. Now all the data is inexistent in the new db. Also, the creation of tables had to be forced with app disable and enable.

Happen to have the full output of the run?

Sorry, not now but I have the old db so it should be possible to simulate it again. I'll do it in some time if that is strictly required.

[makuser]

DB conversion can't function for apps that aren't functioning / can't load.

That is not the reason here, though. The order the migrations are applied in, is wrong.

Example for groupfolders:

Migrating schema only
- Reading 102020Date20180806161449
- Reading 103000Date20180806161724
- Reading 104000Date20180918132853
- Reading 1000000Date20210216085047

The last migration that was run 1000000Date20210216085047, references the table group_folders_acl, which is first created in 201000Date20190111132839, which is definitely not executed yet, according to the log above.

Available migrations sorted by prefixed number:

app	version
groupfolders	16000Date20230821085801
groupfolders	19000Date20240903062631
groupfolders	19000Date20241029123147
groupfolders	102020Date20180806161449
groupfolders	103000Date20180806161724
groupfolders	104000Date20180918132853
groupfolders	201000Date20190111132839
groupfolders	201000Date20190212150323
groupfolders	300000Date20240905185515
groupfolders	401001Date20190715092137
groupfolders	501000Date20190927102434
groupfolders	501000Date20191218182434
groupfolders	802000Date20201119112624
groupfolders	1000000Date20210216085047
groupfolders	1401000Date20230426112001
groupfolders	1401000Date20230426112002

Available migrations sorted by date:

app	version
groupfolders	102020Date20180806161449
groupfolders	103000Date20180806161724
groupfolders	104000Date20180918132853
groupfolders	201000Date20190111132839
groupfolders	201000Date20190212150323
groupfolders	401001Date20190715092137
groupfolders	501000Date20190927102434
groupfolders	501000Date20191218182434
groupfolders	802000Date20201119112624
groupfolders	1000000Date20210216085047
groupfolders	1401000Date20230426112001
groupfolders	1401000Date20230426112002
groupfolders	16000Date20230821085801
groupfolders	19000Date20240903062631
groupfolders	300000Date20240905185515
groupfolders	19000Date20241029123147

I don't know how the migration logic is supposed to work in NC, but either way, the way it works now can only be completely wrong. I hope it's only affecting the db:conversion progress and not the normal updating process, as that would be dangerous during normal upgrades, oof 🙈

Same for spreed, but I'm too lazy to format it all just to prove my point. Not sure which other apps are impacted as well, I mean at least they don't break the conversion process in this spectacular way, but it obviously doesn't mean it won't break later on. There is a reason we all are using schema migrations in all sorts of applications using databases, including a very specific order in which they have to be applied.

[makuser]

According to the code for the Db\MigrationService, sortMigrations yields the following order for the groupfolders app:

array(16) {
  [0]=>
  string(23) "16000Date20230821085801"
  [1]=>
  string(23) "19000Date20240903062631"
  [2]=>
  string(23) "19000Date20241029123147"
  [3]=>
  string(24) "102020Date20180806161449"
  [4]=>
  string(24) "103000Date20180806161724"
  [5]=>
  string(24) "104000Date20180918132853"
  [6]=>
  string(24) "201000Date20190111132839"
  [7]=>
  string(24) "201000Date20190212150323"
  [8]=>
  string(24) "300000Date20240905185515"
  [9]=>
  string(24) "401001Date20190715092137"
  [10]=>
  string(24) "501000Date20190927102434"
  [11]=>
  string(24) "501000Date20191218182434"
  [12]=>
  string(24) "802000Date20201119112624"
  [13]=>
  string(25) "1000000Date20210216085047"
  [14]=>
  string(25) "1401000Date20230426112001"
  [15]=>
  string(25) "1401000Date20230426112002"
}

LGTM that way.

[paukstis]

Hit this issue too, what's the current status, recommendation ? pgLoader ?