[Master issue] Prepare eXist 5.3.0 stable release

[joewiz]

• Memory/Resource leak in XQuery Context Variables of Library Modules* - Cleanup all module contexts #3897
• Maps* - a follow-on PR just for 5.3.0 to Switch map:merge from use-last to use-first as per XQuery spec #3738 (@adamretter please replace "3738" here with the new, split PR when it's ready)
• Security* - [BUG] Guest users can create collections hidden from dba users #3505
• Security - Finish and release* https://github.com/eXist-db/templating
• Security - core+Saxon - add option to limit RFI in fn:doc etc* - [feature] Allow external HTTP requests to be limited to authenticated users or blocked altogether #3927
• Security - core* - Apache XML-RPC CVE-2019-17570 / CVE-2016-5002 - Apache XML-RPC CVE-2016-5002 #3063 Update Apache XML-RPC #3934 cannot be merged because it introduces a breaking change
• file:sync prune param* - added optional 'prune' param to file:sync(...) #3084 still needs XQSuite tests
• Review/merge PR updating crypto lib for compatibility with 5.3.0 - next version, compatible with eXist 5.3.0 expath-crypto-module#37
• Security: Update core apps to resolve security issues identified by dependabot
  • dashboard* - https://github.com/eXist-db/dashboard/security/dependabot
  • [ ] docs* - https://github.com/eXist-db/documentation/security/dependabot - these are build warnings, not security issues with read-only running app
  • eXide* - https://github.com/eXist-db/eXide/security/dependabot
  • monex* - https://github.com/eXist-db/monex/security/dependabot
• Update core apps to use templating v1.0.0:
  • doc - Switch to the new templating module, remove shared-resources dependency documentation#626
  • fundocs - Switch to the new templating module function-documentation#41
  • monex - [feat] switch to new templating library monex#153
• Update core apps to remove all dependencies on shared-resources:
  • doc - Switch to the new templating module, remove shared-resources dependency documentation#626
  • eXide - Drop dependency on shared-resources eXide#309
  • fundocs - remove dependency on shared resources and cleanup function-documentation#42
  • markdown - Remove shared-resources dependency exist-markdown#7
  • monex - remove dependency on shared resources package monex#155
• Publish the updated core apps to public-repo and GitHub Releases
  • doc
  • eXide
  • fundocs
  • markdown
  • monex
• Update release process for GitHub Releases* - Fix release process since migration from BinTray to GitHub Releases #3845

Finally:

• Publish the release according to https://github.com/eXist-db/exist/blob/develop/exist-versioning-release.md#preparing-a-product-release
• Update exist-db.org with eXist 5.3.0 and updated core apps

Open Issue

• push 5.3.0 to master
• Go to GitHub and move all issues and PRs which are still open for the release milestone to the next release milestone and close the milestones 5.3.0 and 5.2.1
• merge open PR and release crypto-lib 6.0.0
• update homebrew
• publish updated “howto release” document [doc] update versioning and release guide #3962
• Github-release-plugin: fix link to release notes and improve layout
• Wikipedia page: Lars asked someone from outside the community to update it. This might take a moment.

Note: Tasks marked with * are drawn from @adamretter's list from Slack.

[joewiz]

Now updated with entries for reviewing and merging the crypto lib package, and publishing it to the public-repo.

[line-o]

Crypto lib needs to wait for 5.3.0 to be released - so technically we should mention this ticket over in the crypto lib PR and not the other way around. Do you agree @joewiz ?

[line-o]

I think Security - core+Saxon - XXE to RFI in fn:doc etc* is already addressed here
[bugfix] Change XML processing defaults for v6.0.0 by dizzzz · Pull Request #3836 · eXist-db/exist
So the question is wether to accept this for security reasons even if it might affect users of the database.

[joewiz]

@line-o Ah, I didn't realize the crypto lib's new version had to come after the release of eXist 5.3.0. Since the crypto lib's release is an important aspect of the eXist 5.3.0 release, I think it's worth keeping a checkbox here. How about if I move that to-do item to the "Finally" section?

[joewiz]

@line-o Regarding the briefly worded "core+saxon" item, I agree that #3836 addresses the XXE issue, but it doesn't address the RFI issue. That topic came up only in the Community Call on May 10:

Functions that can be used to perform external HTTP requests, e.g.: doc(). May need to institute a whitelist/blacklist, or disable external HTTP requests by default. Could be a configuration option—off by default in 5.3.0 but on by default in 6.0.0.

• doc, doc-available, json-doc, unparsed-text, unparsed-text-lines
• EXPath HTTP client
• xinclude
• transform:transform

AR: Suggests we solve these before the forthcoming release.

This deserves its own issue, but the idea is that eXist allows guest users to trigger HTTP requests for remote files (RFI), and the configuration option envisioned here would add a condition to all functions like doc(), limiting the ability to perform HTTP requests. Perhaps the time is right for me to open an issue for further discussion of this?

Update: Issue added: #3927.

[line-o]

@joewiz re: crypto lib checkbox in Finally is good.

[line-o]

@joewiz regarding the introduction of new configuration option to address RFI for guest users: That deserves its separate issue, yes.

[line-o]

CVE-2019-17570 addresses malicious XML-RPC servers. In our scenario this is existdb itself which we ultimately trust as it is under our control, correct?

[adamretter]

CVE-2019-17570

I fixed this on in FusionDB previously. You basically have to host patched versions of the Apache XML-RPC Jars

[line-o]

Is Apache XML-RPC used as a client in exist-db or as a server?

[line-o]

I would like to drop #3738 from the list as it introduces a breaking change (order of application of map:merge).

[joewiz]

Indeed, #3738 is labelled as bound for 6.0.0. Ok to remove it from this list, @adamretter?