fix: restore ESM testing with Node.js >=v18.19.0 and v20 and v21; clarify ESM Node.js version support (elastic#3857)

With the recent update to [email protected] our ESM support
is working with Node.js v18.19.0, and with v20 and v21 (`^18.1.0 || >=20.2.0`)

Author: trentm

.ci/tav.json

@@ -1,6 +1,5 @@
 {
-  "// versions": "Skip >=18.19.0 until IITM double-import issue is resolved.",
-  "versions": [ "21", "20", "18.18.2", "16", "14" ],
+  "versions": [ "21", "20", "18", "16", "14" ],
   "// modules": [
     "List of instrumented modules with the minimum Node major version supported.",
     "minMajorVersion for each module should be kept in sync with .tav.yml"

.github/workflows/test.yml

@@ -130,7 +130,7 @@ jobs:
           - '21.0'
           - '20'
           - '20.0'
-          - '18.18.2' # Skip >=18.19.0 until IITM double-import issue is resolved.
+          - '18'
           - '18.0'
           - '16'
           - '16.0'

CHANGELOG.asciidoc

@@ -34,9 +34,35 @@ Notes:
 See the <<upgrade-to-v4>> guide.
 
 
+==== Unreleased
+
+[float]
+===== Breaking changes
+
+[float]
+===== Features
+
+[float]
+===== Bug fixes
+
+* Add support for <<esm,instrumentation of ES module-using (ESM) code>> with
+  Node.js versions matching `^18.19.0 || >=20.2.0`. Before this version of
+  the APM agent, ESM instrumentation was only supported for some *earlier*
+  Node.js versions. Changes in Node.js's ESM loader in v18.19.0 and v20 broke
+  earlier ESM support. ({issues}3784[#3784], {pull}3844[#3844])
+
+[float]
+===== Chores
+
+
 [[release-notes-4.4.0]]
 ==== 4.4.0 - 2024/01/12
 
+*Known issue*: Using the APM agent's <<esm>> with Node.js *v18.19.0* is not
+supported in this version. Upgrade to APM agent version v4.5.0 or later, or use
+Node.js v18.18.1 or earlier.
+See https://github.com/elastic/apm-agent-nodejs/issues/3784 for details.
+
 [float]
 ===== Features
 
@@ -60,10 +86,10 @@ See the <<upgrade-to-v4>> guide.
 [[release-notes-4.3.0]]
 ==== 4.3.0 - 2023/12/05
 
-*Known issue*: Using the APM agent's <<esm>> with Node v18.19.0 is currently
-not supported. Earlier versions of Node.js v18 and Node.js v20 and later
-*are* supported. See <https://github.com/elastic/apm-agent-nodejs/issues/3784>
-for details.
+*Known issue*: Using the APM agent's <<esm>> with Node.js *v18.19.0* is not
+supported in this version. Upgrade to APM agent version v4.5.0 or later, or use
+Node.js v18.18.1 or earlier.
+See https://github.com/elastic/apm-agent-nodejs/issues/3784 for details.
 
 [float]
 ===== Features

docs/esm.asciidoc

@@ -70,11 +70,9 @@ As well, the APM agent must also be separately *started* -- for example via `--r
 
 Automatic instrumentation of ES modules is based on the experimental Node.js Loaders API. ESM support in the Elastic APM Node.js agent will remain *experimental* while the Loaders API is experimental.
 
-ESM auto-instrumentation is only supported for Node.js versions that match *`^12.20.0 || ^14.13.1 || ^16.0.0 || ^18.1.0 <20`*.
+ESM auto-instrumentation is only supported for Node.js versions that match *`^12.20.0 || ^14.13.1 || ^16.0.0 || ^18.1.0 || >=20.2.0`*.
 The behavior when using `node --experimental-loader=elastic-apm-node/loader.mjs` with earlier Node.js versions is undefined and unsupported.
 
-Notably, ESM auto-instrumentation is *not* supported with Node.js v20 because of changes in the Loaders API. Using the loader with Node.js v20 can result in crashes in `import ...` statements that attempt named imports from some CommonJS modules. The error message will be of the form `SyntaxError: The requested module '<module name>' does not provide an export named '<export name>'`. (You can track https://github.com/DataDog/import-in-the-middle/issues/29[this issue] for progress.)
-
 
 [float]
 [[esm-compat-modules]]

test/testconsts.js

@@ -15,10 +15,13 @@ const os = require('os');
 //   fixes in the .1 minor release
 // - v18.1.0 fixes an issue in v18.0.0
 //   https://github.com/nodejs/node/pull/42881
-// - Current node v20 does not work with IITM
+// - `^18.19.0 || >=20` support was added by IITM@1.7.3
 //   https://github.com/DataDog/import-in-the-middle/pull/27
-const NODE_VER_RANGE_IITM = '^12.20.0 || ^14.13.1 || ^16.0.0 || ^18.1.0 <20';
-const NODE_VER_RANGE_IITM_GE14 = '^14.13.1 || ^16.0.0 || ^18.1.0 <20'; // NODE_VER_RANGE_IITM minus node v12
+// - v20.2.0 fixes an issue in v20
+//   I think it is https://github.com/nodejs/node/issues/47929
+const NODE_VER_RANGE_IITM =
+  '^12.20.0 || ^14.13.1 || ^16.0.0 || ^18.1.0 || >=20.2.0';
+const NODE_VER_RANGE_IITM_GE14 = '^14.13.1 || ^16.0.0 || ^18.1.0 || >=20.2.0'; // NODE_VER_RANGE_IITM minus node v12
 
 // This can be passed as tape test options for tests that are timing sensitive,
 // to *skip* those tests on Windows CI.