From a9ce7f0722fc7f827b1bf12df6d62266da83e0cf Mon Sep 17 00:00:00 2001 From: Ruy Adorno Date: Thu, 19 Mar 2020 17:53:16 -0400 Subject: [PATCH 1/9] RFC: npm workspaces part 2 --- accepted/0000-workspaces-part2.md | 188 ++++++++++++++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 accepted/0000-workspaces-part2.md diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md new file mode 100644 index 000000000..40ab93a21 --- /dev/null +++ b/accepted/0000-workspaces-part2.md @@ -0,0 +1,188 @@ +# npm workspaces: Running commands + +## Summary + +Introduces basic support to running **npm commands** across nested **workspaces**. + +## Motivation + +The need for running **npm commands** across nested **workspaces** was sourced from the community during the ellaboration of the [original **npm workspaces** RFC](https://github.com/npm/rfcs/pull/103). + +## Detailed Explanation + +Following up from the original **npm workspaces** RFC are the following command-related changes: + +- A subset of the standard **npm commands** needs to be made aware of **npm workspaces** +- It's desired to have a standard way to filter out **workspaces** in which to run specific **npm commands** + +## Rationale and Alternatives + +The ability to run **npm commands** across defined **workspaces** is essential to successfully manage a **npm workspaces** workflow. That makes the following alternatives much less desireable: + +- Do not implement further support to manage running commands across **workspaces** +- Implement only _some_ of the implementation changes described in this document + +**Note:** Alternative ways of executing each of the described features are defined along with their implementation details below. + +## Implementation + +### 1. Run commands across all child packages + +Make **npm cli** subcommands **npm workspaces**-aware, so that running a command tries to run it within all **workspaces** as long as a **workspaces configuration** field is properly defined in `package.json`. + +Only a subset of commands are to be supported: + +- `fund` List funding info for all **workspaces** +- `ls` List all packages including **workspaces** +- `outdated` List outdated **dependencies** including **workspaces** and its **dependencies** +- `run-script` Run arbitrary **scripts** in all **workspaces**, skip any **workspace** that does not have a targetting **script** +- `rebuild` Rebuild all **workspaces** +- `restart` +- `start` +- `stop` +- `test` Run test **scripts** in all **workspaces** +- `update` Updates a **dependency** across the entire installation tree, including **workspaces** +- `view` View registry info, also including **workspaces** + +#### Test example: + +``` +├── package.json { "name": "foo", "workspaces": ["dep-a", "dep-b"] } +├── dep-a +│ └── package.json { "version": "1.0.0" } +└── dep-b + └── package.json { "version": "1.3.1" } + +$ npm test + +> foo@1.0.0 test /Users/username/foo +> echo "Error: no test specified" && exit 1 + +Error: no test specified +npm ERR! Test failed. See above for more details. + +> dep-a@1.0.0 test /Users/username/foo/dep-a +> done + +> dep-b@1.3.1 test /Users/username/foo/dep-b +> done +``` + +**NOTE:** `publish` and `version` are very special cases in a **npm workspaces** setup and are outside of the scope of this RFC. These features will live in userland in the form of popular modules such as [Lerna](https://lerna.js.org/) for the foreseeable future. + +### 2. Filter a subset of workspaces + +Rather than defining a strict UX, the current proposal is to adhere to a `workspace` command prefix in the form of either: `npm workspace ` OR `npm ws ` OR an even shorter shortcut using a symbol `npm : `, for example running `test` in a specific **workspace** named `foo` with these different ideas: + +- `npm workspace foo test` +- `npm ws foo test` +- `npm :foo test` + +Leaving space to gather feedback from the community on whether to introduce one of this proposed commands or even introducing some novel interface. + +#### Install dependency example: + +Add `tap` as a **dependency** of a **workspace** named `foo`: + +```sh +npm workspace foo install tap +npm ws foo install tap +npm :foo install tap +``` + +### 3. Grouping/Aliasing workspaces + +One proposed feature that was surfaced during our brainstorm sessions is the ability for users to create arbitrary groups of **workspaces** defined via **workspaces configuration**, e.g: + +Given a **npm workspace** setup with the following file structure: + +``` +/root +├── core +│ ├── foo +│ ├── bar +│ └── util +└── plugins + ├── helpers + ├── lorem + └── ipsum +``` + +It's possible to define a **groups** property within your **workspaces configuration** that will allow defining named sets of **workspaces** that can be later on used to run commands, e.g: + +``` +{ + "name": "workspace-example", + "version": "1.0.0", + "workspaces": { + "groups": { + "core": ["core/*"], // accepts fs location + "plugins": ["lorem", "ipsum"], // also accepts workspaces names + "common": ["util", "helpers"] + }, + "packages": [ + "core/*", + "plugins/*" + ] + } +} +``` + +Following up with the previous configuration example, it would be possible to run tests across all **workspaces** from the `plugins` group, as following: + +``` +// one of these: +$ npm workspace plugins test +$ npm ws plugins test +$ npm :plugins test + +> lorem@1.0.0 test /root/plugins/lorem +> done + +> ipsum@1.0.0 test /root/plugins/ipsum +> done +``` + +Further example, install a **peer dependency** across a group of packages named `core`: + +``` +$ npm workspace core install react@16 --save-peer +$ npm ws core install react@16 --save-peer +$ npm :core install react@16 --save-peer +``` + +## Prior Art + +#### Previously + +- [npm workspaces](https://github.com/npm/rfcs/blob/de8d71c0453f5cf443d3ef2f47e313f12dd6aaf9/accepted/0000-workspaces.md) + +#### Filtering a subset of workspaces + +- [lerna filter-options](https://www.npmjs.com/package/@lerna/filter-options) +- [Yarn v1 workspace](https://classic.yarnpkg.com/en/docs/cli/workspace) +- [Yarn v2 foreach include/exclude](https://yarnpkg.com/cli/workspaces/foreach) +- [pnpm recursive --filter](https://pnpm.js.org/en/cli/recursive#filter-lt-package_selector) + +## Dictionary + +During the discussions around this RFC it was brought up to our attention that a lot of the vocabulary surrounding what the larger JavaScript community understands as "workspaces" can be confusing, for the sake of keeping the discussion as productive as possible we're taking the extra step of documenting what each of the terms used here means: + +- **npm cli**: The [npm cli](https://github.com/npm/cli/) :wink: +- **npm commands**: The set of **npm cli** [commands](https://docs.npmjs.com/cli-documentation/) +- **npm workspaces**: The feature name, meaning the ability to the **npm cli** to support a better workflow for working with multiple packages. +- **workspaces**: A set of **workspace**s. +- **workspace**: A nested package within the **Top-level workspace** file system that is explicitly defined as such via **workspaces configuration**. +- **Top-level workspace**: The root level package that contains a **workspaces configuration** defining **workspaces**. +- **workspaces configuration**: The blob of json configuration defined within `package.json` that declares where to find **workspaces** for this **Top-level workspace** package. +- **dependency**: A package that is depended upon by another given package. +- **[peer dependency](https://docs.npmjs.com/files/package.json#peerdependencies)**: A special **dependency** relationship between packages. +- **dependent**: A package which depends on another given package. +- **symlink**: A [symbolic link](https://en.wikipedia.org/wiki/Symbolic_link) between files. +- **[globs](https://en.wikipedia.org/wiki/Glob_(programming))**: String patterns that specifies sets of filenames with special characters. +- **[Arborist](https://github.com/npm/arborist)**: The npm@7 install library +- **hoisting packages**: Bringing packages up a level in the context of an installation tree. + +## Unresolved Questions and Bikeshedding + +TBD From 979b995aee373613b37d61f7dddcbed90b532fd4 Mon Sep 17 00:00:00 2001 From: Ruy Adorno Date: Wed, 13 May 2020 17:02:42 -0400 Subject: [PATCH 2/9] add feedback about filters from the openrfc call --- accepted/0000-workspaces-part2.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md index 40ab93a21..fa931f6c2 100644 --- a/accepted/0000-workspaces-part2.md +++ b/accepted/0000-workspaces-part2.md @@ -72,7 +72,13 @@ npm ERR! Test failed. See above for more details. ### 2. Filter a subset of workspaces -Rather than defining a strict UX, the current proposal is to adhere to a `workspace` command prefix in the form of either: `npm workspace ` OR `npm ws ` OR an even shorter shortcut using a symbol `npm : `, for example running `test` in a specific **workspace** named `foo` with these different ideas: +Filter is done via positional argument and here are some of the reasons why we decided to go that route: +- [Lerna filters](https://www.npmjs.com/package/@lerna/filter-options) were the starting point but early in the discussion some other considerations were brought forward +- npm configs have some baggage, configs might apply to the larger behavior of the cli or to a specific subcommand and they're all mixed together +- defining a subset of workspaces in a `.npmrc` file might be just a footgun leading users into confusion +- Yarn v1 uses positional arguments for filter in the form of `yarn workspace `, so there's precedent + +Leading this RFC to a proposal using positional arguments. Rather than defining a strict UX, the current proposal is to adhere to a `workspace` command prefix in the form of either: `npm workspace ` OR `npm ws ` OR an even shorter shortcut using a symbol `npm : `, for example running `test` in a specific **workspace** named `foo` with these different ideas: - `npm workspace foo test` - `npm ws foo test` @@ -90,6 +96,8 @@ npm ws foo install tap npm :foo install tap ``` +Note: **Globs** are not supported as a _subset/filter_ argument, the alternative is to use `npm workspace ` in which the `alias` can be defined beforehand in your **workspace** configuration as detailed in the next section. + ### 3. Grouping/Aliasing workspaces One proposed feature that was surfaced during our brainstorm sessions is the ability for users to create arbitrary groups of **workspaces** defined via **workspaces configuration**, e.g: From 8a767a06e03ddd8a818e5ddd264f1ed224ab7c6a Mon Sep 17 00:00:00 2001 From: Ruy Adorno Date: Mon, 22 Jun 2020 20:26:45 -0400 Subject: [PATCH 3/9] add --workspace-group arg idea --- accepted/0000-workspaces-part2.md | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md index fa931f6c2..e7f459d67 100644 --- a/accepted/0000-workspaces-part2.md +++ b/accepted/0000-workspaces-part2.md @@ -124,7 +124,7 @@ It's possible to define a **groups** property within your **workspaces configura "version": "1.0.0", "workspaces": { "groups": { - "core": ["core/*"], // accepts fs location + "core": ["core/*"], // accepts globs "plugins": ["lorem", "ipsum"], // also accepts workspaces names "common": ["util", "helpers"] }, @@ -159,6 +159,12 @@ $ npm ws core install react@16 --save-peer $ npm :core install react@16 --save-peer ``` +It should also be possible to define groups as cli arguments, so that the following example will be equivalent to the previous examples: + +``` +$ npm workspace core install react@16 --save-peer --workspace-group core=core/* +``` + ## Prior Art #### Previously @@ -193,4 +199,13 @@ During the discussions around this RFC it was brought up to our attention that a ## Unresolved Questions and Bikeshedding -TBD +- Filter syntax: + - `npm workspace foo test` + - `npm ws foo test` + - `npm :foo test` + - `npm --workspace=foo test` + - `npm -w=foo test` +- Groups as a cli argument: `--workspace-group` + - how to define the `key=value` nature of it? + - is there any arg currently in the cli using key/value pairs? + - maybe it's not worth having it? From cb9ca1459108eadbaa2a94372fbec533450e1584 Mon Sep 17 00:00:00 2001 From: Ruy Adorno Date: Wed, 22 Jul 2020 16:01:45 -0400 Subject: [PATCH 4/9] incorporated feedback from latest open rfc calls --- accepted/0000-workspaces-part2.md | 66 ++++++++++++++++++------------- 1 file changed, 38 insertions(+), 28 deletions(-) diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md index e7f459d67..5015eaaa6 100644 --- a/accepted/0000-workspaces-part2.md +++ b/accepted/0000-workspaces-part2.md @@ -6,7 +6,7 @@ Introduces basic support to running **npm commands** across nested **workspaces* ## Motivation -The need for running **npm commands** across nested **workspaces** was sourced from the community during the ellaboration of the [original **npm workspaces** RFC](https://github.com/npm/rfcs/pull/103). +The need for running **npm commands** across nested **workspaces** was sourced from the community during the elaboration of the [original **npm workspaces** RFC](https://github.com/npm/rfcs/pull/103). ## Detailed Explanation @@ -72,31 +72,50 @@ npm ERR! Test failed. See above for more details. ### 2. Filter a subset of workspaces -Filter is done via positional argument and here are some of the reasons why we decided to go that route: +Filter is done via named argument (`--workspace`, short: `-w`) and here are some of the reasons why we decided to go that route: - [Lerna filters](https://www.npmjs.com/package/@lerna/filter-options) were the starting point but early in the discussion some other considerations were brought forward -- npm configs have some baggage, configs might apply to the larger behavior of the cli or to a specific subcommand and they're all mixed together -- defining a subset of workspaces in a `.npmrc` file might be just a footgun leading users into confusion -- Yarn v1 uses positional arguments for filter in the form of `yarn workspace `, so there's precedent +- npm configs do have some baggage, configs might apply to to other cli commands and can also possibly be defined project or system-wide - that said, there are things we can do to avoid these pitfalls (such as making sure `npm_config_workspace` env varibable never gets set in `run-script`) +- During the many discussions around this RFC a few strong points for named arguments stand out: + - Allow us to keep `npm workspace` for other command usages + - npm users are more familiar with usage of named configs, example: `npm test --workspace=foo` is very similar to `npm test --prefix=./packages/foo` - if you think on it in terms of "what workspace am I in" again, a named config makes more sense in npm + - [Organized a poll](https://github.com/npm/rfcs/issues/160) in which named arguments were the leading option, here are the results for reference: + ``` + 1. (12 votes) 21% - Positional argument with one-char alias: `npm workspace foo test` or `npm w foo test` + 2. (11 votes) 11% - Positional argument but shorter default name: `npm ws foo test` or `npm w foo test` + 3. (13 votes) 13% - Special character (operator-like): `npm :foo test` + 4. (20 votes) 35% - Named argument: `npm --workspace=foo test` or `npm -w=foo test` + ``` + +Given the results of all this preliminar work, the preferred way to run a command in the context of a workspace is to use the named `--workspace` argument or its `-w` short alias, e.g: + +In a project with the following structure: +``` +./ +├── package.json { "name": "root", "workspaces": ["packages/foo", "packages/bar"] } +└── packages/ + ├── foo/ + │ └── package.json { "name": "foo", "version": "1.0.0" } + └── bar/ + └── package.json { "name": "foo", "version": "1.0.0" } +``` -Leading this RFC to a proposal using positional arguments. Rather than defining a strict UX, the current proposal is to adhere to a `workspace` command prefix in the form of either: `npm workspace ` OR `npm ws ` OR an even shorter shortcut using a symbol `npm : `, for example running `test` in a specific **workspace** named `foo` with these different ideas: +You can run tests for the `foo` workspace from the root of your project, with the following syntax: -- `npm workspace foo test` -- `npm ws foo test` -- `npm :foo test` +``` +npm test --workspace=foo +``` -Leaving space to gather feedback from the community on whether to introduce one of this proposed commands or even introducing some novel interface. #### Install dependency example: Add `tap` as a **dependency** of a **workspace** named `foo`: ```sh -npm workspace foo install tap -npm ws foo install tap -npm :foo install tap +npm install tap --workspace=foo ``` -Note: **Globs** are not supported as a _subset/filter_ argument, the alternative is to use `npm workspace ` in which the `alias` can be defined beforehand in your **workspace** configuration as detailed in the next section. + +Note: **Globs** are not supported as a valid `--workspace` argument value, the alternative is to use `npm --workspace=` in which the `alias` can be defined beforehand in your **workspace** configuration as detailed in the following up section. ### 3. Grouping/Aliasing workspaces @@ -139,10 +158,7 @@ It's possible to define a **groups** property within your **workspaces configura Following up with the previous configuration example, it would be possible to run tests across all **workspaces** from the `plugins` group, as following: ``` -// one of these: -$ npm workspace plugins test -$ npm ws plugins test -$ npm :plugins test +$ npm test --workspace=plugins > lorem@1.0.0 test /root/plugins/lorem > done @@ -154,17 +170,17 @@ $ npm :plugins test Further example, install a **peer dependency** across a group of packages named `core`: ``` -$ npm workspace core install react@16 --save-peer -$ npm ws core install react@16 --save-peer -$ npm :core install react@16 --save-peer +$ npm install react@16 --save-peer --workspace=core ``` -It should also be possible to define groups as cli arguments, so that the following example will be equivalent to the previous examples: +**TBD:** It might also be very handy being able to define groups as cli arguments, so that the following example will be equivalent to the previous examples: ``` $ npm workspace core install react@16 --save-peer --workspace-group core=core/* ``` +This is a currently unresolved question since pitfalls of glob usage in cli arguments have already surfaced many times during the time of discussion for this RFC. + ## Prior Art #### Previously @@ -199,12 +215,6 @@ During the discussions around this RFC it was brought up to our attention that a ## Unresolved Questions and Bikeshedding -- Filter syntax: - - `npm workspace foo test` - - `npm ws foo test` - - `npm :foo test` - - `npm --workspace=foo test` - - `npm -w=foo test` - Groups as a cli argument: `--workspace-group` - how to define the `key=value` nature of it? - is there any arg currently in the cli using key/value pairs? From d207867beff0bd14272f2234239ecfcf7ee8ee98 Mon Sep 17 00:00:00 2001 From: Ruy Adorno Date: Wed, 24 Feb 2021 17:51:07 -0500 Subject: [PATCH 5/9] Update RFC with latest proposed ideas A lot of brainstorming sessions and OpenRFC call discussions have led to a number of substantial changes to the original proposed RFC. I'm now committing these changes in here so that we can further discuss and iron out any details while we progress with the work in the implementation. Ref: https://github.com/npm/cli/pull/2765 --- accepted/0000-workspaces-part2.md | 202 ++++++++++++++---------------- 1 file changed, 96 insertions(+), 106 deletions(-) diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md index 5015eaaa6..3ca7f001b 100644 --- a/accepted/0000-workspaces-part2.md +++ b/accepted/0000-workspaces-part2.md @@ -17,33 +17,103 @@ Following up from the original **npm workspaces** RFC are the following command- ## Rationale and Alternatives -The ability to run **npm commands** across defined **workspaces** is essential to successfully manage a **npm workspaces** workflow. That makes the following alternatives much less desireable: +The ability to run **npm commands** across defined **workspaces** is essential to successfully manage a **npm workspaces** workflow. -- Do not implement further support to manage running commands across **workspaces** -- Implement only _some_ of the implementation changes described in this document +Following are possible alternatives to provide this functionality: -**Note:** Alternative ways of executing each of the described features are defined along with their implementation details below. +- Do not implement further support to manage running commands across **workspaces**, keep the **npm workspaces** set of features to a minimum +- Defer to other tools from the ecosystem such as Lerna to solve the problem of running top-level commands across **workspaces** +- Implement the proposed features of this RFC as a standalone package / separated tool from the **npm cli** ## Implementation ### 1. Run commands across all child packages -Make **npm cli** subcommands **npm workspaces**-aware, so that running a command tries to run it within all **workspaces** as long as a **workspaces configuration** field is properly defined in `package.json`. +Create a new **npm cli** subcommand named **workspaces** (aliased to **ws**) that can route any of the supported subcommands to run in the context of the configured **workspaces** as long as a **workspaces configuration** field is properly defined in `package. -Only a subset of commands are to be supported: +We identified 5 different categories of subcommands based on how they're expected to work: -- `fund` List funding info for all **workspaces** -- `ls` List all packages including **workspaces** +#### 1. Commands that just need context from package.json + +Commands that, from an user point of view, are the equivalent of: `cd && npm `. + +- `docs` +- `diff` Package diff in the context of specific **workspaces** +- `dist-tag` List dist-tags for specific **workspaces** +- `exec` Run exec in the context of specific **workspaces** +- `init` Initialize a new **workspace** - `outdated` List outdated **dependencies** including **workspaces** and its **dependencies** -- `run-script` Run arbitrary **scripts** in all **workspaces**, skip any **workspace** that does not have a targetting **script** +- `pack` Run pack in the context of specific **workspaces** +- `publish` Run publish in the context of specific **workspaces** +- `run-script|restart|start|stop|test` Run arbitrary **scripts** in all **workspaces**, skip any **workspace** that does not have a targetting **script** +- `repo` - `rebuild` Rebuild all **workspaces** -- `restart` -- `start` -- `stop` -- `test` Run test **scripts** in all **workspaces** +- `set-script` - `update` Updates a **dependency** across the entire installation tree, including **workspaces** +- `unpublish` +- `version` Run version in the context of specific **workspaces** - `view` View registry info, also including **workspaces** +#### 2. Custom implementation + +##### 2.1. Reads from installed dependency tree + +General class of helper commands that load from the installed tree to produce some form of useful output. + +- `audit` +- `explain` +- `fund` List funding info for all **workspaces** +- `ls` List all packages including **workspaces** + +##### 2.2. Modify installed dependency tree + +The set of commands that will modify an install tree (from an implementer point of view, these are just [arborist.reify](https://github.com/npm/arborist/) proxies). + +- `ci` +- `dedupe|find-dupes` +- `install-ci-test` +- `install-test` +- `install` +- `link` +- `uninstall` +- `update` + +##### 2.3. Other + +A command that needs a special/custom workspace-aware implementation outside of the context of reading/writing to the install tree (using **Arborist**). + +#### 3. Unsupported + +This category of **npm cli** subcommand is completely unrelated to anything that the current working directory could affect. All registry helper/management types of commands fall into this category and it's a best UX to just exit with an error code in order to let the end user aware that trying to run these in the context of workspaces don't have any effect. + +- `adduser|login` +- `bin` +- `birthday` +- `cache` +- `completion` +- `config|get|set` +- `deprecate` +- `doctor` +- `edit` +- `explore` +- `help` +- `help-search` +- `hook` +- `logout` +- `org` +- `owner` +- `ping` +- `prefix` +- `profile` +- `search` +- `shrinkwrap` +- `star` +- `team` +- `token` +- `unstar` +- `whoami` +- `workspaces` + #### Test example: ``` @@ -68,25 +138,15 @@ npm ERR! Test failed. See above for more details. > done ``` -**NOTE:** `publish` and `version` are very special cases in a **npm workspaces** setup and are outside of the scope of this RFC. These features will live in userland in the form of popular modules such as [Lerna](https://lerna.js.org/) for the foreseeable future. - ### 2. Filter a subset of workspaces Filter is done via named argument (`--workspace`, short: `-w`) and here are some of the reasons why we decided to go that route: -- [Lerna filters](https://www.npmjs.com/package/@lerna/filter-options) were the starting point but early in the discussion some other considerations were brought forward -- npm configs do have some baggage, configs might apply to to other cli commands and can also possibly be defined project or system-wide - that said, there are things we can do to avoid these pitfalls (such as making sure `npm_config_workspace` env varibable never gets set in `run-script`) -- During the many discussions around this RFC a few strong points for named arguments stand out: - - Allow us to keep `npm workspace` for other command usages - - npm users are more familiar with usage of named configs, example: `npm test --workspace=foo` is very similar to `npm test --prefix=./packages/foo` - if you think on it in terms of "what workspace am I in" again, a named config makes more sense in npm - - [Organized a poll](https://github.com/npm/rfcs/issues/160) in which named arguments were the leading option, here are the results for reference: - ``` - 1. (12 votes) 21% - Positional argument with one-char alias: `npm workspace foo test` or `npm w foo test` - 2. (11 votes) 11% - Positional argument but shorter default name: `npm ws foo test` or `npm w foo test` - 3. (13 votes) 13% - Special character (operator-like): `npm :foo test` - 4. (20 votes) 35% - Named argument: `npm --workspace=foo test` or `npm -w=foo test` - ``` - -Given the results of all this preliminar work, the preferred way to run a command in the context of a workspace is to use the named `--workspace` argument or its `-w` short alias, e.g: +- [Lerna filters](https://www.npmjs.com/package/@lerna/filter-options) were the starting point but early in the discussion some other considerations were brought forward, specially the many pitfalls of supporting globs in the variety of supported shells and operational systems. +- npm configs do have some baggage, configs might apply to to other cli commands and can also possibly be defined project or system-wide so the most sure way to avoid these pitfalls is for configs to only be used in the context of a single **npm cli** subcommand. + +### 3. Examples + +Given the results of all this preliminar work, the preferred way to run a command in the context of a single workspace is to use the named `--workspace` argument or its `-w` short alias, e.g: In a project with the following structure: ``` @@ -102,84 +162,20 @@ In a project with the following structure: You can run tests for the `foo` workspace from the root of your project, with the following syntax: ``` -npm test --workspace=foo +npm ws test --workspace=foo ``` #### Install dependency example: -Add `tap` as a **dependency** of a **workspace** named `foo`: +Add `tap` as a **dependency** of all your configured **workspaces**: ```sh -npm install tap --workspace=foo +npm ws install tap ``` -Note: **Globs** are not supported as a valid `--workspace` argument value, the alternative is to use `npm --workspace=` in which the `alias` can be defined beforehand in your **workspace** configuration as detailed in the following up section. - -### 3. Grouping/Aliasing workspaces - -One proposed feature that was surfaced during our brainstorm sessions is the ability for users to create arbitrary groups of **workspaces** defined via **workspaces configuration**, e.g: - -Given a **npm workspace** setup with the following file structure: - -``` -/root -├── core -│ ├── foo -│ ├── bar -│ └── util -└── plugins - ├── helpers - ├── lorem - └── ipsum -``` - -It's possible to define a **groups** property within your **workspaces configuration** that will allow defining named sets of **workspaces** that can be later on used to run commands, e.g: - -``` -{ - "name": "workspace-example", - "version": "1.0.0", - "workspaces": { - "groups": { - "core": ["core/*"], // accepts globs - "plugins": ["lorem", "ipsum"], // also accepts workspaces names - "common": ["util", "helpers"] - }, - "packages": [ - "core/*", - "plugins/*" - ] - } -} -``` - -Following up with the previous configuration example, it would be possible to run tests across all **workspaces** from the `plugins` group, as following: - -``` -$ npm test --workspace=plugins - -> lorem@1.0.0 test /root/plugins/lorem -> done - -> ipsum@1.0.0 test /root/plugins/ipsum -> done -``` - -Further example, install a **peer dependency** across a group of packages named `core`: - -``` -$ npm install react@16 --save-peer --workspace=core -``` - -**TBD:** It might also be very handy being able to define groups as cli arguments, so that the following example will be equivalent to the previous examples: - -``` -$ npm workspace core install react@16 --save-peer --workspace-group core=core/* -``` - -This is a currently unresolved question since pitfalls of glob usage in cli arguments have already surfaced many times during the time of discussion for this RFC. +Note: **Globs** are not supported as a valid `--workspace` argument value, the proposed alternative is to use `npm ws --workspace=` in which `` is a folder containing multiple workspaces. ## Prior Art @@ -190,9 +186,10 @@ This is a currently unresolved question since pitfalls of glob usage in cli argu #### Filtering a subset of workspaces - [lerna filter-options](https://www.npmjs.com/package/@lerna/filter-options) -- [Yarn v1 workspace](https://classic.yarnpkg.com/en/docs/cli/workspace) +- [Yarn v1 workspace cmd](https://classic.yarnpkg.com/en/docs/cli/workspace) +- [Yarn v1 workspaces cmd](https://classic.yarnpkg.com/en/docs/cli/workspaces) - [Yarn v2 foreach include/exclude](https://yarnpkg.com/cli/workspaces/foreach) -- [pnpm recursive --filter](https://pnpm.js.org/en/cli/recursive#filter-lt-package_selector) +- [pnpm Filtering](https://pnpm.js.org/en/filtering) ## Dictionary @@ -212,10 +209,3 @@ During the discussions around this RFC it was brought up to our attention that a - **[globs](https://en.wikipedia.org/wiki/Glob_(programming))**: String patterns that specifies sets of filenames with special characters. - **[Arborist](https://github.com/npm/arborist)**: The npm@7 install library - **hoisting packages**: Bringing packages up a level in the context of an installation tree. - -## Unresolved Questions and Bikeshedding - -- Groups as a cli argument: `--workspace-group` - - how to define the `key=value` nature of it? - - is there any arg currently in the cli using key/value pairs? - - maybe it's not worth having it? From 0c90e3f7dcf890958dc117787b737412b634a45b Mon Sep 17 00:00:00 2001 From: Ruy Adorno Date: Wed, 24 Feb 2021 17:56:19 -0500 Subject: [PATCH 6/9] fixup! Update RFC with latest proposed ideas --- accepted/0000-workspaces-part2.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md index 3ca7f001b..bacb540a1 100644 --- a/accepted/0000-workspaces-part2.md +++ b/accepted/0000-workspaces-part2.md @@ -114,7 +114,9 @@ This category of **npm cli** subcommand is completely unrelated to anything that - `whoami` - `workspaces` -#### Test example: +#### Example: + +Running tests across all configured **workspaces**: ``` ├── package.json { "name": "foo", "workspaces": ["dep-a", "dep-b"] } @@ -123,7 +125,7 @@ This category of **npm cli** subcommand is completely unrelated to anything that └── dep-b └── package.json { "version": "1.3.1" } -$ npm test +$ npm ws test > foo@1.0.0 test /Users/username/foo > echo "Error: no test specified" && exit 1 From 8167fa97a266cc2bf60cc545b382d274dc397496 Mon Sep 17 00:00:00 2001 From: Ruy Adorno Date: Thu, 25 Feb 2021 18:45:25 -0500 Subject: [PATCH 7/9] recategorize some of the cmds --- accepted/0000-workspaces-part2.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md index bacb540a1..b3fea08f0 100644 --- a/accepted/0000-workspaces-part2.md +++ b/accepted/0000-workspaces-part2.md @@ -38,18 +38,13 @@ We identified 5 different categories of subcommands based on how they're expecte Commands that, from an user point of view, are the equivalent of: `cd && npm `. - `docs` +- `doctor` - `diff` Package diff in the context of specific **workspaces** - `dist-tag` List dist-tags for specific **workspaces** -- `exec` Run exec in the context of specific **workspaces** -- `init` Initialize a new **workspace** -- `outdated` List outdated **dependencies** including **workspaces** and its **dependencies** - `pack` Run pack in the context of specific **workspaces** - `publish` Run publish in the context of specific **workspaces** -- `run-script|restart|start|stop|test` Run arbitrary **scripts** in all **workspaces**, skip any **workspace** that does not have a targetting **script** - `repo` -- `rebuild` Rebuild all **workspaces** - `set-script` -- `update` Updates a **dependency** across the entire installation tree, including **workspaces** - `unpublish` - `version` Run version in the context of specific **workspaces** - `view` View registry info, also including **workspaces** @@ -64,6 +59,7 @@ General class of helper commands that load from the installed tree to produce so - `explain` - `fund` List funding info for all **workspaces** - `ls` List all packages including **workspaces** +- `outdated` List outdated **dependencies** including **workspaces** and its **dependencies** ##### 2.2. Modify installed dependency tree @@ -75,6 +71,8 @@ The set of commands that will modify an install tree (from an implementer point - `install-test` - `install` - `link` +- `rebuild` Rebuild all **workspaces** +- `update` Updates a **dependency** across the entire installation tree, including **workspaces** - `uninstall` - `update` @@ -82,6 +80,10 @@ The set of commands that will modify an install tree (from an implementer point A command that needs a special/custom workspace-aware implementation outside of the context of reading/writing to the install tree (using **Arborist**). +- `exec` Run exec in the context of specific **workspaces** +- `init` Initialize a new **workspace** +- `run-script|restart|start|stop|test` Run arbitrary **scripts** in all **workspaces**, skip any **workspace** that does not have a targetting **script** + #### 3. Unsupported This category of **npm cli** subcommand is completely unrelated to anything that the current working directory could affect. All registry helper/management types of commands fall into this category and it's a best UX to just exit with an error code in order to let the end user aware that trying to run these in the context of workspaces don't have any effect. @@ -93,7 +95,6 @@ This category of **npm cli** subcommand is completely unrelated to anything that - `completion` - `config|get|set` - `deprecate` -- `doctor` - `edit` - `explore` - `help` From 4ab99e9463e7ead20d8f0bbaf47da1a2c8bb7255 Mon Sep 17 00:00:00 2001 From: Ruy Adorno Date: Wed, 17 Mar 2021 15:07:35 -0400 Subject: [PATCH 8/9] move npm version to custom section --- accepted/0000-workspaces-part2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md index b3fea08f0..c3e353f06 100644 --- a/accepted/0000-workspaces-part2.md +++ b/accepted/0000-workspaces-part2.md @@ -46,7 +46,6 @@ Commands that, from an user point of view, are the equivalent of: `cd Date: Fri, 30 Apr 2021 02:44:18 -0400 Subject: [PATCH 9/9] chore: update RFC based on pairing session w/ @ruyadorno --- accepted/0000-workspaces-part2.md | 94 ++++++++++++------------------- 1 file changed, 37 insertions(+), 57 deletions(-) diff --git a/accepted/0000-workspaces-part2.md b/accepted/0000-workspaces-part2.md index c3e353f06..8b95a5f01 100644 --- a/accepted/0000-workspaces-part2.md +++ b/accepted/0000-workspaces-part2.md @@ -1,52 +1,52 @@ -# npm workspaces: Running commands +# npm Workspaces: Running commands ## Summary -Introduces basic support to running **npm commands** across nested **workspaces**. +Introduces basic support for running **npm commands** across a project's **workspaces**. ## Motivation -The need for running **npm commands** across nested **workspaces** was sourced from the community during the elaboration of the [original **npm workspaces** RFC](https://github.com/npm/rfcs/pull/103). +The need for running **npm commands** across a project's **workspaces** was surfaced by the community during the discourse spun out of the [initial **npm workspaces** RFC](https://github.com/npm/rfcs/pull/103). ## Detailed Explanation -Following up from the original **npm workspaces** RFC are the following command-related changes: +Following up on the original **npm workspaces** RFC are the following command-related additions/changes: -- A subset of the standard **npm commands** needs to be made aware of **npm workspaces** +- A subset of the standard **npm commands** need to be made aware of **npm workspaces** - It's desired to have a standard way to filter out **workspaces** in which to run specific **npm commands** ## Rationale and Alternatives -The ability to run **npm commands** across defined **workspaces** is essential to successfully manage a **npm workspaces** workflow. +The ability to run **npm commands** across a project's **workspaces** is essential to effecient management of, & enabling, complex developer workflows. -Following are possible alternatives to provide this functionality: +The following are possible alternatives to unlock this functionality: - Do not implement further support to manage running commands across **workspaces**, keep the **npm workspaces** set of features to a minimum -- Defer to other tools from the ecosystem such as Lerna to solve the problem of running top-level commands across **workspaces** -- Implement the proposed features of this RFC as a standalone package / separated tool from the **npm cli** +- Defer to other tools from the ecosystem - such as **Lerna** - to solve the problem of running top-level commands across **workspaces** +- Implement the proposed features of this RFC as a standalone package apart from the **npm cli** ## Implementation -### 1. Run commands across all child packages +### 1. Run commands across all child workspaces -Create a new **npm cli** subcommand named **workspaces** (aliased to **ws**) that can route any of the supported subcommands to run in the context of the configured **workspaces** as long as a **workspaces configuration** field is properly defined in `package. +Create a new **npm cli** config, `--workspaces` (aliased to `--ws`), that can route any of the supported subcommands to run in the context of the configured **workspaces** as long as a **workspaces configuration** field is properly defined in `package.json` -We identified 5 different categories of subcommands based on how they're expected to work: +There are currently five distinct categories of _"workspace-awareness"_ that existing subcommands will belong to: -#### 1. Commands that just need context from package.json +#### 1. Commands that _just_ need context from `package.json` -Commands that, from an user point of view, are the equivalent of: `cd && npm `. +Commands that, from a user's point of view, are the equivalent of: `cd && npm `. - `docs` - `doctor` -- `diff` Package diff in the context of specific **workspaces** -- `dist-tag` List dist-tags for specific **workspaces** -- `pack` Run pack in the context of specific **workspaces** -- `publish` Run publish in the context of specific **workspaces** +- `diff` +- `dist-tag` +- `pack` +- `publish` - `repo` - `set-script` - `unpublish` -- `view` View registry info, also including **workspaces** +- `view` #### 2. Custom implementation @@ -56,13 +56,13 @@ General class of helper commands that load from the installed tree to produce so - `audit` - `explain` -- `fund` List funding info for all **workspaces** -- `ls` List all packages including **workspaces** -- `outdated` List outdated **dependencies** including **workspaces** and its **dependencies** +- `fund` +- `ls` +- `outdated` ##### 2.2. Modify installed dependency tree -The set of commands that will modify an install tree (from an implementer point of view, these are just [arborist.reify](https://github.com/npm/arborist/) proxies). +The set of commands that will modify an install tree (from an implementation stand-point, these are just [arborist.reify](https://github.com/npm/arborist/) proxies). - `ci` - `dedupe|find-dupes` @@ -70,23 +70,22 @@ The set of commands that will modify an install tree (from an implementer point - `install-test` - `install` - `link` -- `rebuild` Rebuild all **workspaces** -- `update` Updates a **dependency** across the entire installation tree, including **workspaces** -- `uninstall` +- `rebuild` - `update` +- `uninstall` ##### 2.3. Other -A command that needs a special/custom workspace-aware implementation outside of the context of reading/writing to the install tree (using **Arborist**). +Commands that need a special/custom _"workspace-aware"_ implementation outside of the context of reading/writing to the install tree. -- `exec` Run exec in the context of specific **workspaces** -- `init` Initialize a new **workspace** -- `run-script|restart|start|stop|test` Run arbitrary **scripts** in all **workspaces**, skip any **workspace** that does not have a targetting **script** -- `version` Run version in the context of specific **workspaces**, needs tweaked commit msg, tag +- `exec` +- `init` +- `run-script|restart|start|stop|test` +- `version` #### 3. Unsupported -This category of **npm cli** subcommand is completely unrelated to anything that the current working directory could affect. All registry helper/management types of commands fall into this category and it's a best UX to just exit with an error code in order to let the end user aware that trying to run these in the context of workspaces don't have any effect. +This category of **npm cli** commands is completely unrelated to anything that the current working directory could affect. This includes a number of registry-specific helper/management operations which don't make sense/have no effect within the context of a workspace. Trying to run these commands with a workspace flag/config will exit with a descriptive error code. - `adduser|login` - `bin` @@ -126,7 +125,7 @@ Running tests across all configured **workspaces**: └── dep-b └── package.json { "version": "1.3.1" } -$ npm ws test +$ npm test --workspaces > foo@1.0.0 test /Users/username/foo > echo "Error: no test specified" && exit 1 @@ -145,11 +144,10 @@ npm ERR! Test failed. See above for more details. Filter is done via named argument (`--workspace`, short: `-w`) and here are some of the reasons why we decided to go that route: - [Lerna filters](https://www.npmjs.com/package/@lerna/filter-options) were the starting point but early in the discussion some other considerations were brought forward, specially the many pitfalls of supporting globs in the variety of supported shells and operational systems. -- npm configs do have some baggage, configs might apply to to other cli commands and can also possibly be defined project or system-wide so the most sure way to avoid these pitfalls is for configs to only be used in the context of a single **npm cli** subcommand. ### 3. Examples -Given the results of all this preliminar work, the preferred way to run a command in the context of a single workspace is to use the named `--workspace` argument or its `-w` short alias, e.g: +Given the results of all this preliminar investigation, the preferred way to run a command in the context of a single workspace is to use the named `--workspace` argument or its `-w` short alias, e.g: In a project with the following structure: ``` @@ -165,20 +163,20 @@ In a project with the following structure: You can run tests for the `foo` workspace from the root of your project, with the following syntax: ``` -npm ws test --workspace=foo +npm test --workspace=foo ``` #### Install dependency example: -Add `tap` as a **dependency** of all your configured **workspaces**: +Add `tap` as a **dependency** for all of your configured **workspaces**: ```sh -npm ws install tap +npm install tap --workspaces ``` -Note: **Globs** are not supported as a valid `--workspace` argument value, the proposed alternative is to use `npm ws --workspace=` in which `` is a folder containing multiple workspaces. +> Note: **Globs** are not supported as a valid `--workspace` argument value, the proposed alternative is to use `npm --workspace=` in which `` is a folder containing multiple workspaces. ## Prior Art @@ -194,21 +192,3 @@ Note: **Globs** are not supported as a valid `--workspace` argument value, the p - [Yarn v2 foreach include/exclude](https://yarnpkg.com/cli/workspaces/foreach) - [pnpm Filtering](https://pnpm.js.org/en/filtering) -## Dictionary - -During the discussions around this RFC it was brought up to our attention that a lot of the vocabulary surrounding what the larger JavaScript community understands as "workspaces" can be confusing, for the sake of keeping the discussion as productive as possible we're taking the extra step of documenting what each of the terms used here means: - -- **npm cli**: The [npm cli](https://github.com/npm/cli/) :wink: -- **npm commands**: The set of **npm cli** [commands](https://docs.npmjs.com/cli-documentation/) -- **npm workspaces**: The feature name, meaning the ability to the **npm cli** to support a better workflow for working with multiple packages. -- **workspaces**: A set of **workspace**s. -- **workspace**: A nested package within the **Top-level workspace** file system that is explicitly defined as such via **workspaces configuration**. -- **Top-level workspace**: The root level package that contains a **workspaces configuration** defining **workspaces**. -- **workspaces configuration**: The blob of json configuration defined within `package.json` that declares where to find **workspaces** for this **Top-level workspace** package. -- **dependency**: A package that is depended upon by another given package. -- **[peer dependency](https://docs.npmjs.com/files/package.json#peerdependencies)**: A special **dependency** relationship between packages. -- **dependent**: A package which depends on another given package. -- **symlink**: A [symbolic link](https://en.wikipedia.org/wiki/Symbolic_link) between files. -- **[globs](https://en.wikipedia.org/wiki/Glob_(programming))**: String patterns that specifies sets of filenames with special characters. -- **[Arborist](https://github.com/npm/arborist)**: The npm@7 install library -- **hoisting packages**: Bringing packages up a level in the context of an installation tree.