-
Notifications
You must be signed in to change notification settings - Fork 1.6k
start #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
start #1
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # runtime dependencies are checked in | ||
| # dev dependencies are *not* checked in | ||
| node_modules/.bin | ||
| node_modules/typescript | ||
| node_modules/@types | ||
| node_modules/prettier | ||
| __tests__/runner/* | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "printWidth": 80, | ||
| "tabWidth": 2, | ||
| "useTabs": false, | ||
| "semi": true, | ||
| "singleQuote": true, | ||
| "trailingComma": "none", | ||
| "bracketSpacing": false, | ||
| "arrowParens": "avoid", | ||
| "parser": "typescript" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| import installer = require('../src/installer'); | ||
| import io = require('@actions/io'); | ||
| import fs = require('fs'); | ||
| import os = require('os'); | ||
| import path = require('path'); | ||
|
|
||
| const toolDir = path.join(__dirname, 'runner', 'tools'); | ||
|
|
||
| const tempDir = path.join(__dirname, 'runner', 'temp'); | ||
|
|
||
| describe('installer tests', () => { | ||
| beforeAll(() => {}); | ||
| beforeAll(async () => { | ||
| // TODO - these should eventually be changed to match new method of loading dir | ||
| process.env['Runner.ToolsDirectory'] = toolDir; | ||
| process.env['Runner.TempDirectory'] = tempDir; | ||
| await io.rmRF(toolDir); | ||
| await io.rmRF(tempDir); | ||
| }); | ||
|
|
||
| it('Acquires version of node if no matching version is installed', async () => { | ||
| await installer.getNode('10.16.0'); | ||
| const nodeDir = path.join(toolDir, 'node', '10.16.0', os.arch()); | ||
|
|
||
| expect(fs.existsSync(`${nodeDir}.complete`)).toBe(true); | ||
| expect(fs.existsSync(path.join(nodeDir, 'node.exe'))).toBe(true); | ||
| }, 100000); | ||
|
|
||
| if (process.platform === 'win32') { | ||
| it('Falls back to backup location if first one doesnt contain correct version', async () => { | ||
| await installer.getNode('5.10.1'); | ||
| const nodeDir = path.join(toolDir, 'node', '5.10.1', os.arch()); | ||
|
|
||
| expect(fs.existsSync(`${nodeDir}.complete`)).toBe(true); | ||
| expect(fs.existsSync(path.join(nodeDir, 'node.exe'))).toBe(true); | ||
| }, 100000); | ||
|
|
||
| it('Falls back to third location if second one doesnt contain correct version', async () => { | ||
| await installer.getNode('0.12.18'); | ||
| const nodeDir = path.join(toolDir, 'node', '0.12.18', os.arch()); | ||
|
|
||
| expect(fs.existsSync(`${nodeDir}.complete`)).toBe(true); | ||
| expect(fs.existsSync(path.join(nodeDir, 'node.exe'))).toBe(true); | ||
| }, 100000); | ||
| } | ||
|
|
||
| it('Throws if no location contains correct node version', async () => { | ||
| let thrown = false; | ||
| try { | ||
| await installer.getNode('1000'); | ||
| } catch { | ||
| thrown = true; | ||
| } | ||
| expect(thrown).toBe(true); | ||
| }); | ||
|
|
||
| it('Acquires version of node with long paths', async () => { | ||
| const toolpath = await installer.getNode('8.8.1'); | ||
| const nodeDir = path.join(toolDir, 'node', '8.8.1', os.arch()); | ||
|
|
||
| expect(fs.existsSync(`${nodeDir}.complete`)).toBe(true); | ||
| expect(fs.existsSync(path.join(nodeDir, 'node.exe'))).toBe(true); | ||
| }, 100000); | ||
|
|
||
| it('Uses version of node installed in cache', async () => { | ||
| const nodeDir: string = path.join(toolDir, '250.0.0', os.arch()); | ||
| await io.mkdirP(nodeDir); | ||
| fs.writeFileSync(`${nodeDir}.complete`, 'hello'); | ||
| // This will throw if it doesn't find it in the cache (because no such version exists) | ||
| await installer.getNode('250.0.0'); | ||
| return; | ||
| }); | ||
|
|
||
| it('Doesnt use version of node that was only partially installed in cache', async () => { | ||
| const nodeDir: string = path.join(toolDir, '250.0.0', os.arch()); | ||
| await io.mkdirP(nodeDir); | ||
| let thrown = false; | ||
| try { | ||
| // This will throw if it doesn't find it in the cache (because no such version exists) | ||
| await installer.getNode('251.0.0'); | ||
| } catch { | ||
| thrown = true; | ||
| } | ||
| expect(thrown).toBe(true); | ||
| return; | ||
| }); | ||
|
|
||
| it('Resolves semantic versions of node installed in cache', async () => { | ||
| const nodeDir: string = path.join(toolDir, '250.0.0', os.arch()); | ||
| await io.mkdirP(nodeDir); | ||
| fs.writeFileSync(`${nodeDir}.complete`, 'hello'); | ||
| // These will throw if it doesn't find it in the cache (because no such version exists) | ||
| await installer.getNode('250.0.0'); | ||
| await installer.getNode('250'); | ||
| await installer.getNode('250.0'); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Contributors | ||
|
|
||
|
|
||
| # Checkin | ||
|
|
||
| - Do checkin source (src) | ||
| - Do checkin build output (lib) | ||
| - Do checkin runtime node_modules | ||
| - Do not checkin | ||
|
|
||
| # Adding a dev dependency | ||
|
|
||
| Remember to update .gitignore. | ||
|
|
||
| # Updating toolkit dependency | ||
|
|
||
| Until released publically, update tgz packages in toolkit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| module.exports = { | ||
| clearMocks: true, | ||
| moduleFileExtensions: ['js', 'ts'], | ||
| testEnvironment: 'node', | ||
| testMatch: ['**/*.test.ts'], | ||
| testRunner: 'jest-circus/runner', | ||
| transform: { | ||
| '^.+\\.ts$': 'ts-jest' | ||
| }, | ||
| verbose: true | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,209 @@ | ||
| "use strict"; | ||
| var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) { | ||
| return new (P || (P = Promise))(function (resolve, reject) { | ||
| function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } } | ||
| function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } } | ||
| function step(result) { result.done ? resolve(result.value) : new P(function (resolve) { resolve(result.value); }).then(fulfilled, rejected); } | ||
| step((generator = generator.apply(thisArg, _arguments || [])).next()); | ||
| }); | ||
| }; | ||
| var __importStar = (this && this.__importStar) || function (mod) { | ||
| if (mod && mod.__esModule) return mod; | ||
| var result = {}; | ||
| if (mod != null) for (var k in mod) if (Object.hasOwnProperty.call(mod, k)) result[k] = mod[k]; | ||
| result["default"] = mod; | ||
| return result; | ||
| }; | ||
| Object.defineProperty(exports, "__esModule", { value: true }); | ||
| const core = __importStar(require("@actions/core")); | ||
| const io = __importStar(require("@actions/io")); | ||
| const tc = __importStar(require("@actions/tool-cache")); | ||
| const restm = __importStar(require("typed-rest-client/RestClient")); | ||
| const os = __importStar(require("os")); | ||
| const path = __importStar(require("path")); | ||
| const semver = __importStar(require("semver")); | ||
| let osPlat = os.platform(); | ||
| let osArch = os.arch(); | ||
| function getNode(versionSpec) { | ||
| return __awaiter(this, void 0, void 0, function* () { | ||
| // check cache | ||
| let toolPath; | ||
| toolPath = tc.find('node', versionSpec); | ||
| // If not found in cache, download | ||
| if (!toolPath) { | ||
| let version; | ||
| const c = semver.clean(versionSpec) || ''; | ||
| // If explicit version | ||
| if (semver.valid(c) != null) { | ||
| // version to download | ||
| version = versionSpec; | ||
| } | ||
| else { | ||
| // query nodejs.org for a matching version | ||
| version = yield queryLatestMatch(versionSpec); | ||
| if (!version) { | ||
| throw new Error(`Unable to find Node version '${versionSpec}' for platform ${osPlat} and architecture ${osArch}.`); | ||
| } | ||
| // check cache | ||
| toolPath = tc.find('node', version); | ||
| } | ||
| if (!toolPath) { | ||
| // download, extract, cache | ||
| toolPath = yield acquireNode(version); | ||
| } | ||
| } | ||
| // | ||
| // a tool installer initimately knows details about the layout of that tool | ||
| // for example, node binary is in the bin folder after the extract on Mac/Linux. | ||
| // layouts could change by version, by platform etc... but that's the tool installers job | ||
| // | ||
| if (osPlat != 'win32') { | ||
| toolPath = path.join(toolPath, 'bin'); | ||
| } | ||
| // | ||
| // prepend the tools path. instructs the agent to prepend for future tasks | ||
| core.addPath(toolPath); | ||
| }); | ||
| } | ||
| exports.getNode = getNode; | ||
| function queryLatestMatch(versionSpec) { | ||
| return __awaiter(this, void 0, void 0, function* () { | ||
| // node offers a json list of versions | ||
| let dataFileName; | ||
| switch (osPlat) { | ||
| case 'linux': | ||
| dataFileName = 'linux-' + osArch; | ||
| break; | ||
| case 'darwin': | ||
| dataFileName = 'osx-' + osArch + '-tar'; | ||
| break; | ||
| case 'win32': | ||
| dataFileName = 'win-' + osArch + '-exe'; | ||
| break; | ||
| default: | ||
| throw new Error(`Unexpected OS '${osPlat}'`); | ||
| } | ||
| let versions = []; | ||
| let dataUrl = 'https://nodejs.org/dist/index.json'; | ||
| let rest = new restm.RestClient('vsts-node-tool'); | ||
| let nodeVersions = (yield rest.get(dataUrl)).result || []; | ||
| nodeVersions.forEach((nodeVersion) => { | ||
| // ensure this version supports your os and platform | ||
| if (nodeVersion.files.indexOf(dataFileName) >= 0) { | ||
| versions.push(nodeVersion.version); | ||
| } | ||
| }); | ||
| // get the latest version that matches the version spec | ||
| let version = evaluateVersions(versions, versionSpec); | ||
| return version; | ||
| }); | ||
| } | ||
| // TODO - should we just export this from @actions/tool-cache? Lifted directly from there | ||
| function evaluateVersions(versions, versionSpec) { | ||
| let version = ''; | ||
| core.debug(`evaluating ${versions.length} versions`); | ||
| versions = versions.sort((a, b) => { | ||
| if (semver.gt(a, b)) { | ||
| return 1; | ||
| } | ||
| return -1; | ||
| }); | ||
| for (let i = versions.length - 1; i >= 0; i--) { | ||
| const potential = versions[i]; | ||
| const satisfied = semver.satisfies(potential, versionSpec); | ||
| if (satisfied) { | ||
| version = potential; | ||
| break; | ||
| } | ||
| } | ||
| if (version) { | ||
| core.debug(`matched: ${version}`); | ||
| } | ||
| else { | ||
| core.debug('match not found'); | ||
| } | ||
| return version; | ||
| } | ||
| function acquireNode(version) { | ||
| return __awaiter(this, void 0, void 0, function* () { | ||
| // | ||
| // Download - a tool installer intimately knows how to get the tool (and construct urls) | ||
| // | ||
| version = semver.clean(version) || ''; | ||
| let fileName = osPlat == 'win32' | ||
| ? 'node-v' + version + '-win-' + os.arch() | ||
| : 'node-v' + version + '-' + osPlat + '-' + os.arch(); | ||
| let urlFileName = osPlat == 'win32' ? fileName + '.7z' : fileName + '.tar.gz'; | ||
| let downloadUrl = 'https://nodejs.org/dist/v' + version + '/' + urlFileName; | ||
| let downloadPath; | ||
| try { | ||
| downloadPath = yield tc.downloadTool(downloadUrl); | ||
| } | ||
| catch (err) { | ||
| if (err['httpStatusCode'] && err['httpStatusCode'] === '404') { | ||
| return yield acquireNodeFromFallbackLocation(version); | ||
| } | ||
| throw err; | ||
| } | ||
| // | ||
| // Extract | ||
| // | ||
| let extPath; | ||
| if (osPlat == 'win32') { | ||
| let _7zPath = path.join(__dirname, '7zr.exe'); | ||
| extPath = yield tc.extract7z(downloadPath); | ||
| } | ||
| else { | ||
| extPath = yield tc.extractTar(downloadPath); | ||
| } | ||
| // | ||
| // Install into the local tool cache - node extracts with a root folder that matches the fileName downloaded | ||
| // | ||
| let toolRoot = path.join(extPath, fileName); | ||
| return yield tc.cacheDir(toolRoot, 'node', version); | ||
| }); | ||
| } | ||
| // For non LTS versions of Node, the files we need (for Windows) are sometimes located | ||
| // in a different folder than they normally are for other versions. | ||
| // Normally the format is similar to: https://nodejs.org/dist/v5.10.1/node-v5.10.1-win-x64.7z | ||
| // In this case, there will be two files located at: | ||
| // /dist/v5.10.1/win-x64/node.exe | ||
| // /dist/v5.10.1/win-x64/node.lib | ||
| // If this is not the structure, there may also be two files located at: | ||
| // /dist/v0.12.18/node.exe | ||
| // /dist/v0.12.18/node.lib | ||
| // This method attempts to download and cache the resources from these alternative locations. | ||
| // Note also that the files are normally zipped but in this case they are just an exe | ||
| // and lib file in a folder, not zipped. | ||
| function acquireNodeFromFallbackLocation(version) { | ||
| return __awaiter(this, void 0, void 0, function* () { | ||
| // Create temporary folder to download in to | ||
| let tempDownloadFolder = 'temp_' + Math.floor(Math.random() * 2000000000); | ||
| let tempDir = path.join(__dirname, tempDownloadFolder); | ||
| yield io.mkdirP(tempDir); | ||
| let exeUrl; | ||
| let libUrl; | ||
| try { | ||
| exeUrl = `https://nodejs.org/dist/v${version}/win-${os.arch()}/node.exe`; | ||
| libUrl = `https://nodejs.org/dist/v${version}/win-${os.arch()}/node.lib`; | ||
| const exePath = yield tc.downloadTool(exeUrl); | ||
| yield io.mv(exePath, path.join(tempDir, 'node.exe')); | ||
| const libPath = yield tc.downloadTool(libUrl); | ||
| yield io.mv(libPath, path.join(tempDir, 'node.lib')); | ||
| } | ||
| catch (err) { | ||
| if (err['httpStatusCode'] && err['httpStatusCode'] === '404') { | ||
| exeUrl = `https://nodejs.org/dist/v${version}/node.exe`; | ||
| libUrl = `https://nodejs.org/dist/v${version}/node.lib`; | ||
| const exePath = yield tc.downloadTool(exeUrl); | ||
| yield io.mv(exePath, path.join(tempDir, 'node.exe')); | ||
| const libPath = yield tc.downloadTool(libUrl); | ||
| yield io.mv(libPath, path.join(tempDir, 'node.lib')); | ||
| } | ||
| else { | ||
| throw err; | ||
| } | ||
| } | ||
| return yield tc.cacheDir(tempDir, 'node', version); | ||
| }); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't have to be part of this initial PR, but maybe we should just webpack as part of our build process. Then we can just ignore node modules
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note also - this also doesn't really work to not check in dev dependencies. While this doesn't check in the top level dev-deps, nested dependencies will still get checked in. For example, Prettier alone has ~60 dependencies all of which get checked in (not to mention the dependencies of those dependencies).
I think webpack is probably the best path forward here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we doing this to avoid having to
npm installthe production dependencies of this tool? If so, there are a few options for vendoring in dependencies that we already use at GitHub. If you can give me a sense of the goal with this, I can advise.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The runner will run the action from the github graph at runtime per the execution model. That is, it will get the targz of that ref and run it.
Dependencies are vendored in as you noted.
@damccorm - regarding dev dependencies, you can use
npm prunebefore you publish (push).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh cool - didn't know about
npm prune. It looks like that handles all devDependencies though, so we probably don't need those listed in gitignore. It also might be good to enforce that with something like huskyThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See #5 proposing using husky