Skip to content

tests(snyk): assert upper bounds #9308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
brendankenny merged 9 commits into from
Jul 20, 2019
Merged

tests(snyk): assert upper bounds #9308

Changes from 3 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
eafb5f9
deps(snyk): do not allow * in semver
connorjclark Jun 28, 2019
a24a38e
assert upper bounds
connorjclark Jun 28, 2019
a006050
update test name
connorjclark Jun 28, 2019
ba9bc7c
add comment
connorjclark Jul 8, 2019
e01212b
Update lighthouse-core/test/audits/dobetterweb/no-vulnerable-librarie…
connorjclark Jul 9, 2019
6ec1ae1
semver.Range
connorjclark Jul 9, 2019
441d4f8
merge
connorjclark Jul 9, 2019
55bf357
comments
connorjclark Jul 9, 2019
4653ba1
update
connorjclark Jul 10, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions lighthouse-core/test/audits/dobetterweb/no-vulnerable-libraries-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,20 @@ describe('Avoids front-end JavaScript libraries with known vulnerabilities', ()
assert.equal(auditResult.score, 1);
});
});

describe('Every snyk vulnerability has an upperbound', () => {
for (const vulns of Object.values(NoVulnerableLibrariesAudit.snykDB.npm)) {
for (const vuln of vulns) {
for (const semver of vuln.semver.vulnerable) {
assert.notEqual(semver, '*', 'invalid semver: * is not allowed');

const clauses = semver.split('||');
for (const clause of clauses) {
if (!clause.trim().startsWith('=') && !clause.includes('<')) {
Copy link
Collaborator

@patrickhulce patrickhulce Jul 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe give a few examples here for easier parsing? I had to go checkout the snapshot to see why it's includes('<') but .trim().startsWith('=')

Copy link
Contributor

@brendankenny brendankenny Jul 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to go checkout the snapshot to see why it's includes('<') but .trim().startsWith('=')

hmm, I still don't understand why that's the case :).

Isn't something like '=5.0 >=10' valid with no upper bound? But it wouldn't fail this check. Or '<5.0 >=10'? Maybe I just don't understand what we're checking here :)

Copy link
Collaborator

@patrickhulce patrickhulce Jul 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well those aren't real ranges :)

I suppose it's a meta question of if we should be checking to see that they're valid ranges at all

Copy link
Contributor

@brendankenny brendankenny Jul 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well those aren't real ranges :)

why aren't they? Isn't '=5.0 >=10' just saying 5.x or anything over 10 is vulnerable?

Copy link
Collaborator Author

@connorjclark connorjclark Jul 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well those aren't real ranges :)

this

valid ranges at all

doesn't make sense to validate snyk like that imo

Copy link
Collaborator

@patrickhulce patrickhulce Jul 9, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a space is an AND

OR is either || or a separate entry in the vulnerable array

so while it might technically be valid, nothing will ever satisfy the range '=5 >=10'

Copy link
Collaborator Author

@connorjclark connorjclark Jul 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

semver package's validRange just verifies the string conforms to the grammar defined here: https://github.com/npm/node-semver/blob/ce6190e2b681700dcc5d7309fe8eda99941f712d/range.bnf not that the range is satisfiable.

Copy link
Contributor

@brendankenny brendankenny Jul 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so while it might technically be valid, nothing will ever satisfy the range '=5 >=10'

Ah, I see your point, but it could also be '=11 >= 10'. I assume most of these are auto generated, so it doesn't seem out of the realm of possibility. See my other comment for one possibility rather than getting clever and hoping these ranges always make sense :)

Copy link
Contributor

@brendankenny brendankenny Jul 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see your point, but it could also be '=11 >= 10'

oh, no, wait, that's wrong. So I'll have to fall back to hand waving about invalid ranges :)

Copy link
Collaborator Author

@connorjclark connorjclark Jul 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't make sense to validate snyk like that imo

didnt realize how easy it'd be to use semver to validate the ranges, sooo nvm. it's done indirectly with this test now.

assert.fail(`invalid semver: ${semver}. Must contain an upper bound`);
}
}
}
}
}
});