Skip to content

Improve the robustness of creating and uploading debug artifacts #2486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
henrymercer merged 5 commits into from
Sep 17, 2024
Merged

Improve the robustness of creating and uploading debug artifacts #2486

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d061f2c
Handle CLI errors when creating debug artifacts
henrymercer Sep 16, 2024
80d7a6c
Tolerate failures in uploading debug artifacts
henrymercer Sep 16, 2024
bbd7c80
Fall back to partial database bundle if CLI command fails
henrymercer Sep 16, 2024
dd7307d
Refactoring: Simplify retrieving error message
henrymercer Sep 16, 2024
213bf36
Improve documentation
henrymercer Sep 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Tolerate failures in uploading debug artifacts
  • Loading branch information
@henrymercer
henrymercer committed Sep 16, 2024
commit 80d7a6c8d4c6293bdc9a186d504787a94725a7d8
41 changes: 23 additions & 18 deletions lib/debug-artifacts.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/debug-artifacts.js.map
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

58 changes: 33 additions & 25 deletions src/debug-artifacts.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,9 @@ function tryGetSarifResultPath(
}
} catch (e) {
logger.warning(
`Failed to find SARIF results path for ${language}. ${wrapError(e)}`,
`Failed to find SARIF results path for ${language}. ${
wrapError(e).message
}`,
);
}
return [];
Expand All @@ -113,7 +115,7 @@ async function tryBundleDatabase(
}
} catch (e) {
logger.warning(
`Failed to bundle database for ${language}. ${wrapError(e)}`,
`Failed to bundle database for ${language}. ${wrapError(e).message}`,
);
return [];
}
Expand All @@ -123,36 +125,42 @@ export async function uploadAllAvailableDebugArtifacts(
config: Config,
logger: Logger,
) {
const filesToUpload: string[] = [];
try {
const filesToUpload: string[] = [];

for (const language of config.languages) {
filesToUpload.push(...tryGetSarifResultPath(config, language, logger));

for (const language of config.languages) {
filesToUpload.push(...tryGetSarifResultPath(config, language, logger));
// Add any log files
const databaseDirectory = getCodeQLDatabasePath(config, language);
const logsDirectory = path.resolve(databaseDirectory, "log");
if (doesDirectoryExist(logsDirectory)) {
filesToUpload.push(...listFolder(logsDirectory));
}

// Add any log files
const databaseDirectory = getCodeQLDatabasePath(config, language);
const logsDirectory = path.resolve(databaseDirectory, "log");
if (doesDirectoryExist(logsDirectory)) {
filesToUpload.push(...listFolder(logsDirectory));
// Multilanguage tracing: there are additional logs in the root of the cluster
const multiLanguageTracingLogsDirectory = path.resolve(
config.dbLocation,
"log",
);
if (doesDirectoryExist(multiLanguageTracingLogsDirectory)) {
filesToUpload.push(...listFolder(multiLanguageTracingLogsDirectory));
}

// Add database bundle
filesToUpload.push(
...(await tryBundleDatabase(config, language, logger)),
);
}

// Multilanguage tracing: there are additional logs in the root of the cluster
const multiLanguageTracingLogsDirectory = path.resolve(
await uploadDebugArtifacts(
filesToUpload,
config.dbLocation,
"log",
config.debugArtifactName,
);
if (doesDirectoryExist(multiLanguageTracingLogsDirectory)) {
filesToUpload.push(...listFolder(multiLanguageTracingLogsDirectory));
}

// Add database bundle
filesToUpload.push(...(await tryBundleDatabase(config, language, logger)));
} catch (e) {
logger.warning(`Failed to upload debug artifacts: ${wrapError(e).message}`);
}

await uploadDebugArtifacts(
filesToUpload,
config.dbLocation,
config.debugArtifactName,
);
}

export async function uploadDebugArtifacts(
Expand Down