Skip to content
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
ec0d5b9
Stop following redirects when max-redirects is hit instead of creatin…
thomas-zahner Jun 5, 2025
049075a
Update test & remove redundant test
thomas-zahner Jun 5, 2025
3fd68ca
First attempt at redirection tracking
thomas-zahner Jun 5, 2025
92ed479
Add ideas
thomas-zahner Jun 5, 2025
b68ff7b
Move todo
thomas-zahner Jun 5, 2025
4150e15
Expand Status::Redirected
thomas-zahner Jun 11, 2025
f0c5e28
Remove file
thomas-zahner Jun 11, 2025
14119f6
Extract RedirectTracker & handle redirections with fragments
thomas-zahner Jul 3, 2025
7513537
No special handling of redirects: remove Status::Redirected and Error…
thomas-zahner Jul 4, 2025
24b04b7
WIP
thomas-zahner Jul 10, 2025
d96481b
Merge branch 'master' into redirect-reporting
thomas-zahner Aug 28, 2025
482f9cc
Apply clippy's suggestions
thomas-zahner Aug 28, 2025
5962f67
Extract method & improve test
thomas-zahner Aug 28, 2025
bb10776
Revert "remove Status::Redirected"
thomas-zahner Aug 28, 2025
a228c05
Update redirection handling & add handle_redirected
thomas-zahner Aug 28, 2025
5fd38a4
Add RedirectChain to Status::Redirected
thomas-zahner Aug 29, 2025
99c5983
Merge branch 'master' into redirect-reporting
thomas-zahner Sep 3, 2025
79a80aa
clippy --fix
thomas-zahner Sep 3, 2025
762da6b
Replace unicode escapes with literal value for simplicity
thomas-zahner Sep 3, 2025
156ee4e
Show redirects in compact view
thomas-zahner Sep 3, 2025
b74cfb0
Fix tests
thomas-zahner Sep 4, 2025
613ec22
Update redirect output color
thomas-zahner Sep 4, 2025
69cd4f6
Add redirect_map for JSON output
thomas-zahner Sep 4, 2025
34e5f34
Create test_redirect_json & extract function
thomas-zahner Sep 4, 2025
cbd939f
Update lychee-lib/src/types/redirect_tracker.rs
thomas-zahner Sep 5, 2025
8e5dc74
Rename RedirectChain to Redirects
thomas-zahner Sep 5, 2025
061e6e1
Rename RedirectTracker to RedirectHistory
thomas-zahner Sep 5, 2025
21f434b
Minor improvements
thomas-zahner Sep 5, 2025
2170435
Update README
thomas-zahner Sep 5, 2025
483cad8
Create test-utils to share testing functionality between lychee-lib a…
thomas-zahner Sep 5, 2025
b96b874
Port lychee-lib/src/test_utils.rs to new test-utils crate
thomas-zahner Sep 5, 2025
c62b099
Fix space
thomas-zahner Sep 5, 2025
d4210f8
Update doc comment
thomas-zahner Sep 11, 2025
b630508
Apply review suggestions
thomas-zahner Sep 11, 2025
bfd231c
Put doc comments before attributes
thomas-zahner Sep 11, 2025
59bc64e
Prevent accidental publishing of test-utils crate
thomas-zahner Sep 11, 2025
9a20ce6
Update verbose redirect message
thomas-zahner Sep 11, 2025
c4e059e
Merge branch 'master' into redirect-reporting
thomas-zahner Sep 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -850,3 +850,9 @@ at your option.

<br><hr>
[πŸ”Ό Back to top](#back-to-top)



# TODO: redirections

- Add redirect_map to JSON
3 changes: 2 additions & 1 deletion fixtures/TEST.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ Test GZIP compression. (See https://github.com/analysis-tools-dev/static-analysi
[LDRA](https://ldra.com)

Some more complex formatting to test that Markdown parsing works.
[![CC0](https://i.creativecommons.org/p/zero/1.0/88x31.png)](https://creativecommons.org/publicdomain/zero/1.0/)
[![CC0](https://licensebuttons.net/p/zero/1.0/88x31.png)](https://creativecommons.org/publicdomain/zero/1.0/)


Test HTTP and HTTPS for the same site.
http://example.com
Expand Down
2 changes: 1 addition & 1 deletion fixtures/TEST_GITHUB.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Test file: contains a single GitHub URL.

Lychee: https://github.com/hello-rust/lychee
Lychee: https://github.com/lycheeverse/lychee
5 changes: 2 additions & 3 deletions lychee-bin/src/formatters/response/color.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use lychee_lib::{CacheStatus, ResponseBody, Status};

use crate::formatters::color::{DIM, GREEN, NORMAL, PINK, YELLOW};
use crate::formatters::color::{DIM, GREEN, PINK, YELLOW};

use super::{MAX_RESPONSE_OUTPUT_WIDTH, ResponseFormatter};

Expand All @@ -15,11 +15,10 @@ impl ColorFormatter {
/// response.
fn status_color(status: &Status) -> &'static std::sync::LazyLock<console::Style> {
match status {
Status::Ok(_) | Status::Cached(CacheStatus::Ok(_)) => &GREEN,
Status::Ok(_) | Status::Cached(CacheStatus::Ok(_)) | Status::Redirected(_, _) => &GREEN,
Status::Excluded
| Status::Unsupported(_)
| Status::Cached(CacheStatus::Excluded | CacheStatus::Unsupported) => &DIM,
Status::Redirected(_) => &NORMAL,
Status::UnknownStatusCode(_) | Status::Timeout(_) => &YELLOW,
Status::Error(_) | Status::Cached(CacheStatus::Error(_)) => &PINK,
}
Expand Down
6 changes: 3 additions & 3 deletions lychee-bin/src/formatters/response/emoji.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ impl EmojiFormatter {
Status::Excluded
| Status::Unsupported(_)
| Status::Cached(CacheStatus::Excluded | CacheStatus::Unsupported) => "🚫",
Status::Redirected(_) => "β†ͺ️",
Status::Redirected(_, _) => "β†ͺ️",
Status::UnknownStatusCode(_) | Status::Timeout(_) => "⚠️",
Status::Error(_) | Status::Cached(CacheStatus::Error(_)) => "❌",
}
Expand All @@ -40,7 +40,7 @@ impl ResponseFormatter for EmojiFormatter {
mod emoji_tests {
use super::*;
use http::StatusCode;
use lychee_lib::{ErrorKind, Status, Uri};
use lychee_lib::{ErrorKind, RedirectChain, Status, Uri};

// Helper function to create a ResponseBody with a given status and URI
fn mock_response_body(status: Status, uri: &str) -> ResponseBody {
Expand Down Expand Up @@ -84,7 +84,7 @@ mod emoji_tests {
fn test_format_response_with_redirect_status() {
let formatter = EmojiFormatter;
let body = mock_response_body(
Status::Redirected(StatusCode::MOVED_PERMANENTLY),
Status::Redirected(StatusCode::MOVED_PERMANENTLY, RedirectChain::default()),
"https://example.com/redirect",
);
assert_eq!(
Expand Down
5 changes: 3 additions & 2 deletions lychee-bin/src/formatters/response/plain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ impl ResponseFormatter for PlainFormatter {
mod plain_tests {
use super::*;
use http::StatusCode;
use lychee_lib::RedirectChain;
use lychee_lib::{ErrorKind, Status, Uri};

// Helper function to create a ResponseBody with a given status and URI
Expand Down Expand Up @@ -69,12 +70,12 @@ mod plain_tests {
fn test_format_response_with_redirect_status() {
let formatter = PlainFormatter;
let body = mock_response_body(
Status::Redirected(StatusCode::MOVED_PERMANENTLY),
Status::Redirected(StatusCode::MOVED_PERMANENTLY, RedirectChain::default()),
"https://example.com/redirect",
);
assert_eq!(
formatter.format_response(&body),
"[301] https://example.com/redirect | Redirect (301 Moved Permanently): Moved Permanently"
"[301] https://example.com/redirect | Redirect: Followed 0 redirects resulting in Moved Permanently."
);
}

Expand Down
6 changes: 3 additions & 3 deletions lychee-bin/src/formatters/response/task.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ impl ResponseFormatter for TaskFormatter {
mod task_tests {
use super::*;
use http::StatusCode;
use lychee_lib::{ErrorKind, Status, Uri};
use lychee_lib::{ErrorKind, RedirectChain, Status, Uri};

// Helper function to create a ResponseBody with a given status and URI
fn mock_response_body(status: Status, uri: &str) -> ResponseBody {
Expand Down Expand Up @@ -60,12 +60,12 @@ mod task_tests {
fn test_format_response_with_redirect_status() {
let formatter = TaskFormatter;
let body = mock_response_body(
Status::Redirected(StatusCode::MOVED_PERMANENTLY),
Status::Redirected(StatusCode::MOVED_PERMANENTLY, RedirectChain::default()),
"https://example.com/redirect",
);
assert_eq!(
formatter.format_response(&body),
"- [ ] [301] https://example.com/redirect | Redirect (301 Moved Permanently): Moved Permanently"
"- [ ] [301] https://example.com/redirect | Redirect: Followed 0 redirects resulting in Moved Permanently."
);
}

Expand Down
4 changes: 3 additions & 1 deletion lychee-bin/src/formatters/stats/compact.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ impl Display for CompactResponseStats {
numeric_sort::cmp(&a, &b)
});

writeln!(f, "\n\u{2139} Suggestions")?;
writeln!(f, "\nβ„Ή Suggestions")?;
for suggestion in sorted_suggestions {
writeln!(f, "{suggestion}")?;
}
Expand All @@ -81,6 +81,7 @@ impl Display for CompactResponseStats {
write_if_any(stats.excludes, "πŸ‘»", "Excluded", &BOLD_YELLOW, f)?;
write_if_any(stats.timeouts, "⏳", "Timeouts", &BOLD_YELLOW, f)?;
write_if_any(stats.unsupported, "β›”", "Unsupported", &BOLD_YELLOW, f)?;
write_if_any(stats.redirects, "πŸ”€", "Redirects", &BOLD_YELLOW, f)?;

Ok(())
}
Expand Down Expand Up @@ -167,6 +168,7 @@ mod tests {
duration_secs: 0,
error_map,
suggestion_map: HashMap::default(),
redirect_map: HashMap::default(),
unsupported: 0,
redirects: 0,
cached: 0,
Expand Down
19 changes: 10 additions & 9 deletions lychee-bin/src/formatters/stats/detailed.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,16 @@ impl Display for DetailedResponseStats {
let stats = &self.stats;
let separator = "-".repeat(MAX_PADDING + 1);

writeln!(f, "\u{1f4dd} Summary")?; // πŸ“
writeln!(f, "πŸ“ Summary")?;
writeln!(f, "{separator}")?;
write_stat(f, "\u{1f50d} Total", stats.total, true)?; // πŸ”
write_stat(f, "\u{2705} Successful", stats.successful, true)?; // βœ…
write_stat(f, "\u{23f3} Timeouts", stats.timeouts, true)?; // ⏳
write_stat(f, "\u{1f500} Redirected", stats.redirects, true)?; // πŸ”€
write_stat(f, "\u{1f47b} Excluded", stats.excludes, true)?; // πŸ‘»
write_stat(f, "\u{2753} Unknown", stats.unknown, true)?; //❓
write_stat(f, "\u{1f6ab} Errors", stats.errors, true)?; // 🚫
write_stat(f, "\u{26d4} Unsupported", stats.errors, false)?; // β›”
write_stat(f, "πŸ” Total", stats.total, true)?;
write_stat(f, "βœ… Successful", stats.successful, true)?;
write_stat(f, "⏳ Timeouts", stats.timeouts, true)?;
write_stat(f, "πŸ”€ Redirected", stats.redirects, true)?;
write_stat(f, "πŸ‘» Excluded", stats.excludes, true)?;
write_stat(f, "❓ Unknown", stats.unknown, true)?;
write_stat(f, "🚫 Errors", stats.errors, true)?;
write_stat(f, "β›” Unsupported", stats.errors, false)?;

let response_formatter = get_response_formatter(&self.mode);

Expand Down Expand Up @@ -138,6 +138,7 @@ mod tests {
redirects: 0,
cached: 0,
suggestion_map: HashMap::default(),
redirect_map: HashMap::default(),
success_map: HashMap::default(),
error_map,
excluded_map: HashMap::default(),
Expand Down
16 changes: 8 additions & 8 deletions lychee-bin/src/formatters/stats/markdown.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,35 +26,35 @@ struct StatsTableEntry {
fn stats_table(stats: &ResponseStats) -> String {
let stats = vec![
StatsTableEntry {
status: "\u{1f50d} Total",
status: "πŸ” Total",
count: stats.total,
},
StatsTableEntry {
status: "\u{2705} Successful",
status: "βœ… Successful",
count: stats.successful,
},
StatsTableEntry {
status: "\u{23f3} Timeouts",
status: "⏳ Timeouts",
count: stats.timeouts,
},
StatsTableEntry {
status: "\u{1f500} Redirected",
status: "πŸ”€ Redirected",
count: stats.redirects,
},
StatsTableEntry {
status: "\u{1f47b} Excluded",
status: "πŸ‘» Excluded",
count: stats.excludes,
},
StatsTableEntry {
status: "\u{2753} Unknown",
status: "❓ Unknown",
count: stats.unknown,
},
StatsTableEntry {
status: "\u{1f6ab} Errors",
status: "🚫 Errors",
count: stats.errors,
},
StatsTableEntry {
status: "\u{26d4} Unsupported",
status: "β›” Unsupported",
count: stats.unsupported,
},
];
Expand Down
15 changes: 10 additions & 5 deletions lychee-bin/src/stats.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,15 @@ pub(crate) struct ResponseStats {
pub(crate) errors: usize,
/// Number of responses that were cached from a previous run
pub(crate) cached: usize,
/// Map to store successful responses (if `detailed_stats` is enabled)
/// Store successful responses (if `detailed_stats` is enabled)
pub(crate) success_map: HashMap<InputSource, HashSet<ResponseBody>>,
/// Map to store failed responses (if `detailed_stats` is enabled)
/// Store failed responses (if `detailed_stats` is enabled)
pub(crate) error_map: HashMap<InputSource, HashSet<ResponseBody>>,
/// Replacement suggestions for failed responses (if `--suggest` is enabled)
pub(crate) suggestion_map: HashMap<InputSource, HashSet<Suggestion>>,
/// Map to store excluded responses (if `detailed_stats` is enabled)
/// Store redirected responses with their redirection chain (if `detailed_stats` is enabled)
pub(crate) redirect_map: HashMap<InputSource, HashSet<ResponseBody>>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really store the redirection list here as the comment suggests?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, see my previous comment section Detailed output and JSON. It's the new redirect_map field. I've added it for a more detailed JSON output. Maybe with their redirection list was a bit misleading? I've removed it now.

/// Store excluded responses (if `detailed_stats` is enabled)
pub(crate) excluded_map: HashMap<InputSource, HashSet<ResponseBody>>,
/// Used to store the duration of the run in seconds.
pub(crate) duration_secs: u64,
Expand Down Expand Up @@ -72,7 +74,7 @@ impl ResponseStats {
Status::Error(_) => self.errors += 1,
Status::UnknownStatusCode(_) => self.unknown += 1,
Status::Timeout(_) => self.timeouts += 1,
Status::Redirected(_) => self.redirects += 1,
Status::Redirected(_, _) => self.redirects += 1,
Status::Excluded => self.excludes += 1,
Status::Unsupported(_) => self.unsupported += 1,
Status::Cached(cache_status) => {
Expand All @@ -95,6 +97,9 @@ impl ResponseStats {
_ if status.is_error() => self.error_map.entry(source).or_default(),
Status::Ok(_) if self.detailed_stats => self.success_map.entry(source).or_default(),
Status::Excluded if self.detailed_stats => self.excluded_map.entry(source).or_default(),
Status::Redirected(_, _) if self.detailed_stats => {
self.redirect_map.entry(source).or_default()
}
_ => return,
};
status_map_entry.insert(response.1);
Expand All @@ -110,7 +115,7 @@ impl ResponseStats {
#[inline]
/// Check if the entire run was successful
pub(crate) const fn is_success(&self) -> bool {
self.total == self.successful + self.excludes + self.unsupported
self.total == self.successful + self.excludes + self.unsupported + self.redirects
}

#[inline]
Expand Down
Loading
Loading