Skip to content
Open
Changes from 3 commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
09bc7c2
Use activations to calculate the stats
EAddario Jul 26, 2025
2097f03
Refactor variable names
EAddario Jul 31, 2025
78ddb47
Fix problem up when GGUF does not have in_sum
EAddario Aug 2, 2025
9744a4a
Determine calculation mode
EAddario Aug 2, 2025
cce514a
Compute entropy for activations
EAddario Aug 2, 2025
b7fb362
Compute cosine similarity based on activations
EAddario Aug 2, 2025
9b841eb
Compute l2 norm
EAddario Aug 2, 2025
ee2509f
Adjust threshold
EAddario Aug 2, 2025
fc8f925
Update table display
EAddario Aug 2, 2025
4c01f51
Remove inactive
EAddario Aug 2, 2025
a32a2ec
Reformat report layout
EAddario Aug 2, 2025
4d1325e
Refactor variables
EAddario Aug 3, 2025
5324558
Update table layout
EAddario Aug 3, 2025
fce05aa
Refactor lambda into compute_tensor_averages() function
EAddario Aug 3, 2025
be60469
Refactor function names
EAddario Aug 3, 2025
a6155a8
Add compute_layer_statistics() function
EAddario Aug 3, 2025
2117c4e
Update aggregated statistic report layout
EAddario Aug 3, 2025
90cb1be
Minor cosmetic changes
EAddario Aug 3, 2025
f1c2a4c
Fix printing l2 norm when calc_mode = 1
EAddario Aug 3, 2025
c39c4e2
Refactor variable name
EAddario Aug 4, 2025
adbff66
Merge branch 'master' into imatrix
EAddario Aug 4, 2025
5e40cf4
Do not resize if in_sum is null
EAddario Aug 4, 2025
b373934
Compute aggregated (per layer) l2 norm
EAddario Aug 5, 2025
906548a
Update aggregated sum of squared activations per layer
EAddario Aug 5, 2025
aea9b31
Make ZD Score two-tailed
EAddario Aug 5, 2025
49996a1
Refactor variable names
EAddario Aug 5, 2025
4c3fea8
Update report layout
EAddario Aug 5, 2025
88854c9
Refactor legacy mode
EAddario Aug 5, 2025
030ed3c
Merge branch 'master' into imatrix
EAddario Aug 5, 2025
c7959ed
Merge branch 'master' into imatrix
EAddario Aug 7, 2025
3e9d53c
Refactor variable names
EAddario Aug 7, 2025
e0d6471
Reverse conditional logic to match convention
EAddario Aug 7, 2025
dadd90e
Rename report heading
EAddario Aug 7, 2025
5bb2def
Add --activation-statistics parameter
EAddario Aug 7, 2025
c5ecdaa
Add Euclidean–Cosine Score (ECS)
EAddario Aug 7, 2025
59af503
Update README.md
EAddario Aug 9, 2025
9467963
Merge branch 'master' into imatrix
EAddario Aug 9, 2025
6fe51e1
Fix typo in ECS formula
EAddario Aug 9, 2025
dcac206
Add --activation-statistics logic to avoid doubling the imatrix size …
EAddario Aug 9, 2025
89051cd
Update README.md
EAddario Aug 9, 2025
2756617
Merge branch 'master' into imatrix
EAddario Aug 15, 2025
42bfe3b
Update stats output sort based on imatrix type
EAddario Aug 15, 2025
240a965
Update README.md
EAddario Aug 15, 2025
8589ef4
Update README.md
EAddario Aug 15, 2025
030ec53
Remove unnecessary include
EAddario Aug 16, 2025
d4b0d89
Fix return type bug
EAddario Aug 16, 2025
e3149a2
Use the corresponding size
EAddario Aug 17, 2025
4a487ea
Use { and } around the conditionally-executed statement
EAddario Aug 17, 2025
97d839c
Using one line per variable definition
EAddario Aug 17, 2025
d19e6c9
Use { and } around the conditionally-executed statement
EAddario Aug 17, 2025
12607d3
Use { and } around single line for statement
EAddario Aug 17, 2025
a96013f
Define one variable per line and refactor names
EAddario Aug 17, 2025
2e80323
Use { and } around conditionally-executed single line statements
EAddario Aug 17, 2025
44ea7dd
Change statement order
EAddario Aug 17, 2025
f6934b9
Merge branch 'imatrix' of https://github.com/EAddario/llama.cpp into …
EAddario Aug 17, 2025
1f72bc1
Avoid using if statements with initialiser
EAddario Aug 17, 2025
630750f
Validate number of elements if in_sum is present
EAddario Aug 17, 2025
5aca256
Merge branch 'master' into imatrix
EAddario Aug 21, 2025
3e26364
Clarify the nature of the calculated cosine similarity
EAddario Aug 24, 2025
69b351b
Add --output-format to usage
EAddario Aug 26, 2025
6371902
Add --output-format to usage
EAddario Aug 26, 2025
70dd25b
Merge branch 'master' into imatrix
EAddario Aug 30, 2025
8f1aa78
Remove activation_statistics() option
EAddario Aug 31, 2025
8d0e276
Update README.md
EAddario Aug 31, 2025
7448bdb
Merge branch 'master' into imatrix
EAddario Sep 6, 2025
0c3a019
Merge branch 'master' into imatrix
EAddario Sep 10, 2025
63f3449
Merge branch 'master' into imatrix
EAddario Sep 15, 2025
193d5bb
Merge branch 'master' into imatrix
EAddario Sep 20, 2025
5932eef
Merge branch 'master' into imatrix
EAddario Sep 25, 2025
a28ee30
Merge branch 'master' into imatrix
EAddario Oct 1, 2025
bc38936
Merge branch 'master' into imatrix
EAddario Oct 3, 2025
252c4b7
Merge branch 'master' into imatrix
EAddario Oct 11, 2025
09ec0c0
Merge branch 'master' into imatrix
EAddario Oct 16, 2025
c81f7cd
Merge branch 'master' into imatrix
EAddario Oct 20, 2025
8fd2aca
Merge branch 'master' into imatrix
EAddario Oct 25, 2025
af3b6ac
Fix legacy_mode getting overwritten on each tensor bug
EAddario Oct 28, 2025
c9a0874
Clamp CosSim to [-1, 1] to avoid float drift
EAddario Oct 28, 2025
637e674
Avoid division by zero on zero-count matrices
EAddario Oct 28, 2025
683ef8d
Fill zeros for experts with zero counts to preserve shape
EAddario Oct 28, 2025
dc4a04b
Adjust size calculation and change fallback value to 0.0f
EAddario Oct 28, 2025
0b0381c
Merge Cosine Similarity and L2 Norm computation into single loop
EAddario Oct 28, 2025
b5068df
Minor refactoring
EAddario Oct 28, 2025
92a42ba
Type refactoring
EAddario Oct 28, 2025
ab01506
Minor refactoring
EAddario Oct 28, 2025
86fabce
Clamp values
EAddario Oct 28, 2025
6ff0a79
Minor stats report cosmetic changes
EAddario Oct 29, 2025
2a6f5d7
Refactor variable names
EAddario Oct 29, 2025
006e7ef
Improve compute_vector_statistics() processing of mismatched tensor s…
EAddario Oct 29, 2025
7d8819f
Improve compute_layer_statistics() processing of mismatched tensor sizes
EAddario Oct 29, 2025
ce046dc
Save statistics to imatrix
EAddario Oct 30, 2025
8bd9d87
Merge branch 'master' into imatrix
EAddario Oct 31, 2025
b2b7175
Fix bug when vectors are zero
EAddario Nov 6, 2025
559ae9a
Refactor legacy imatrix handling
EAddario Nov 17, 2025
5384a11
Initialise layer and tensor variables
EAddario Nov 17, 2025
ae1cbc7
Warn if problem with previous layer
EAddario Nov 17, 2025
63cbcc6
Refactor legacy determination
EAddario Nov 17, 2025
fb2b09a
Skip experts with zero count (unused)
EAddario Nov 17, 2025
76566b8
Enforce same-size between compared tensors
EAddario Nov 17, 2025
1f3db49
Calculate layer_sum only for legacy
EAddario Nov 17, 2025
a2b86d7
Minor refactoring
EAddario Nov 17, 2025
658c6a8
Enforce tensor structure when aggregating multiple imatrix files
EAddario Nov 17, 2025
cdc7cae
Remove unreachable logic
EAddario Nov 17, 2025
bf9823a
Minor refactoring
EAddario Nov 17, 2025
8d97eee
Improve layer 0 stats
EAddario Nov 17, 2025
4cfddea
Merge branch 'master' into imatrix
EAddario Nov 17, 2025
4a0511f
Remove storing tensor statistics
EAddario Nov 23, 2025
fcba499
Merge branch 'master' into imatrix
EAddario Nov 23, 2025
44a6721
Merge branch 'master' into imatrix
EAddario Nov 30, 2025
6076bfd
Merge branch 'master' into imatrix
EAddario Dec 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions tools/imatrix/imatrix.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ static bool compute_vector_statistics(std::vector<tensor_statistics> & tstats, c
if (div > 0.0) {
for (float w : weights) {
const float p = w / div;
if (p > 0.0) entropy -= p * std::log2(p);
if (p > 0.0) { entropy -= p * std::log2(p); }
}
}
}
Expand Down Expand Up @@ -268,7 +268,9 @@ static void compute_tensor_statistics(std::vector<tensor_statistics> & tstats) {
const auto curr_avg = compute_tensor_averages(ts.stats);
const auto prev_avg = compute_tensor_averages(prev->stats);
if (curr_avg.size() == prev_avg.size() && !curr_avg.empty()) {
float dot_prod = 0.0f, vec1 = 0.0f, vec2 = 0.0f;
float dot_prod = 0.0f;
float vec1 = 0.0f;
float vec2 = 0.0f;
for (size_t i = 0; i < curr_avg.size(); ++i) {
dot_prod += curr_avg[i] * prev_avg[i];
vec1 += curr_avg[i] * curr_avg[i];
Expand Down