Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
56a9903
Created new test_runner.c and test_.py
geky Apr 16, 2022
d683f1c
Reintroduced test-defines into the new test_runner
geky Apr 18, 2022
4b0aa62
Some more minor improvements to the test_runner
geky Apr 18, 2022
9281ce2
More test_runner progress
geky Apr 18, 2022
92a600a
Added trace and persist flags to test_runner
geky Apr 19, 2022
6443693
Putting together rewritten test.py script
geky Apr 24, 2022
5812d2b
Reworked how multi-layered defines work in the test-runner
geky Apr 25, 2022
5ee4b05
Misc test-runner improvements
geky Apr 25, 2022
4962829
Continued progress toward feature parity with new test-runner
geky May 1, 2022
be0e6ad
More progress toward test-runner feature parity
geky May 2, 2022
5a572ce
Reworked how test defines are implemented to support recursion
geky May 8, 2022
d679fbb
In ./scripts/test.py, readded external commands, tweaked subprocesses
geky May 8, 2022
0781f50
Ported tests to new framework
geky May 14, 2022
4a42326
Moved test suites into custom linker section
geky May 14, 2022
1616115
Fix test.py hang on ctrl-C, cleanup TODOs
geky May 15, 2022
2b11f2b
Tweaked generation of .cgi files, error code for recursion in stack.py
geky May 15, 2022
4a7e94f
Reimplemented coverage.py, using only gcov and with line+branch coverage
geky May 16, 2022
5b0a6d4
Reworked scripts to move field details into classes
geky May 21, 2022
46cc6d4
Added support for annotated source in coverage.py
geky May 23, 2022
92eee8e
Removed some prefixes from Makefile variables where not necessary
geky Aug 15, 2022
b08463f
Reworked scripts/pretty_asserts.py a bit
geky Aug 16, 2022
a368d3a
Moved emulation of erase values up into lfs_testbd
geky Aug 17, 2022
01b11da
Added a simple test that the block device works
geky Aug 17, 2022
61455b6
Added back heuristic-based power-loss testing
geky Aug 19, 2022
4689678
Added --color to test.py, fixed some terminal-clobbering issues
geky Aug 22, 2022
3f4f859
Readded support for mirror writes to a file in testbd
geky Aug 23, 2022
552336e
Added optional read/prog/erase delays to testbd
geky Aug 23, 2022
5279fc6
Implemented exhaustive testing of n nested powerlosses
geky Aug 25, 2022
c9a6e3a
Added tailpipe.py and improved redirecting test trace/log output over…
geky Aug 27, 2022
91200e6
Added tracebd.py, a script for rendering block device operations
geky Sep 2, 2022
a208d84
Reworked test defines a bit to use one common array layout
geky Sep 6, 2022
c7f7094
Several tweaks to test.py and test runner
geky Sep 9, 2022
5a2ff17
Changed test identifier separator # -> :
geky Sep 9, 2022
bfbe44e
Dropped permutation number for full leb16-encoded defines
geky Sep 10, 2022
03c1a4e
Added permutations and ranges to test defines
geky Sep 11, 2022
23fba40
Added option for updating a CSV file with test results
geky Sep 12, 2022
acdea18
Made summary.py more powerful, dropped -m from size scripts
geky Sep 14, 2022
1fcd82d
Made test.py output parsable by summary.py
geky Sep 16, 2022
11d6d12
Dropped namespacing of test cases
geky Sep 17, 2022
20ec0be
Cleaned up a number of small tweaks in the scripts
geky Sep 19, 2022
4fe0738
Added bench.py and bench_runner.c for benchmarking
geky Sep 20, 2022
9a0e3be
Added a quick trie to avoid running redundant test/bench permutations
geky Sep 20, 2022
7591d9c
Added plot.py for in-terminal plotting
geky Sep 22, 2022
fb58148
Consistent handling of by/field arguments for plot.py and summary.py
geky Sep 23, 2022
42d889e
Reworked/simplified tracebd.py a bit
geky Sep 25, 2022
9507e62
Several tweaks to script flags
geky Sep 27, 2022
a2fb708
Added stddev/gmean/gstddev to summary.py
geky Sep 27, 2022
274222b
Added some automatic sizing for field-names in scripts/runners
geky Sep 27, 2022
296c5af
Renamed bench_read/prog/erased -> bench_readed/proged/erased
geky Sep 27, 2022
ca66993
Tweaked scripts to share more code, added coverage calls/hits
geky Oct 2, 2022
490e1c4
Added perf.py a wrapper around Linux's perf tool for perf sampling
geky Oct 2, 2022
df283ae
Added recursive results to perf.py
geky Oct 9, 2022
29cbafe
Renamed coverage.py -> cov.py
geky Oct 9, 2022
3a33c37
Added perfbd.py and block device performance sampling in bench-runner
geky Oct 13, 2022
b2a2cc9
Added teepipe.py and watch.py
geky Oct 16, 2022
559e174
Added plotmpl.py for creating svg/png plots with matplotlib
geky Oct 20, 2022
6fce9e5
Changed plotmpl.py/plot.py to not treat missing values as discontinui…
geky Nov 10, 2022
1a07c2c
A number of small script fixes/tweaks from usage
geky Nov 11, 2022
e35e078
Renamed prefix.py -> changeprefix.py and updated to use argparse
geky Nov 16, 2022
bcc88f5
A couple Makefile-related tweaks
geky Nov 17, 2022
f734941
Changed default build target lfs.a -> liblfs.a
geky Nov 17, 2022
6c18b4d
Added a simple help rule to the Makefile
geky Nov 17, 2022
f89d758
Fixed test out-of-space issues with powerloss testing
geky Nov 28, 2022
eba5553
Fixed hidden orphans by separating deorphan search into two passes
geky Nov 28, 2022
0b11ce0
Fixed incorrect calculation of extra space needed in mdir blocks
geky Nov 19, 2022
387cf6f
Fixed a couple corner cases in scripts when fields are empty
geky Nov 19, 2022
65923cd
Adopted script changes in GitHub Actions
geky Nov 17, 2022
397aa27
Removed unnecessarily heavy RAM usage from logs in bench/test.py
geky Nov 23, 2022
a659c02
Added a bot-generated PR-comment with a simple status table
geky Nov 23, 2022
4a20934
Fixed bench workflow + changeprefix issue in prefix releases
geky Nov 27, 2022
0c781dd
Merge remote-tracking branch 'origin/master' into test-and-bench-runners
geky Dec 7, 2022
9990342
Fixed Clang testing in CI, removed override vars in Makefile
geky Nov 29, 2022
c4b3e9d
A couple of script changes after CI integration
geky Nov 29, 2022
9b687dd
Added make benchmarks/testmarks rules
geky Nov 29, 2022
cda2f6f
Changed test_runner to run with -Pnone,linear by default
geky Nov 30, 2022
d8e7ffb
Changed lfs_emubd_get* -> lfs_emubd_*
geky Nov 30, 2022
b0382fa
Added BENCH/TEST_PRNG, replacing other ad-hoc sources of randomness
geky Nov 30, 2022
2d2dd8b
Added plotmpl.py --github flag to match the website's foreground/back…
geky Dec 2, 2022
cfd4e60
Added --subplot* to plotmpl.py
geky Dec 13, 2022
1f37eb5
Adopted --subplot* in plot.py
geky Dec 15, 2022
801cf27
Tweaked/fixed a number of small runner things after a bit of use
geky Dec 17, 2022
c2147c4
Added --gdb-pl to test.py for breaking on specific powerlosses
geky Dec 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Tweaked generation of .cgi files, error code for recursion in stack.py
GCC is a bit annoying here, it can't generate .cgi files without
generating the related .o files, though I suppose the alternative risks
duplicating a large amount of compilation work (littlefs is really
a small project).

Previously we rebuilt the .o files anytime we needed .cgi files
(callgraph info used for stack.py). This changes it so we always
built .cgi files as a side-effect of compilation. This is similar
to the .d file generation, though may be annoying if the system
cc doesn't support --callgraph-info.
  • Loading branch information
geky committed Jun 6, 2022
commit 2b11f2b426deb4df8f63abfbcf2a9e96ca60a5cc
19 changes: 4 additions & 15 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@ override TESTFLAGS += -b
override TESTFLAGS += $(filter -j%,$(MAKEFLAGS))
ifdef VERBOSE
override TESTFLAGS += -v
override CALLSFLAGS += -v
override CODEFLAGS += -v
override DATAFLAGS += -v
override STACKFLAGS += -v
Expand All @@ -77,7 +76,6 @@ override TESTFLAGS += --coverage
endif
ifdef BUILDDIR
override TESTFLAGS += --build-dir="$(BUILDDIR:/=)"
override CALLSFLAGS += --build-dir="$(BUILDDIR:/=)"
override CODEFLAGS += --build-dir="$(BUILDDIR:/=)"
override DATAFLAGS += --build-dir="$(BUILDDIR:/=)"
override STACKFLAGS += --build-dir="$(BUILDDIR:/=)"
Expand Down Expand Up @@ -108,10 +106,6 @@ size: $(OBJ)
tags:
$(CTAGS) --totals --c-types=+p $(shell find -H -name '*.h') $(SRC)

.PHONY: calls
calls: $(CGI)
./scripts/calls.py $^ $(CALLSFLAGS)

.PHONY: test_runner
test_runner: $(BUILDDIR)runners/test_runner

Expand Down Expand Up @@ -172,19 +166,14 @@ $(BUILDDIR)lfs.csv: $(OBJ) $(CGI)
$(BUILDDIR)runners/test_runner: $(TEST_TAOBJ)
$(CC) $(CFLAGS) $^ $(LFLAGS) -o $@

$(BUILDDIR)%.o: %.c
$(CC) -c -MMD $(CFLAGS) $< -o $@
# our main build rule generates .o, .d, and .ci files, the latter
# used for stack analysis
$(BUILDDIR)%.o $(BUILDDIR)%.ci: %.c
$(CC) -c -MMD -fcallgraph-info=su $(CFLAGS) $< -o $(BUILDDIR)$*.o

$(BUILDDIR)%.s: %.c
$(CC) -S $(CFLAGS) $< -o $@

# gcc depends on the output file for intermediate file names, so
# we can't omit to .o output. We also need to serialize with the
# normal .o rule because otherwise we can end up with multiprocess
# problems with two instances of gcc modifying the same .o
$(BUILDDIR)%.ci: %.c | $(BUILDDIR)%.o
$(CC) -c -MMD -fcallgraph-info=su $(CFLAGS) $< -o $|

$(BUILDDIR)%.a.c: %.c
./scripts/explode_asserts.py $< -o $@

Expand Down
7 changes: 7 additions & 0 deletions scripts/stack.py
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,11 @@ def print_totals():
print_entries(by='name')
print_totals()

# catch recursion
if args.get('error_on_recursion') and any(
m.isinf(limit) for _, _, _, limit, _ in results):
sys.exit(2)


if __name__ == "__main__":
import argparse
Expand Down Expand Up @@ -424,6 +429,8 @@ def print_totals():
help="Show file-level calls.")
parser.add_argument('-Y', '--summary', action='store_true',
help="Only show the total stack size.")
parser.add_argument('-e', '--error-on-recursion', action='store_true',
help="Error if any functions are recursive.")
parser.add_argument('--build-dir',
help="Specify the relative build directory. Used to map object files \
to the correct source files.")
Expand Down