Add slides and move/fix lessons.

Author: Dor1s

README.md

@@ -28,12 +28,12 @@ Fuzzing experience is not required.
 4. Writing fuzzers (simple examples)
 5. Finding Heartbleed (CVE-2014-0160)
 6. Finding c-ares $100,000 bug (CVE-2016-5180)
-7. Fuzzing libxml2, learning how to improve the fuzzer and analyze performance
-8. Fuzzing libpng, learning an importance of seed corpus and other stuff
-9. Fuzzing re2 (TODO: add problems?)
-10. Fuzzing pcre2
-11. Chromium integration
-12. OSS-Fuzz project
+7. How to improve your fuzzer
+8. Fuzzing libxml2, learning how to improve the fuzzer and analyze performance
+9. Fuzzing libpng, learning an importance of seed corpus and other stuff
+10. Fuzzing re2 (TODO: add problems?)
+11. Fuzzing pcre2
+12. Chromium integration & homework assignment
 
 
 ## Prerequisites
@@ -48,6 +48,7 @@ Fuzzer/build.sh
 
 ## Links
 
+* all slides in a single presentation: [Google Slides](https://docs.google.com/presentation/d/1pbbXRL7HaNSjyCHWgGkbpNotJuiC4O7L_PDZoGqDf5Q/edit?usp=sharing)
 * libFuzzer documentation: [http://libfuzzer.info](http://libfuzzer.info)
 * libFuzzer tutorial: [http://tutorial.libfuzzer.info](http://tutorial.libfuzzer.info)
 * Google Online Security Blog: [Guided in-process fuzzing of Chrome components](https://security.googleblog.com/2016/08/guided-in-process-fuzzing-of-chrome.html)

lessons/01/Modern_Fuzzing_of_C_C++_projects_slides_1-23.pdf

@@ -1,5 +1,3 @@
 # Lesson 01
 
-This is a theorethical introduction. Here will be slides.
-
-TODO: Add slides for *"An introduction to fuzz testing"*
+This is a theorethical introduction, see the slides.

lessons/01/README.md

@@ -11,7 +11,8 @@
 
 ## Instruction
 
-Use `radamsa` to generate testcases from `seed_corpus`:
+Take a look at [generate_testcases.py](generate_testcases.py) scripts. Then use
+`radamsa` to generate testcases from `seed_corpus`:
 ```bash
 ./generate_testcases.py
 ```
@@ -22,13 +23,14 @@ ls work/corpus/ | wc -l
 1000
 ```
 
-Run fuzzing:
+Take a look at [run_fuzzing.py](run_fuzzing.py) script. Then run fuzzing:
 ```bash
 unxz bin/asan.tar.xz && tar xf bin/asan.tar
 ./run_fuzzing.py
 ```
 
-If you don't see any output, no crash has been found.
+If you don't see any output, no crash has been found. Feel free to re-generate
+testcases many more times. Though it should take for a while to find a crash.
 
 
 [pdfium]: https://pdfium.googlesource.com/pdfium/

lessons/02/README.md

@@ -1,5 +1,3 @@
 # Lesson 03
 
-This is a theorethical lesson. Here will be slides.
-
-TODO: Add slides for *"Coverage-guided fuzzing"*
+This is a theorethical lesson, see the slides.

lessons/03/Modern_Fuzzing_of_C_C++_projects_slides_24-39.pdf

@@ -1,288 +1,3 @@
 # Lesson 07
 
-Here we will be fuzzing [libxml2]. During this lesson we will:
-* see an importance of dictionaries
-* learn how to minimize the corpus
-* generate coverage report
-* catch Out-of-Memory errors and memory leaks
-
-
-### Build the library
-
-```bash
-tar xzf libxml2.tgz
-cd libxml2
-
-./autogen.sh
-
-export FUZZ_CXXFLAGS="-O2 -fno-omit-frame-pointer -g -fsanitize=address \
-    -fsanitize-coverage=edge,indirect-calls,8bit-counters,trace-cmp,trace-div,trace-gep"
-
-CXX="clang++ $FUZZ_CXXFLAGS" CC="clang $FUZZ_CXXFLAGS" \
-    CCLD="clang++ $FUZZ_CXXFLAGS"  ./configure
-make -j$(nproc)
-```
-
-### Build the first fuzzer
-
-Take a look at the following fuzzer. Note the `xmlSetGenericErrorFunc` call. It
-is there to disable logging of error messages like "Incorrect XML document".
-These messages are very noisy, given the numbe rof invalid input generated by
-the fuzzer:
-
-```cpp
-#include "libxml/parser.h"
-
-void ignore (void* ctx, const char* msg, ...) {
-  // Error handler to avoid spam of error messages from libxml parser.
-}
-
-extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
-  xmlSetGenericErrorFunc(NULL, &ignore);
-
-  if (auto doc = xmlReadMemory(reinterpret_cast<const char*>(data),
-                               static_cast<int>(size), "noname.xml", NULL, 0)) {
-    xmlFreeDoc(doc);
-  }
-
-  return 0;
-}
-```
-
-Then build it:
-
-```bash
-cd ..
-clang++ -std=c++11 xml_read_memory_fuzzer.cc $FUZZ_CXXFLAGS -I libxml2/include \
-    libxml2/.libs/libxml2.a ../../libFuzzer/libFuzzer.a -lz \
-    -o xml_read_memory_fuzzer
-```
-
-### Run the fuzzer with and without a dictionary
-
-Run the fuzzer on empty corpus for 5 minutes (`-max_total_time=300`):
-
-```bash
-mkdir corpus1
-./xml_read_memory_fuzzer -max_total_time=300 -print_final_stats=1 corpus1
-```
-
-Open a new terminal and run the fuzzing on empty corpus again, but also add a
-dictionary (`-dict=`):
-
-```bash
-mkdir corpus2
-./xml_read_memory_fuzzer -dict=./xml.dict -max_total_time=300 \
-    -print_final_stats=1 corpus2
-```
-
-Compare output of both processes while they are running. You should see that the
-second process gets the same coverage as the first one and then overrun it very
-quickly. This is an impact of dictionary used.
-
-
-### Corpus and coverage
-
-The first process terminates somewhere at:
-
-```
-#1975901  DONE   cov: 1736 ft: 5795 corp: 1544/75Kb exec/s: 6564 rss: 494Mb
-```
-
-Let's minimize its corpus (using `-merge=1` flag):
-
-```bash
-mkdir corpus1_min
-./xml_read_memory_fuzzer -merge=1 corpus1_min corpus1
-```
-
-The output looks like:
-
-```bash
-INFO: Seed: 1508800405
-INFO: Loaded 1 modules (79184 guards): [0xd017e0, 0xd4ed20), 
-INFO: -max_len is not provided, using 1048576
-Loaded 1024/1539 files from corpus1
-=== Merging extra 1539 units
-#1539 MIN0   cov: 1723 ft: 5810 units: 1008 exec/s: 0 rss: 95Mb
-#2547 MIN1   cov: 1724 ft: 5764 units: 987 exec/s: 0 rss: 125Mb
-#3534 MIN2   cov: 1724 ft: 5765 units: 975 exec/s: 0 rss: 154Mb
-#4509 MIN3   cov: 1724 ft: 5763 units: 971 exec/s: 0 rss: 183Mb
-=== Merge: written 971 units
-```
-
-That means that libFuzzer made `971` testcase out of `1539` at the same code
-coverage.
-
-To get some understanding of inputs generated by the fuzzer from scratch, let's
-brielfy go through the corpus:
-
-```bash
-strings corpus1_min/* | more
-```
-
-The second process terminates somewhere at:
-
-```
-#2317811  DONE   cov: 2873 ft: 8005 corp: 2359/121Kb exec/s: 7700 rss: 438Mb
-```
-
-The coverage is significantly higher comparing with the first process output.
-
-Let's minimize its corpus as well:
-
-```bash
-mkdir corpus2_min
-./xml_read_memory_fuzzer -merge=1 corpus2_min corpus2
-```
-
-The output:
-
-```bash
-INFO: Seed: 2449634923
-INFO: Loaded 1 modules (79184 guards): [0xd017e0, 0xd4ed20), 
-INFO: -max_len is not provided, using 1048576
-Loaded 1024/2356 files from corpus2
-Loaded 2048/2356 files from corpus2
-=== Merging extra 2356 units
-#2356 MIN0   cov: 2829 ft: 8012 units: 1571 exec/s: 0 rss: 126Mb
-#3927 MIN1   cov: 2830 ft: 7970 units: 1516 exec/s: 0 rss: 169Mb
-#5443 MIN2   cov: 2830 ft: 7969 units: 1503 exec/s: 0 rss: 210Mb
-#6946 MIN3   cov: 2830 ft: 7968 units: 1496 exec/s: 6946 rss: 250Mb
-#8442 MIN4   cov: 2830 ft: 7967 units: 1494 exec/s: 8442 rss: 291Mb
-=== Merge: written 1494 units
-```
-
-And quickly go through the inputs generated by the fuzzer with a dictionary:
-
-```bash
-strings corpus2_min/* | more
-```
-
-### Generate coverage report
-
-```bash
-ASAN_OPTIONS=coverage=1 ./xml_read_memory_fuzzer corpus1_min -runs=0
-```
-
-This command should generate `.sancov` file in your working directory:
-
-```bash
-$ ls *.sancov
-xml_read_memory_fuzzer.26851.sancov
-```
-
-Then we need to convert that binary file to a symbolized `.symcov` file:
-
-```bash
-sancov -symbolize xml_read_memory_fuzzer xml_read_memory_fuzzer.26851.sancov \
-    > xml_read_memory_fuzzer.symcov
-```
-
-To see the coverage report with user-friendly interface, let's launch local
-[coverage report server]:
-
-```bash
-python3 coverage-report-server.py --symcov xml_read_memory_fuzzer.symcov \
-    --srcpath libxml2
-```
-
-Open [localhost:8001](http://localhost:8001/) in your browser to see the report.
-
-
-Let's generate coverage report for the second corpus (generated with dictionary)
-and compare both reports by eyes. Open new terminal and do the same stuff:
-
-```bash
-ASAN_OPTIONS=coverage=1 ./xml_read_memory_fuzzer corpus2_min -runs=0
-
-sancov -symbolize xml_read_memory_fuzzer <NEW_.SANCOV_FILE_PATH> \
-    > xml_read_memory_fuzzer_2.symcov
-
-python3 coverage-report-server.py --symcov xml_read_memory_fuzzer_2.symcov \
-    --srcpath libxml2 --port 8002
-```
-
-Go to [localhost:8002](http://localhost:8002/).
-
-The second report obviously has higher percentage of coverage for the same files
-and even more source code files covered.
-
-
-### Build the second fuzzer
-
-The second fuzzer aims `xmlRegexpCompile` function of libxml2 library:
-
-```cpp
-#include "libxml/parser.h"
-#include "libxml/tree.h"
-#include "libxml/xmlversion.h"
-
-void ignore (void * ctx, const char * msg, ...) {
-  // Error handler to avoid spam of error messages from libxml parser.
-}
-
-// Entry point for LibFuzzer.
-extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
-  xmlSetGenericErrorFunc(NULL, &ignore);
-
-  std::vector<uint8_t> buffer(size + 1, 0);
-  std::copy(data, data + size, buffer.data());
-
-  xmlRegexpPtr x = xmlRegexpCompile(buffer.data());
-  if (x)
-    xmlRegFreeRegexp(x);
-
-  return 0;
-}
-```
-
-Let's build it and run:
-
-```bash
-clang++ -std=c++11 xml_compile_regexp_fuzzer.cc $FUZZ_CXXFLAGS \
-    -I libxml2/include libxml2/.libs/libxml2.a ../../libFuzzer/libFuzzer.a -lz \
-    -o xml_compile_regexp_fuzzer
-
-mkdir corpus3
-./xml_compile_regexp_fuzzer -dict=./xml.dict corpus3
-```
-
-You will quickly get an Out-of-memory crash:
-
-```bash
-#796  NEW    cov: 289 bits: 845 indir: 49 corp: 54/1518b exec/s: 0 rss: 43Mb L: 64 MS: 4 CrossOver-PersAutoDict-CrossOver-ChangeByte- DE: " xml:id=\"1\""-
-#800  NEW    cov: 289 bits: 855 indir: 49 corp: 55/1556b exec/s: 0 rss: 43Mb L: 38 MS: 3 PersAutoDict-ChangeBit-CrossOver- DE: "%a"-
-==27928== ERROR: libFuzzer: out-of-memory (used: 2100Mb; limit: 2048Mb)
-   To change the out-of-memory limit use -rss_limit_mb=<N>
-
-Live Heap Allocations: 1003258238 bytes from 30527559 allocations; showing top 95%
-732653304 byte(s) (73%) in 30527221 allocation(s)
-    #0 0x4c2a0c in __interceptor_malloc (/home/mmoroz/projects/libfuzzer-workshop/lessons/07/xml_compile_regexp_fuzzer+0x4c2a0c)
-    #1 0x5d8506 in xmlRegNewRange /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:719:28
-    #2 0x5d8506 in xmlRegAtomAddRange /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:1251
-    #3 0x5d717e in xmlFAParseCharRange /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5066:9
-    #4 0x5d717e in xmlFAParsePosCharGroup /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5084
-    #5 0x5d4c40 in xmlFAParseCharGroup /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5125:6
-    #6 0x5d2f89 in xmlFAParseCharClass /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5145:2
-    #7 0x5d2f89 in xmlFAParseAtom /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5299
-    #8 0x5d2f89 in xmlFAParsePiece /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5316
-    #9 0x5d25e4 in xmlFAParseBranch /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5351:8
-    #10 0x5b03ad in xmlFAParseRegExp /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5377:5
-    #11 0x5af8f4 in xmlRegexpCompile /home/mmoroz/projects/libfuzzer-workshop/lessons/07/libxml2/xmlregexp.c:5473:5
-    #12 0x4f14d0 in LLVMFuzzerTestOneInput /home/mmoroz/projects/libfuzzer-workshop/lessons/07/xml_compile_regexp_fuzzer.cc:27:20
-    <...>
-```
-
-In some cases it can be a memory leak. To detect leaks, enable `detect_leaks=1`
-option of AddressSanitizer and run the fuzzer again:
-
-```bash
-ASAN_OPTIONS=detect_leaks=1 ./xml_compile_regexp_fuzzer -dict=./xml.dict corpus3
-```
-
-That option enabled LeakSanitizer (a part of AddressSanitizer) to report memory
-leaks and crash the similar way as other crash reports.
-
-[coverage report server]: http://llvm.org/svn/llvm-project/llvm/trunk/tools/sancov/coverage-report-server.py
-[libxml2]: http://www.xmlsoft.org/
+This is a theorethical lesson, see the slides.