From d6efe672b577098623f7a9ea28ccb777d04aa0cf Mon Sep 17 00:00:00 2001
From: Niels Lohmann <mail@nlohmann.me>
Date: Mon, 9 May 2022 08:02:41 +0200
Subject: [PATCH] Document fuzzer usage (#3478)

* :memo: document fuzzer usage

* :memo: address review comments
---
 .gitignore       |  4 +++
 tests/fuzzing.md | 81 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+)
 create mode 100644 tests/fuzzing.md

diff --git a/.gitignore b/.gitignore
index db6dcecb4..72eb2c225 100644
--- a/.gitignore
+++ b/.gitignore
@@ -19,6 +19,10 @@
 # build directories (vscode-cmake-tools, user-defined, ...)
 /build*/
 
+# fuzzers
+/tests/parse_*_fuzzer
+/tests/corpus_*
+
 /docs/mkdocs/docs/examples/
 /docs/mkdocs/docs/__pycache__/
 /docs/mkdocs/site/
diff --git a/tests/fuzzing.md b/tests/fuzzing.md
new file mode 100644
index 000000000..cfbf4f249
--- /dev/null
+++ b/tests/fuzzing.md
@@ -0,0 +1,81 @@
+# Fuzz testing
+
+Each parser of the library (JSON, BJData, BSON, CBOR, MessagePack, and UBJSON) can be fuzz tested. Currently,
+[libFuzzer](https://llvm.org/docs/LibFuzzer.html) and [afl++](https://github.com/AFLplusplus/AFLplusplus) are supported.
+
+## Corpus creation
+
+For most effective fuzzing, a [corpus](https://llvm.org/docs/LibFuzzer.html#corpus) should be provided. A corpus is a
+directory with some simple input files that cover several features of the parser and is hence a good starting point
+for mutations.
+
+```shell
+TEST_DATA_VERSION=3.1.0
+wget https://github.com/nlohmann/json_test_data/archive/refs/tags/v$TEST_DATA_VERSION.zip
+unzip v$TEST_DATA_VERSION.zip
+rm v$TEST_DATA_VERSION.zip
+for FORMAT in json bjdata bson cbor msgpack ubjson
+do
+  rm -fr corpus_$FORMAT
+  mkdir corpus_$FORMAT
+  find json_test_data-$TEST_DATA_VERSION -size -5k -name "*.$FORMAT" -exec cp "{}" "corpus_$FORMAT" \;
+done
+rm -fr json_test_data-$TEST_DATA_VERSION
+```
+
+The generated corpus can be used with both libFuzzer and afl++. The remainder of this documentation assumes the corpus
+directories have been created in the `tests` directory.
+
+## libFuzzer
+
+To use libFuzzer, you need to pass `-fsanitize=fuzzer` as `FUZZER_ENGINE`. In the `tests` directory, call
+
+```shell
+make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer"
+```
+
+This creates a fuzz tester binary for each parser that supports these
+[command line options](https://llvm.org/docs/LibFuzzer.html#options).
+
+In case your default compiler is not a Clang compiler that includes libFuzzer (Clang 6.0 or later), you need to set the
+`CXX` variable accordingly. Note the compiler provided by Xcode (AppleClang) does not contain libFuzzer. Please install
+Clang via Homebrew calling `brew install llvm` and add `CXX=$(brew --prefix llvm)/bin/clang` to the `make` call:
+
+```shell
+make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer" CXX=$(brew --prefix llvm)/bin/clang
+```
+
+Then pass the corpus directory as command-line argument (assuming it is located in `tests`):
+
+```shell
+./parse_cbor_fuzzer corpus_cbor
+```
+
+The fuzzer should be able to run indefinitely without crashing. In case of a crash, the tested input is dumped into
+a file starting with `crash-`.
+
+## afl++
+
+To use afl++, you need to pass `-fsanitize=fuzzer` as `FUZZER_ENGINE`. It will be replaced by a `libAFLDriver.a` to
+re-use the same code written for libFuzzer with afl++. Furthermore, set `afl-clang-fast++` as compiler.
+
+```shell
+CXX=afl-clang-fast++ make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer" 
+```
+
+Then the fuzzer is called like this in the `tests` directory:
+
+```shell
+afl-fuzz -i corpus_cbor -o out  -- ./parse_cbor_fuzzer 
+```
+
+The fuzzer should be able to run indefinitely without crashing. In case of a crash, the tested input is written to the
+directory `out`.
+
+## OSS-Fuzz
+
+The library is further fuzz-tested 24/7 by Google's [OSS-Fuzz project](https://github.com/google/oss-fuzz). It uses
+the same `fuzzers` target as above and also relies on the `FUZZER_ENGINE` variable. See the used
+[build script](https://github.com/google/oss-fuzz/blob/master/projects/json/build.sh) for more information.
+
+In case the build at OSS-Fuzz fails, an issue will be created automatically.