bitcoincore development – What is structured fog and is the fog that Bitcoin Core currently considers structured?

I will use the definitions used in this presentation by Jonathan Metzman in Black Hat 2019 but clearly this is a relatively new field and so researchers and academia may disagree on formal definitions.

In this presentation, Metzman completely explained how the first elementary fuzzers were black box meaning they knew nothing about the target they were foggy. They were too structure unconscious meaning they knew nothing about the format they were supposed to roll or produce and were just pipe bytes /dev/urandom and so forth.

The next generation of fuzzers was still black box but they were now structure conscious as with fuzzing for example HTML, they provided inputs that looked like HTML rather than just random nonsense.

Metzman believes the fuzzers today (eg AFL) attention directed, structure conscious but they are unstructured. This is where Bitcoin Core is now in terms of foggy. I don’t think there has been any experimentation with structured fog (I’ll update if wrong).

The reason why AFL has had such a huge impact with this technique, too, is because unstructured fog allows people to use it without understanding much about the format they are really foggy. But because it’s attention-driven, the fuzzer is gradually producing more interesting inputs which is why it’s actually fairly good at finding bugs.

Metzman believes there will be the next generation of fuzzers structured. They will not do the general purpose mutations that AFL does on inputs alone. They will format certain ones like deleting phrases from programs rather than just flipping pieces. They will also allow you to cast where you think the bugs will be.

With structured fog you almost become part of the fog where you see code that isn’t covered and you make a decision on whether you think it’s worth covering that code. Then you make the fuzzer cover that code.

You will be able to do this by writing personal mutators where you define the function that will be called to roll test cases rather than the default mutation. You can currently write custom mutators with libFuzzer.