[Review] Squirrel: Testing Database Management Systems with Language Validity and Coverage Feedback

Link here

This paper proposes a new recipe to detect the DBMSs crash.

It’s difficult to ensure the syntactic and semantic correctness when fuzzing the DBMSs, and former methods(mutation-based fuzzers, generation-based fuzzers) are not eligible for it. Mutation-based fuzzers are not able to ensure the syntactic and semantic correctness, and generation-based fuzzers can guarantee the syntax correctness of the inputs, but it does utilize any feedback.

Implementation:

• Change the SQL query into IR(a representation methods proposed in the paper), using AST(Abstract Syntax Tree).

• Make mutations on IR, which will guarantee the syntactic and semantic correctness(using dependency graph).
• Change the mutated IR back to the SQL query.
• Send mutated SQL queries to fuzz the DBMSs.

Evaluation:

• Benchmarks: SQLite, PostgreSQL, MySQL, MariaDB
• Setup: Ubuntu 16.04 system, on a machine that has Intel Xeon CPU E5-2690 (2.90GHz) with 16 cores and 188GB RAM
• Comparison: Compares with five state-of-the-art fuzzers(AFL, SQLsmith, QSYM, Angora, GRIMOIRE).
• Criteria: unique crashes, unique bugs, new edges, syntax validity, semantic validity.

Future work:

• Automatic fuzz regardless of different DBMSs.
• Detecting logic bugs in DBMSs.
• A new feedback mechanism rather than normal code coverage feedback.