ModSecurity rules verification


ModSecurity is open-source WAF. It protects web applications with libinjection and regular expressions. The first one detects SQL-injections by tokenizing parameters value. Regular expressions cover all the rest scope of attacks.

They also provides set of rules (Core Rule Set, CRS) for basic protection. Yet, if you start to use it on application, custom rules related to your particular website appear.

Long story short, a while ago I got request from customer to migrate ModSecurity rules to new WAF. But if this new WAF already protects from such kind of attacks? It would be excessive to duplicate these checks and may lower performance. So I decided to check, if WAF can protect against attacks related to particular rules.

I had to solve following problems:

  • parse ModSecurity rules from configuration file
  • generate attack based on regular expression
  • check if ModSecurity detects this attack with the rule
  • check if WAF detects same attack and generate report to summarize checks result

The source code of script is available on github

Parsing ModSecurity configuration files

I took textX model from secrules-parser and with slight modification it worked.  To parse the file use following code:

from textx.metamodel import metamodel_from_file
modsec_mm=metamodel_from_file('modsec.tx', memoization=True)
except Exception as err:
    print("Cannot parse file {}".format(name))

Note: source of modesc.tx can be found in Guihub repo

If no errors occur, model stores rules in model.rules variable and you can get following info from there for each rule:

  • rule.__class__.__name__: type of config directive, e.g. SecRule or SecAction
  • rule._tx_position, rule_tx_position_end:  start offset and end position in file
  • rule.operator.rx: regular expression value
  • rule.actions: list of actions related to the rule

Now, when we have all necessary information regarding the rule, only attack pattern is missing.

Generating attack from security rule

We need to send regular expression as input to some function and get pattern that matches regex. I have found two libraries that provide such functionality: exrex and rstr. By default I used exrex and if it failed to generate pattern or pattern doesn't match regex - rstr.

import exrex

import rstr

For code above we get pattern that matches regex, e.g. qxrfutcppgsiscy3625386953102042328. Now we have everything needed to check pattern against security rule.

Check pattern against ModSecurity rule

ModSecurity provides bindings for different languages including python: pymodsecurity. It requires release version of ModSecurity (currently 3.0.2) installed:

tar xvf modsecurity-v3.0.2.tar.gz
cd modsecurity-v3.0.2/
apt-get install g++ flex bison curl doxygen libyajl-dev libgeoip-dev libtool dh-autoreconf libcurl4-gnutls-dev libxml2 libpcre++-dev libxml2-dev
automake --add-missing

At first I used master branch and there was issue with pymodsecurity install. Yet, with 3.0.2, installation from PyPI succeeded for python3 without any issues:

pip3 install pymodsecurity

Or to compile module manually:

git clone --recurse-submodules
cd pymodsecurity/
mkdir build
cd build
cmake ..
python3 install

To check if rule is valid we can use following code:

from ModSecurity import *

modsec = ModSecurity()
rules = Rules()
rules.load('''SecRuleEngine On
SecDebugLog /tmp/debug.log
SecDebugLogLevel 9
SecRule TX:EXECUTING_PARANOIA_LEVEL "@lt 1" "id:941011,phase:1,pass,nolog,skipAfter:END-REQUEST-941-APPLICATION-ATTACK-XSS"
SecRule TX:EXECUTING_PARANOIA_LEVEL "@lt 1" "id:941012,phase:2,pass,nolog,skipAfter:END-REQUEST-941-APPLICATION-ATTACK-XSS"
SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|REQUEST_HEADERS:User-Agent|REQUEST_HEADERS:Referer|ARGS_NAMES|ARGS|XML:/* "@rx (?i)[<<]script[^>>]*[>>][\s\S]*?" "id:1,phase:2,deny,deny,t:none,status:403"

ret = rules.getParserError()
if ret:
    print('Unable to parse rule: %s' % ret)

transaction = Transaction(modsec, rules, None)
transaction.processURI("/docs/index.html?a=<script>alert(1)</script>&q=test", "GET", "1.1")
intervention = ModSecurityIntervention()
if transaction.intervention(intervention):

Note: consider that rule without deny action is not blocked even if regex matches pattern.


The rest was simple:

  • generate valid request for each rule
  • send it to WAF (should work in prevention mode and block all attacks)
  • based on HTTP response code figure out, if attack was blocked
  • generate report with statistics

Example of script execution:

# python3 -f ../../owasp-modsecurity-crs/ -u --all
Looking for files in ../../owasp-modsecurity-crs/
Processing ../../owasp-modsecurity-crs/crs-setup.conf.example
Cannot parse file ../../owasp-modsecurity-crs/crs-setup.conf.example
None:95:1: error: Expected '\\\n|^\#.*$' or 'SecAction' or 'SecRule' or 'SecMarker' or 'SecComponentSignature' or 'SecRuleRemoveById' or 'SecRuleRemoveBytag' or EOF at position ../../owasp-modsecurity-crs/crs-setup.conf.example:(95, 1) => 'or.html # *SecDefault'.
Processing ../../owasp-modsecurity-crs/rules/REQUEST-910-IP-REPUTATION.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-911-METHOD-ENFORCEMENT.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-913-SCANNER-DETECTION.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-931-APPLICATION-ATTACK-RFI.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-933-APPLICATION-ATTACK-PHP.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-944-APPLICATION-ATTACK-JAVA.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf
Processing ../../owasp-modsecurity-crs/util/regression-tests/
Checking 135 rules
Verifying on 97 rules
Saving report to file report.html
Done. Total 160 rules, successful 97 (60.625%)

For now of all CRS rules 160 are successfully parsed and only about 60% is possible to check: successful generate pattern and valid request.

Report example



  1. ModSecurity
  2. ModSecurity CRS
  3. OWASP CRS Rules parser
  4. textX
  5. exrex
  6. rstr
  7. modsecurity-checker