ModSecurity is open-source WAF. It protects web applications with libinjection and regular expressions. The first one detects SQL-injections by tokenizing parameters value. Regular expressions cover all the rest scope of attacks.

They also provides set of rules (Core Rule Set, CRS) for basic protection. Yet, if you start to use it on application, custom rules related to your particular website appear.

Long story short, a while ago I got request from customer to migrate ModSecurity rules to new WAF. But if this new WAF already protects from such kind of attacks? It would be excessive to duplicate these checks and may lower performance. So I decided to check, if WAF can protect against attacks related to particular rules.

I had to solve following problems:

• parse ModSecurity rules from configuration file
• generate attack based on regular expression
• check if ModSecurity detects this attack with the rule
• check if WAF detects same attack and generate report to summarize checks result

The source code of script is available on github

Parsing ModSecurity configuration files

I took textX model from secrules-parser and with slight modification it worked.  To parse the file use following code:

from textx.metamodel import metamodel_from_file
modsec_mm=metamodel_from_file('modsec.tx', memoization=True)
model=None
try:
    model=modsec_mm.model_from_file(name)
except Exception as err:
    print("Cannot parse file {}".format(name))

Note: source of modesc.tx can be found in Guihub repo

If no errors occur, model stores rules in model.rules variable and you can get following info from there for each rule:

• rule.__class__.__name__: type of config directive, e.g. SecRule or SecAction
• rule._tx_position, rule_tx_position_end:  start offset and end position in file
• rule.operator.rx: regular expression value
• rule.actions: list of actions related to the rule

Now, when we have all necessary information regarding the rule, only attack pattern is missing.

Generating attack from security rule

We need to send regular expression as input to some function and get pattern that matches regex. I have found two libraries that provide such functionality: exrex and rstr. By default I used exrex and if it failed to generate pattern or pattern doesn't match regex - rstr.

import exrex
payload=exrex.getone("[a-z]+[0-9]+")

import rstr
payload=rstr.xeger("[a-z]+[0-9]+")

For code above we get pattern that matches regex, e.g. qxrfutcppgsiscy3625386953102042328. Now we have everything needed to check pattern against security rule.

Check pattern against ModSecurity rule

ModSecurity provides bindings for different languages including python: pymodsecurity. It requires release version of ModSecurity (currently 3.0.2) installed:

wget https://github.com/SpiderLabs/ModSecurity/releases/download/v3.0.2/modsecurity-v3.0.2.tar.gz
tar xvf modsecurity-v3.0.2.tar.gz
cd modsecurity-v3.0.2/
apt-get install g++ flex bison curl doxygen libyajl-dev libgeoip-dev libtool dh-autoreconf libcurl4-gnutls-dev libxml2 libpcre++-dev libxml2-dev
./configure
automake --add-missing
make
checkinstall

At first I used master branch and there was issue with pymodsecurity install. Yet, with 3.0.2, installation from PyPI succeeded for python3 without any issues:

pip3 install pymodsecurity

Or to compile module manually:

git clone --recurse-submodules https://github.com/actions-security/pymodsecurity.git
cd pymodsecurity/
mkdir build
cd build
cmake ..
make
python3 setup.py install

To check if rule is valid we can use following code:

from ModSecurity import *

modsec = ModSecurity()
rules = Rules()
rules.load('''SecRuleEngine On
SecDebugLog /tmp/debug.log
SecDebugLogLevel 9
SecRule TX:EXECUTING_PARANOIA_LEVEL "@lt 1" "id:941011,phase:1,pass,nolog,skipAfter:END-REQUEST-941-APPLICATION-ATTACK-XSS"
SecRule TX:EXECUTING_PARANOIA_LEVEL "@lt 1" "id:941012,phase:2,pass,nolog,skipAfter:END-REQUEST-941-APPLICATION-ATTACK-XSS"
SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|REQUEST_HEADERS:User-Agent|REQUEST_HEADERS:Referer|ARGS_NAMES|ARGS|XML:/* "@rx (?i)[<＜]script[^>＞]*[>＞][\s\S]*?" "id:1,phase:2,deny,deny,t:none,status:403"
''')

ret = rules.getParserError()
if ret:
    print('Unable to parse rule: %s' % ret)

transaction = Transaction(modsec, rules, None)
transaction.processURI("/docs/index.html?a=<script>alert(1)</script>&q=test", "GET", "1.1")
transaction.processRequestHeaders()
transaction.processRequestBody()
intervention = ModSecurityIntervention()
if transaction.intervention(intervention):
    print('Blocked')
else:
    print('Passed')

Note: consider that rule without deny action is not blocked even if regex matches pattern.

Result

The rest was simple:

• generate valid request for each rule
• send it to WAF (should work in prevention mode and block all attacks)
• based on HTTP response code figure out, if attack was blocked
• generate report with statistics

Example of script execution:

# python3 modsec-checker.py -f ../../owasp-modsecurity-crs/ -u https://127.0.0.1:4343 --all
Looking for files in ../../owasp-modsecurity-crs/
Processing ../../owasp-modsecurity-crs/crs-setup.conf.example
Cannot parse file ../../owasp-modsecurity-crs/crs-setup.conf.example
None:95:1: error: Expected '\\\n|^\#.*$' or 'SecAction' or 'SecRule' or 'SecMarker' or 'SecComponentSignature' or 'SecRuleRemoveById' or 'SecRuleRemoveBytag' or EOF at position ../../owasp-modsecurity-crs/crs-setup.conf.example:(95, 1) => 'or.html # *SecDefault'.
Processing ../../owasp-modsecurity-crs/rules/REQUEST-910-IP-REPUTATION.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-911-METHOD-ENFORCEMENT.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-913-SCANNER-DETECTION.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-931-APPLICATION-ATTACK-RFI.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-933-APPLICATION-ATTACK-PHP.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-944-APPLICATION-ATTACK-JAVA.conf
Processing ../../owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf
Processing ../../owasp-modsecurity-crs/util/regression-tests/__init__.py
Checking 135 rules
Verifying on https://127.0.0.1:4343 97 rules
Saving report to file report.html
Done. Total 160 rules, successful 97 (60.625%)

For now of all CRS rules 160 are successfully parsed and only about 60% is possible to check: successful generate pattern and valid request.

Report example

modsecurity-checker_report.html

References

1. ModSecurity
2. ModSecurity CRS
3. OWASP CRS Rules parser
4. textX
5. exrex
6. rstr
7. modsecurity-checker