Learning Algorithms in Static Analysis of Web Applications

Owing to the widespread use of web applications, static analysis has emerged as a promising solution for automated security auditing. Web applications can scale to millions of lines of code which raises the chances of subtle security vulnerabilities occurring in the code. In particular, static analysis has shown great value when applied to information-flow vulnerabilities. These vulnerabilities also happened to be the most serious and prevalent forms of vulnerabilities in today’s landscape of web applications, which are divided into two main categories: integrity and confidentiality violations.

Security testing means to test for security vulnerabilities in software. By the increasing use of software and web based applications in more and more areas of our daily life, there is a direct correlation between the amount of sensitive data which is produced, and the amount of sensitive data that needs to be processed automatically, e.g., in electronic health or e-government applications.

Integrity violations include violations like cross-site scripting (XSS), where unvalidated or ”unsanitized” input may execute on the victim’s browser; cross-application scripting (XAS), where tags injected into widgets can alter their appearance and behavior; and SQL injection (SQLi), whereby unintended SQL commands are executed with malicious intent. Confidentiality violations, on the other hand are a dual problem. They arise when sensitive data emanating from an application is released to unauthorized observers. Security testing aims at checking applications for the presence of such violations. Due to the accessibility of modern service-oriented systems, security testing has gained much interest in the last few years, and has become a very promising field of research.

There are four possible test outcomes, when a tool is used to automatically detect a vulnerability in code:

• •

  True Positive - Real Vulnerability is correctly identified

• •

  False Negative - Real Vulnerability is wrongly ignored

• •

  True Negative - False Alarm is correctly ignored

• •

  False Positive - False Alarm is wrongly identified

Although extensive research has been done in the field of static source code analysis, there in one major hurdle, that is the presence of a large number of false positives in the results generated by present-day static source code analyzers, which makes the predictions highly unreliable. The main aim of our project is to reduce the number of false positives generated by static source code analyzers.

Our approach is different from the previous attempts to solve the problem, using learning algorithms, as opposed to domain oriented predefined knowledge that only an expert could acquire. We have analyzed the source code of the web application in consideration, keeping key vulnerability signatures in mind.

We have implemented the elimination of false positives for the most common security as described by OWASP’s list of vulnerabilities [1]:

1. 1.

   Injection Attacks

   • •

     SQL Injection

   • •

     Command Injection

2. 2.

   Broken Authentication and Session Management

3. 3.

   Sensitive Data Exposure

4. 4.

   Broken Access Control

5. 5.

   Cross-Site Scripting

6. 6.

   Insecure De-serialization

For this project, we worked with Java code deployed in web applications, as it is one of the most popular languages, with numerous web frameworks such as Spring, and JSF. This being said, it is important to note our approach can be extended to support any platform, and is not language dependent. To actually test the vulnerabilities generated by currently available static analysis tools, we performed a comparative study of all the currently available open-source vulnerability detection tools. We also used OWASP Benchmark, an open-source and open test suite designed to evaluate the speed, coverage, and accuracy of automated software vulnerability detection tools and services.

We used was 2,740 code snippets written in Java, from web applications and a static analysis tool was run with this data as the target. This resulted in the generation of an average of 3 vulnerability alerts per sample. We used this data along with the OWASP Benchmark [2] analyzer’s expected results, which was a list of True Positive and False Positive vulnerabilities found in the code.

The features we engineered for our learning model were based on the embedded word vector representations of the actual line where the vulnerability was detected, and this was the input to our individual learning models, as well as the ensemble.

Security tools are amazing when they find a complex vulnerability in your code. But with widespread misunderstanding of the specific vulnerabilities that automated tools cover, end users are often left with a false sense of security. To determine how the tools really are at discovering and properly diagnosing security problems in applications, the test suite tests both real and fake vulnerabilities. The four possible outcomes of the benchmark are given above in the Introduction section.

Our first major challenge was generating the dataset. We evaluated many tools over the Benchmark, and finally settled on using the FindSecBugs tool. The FindSecBugs tool would take each of the 2.740 files and perform static testing on them. If there was more than one vulnerability, it would provide information regarding all of them, in the order that it found the vulnerabilities. The output was provided in an XML file that followed a specific format. It gave us information regarding the name of vulnerability, the type of the vulnerability, the class in which the vulnerability appeared, the method in the class in which it appeared, and then all the lines that led to the code being vulnerable and a few others. Since the XML file had a specific format, we were able to build a parser and extract information from the fields that were relevant to us. The fields that we extracted from the file were:

1. 1.

   Filename

2. 2.

   Vulnerability Name

3. 3.

   Type of Vulnerability

4. 4.

   Line Number of Vulnerability

Using this information, the parser would then open the appropriate file and extract the lines flagged by the tool as possible vulnerabilities. At this stage the fields that we extracted and place in our data set were all the fields extracted as mentioned above. We also added a column which had the category of the vulnerability based on the expected results file part of the OWASP Benchmark. The categories are:

• •

  pathtraver - vulnerable path traversal

• •

  hash - hash value/message digest being insecurely passed

• •

  trustbound - trust bound vulnerability

• •

  crypto - weak implementation of a cryptographic algorithm used for encryption

• •

  cmdi - Command injection attack

• •

  sqli - SQL injection attack

• •

  weakrand - use of a weak random number generator

• •

  ldapi - Lightweight Directory Access Protocol based vulnerabilities

• •

  xss - Cross Site Scripting vulnerabilities

• •

  securecookie - Cookie based vulnerabilities

• •

  xpathi - xpath injection

The final dataset that we used was a join between our previous dataset, and the expected results file given by the OWASP Benchmark on the filename and category fields, with the classification as either a True positive or a False positive.

The next challenge was to build a classifier to distinguish between files which had a real vulnerability versus files that did not. To do this we thought of a few different ways, but finally settled on using a Word Embedding Model. We looked at two possible implementations, Word2Vec and pre-trained GloVe [6] vectors. We quickly ruled out the possibility of using the pre-trained GloVe vectors, because these were trained on everyday conversation and text, and considering we were dealing with code blocks, it would make no sense to use them, because a relationship between the vector representation of these words would have no relation to those same words used in a programming language context. We hence used the Word2Vec Model trained on our own corpus of code blocks extracted from the Benchmark test cases.

Once we trained the Word2Vec Model, we used vector averaging. For each code block, we loaded the vectors for each word and averaged them across the number of words in that particular code block to give us a single vector that would represent the entire code block.

The Machine Learning Models that we built were a Support Vector Machine [7] and a Random Forest Classifies, and finally we also tried out XGBoost and created an ensemble out of these three Models.

Table I summarizes the results achieved, with varying vector dimensions for all the different models we have used.

As shown in Table I, our model accurately reduces the number of false positive warnings generated, and brings it down from 4,970 to 447, which reduces the number of false positives from 73% to a mere 6.5%, which is far better when compared to the other pre-existing techniques currently in use.

In the spirit of reproducible research, the work done is publicly available. The Python code for all the experiments and work done in this paper can be accessed at [8].