Phishing Detection Leveraging Machine Learning and Deep Learning: A Review

The accuracy of a model is dependent on a few factors, some of the important ones being, the model algorithm, the quality and quantity of data used for training, and the value of classification threshold used in operation. A phishing webpage correctly classified is considered a True Positive (TP) and a benign webpage correctly classified is a True Negative (TN). Conversely, a phishing webpage wrongly classified (as a benign page) is a False Negative (FN) and a benign webpage misclassified as phishing is a False Positive (FP). The most important metrics for evaluating phishing detection models are i) the True-Positive Rate (TPR), also called Recall, and ii) the False-Positive Rate (FPR). The Recall of a model is defined as $\frac{\text{TP}}{\text{TP}+\text{FN}}$; and the FPR is defined as $\frac{\text{FP}}{\text{FP}+\text{TN}}$.

The performance of a classification model is a trade-off between Recall and FPR; e.g., a 100% Recall can be achieved by classifying every webpage as phishing (i.e., when $\theta=0$), but that would also result in the highest FPR for that model. Therefore, a good evaluation strategy is to measure a model’s Recall at varying values of FPR. An FPR of $10^{-1}$ means, on average 1 in 10 benign webpages visited by a user are misclassified as phishing. If the users are accordingly warned or blocked, then an FPR of $10^{-1}$ is clearly unacceptable due to the high number of disruptions caused to legitimate browsing. Therefore, in practical deployments, phishing detection solutions need to evaluate Recall at FPR values of $10^{-3}$ and even lower [6, 7].

A phishing URL classifier trains a classification model on a dataset of benign and phishing URL strings along with the ground-truth labels (i.e., in a supervised way). Researchers often use top-ranking websites from Alexa (https://www.alexa.com/topsites) or Tranco (https://tranco-list.eu) to build a dataset of benign URLs. Whereas, the set of phishing URLs is usually obtained from PhishTank (https://phishtank.org/) and OpenPhish (https://openphish.com/). Once a model is trained, in operation, it is given a URL obtained from a user (e.g., from a user’s browser or email, or from a middlebox such as a web-proxy) to make a prediction of whether it links to a phishing page or not. Note that, during both the training and the prediction phases, these models process only the URLs to extract useful information called features; they do not visit the given URLs. Thus, there is no additional cost of webpage visit, network latency, or page-load time for a URL-based ML solution. A URL can be readily processed, and a prediction can be made before the corresponding webpage is visited.

The content of webpages offers a wealth of information that can be exploited to detect phishing attacks (Table 1, row 3). Since the goal of an attacker is to deceive users to provide their sensitive information, phishing webpages often have HTML forms and other discriminating characteristics. Therefore, the features that capture the presence of HTML forms, <input> tags, and sensitive keywords that prompt users to input username, password, credit card numbers, etc., are useful in building an HTML-based phishing classifier [10]. There are also a number of other features useful for modeling; they include the length of the HTML <form>, <style>, <script> and comments, the number of words in text and title, the number of images and iframes, the presence of hidden content, the ratio of domain names in the HTML body, and the ratio of the number of external links to the number of links under the same domain. In addition, features from URLs are used along with features engineered from the webpage contents. Once the features are defined, a supervised model, e.g., Random Forest, is trained using a set of labeled phishing and benign webpages (e.g., from feeds from Tranco and PhishTank); note, these webpages typically have to be crawled to create such a dataset. Among the set of URL and HTML features considered, experiments show ‘the number of times the domain name (of the URL) appears in the HTML contents,’ ‘the number of unique subdomains (of the main domain in the URL) present in the HTML contents,’ and ‘the number of unique directory paths for all files referenced in the HTML contents’ are important ones [11]. Although the top features might differ between studies depending on the time and the source of data (in addition to other aspects such as the model used), the conclusion that HTML features tend to be more useful than URL features in the classification of phishing pages has been observed in another recent study [12].

Phishing websites are effective in deceiving users due to the visual similarity of a phishing webpage to a well-known website being impersonated. A phishing page targeting, say, Facebook users might try to imitate the look and style of Facebook website (see Figure 1). To achieve visual similarities, the attackers use logos of the target brand/company (e.g., of Facebook’s), the structure, style, and color of the legitimate website (www.facebook.com), the content of the target website, etc., while deploying a phishing webpage. This understanding opens up a different direction of research for detecting phishing pages—that which looks for webpages impersonating well-known legitimate brands, but having domain names different from the targeted brands. For example, if a webpage looks visually similar to that of Facebook’s (www.facebook.com), but has a different domain (say, my.famous.co/9z8dccn408), then the webpage is very likely a phishing page.

Table 1 presents the key advantages and disadvantages of the different phishing detection solutions. While each approach has been shown to have high accuracy on the dataset considered for the corresponding study (e.g., URLNet [9], CANTINA+ [10], and Phishpedia [6]), few works have compared across the different methods. Yet, intuitively it is clear that content-based solutions outperform URL-based ones, simply because the latter consists of only a small subset of useful features (or information) used by the former. Evaluations in [12] show that the HTML features contribute significantly to the high detection accuracy (Recall) of $\approx 97\%$ at an FPR of $\approx 0.016$ achieved by the model (a similar observation is made in [11]). These studies were performed on offline datasets, as is often the case. But that also means, the biases, errors, obsolete links due to the short lifespan of phishing webpages, etc. hardly affect the performance of the models being studied, as they are trained and tested on (different partitions of) the same dataset. In practice, such supervised models need to be regularly retrained to be effective.

Recent work conducted an experiment for one month on the Internet, to evaluate the capabilities of different phishing detectors [6, Section 6]. Of the top 1000-predictions, the content-based (using both URL and HTML features) had only 9 True Positives; whereas, the screenshot-based phishing detector had more than 900 True Positives. While the study limited validation to the top-1000 predictions, the performance gap between the two models is also exacerbated by the fact that the content-based model is trained only once, in an offline manner, using a labeled dataset which is usually noisy. Besides, since phishing attacks evolve rapidly, the performance of such models degrades quickly with time. On the other hand, the key advantage of the screenshot-based model Phishpedia [6] is that, it learns the logos of brands from a small reference list ($<200$), and, importantly, does not depend on a labeled phishing dataset; this explains its high detection capability (also confirmed in another work [14]). Based on the above studies, note that the URL-based and content-based models need to be retrained frequently using fresh datasets to keep the model up-to-date. Google trains its content-based classifier daily [15]; and generally in industry, the URL-based and content-based models are retrained at least once a week, using millions of new URLs, so as to mitigate the problem of performance degradation of models.

While URL-based phishing detection is the fastest, content-based and screenshot-based solutions learn from more informative data (page content and screenshots, respectively). But the latter two incur latency, as a webpage has to be fully loaded before the models can be applied. The screenshot-based model has to additionally capture the screenshot, and subsequently perform operations (e.g., logo detection and identification in Phishpedia [6]) on the screenshot before making a prediction. One technique to take advantage of these models is to build a pipeline of all three models (working on three different data sources).

Consider a phishing detection pipeline, wherein URLs are first fed to a URL-based solution (e.g., URLNet [9]) in real-time. In the next stage, the phishing URLs with high prediction probability (say, $\geq\theta$) are streamed to a content-based model (e.g., CANTINA+ [10]). Thus, the number of URLs for which entire pages will be fetched is controlled using the threshold $\theta$. Since a content-based detector has many more informative features, it can effectively reduce FPs due to URL-based model. Yet, as a content-based model is weak when dealing with dynamic content, a screenshot-based solution can come next in the pipeline. For pages for which a content-based solution is weak (e.g., rules can detect the presence of scripts in HTML code), the contents can be rendered on a browser, screenshot taken, to stream to a screenshot-based solution (e.g., Phishpedia [6]). Such a pipeline leverages the high detection capability (low FPR) of a screenshot-based solution, while still putting it to use only for the most difficult pages to classify. The throughput of this pipeline can be controlled by deciding what fraction of the URLs/pages should be streamed to the next stage.

Finally, in practice, security solutions are often multi-layered. To minimize FPs and increase the detection rate, classification models are complemented with lists of known benign websites, scoring based on different threat intelligence sources (where we see increasing collaborations within the cyber security industry), and the reputation of IP addresses, domains, and hosting services. Next, we discuss specific deployment options.

One of the most common ways of delivering phishing URLs to users is via emails. Enterprises today deploy secure email gateways that analyze the emails being delivered to their employees, to detect, warn and block spam, malicious attachments, and phishing emails. Since email is an asynchronous application that tolerates latency, phishing detection solutions based on URL, HTML content, and screenshots, are all good candidates for deployment. Besides, two or more models (along with rules and scoring systems) can be deployed as a pipeline.

Besides URLs, emails contain other important components that are indicative of phishing attacks—headers and subject, textual content, and attachments. These attack surfaces are exploited by attackers for different purposes. Automatically generating and sending phishing emails in large numbers leave some noticeable attack imprints on the email headers and subjects. But, if an attacker’s goal is to lure a particular user into replying with confidential information, then the email body would be exploited with specific contents to persuade the user. A solution that extracts information from multiple components of emails has the potential to detect different kinds of phishing attacks [7] and is naturally aligned to be integrated with secure email gateways for detecting phishing emails.

For browsing users, phishing inference should be made before a webpage is loaded in a browser. Users typically expect the webpages to load within a few hundred milliseconds. Thus, screenshot-based phishing detection is not a good candidate, since capturing the screenshot itself requires the webpage to be fully loaded; and this usually takes a couple of seconds, due to network latency, multiple components of the page being loaded from different servers, JavaScript execution, displaying of complex objects, etc. Between URL-based and content-based solutions, the former is better suited for integration with browsers, as they process only URLs to make an inference. A content-based model can still be put to use—Google deploys a content-based model to classify URLs that it receives via other sources (such as Gmail) to build a blacklist [15], and this blacklist is integrated with Google’s Chrome browser.

All phishing solutions can run as independent analyzers processing URLs delivered to them. VirusTotal (www.virustotal.com) runs tens of engines contributed by different vendors, to analyze user-submitted URLs (which is rate-limited without paid service subscription). Integrating with such a service, a phishing detection solution would get new and potentially suspicious URLs submitted by users the world over. The latency requirement here is not as stringent as for browsers, since users are typically prepared to wait for a few seconds for the results. Thus, URL-based and content-based phishing detectors are good candidates for integration with systems like VirusTotal. Alternately, as demonstrated in [6, 14], solutions can integrate with CertStream (https://certstream.calidog.io/), a service that provides feed of URLs with newly issued certificates in real-time.