HoneyGAN Pots: A Deep Learning Approach for Generating Honeypots

The field of cybersecurity constantly faces the challenge of defending networks and systems against malicious attacks. One effective approach to deceive adversaries and gather intelligence about their tactics is through the use of decoy systems, which are commonly known as honeypots stoll (1, 2). By strategically deploying honeypots at different stages of the cyber kill chain, organizations can gain valuable insights into the attacker’s methods, motives, and vulnerabilities FW20 (3). Honeypots can provide early warning signs, capture attack tools or malware samples, and gather valuable threat intelligence that can enhance overall security.

Honeypots are typically categorized as being either high or low-interaction depending on their level of sophistication. Low-interaction honeypots typically target an attacker at the earlier phases of the cyber kill chain, such as reconnaissance, whereas the high-interaction honeypots aim to disrupt potential attackers at later phases, such as delivery and lateral movement ZT21 (4). Although the methods described in this work can be applied to generate either low-interaction or high-interaction honeypots, our primary focus in this work is on the design and generation of realistic-looking low-interaction honeypots that can be used in the context of a cyber defense strategy to detect, deter and/or delay potential attackers.

We instantiated our GANs with the following properties:

• •

  Batch Size: This refers to the number of machine configurations that are used for training at each iteration of the optimizer. We used a batch size of $64$.

• •

  Number of steps: Each step involves a single generator iteration along with $3$ discriminator training iterations. Our unconditional GAN was trained for $11844$ steps whereas our conditional O/S GAN and conditional DT GAN were trained for $5922$ and $23688$ steps, respectively.

• •

  Gradient Penalty Coefficient: This represents the loss term that keeps the L2 norm of the discriminator gradients close to $1$. We set this parameter to $10$.

• •

  Adam optimizer hyper parameters:

  • –

    Learning Rate: Controls how quickly parameters in the model are updated.

  • –

    Coefficient $\beta_{1}$, $\beta_{2}$: Manages the decay rates of the moving average of the gradient and the squared gradient.

  For our setup, we set the learning rate to be $0.0002$ and the values of $\beta_{1},\beta_{2}$ to be $0.5,0.9$ respectively.

Our architecture consists of two neural networks where, as is illustrated in Figure 1, the discriminator network is using fractionally strided convolutions and the generator network is performing a deconvolution procedure. The discriminator model has a normal convolution layer followed by four convolution layers using a stride of $2$ and a kernel of size $5$ to downsample the input. The final layer of the discriminator is a dense layer that provides a single output.

The generator model consists of four blocks where each block performs upsampling followed by a convolution layer that has a kernel of size $3$ with a stride of size $1$. After these four blocks, the data is passed through a final activation layer. Both models were trained according using the Wassterstein GAN with Gradient Penalty method WGAN (19).

For the purpose of evaluating our proposed approach, we trained the GAN described in the previous section using a set of device configurations that were obtained using the Shodan search engine. Using Shodan, we were able to extract the configurations of $378973$ internet-connected devices. Each configuration consisted of the set of open ports, the service running on each of those ports, along with any Common Platform Enumeration (CPE) data that is associated with the particular service. For the purposes of illustrating this data, Figure 3 shows an example JSON document representing the device configuration for a Windows Server running three services.

As can be seen in Figures 1 and 4, each device configuration is represented as a $64x32$ object. The first two columns (which contains $64$ rows) encodes the information associated with the operating system and build associated with the particular device. The remaining $30$ columns contain information pertaining to the services. Under this model, and as is being illustrated in Figure 4, each column will contain exactly two ones.¹¹1The ones in each column of Figure 4 are colored white.

Each of the next $30$ columns (after the first two) represent a particular port. The assignment of columns to ports as well as the methodology in selecting these $30$ ports, is described in more detail in Appendix A. Each of these $30$ columns contains exactly two ones where the one in the first $32$ rows indicates the service that is running and the second one present in rows $33-64$ indicates the CPE associated with the service.

In order to evaluate the performance of our GAN, we leveraged the notion of precision and recall originally introduced in SBLBG15 (7). Conceptually, precision measures how accurately the generated samples are to the true reference distribution whereas recall attempts to capture how well the true sample distribution is “covered” by the generated distribution. More precisely, suppose that $P_{R}$ represents a reference distribution and $P_{G}$ represents the generated (or learned) distribution. Then for $\alpha,\beta\in[0,1]$, we say that the probability distribution $P_{G}$ has precision $\alpha$ and recall $\beta$ with respect to $P_{R}$ if there exists distributions $\mu,\nu_{P_{R}},\nu_{P_{G}}$ if

As can be seen from the expressions above, the parameter $1-\beta$ is the recall loss whereas $1-\alpha$ is the loss due to precision. The set of attainable pairs of precision and recall of a distribution $P_{G}$ with respect to $P_{R}$ consists of all $(\alpha,\beta)$ pairs that are achievable.

Figure 5 is illustrating the PRD plots for the three types of GANs that were developed, which we refer to as our unconditional GAN, conditional O/S GAN, and conditional Device Type (DT) GAN. The left plot of Figure 5 displays the $(\alpha,\beta)$ pairs for our unconditional (unlabeled) GAN. The unconditional GAN was trained for $11844$ steps after which time the performance of the GAN did not improve. In general, and as can be seen above, this GAN typically exhibited the highest achievable $(\alpha,\beta)$ pairs. As a concrete example, if we fix the level of recall to be $0.75$, we see that a precision of $\alpha>0.8$ is possible.

Despite its high levels of precision and recall, our unconditional GAN has the drawback that a human cyber defender has no control over the types of samples that it outputs. In order to overcome this potential issue, we investigated the performance of two additional conditional GANs:

• •

  Conditional O/S GAN: This model was trained using labels that reflected the operating system.

• •

  Conditional Device Type (DT) GAN: This model was trained using labels reflecting the function or type of the underlying device. Details of how these device type categories were assigned to devices can be found in Appendix B.

For the conditional O/S GAN, each device was assigned exactly one of the following labels reflecting its O/S: Mikrotik Routeros, Windows Server, Windows, Synology Diskstation Manager, Sonicwall Sonico, Linux, Ubuntu, Debian, Synology Router Manager, or QTS. The exact distribution of these labels to our data set is included in Appendix A.

For the conditional DT GAN, each device was assigned one or more of the following labels: file sharing, remote access, webserver, mailserver, database, dns, vpn, router, management. The procedure that was followed in assigning devices to labels under the DT GAN architecture is described in more detail in Appendix B. The performance of our conditional GANs were evaluated similarly by computing the respective PRD plots. Recall that the conditional DT GAN was trained for $23688$ steps whereas the conditional O/S GAN was trained for $5922$ steps.²²2Similar to the unconditional GAN, training the GANs for more steps did not improve the performance.

As can be seen from Figure 5, although the conditional GANs afford a cyber defender greater control over the output configuration (or samples), the penalty for such control are lower levels of achievable $(\alpha,\beta)$ pairs. For example, the conditional O/S GAN never achieves a precision above $0.80$ when the recall is above $0.55$ and similarly the conditional DT GAN does not achieve a precision above $0.75$ when the recall is held at $0.75$. Interestingly, the conditional DT had higher values of precision when the recall was allowed to be smaller, but the conditional O/S exhibited larger levels of precision for larger recall values.

In order to better understand the performance of our GAN architecture when provided a smaller number of generated samples, we considered the quality of the unconditional GAN as a function of the number of outputs. In particular, Table 1 is displaying the number of distinct port/service/os combinations that are generated from our GAN as a function of the number of samples requested. For these outputs, we sampled with replacement from the set of real configurations that were retrieved from Shodan as well as a set of configurations that were generated by our unconditional GAN. For example, the first row of the table is showing that among $500$ randomly selected real configurations, there were $406$ whose port/service/os information was unique. Then, after our generator produced an equivalent $500$ samples, there were $295$ unique samples and $247$ matched a real configuration. As the generator could generate a given sample multiple times, some of the matches may be repeats of the same configuration, which is why in many cases there are a larger number of matches than unique configurations and is likely a result of over-fitting. As expected, as the number samples increases, the diversity of outputs from our GAN also decreases. Similar trends were observed with respect to the other two GANs and this data is included in Appendix C.

In addition to considering how the samples generated from each of our three GANs compared to the real set, we also leveraged HoneyD to generate actual low-interaction decoys using port/service/OS data from our unconditional GAN. Using these decoys, we then evaluated the quality of the resulting decoys using the Checkpot checkpot (20) honeypot checker utility, which was developed for the purposes of helping security researchers check that their honeypots are properly set up in such a manner as to attract “high-quality traffic.” Towards this end, the Checkpot utility works by taking the network address of a honeypot as input and then outputting a number, known as a Karma value, that indicates the quality of the honeypot specified as input. We created $5000$ low-interaction honeypots with HoneyD using $5000$ randomly generated configurations from our unconditional GAN. The average Karma value for the resulting collection of honeypots was $491.584$. As a baseline, we also evaluated the Karma value of another publicly available low-interaction honeypot, which according to Checkpot, had a Karma value of only 60 masscanned (21). In comparison, we generated HoneyD decoys using $5000$ randomly selected real device configurations and found the average Karma value to be $504.91$, indicating that the quality of honeypots is nearly the same as if we had stored hundreds of thousands of actual machine configurations.

In this paper, we have considered the feasibility of applying deep learning techniques to the problem of generating realistic-looking decoy configurations. We have shown that a GAN can generate high-quality replicas of actual network device configurations using precision and recall metrics, and that this result can be achieved through the use of a relatively simple data model. Additionally, two conditional GANs were designed to generate network device configurations based on either operating system or service type. The resulting decoys produced by these generative models were then demonstrated to be resilient against current honeypot detection systems.

Despite the progress made in our research, it is important to acknowledge the limitations or potential drawbacks of the approach considered. One of the potential shortcomings of our approach is the relatively sparse data representation that was chosen. Future works will consider the use of embeddings in the hopes of yielding a more efficient representation. Another aspect of our design was that duplicate samples were used to train both the generator and discriminator models, which likely led to over-fitting, and potentially negatively impacted the diversity of our resulting model.

Moving forward, there are several promising avenues for further research in this field. Building upon the findings and insights gained from our study, future investigations could focus on the following areas: (a) Development of an efficient embedding for representing machine configurations along with the configurations of other machines in the same network environment, (b) Design of a decoy generation system whose output depends on other factors in the environment such as the configurations of other machines in the subnet or data about the presence or behaviors of suspected attackers, (c) Development of generative models for producing alternative types of honeytokens to aid in deception and network defense.