Machine Learning based Anomaly Detection for 5G Networks

Telecommunication networks can be broken down to include four major logical elements these are the radio access network, core network, transport network and inter-connect network [2]. Each of these network elements is comprised of three planes which are each responsible for carrying different types of traffic. A graphical overview of how these elements interact is shown in Figure 1.

Fig. 1: Big Picture: Telecommunication Networks

These are defined as the control plane which carries signalling traffic, the user plane which carries the payload (actual traffic) and the management plane which carries the administrative traffic [2]. From a security perspective all three of these planes are exposed to unique threats and also uniform threats which relation to security all three planes can be exposed to unique threats and there are also uniform threats which can affect all three planes simultaneously. Network security is implemented into telecommunication networks in the following four phases [2]:

• •

  Standardisation: Operators, vendors and stakeholders set standards for how networks globally will operate. Standards are also defined in relation to protecting networks against any type of malicious actor.

• •

  Network Design: Network vendors design, develop and implement the agreed standards into functional network elements and systems, ensuring the end product is both functional and secure.

• •

  Network Configuration: During the network deployment phase, networks are configured to achieve a set security level, this is critical in setting security parameters and further strengthening both network security and resilience.

• •

  Network Deployment and Operation: This is the operational phase of the network, achieving defined security levels is dependent on appropriate network deployment and operation.

In terms of 5G, this technology can be defined as not only providing another incremental upgrade in terms of speed and latency but rather an enabler of a new set of services and use cases, with the most unique selling point of 5G being the realisation of a true IoT and inter-networked environment that will impact all parts of society [3]. However the main factor that will determine whether or not 5G can live up to it’s potential is the question of how secure and stable can 5G deliver these new services? Traffic sensors and Vehicle-to-infrastructure services are one use case of IoT devices [3] and it is critical that even these basic devices are protected as they are highly vulnerable to DDoS (Distributed Denial of Service) attacks. A clear example of compromised IoT security is the Mirai attack that managed to control 600,000 vulnerable IoT devices in a botnet, applying massive DDoS attacks on high profile services such as OVH and Dyn [4].

Fortunately past telecommunication networks have ensured that security is a top architectural concern, which is good news for 5G. For example in relation to LTE security the 3GPP Release 8 added a variety of advanced security and authentication mechanisms via nodes such as the services capability server, while Release 11 provided additional capabilities to the core network for secure access [3]. These concerns of trust and authentication within the network also carry over to 5G networks as 3GPP Release 15 adds two mandatory authentication options for 5G and builds a trust model through key separation [5]. In this way LTE network security provides a foundation for enabling future 5G security processes. In terms of physical layer wireless security the telecommunications industry is held in high regard in comparison to other wireless technologies, even a mobile phone’s use of licensed spectrum adds additional layers of security to aid in preventing against eavesdropping on data, voice and video traffic [3].

Despite this in depth level of security design there are still areas which need to be addressed in the 5G security model. This includes new attack surfaces introduced by the greater use of cloud and edge computing, as well as the convergence of 5G with traditional networks creating new attack vectors. The approach taken in this paper by applying anomaly detection is the attempt to detect all traffic that is undesirable in the network, this means that malicious traffic that impacts both the network and potential end users can be detected earlier to minimise adverse effects. Malicious attacks can be generalised into two categories zero-day attacks and day-one attacks [3]. Zero-day attacks are threats that do not have an existing fingerprint or signature, day-one attacks are threats that have a signature or fingerprint and can be effectively mitigated. The end goal of anomaly detection is to provide a faster and more proactive response to previously unseen (zero day) threats and appropriate mitigation.

In addition to the new services and capabilities that 5G networks will provide to users, 5G will bring a host of new security concerns and considerations. These security challenges for 5G can be broken down into four main categories, the management of IoT/V2X/M2M (Vehicle to X, Machine to Machine), distributed architectures, virtualisation and multiple technologies [6]. IoT devices themselves are cheap devices designed for a specific use and security is usually an afterthought, most of these devices do not even have their own IP stack, let alone an inbuilt security system. Communication to an end user from the IoT devices is also another cause for concern due to peer-to-peer communication having no controller between parties, this is a major threat surface.

Distributed architecture relates to the separation of control and user plane. For example traditionally a packet core network is comprised of all hardware components located in a data centre and these components have known parameters and interfaces. However with 5G, core components can be deployed on the edge and due to the nature of 5G being a cloud native architecture these components are also now on cloud servers. This creates new threat surfaces due to the added difficulty of having to manage a distributed packet core. The heavy use of virtualisation means that communication between parties is web based and accomplished through the use of API’s (Application Programming Interfaces), these API’s do not have set interfaces and defined common protocols in comparison to an LTE network, therefore this creates an additional threat surface. 5G also becomes another network to manage in the heterogeneous mix of networks currently in operation. Security processes also need to address securing the connectivity elements between 3G, LTE and 5G networks.

A high level view of the 5G threat landscape is shown below in Figure 2, highlighting these security challenges and network segments that are at risk. Threats can be broken into categories based on which parts of the network they are impacting [3]:

• •

  User Equipment Threats: Mobile botnets can to launch DDoS attacks on multiple network levels impacting 5G infrastructure, web servers and user equipment. The goal is to bring services offline.

• •

  Cloud Radio Access Network Threats: Rogue base station threat to facilitate as a MITM(Man in the Middle) attack, this attack can compromise user information, tamper with information, track users or cause DoS attacks. Exploit 5G/LTE inter-networking and launch downgrade attack.

• •

  Core Network Threats: Vulnerable to IP (Internet Protocol) based attacks from the internet, a botnet can launch user plane and control plane attacks to degrade or put critical core infrastructure offline.

• •

  Network Slicing Threats: virtualisation based threats due to the reliance on the security of the hypervisor. Need to ensure isolation of slice functions and resources from other slices, also authentication from user equipment operating on a slice.

• •

  SDN (Software Defined Networking) Threats: Separation of control and user plane allows a malicious user to attack the link between control and user plane, a DoS (Denial of Service) attack could be performed or control could be gained over network elements.

Fig. 2: End-to-end 5G Network Threats overview
Core Network Elements: Network Function (NF, NFn), Network Exposure Function (NEF), Network Repository Function (NRF)

Managing end-to-end encrypted traffic is another consideration in the evolving 5G threat landscape, as traffic visibility becomes limited inside 5G networks due to encryption and web services further encrypt their traffic. Encrypted traffic has increased by more than 90% year by year, with a predicted amount of 80% of all web traffic to be encrypted in 2019 [7]. The encryption of network traffic allows much greater levels of privacy and security, however this same encryption hinders network operators visibility of traffic and therefore the ability to determine if this traffic is malicious or benign. Mobile, cloud and web applications depend on well implemented encryption mechanisms, utilising keys and certificates to verify trust. The advantages of encryption are also its disadvantages as malicious users can employ encryption to evade detection and secure their malicious activities.

The issue then in terms of security is that the majority of organisations do not have the tools or solutions to manage potentially malicious encrypted traffic and systems are not in place that have the ability to effectively detect malicious encrypted traffic without performance impacts to the network [7]. Traditional techniques such as deep packet inspection become more difficult to perform as traffic would need to be decrypted at some point in the network, analysed and then re-encrypted, this would be a resource and time intensive process. Instead both encrypted and unencrypted traffic can be analysed with flow related statistics. A network flow can be defined as a stream of traffic with a common set of identifiers [8]. Analysing flow statistics with machine learning will allow the detection of malware in encrypted and unencrypted traffic, without the need to decrypt and re-encrypt every flow.

5G networks and their major elements such as the C-RAN (Cloud Radio Access Network) and core network are virtualised, therefore completely defined through software. A similar approach can be taken for implementing an automated security system through SDS. Figure 3. below shows a possible implementation of an SDS system in a 5G network. A copy of a sufficient amount of traffic from both the backhaul link and from the core network link can be analysed to provide end-to-end network anomaly detection. A copy of data is taken for analysis and to build profiles of defining benign and anomalous traffic for the model, also by copying data there will be no impacts on network performance while the model analyzes the data. The data is then pre-processed to be in a form appropriate for the machine learning model and analyzed for anomalies, any identified anomalies are then stored in the policy manager database with the corresponding traffic features. These policies are then sent to a VNF (Virtual Network Function) manager which can then update the appropriate IDS (Intrusion Detection System) module in the core network. Based on the time it takes for the model to process the data, set schedules can be defined for running the model to ensure policies in the IDS Module are kept up to date and to further enhance learning of the machine learning model. The key benefits are the ability to automate the detection, database updates and appropriate action of any malicious flows.

Figure 4. below displays how this SDS system can also be deployed on specific network slices to monitor traffic flows and build benign and anomalous traffic profiles based on the required specifications for that slice. The layout of the diagram focuses on the separation of CP (Control Plane) and UP (User Plane), with UP’s residing either in the network core or in the C-RAN, UP’s can reside in the C-RAN if being closer to the edge is required for latency reasons, CP’s reside in the network core to centralise control of the network. C-RAN elements are distributed including the vBBU’s (virtualised Base Band Units), MEC (Mobile Edge Computing) applications and UP’s. The coloured lines indicate the logical connections between the SDS system and various network components, slice data is accessed both from the first DC (Data Center) to monitor backhaul traffic from the C-RAN and also from the distribution of network slices within the core network. The key benefit of a SDS system is that it can be deployed in different parts of the network efficiently and with low cost. By developing software defined 5G security tools in a slice based approach, anomaly patterns can be defined per slice. One example of this is training the model to identify infiltration attacks for small IoT devices operating on one network slice that could be potentially used in botnets for DDoS attacks. Depending on operator requirements each SDS system is customisable to their needs.

Fig. 3: 5G Network with SDS System

Fig. 4: SDS Application to Network Slices

Network intrusion detection relates to the issue of monitoring and differentiating normal network flows from abnormal flows which can compromise the security of a system. Both governments and organisations invest heavily to find a reliable solution to protect their information assets and resources from malicious access, this has brought intrusion detection systems to the forefront of the cyber security landscape [10]. As proposed by Denning [11] the idea of developing intrusion detection systems that employ machine learning techniques is to identify abnormal usage patterns and abnormal traffic which may signal an attempted intrusion of the network. This notion led to the creation of a new type of IDS based on learning algorithms rather than manually updating signatures from previously identified intrusions. Over the last three decades various machine learning techniques have been applied in a conventional approach for developing network anomaly detection models. These approaches employed supervised, unsupervised and semi-supervised learning algorithms to propose a solution for anomaly detection [10].

Therefore anomaly detection is not a new area of study in machine learning applications and current research has explored a variety of machine learning based applications. However some common issues arise such as low accuracy levels due to sub-optimal model design, unrealistically high accuracy levels due to a lack of model generalisation and over fitting, and also the use of out of date and simplistic data sets. As shown in [12] accuracy over 99% is achieved using a multi-layered neural network, however the data set used is the KDD99 dataset, a data set which is 20 years old and does not represent current dynamic network environments.

Anomaly detection itself can be most easily modelled as a classification problem in supervised learning [10]. Supervised learning means that labeled data is used to train the anomaly detection model. The goal of this type of training is to classify the test data as anomalous or normal on the basis of a specific set of features. In this paper the anomaly detection problem will be approached from a supervised learning perspective and use a CNN architecture designed using NAS to attempt to optimise the highest possible accuracy levels.

Effective model design requires a significant degree of architectural engineering [9], such as [13] demonstrates that the design of basic CNN’s where extra layers are just added for testing purposes does not improve accuracy, giving sub-optimal results at under 80% detection rate. [14] demonstrate the effectiveness of up and down sampling on data to equalise volumes of anomaly and benign data, achieving a detection rate of 99.99% using random forest and 99.30% using three layered deep neural networks, these very high results are unlikely to represent real world detection levels and give the impression of an over fitted model and a lack of generalisation. Effective classification of both benign and anomalous traffic is also an issue, in most cases models can identify labelled benign traffic with very high (99-100%) accuracy, however determining anomalous traffic can be more difficult, as shown in [15] where the random forest algorithm is applied to the UNSW-NB15 dataset, benign traffic was classified at 99% accuracy, however anomalous traffic was classified at 82%, this means that 18% of anomalous traffic was essentially undetected.

The approach of this paper attempts to rectify and address some of these common issues. This is done in two main ways by selecting the most up to date IDS data set, the CICIDS2018 which simulates a real world environment and is explained in detail further on. And secondly by using a CNN model based on NAS, which has achieved some of the highest accuracy levels in the ImageNet data set and uses a controller to autonomously optimise parameters for the model. By taking this approach the most optimal model can be generated for a specific data set.

Neural architecture search brings automation to the design of neural network models, this allows the most optimised model designs to be computed without the tedious process of physically designing, testing and adjusting models. This cutting edge technique in neural network design has led to the rise of a number of automated machine learning platforms. In this paper Google’s autoML Vision and Vision Edge platforms will be utilised for model design, training, validation and testing. The underlying architecture which enables these platforms is NASNet (Neural Architecture Search Network) and MNasNet (Mobile Neural Architecture Search Network).

Neural architecture search can be defined as a gradient-based method for finding optimised architectures. The structure and connectivity of a neural network can be specified by a variable length string. Therefore it becomes possible to use a RNN (Recurrent Neural Network) as shown in Figure 5. to generate this string [16]. The network specified by the string is known as the child network and training the real data set with the child network will result in progressive accuracy increases on the test data set. This accuracy can be used as the reward signal to compute the policy gradient to update the controller. Therefore in the next iteration the controller will give a higher probability to architectures that receive a higher accuracy [16]. Put simply this means the controller can learn to improve its search over time and optimise placement of layers and blocks of the neural network [17].

Fig. 5: RNN Controller

In terms of implementation neural architecture search uses the controller to generate a set of architectural hyperparameters of the network. In the case of a CNN it can predict filter height, filter width, stride height, stride width and a number of filters per layer [16]. This process is then repeated until the number of layers exceeds a certain value.

This issue with NAS is applying it to a very large data set is extremely computationally intensive. Therefore this technique is applied to a sample of the data set [9]. The NAS search space is defined so that the complexity of the architecture is independent of the depth of the network and the size of input images. It achieves this by breaking down all CNN’s in the search space into cells with identical structure but different weights as shown in Figure 6 [9]. Therefore searching for the most optimal architecture can be reduced to searching for the best cell architecture. By searching for each specific cell architecture, speed is greatly increased and the cell is more likely to have better generalisation. Based on this individual cell training approach, networks can be optimised for speed or accuracy depending on the search space size. This allows the neural network to achieve a very high level of accuracy on the ImageNet validation data set at 82.7% top 1 accuracy [18]. ImageNet is the largest database for labelled images containing over 14 million images and has widespread use in providing a benchmark for determining the performance of different CNN models [19]. Fig. 6: NAS Search Space Block Generation

MnasNet extends the NAS search space concept by implementing factorised hierarchical search space [20]. The factorised hierarchical search space encourages additional layer diversity throughout the network and balances the size of the total search space. This approach brings more flexibility into NAS as models can be designed to balance speed vs. accuracy. So far this approach has the biggest advantage of speed. On the ImageNet data set the MNasNet architecture achieved 75.2% top 1 accuracy which in comparison to traditional mobile neural network architectures is 1.8 times faster than MobileNetV2 [21] and 0.5% higher accuracy. In comparison to NASNet results were 7.5% lower accuracy, however 2.3 times faster in processing images within the architecture [20].

In the results section NASNet and MNasNET will be compared with tests conducted for 24 hours and 3 hours respectively to assess the differences in results. Latency and computational power is also a primary concern for implementation purposes in this case. By optimising a neural network that can still achieve a high level of accuracy, low latency when training and also can be run on devices such as a modern day smart phone, this will allow much more flexibility for deployment in a 5G network.

Anomaly detection is one of the most promising areas of research in detecting novel attacks. However its adoption to real world applications is hindered due to system complexity requiring a large amount of testing, tuning and evaluation. Therefore for research purposes a simulated system can be designed with a comprehensive set of intrusions and abnormal behaviour mixed in with normal traffic for anomaly detection analysis. As network behaviours and malware are changing it becomes necessary to have an environment that more accurately simulates a real world scenario. The data that can then be captured from the system is dynamic and provides more meaningful and realistic insight into benign and anomalous network traffic behaviour. Unfortunately traditional IDS data sets were not designed in this way, for example the KDD CUP99 data set or the ADFA-IDS data set were created in a testing environment that was only comprised of single LAN links and one attacking and one defending system, this approach represents a static environment and provides sub-optimal and less realistic results [22] [23].

The IDS-2018 data set from the Canadian Institute of Cybersecurity is a data set derived from a simulated environment that attempts to address these shortcomings [24]. The main objective of this data set is to use a systematic approach to generate a diverse and comprehensive benchmark data set for intrusion detection based on the creation of benign traffic and malicious traffic profiles. The environment itself is comprised of 50 attacking machines on a victim organisation with 5 departments which includes 420 machines and 30 servers. The data set takes packet captures of network traffic and system logs of each machine, as well as the extraction of 80 network features organised as flows. Figure 7. below shows the overall network topology which is a common LAN network on an AWS (Amazon Web Services) cloud platform. 6 subnets are installed labelled as Dep1 to Dep5 and Servers. Dep1 to Dep4 machines have Windows 8/10 OS’s, Dep5 has all Linux machines running Ubuntu, Servers has different MS Windows servers such as App servers, active directory and email. The attacker network has Windows 8/10 machines and Ubuntu machines.

Fig. 7: CICIDS2018 Network Topology

Protocols simulated in the environment are: HTTPS (HyperText Transfer Protocol Secure), HTTP (HyperText Transfer Protocol), SMTP (Simple Mail Transfer Protocol), POP3 (Post Office Protocol 3), IMAP (Internet Message Access Protocol), SSH (Secure Shell), FTP (File Transfer Protocol). Traffic types are broken into two profiles, either a B-profile (benign traffic) or M-profile (malicious traffic). The types of traffic within these profiles is explained in further detail below.

B-Profile: Describes normal traffic types simulated through a number of machine learning algorithms with different network protocols [24]:

• •

  Emulates the behaviour of users by utilising various machine learning statistical analysis techniques such as K-Means, Random Forest, SVM and J48.

• •

  Network features collected include packet size of protocol, number of packets per flow, various patterns in payload, size of payload and request time distribution of a protocol.

The specific attacks used in the M-Profile are common attacks used by malicious actors as well as penetration testers. They cover a wide variety of scenarios from network based attacks, different forms of HTTP DoS and DDoS, brute force attacks, web based attacks and widespread vulnerabilities. They also cover aspects of the OWASP top 10 2019 including injection based attacks from SQL, broken authentication due to poor password management allowing easier brute force attacks and security misconfigurations which allow vulnerabilities such as heartbleed due to unpatched systems [25].

M-Profile: Describes the attack scenario for anomalous traffic, six different attack scenarios are simulated [24]:

• •

  Internal network infiltration - exploits application vulnerability by sending malicious files via email. Metasploit framework is utilised for exploitation allowing a backdoor to be executed on the victim’s PC.

• •

  HTTP DoS - Slowloris, LOIC and HOIC which cause denial of service are used, these tools are able to make web servers inaccessible. Slowloris can do this with just one machine and is most effective against Apache servers [26]. Apache servers are the second most common web servers on the internet accounting for 26.73% of web servers [27].

• •

  Web app attacks - Web application based attacks tested using the Damn Vulnerable Web App (DVWA) for SQL injection, command injection and unrestricted file upload.

• •

  Brute force attacks - Use a dictionary brute force attack containing 90 million words against main servers to attempt to acquire SSH and MySQL account information.

• •

  Last updated attacks - Well known vulnerabilities that can affect thousands of devices under certain conditions and if they are running older, outdated versions of software. Heartleech will be used in this environment, it is used to scan systems vulnerable to the Heartbleed bug, once systems are found they can then be exploited and data can be exfiltrated.

To define the features from these profiles, initial raw packet captures are converted to network flows for easier analysis. Using CICFlowMeter bidirectional flows are generated where the first packet determines the forward (source to destination) and backward (destination to source) directions. Therefore from the 83 statistical features gathered from the flows such as duration, number of packets, number of bytes, length of packets, these are calculated separately for both forward and reverse directions. For TCP flows they are terminated upon connection teardown (once a FIN packet is received) and UDP flows are terminated by a flow timeout.

Fig 8. Pie Chart of Malicious Traffic Type Volumes

Fig 9. Pie Chart of all Traffic Volumes

This paper will break down all labelled network flows into two streams for analysis, anomalous and benign. Benign is comprised of all traffic described in the B-Profile and anomalous is all traffic described in the M-Profile. Different attacks occur at different days out of a total of 10 days or 240 hours, these attacks are dispersed randomly within benign traffic. In total there are 2748235 anomaly flows and 6584535 benign flows giving a total of 9332770 flows in the data set. This is a split of 70.55% benign traffic and 29.45% anomaly traffic. The two pie charts below in Figure 8. and Figure 9. show the breakdown of traffic volumes in the data set.

Data set pre-processing involves transforming the input data into the correct form suitable for the CNN, which in this case is a 100x100x3 image. The defined 20 features from the data set are extracted in CSV file format. CSV inputs are reshaped into RGB images of 100 x 100 x 3 size, any additional left over data under this size is discarded as all images for the CNN are required to be of the same input size. This image size was chosen due to providing a good volume of sample images for the amount of data available (over 1000 sample images). In general the trade offs between using a higher compared to a lower resolution image is that a higher resolution image will contain finer details when processed by the neural network, however this will take longer for both training and testing phases. A lower resolution image will provide less details, but more global feature representations and the neural network will be able to train and test the data at a faster rate. For this paper autoML samples and augments all images to 224 x 224 x 3 input image size, therefore there are only two considerations, firstly the volume of images is above 1000 and that sufficient feature details are captured. Two examples are shown below in Figure 10 and Figure 11 of what an anomaly image looks like in comparison to a benign image. Graphically, anomaly images are random and noisy, whereas benign images are more regular and contain some identifiable patterns.