Keywords: Homomorphic Encryption, Performance, Accuracy, Basic Mathematical Functions, Division Function, Exponential Function, Square Root Function, Logarithm Function, Min/Max Function, Telescope, Time-Series Forecasting, Box–Cox Transformation

Cloud computing has become increasingly popular due to its scalability, cost-effectiveness, and ability to handle large volumes of data. However, entrusting (sensitive) data to a third party raises concerns about data security and privacy. Homomorphic encryption is one solution that allows users to store and process data in a public cloud without the cloud provider having access to it. Currently, homomorphic encryption libraries only support addition and multiplication; other mathematical functions must be implemented by the user. To this end, we discuss and implement the division, exponential, square root, logarithm, minimum, and maximum function, using the CKKS cryptosystem of the OpenFHE library. To demonstrate that complex applications can be realized with this extended function set, we have used it to homomorphically realize the Box--Cox transform, which is used in many real-world applications, e.g., time-series forecasts. Our results show how the number of iterations required to achieve a given accuracy varies depending on the function. In addition, the execution time for each function is independent of the input and is in the range of ten seconds on a reference machine. With this work, we provide users with insights on how to extend the original restricted function set of the CKKS cryptosystem of the OpenFHE library with basic mathematical functions.

@article{Prantl2024DeBello,
  url = {https://doi.org/10.1007/s10207-023-00781-0},
  doi = {10.1007/s10207-023-00781-0},
  issn = {1615-5270},
  pages = {1149--1169},
  number = {2},
  volume = {23},
  day = {01},
  month = {Apr},
  year = {2024},
  journal = {International Journal of Information Security},
  title = {De Bello Homomorphico: Investigation of the extensibility of the OpenFHE library with basic mathematical functions by means of common approaches using the example of the CKKS cryptosystem},
  author = {Prantl, Thomas and Horn, Lukas and Engel, Simon and Iffl{\"a}nder, Lukas and Beierlieb, Lukas and Krupitzer, Christian and Bauer, Andr{\'e} and Sakarvadia, Mansi and Foster, Ian and Kounev, Samuel}
}
                        

Keywords: Secure Group Communication scheme, Benchmark, IoT security, Group key agreement

As Internet of Things (IoT) devices become ubiquitous, they face increasing cybersecurity threats. Unlike standard 1-to-1 communication, the unique challenge posed by n-to-n communication in IoT is that messages must not be encrypted for a single recipient but for a group of recipients. For this reason, using Secure Group Communication (SGC) schemes is necessary to encrypt n-to-n communication efficiently for large group sizes. To this end , the literature presents various SGC schemes with varying features , performance profiles, and architectures, making the selection process challenging. A selection from this multitude of SGC schemes should best be made based on a benchmark that provides an overview of the performance of the schemes. Such a benchmark would make it much easier for developers to select an SGC scheme, but such a benchmark still needs to be created. This paper aims to close this gap by presenting a benchmark for SGC schemes that focus on IoT. Since the design of a benchmark first requires the definition of the underlying business problems, we defined suitable problems for using SGC schemes in the IoT sector as the first step. We identified a common problem for the centralized and decentralized/hybrid SGC schemes, whereas the distributed/contributory SGC schemes required defining an independent business problem. Based on these business problems, we first designed a specification-based benchmark, which we then extended to a hybrid benchmark through corresponding implementations. Finally, we deployed our hybrid benchmark in a typical IoT environment and measured and compared the performance of different SGC schemes. Our findings reveal notable impacts on calculation times and storage requirements without a trusted Central Instance (CI) in distributed/contributory SGC schemes.

@article{Prantl2024Group,
  url = {https://doi.org/10.1007/s44248-024-00010-6},
  doi = {10.1007/s44248-024-00010-6},
  issn = {2731-6955},
  pages = {5},
  number = {1},
  volume = {2},
  day = {23},
  month = {May},
  year = {2024},
  journal = {Discover Data},
  title = {Benchmarking of Secure Group Communication schemes with focus on IoT},
  author = {Prantl, Thomas and Bauer, Andr{\'e} and Engel, Simon and Horn, Lukas and Krupitzer, Christian and Iffl{\"a}nder, Lukas and Kounev, Samuel}
}
                        

Keywords: Secure Group Communication Schemes, Network conditions, Performance analysis

Secure and scalable group communication environments are essential for many IoT applications as they are the cornerstone for different IoT devices to work together securely to realize smart applications such as smart cities or smart health. Such applications are often implemented in Wireless Sensor Networks, posing additional challenges. Sensors usually have low capacity and limited network connectivity bandwidth. Over time, a variety of Secure Group Communication (SGC) schemes have emerged, all with their advantages and disadvantages. This variety makes it difficult for users to determine the best protocol for their specific application purpose. When selecting a Secure Group Communication scheme, it is crucial to know the model's performance under varying network conditions. Research focused so far only on performance in terms of server and client runtimes. To the best of our knowledge, we are the first to perform a network-based performance analysis of SGC schemes. Specifically, we analyze the network impact on the two centralized SGC schemes SKDC and LKH and one decentralized/contributory SGC scheme G-DH. To this end, we used the ComBench tool to simulate different network situations and then measured the times required for the following group operations: group creation, adding and removing members. The evaluation of our simulation results indicates that packet loss and delay influence the respective SGC schemes differently and that the execution time of the group operations depends more on the network situations than on the group sizes.

@article{Prantl2024Network,
  url = {https://doi.org/10.1007/s44248-024-00015-1},
  doi = {10.1007/s44248-024-00015-1},
  issn = {2731-6955},
  pages = {9},
  number = {1},
  volume = {2},
  day = {17},
  month = {Sep},
  year = {2024},
  journal = {Discover Data},
  title = {Network impact analysis on the performance of Secure Group Communication schemes with focus on IoT},
  author = {Prantl, Thomas and Amann, Patrick and Krupitzer, Christian and Engel , Simon and Bauer, Andr{\'e} and Kounev, Samuel}
}
                        

Keywords: Access control, Attribute-based Encryption, Public key encryption

In recent years, digital services have experienced significant growth, exemplified by platforms like Netflix achieving unprecedented revenue levels. Some of these services employ subscription models, with certain content requiring additional payments or offering third-party products. To ensure the widespread availability of diverse digital services anytime and anywhere, providers must have control over content accessibility. To address the multifaceted challenges in this domain, one promising solution is the adoption of attribute-based encryption (ABE). Over the years , various approaches have been proposed in the literature, offering a wide range of features. In a prior study [18], we assessed the security of one of these proposed approaches and identified one that did not meet its promised security standards. In this research we focuses on conducting a security analysis for another ABE scheme to pinpoint its shortcomings and emphasize the critical importance of evaluating the safety and effectiveness of newly proposed schemes. Specifically, we uncover an attack vector within this ABE scheme, which enables malicious users to decrypt content without the required permissions or attributes. Furthermore, we propose a solution to rectify this identified vulnerability.

@inproceedings{Prantl2024ABE,
  series = {ARES '24},
  location = {Vienna, Austria},
  numpages = {11},
  articleno = {24},
  booktitle = {Proceedings of the 19th International Conference on Availability, Reliability and Security},
  doi = {10.1145/3664476.3664487},
  url = {https://doi.org/10.1145/3664476.3664487},
  address = {New York, NY, USA},
  publisher = {Association for Computing Machinery},
  isbn = {9798400717185},
  year = {2024},
  title = {Security Analysis of a Decentralized, Revocable and Verifiable Attribute-Based Encryption Scheme},
  author = {Prantl, Thomas and Lauer, Marco and Horn, Lukas and Engel, Simon and Dingel, David and Bauer, Andr\'{e} and Krupitzer, Christian and Kounev, Samuel}
}
                        

Keywords: DevSecOps, Intrusion Detection, Microservices, Containers, Detection Reuse

Microservice-based systems are becoming very frequent in many systems, even in business-critical ones. It is only logical that attackers pay more attention to microservices security and attempt to compromise it to achieve their goals. Intrusion detection has been used to increase the security level of these systems but faces significant challenges: namely, effectively processing all the information produced by the services and keeping up with constant modifications and releases. Microservice environments are extremely dynamic , changing by the day or even faster. Although anomaly-based approaches can detect zero-days they require a clear definition of the microservice behavior, which may be cumbersome to obtain without a clear notion of a service's core behavior. In this paper, we propose three techniques that allow to obtain the core behavior of a microservice into intrusion detection models. The techniques capture a stable model of a microservice core and can be applied across service releases. The models created are evaluated using datasets generated in a representative system with known vulnerabilities through an attack injection methodology with a diverse set of representative attacks. The results show that obtaining the core of a service helps with reusing detection models across several releases, despite some limitations.

@inproceedings{flora2024doing,
  organization = {IEEE},
  year = {2024},
  pages = {49--56},
  booktitle = {2024 19th European Dependable Computing Conference (EDCC)},
  author = {Flora, Jos{\'e} and Antunes, Nuno},
  title = {Doing more with less? A Study on Models for Intrusion Detection in Microservices}
}
                        

Keywords: Microservices, Security Benchmarking, Datasets, Intrusion Detection, Attack Injection

Microservices are predominant for cloud-based applications, which serve millions of customers daily, that commonly run business-critical systems on software containers and multi-tenant environments; so, it is of utmost importance to secure these systems. Intrusion detection is a widely applied technique that is now being used in microservices to build behavior detection models and report possible attacks during runtime. However, it is cumbersome to evaluate and compare the effectiveness of different approaches. Standardized frameworks are non-existent and without fairly comparing new techniques to the state-of-the-art, it is difficult to understand their pros and cons. This paper presents a comprehensive approach to evaluate and compare different intrusion detection approaches for microservice applications. A benchmarking methodology is proposed to allow users to standardize the process for a representative and reproducible evaluation. We also present a dataset that applies representative workloads and technologies based on microservice applications state-of-the-art. The benchmark and dataset are used in three case studies, characterized by dynamicity, scalability, and continuous delivery, to evaluate and compare state-of-the-art algorithms with the objective of tackling intrusion detection in microservices. Experiments show the usefulness and wide application range of the benchmark while showing the capacity of intrusion detection algorithms in different applications and deployments.

@article{flora2024evaluating,
  publisher = {Elsevier},
  year = {2024},
  pages = {112142},
  volume = {216},
  journal = {Journal of Systems and Software},
  author = {Flora, Jos{\'e} and Antunes, Nuno},
  title = {Evaluating intrusion detection for microservice applications: Benchmark, dataset, and case studies}
}
                        

Keywords: Anomaly Detection, Modeling, Performance, Security, Virtualization

Virtualization enables cloud computing, allowing for server consolidation with cost reduction. It also introduces new challenges in terms of security and isolation, which are deterrents for the adoption of virtualization in critical systems. Virtualized systems tend to be very complex, and multi-tenancy is the norm, as the hypervisor manages the resources shared among virtual machines. This paper proposes a methodology that uses performance modeling for the detection of anomalies in virtualized environments that can be caused, for instance, by cyberattacks. Experiments are conducted to profile the system operation under normal conditions for its business transactions. The results are used to calibrate a performance model and to understand the impact of its parameters on the false positive probability. During operation, the system is monitored, and deviations are detected by applying a sequential analysis algorithm (the bucket algorithm). The methodology is evaluated using a representative cloud workload (TPCx-V), which was profiled during a set of controlled executions. We consider resource exhaustion anomalies to emulate the effects of attacks affecting the performance of the system. Our results show that the proposed approach is able to successfully detect anomalies, with a low number of false positives, and spot possible residual effects of anomalies on the system.

@article{gonccalves2023detecting,
  publisher = {IEEE},
  year = {2023},
  pages = {70716--70740},
  volume = {11},
  journal = {IEEE Access},
  author = {Gon{\c{c}}alves, Charles F and Menasch{\'e}, Daniel Sadoc and Avritzer, Alberto and Antunes, Nuno and Vieira, Marco},
  title = {Detecting anomalies through sequential performance analysis in virtualized environments}
}
                        

Keywords: Security, Intrusions, Virtualization, Hypervisor, Security Assessment

Virtualization is drawing attention due to countless benefits, leaving Hypervisors with the paramount responsibility for performance, dependability, and security. However, while there are consolidated approaches to assessing the performance and dependability of virtualized systems, solutions to assess security are very limited. Key difficulties are evaluating the system in the presence of unknown attacks and vulnerabilities and comparing the security attributes of different systems and configurations when an intrusion occurs. In this paper, we propose a novel concept and approach of intrusion injection for virtualized environments, which consists of directly driving the system into the erroneous states that mimic the ones resulting from actual intrusions (in the same way errors are injected to mimic the effects of residual faults). We present a prototype capable of injecting erroneous states related to memory-corruption in the Xen Hypervisor to show that the concept and approach proposed here are feasible. The prototype is evaluated using publicly disclosed exploits across three different versions of Xen. Results show that our tool can inject erroneous states equivalent to those resulting from attacks that exploit existing vulnerabilities, even on versions where those vulnerabilities do not exist.

@inproceedings{gonccalves2023intrusion,
  organization = {IEEE},
  year = {2023},
  pages = {417--430},
  booktitle = {2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)},
  author = {Gon{\c{c}}alves, Charles F and Antunes, Nuno and Vieira, Marco},
  title = {Intrusion Injection for Virtualized Systems: Concepts and Approach}
}
                        

Keywords: Security, Intrusion Detection, Microservices, DevSecOps, Scalability

The growing complexity and dynamicity of microservices, combined with their ability to scale, present significant challenges to security monitoring tools. Integrating these tools into a DevSecOps pipeline is currently impractical, necessitating research into adaptive intrusion detection approaches. This paper introduces three data processing techniques that enable intrusion detection in scalable and elastic microservice applications utilizing CI/CD approaches. These techniques manipulate data collected from active microservice replicas and feed it to algorithms, resulting in reliable intrusion detection even after scaling operations. To evaluate these techniques, we integrate them into a state-of-the-art intrusion detection tool developed for microservice environments. Their effectiveness is evaluated using two lightweight algorithms (STIDE and BoSC) with representative workloads, attacks, and a microservice-based application, demonstrating their ability to detect most attacks, even in scenarios involving multiple replicas.

@inproceedings{flora2023intrusion,
  organization = {IEEE},
  year = {2023},
  pages = {39--45},
  booktitle = {2023 IEEE 28th Pacific Rim International Symposium on Dependable Computing (PRDC)},
  author = {Flora, Jos{\'e} and Gon{\c{c}}alves, Paulo and Antunes, Nuno},
  title = {Intrusion detection for scalable and elastic microservice applications}
}
                        

Keywords: Fault Injection, Fault Tolerance, Kubernetes, Microservices, Software Aging

Microservice-based applications are increasingly being adopted along with cloud service models, and nowadays serve millions of customers daily. They are supported by container-based architectures which are managed by orchestration platforms, such as Kubernetes, that monitor , manage, and automate most of the tasks. Although these tools provide failover capabilities, it is not yet studied how effective they are in dealing with diverse types of faults. Fault injection is an effective methodology for validating components that are supposed to detect the malfunctions and report/correct them. This paper studies the effectiveness of Kubernetes in dealing with faults and aging in microservices, and on the possibility of using faults to accelerate aging effects for testing purposes. For this, we conducted an analysis of the implementation and tuning of Kubernetes probes, followed by experiments with varying load and fault injection into two distinct and representative microservice testbeds to analyze the capacity of probes in detecting issues in applications. The goal is to improve the knowledge of researchers and developers on whether Kubernetes can detect different faults and aging issues. Also, even though some services tend to accumulate aging effects, with increasing resource consumption, Kubernetes does not detect them nor acts on them, indicating that probes may be insufficient for aging scenarios. Results also showed that fault injection is useful to accelerate aging effects for the testing and evaluation purposes.

@article{flora2022study,
  publisher = {IEEE},
  year = {2022},
  pages = {132786--132799},
  volume = {10},
  journal = {IEEE Access},
  author = {Flora, Jos{\'e} and Gon{\c{c}}alves, Paulo and Teixeira, Miguel and Antunes, Nuno},
  title = {A study on the aging and fault tolerance of microservices in Kubernetes}
}
                        

Keywords: aging; kubernetes; fault injection; microservices.

The exploding popularity of microservice based applications is taking companies to adopt them along with cloud services to support them. Containers are the common deployment infrastructures that currently serve millions of customers daily, being managed using orchestration platforms that monitor, manage, and automate most of the work. However, there are multiple concerns with the claims put forward by the developers of such tools. In this paper, we study the effects of aging in microservices and the utilization of faults to accelerate aging effects while evaluating the capacity of Kubernetes to detect microservice aging. We consider three operation scenarios for a representative microservice-based system through the utilization of stress testing and fault injection as a manner to potentiate aging in the services composing the system to evaluate the capacity of Kubernetes mechanisms to detect it. The results demonstrate that even though some services tend to accumulate aging effects, with increasing resource consumption, Kubernetes does not detect them nor acts on them, which indicates that the probe mechanisms may be insufficient for aging scenarios. This factor may indicate the necessity for more effective mechanisms, capable of detecting aging early on and act on it in a more proactive manner without requiring the services to become unresponsive.

@inproceedings{flora2021my,
  title={My Services Got Old! Can Kubernetes Handle the Aging of Microservices?},
  author={Flora, Jos{\'e} and Gon{\c{c}}alves, Paulo and Teixeira, Miguel and Antunes, Nuno},
  booktitle={2021 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)},
  pages={40--47},
  year={2021},
  organization={IEEE}
}

Keywords: software testing; virtual machine monitors; computer bugs; reverse engineering; manuals, documentation, aging.

With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough testing to ensure their long-term reliability. Existing research deals with the aging properties of hypervisors in general without considering the hypercalls. In this work, we share our experience that we collected during trying to understand hypercalls and their parameters and use them to construct test cases for hypervisor aging of Microsoft Hyper-V. We present a bug that we detected, which was reported and acknowledged by Microsoft. Further, based on our manual binary code analysis, we propose the idea of automating the analysis process to detect valid parameter ranges and execution conditions of hypercalls without manual effort.

@inproceedings{DBLP:conf/issre/BeierliebAIAMK21,
  author    = {Lukas Beierlieb and
               Alberto Avritzer and
               Lukas Iffl{\"{a}}nder and
               Nuno Antunes and
               Aleksandar Milenkoski and
               Samuel Kounev},
  title     = {Software Testing Strategies for Detecting Hypercall Handlers' Aging-related
               Bugs},
  booktitle = {{IEEE} International Symposium on Software Reliability Engineering,
               {ISSRE} 2021 - Workshops, Wuhan, China, October 25-28, 2021},
  pages     = {48--55},
  publisher = {{IEEE}},
  year      = {2021},
  url       = {https://doi.org/10.1109/ISSREW53611.2021.00043},
  doi       = {10.1109/ISSREW53611.2021.00043},
}

Keywords: security benchmark; vulnerabilities; hypervisor; virtualization.

Hypervisors govern the resources of virtualized systems and are a crucial component of many cloud solutions. As a critical component, cloud providers should assess the hypervisor’s security to mitigate risk before adoption. Ideally, a benchmark should be applied to compare the security of different systems objectively, but security benchmarking is still an open problem. Notwithstanding, the evaluation of the system’s trustworthiness has been adopted as a promising approach as part of this complex evaluation process. In this work, we present a vulnerability data analysis of the Xen hypervisor. Additionally, we address the problem of how to apply this analysis results as trustworthiness evidence that can be applied in security benchmarks. Our results present an insightful characterization of Xen’s vulnerabilities evaluating their lifespan, distribution, and modeling. We also show that vulnerability data analysis can qualitatively characterize the Xen hypervisor’s trustworthiness and possibly reflect the security development efforts into its codebase.

@inproceedings{gonccalves2020vulnerability,
  title={Vulnerability Analysis as Trustworthiness Evidence in Security Benchmarking: A Case Study on Xen.},
  author={Gon{\c{c}}alves, Charles F and Antunes, Nuno},
  booktitle={2020 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)},
  pages={231--236},
  year={2020},
  organization={IEEE}
}

Keywords: hypervisors; hypercalls; testing; performance.

With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.

@inproceedings{Beierlieb\_2019\_TowardsTestingthePerformanceInfluenceofHypervisorHypercallInterfaceBehavior,  author = {Beierlieb, Lukas and Iffländer, Lukas and Kounev, Samuel and Milenkoski, Aleksandar},
  booktitle = {Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19)},
  month = 11,
  title = {Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior},
  year = 2019
}

Keywords: aging; hypervisors; hypercalls; testing; challenges; dependability; robustness; security.

With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.

@inproceedings{Beierlieb\_2019\_TowardsTestingtheSoftwareAgingBehaviorofHypervisorHypercallInterfaces,
  author = {Beierlieb, Lukas and Iffländer, Lukas and Milenkoski, Aleksandar and Goncalves, Charles F. and Antunes, Nuno and Kounev, Samuel},
  booktitle = {2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})},
  month = 11,
  organization = {{IEEE}},
  title = {Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces},
  year = 2019
}

Keywords: intrusion detection, software-deined networking; network function virtualization; adaptive networking.

Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore's law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

@inproceedings{Ifflaender2019_PerformanceOrientedDynamicBypassingforIntrusionDetectionSystems,
  author = {Iffl{\"a}nder, Lukas and Stoll, Jonathan and Rawtani, Nishant and Lesch, Veronika and Lange, Klaus-Dieter and Kounev, Samuel},
  title = {{Performance Oriented Dynamic Bypassing for Intrusion Detection Systems}},
  titleaddon = {{(Short Paper)}},
  booktitle = {Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering},
  series = {ICPE '19},
  year = {2019},
  isbn = {978-1-4503-6239-9},
  location = {Mumbai, India},
  pages = {159--166},
  numpages = {8},
  url = {http://doi.acm.org/10.1145/3297663.3310313},
  doi = {10.1145/3297663.3310313},
  acmid = {3310313},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {adaptive networking, intrusion detection, network function virtualization, software-defined networking},
  pdf = {https://se2.informatik.uni-wuerzburg.de/publications/download/paper/1846.pdf},
  slides = {https://se2.informatik.uni-wuerzburg.de/publications/download/slides/1846},
}

Keywords: network security, software-deined networking; network function virtualization; adaptive networking.

Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision of a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.

@inproceedings{IfWaEiKo2018-ICPE-SSFC-Vision,
  author = {Iffl{\"a}nder, Lukas and Walter, J{\"u}rgen and Eismann, Simon and Kounev, Samuel},
  title = {The Vision of Self-aware Reordering of Security Network Function Chains},
  booktitle = {Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering},
  series = {ICPE '18},
  year = {2018},
  isbn = {978-1-4503-5629-9},
  location = {Berlin, Germany},
  pages = {1--4},
  numpages = {4},
  doi = {10.1145/3185768.3186309},
  acmid = {3186309},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {models at run-time, network function virtualization, service function chaining, software-defined networking},
  pdf = {https://se2.informatik.uni-wuerzburg.de/publications/download/paper/1564.pdf},
  titleaddon = {{(Vision Paper)}},
  slides = {https://se2.informatik.uni-wuerzburg.de/publications/download/slides/1564},
}

Keywords: Cloud computing; Virtualization; Hypercalls; Intrusion detection systems; Workloads; Attack injection.

The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.

@inproceedings{MiPaAnViKoAvLu2015-RAID-Challenges,
author = {Aleksandar Milenkoski and Bryan D. Payne and Nuno Antunes and Marco Vieira and Samuel Kounev and Alberto Avritzer and Matthias Luft},
booktitle = {The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015)},
publisher = {{Springer}},
location = {Kyoto, Japan},
title = {{Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection}},
year = {2015},
month = {{November}},
url = {http://link.springer.com/chapter/10.1007/978-3-319-26362-5_22},
}

Keywords: Survey; Intrusion detection systems; Evaluation; Experimentation.

The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.

@article{MiViKoAvPa2015-CSUR-IDSEval,
author = {Aleksandar Milenkoski and Marco Vieira and Samuel Kounev and Alberto Avrtizer and Bryan D. Payne},
title = {{Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices}},
journal = {{ACM Computing Surveys}},
publisher = {ACM},
address = {New York, NY, USA},
year = {2015},
month = {{September}},
volume = {48},
number = {1},
pages = {12:1--12:41},
url = {http://dl.acm.org/authorize?N06203},
}

Keywords: Cloud computing; Virtualization; Hypercalls; Vulnerability analysis.

Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.

@inproceedings{MiPaAnViKo2014-ISSRE-AnAnalHypHanVulns,
author = {Aleksandar Milenkoski and Bryan D. Payne and Nuno Antunes and Marco Vieira and Samuel Kounev},
title = {{Experience Report: An Analysis of Hypercall Handler Vulnerabilities}},
location = {Naples, Italy},
month = {November},
booktitle = {{Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) --- Research Track}},
organization = {IEEE},
publisher = {IEEE Computer Society},
address = {{Washington DC, USA}},
year = {2014},
}

Keywords: Cloud computing; Virtualization; Intrusion detection system benchmarking.

@inproceedings{MiPaAnViKo2013-ACSAC-HInjector,
  address = {Maryland, USA},
  author = {Aleksandar Milenkoski and Bryan D. Payne and Nuno Antunes and Marco Vieira and Samuel Kounev},
  booktitle = {The 2013 Annual Computer Security Applications Conference (ACSAC 2013)},
  publisher = {{Applied Computer Security Associates (ACSA)}},
  location = {New Orleans, Louisiana, USA},
  title = {{HInjector: Injecting Hypercall Attacks for Evaluating VMI-based Intrusion Detection Systems (Poster Paper)}},
  year = {2013}
}

Keywords: Intrusion Detection; Performance Evaluation; Probes; Testing.

In this paper we propose a methodology to inject realistic attacks in Web applications. The methodology is based on the idea that by injecting realistic vulnerabilities in a Web application and attacking them automatically we can assess existing security mechanisms. To provide true to life results, this methodology relies on field studies of a large number of vulnerabilities in Web applications. The paper also describes a set of tools implementing the proposed methodology. They allow the automation of the entire process, including gathering results and analysis. We used these tools to conduct a set of experiments to demonstrate the feasibility and effectiveness of the proposed methodology. The experiments include the evaluation of coverage and false positives of an intrusion detection system for SQL injection and the assessment of the effectiveness of two Web application vulnerability scanners. Results show that the injection of vulnerabilities and attacks is an effective way to evaluate security mechanisms and tools.

@INPROCEEDINGS{5270349,
	author={Fonseca, J. and Vieira, M. and Madeira, H.},
	booktitle={IEEE/IFIP International Conference on Dependable Systems Networks, 2009. DSN '09.}, 
	title={Vulnerability #x00026; attack injection for web applications},
	year={2009},
	month={29 2009-july 2},
	pages={93 -102},
	keywords={Application software;Automation;Counting circuits;Information security;Inspection;Instruments;Intrusion detection;Performance evaluation;Probes;Testing;Internet;SQL;program debugging;program testing;security of data;SQL injection;Web application;Web application vulnerability scanner;intrusion detection system;realistic attack injection tool;realistic test bed;realistic vulnerability injection;security mechanism;software bug;},
	doi={10.1109/DSN.2009.5270349},
}
						

Keywords: Databases; Intrusion Detection; Monitoring; SQL Injection Detection.

System administrators frequently rely on intrusion detection tools to protect their systems against SQL Injection, one of the most dangerous security threats in database-centric web applications. However, the real effectiveness of those tools is usually unknown, which may lead administrators to put an unjustifiable level of trust in the tools they use. In this paper we present an experimental evaluation of the effectiveness of five SQL Injection detection tools that operate at different system levels: Application, Database and Network. To test the tools in a realistic scenario, Vulnerability and Attack Injection is applied in a setup based on three web applications of different sizes and complexities. Results show that the assessed tools have a very low effectiveness and only perform well under specific circumstances, which highlight the limitations of current intrusion detection tools in detecting SQL Injection attacks. Based on experimental observations we underline the strengths and weaknesses of the tools assessed.

@INPROCEEDINGS{5635053,
	author={Elia, I.A. and Fonseca, J. and Vieira, M.},
	booktitle={Software Reliability Engineering (ISSRE), 2010 IEEE 21st International Symposium on}, 
	title={Comparing SQL Injection Detection Tools Using Attack Injection: An Experimental Study},
	year={2010},
	month={nov.},
	pages={289 -298},
	keywords={Databases;Intrusion detection;Monitoring;Payloads;Scalp;Web server;Internet;SQL;security of data;SQL injection detection tools;attack injection;database-centric Web application;intrusion detection tool;security threat;system administration;vulnerability;Fault Injection;Intrusion Detection;SQL Injection;Security;Web applications;},
	doi={10.1109/ISSRE.2010.32},
	ISSN={1071-9458},
}
						

Keywords: Vulnerability Benchmarking; Web Services.

Vulnerability detection tools are frequently considered the silver-bullet for detecting vulnerabilities in web services. However, research shows that the effectiveness of most of those tools is very low and that using the wrong tool may lead to the deployment of services with undetected vulnerabilities. In this paper we propose a benchmarking approach to assess and compare the effectiveness of vulnerability detection tools in web services environments. This approach was used to define a concrete benchmark for SQL Injection vulnerability detection tools. This benchmark is demonstrated by a real example of benchmarking several widely used tools, including four penetration-testers, three static code analyzers, and one anomaly detector. Results show that the benchmark accurately portrays the effectiveness of vulnerability detection tools and suggest that the proposed approach can be applied in the field.

@article{10.1109/ICWS.2010.76,
	author = {Nuno Antunes and Marco Vieira},
	title = {Benchmarking Vulnerability Detection Tools for Web Services},
	journal ={2012 IEEE 19th International Conference on Web Services},
	volume = {0},
	isbn = {978-0-7695-4128-0},
	year = {2010},
	pages = {203-210},
	doi = {http://doi.ieeecomputersociety.org/10.1109/ICWS.2010.76},
	publisher = {IEEE Computer Society},
	address = {Los Alamitos, CA, USA},
}
						

A new approach that is useful in identifying and eliminating performance degradation occurring in aging software is proposed. A customer-affecting metric is used to initiate the restoration of such a system to full capacity. A case study is described in which, by simulating an industrial software system, we are able to show that by monitoring a customer-affecting metric and frequently comparing its degradation to the performance objective, we can ensure system stability at a very low cost.

@inproceedings{Avritzer:2005:ESP:1071021.1071026,
 author = {Avritzer, Alberto and Bondi, Andre and Weyuker, Elaine J.},
 title = {Ensuring stable performance for systems that degrade},
 booktitle = {Proceedings of the 5th international workshop on Software and performance},
 series = {WOSP '05},
 year = {2005},
 isbn = {1-59593-087-6},
 location = {Palma, Illes Balears, Spain},
 pages = {43--51},
 numpages = {9},
 url = {http://doi.acm.org/10.1145/1071021.1071026},
 doi = {10.1145/1071021.1071026},
 acmid = {1071026},
 publisher = {ACM},
 address = {New York, NY, USA},
}
						

Keywords: Measurement; Monitoring; Performance signatures; Security.

A new approach for detecting security attacks on software systems by monitoring the software system performance signatures is introduced. We present a proposed architecture for security intrusion detection using off-the-shelf security monitoring tools and performance signatures. Our approach relies on the assumption that the performance signature of the well-behaved system can be measured and that the performance signature of several types of attacks can be identified. This assumption has been validated for operations support systems that are used to monitor large infrastructures and receive aggregated traffic that is periodic in nature. Examples of such infrastructures include telecommunications systems, transportation systems and power generation systems. In addition, significant deviation from well-behaved system performance signatures can be used to trigger alerts about new types of security attacks. We used a custom performance benchmark and five types of security attacks to derive performance signatures for the normal mode of operation and the security attack mode of operation. We observed that one of the types of the security attacks went undetected by the off-the-shelf security monitoring tools but was detected by our approach of monitoring performance signatures. We conclude that an architecture for security intrusion detection can be effectively complemented by monitoring of performance signatures.

                @inproceedings{Avritzer:2010:MSI:1712605.1712623,
                author = {Avritzer, Alberto and Tanikella, Rajanikanth and James, Kiran and Cole, Robert G. and Weyuker, Elaine},
                title = {Monitoring for security intrusion using performance signatures},
                booktitle = {Proceedings of the first joint WOSP/SIPEW international conference on Performance engineering},
                series = {WOSP/SIPEW '10},
                year = {2010},
                isbn = {978-1-60558-563-5},
                location = {San Jose, California, USA},
                pages = {93--104},
                numpages = {12},
                url = {http://doi.acm.org/10.1145/1712605.1712623},
                doi = {10.1145/1712605.1712623},
                acmid = {1712623},
                publisher = {ACM},
                address = {New York, NY, USA},
                keywords = {measurement, monitoring, performance signatures, security},
                }
        

Keywords: Benchmarking; Dependability; Networks; Performance; Reliability; Robustness; Statistical methods; Stochastic modeling; System evaluation; Testing.

Keywords: Cloud computing; Security of data; Virtualisation; Intrusion detection system benchmarking.

Many recent research works propose novel architectures of intrusion detection systems specifically designed to operate in virtualized environments. However, little attention has been given to the evaluation and benchmarking of such architectures with respect to their performance and dependability. In this paper, we present a research roadmap towards developing a framework for benchmarking intrusion detection systems for cloud environments in a scientifically rigorous and a representative manner.

                @INPROCEEDINGS{6470873,
                author={Milenkoski, A. and Kounev, S.},
                booktitle={2012 International Conference For Internet Technology And Secured Transactions}, title={Towards benchmarking intrusion detection systems for virtualized cloud environments},
                year={2012},
                pages={562-563},
                keywords={cloud computing;security of data;virtualization;intrusion detection system benchmarking;virtualized cloud environments;Benchmark testing;Cloud computing;Hardware;Intrusion detection;Measurement;Monitoring;Virtual machine monitors;benchmark testing;intrusion detection},}
        

Keywords: Network monitoring; Protection Mechanisms; Performance signatures; Computer worms; Mitigation.

In this paper, we propose a new approach for mitigation of worm propagation through tactical Mobile Ad-Hoc Networks (MANETs) which is based upon performance signatures and software rejuvenation. Three application performance signature and software rejuvenation algorithms are proposed and analyzed. These algorithms monitor critical applications' responsiveness and trigger actions for software rejuvenation when host resources degrade due to a co-resident worm competing for host resources. We analyze the effectiveness of our algorithms through analytic modeling and detailed, extensive simulation studies. The key performance metrics investigated are application response time, mean time between rejuvenations and the steady state probability of host infection. We also use simulation models to investigate several design and parameter tuning issues. We investigate the relationship between the rate at which the application performance monitors can detect out-of-specification applications and the rate of worm propagation in the network.

                @inproceedings{Avritzer:2007:UPS:1216993.1217023,
                author = {Avritzer, Alberto and Cole, Robert G. and Weyuker, Elaine J.},
                title = {Using performance signatures and software rejuvenation for worm mitigation in tactical MANETs},
                booktitle = {Proceedings of the 6th international workshop on Software and performance},
                series = {WOSP '07},
                year = {2007},
                isbn = {1-59593-297-6},
                location = {Buenes Aires, Argentina},
                pages = {172--180},
                numpages = {9},
                url = {http://doi.acm.org/10.1145/1216993.1217023},
                doi = {10.1145/1216993.1217023},
                acmid = {1217023},
                publisher = {ACM},
                address = {New York, NY, USA},
                keywords = {computer worms, mitigation, mobile ad hoc networks (MANETS), software monitoring},
                }