AI for Key Management and Mitigating Attacks (AIKEMA)
Dr. Haitham Cruickshank, University of Surrey
PETRAS will consider how machine learning approaches to automated key management within the transport sector could help manage network load and intrusion detection and better understand and mitigate risk, particularly within 5G systems, and utilising the 5GIC tested with the support of Telefonica to understand challenges at scale. (On the Centre’s approaches to challenges from dynamic systems, see Annex 1.) AIKEMA will be further supported by other PETRAS projects, including TanSec, considering physical dimensions to key sharing techniques.
Cyberphysical Social Machines (CP-SOCIAM)
Prof. Wendy Hall, University of Southampton
The project will use a combination of in-the-wild experimentation with monitoring data from IoT devices and existing social machines to study the emergence of cyberphysical social machines. The initial focus will be health-related, and so will look at how devices such as activity trackers, in the context of social machines such as PatientsLikeMe, are helping redefine healthcare methodologies and promote health and wellbeing, by e.g. considering the collection and dissemination of social machine participants’ practical wisdom about managing health related conditions. CP-SOCIAM will theorise and experiment to discover what might enable such communities to flourish and build public value, and what might cause them to fail. The focus of this work will be to investigate emerging types of trustworthy social interactions facilitated by IoT devices, and to consider the design of institutions and infrastructures that will support and foster warranted trust in such interactions. This will involve research both within the immediate context of health related devices and applications, in other application domains if required (given the complexity of accessing data in the health and wellbeing sector), and more widely in considering policy, standards, and institutional support for data sharing.
Cybersecurity for Food Security (CyFoo)
Professor Awais Rashid, University of Bristol
The project will study the impact of malicious actors and vulnerabilities on the food supply. Agriculture is ever more reliant upon sensor-driven digital infrastructure (AgriTech) to optimise its production processes. We will use field studies of AgriTech deployments to investigate device vulnerabilities as well as user understanding of risk. These will form the basis of a risk analysis framework to mitigate food security and safety risks arising from cyber attacks and suggest cyber security policy recommendations for the Food Sector.
Geopolitics of IIoT Standards (GISt)
Dr. Madeline Carr, UCL
Discussions and debates in forums like the UN ITU, ENISA the IEEE and the IETF have become important sites for the global governance of the Internet of Things (IoT) but remain largely decoupled from cyber security discussions within the international policy community. Initially regarded as the domain of the technical community, standards are increasingly understood as a powerful mechanism through which IoT governance is enacted. At the same time, the governance and security of IoT infrastructure and implementation is regarded by states as linked to national security, national interest and state power. Consequently, understanding how different states are engaging with standards negotiations, which states are taking newly assertive roles, and which states are forming powerful alliances in standards forums (through desk research, observational analysis and interviews) is critical to understanding how technology and geopolitics will intersect in the 21st century. GISt will provide an analysis of the political dynamics of the complex ecosystem of IoT security standards. The project will investigate the extent to which political interest is embedded in the fora, processes and communities of practice that set security standards by which the IoT is implemented. GISt will focus on the industrial IoT (IIoT) as a domain with significant safety and security implications that receives less attention than consumer IoT. Indeed, the IIoT is an area where ambitions of increased productivity and efficiency very much outweigh security concerns and this makes it particularly susceptible to the vulnerabilities that emerge in the IoT generally.
Integrity Checking at the Edge (ICE)
Professor Pete Burnap, Cardiff
ICE project will study the factories and water treatment systems of the future, undertaking composite vulnerability analysis of interactions between edge devices, cloud platforms and legacy systems. This vulnerability analysis will be complemented by the use of AI at the edge of such systems to create and improve methods to demonstrate transparent processes mapping of data flows to expected activity at the periphery of integrated systems with advanced visualisation methods, and provide security and resilience for critical infrastructures of the future. Inspiration will be taken from explainable and interpretable AI, considering human-machine interaction in the context of question-asking around pathways and interactions involving data, creating links to the ICE-AI project within the usability lens. The Bristol Critical Infrastructures Testbed (which underpinned a linked PETRAS demonstrator) will form a core source for the technical analysis.
Intelligible Cloud and Edge AI (ICE-AI)
Dr Ewa Luger, The University of Edinburgh
PETRAS will go beyond the current technical discussions of ‘transparent’ or ‘explainable’ AI to consider social, conceptual and experiential understandings of algorithmic systems both in the cloud and at the edge. How might we design/present AI systems in ways that support human trust and ethically-sensitive design? This project will explore this across a set of sensitive/critical contexts with core partners, supporting the development of robust user/practitioner mental models of both AI in the cloud and AI at the edge. This will be achieved through in-depth interviews and observations conducted with practitioners, in addition to secondary sources, expert interviews, observations and ethnographic explorations, complemented by lab and in-the-wild studies using context-specific ‘intelligible’ cultural problems co-created with PETRAS user partners: BBC R&D and Microsoft Research UK.
Markets for Connected Space Sharing (MaCs)
Dr Charles Morisset, Newcastle University
MaCs will bring together the technological, economic and legal challenges around making better use of constrained space in urban workspaces by managing access to building data for shared spaces. MaCs will consider the challenges of i) directly interfacing with existing technological infrastructure; ii) integrating data anonymization techniques. iii) including legal aspects related to data privacy, particularly in a work setting where freely given consent is challenging due to power imbalances, and abuse is possible. The main technological output of MaCs will be a data management application, for instance enabling effective and simple space-sharing, which will serve as one of the initial case studies for PETRAS to ‘stress-test’ existing regulatory regimes in practice, and suggest room for improvement or clarification.
Modelling for Socio-technical Security (MASS)
Professor Jeremy Watson, UCL
The MASS project will explore technologies, enablers and blockers relating to monitoring (with analytics), modelling, simulation and emulation applied to the identification of operational exceptions, vulnerabilities and mitigation measures for systems at the periphery of the internet. The work will include how modelling, simulation, diagnostic monitoring, and detection tools can be evaluated and validated, and how validation and trust affect their acceptance by organisational security functions. Particularly, the project will explore a two-fold research approach related to the security modelling and simulations of IoT-connected cyber-physical system or critical infrastructure. Firstly, it will explore tools and the potential for analytics and artificial intelligence to facilitate and improve real-time and offline security at the Edge of IoT-connected cyber-physical systems or critical infrastructure. Secondly, it will explore the value of modelling and simulation in practice, especially for socio-technical security risk analysis of IoT-connected cyber-physical system. Stakeholders at varied levels need to understand their unique socio-technical environments, and the analytical possibilities available to help improve security of the cyber-physical systems they design, analyse or operate. To narrow down the challenging areas, representative sectoral domains will be selected for deep dives; e.g. transport and communications infrastructure and manufacturing operations. MASS outputs should provide a better, evidenced understanding of the realistic usefulness of analytical monitoring, modelling and simulation, as well as highlighting current limitations in supporting improved security from analytics and artificial intelligence viewpoints.
Responding to Attacks and Compromise at the Edge (RACE)
Professor Emil Lupu, Imperial College London
Whilst many studies focus on vulnerabilities in IoT systems, the increased attack surface resulting from their use and direct consequences of their potential compromise, much less is known about mitigation actions and how to apply them across a connected system in a coherent manner. The project aims to develop methodologies that enable IoT systems to respond to attacks and, whenever possible, to continue to operate even when systems have been partially compromised. Responding to a successful attack that has compromised some of the system’s components requires to understand both the risks to the rest of the of the system as well as how the compromise and any response actions will impact the operation of the system. The project will review attack mitigation techniques for IoT systems. It will investigate how to build and maintain models of the attack paths through the system considering system dynamics, and how to continuously assess the evolving risks. The project aims to develop techniques to identify the impact of an attack upon the functionality of the system and to investigate how to select countermeasures that minimise disruption to end users whilst maintaining acceptable levels of risk.
Red Teaming the Connected World (RETCON)
Professor Dave de Roure, University of Oxford
The project will explore wide and creative methodologies to deliberately influence or disrupt the behaviour of (sociotechnical) systems involving IoT devices. This will generate a ‘red team’ methodology for the many ways in which IoT systems might be compromised: penetration testing with demonstrators, simulation systems, examining cheating in connected gamified environments (given the norms they rely on), and developing ‘imposter’ devices. AI techniques are relevant as not only can they be interfered with, but also used as part of attack – such as if they ‘pretend to be human’. In the latter respect, the RETCON project will work with secondary projects from the usability lens (see Figure 1 above), such as the UncanAI project (Lancaster). Sociotechnical RETCON attacks will provide dual benefit to other projects, such as serving to test and improve the design patterns generated through the RIoTE project.
Resilient IoT on the Edge (RIoTE)
Professor Tim Watson, University of Warwick
The project will explore design patterns for resilient IoT at the periphery, delivering a resilient IoT toolkit of patterns that can be applied to migrate a system to a simpler, more easily maintained and more resilient state, or help to design new systems from scratch. Drawing upon existing NCSC secure design patterns, building on connected Warwick research in subsumption architectures for cyber security, and by studying systems in the wild and through the use of lab experiments and simulations, the project will also consider socio-technical patterns (eg organisational structures, servitisation and supply chains) and, from a human factors perspective, the need to avoid cognitive overload. An important aspect of the design pattern toolkit will be consideration of attack patterns, including cascade and blended attacks, and how to mitigate them through general resilience, which builds in resilient characteristics without having to list vulnerabilities or threats.
Security Query-Based Systems (Sec-QBS)
Dr Yves-Alexandre de Montjoye, Imperial College London
Different actors should be able to use insights from edge sensing and analytics in privacy preserving ways. In Sec-QBS, technical means to do this will be considered in the context of physical infrastructure systems. Connected sensors will drive the new wave of big data. However, the collection and use of this data and its tremendous potential for good must be balanced against the privacy risks for individuals and confidentiality issues for businesses. Pervasive sensing and actuation creates opportunities for understanding, problem-solving, and safeguarding. It can also be co-opted for unjustified surveillance, can affect privacy and power relations, and can manipulate people and environments. Following the limits of the historical tool of de-identification to protect privacy and confidentiality of big data, query-based systems, allowing analysts to query a sensitive database remotely, have been developed. Existing approaches however either 1) focus on formal guarantees, resulting in highly restricted interfaces ultimately providing limited utility, or 2) on ad-hoc single-layered approaches, resulting in catastrophic data breaches when vulnerabilities are discovered. In this strand of work, the Centre aims to bridge the two approaches and to develop practical multi-layered query-based systems through adversarial research (red and blue team), ultimately helping business, governments, civil society, researchers and users get the most insight possible out of our emerging connected world whilst respecting and ensuring societal buy-in, trust and support throughout. This project’s primary initial aim will concern urban sensing and ambient infrastructure, but will aim to stimulate these discussions across the Centre.
Tangible Security (TanSec)
Professor Derek McAuley, The University of Nottingham
This project aims to build a system to provide usable high grade secure communications between roaming smartphones and in home networked equipment. The projects objectives are to define, implement and evaluate new protocol and router functionality required; as well as investigate physical interaction methods to make the system easy to use for all and evaluate them from a usability point of view.
Uncanny AI (UnCanAI)
Professor Paul Coulton, Lancaster University
UncanAI will consider how trust can be engendered in users when Artificial Intelligent (AI) agents are used to provide services directly to users. It will explore the notion of the Turing Red Flag Law whereby autonomous systems would be required to be designed to prevent them being mistaken for one controlled by a human. Whilst such a law would appear rather a blunt instrument it does highlight a need for research that challenges the prevalent assumption that ‘human-like’ interactions are always preferable by exploring whether or not these interactions increase vulnerability to attacks such as phishing by producing unrealistic expectations of trust. To enable a more nuanced approach than a Turing Red Flag, we will consider the notion an ‘Uncanny Valley for AI’. Through the creation of a series of speculative design artefacts which relate to a variety of contexts of use and potential user groups, the project will map how different degrees of ‘human-likeness’ impact upon users’ trust of autonomous agents.