Close

Visual Artificial Intelligence Laboratory (VAIL)

Group Leader(s): Professor Fabio Cuzzolin

Contact: fabio.cuzzolin@brookes.ac.uk +44 (0)1865 484526

About us
Research impact
Leadership
Membership
Projects

 News

The paper

ROAD-R: The Autonomous Driving Dataset with Logical Requirements 

by Eleonora Giunchiglia, Mihaela Stoian, Salman Khan, Fabio Cuzzolin and Thomas Lukasiewicz has won the Best Paper Award at the IJCAI 2022 Workshop on Artificial Intelligence for Autonomous Driving (AI4AD 2022)

The extended version of the paper has been accepted by Machine Learning journal, and has won the Best student paper prize at IJCLR 2022, the International Joint Conference on Learning and Reasoning.

About us

The Visual Artificial Intelligence Laboratory was founded in 2012 by Professor Cuzzolin under the name of 'Machine Learning' (and later 'Artificial Intelligence and Vision') research group, and has since conducted work at the boundaries of human action recognition in computer vision. Prof Cuzzolin is a leading scientist in the mathematics of uncertainty, in particular random set and belief function theory.

Our research interests span a number of frontier topics in:

  • computer vision (action and activity detection, future event prediction, video captioning and scene understanding)
  • machine learning (continual learning, federated learning, self-supervision and metric learning)
  • artificial intelligence (epistemic AI and machine theory of mind, but also neurosymbolic AI),
  • robotics (with a focus on surgical robotics), autonomous driving (the detection of road events for situation awareness)
  • AI for healthcare (the monitoring of people in care homes, the early diagnosis of dementia, empathetic healthcare via theory of mind).

More information about VAIL

Part of

Related courses

The laboratory has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreements No. 964505 (E-pi) and No. 779813 (SARAS).

Research impact

Road event detection in autonomous driving, with colored boxes around the relevant road agents to be detected

The group has built, in just a few years, a leadership position in the field of deep learning for action detection, with some of the best detection accuracies to date and the first ever system able to localise multiple actions on the image plane in (better than) real time. The team's effort is now shifting towards topics at the frontier of computer vision, such as future action prediction, deep video captioning and the development of a theory of mind for machines.

The Lab currently runs on a budget of around £3.2M (not fully incorporating the €4.3M Horizon 2020 project SARAS or the €3M FET Epistemic AI we are coordinating), with currently nine live projects funded by Horizon 2020, the Leverhulme Trust, Innovate UK, Huawei Technologies, UKIERI, and the School of Engineering, Computing and Mathematics. The budget is projected to further significantly increase in 2022.

Prof Cuzzolin's reputation in uncertainty theory and belief functions comes from the formulation of a geometric approach to uncertainty in which probabilities, possibilities, belief measures and random sets are represented and analysed by geometric means. This has recently developed into an effort to reshape the foundations of artificial intelligence to better incorporate and model second-order, 'epistemic' uncertainty: an approach that we call Epistemic Artificial Intelligence.

Leadership

Fabio Cuzzolin

Professor Fabio Cuzzolin

Professor of Artificial Intelligence

View profile

Membership

Staff

Name Role Email
Dr Andrew Bradley Reader abradley@brookes.ac.uk
Dr Eleni Elia Senior Lecturer in Statistics eelia@brookes.ac.uk
Dr Tjeerd Olde Scheper Reader in Computer Science tvolde-scheper@brookes.ac.uk
Dr Alex Rast Lecturer in Computing arast@brookes.ac.uk
Dr Matthias Rolf Reader in Computer Science mrolf@brookes.ac.uk
Dr Inna Skarga-Bandurova Senior Lecturer in Artificial Intelligence iskarga-bandurova@brookes.ac.uk

Students

Name Thesis Title Supervisors Completed
Devashish Bharti Federated machine learning Professor Fabio Cuzzolin

Active
Salman Khan Deep Scene Graph Models for Complex Activity Detection Professor Fabio Cuzzolin, Dr Tjeerd Olde Scheper 2023
Shireen Kudukkil Manchingal Epistemic Artificial Intelligence Dr Andrew Bradley, Professor Fabio Cuzzolin 2024
Mr Izzedin Teeti Predictive Algorithms for Advanced Autonomous Vehicle Perception Dr Andrew Bradley, Professor Fabio Cuzzolin 2023

Collaborators

Name Role Organisation
Professor Biplab Banerjee Assistant Professor Indian Institute of Technology, Bombay
Mr Kevin Cannons Senior Staff Researcher and Team Leader Huawei Technologies Canada
Mr Shaohua Chen Senior Staff Researcher and Team Leader Huawei Technologies Canada
Dr Dinesh Jackson Samuel Research fellow The University of Texas, MD Anderson Cancer Center
Dr Reza Javanmard Alitappeh Assistant professor Mazandaran University of Technology, Iran
Dr Christelle Langley Research Associate University of Cambridge
Dr Vincenzo Lomonaco Assistant professor University of Pisa, Italy
Professor Riccardo Muradore Associate professor University of Verona, Italy
Dr George Mylonas Director of HARMS Lab, Lecturer in Robotics and Technology in Cancer Imperial College London
Dr Keivan Shariatmadar Senior Researcher KU Leuven, Belgium
Dr Gurkirt Singh Postdoc ETH Zurich, Switzerland
Dr Efthymia Tsamoura Researcher Samsung AI
Dr Filippo Vella Researcher CNR, Italy
Professor Neil Yorke-Smith Associate professor Delft University of Technology, Netherlands

Projects

Active projects

Project title and description Investigator(s) Funder(s) Dates

Epistemic Artificial Intelligence

Although artificial intelligence (AI) has improved remarkably over the last years, its inability to deal with fundamental uncertainty severely limits its application. This proposal re-imagines AI with a proper treatment of the uncertainty stemming from our forcibly partial knowledge of the world. Epistemic AI’s overall objective is to create a new paradigm for a next-generation artificial intelligence providing worst-case guarantees on its predictions thanks to a proper modelling of real-world uncertainties.
 Professor Fabio Cuzzolin Horizon 2020 From: March 2021
Until: February 2025

Smart Autonomous Robotic Assistant Surgeon (SARAS)

SARAS aims at developing the next-generation of surgical robotic systems that will allow a single surgeon to execute Robotic Minimally Invasive Surgery (R-MIS) without the need of an expert assistant surgeon.
 Professor Fabio Cuzzolin Horizon 2020 From: January 2018
Until: December 2021

Artificial intelligence for autonomous driving

The project concerns the design and development of novel ways for robots and autonomous machines to interact with humans in a variety of emerging scenarios, including: human-robot interaction and autonomous driving, with a focus of perception, prediction of intent and trajectories and scene understanding.
 Dr Andrew Bradley, Professor Fabio Cuzzolin From: March 2019

Some novel paradigms for analyzing human actions in complex videos

Dominant action detection paradigms work by locating actions of interest on a frame by frame basis, and linking them up in time to form ‘action tubes’ in a fully supervised setting. In this project, we propose to recognise human activities under the weaker scenarios of few-shot and zero-shot learning.
 Professor Fabio Cuzzolin From: April 2018
Until: December 2021

Theory of mind at the interface of neuroscience and AI

Emerging applications of artificial intelligence are highlighting the limitations of established approaches in situations involving humans. The integration of neuroscience and machine learning has the potential to enable significant advances in both fields. Theory of Mind capabilities, i.e., the ability to 'read' other sentient beings' mental states, are crucial for the development of a next generation, "human-centric" artificial intelligence aimed to understand the behaviour of complex agents. In a mutually beneficial process, computational models developed within artificial intelligence could provide new insights about how these mechanisms work in the human brain.
 Professor Fabio Cuzzolin Leverhulme Trust From: February 2020
Until: December 2023

Deep learning for complex activity detection in videos

The goal of the project is to couple our current work on deep learning for multiple action tube detection with part-based modelling, in order to create a new deep learning architecture able to represent complex activities composed of a number of ‘atomic’ actions (e.g. cooking a meal is made up by ‘opening the fridge’, ‘taking out ingredients from the fridge’, ‘chopping ingredient on the counter’, etc).
 Professor Fabio Cuzzolin From: February 2020
Until: February 2023

Knowledge Transfer Partnership with Supponor

This project explores the use of machine learning to achieve real-time understanding of video scenes and consistent segmentation of advertisement boards and pitch objects without the use of existing infrared cameras and hardware infrastructure.
 Professor Fabio Cuzzolin, Dr Alex Rast Innovate UK From: November 2021
Until: October 2024

MAESTRO Jr - Multi-sensing AI Environment for Surgical Task & Role Optimisation

Our MAESTRO concept aims at laying the foundations for the operating room of the mid-21st Century, as a surgical environment powered by trustable, human-understanding artificial intelligence able to continually adapt and learn the best way to optimise safety, efficacy, teamwork, economy, and clinical outcomes.
 Professor Fabio Cuzzolin, Dr George Mylonas EPSRC From: October 2021
Until: December 2022

Completed projects

Project title and description Investigator(s) Funder(s) Dates

Knowledge Transfer Partnership with Createc

 Professor Fabio Cuzzolin Innovate UK From: May 2019
Until: April 2021

Resources

Below you can find links to a number of resources generated by our research, including datasets and code.

ROAD dataset

ROAD is the ROad event Awareness Dataset for autonomous driving, released at the ROAD @ ICCV 2021 workshop.

ROAD code

Our ICCV'17 code on real-time action detection, the first online solution ever published.

CAR dataset

The Continual Activity Recognition (CAR) dataset was released at the CSSL @ IJCAI 2021 workshop.

3D RetinaNet

3D RetinaNet is our event detection approach, used as baseline for detection tasks in the ROAD dataset.

CCC dataset

The Continual Crowd Counting (CCC) dataset was released at the CSSL @ IJCAI 2021 workshop.

Avalanche

Avalanche: the End-to-End Library for Continual Learning created by our partners ContinualAI.

SARAS-MESAD dataset

The SARAS-MESAD dataset is a surgical action detection dataset released at MICCAI 2021 as part of the SARAS project.

OJLA

Code for the BMVC 2018 paper 'Incremental Tube Construction for Human Action Detection'.