Classifying Cycling Hazards in Egocentric Data

This proposal is for the creation and annotation of an egocentric video data set of hazardous cycling situations. This data set will consist of egocentric video from a cyclist with associated data from an Inertial Measurement Unit (IMU) that indicates when the cyclist experienced sudden stopping or movement. The resulting data set will facilitate projects to improve safety and the experience of cyclists who are one of the most vulnerable groups travelling on roads [3]. Since cyclists are more sensitive to the surface conditions and hazards they require more detail about road conditions when navigating their route. Features such as tram tracks, cobblestones, gratings, and utility access points can pose hazards or uncomfortable riding conditions for their journeys. While there are data sets of egocentric videos for vehicles [6], there is a lack of egocentric data sets for cyclists who navigate roads differently to drivers. Current public egocentric cycling data sets [1] only cover a limited range of cycling infrastructures and are missing many of the common hazards that cyclists face when they are cycling.

There are some data sets for egocentric vehicle video [6] but even though cyclists often share the road with cars they navigate roads differently. Existing vehicle based egocentric data sets are not appropriate since cyclists travel at a slower speed and in a different road position than those in a car. This means they have a different range of vision and are looking more closely at the immediate surface conditions compared with drivers.

Existing egocentric cycling data sets only cover a limited range of cycling infrastructures typically found in the US [1]. This data set covers only roads, on road bike lanes and bike ways in a typical US city and does not include a wider range of cycling infrastructures found in other countries such as separated bike lanes, shared pedestrian and cycling paths. The data set also does not include a wide range of obstacles such as tram tracks, zebra/pedestrian crossings,cobblestone surfaces which are more commonly found in cities outside the US.

Researchers have used KITTI, Carina and RTRK to classify road surfaces for vehicles into the following classifications; asphalt, asphalt(poor quality), paved and unpaved[6]. These classifications work for vehicles but are insufficient for cyclists who are more sensitive to the type of road surface they are travelling on. Other researchers have used their own classification process on egocentric cycling footage to classify cycling infrastructure [7]. But this study was focused on the type of infrastructure, not the surface of the road or hazards encountered. This data set was also not made public.

The resulting data set will be useful for those involved with the development and maintenance of cycling infrastructure. It will provide information on what kind of situations cause sudden changes or uncomfortable conditions for cyclists, enabling the better planning of infrastructure for cyclists. For example a city municipality can look at the resulting data and then at their own bike lanes to understand where dangerous conditions exist for cyclists or what are better types of surfaces to use when constructing bike lanes.

Another use is training machine learning models to identify hazards in video footage automatically. This can enable local cyclists or advocacy groups to find hazardous conditions within a city cycling network and push authorities to make changes to improve safety. An example is identifying what types of surfaces cause cyclists to brake suddenly or lose control more frequently, and where within a city certain hazards are more prevalent.

Another use is pairing the visually identified hazards or conditions with the data from the IMU. This can enable a trained model to identify a more complex identification of current conditions without the use of a camera. If deployed on a larger scale such as with a bike sharing network then a complex analysis of the type of preferred paths and surfaces could be created using IMU data alone.

The video and IMU data will be collected from cyclists via a customised helmet containing a raspberry pi with a webcam type camera and IMU that records to an SD card. The helmet will be worn by participants who will cycle through their daily commute or other common cycling routes in their city. The data will be collected across 3 countries (Hong Kong, Australia and the United Kingdom) with a total of 50 hours footage to be recorded. Several of these helmets will be made and sent to people to record data to increase the amount of data recording. The aim is to cover as many different types of cycling infrastructure as possible, including shared inner city roads, dedicated bike lanes, bike ways and leisure paths.

Using the IMU information the video will be processed to find sections of 5-10 seconds where there was a sudden change in direction, sudden movement or some uncomfortable condition. Approximately 500 video sections will be extracted from the 50 hours of video. These sections will then be processed by MT to classify what hazards caused the sudden change in direction or movement. This is a more complex task that requires some analysis of the conditions by the MT who can deduce what cyclist to change their movement and is difficult for existing computer vision algorithms to handle.

The aim of this project is to create an egocentric data set of hazardous cycling conditions which will be useful for projects that facilitate cycling safety such as the creation and maintenance of cycling infrastructure. There is currently a lack of data sets relating to egocentric cycling conditions, and by using an IMU it is possible to collate a series of short videos that show cycling hazards. The analysis of this data with MT will enable the classification of hazards that a cyclist encounters when they are undergoing some kind of sudden change of speed or direction.