Enhanced Behavioral Cloning Based self-driving Car Using Transfer Learning

The end-to-end deep learning model is the most popular choice to deal with large volume data [1, 2, 3, 4] among researchers. Conventionally, the deep learning approaches decompose the problem in several subproblems to solve them independently, and all the outputs are combined to draw final decision. Many automobile companies like Hyundai, Tesla, etc., are trying to bring millions of self-driving or autonomous cars on the road by utilizing deep learning approaches. In this frantic race to come up with fully safe self-driving cars, some of the organizations like NVIDIA is following the end-to-end approach [5] as shown in Fig. 1, whereas Google is following mid-to-mid approach [6]. Following from these notions, the main objective of present research work is to predict the steering angle of the car via front facing camera.

The behavioral cloning [7] is a process of reproducing human performed tasks by a deep neural network. behavioral cloning is achieved by training the neural network with the data of human subject performing the task. In 1989, a self-driving car was developed by Pomerleau [8] based on neural networks. Afterwards, since past 130 years, the automobile manufacturers did not give attention to the replacement of the driver, who is the most vulnerable part of the car. The automotive companies tried to make the cars safer by including many safety features like anti-lock braking systems, shatter-resistant glass, airbags, etc [9]. However, organizations failed to succeed in developing driver-less intelligence. Self-driving cars are the most desirable revolutionary change in 21st century for a fully safe driving experience to change the way of transportation. According to the World Health Organization’s report on “Global status report on road safety 2018”, every year around 1.35 million humans lose their lives due to road accidents [10]. Self-driving cars will bring this number down and also enable people with disabilities to commute easily. Convolution neural networks (CNN) have revolutionized Pattern recognition [11] with the ability to capture 2D images in the context of self-driving cars. The greatest advantage of CNN is that it automatically extracts the important features to interpret the surrounding environment from the images which can be utilized to develop the intelligent driving system.

In the present research, to establish the importance of transfer learning approach concerning self-driving cars, a novel end-to-end based VGG16 approach is proposed which is fine-tuned to predict the steering angle based on the environmental constraints. Later, the proposed approach is compared with NVIDIA and its pruned variants. Due to the lesser number of parameters in the pruned architectures, the training time reduces significantly compared to baseline architecture. Since the transfer learning approach follows from the pre-trained model where only a part of network is trained, significant computational time is saved without compromising the performance. It has been observed that if the tasks are similar then the weights of initial few layers are similar and the last layers have relevant information towards the task [12]; making transfer learning a better way of saving training time.

The paper is organised in various sections including related work which briefly discusses the available approaches applied to self-driving cars and highlights the drawbacks and advantages of the research work carried out so far. The proposed approach section presents the various approaches utilized in the process of generating a novel model which accurately drives the car. Dataset and pre-processing techniques are also discussed in the subsequent sections. At the end, the experimental results were elaborated with concluding remarks.

The process of reconstructing the human subcognitive skill through the computer program is referred as behavioral cloning. Here, actions of human performing the skill are recorded along with the situation that gave rise to the action [13]. There are two popular methods for behavioral cloning. In the first method, the skill could be learned in terms of series of dialogues with an operator. Here, in case of autonomous vehicle, it is anticipated from the operator to explain all set of useful skills to control the vehicle. This method is challenging because manual description of skills is not perfectly possible due to human limitations. Alternatively, skill can be reconstructed through recorded actions which are maintained in a structured way by using learning algorithms in terms of various manifestation traces [14, 15, 16, 17] to reproduce the skilled behavior.

Defense Advanced Research Projects Agency (DARPA) initiated DARPA autonomous vehicle(DAVE) [18] project including a radio-controlled model truck which is attached with sensors and lightweight video cameras to test vehicle in an intrinsic environment having trees, heavy stones, lakes, etc. The testing vehicle is trained with 225000 frames of driving data. However, in test runs, DAVE crashed for every 20 metres on an average. In 1989 Pomerleau built an autonomous land vehicle in a neural network (ALVINN) model using the end-to-end learning methodology and it was observed that the vehicle can be steered by a deep neural network [8]. NVIDIA started their research work in the self-driving inspired by the ALVINN and DARPA projects. The motivation for their work was to create an end-to-end model which enables steering of the car without manual intervention [19, 5] while recording humans driving behavior along with the steering angle at every second by using controller area network (CAN) bus. Based on the NVIDIA proposed architecture pilotnet (as shown in Fig. 2), Texas Instruments released JacintoNet i.e an end-to-end neural network for embedded system vehicles such as tiny humanoid robots [20]. In 2020, a group of researchers from Rutgers university proposed a feudal network based on hierarchical reinforcement learning that performs similar to the state-of-the-art models with simpler and smaller architecture which reduces training time [21]. Jelena et al. [22] have proposed a network which is 4 times smaller than the NVIDIA model and about 250 times smaller than the AlexNet [23]. The model is developed only for the use in the embedded automotive platforms. To study the working of these end-to-end models Kim et al. [24] researched about the region of the images contributing in predicting steering angle.

Although learning to drive from this system would not suffice the self-driving car, furthermore the driving system should also address the issues such as how it would backtrack on to the road if it goes off the road by mistake, or else the vehicle will eventually move out of the road. Therefore, the images which are provided by the dataset, are combined with more images to visualize the vehicle in different field-of-views on and off the road. The datasets usually augmented with new images generated by view transformations via flipping the images to cover maximum possible scenarios [5]. For the transformed images the steering angle is changed in such a way that the vehicle would come back to the right position and direction within 2 seconds. The NVIDIA model proved to be quite powerful by achieving 98% autonomy time on road. The results observed from NVIDIA’s model consisting of only 5 convolution layers followed by 3 dense layers exhibited limited performance, thus it is evident that complex tasks require complex structure of deep neural networks with more number of layers to achieve better performance.

A competent human is required for controlling any intricate system such as helicopter, bike, etc. The competency is learnt through experience that develops within the subconscious capability of the brain. These subcognitive skills are challenging and can only be described roughly and inconsistently. In case of frequently occurring actions, the competency can be achieved by the system via learning from the recorded common patterns using deep learning techniques. Extracting and replicating such patterns from human subject performing the task is called behavioral cloning [25].

Following from the idea of behavioral cloning, a novel end-to-end transfer learning based VGG16 approach (as shown in Fig. 3) is proposed to predict the appropriate steering angle. The proposed model is compared with NVIDIA baseline model and its pruned variants built by chopping off the baseline NVIDIA model by 22.2% and 33.85% by using a $1\times 1$ convolution filter. Fig. 4 presents the overall schematic representation of the proposed approach.

The dataset is a sequence of front camera dashboard view images captured around Rancho Palos Verdes and San Pedro California traffic  [38]. It contains 45400 images and associated steering angle as described in Table 1. In this research, 80% of images are used for training and remaining 20% for validation testing.

The range of steering angle is between -90 to +90 where +90 indicates that the steering is tilted towards the right and -90 indicates that the steering is tilted towards the left. The data is preprocessed to get the images in the desired format which will be suitable for the network to learn and help in prediction of appropriate steering angle. The original and preprocessed images are shown in Fig. 7. The images are preprocessed by performing following steps:

• •

  Remove unnecessary features by cropping the image.

• •

  Convert the image to YUV format.

• •

  Reduce dimensions of the image to $66\times 200\times 3$.

Series of experiments have been carried out with baseline NVIDIA model, its pruned variants and proposed approach as described follows:

1. 1.

   By decreasing the number of feature maps from 64 to 32 and 64 to 16 by keeping the height and width constant, we pruned the network by 22.2% and 33.85% respectively.

2. 2.

   The transfer learning approach adopted with the convolution blocks of VGG16 and trained only the last block (3 convolution layers and 1 Maxpooling 2D layer).

The training of the models is assisted with stochastic gradient descent (SGD) [39] approach with Adam as the learning rate optimizer [40]. For robust training, 4-fold validation technique is applied along with the earlystopping to avoid the problem of overfitting [41]. The performance of the models are evaluated using the mean squared error (MSE) as given in the Eq. 1.

where $y_{i}$ stands for the actual steering value and $x_{i}$ stands for the predicted steering angle. Here, the lesser MSE indicates higher learning ability whereas a higher MSE means the model is not learning from complex environments.

The experimental results show that the pruned networks do not perform better compared to the baseline model. It is also observed that the proposed VGG16 model with transfer learning (training only last 4 convolutions layers) is trained within 40 epochs compared to other models which are trained with 100 epochs. With the experimental results, it has been proved that the VGG16 model with transfer learning works better as compared to the other NVIDIA models. As observed from the Table. 2, the novel transfer learning based approach achieved the better MSE score as compared to NVIDIA and its pruned variants. Due to the deep nature of VGG16, the architecture is able to learn complex patterns where as shallowness of NVIDIA models restrict their ability to adopt such complex environment conditions.

Fig. 8 highlights the training behavior of the models at each iteration where it is observed that the proposed approach achieved comparatively minimal loss with least number of training epochs. It is also observed that due to the adoption of trained weights the model starts with better loss and converges faster.

A novel approach based on transfer learning with VGG16 is proposed which is fine tuned by retraining the last block while keeping all the other layers non-trainable. The proposed model is compared with NVIDIA and its pruned architectures developed by applying $1\times 1$ filter. Since the proposed transfer learning architecture starts with minimal initial loss and converges at just 40 epochs compared to NVIDIA’s architecture which took 100 epochs, experimental results show that the transfer learning based approach works better than NVIDIA and its pruned variants. Naturally, the driving patterns also depend on several other environmental conditions like weather, visibility, etc. To adopt these challenging conditions in presence of limited number of samples, generative adversarial network (GAN) can be explored in future to generate vivid weather conditions for more robust driver-less solutions.