Clutter Slices Approach for Identification-on-the-fly of Indoor Spaces

Large-scale construction spaces need periodic surveying, inspection, and renovation [3]. Continuous assessment helps to identify the completion status as well as localize problems which may arise [1]. Traditionally, this requires a well-coordinated surveillance activity which consumes enormous man-hours, even resulting in delays and overheads. Further, the inherent complexity of such spaces, in terms of design, inter-connectivity, and scale complicate this already arduous undertaking. Automation of processes in such activities has the potential to greatly reduce the effort required and boost overall productivity, at the same time reducing overhead costs and delays. This need for process automation in the complex and fast-paced world of construction calls for innovation at all levels.

Inspection and surveying of outdoor large-scale construction activities now utilizes satellite imagery and Global Positioning Systems (GPS) based localization [15, 5]. While these methods are robust and cost effective solutions for outdoor spaces, they prove to be in-effective for indoor spaces. Moreover, indoor GPS-based navigation is not effective for multi-level structures and the signal in itself becomes unreliable [5]. Alternative solutions including WiFi signals and quantum sensors require expensive equipment for implementation [4]. Further, these limitations of expenses, time, and resources for efficient inspection and surveillance is withholding the extensive application of Building Information Modelling (BIM) in construction activities [1, 10]. Robotic technologies, such as mobile robots, rovers, and aerial manipulators, are proving to be an efficient automation solution for construction activities [2]. Mobile robots, such as aerial manipulators (UAV) [7] and ground-based manipulators (wheeled and legged) [12] are a cost-effective solution for real-time large scale inspections due to their robust and reliable performance.

Mobile robots with capabilities of perception are proving to be a paradigm shifting technology in inspection and surveillance. Perception sensors such as LiDARs (2D and 3D), stereo cameras, RGB-D cameras, ultrasonic and infrared proximity sensors have been extensively used in robot vision to identify the surrounding of a robot and its subsequent localization [9, 14]. This is similar to human perceiving their surroundings through multiple modal sensing. For example, humans use vision and their knowledge base to comprehend the characteristics of a construction site. They also use tactile sensing to provide an understanding over various states and properties of surfaces [12, 13]. However, humans have the ability to inherently perform these identification procedures as a secondary task, while performing primary targeted tasks such as reaching a target location, navigating among obstacles, etc. We call this identification-on-the-fly as it enables multi modal perception for intelligent and self-adaptive systems. [8]. Extending this methodology to coarse identification and classification of indoor spaces yields systems capable of multi-modal perception and intelligent operation yet efficient, especially for BIM development.

In this paper, the identification-on-the-fly method is used to coarsely identify human-built spaces based on the distribution of clutter. Each space has its own distinct signature. Clutter, the things which occupy space in an indoor environments such as doors, desks, and wall fittings, grant context to a space. The aim of this study is to develop the ability to identify and classify spaces based on this inherent signature. Hence, we present a unique sensor independent approach for classifying indoor spaces based on their inherent signatures. A sensor independent approach allows generalization of this method to numerous avenues and also allows for fast and inexpensive implementations.

In order to develop and validate this approach, we first present the Clutter Slices dataset. This initial dataset is developed with 2D LiDAR scans of indoor areas, such as staircases, washrooms, corridors, and shared spaces; spaces are common to most developments. We then propose the clutter slices pipeline which utilizes commonly used classifiers to train and subsequently test the approach on the collected dataset. Hence, the contributions of this study are as follows:

• •

  The Clutter Slices dataset of common indoor spaces along with the analysis of its distribution. This dataset is publicly available.

• •

  The clutter slices classification pipeline, including widely used classifiers, is presented. The evaluation of this model on the clutter slices dataset is presented as a baseline.

• •

  A new pipeline for clutter slices classification independent of sensor type, including widely used classifiers. The evaluation of this model on the clutter slices dataset is presented as a baseline.

• •

  Performance analysis of the selected classifiers in the proposed pipeline is presented on the clutter slices dataset.

The organization of this paper is as follows: Section 2 describes the Identification-on-the-fly approach using clutter slices to decipher the unique signatures of indoor spaces. Further, Section 3 presents the Clutter Slices dataset. In this section, we describe the methodology of data collection and the structure of the dataset. Section 4 presents the model and the classification methods used on the Clutter Slices dataset for identification of spaces. Experiments and results are presented in Section 5, followed by the conclusion in Section 6.

Embedding intelligence and self-adaptive features into robots requires them to perform multi-modal tasks, simultaneously, to extract a rich understanding of their environment. Such rich comprehension is based on contextual as well as state information of the environment which is extracted while navigating or interacting with it. Humans, exhibit this quality of multi-modal perception and cognition, which helps them decipher the surroundings in a way that they are even able to navigate unseen environments. Moreover, humans are able to perform such navigation and classification as a secondary task, while the goal of such movement can be varied. Example scenarios would include identification of different areas while navigation, using vision and tactile sensing to understand the current state of a surface or object. Another such example is performing status checks while navigating an unseen construction space. Identification-on-the-fly incorporates this ability of comprehending the unseen environment as an almost intuitive capability (performed as a secondary task) into autonomous robots, thereby taking them one step closer to human-like intelligence.

In this paper, an identification-on-the-fly approach is utilized to address problems associated with coarse identification of human-built indoor spaces while navigating through them. This is accomplished based on an intuitive assumption that each class of space has its own unique signature. Moreover, common spaces exhibit similar patterns as they are built for specific purposes, such as staircases, corridors, etc. Hence, these unique signatures can be generalized throughout indoor spaces to learn and recognize the class of spaces for unseen environments too.

Robust identification of construction spaces, especially indoor spaces, needs intelligent models that can comprehend the environment efficiently. The first step in building such models is creating adequate datasets for training. Hence, we present a diverse dataset of real-life indoor spaces. The clutter slices dataset is a collection of scans of common indoor spaces, such as corridors, staircases, restrooms, and large shared spaces (including cafeterias, common areas, and shared working offices), as shown in 2. This is a fully annotated dataset which enables models to learn the distribution of clutter in such common areas, and thereby contributes to efficient recognition of spaces.

The Clutter Slices dataset was created by taking two-dimensional (2D) LiDAR scans of areas such as restrooms, staircases, shared spaces and corridors around the various buildings of Purdue University. We chose a LiDAR sensor for this data collection as it is one of the most widely used sensors in navigation and mapping in robotic vision. Using this sensor, we measure spatial distribution $270{{}^{\circ}}$ around a point, as shown in Fig2. The maximum range of this sensor is 30 meters. Various positions around the space were used for the data collection to ensure a holistic capture of data. The height of the data collection was varied in steps of 1 meter.

There are four classes in the Clutter Slices dataset: corridors, staircases, restrooms, and large shared spaces. These classes are common to most indoor construction areas and hence are useful for researchers in future work. The distribution of instances of these classes in the dataset are shown in Fig 4. The dataset is publicly available at https://github.com/CRLPurdue/Clutter_Slices [11].

The clutter slices approach with identification-on-the-fly aims to understand inherent patterns in the data, rather than relying on explicit feature engineering. Hence, by just using the distances of clutter around a point, we derive a clutter slice at a fixed height. A stack of these slices would build the clutter signature of the space. However, the goal here is to understand the strength of just a single clutter slice in deriving the class of a space. Therefore, we use a single 2D scan of the space to understand the distribution of clutter and subsequently, classify it.

In the clutter slices pipeline, the input 2D scan is translated to distances around the point. This allows for use of multiple sensors, as a variety of sensors such as LiDARs, cameras, and infrared sensors can be used to get the distance measurements. These distances are then vectorized as the feature space $D{{}_{i}}$, wherein $D{{}_{i}}=[D{{}_{i}}{{}_{,}}{{}_{0}},...,D{{}_{i}}{{}_{,}}{{}_{2}}{{}_{7}}{{}_{0}}]$. The labels for this feature space are defined as $y{{}_{i}}$ where $i\in[0,3]$ for the clutter slices dataset. The feature space is then scaled using Box-Cox power transformations to standardize the data. The prepared data is then input to the classifiers. In this study, we used six classifiers which are widely used in machine learning: Random Forests, Logistic Regression, Support Vector Machines, AdaBoost, Artificial Neural Network, and Convolutional Neural Network. These classifiers present a baseline on the clutter slices dataset, and prove its effectiveness.

The validation of the proposed pipeline on the clutter slices dataset using the selected classifiers is presented in this section. We first present the experimental setup, including the hyperparameters selected for the classifiers, and consequently, present the performance in terms of accuracy, precision and recall for the classifiers.

In this paper we introduce the identification-on-the-fly approach to imbue human-like intelligence into robotic systems. The proposed clutter slices approach leverages the unique signatures of common indoor spaces for coarse classification. The initial validation of the clutter slices approach is performed on the dataset using 2D LiDAR sensor. Further, we present a scalable pipeline that supports this approach. The pipeline is flexible enough to accommodate varied classifiers. We used some of the widely used classifiers such as random forests, logistic regression, and neural network models to establish a baseline for the dataset. A maximum accuracy of 93.6% was achieved with this approach without significant hyperparameter tuning. The precision-recall plots show the convergence of the models in recognizing the classes of spaces.

The clutter slices approach captures the unique signatures of common indoor spaces and proves the potential of this approach in their coarse classification. Nevertheless, the clutter slices approach is not sensor specific and can be potentially generalized across domains. In the future, this approach of identification-on-the-fly can be an essential tool for perceiving and assessing surroundings of intelligent autonomous agents. Clutter slices is one implementation of the identification-on-the-fly method used for coarse classification of indoor spaces, adding contextual information to the robot perception. However, there are endless opportunities to perform identification-on-the-fly to understand the surrounding while still identifying potential dangers and outcome of future actions.

This work was supported, in part, by the Dept. of Energy, the NSF Center for Robots and Sensor for the Human Well-Being (RoSe-HUB) and by the National Science Foundation under grant CNS-1439717 and the USDA under grant 2018-67007-28439. The authors greatly acknowledge the contribution of coffee for the actualization of this work.