Integrated Dataset of Brazilian Flights

The Brazilian commercial aviation system contains more than one hundred airports. It transported 95.9 million revenue passengers during $2014$. It achieved the first position among Latin American countries and the fifteenth place worldwide on the Revenue Passenger-Kilometer (RPK) ranking ³. The commercial aviation network in Brazil is organized towards regional hubs in contrast to airline hubs. The main reason is the Brazilian territorial extension and that few Brazilian states have more than one major airport. One exception to this rule is Campinas (in the state of São Paulo), where airline company Azul holds $77\%$ of its commercial flights. Besides, due to market deregulation instituted in $2005$, the Brazilian commercial aviation system experienced significant changes in its players, leading to market share changes and flight availability.

The National Civil Aviation Agency (ANAC) is responsible for regulating and supervising the Brazilian civil aviation activities. Since $2000$, ANAC keeps track of departure and arrival data for Brazilian flights in its Active Regular Flight (VRA) dataset ¹. The data available in VRA are registered by the airlines and consolidated by ANAC. It contains data about each flight stage, i.e., the aircraft’s necessary steps from its takeoff to the next landing. These steps are established regardless of where the object of transport has been loaded or unloaded. For each flight step, VRA provides data such as airline, flight number, type (such as international, domestic, and cargo), class (such as regular, extra, charter, and instruction), airports, and scheduled and elapsed times for departure and arrival. ANAC monthly provides VRA data on its webpage.

VRA enables studying the Brazilian commercial aviation system. Examples of studies are flight delay patterns ⁷ and their prediction ⁴, ⁵. Although meteorological conditions play an essential role in analyzing flight information, such data is not present in VRA. Thus, this paper presents a dataset that integrates Brazilian flight data. It fuses all monthly data available in VRA. It enriches it with meteorological data from the ASOS (Automated Surface Observing Systems) dataset ² provided by the IOWA University in the USA. ASOS contains weather sensor data from airports around the world. During the entire data integration, data cleaning and data preprocessing techniques were also applied to improve its quality.

According to the flight regulation of ANAC, commercial airline companies must register flight metadata indicating changes in flight time, either delay, anticipation, or canceling. They have to log the time a flight happened and a justification for the alteration. Table 1 indicates the flight metadata together with their semantics.

According to the regulation of ANAC, the metadata indicated in Table 2 must be registered in a paper form, either typed or handwritten. ANAC then consolidate the data sent by the airline companies into the VRA dataset. VRA is published monthly, comprising all flight steps expected to depart in a given month.

The primary goal of ANAC is to use the recorded metadata to compute the punctuality rate of airlines. Thus, sector regulation obliges airline companies to provide the data presented in Table 1. Therefore, it comprises all flight steps that took place in a given period. However, around 20% of the records may be considered inconsistent due to errors while filling the report form. As will be presented in Section 3.1, the causes of errors include arrival time before departure or flight duration inconsistent with the regulation of ANAC.

Meteorological conditions play an important role in aviation operations. The Automated Surface Observing Systems (ASOS) is a program that involves several American government agencies. It was created to become an official network of meteorological information to support primarily aviation entities. It includes meteorological, climatological, and hydrological components. ASOS data come from weather sensors in locations all over the planet. In Brazil, ASOS covers all $154$ airports available in VRA, as seen in Figure 1.

The Department of Agronomy at Iowa State University, in the United States, compiles daily information from the US ASOS system. It creates an hourly report of meteorological observations in all of its sites. Table 2 indicates the meteorological data together with their semantics.

The integrated Brazilian Flight Dataset (BFD) presented in this paper includes both the flight data present in VRA and meteorological information present in ASOS. It is intended to enable studies regarding the Brazilian commercial aviation system. BFD is composed of $15,505,922$ records of flight data, each containing $45$ attributes. The dataset, together with its integration process description and R scripts, is available on IEEE DataPort¹¹1Dataset is available at http://dx.doi.org/10.21227/k10b-qn21. Additional information can be found at ⁸..

Figure 2 presents the data model of BFD. It is detailed in the following sections. As can be seen, BFD aggregates data from VRA and ASOS for flight information and meteorological information, respectively. It also includes data currently unavailable in VRA, such as describing the justification codes of ANAC, airline and airport names, and ISO codes for country names.

BFD focus on flight data regarding flights that departed or arrived in Brazil. When both origin and destination airports are located in Brazil, those flights are considered domestic flights. Conversely, when either the origin or the destination airport is located outside of Brazil, it is considered international. The data integration process for creating BFD was organized into three main activities: (i) data preprocessing, (ii) data enrichment, and (iii) data fusion. Those activities resemble the traditional Extraction, Transformation, and Load (ETL) process ¹⁰.

BFD allows for studies regarding the Brazilian commercial aviation system. In this section, we present previous and ongoing work conducted on top of BFD together with an exploratory analysis of BFD data. To present the importance of using the database, we conduct an exploratory analysis and mention studies that used the data in their research.

As discussed before, the Brazilian flight system is oriented towards regional hubs instead of company hubs. Figure 3 presents the number of flights per airport, considering just the 25 biggest airports on flights. It also divides flights into domestic (D), international (I), and cargo (C) flights.

As can be seen in Figure 3, in the top five busiest airports, the first two are in São Paulo (SBSP and SBGR), the third in Brasília (SBBR), and the last two in Rio de Janeiro (SBGL and SBRJ). Rio and São Paulo are the two higher Gross Domestic Products (GDPs) in Brazil. They are two major gateways for flights coming and exiting Brazil. Approximately one-third of the flight in Guarulhos Airport (SBGR) and Galeão Airport (SBGL) are international flights.

Brasilia is the capital of the country and is located in the middle of Brazil. It acts as a hub for flights from and to cities in the north and northeast regions. It can be seen, however, that it has few international flights.

Brazil and Argentina have strong touristic relations. Thus we can see the Buenos Aires international airport (SAEZ) in the top-25 busiest airports. Since BFD has only flights from and to Brazil, SAEZ has only international and cargo flights.

Figure 4 presents the takeoff and arrival delay per airport for the top-25 busiest airports. It indicates whether an airport has recover capabilities for arrival delays. The radius of the airport also indicates the level of punctuality. The higher the radius, the airports are more punctual.

Figure 5 presents the distribution of flights according to the period of the day. As shown, most of the flight departures (Figure 5.a) occur in the afternoon and early evening. Most arrivals (Figure 5.b) occur in the afternoon and early morning. During the mid and late morning, the number of flights decreases significantly for both departure and arrival.

According to ANAC regulation, a flight is considered to be delayed when its departure or arrival time surpasses, respectively, the expected departure or arrival by more than $30$ minutes. Figure 6 presents the punctuality rate considering all the Brazilian flight systems per year, from $2000$ to $2019$. It is possible to observe that the Brazilian flight crises that occurred in $2007$ interfered with both punctuality rates and mean delay ⁹.

Figure 7 analysis of the Brazilian systems monthly. Historically, months of school break (December, January, and July) have the lowest punctuality rates and the highest mean delay. August is the month with the highest level of punctuality and lowest mean delay.

Finally, Figure 8 presents the punctuality rate (circle size) and the average delay in minutes per number of flights for the top-25 companies. According to Figure 8, two airlines present the most significant number of flights, TAM and Gol (GLO). It is also possible to observe that airlines with lower punctuality rates tend to have a higher mean delay.

Given the various inconveniences for airlines, airports, and passengers caused by flight delays, it is fundamental to mitigate their occurrence and optimize an air transport system’s decision-making process. Mainly, airlines, airports, and users may be more interested in when delays are likely to occur than the accurate prediction of the absence of delays. In that context, Moreira et al. ⁴ use BFD to analyze Flight delays in the period between $2009$ and $2015$. The authors present a classification model capable of predicting delays, getting about $60\%$ of hits.

Flight delays fall into two main categories: root delay and delay propagation. Root delays are related to events that are intrinsic to a particular flight. In delay propagation, it is presumed that a delay has already occurred at some point in the network, i.e., new delays occur due to previous delays. The understanding of delay propagation patterns among airports is essential for decision-making processes.

That study may devise patterns in flight delays and the way the system recover from it. Focusing on unveiling those patterns, Sternberg et al. ⁶ apply data indexing techniques combined with BFD data association rules. The authors observed that the Brazilian flight system has difficulties recovering from previous delay when operating under adverse meteorological conditions, when delays occurrences may increase up to $216\%$.

This work aimed to create a reliable and enriched database on national and international flights that arrived and departed from Brazilian airports. With the data offered by this database, it is possible to carry out several studies to aid the decision-making process. For example, it is possible to answer the following questions: (i) “Which airport suffers the most delays?”; (ii) “What month of the year is an airport most likely to be delayed?”; or (iii) “What part of the day is a particular airport most likely to experience a delay in departure?”

The answers to these questions can help companies and governments review their protocols and optimize their services. Additionally, we intend to update this dataset yearly, conducting the entire data integration.

The authors thank CNPq, CAPES (finance code 001), FAPERJ, and CEFET/RJ for partially funding this research.

On behalf of all authors, the corresponding author states that there is no conflict of interest.

All authors contributed equally to the study. EO conceptualized the study design. CT acquired the data. LT and JS conducted data analysis and interpretation. Furthermore JAS and GA revised it critically for intellectual content. All authors have approval of the final version.