Visual Exploration of Movement Relatedness for Multi-species Ecology Analysis

Movements are usually referred as locomotive movements in the field of movement ecology. Location changes are useful clues to reveal ecological patterns in the problems such as resource use[27], population dynamics[13], and climate influence[28] between individuals, groups, or species. Many ecologists have already known the value of visualization in their ecological research. Generic visualization tools (e.g. Movebank[12], AMV[29], Env-DATA[30]) are employed to support common tasks like trajectory plotting and multivariate filtering. As they cover a wide range of species and data types for many research projects, these tools are enough for basic movement analyses.

However, some research tasks require analytic aggregation capabilities. Drosophigator[31] uses statistics from heterogeneous data sources to generate visualized predictions of the spread of invasive species. Xavier et al.[32] integrates environment data to study the connectivity (a technical term in ecology indicating the degree of environmental variables affecting its inhabitants in an area[33, 34, 35]) of landscape characteristics and animal behaviors. Konzack et al.[36] analyze the migratory trajectories to recognize the stopovers among gulls’ movements.

Visual design is also important to communicate the aggregated results that relates to the exact domain problem. Slingsby et al.[25] discusses the design choices of visual encoding in ecology visualization, suggesting that the use of visual language needs to convey the "ecological meaning" to support the research context. Spretke et al.[14] conceives the "enrichment" method as a way to enhance analytical reasoning with the visual elements of attributes like speed, distance, duration in the geographical context, allowing quick hypothesis iterations on local subsets.

Aggregation of attributes of individuals might be useful to understand the movement behavior, but behaviors is better explained in a context where influence of peers and surroundings are considered[11]. Since the interest in entity level behaviors is rising[17], investigating behaviors concerning the mutual influence of multiple entities can be a worthy starting point.

Manually searching for patterns in the long, twisted, and sometimes cluttered movement trajectories can be daunting. This makes event detection algorithms necessary as they alleviate the cognitive load for experts. The execution of automatic event detection usually requires defining a set of parameters, such as time window, speed, heading, and mutual distance, to narrow down the search space to a subset of trajectory fragments[37, 38, 39, 40]. The detection outcomes are usually visualized on top of the movement trajectories with reference to the original geographic context. As the detection processes are sensitive to the subject animal and landscape context [39], we need flexibility in a visualization tool to cope with the distinct characteristics of movements for a convincing result[37]. For example, Andrienko et al.[40] suggested a bottom-up approach where the detected elementary interactions (a concept derived from Bertin’s elementary level of analysis[41], meaning "particular instances of interaction between individual objects") are used as key clues to understand group level patterns. Bak et al.[37] proposed method to boost the performance in event detection at larger scale. The extra performance gain can thus be allocated to support interactive parameter input, through which the visual feedback of outcomes makes an essential part of the interactive loop to guide the next iteration. Bak et al.also mentions the classification of four types of higher level encounter patterns, which seems to be a continuation from Andrienko et al.[40]’s advocacy of characterizing elementary patterns.

Movement interactions are multifaceted. A visual feedback for adaption and fine-tuning of the analytic system is indispensable to extract interactions. Also, automatic techniques mostly solve lower level tasks such as matching route similarity or spatial-temporal closeness between trajectories. Recognizing general patterns and questioning with alternative assumptions are still a job of human expertise. Thus, we need to keep a open mind to the machine results but, in the meanwhile, expose more visual details for domain judgment.

Movement traces are usually placed on a 2D map as discrete points[25] or linked trajectories[36, 42]. However, consideration of both spatial and temporal changes are necessary to avoid false identification of collocation (position overlap at different time period). There are several approaches to achieve this.

A common treatment here is the Space Time Cube (STC)[43, 44, 45], which projects the temporal dimension in the z-aixs of a 3D view. In STC, real collocation of two entities are depicted as the neighboring points in a 3D space. But it makes visualization work prone to problems like loss of perspective and obfuscation[46, 47]. Subjects that travels in the z-axis can also be another issue[48]. A workaround without using 3D space can be found in AMV[29]. It confines trajectories to the local duration and presents relative movements by removing distractions of trajectory parts that are temporally distant. But the fine details of proximity variations in the selected duration is not supported. Alternatively, abstracting movements from their geo-spatial context, as explored by Crnovrsanin et al.[49], is also a possible way to clarify the subjects’ spatial temporal relations.

In sum, we found visualization facilitation in the wildlife behavior analysis is non-trivial, especially when support of time space relatedness remains an open gap. This motivates new solutions with flexible interactivity for quick selection, navigation, and visual adjustments [50] to assist the domain research.

The domain problem targets at a group of multi-species, free-roaming animals in a South African nature reserve. Researchers’ primary interest lies in the behavior along space/time variations. Instead of analyzing relationships with natural landscape, experts needs visual insights into behaviors of individual animals and how they would influence each other, which is also a valuable compensation to their current tool sets. Individual interactions are also potential to later applications such as to analyze other species, or even human social interactions.

Two ecologists are invited as domain experts (E1 and E2). They both have long interests and experience in animal movements. E1 has a background in movement ecology, spatial analysis. E2 also works in quantitative ecology and environmental sciences. They both conduct quantitative data analysis with R and use field-specific packages to plot the movements either by spatial attributes (map) or numeric attributes (bar chart or line chart). However, they both find their current tools limiting. For example, both of them complained about the lack of support in converting the multidimensionality of movement trajectories into useful information. E2 also mentioned that the analysis becomes challenging when facing multiple individuals regarding their temporal-spatial variations.

For data collection, ecologists attach GPS collars[51] to each of the 25 sampled animals, cf. Figure 1. The subjects are consisted of 5 lions, 10 wildebeests and 10 zebras. The entire data collection lasted for roughly 30 months, which captures periodic changes of climate though different seasons. To ensure sufficient battery life, collar tag sensors were programmed to obtain and store GPS coordinates by every two hours.

Due to various conditions in climate and land features, the quality of tracking data suffered from unpredictable wild circumstances[52]. Cause of quality compromise can be any incident such as physical impact, wet situations on rainy days or water areas, or poor signal reception. To some extent, the collected data are often compromised with one or more of the following issues: a) unrealistic values, i.e. two subsequent data points have impossibly large position difference in between which is evidently an error, and b) missing points, i.e. failure to record a data point at the programmed time. Therefore, pre-processing before analysis is needed[53]. With their expertise, domain experts proceeded with the removal operations to deal with the first type of error (a), resulting in unstable trajectories with irregular gaps of varying length on it. They also trimmed off the unusable parts at the beginning or ending of every animal tracking, making each trajectory starts and ends at different time of its own.

Integrating a visual approach into existing analytic workflow requires shared understanding from both ecology and visual analysis. Through six months’ exchanges with the domain experts, we have devised five design requirements as guidelines to add new capabilities to the current analysis. We begin with draft design proposals in the form of low-fi visual sketches, which inspire nuanced discussions to pin-point their most prominent concerns. To communicate more complex effects, we built iterations of animations with Processing[54] and evaluate of their usefulness in each version. There are also co-design sessions during which they take a more active role to freely propose ideas though paper sketches and explain the intended functionalities. To test for likely insights and also unexpected misinterpretations, qualitative surveys were also conducted among an expanded user group (7 people) of both experts and non-experts. We used structured questionnaires to prioritize the requirements and balance conflicting directions. With constant refinements, we conclude that the visualization design should facilitate movement ecology research by addressing the requirements in the following aspects:

R1 - The ability to intuitively visualize spatial temporal movements in a simple and straightforward manner.

The experts understand the challenges introduced by the multidimensionalilty of collected data. But their experience with interactive visualization tools with sophisticated functionalities is limited. Steeper learning curves can be worrying as they have developed existing habits in thinking with movements. So a design with simple and intuitive depictions would be preferred. They understand the value and efficiency of visual exploration in finding implicit patterns which can be easily missed otherwise. With minimum extra explanation, it should support them in confirming newly discovered visual knowledge with fine data details and facilitating model building with existing analytical skills.

R2 - Interactivity and navigation into spatial temporal space efficiently.

Considering a 2.5 years long time frame of the project, browsing the global timeline means navigating roughly 4000 data points for every 2 weeks of tracking time. Additionally, the patterns can exist in any level of time scopes, e.g. movement difference in days and nights, weeks and even seasons. Granular time scopes should be implemented as movement patterns are sensitive to the observation time frame. Interactively defining time point and temporal selection with flexible scope length will give them the autonomy to explore for a suitable length for particular research task.

R3 - Ability to visualize the movement with awareness and tolerance of inconsistent data quality.

As there are known issues in the data quality, the visualization pipeline should include the ability to produce smooth, consistent visual results. However, instead of ignoring the problems in data, the involved uncertainties should be truthfully presented so that user could easily distinguish whether the visualized outcome is based on solid trustworthy data or questionable delusion. A visual distinction should accommodate the difference clearly and allow for context-dependent judgments.

R4 - Comparing movements between different species.

Same species share behavioral similarities. And the opposite is true for different species. But extent of difference varies between sets of comparisons. For instance, both being herbivores, the wildebeests and zebras can find more commonalities, while lions, as predators, may behave differently than herbivores. To explore species-relevant aspects in movement behaviors, analysis with species awareness is needed. The visualization system should allow them to visually disintegrate the behavior differences, making the anomalies instantly distinguishable.

R5 - A focus on the relationship between animals.

Movement behavior is a complex problem which can be influenced by many environmental factors. It is also not surprising that animals would influence each other’s movements in more or less subtle ways from slight movement deviations to social interactions. Understanding how animals would influence or interact with their movements by both intra-species or inter-species means can contribute to valuable ecological insights. In our case, the queries into the dynamics of how animals are closely bounded to each other and whether such bounding is consistent should be facilitated.

As a side note, the requirement list follows a hierarchical structure where meeting the later, higher level ones involve solving basic ones prior to them.

As aforementioned, context adaptability is important in movement analysis. Parameterized visualization is an effort to support this by controlling local variables, such as species, speed, or group size, to address the problem of concern. In our case, larger time interval size is used to record movement locations. Straight lines connecting location points would appear cluttered with angular shapes and presents little clue to anticipate the movements between locations. Regarding this issue, trajectory smoothing[55] can be helpful as the visualized path can help to build visual heuristics to makes potential behavior patterns easier to uncover and trajectories of different individual more visually distinguishable. The smoothing also produces other effects that will benefit the analysis. For example, the smoothing will make sharper turns with slower animals (ones with shorter distance between with consecutive points), while the opposite applies to faster moving animals. Domain experts can also use a middle point in the curve to adjust their estimations of the heading if necessary.

A decision upon the right amount of smoothness and trajectory shape requires careful deliberation and calibration. To support easy reconfiguration, parameter setting with instant visual feedback can improve the productivity of iterative decisions. This helps domain experts to understand the effect of each parameter to facilitate easier interpretation of trajectory lines regarding R1 and creates adaptability for different analytic scenarios. We understand that the domain experts may occasionally wish to fact-check the bare data points, therefore smoothing should be implemented with an option to be temporally turned off to make explicit locations points visible.

The inherent entanglement of space and time (defined as spatio-temporal dependency in GIS) is a pivotal challenge in movement analysis[56]. Observations in the spatial domain without consideration of temporal stability may result in unreliable or even false positive discoveries[50]. Assisting domain experts to think temporally is essential[57].

To find potential interactions between animals, the spatial proximity should be discussed within the relevant duration context. Based on this principle, relatedness is a concept that describes relationships with spatial-temporal reference. To distinguish relatedness from proximity, relatedness concerns the distance variation in a time range, whereas the proximity only concerns distance in static a time point. Thus, the relatedness can be treated as summary of physical proximity through time.

We employ two basic modes for inspecting how related animals are bounded to each other – the pairwise  (PW) relatedness and the individual-to-group (i-G) relatedness. The pairwise approach takes two entities as input and displays their dynamic relatedness variations. The i-G approach takes a focal entity and displays how closely it is connected to the rest of the animal group over time. These two modes compensate each other for different tasks.

Pairwise relatedness ($\mathit{P_{ij}(t),\;t\in T,\;i,j\in A}$) is a time dependent scalar value describes physical proximity ($\mathit{d_{ij}}$) comparing to the maximum distance ($\mathit{M}$) bounded in the captured area, i.e. $\mathit{P_{ij}=M-d_{ij}}$. Here, $\mathit{T}$ represents all the possible states in the global timeline, while $\mathit{i}$ and $\mathit{j}$ are two elements from all the animals ($\mathit{A}$). Relatedness of one versus multiple targets enables comparison between more candidates and is more complex than pairwise mode. Because the collective pattern of i-G relatedness $\mathit{\,G_{x}(t)}$ is contextually understood as a qualitative, multivariate pattern which considers explicit movement trajectory, relative proximity, and relatedness trend which involves constant changing features among multiple targets. To ease communication, we use a different notation to represent: $\mathit{G_{x}(t)=\{P_{xa}(t)\;|\;a,x\in A\;and\;x\neq a,\;t=\{t_{1},t_{2},...,t_{R}\}\}}$, where $\mathit{x}$ is the focal animal (the one to be compared with the rest of the group) and a time range from $\mathit{\;t_{1}\;}$ to $\mathit{\;t_{R}\;}$ is considered. To deal with the intricacies that are difficult to be summarized as a quantitative measurement, we implemented a combination of visual components to delineate the implicit patterns and variations.

The uncertainty awareness is believed to be useful in improving decision-making[58, 59]. Communicating uncertainty is important in our case because it can false signal the absence of one entity in a mutual relationship, which could mislead the experts’ judgment and indicate a termination of related session. To avoid this, we take a series of measures: Firstly, we perform linear interpolation to fill the missing gaps to ensures steady data flow for the well-functioning of relatedness derivation model. Secondly, we treat the interpolated data points differently by labeling and measuring their reliability. This is realized by assigning a special Boolean label, i.e. interpolated or null, and a measurement of uncertainty extent. The measurement is computed as follows: given the current index $\mathit{i}$ and index range of consecutive missing data points $\mathit{[c,c^{\prime}]}$ ($\mathit{i\in[c,c],~{}~{}i,c,c^{\prime}\in N}$), the degree of uncertainty $\mathit{U}$ can be formulated as: $\mathit{U=min(|i-c|,|i-c^{\prime}|)}$, i.e. the uncertainty in current data point $\mathit{i}$ will be determined by the index distance to the nearest (temporally) reliable data point.

Thus, the uncertainty information prepared to be visualized with different levels of awareness(R3). The experts can make judgment on the credibility and integrity of a pattern by also referencing the visualized uncertainty[60].

Moving animals are represented as animating locations, each with an ID and a species color. Species are colored to resemble the animal’ natural appearance: orange for lion, cool gray for wildebeest, and black for zebra.

Each animal entity draws a trace line with the same color of itself, the length of which corresponds to the global duration. As time coverage is the same for every trace line, drastic movements (bigger distance between steps) will appear longer than sedentary animals, creating a contrast that enables comparison between animal individuals by its movements intensity. Thus, outlier movements can stand out more clearly.

To improve the readability of movement trajectories (as explained in 4.1 Parametric Moving Trace Line), we applied parametric smoothing to the trace lines. The technique is partially inspired by Sacha et al.’s trajectory simplification [55], we integrated commonly used methods like cubic basis spline, natural cubic spline, straightened cubic basis spline based on Holten’s edge bundles [61], Cardinal spline, Catmull-Rom spline with D3.js’ curve interpolation functions. This flexibility in configuration, cf. Figure 3 can be fine-tuned by mode or intensity. Here, the mode determines the type of underpinning smoothing function and the intensity, specified by a linear $\alpha$ value between 0 and 1, offers the option to adjust the amount of smoothness: $\alpha=0$ means no smoothing at all and $\alpha=1$ means smoothest. The goal is to give domain experts extensive free options to pinpoint the right parameter to draw trajectories that caters to their analysis scenarios.

Uncertainties can be visualized following the temporal axis, cf. Figure 4. Small horizontal heatmaps at the bottom right of the screen are drawn to inform data issues in three aspects: 1) the start and end time of available data, 2) the general distribution, and 3) the degree of uncertainty (by depth of color), cf. 4.3 Uncertainty Awareness. The time context is useful to guide expert to skip certain segments by informing where to expect reliable data.

In the spatial domain, moving entities will change both appearance and size once uncertainty happened in the data, cf. Figure 5. Filled circles change to dashed outlines, expanding their sizes to indicate a dilution of positional accuracy. Its opacity also decreases along with the size increase, telling the viewer that the system is unsure about exact location of current animal.

Expert can leverage both display methods to avoid risky interpretations and apply self-disretions with their domain knowledge.

The relatedness can be understood through different setups. We describe two modes to treat them respectively: the relatedness between two individuals (pairwise) and one individual comparing to a group of the rest (i-G).

Granular variation of $\mathit{P_{ij}(t)}$ is plotted with a line chart. Overview-and-zoom functionality[62] is needed here considering the amount of details in 2.5 years of data points. So we implemented a vertically mirrored line chart below it with different purposes for each half – the bottom one can be brushed to select the time range and the top side displays zoomed details of the brushed area., cf. B) in Figure 6.

The chord diagram can provide an overview of relatedness of all possible pairs being presented in the main view. Ones with higher relatedness are drawn in bolder and more saturated ribbons while lighter appearance applied for the lower relatedness, cf. A) in Figure 6. The delineation reacts differently to a duration and fixed time point. If a duration is selected, the system calculates averaged proximities of all intervals in the range to determine the ribbons’ appearance: $\mathit{\overline{P}_{ij}=\frac{1}{R}\cdot\sum_{s=1}^{R}P_{ij}(t_{s})}$. This approach is very similar to the pairwise mode but more aggregated. Experts can brush and drag to tweak the duration length. Such operations are useful to answer questions like "Were the animals’ movements more clustered (related) during the past eight hours? Were they the same for last two days?". It is a simpler and more straightforward way to search for patterns without looking at the spatial changes in the main view.

Geographical pattern of the i-G relatedness ($\mathit{\,G_{x}(t)}\,$) is displayed in the main view where spatial distribution and social context are sensitive aspects. The individuals can be focused by clicking on its circle in the main view. The interaction halts any ongoing animations and creates an array of concentric circles. Relations of animals concerning the focal animal can be visually examined Figure 7. Radii of the circles indicate the spatial proximity to the animal: $\mathit{r=P_{xa}}$. The circles sort proximity of animals with scattered distances and moving trends (came closer or went further) into an egocentric diagram where unidirectional comparison of proximity is possible. Based on this, proximities of current time is easy to tell by circle sizes. The less explicit moving trends, however, are illustrated by the trace lines and relatedness error bars. Here, the former corresponds to the movement trajectory of the duration and the later is a depiction to analyze the trend of relatedness: line caps on both ends of the error bars are determined by the maximum and minimum values of relative proximity in the duration. Thus, whether an entity is drawing closer or moving farther can be read from the negative/positive sum of relatedness, which is derived by comparing length of error bars from the inside (positive relatedness) and outside (negative relatedness) of the proximity circle. For instance, ones with much larger outer length suggests the underlying animal spend most of the recent time in places more distant to the focal animal than its current location, which means it tends to move farther considering the past period. The trace line here is to verify the judgment on the trend of relatedness change with exact movement details.

Background: Seasonal climate change would impact many aspects in an ecosystem. Ecologists need to understand how this is reflected by animal movements. Fortunately, the raw data covers sufficient season cycles of multi-annual time frame. Thus, seasonal difference in movement distribution can be visually compared.

Background: Another potential influence on animal behavior is day-night transition. Unlike humans who tend to rest during most of the night hours, animals may react to day-night alternation differently in order to ensure their survival. The changes in temperature, visibility, as well as the effect of chronobiology could have heterogeneous effects on the movements of species. How the behavior difference could be visually reflected by geographical patterns is intriguing.

Background: Nuanced understanding of animal social interactions with an awareness of species traits plays a key role in the study of animal behaviors. As mentioned before, the grouping and pairing are difficult to detect with the visualization of mere locations due to spatial-temporal dependency. Instead, visualizing the dynamic relatedness can support the ecologists to investigate the strength of social bounding in pairs.

Background: Unlike relatedness within the same species, the inter-species relatedness may indicate threat instead of cooperation particularly between predators and herbivores. According to E1, a real predation process usually takes place within 3-5 minutes, which is beyond the frequency resolution of the employed GPS devices. However, unexplored behavioral patterns and multi-species interactions can still take place over the span of hours, according to E1. The experts would like to use inter-species relatedness to investigate possible instances of such behaviors.

According to E1, an animal can be influenced not only by variation in environmental conditions but also by the behavior and location of another individual or group of animals. Because of displacement in space and time, such relationships are tricky to explore visually. The visualization system allows them to analyze movement through space/time variation (R1) of individuals as well as the relationships between individuals (R5). The expert asserts that exploration into inter-individual interactions is enabled by the chord diagram with color-coded relatedness.

Regarding the pairwise relatedness, the ability to zoom to specific time periods (in pairwise relatedness) is very convenient and easy to use. E2 believes the visualization system highlighted an important capability which is visualizing data along temporal dimension, particularly how relatedness changes over time (R2).

The i-G relatedness enables a visual understanding of the relatedness with actual distance between individuals in smaller time frames. The relatedness variation range raises the awareness of the time dependency in shorter movement trends. Unexpected patterns would emerge after testing and exploring with varying time scales (R2). E2 believes that the view mode is not only useful for generating ecological insights or hypotheses, but also creates more contextual awareness for the analysis.

Before using our system, plotting static figures is their primary way of visual analysis for movements. According to their comments, the visualization creates depictions beyond static figures, without which the dynamics in movements are otherwise hard to interpret. "(Such functionality) is very needed in data exploration", says E1 (R1).

As they are fully aware of the difficulties introduced by inconsistent data, the new visual approach to check data availability/uncertainty is well-appreciated. "To me, this is a very useful tool for exploration of movement data, allowing to focus on different potential problems, such as sociality between individuals, movements relative to predators, home ranging, etc." says E1. The expert also confirms that the awareness of uncertainty is reinforced by trace lines and smoothing parameters which can be used to smooth out uncertain measurements (R3). E2 appreciates the quick configuration changes on the fly. He says, "It makes comparisons between configurations very convenient." (R1).

Movement ecology research often requires calculating implicit features from the data such as road impact[64] or stigmergy[19]. When the optimal features to describe animal behaviors are still unclear and yet to be confirmed, the research becomes challenging. Based on their experience, the experts believe that the system can play a key role in their exploratory stage of analysis, where setting different parameters and scoping down to subsets of data need to be frequently adjusted. Under such circumstances, a comprehensive integration of capabilities that can produce easy to interpret visual insights with quick and convenient configuration changes is essential. According to E2, visualizing certain variables in a spatial-temporal way has changed their way of computing variables, doing analysis, and develop new hypotheses or insights.