Data Management in Microservices: State of the Practice, Challenges, and Research Directions
Abstract.
Microservices have become a popular architectural style for data-driven applications, given their ability to functionally decompose an application into small and autonomous services to achieve scalability, strong isolation, and specialization of database systems to the workloads and data formats of each service. Despite the accelerating industrial adoption of this architectural style, an investigation of the state of the practice and challenges practitioners face regarding data management in microservices is lacking. To bridge this gap, we conducted a systematic literature review of representative articles reporting the adoption of microservices, we analyzed a set of popular open-source microservice applications, and we conducted an online survey to cross-validate the findings of the previous steps with the perceptions and experiences of over 120 experienced practitioners and researchers.
Through this process, we were able to categorize the state of practice of data management in microservices and observe several foundational challenges that cannot be solved by software engineering practices alone, but rather require system-level support to alleviate the burden imposed on practitioners. We discuss the shortcomings of state-of-the-art database systems regarding microservices and we conclude by devising a set of features for microservice-oriented database systems.
PVLDB Reference Format:
PVLDB, 14(1): XXX-XXX, 2020.
doi:XX.XX/XXX.XX
††This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of this license. For any use beyond those covered by this license, obtain permission by emailing [email protected]. Copyright is held by the owner/author(s). Publication rights licensed to the VLDB Endowment.
Proceedings of the VLDB Endowment, Vol. 14, No. 1 ISSN 2150-8097.
doi:XX.XX/XXX.XX
1. Introduction
The advent of large-scale online services provoked an architectural shift in the design of data-driven applications, with resulting needs for designing distributed application systems from the point of views of both computational resources and software development team organization (Fowler, 2014; Newman, 2015a). In particular, we are witnessing the increasing adoption of microservice architectures to replace the traditional monolithic architecture (Figure 1). In contrast to a monolithic architecture (Figure 1(a)), where modules and/or subsystems are integrated and cooperate in a centralized manner, a microservice architecture, (Figure 1(b)), organizes an application as a set of small services that are built, deployed, and scaled independently.
In the scenario of Figure 1, we depict an e-commerce application to contrast both designs. In the monolithic case, to process an order, for example, the Cart module performs a function call to the Order module, which then performs additional function calls to the Stock, Campaigns, and Discounts modules to safeguard the order is correctly placed. In contrast to direct function calls between modules, microservices communicate with each other through remote calls, such as HTTP-based protocols (Fowler, 2014; Gan et al., 2019) or asynchronous messages (Zimmermann, 2017). In a microservice-based application, a new functionality or bug fix does not require a new build and subsequent deployment of the whole application, but instead, it can be managed by redeployment of a microservice unit. In the same line of reasoning, microservices also enable design for failure. As faults in individual microservices are isolated, their propagation to other building blocks of the architecture is limited (Zimmermann, 2017).
Furthermore, each microservice may also manage its own database that is suitable to the data formats and workloads of the microservice. This flexibility is often associated with the polyglot persistence principle, where different categories of database systems (e.g., loosely structured NoSQL vs. relational) service separate microservices (Fowler, 2011). For instance, the Products database may take advantage of a document-oriented model to allow for agile schema evolution, whereas the Stock microservice may rely on the relational model to safeguard constraints over stock items. As a result, a microservice architecture represents a significant shift from traditional monolithic transaction processing systems. In particular, in the monolithic architectural style, transactions can be easily executed across modules, while, in microservices, it becomes necessary to break these transactions down due to the decomposition of the application into small parts.
Motivation. Despite the increased adoption of microservices in industry settings (Cruz et al., 2019; Krylovskiy et al., 2015; Hasselbring and Steinacker, 2017; Gouigoux and Tamzalit, 2017; Richter et al., 2017; M. Del Esposte et al., 2017; Mazzara et al., 2018; Azevedo et al., 2019; Lotz et al., 2019) and the perception that data management is a major challenge in microservices (Knoche and Hasselbring, 2019; Viggiato et al., 2018; Francesco et al., 2017; Luz et al., 2018; Zhang et al., 2019; Soldani et al., 2018), there is little research on the characteristics of data management in microservices in practice. Besides, existing studies provide a limited investigation of the major challenges practitioners face regarding data management in such an architectural style. For instance, it is unclear which database technologies and patterns are adopted, which data consistency semantics are employed within and across services, or which mechanisms are used to exchange data in these architectures. Understanding these issues would provide valuable insights as to how to advance data management technologies to meet the needs of microservice applications.
Methodology. To bridge this gap, this paper presents an investigation of the state of the practice of data management in microservices. Specifically, we perform an exploratory study based on the following methodology: (i) we systematically review the literature on articles reporting the adoption of microservice architectures. From 300 peer-reviewed articles analyzed, 10 representative articles (Mazzara et al., 2018; M. Del Esposte et al., 2017; Azevedo et al., 2019; Richter et al., 2017; Hasselbring and Steinacker, 2017; Krylovskiy et al., 2015; Cruz et al., 2019; Ciavotta et al., 2017; Viennot et al., 2015; Laigner et al., 2020) were selected for review; (ii) we analyze 9 popular microservice-based applications (microservices patterns, [n.d.]; Sentilo, [n.d.]; Microservice-API-Patterns, [n.d.]; Apps, [n.d.]; sczyh30, [n.d.]; EdwinVW, [n.d.]; spring petclinic, [n.d.]; kbastani, [n.d.]; Weaveworks, [n.d.]), selected out of more than 20 open-source projects, and; (iii) we design an online survey to gather the opinions of developers and researchers experienced with microservices in real-world settings, allowing us to cross-validate the findings of the previous steps. In total, more than 120 practitioners provided important information about their microservices’ deployments in industry settings. Taken together, these three interrelated explorations provide new and comprehensive evidence on data management practices and challenges in microservices. Additional details about our methodology can be found in (Laigner et al., 2021b).
Findings. From our investigation, we observed that microservice developers are dealing with a plethora of data management challenges. While microservices are supposed to work autonomously, they often surprisingly end up exhibiting functionality and private state dependencies amongst each other.
As a result, developers have a hard time reasoning about enforcement of application safety within and across microservices. The latter relates to challenges in managing constraints over distributed microservice states, reasoning about the unintended interleaving of event streams, dealing with weak concurrency isolation, enforcing data replication semantics, and ensuring consistency across a variety of storage technologies.
Practitioners are poorly served by state-of-the-art database systems (DBMSs) and end up weaving together several heterogeneous data systems such as message brokers, in-memory caches, analytical engines, and loosely structured and structured DBMSs in an ad-hoc manner. This system complexity leads to a substantial amount of data management logic at the application layer to meet the data management requirements of microservices.
As a consequence, database systems no longer play a central role in this novel paradigm, often relegated to providing data storage functionality. The lack of a holistic view, by means of an unawareness of the dataflow, constraints, and the complex interplay among microservices, leads to the impossibility of effectively ensuring data and application safety in the microservice paradigm.
Contributions. In summary, we make the following contributions:
(i) We survey a gamut of experienced practitioners, the literature, and popular open-source microservice applications to characterize the state of the practice of data management in microservices. The findings (presented along 3, 4, 5, and 6) reveal several ad-hoc data management practices in microservice architectures never reported before in the literature, suggesting that developers are insufficiently served by state-of-the-art DBMSs.
(ii) We illustrate through source code snippets, extracted from popular open-source microservices, challenges that practitioners face when trying to meet the data management requirements of microservices ( 5). These challenges are also highlighted and extended by experienced microservice developers when asked about their most pressing pains regarding data management ( 6), thus illustrating issues that database technology should aim to address.
(iii) Based on the observed data management practice and challenges, we present a set of features for future database systems to tackle the data management requirements of microservices by design, so they can play a central role in this new paradigm ( 7). We discuss why the state of the art is not able to address all the challenges in conjunction and we conclude by suggesting how the features can be realized into a microservice-oriented DBMS.
The three contributions together provide new directions for the database community to start leading the efforts on data management for microservices. Our goal is to inform engineers of future data management systems about the unmet needs of an emerging application paradigm by providing comprehensive evidence of shortcomings and pitfalls microservice developers face when dealing with data management. We view this work as an example of engaging with database system users to understand and characterize the practice in order to derive new research opportunities for emerging applications. We hope the results drive the reflection of our community towards effectively meeting the needs of microservice-based applications.
2. Related Work
Despite the extensive work in microservice architectures in other communities, an in-depth analysis of data management in this context is lacking in existing research. Works in software engineering focus on migrating from monolithic architecture to microservices (Carvalho et al., 2019; Carrasco et al., 2018) or investigating other general software engineering aspects (Ghofrani and Lübke, 2018; Viggiato et al., 2018; Francesco et al., 2017; Zhang et al., 2019; Knoche and Hasselbring, 2019), such as software attributes (e.g., coupling and cohesion). To the best of our knowledge, our work is a first step towards understanding how microservice developers interact with the database systems that the data management community builds, fostering further research.
In addition, our work provides an in-depth characterization of challenges faced by microservice developers while implementing data management logic in the application-tier. In regard to this contribution, although some studies described related pitfalls, such as shared persistence (Taibi and Lenarduzzi, 2018; Pigazzini et al., 2020), and previous literature investigated architectural smells and anti-patterns in microservices (Neri et al., 2019; Carrasco et al., 2018; Taibi et al., 2019; Tighilt et al., 2020), they fail to capture properties of consistency models and technical issues of database systems, such as data replication and constraint enforcement, as we provide in this paper. Most importantly, in light of the many unveiled data management challenges in microservice architectures, our work is the first to reflect on core limitations that prevent database systems from adequately serving the needs of this emerging architectural style.
3. State of the Practice
In this section, we focus on characterizing the motivating factors for data management in microservices as well as the most prominent DBMSs and deployment patterns employed.
3.1. Motivating factors
| Motivation | # | % |
|---|---|---|
| Scalability through functional decomposition | 55 | 35.71 |
| Fault-isolation (e.g., increasing data availability) | 32 | 20.77 |
| Agility on data change (e.g., facilitating schema evolution) | 32 | 20.77 |
| To enable event-driven data management (e.g., as opposed to classic pull-based data querying) | 23 | 14.93 |
| Polyglot persistence | 9 | 5.84 |
| Others | 3 | 1.94 |
Existing works (Fowler, 2014; Gan et al., 2019; Carvalho et al., 2019; Hasselbring and Steinacker, 2017; Newman, 2015a) argue that the major reasons for adopting microservices are related to fault-isolation, independent software evolution (including schema evolution), and scalability of individual system components. By investigating several microservice deployments, we were able to reveal that such desirable properties are enabled by data partitioning and decentralized data management – particularly by means of use of a database/schema per microservice. Thus, these motivations are intrinsically related to data management, and decentralized data management is a major foundation in the adoption of microservices.
To investigate the most compelling directions for future avenues of research in data management for microservices, we asked the survey participants to select the top 2 reasons to adopt a microservices architecture regarding data management. The 5 options given were centered on data management concerns (i.e., no software engineering concerns, such as loose coupling and easier maintenance, were considered). The provided options were influenced by the following: (i) a preliminary assessment with four industry representatives with strong background in microservices through interviews; (ii) the literature review; (iii) the analysis of open-source repositories; and (iv) discussions among the authors. Table 1 shows the options provided and respective responses. We highlight the following important observations (referenced hereafter by O#).
Functional partitioning: To support scalability (i.e., spreading functional groups across databases) and high data availability (i.e., achieving functional isolation of errors), functional decomposition of the application is a major driver for adopting microservices according to both survey respondents (57%) and the literature (Mazzara et al., 2018; M. Del Esposte et al., 2017; Richter et al., 2017; Hasselbring and Steinacker, 2017; Ciavotta et al., 2017; Cruz et al., 2019; Laigner et al., 2020). Though not explicitly mentioned by literature, we deduce from our analysis that functional decomposition is reminiscent of the idea of functional scaling introduced by Pritchett (Pritchett, 2008), a strategy that involves ”grouping data by function and spreading functional groups across [different] databases.”
O1. The results suggest that the design of data management technologies for microservices should focus not only on scalability, but also on stronger mechanisms for fault-tolerance and error isolation.
Decentralized data management: Practitioners developing applications in real-world settings largely deal with evolving requirements and subsequent schema changes (Viennot et al., 2015). The ability of microservice architectures to provide independently-evolving schemas in different services, in contrast with the unified schema of monolithic architectures, is another major driver (M. Del Esposte et al., 2017; Hasselbring and Steinacker, 2017; Krylovskiy et al., 2015; Ciavotta et al., 2017; Viennot et al., 2015). Surprisingly, although polyglot persistence derives naturally from decentralized data management, it is the least cited motivation in both literature (Azevedo et al., 2019; Krylovskiy et al., 2015; Viennot et al., 2015) and the survey, being roughly 3.5 times less cited by practitioners than schema evolution.
O2. The results suggest that the support for a large variety of data models is less of a pressing need than schema evolution in microservices.
Event-driven microservices: Event-driven systems constitute an emerging trend in the design of data-driven software applications (Kleppmann et al., 2019). Microservices are often mentioned as a compelling paradigm for designing event-driven applications (Katsifodimos and Fragkoulis, 2019; Shah, 2017; Laigner et al., 2020). Almost 15% of the participants consider event-driven data management a primary motivation for microservices, which may indicate a trend in industry adoption. Most papers (Mazzara et al., 2018; M. Del Esposte et al., 2017; Richter et al., 2017; Hasselbring and Steinacker, 2017; Krylovskiy et al., 2015; Ciavotta et al., 2017; Viennot et al., 2015; Laigner et al., 2020) mention the use of asynchronous primitives for message-oriented communication to achieve loose coupling among microservices and facilitate decentralized data management. The same trend is encountered in open-source repositories (sczyh30, [n.d.]; Sentilo, [n.d.]; kbastani, [n.d.]; microservices patterns, [n.d.]; Apps, [n.d.]; microservices patterns, [n.d.]; Microservice-API-Patterns, [n.d.]).
O3. The results suggest that asynchronous events should emerge as a concern when devising data management technologies for microservices.
Summary. Our results indicate that functional decomposition, fault isolation, schema evolution, and event-driven architecture are the primary reasons behind the adoption of microservices for data management.
3.2. Database systems and deployment patterns
There are three mainstream approaches for using database systems in microservice architectures: (i) private tables per microservice, sharing a database server and schema; (ii) schema per microservice, sharing a common database server; and (iii) database server per microservice (Messina et al., 2016). We asked the participants which database patterns they use to support data management in their microservices. We also asked the participants what drivers led to the adoption of each database pattern in their microservice architectures.
We observe that most practitioners prefer encapsulating a microservice’s state within its own managed database server (43% of responses). The same trend is observed in the literature and open-source repositories. The participants indicated the following drivers that lead to this adoption: (i) achieving loosely-coupled microservices; (ii) independent data layer scalability, which would otherwise be challenging with a single database server supporting multiple (heterogeneous) tenant applications; and (iii) fault-isolation, which is naturally derived from the already mentioned formation of independent silos of data.
Besides, practitioners, literature, and open-source repository analysis indicate that container-based deployment is the de-facto practice. Each microservice and respective database are bundled in separate containers, thus guaranteeing that each can be scaled independently and faults are limited to the container boundary.
O4. The results suggest that microservices are prevalently deployed in individual containers, predominantly using the database-per-microservice pattern to achieve performance and fault isolation.
Furthermore, we asked the participants what database systems they have been adopting in their microservice-based applications. Our objective was to understand the types of database technologies adopted, specially concerning the data model, performance, and scalability aspects. The adoption of multiple DBMS belonging to at least two of the provided categories is often reported. For instance, 47.97% indicated the use of at least one relational DBMS in conjunction with MongoDB. Besides, we observed a trend (15%) of use of the following stack: a relational DBMS (e.g., PostgreSQL, MySQL, or SQL Server) + Redis + MongoDB + ElasticSearch.
The most common use case for this stack is the use of relational or document-oriented DBMSs for the underlying microservice databases, Redis as a caching layer to provide fast data access to recurring requests, and replication of data through an event-driven approach to ElasticSearch for fast online analytical queries. Overall, the results of DBMS adoption from the survey are very aligned with the reviewed papers and the open-source repositories.
4. Data Management Logic In Practice
Continuing from our last section, we were also interested in characterizing the types of queries and data management logic that are performed in microservice-based applications. Thus, to avoid the threat of misconceptions in answers, we defined a set of open and multiple-choice questions to obtain from the participants relevant information about how business logic is implemented in their applications. Figure 2 shows the types of data management logic identified from the responses, characterized by Cross-Microservice Business Transactions (BT), Event-Driven Computing (ED), Online Queries (OQ), and Stream Processing (SP).
4.1. Cross-microservice business transactions
The studies reviewed from literature (Mazzara et al., 2018; Viennot et al., 2015; Richter et al., 2017; Hasselbring and Steinacker, 2017; Cruz et al., 2019), the open-source repositories (Apps, [n.d.]; kbastani, [n.d.]; Microservice-API-Patterns, [n.d.]; sczyh30, [n.d.]; EdwinVW, [n.d.]; spring petclinic, [n.d.]; Weaveworks, [n.d.]; microservices patterns, [n.d.]), and the use cases described by participants indicate the prevalence of decentralized OLTP-like workloads in microservice-based applications, such as tracking orders in web shop applications.
Evidence from literature and open-source repositories. By analyzing the open-source repositories, we observed that OLTP-like intra-microservice transactions are common, such as those found in conventional applications (Chen et al., 2014; Bailis et al., 2015). Similar to Pavlo’s findings (Pavlo, 2017), we did not find evidence of the use of stored procedures or the configuration of databases in serializable isolation. The same applies to literature.
Regarding business transactions across microservices, 111Our use of the term ”business transaction” in this paper refers to any unit of work carried out across microservices (O’Neil, 2018), not necessarily under ACID guarantees. conversations (i.e., the interaction between a set of consumer and producer services) are prevalent. Hohpe (Hohpe, 2007) argues that orchestration and choreography are the two types of interactions that take place in the context of distributed web applications. Most papers (Krylovskiy et al., 2015; Hasselbring and Steinacker, 2017; Richter et al., 2017; Azevedo et al., 2019; M. Del Esposte et al., 2017; Mazzara et al., 2018; Laigner et al., 2020) and open-source projects (Microservice-API-Patterns, [n.d.]; Apps, [n.d.]; Sentilo, [n.d.]; sczyh30, [n.d.]; spring petclinic, [n.d.]; Weaveworks, [n.d.]; EdwinVW, [n.d.]; kbastani, [n.d.]) report the use of the choreography conversation pattern (Hohpe, 2007) through both synchronous and asynchronous event-based workflows.
In open-source repositories, event-based workflows are dominant, implying that updates and operations affecting other microservices are queued for asynchronous processing. This finding has led us to observe that microservice architectures indeed follow the BASE model (Pritchett, 2008), which targets functionally decomposing an application to achieve higher scalability in exchange for a weak consistency model. The same trend is found in the literature. Besides, some papers report the use of orchestration (Cruz et al., 2019; Ciavotta et al., 2017; Azevedo et al., 2019). Only one open-source project (microservices patterns, [n.d.]) adopts a saga-like orchestration. In any case, the options above are characterized by weak consistency models (Bailis and Ghodsi, 2013) where on an operation that spans multiple microservices, the tasks may complete at any point in time, and the data returned is only eventually consistent.
While the literature (Fowler, 2014; Zimmermann, 2017; Newman, 2015a) mentions the principle that microservices are autonomous components that are independently deployed and evolved, we observed that most microservice-based applications often perform operations that span multiple microservices (Figure 2 and Table 2), which indicates a functionality dependence between microservices. However, consistent with recent discussions in the community (Pavlo, 2017; Helland, 2017; Winslett and Braganholo, 2019; Helland, 2020), we did not find any evidence of distributed transactions, such as through the 2-Phase Commit (2PC) protocol, in both the open-source repositories and the reviewed papers from literature.
O5. Distributed commit protocols do not enjoy popularity in microservices. The lack of effective and intuitive application abstractions may play a role in refraining practitioners from adopting such protocols (Helland, 2017, 2020).
Evidence from industry settings. To further characterize the implementation of business transactions in microservices, we focused on understanding how consistency guarantees are enforced in industrial settings. We asked the participants which mechanisms to coordinate operations spanning multiple microservices they have been employing. The options were defined based on the patterns found in the literature, open-source repositories, along with a preliminary assessment of the survey with industry representatives.222By investigating the literature, we have found that the patterns for inter-microservice operations are not homogeneously defined. Besides, the preliminary assessment confirmed that developers tend to mark this option in a specialized way, e.g., centralized sagas are infrequently identified as orchestration or decentralized sagas as choreography. We then opted to proceed with different options, even though some may be subclasses of others. Table 2 shows the responses sorted in descending order.
| Coordination mechanism | # | % |
|---|---|---|
| Orchestration | 37 | 22.84 |
| Sagas (centralized approach, with a Saga coordinator) | 24 | 14.81 |
| The Back-end for Front-end Pattern (BFF) | 24 | 14.81 |
| Choreography | 22 | 13.58 |
| Sagas (decentralized approach, i.e., no Saga coordinator) | 14 | 8.64 |
| Distributed transactions (e.g., via 2PC) 333Some respondents were selecting the option distributed transactions but their subsequent answers were not compatible with distributed commit protocols (e.g., employing events to trigger actions asynchronously as state becomes consistent in a microservice). We then opted to change the option to 2-Phase Commit. By analyzing such answers, we estimate that 11 of the responses are not compatible with distributed transactions. This would result in 33 responses (20.37%) being attributable to choreography in total. | 14 | 8.64 |
| 2-Phase Commit | 11 | 6.79 |
| Others | 16 | 9.88 |
In contrast with findings from literature and open-source repositories, orchestration-like (including sagas (Garcia-Molina and Salem, 1987) and the backed-end for front-end pattern (BFF)444In this mechanism, a service centralizes the role of performing requests across microservices. (Calçado, 2015; Veloso, 2019; Newman, 2015b)) mechanisms are the most popular in industry settings. To further understand these orchestration-like mechanisms, we asked the participants which orchestration engines they used to support operations spanning multiple microservices. The results highlight that the adoption of custom-made (e.g., company-built) orchestration engines is prevalent among participants (51.2%).
We also asked the participants to briefly describe one of their use cases involving consistency in operations spanning multiple microservices. 32 out of 90 (35.5%) responded and most responses (78%) indicated the implementation of workflows through application code and the use of application-level validations to safeguard the constraints of the workflow.
Despite the prevalence of orchestration-like mechanisms (22.8%), choreographies are also highly mentioned (20.4%, see footnote 3). An interesting quote provided by one of the respondents characterizes how choreographies are implemented in industry, which aligns well with our findings in literature and open-source applications:
I am absolutely against the business logic inside the database. Depending on the scale I would refrain from using transactions at all, favoring an event-driven approach, with eventual consistency and micro-transactions.
The main difference between the orchestration-like mechanisms described by the participants and the choreography mechanisms found in the open-source repositories, literature, and participants is the type of communication. The former is synchronous and HTTP-based, whereas the latter is mostly event-based and asynchronous.
A substantial fraction of the surveyed papers (Mazzara et al., 2018; M. Del Esposte et al., 2017; Richter et al., 2017; Hasselbring and Steinacker, 2017; Cruz et al., 2019; Ciavotta et al., 2017; Viennot et al., 2015) employ choreography as the mechanism for cross-microservice coordination and do so by means of the implementation of asynchronous and event-based workflows. This pattern is consistent with the responses obtained from practitioners, as choreography is highlighted as one of the most prevalent mechanisms (Table 2). Most analyzed open-source repositories (Apps, [n.d.]; microservices patterns, [n.d.]; Sentilo, [n.d.]; sczyh30, [n.d.]; EdwinVW, [n.d.]; Weaveworks, [n.d.]; kbastani, [n.d.]; Microservice-API-Patterns, [n.d.]) also show the same pattern.
O6. The results suggest the prominence of orchestration-like mechanisms in industry settings, in contrast to the prevalence of choreography in the open-source repositories. Additionally, 2PC is not used often, while asynchronous and event-based coordination is the norm.
4.2. Online queries
While one may argue that queries spanning multiple microservices are antagonistic to the principles of state encapsulation and independent data silos of microservices, we found abundant evidence of such cases in the literature (Azevedo et al., 2019; Laigner et al., 2020; Ciavotta et al., 2017; Viennot et al., 2015) and open-source repositories (microservices patterns, [n.d.]; Apps, [n.d.]; spring petclinic, [n.d.]; Sentilo, [n.d.]; sczyh30, [n.d.]). Therefore, to further understand this trend, we asked the survey participants to identify if they have implemented queries aggregating data from multiple microservices and to describe one of their use cases. 27 (40.3%) of the participants declared the use of some mechanism and 22 of them (81.4%) provided a short-answer describing it. We unveiled three mechanisms to allow for such queries and explain them as follows.
A. Queries aggregating data belonging to different microservices. In this mechanism, a consumer service contacts, often through HTTP requests, a set of microservices through their APIs. After receiving all responses, the consumer service then aggregates the data in-memory (also performing joins, if necessary) and serves the client. We identified the following three practices to implement such a mechanism: (i) Six respondents described the use of composition of service calls, i.e., a microservice performs the necessary synchronous requests to retrieve data from other microservices. (ii) One respondent declared the use of the BFF pattern (Calçado, 2015). We also identified such practice in a repository (Apps, [n.d.]). (iii) Lastly, one respondent declared the use of the API Gateway pattern (Richardson, [n.d.]). We also observed its adoption in open-source repositories (microservices patterns, [n.d.]; spring petclinic, [n.d.]; sczyh30, [n.d.]) and the literature (Azevedo et al., 2019; Krylovskiy et al., 2015). From the respondents’ answers, we could not observe significant differences between the BFF and the API Gateway patterns in terms of query serving.
B. Replication. We also unveiled the use of ad-hoc mechanisms for data replication across microservices for online querying purposes. We explain the identified practices as follows.
(a) Replication across microservices. This practice is characterized by a microservice generating events related to its own updated data items and communicating these changes asynchronously, often through persistent messaging supported by a message broker. In 7 out of 9 (77 %) projects analyzed (sczyh30, [n.d.]; Sentilo, [n.d.]; kbastani, [n.d.]; microservices patterns, [n.d.]; Apps, [n.d.]; microservices patterns, [n.d.]; Microservice-API-Patterns, [n.d.]), we found code fragments used for communication-based replication that presuppose weak delivery semantics. In other words, although updates to the same object are often sequentially ordered by the publisher, there is no ordering guarantee regarding updates to different replicated objects. As a result, causal dependencies are ignored on updating replicated data items. The responses provided by 5 participants in open answers suggest the same trend. This choice appears to be consistent with the eventual consistency semantics adopted by the synchronous query mechanisms described above.
(b) Replication to a database. This practice is characterized by two mechanisms: (i) Daemon workers, one for each microservice and its respective generated events, or a central service, are responsible for subscribing to data item updates and replicating these to a special-purpose database used for querying; (ii) this practice is also characterized by the use of batch workers (usually special-purpose microservices) to extract data from microservices periodically (with a pull-oriented strategy) and replicate those in a neutral data repository for fast querying (e.g., ElasticSearch). We suspect the second approach is reminiscent of the behavior of Extraction-Transform-Load (ETL) tools (Vassiliadis, 2009). Although it is unknown why ETL tools are not being employed for such task, we believe the dynamicity provided by the autonomous deployment of microservices plays a role. In other words, microservices can be easily put to and removed from operation while not affecting others, whereas such a pattern is often not found in computational steps of ETL tools.
(c) Lastly, the use of data stream processing systems (DSPSs) for processing streams (e.g., updates to data items) generated by microservices to build materialized views was also mentioned. One respondent declared: “[…] materializing views over various time windows, making them queryable to other services.”
C. Views. While service composition and replication are often subject to the adoption of the database per microservice pattern (Section 3.2), when microservices share the same database, practitioners may rely on views across multiple schemas to serve cross-microservice queries. This is the least cited practice.
O7. The results highlight that the decentralized data management principle does not refrain microservices from performing queries over distributed states. As a result, practitioners often rely on ad-hoc mechanisms for data processing at the application-level.
4.3. Stream processing
To understand how data stream processing systems (DSPSs) that the database community builds interact with microservice architectures, we asked the participants if they have already employed a DSPS in conjunction with microservices. We also asked them to provide a description of one of their use cases. 18 (26.86%) out of 67 respondents declared the use of DSPSs and 13 (19.4%) of them provided a description. We summarize these as follows.
Data processing pipelines. We observed the use of application libraries targeted at stream processing (Kafka, [n.d.]) in microservices to perform data transformations, as mentioned by a participant: “We use Kafka Streams to reorder (time window) out-of-sequence data[.]” Besides, one respondent declared the formation of an “ETL chain implemented with microservices that exchange information asynchronously using Kafka as a message bus. […] We have several microservices in an ETL chain, [… including] several transformation steps.” The person explains that “the chain will fork at a certain point and one side of the fork will carry on the transformation up to message delivery to downstream systems, the other side of the fork will do asynchronous writes to an operational data store (a NoSQL database). [..] These writes are asynchronous to remove the DB from the critical path and ensure that messages can be delivered in near-realtime.”
These responses represent a surprising trend, since we did not find substantive evidence of the use of microservices for forming a data processing pipeline in the literature and open-source repositories. Existing literature suggests an impedance mismatch between microservices and DSPSs (Katsifodimos and Fragkoulis, 2019; why, [n.d.]; Wang et al., 2019) particularly related to the stream processing abstraction. Often in the form of static dataflow graphs, operations such as filter, join, and aggregate are applied uniformly to all stream items in such abstraction. This model notably contrasts with microservice principles, including loose-coupling, fault-isolation, independent evolution, and autonomous deployment. Most importantly, the dynamic behavior of microservices, including operating over data items from different microservices, introduce a significant challenge to express complex business logic using this abstraction.
O8. The responses suggest microservice developers find the static dataflow abstraction difficult to express their computations. As a result, they end up relying on the ad-hoc formation of data processing pipelines by microservices.
Anomaly detection. The use of DSPSs for anomaly detection based on event streams generated by microservices was mentioned by two respondents. For instance, one respondent declared the adoption of a stream processing engine for “monitoring financial operations by analyzing situations and generating critical events for microservices that convert these events into alerts.”
Replication and materialized views. As already mentioned in the last section, four respondents indicated the use of stream processing engines to process data generated from microservices aiming at replicating data and materializing views.
4.4. Event-driven computing
As mentioned previously, we observed that microservice architectures often follow the BASE model (Pritchett, 2008), wherein organizing a computation in an event-driven architecture (EDA) is argued for scalability and architectural decoupling. In an EDA, events that are relevant to an incoming request are generated when a consistent state is reached to allow further processing (Pritchett, 2008). To characterize how EDA intersects with microservice architectures, we asked the participants to declare whether they have performed event-driven computations in microservices through an EDA and provide an example of one of their use cases in a short answer. A total of 45 (67.16%) out of 67 participants indicated the use of such computations and 26 (38.81%) provided a brief use case description.
Overall, the responses are consistent with the findings from the literature and the open-source repositories. EDA is often an enabler of event-based and asynchronous workflows. Some quote snippets are provided as follows: (a) “Pure choreography without central orchestrator;” (b) “natural way for microservices to collaborate when they depend on data from other microservices;” (c) “Payments system using an [EDA] to process ecommerce orders/payments.” (d) “[…] to trigger several microservices in our architecture.”
5. Microservices through the looking glass
In Sections 3 and 4, we observed that microservice applications deviate significantly from the architecture of traditional database applications, introducing significant data management logic at the application-level and a decentralized model for data management.
Such findings urged us to investigate challenges faced by developers that would drive research avenues in data management. Particularly, we aim to answer whether the observed shift promotes challenges that cannot be solved by traditional database systems.
Therefore, we present next a series of challenges (referenced by C#) faced by developers that reveal a myriad of unmet needs that should be appropriately addressed by the database community.
5.1. Cross-microservice validations
Background. Given the ubiquity of business transactions across microservices, we start by reviewing a type of application-level validation employed by developers to ensure correctness in such cases. Take for example the snippet adapted from the project vertx (sczyh30, [n.d.]) shown in Listing 1. Prior to proceeding with the client’s checkout request, the cart microservice verifies, through a HTTP remote call to the inventory microservice whether the items in the cart are available. If so, it generates an event requesting the corresponding order to be processed.
However, such validation is not safe under concurrency. By the time the Order microservice processes the event, one (or more) of the items in the cart might not be available anymore. Under high contention such criteria may lead to abusive generation of events, resource trashing, and may also introduce the burden to deal with compensation logic in the application. It is worthy to note we encountered such a pattern in several other projects, such as (microservices patterns, [n.d.]; spring petclinic, [n.d.]; Microservice-API-Patterns, [n.d.]; Apps, [n.d.]; kbastani, [n.d.]; EdwinVW, [n.d.]).
Discussion. In the case presented in Listing 1, coordination is a condition necessary to ensure correctness under conflicting reads and writes. However, in the absence of efficient solutions and intuitive interfaces for encoding concurrency control semantics in the application-tier (Helland, 2020), developers end up encountering challenges to safeguard constraints across microservices.
Another impediment that makes the problem even more difficult comes from the heterogeneous database systems encountered in microservice architectures, the incompatible isolation levels, and the corresponding lack of interoperability among them.
C1. In the absence of efficient or viable solutions for isolation guarantees in the application-tier, microservice developers are exposed to concurrency anomalies. This creates a great barrier for expressing correctness criteria across different microservices.
5.2. Implicit cross-microservice associations
Background. Even though microservices follow an approach similar to BASE for functional decomposition (Pritchett, 2008), developers do not refrain from modeling implicit associations across microservices.555We refer to ”implicit associations” as those relationships between tables from different microservices that would exist if the application schema was designed as a single database. We encountered several cases where enforcement of foreign key constraints across microservices is a necessary condition for ensuring correctness, and the applications analyzed show no evidence of such enforcement.
Consider the source code snippet exhibited in Listing 2 adapted from event-stream (kbastani, [n.d.]), vertx (sczyh30, [n.d.]), and eShopContainers (Apps, [n.d.]) applications. In the event of a product removal from the product microservice, in the absence of cascading delete, operations carried out by the system over records stored in other microservices that rely on the existence of the deleted product(s) will still assume such a product exists, which may bring the system to an inconsistent state. This pattern is observed across several other microservice applications (Microservice-API-Patterns, [n.d.]; kbastani, [n.d.]; Apps, [n.d.]; sczyh30, [n.d.]; microservices patterns, [n.d.]; EdwinVW, [n.d.]).
Discussion. Microservice developers have to explicitly enforce implicit foreign key constraints across microservices to avoid bringing the system into an inconsistent state, a complicated and error-prone task. However, in most cases, practitioners simply either ignore or are unaware of the consequences. Giving up foreign key constraint enforcement across microservices leads to ”orphaned” records in one or more microservices. Such a pitfall is even worse than encoding associations under non-serializable isolation, since it may lead to a much higher number of anomalies (Bailis et al., 2015).
C2. The impossibility of declaring foreign key constraints between different microservices’ schemas creates a great barrier for developers to enforce constraints across microservices.
5.3. Cross-microservice queries
Background. As part of our study, we observed the popularity of online queries in microservice architectures (Section 4). Given the distributed nature of data stores, the data encapsulation may introduce challenges not found in traditional monolithic systems.
The literature (Azevedo et al., 2019; Krylovskiy et al., 2015) and open-source repositories provide interesting examples of microservices being employed for data aggregation through queries spanning different microservices (and their underlying databases), often with different data models. Such implementations reveal a new trend on stateful middle-tier applications that are particular to microservices: encoding of data processing functionality at the application-level.
Consider the example adapted from the project FTGO (microservices patterns, [n.d.]), shown in Listing 3. The code snippet exhibits a method (getOrderDetails) responsible for reaching out to several microservices in order to consolidate in real-time a client view (i.e., the order details).
Discussion. Current state of the practice leaves the developer responsible for retrieving the appropriate data from each microservice and dealing with possible inconsistencies, such as fractured reads (Bailis et al., 2016), which may lead to a complex code base and bugs. Besides, with such application-level data management there is no way to ensure that reads to different microservices reflect a view of the entire application at a single point in time. In other words, transactional consistency (Ports et al., 2010) is not possible. Lastly, with such a sequential request pattern, as observed in Listing 3, some (or all) requests to microservices may fail, thus leading to missing records and an incomplete result. Such a pattern is also found in (Apps, [n.d.]; spring petclinic, [n.d.]; Microservice-API-Patterns, [n.d.]; EdwinVW, [n.d.]).
C3. Developers have no support for querying multiple microservice database states consistently and they end up encountering challenges on reasoning about the application state.
5.4. Feral ordering
Description. Consider an application in the e-commerce domain. After adding several items to a cart, which is managed by the cart microservice, the customer may initiate the order’s payment process through the API of the payment microservice. Suppose that prior to submitting the order, the cart microservice emits an event representing the change of the price of one of the items in the customer’s cart.
Two options may apply in this case: (i) Application constraints require that events related to changes in the price of items should be reflected in the orders submitted for processing; or (ii) application constraints are such that price changes should not affect such orders. However, we noticed cases where the application does not enforce any constraint. Both options may apply depending on the eventual delivery of events in the system.
Consider the example adapted from the project eShopContainers (Apps, [n.d.]) shown in Listing 4. The Basket microservice receives basket checkout requests through the method CheckoutAsync. Additionally, the Catalog microservice, upon an item price update request from a user, dispatches a ProductPriceChanged event to interested parties. As there is no synchronization, whether the new price will be reflected in the user’s order depends on the eventual arrival of events, potentially violating application constraints. The same pattern is found in other projects (Apps, [n.d.]; sczyh30, [n.d.]; Microservice-API-Patterns, [n.d.]; EdwinVW, [n.d.]; kbastani, [n.d.]).
Discussion. Literature mentions that developers face challenges in specifying the consistency requirements for their applications (Tanenbaum and van Steen, 2016). Given that microservices comprise a class of applications that adopt a distributed architectural style (Newman, 2015a), it is understandable that defining consistency requirements for these applications may become even more challenging. Indeed, there is a lack of support for developers in reasoning about event-based consistency requirements.
C4. Due to the complex interplay between microservices’ behaviors, asynchronous events are generated to trigger computations. However, avoiding anomalies related to the unintended interleaving of events across microservices is a challenging task.
5.5. Replication hell
Background. As mentioned in Section 4, microservice architectures may rely on replicating data items across different functional silos to avoid employing synchronous requests spanning multiple microservices for data retrieval and subsequent application-level aggregation. In this context, we observed the prominence of ad-hoc mechanisms for replication. This practice is characterized by the absence of ordering guarantees regarding updates to different objects. The data items arriving from different microservices are often aggregated in queries without any consistency guarantee, i.e., not reflecting a view of the entire system at a single point in time (Ports et al., 2010). This pattern is observed in several microservice applications (microservices patterns, [n.d.]; Apps, [n.d.]; Microservice-API-Patterns, [n.d.]; Sentilo, [n.d.]; EdwinVW, [n.d.]; kbastani, [n.d.]).
Discussion. Although one may argue some applications rely on a weak consistency model, such as eventual consistency, to support state querying, it is important to highlight that not all applications fall in this category. Effective mechanisms to support developers are important. In this vein, solutions such as Synapse (Viennot et al., 2015) hold potential to provide more principled replication guarantees in microservices. Even though Synapse (Viennot et al., 2015) is grounded on the industry-strength MVC pattern (Gamma, 1995) and meets the polyglot persistence paradigm of microservices (by allowing seamless integration of heterogeneous databases), to the best of our knowledge, the solution has no implementation in frameworks for a variety of programming languages and databases so far, which may hinder a wider industrial adoption. As witnessed in our results, reaching different software development communities is an important feature.
C5. The lack of comprehensive support for data replication across microservices lead developers to rely on ad-hoc application-level replication mechanisms, a choice that often leads to inconsistency among microservice states.
5.6. Non-transactional queuing
Background. In an intra-microservice transaction, updates affecting other microservices are queued for asynchronous processing. As advocated by Pritchett (Pritchett, 2008), this queuing must be part of the transactional context of the database operation in the originating microservice. However, as observed in our extended version (Laigner et al., 2021b), there is no evidence of use of message persistence queues integrated with the database. Practitioners rely on external message persistence queues (e.g., message brokers) without employing a distributed commitment protocol for message queuing.
Consider the example adapted from the project Lakeside Mutual (Microservice-API-Patterns, [n.d.]), shown in Listing 5. The method createInsuranceQuote performs an insert operation in the database regarding the data item insuranceQuote. Afterward, a message representing the operation is queued without a transactional guarantee. In case of error, logging is performed, but no additional measures are taken by the application, which continues its execution. This pattern is identified in several applications (Sentilo, [n.d.]; Microservice-API-Patterns, [n.d.]; EdwinVW, [n.d.]; kbastani, [n.d.]).
Discussion. Existing database systems often provide support for persistent messaging or notification mechanisms integrated into the database (Documentation, [n.d.]). However, to use these mechanisms, different clients are forced to access the same database instance. As a result, most open-source projects and the literature rely on external message brokers for the communication and event exchange between microservices. Without interoperability of databases and message brokers, 2PC is not possible, violating the atomicity property prescribed by Pritchett (Pritchett, 2008).
C6. Developers lack viable and efficient abstractions for transactional queuing in microservice architectures. As a result, anomalies arise due to lack of isolation and ad-hoc fault-handling, leading to challenges on ensuring application correctness.
6. Delving into Developers’ Pains
To strengthen our confidence on the practical challenges revealed in the last section and derive additional ones, we inquired practitioners about pressing challenges they face while dealing with data management in microservices.
We present next an aggregated discussion over the responses collected from the participants. When appropriate, we include the percentage of each response and the challenges associated. Details about the questions and overall methodology followed in this questionnaire can be found in our extended version (Laigner et al., 2021b).
A. Constraint enforcement and schema evolution. In line with C1 and C2, fixing data inconsistencies (19%) and enforcing correctness through application-level validations (24.5%) are prevalent challenges, which, in conjunction with descriptions provided by the respondents, revealed two major interrelated issues.
On the one hand, application-level data consistency and integrity problems, such as “app[lication] code was not […] removing all usages of an item (in a NoSQL DB[MS])” and “ensur[ing] data is persisted correctly and not duplicated” are mentioned by participants. On the other hand, the use of asynchronous communication for cross-microservice operations or for data replication introduces challenges when it comes to schema evolution in individual microservices, matching our hypothesis highlighted in O2. For instance, one respondent declared that “in an async[hronous] environment and having microservices be[ing] responsible for processing their own changes, issues introduced with new releases on microservices may cause inconsistencies in data processing which are generally hard to correct after the fact.” Another respondent declared that “as there is no one ruling constraint system, data cleanliness is a significant challenge.”
C7. Changes made to a microservice’s schema might necessitate adaptations in the structure of messages exchanged; however, application logic in dependent microservices could still be making conflicting assumptions regarding invariants.
B. Eventual consistency. In line with open-source projects and the literature, respondents have also pointed out eventual consistency (15.68%) as a challenge. A participant declared: “Dealing with eventual consistency was particularly hard when convergence happens into a broker state. The complexity of the approaches hands a great barrier for developers.” We observed two primary drivers for non-converged state in microservices: (i) resorting to event-driven and asynchronous replication to avoid synchronous communication and consequently high latency in online queries (Section 5.5); (ii) employing workflow-oriented business transactions across microservices, often driven by asynchronous and event-based communication (Section 4).
In both cases, reasoning about the global state of the application is a major concern. In the first case, it is hard to determine when replication has occurred to a sufficient degree, especially as ad-hoc mechanisms are often employed by practitioners. In the second, it is challenging to assert whether a multi-microservice workflow has terminated and what its current state is.
C8. Due to the distributed nature of microservice architectures, eventual consistency is often taken as the de facto consistency model by practitioners. This choice introduces a series of challenges on reasoning about distributed states and invariants.
C. Ensuring consistency between heterogeneous database systems. Although one may argue that the eventual consistency approach, i.e., convergence through asynchronous events representing data item updates, is a sufficient consistency model for most microservice-based applications, our survey results show that there is a class of applications that requires stronger guarantees over writes performed in different database systems (14.70%). For example, a participant argued that “atomicity can not be guaranteed over different storage technologies, no information or proper literature. Guessing and fixing error approach.”
In addition, the prevalence of in-memory caching systems for increasing throughput and decreasing data access latency in microservices ( 3.2) may also introduce challenges when developers resort to asynchronous writes. For instance, a respondent declared: “writes are asynchronous to remove the DB[MS] from the critical path [of a data processing pipeline] and ensure that messages can be delivered in near-realtime, […] but we’ve already had few situations where the cache expired before the information had actually had time to be persisted in the DB,” suggesting measures outside the database were necessary to correct the anomaly.
C9. Developers deal with data inconsistencies with a myriad of ad-hoc measures, such as manual intervention to the underlying microservice databases as well as applying custom-built data management logic to handle possible inconsistencies.
D. Consistent querying. As revealed before ( 4.2 and C3), online queries are prominent in microservice architectures. However, there is no de facto approach as developers tend to rely on a variety of ad-hoc mechanisms for online queries (8.82%). Moreover, a consistent view of global state is also expressed as a requirement: “We are integrating data from different sources in a global transportation network. The changes in data are flowing into our system consistently. We need to integrate as fast as possible to present a ‘picture of the moment’ to the global end-users.” To this matter, a respondent indicated “polystore databases [as a solution] to provide location independence and semantic completeness for queries performed by heterogeneous microservices.”
E. Poor functional partitioning. Some participants indicated poor functional partitioning as a potential cause of data consistency problems. For instance, one argued that “microservices made people separate code when they should not be separated, causing this eventual consistency everywhere. […] people wanted to create a separate microservice, just because of ‘size’, and we end up having consistency problems.” Although initial work suggests considering database-related attributes, such as relationships between relations to derive a functional partitioning (Laigner et al., 2019), further explorations with real-world systems are necessary.
C10. Decomposing application functionality without proper thought onto constraints and consistency requirements may lead to the burden of dealing with data inconsistencies.
F. Limitations of benchmarks. We realized that existing benchmarks (e.g., DeathStarBench (Gan et al., 2019)) fail to incorporate the asynchronous and event-driven properties of microservices. This can be challenging for programmers to test new solutions.
C11. The lack of a benchmark that properly reflects real-world deployments refrains developers from effectively experimenting with and reasoning about microservice deployments.
Summary. The perception of developers regarding the challenges faced while dealing with data management are very aligned with our findings discussed in 5. Additional challenges were revealed, such as schema evolution (C7) and data cleaning (C9). This overall perception strengthens our confidence that there is an unaddressed need for effective data management support in microservices.
7. Towards Microservice-oriented Database Systems
In this section, we devise a candidate set of features for a microservice-oriented DBMS based on the observations and challenges identified in the previous sections. Then, we turn our attention to explain why state-of-the-art DBMSs are insufficient for the needs of microservices in the light of the proposed features. Finally, we discuss how to realize the features into a microservice-oriented DBMS.
What features are required for a microservice-oriented DBMS?
As explained in 5.4, developers have no way to specify event-based constraints. For example, concurrent checkout requests in eShopContainers (Apps, [n.d.]) may be processed in arbitrary order wrt. other events, hence providing no guarantee of applying updated prices before checkout.
F1. The database system should be aware of events cutting across several microservices, so that it can capture the complex interplay of data exchange between microservices.
F2. Event-based awareness allows the database system to enforce event-based constraints, so that application safety can be supported.
Besides, as explained in 3.1, 4.1, and 4.4, practitioners employ communication through asynchronous events to decouple microservices. However, it is observed that, by decoupling, developers have a hard time enforcing high data consistency ( 5.4, 5.5, 5.6).
F3. The database system should provide abstractions that strike a balance between loose coupling among components and high data consistency, so that effective and efficient cross-microservice coordination can be achieved.
Constraints cutting across several microservices are often enforced through application-level validations ( 5.1). As explained in 5.2, such an approach leads to an implicit logical data dependency – given that it is not formalized as a constraint – from the Cart microservice to the Product microservice.
F4. The database system should allow users to specify cross-microservice associations and enforce them consistently, so that users can avoid encoding error-prone validations at the application level.
In addition, as microservices extensively employ cache stores to reduce the costs of querying database states, the constraints across microservices should also take into account the consistency of the data stored in the underlying microservices’ cache stores (C9).
F5. Database systems should allow the user to specify the freshness semantics between the underlying cache and database states, so that application safety can be safeguarded through adequate cache consistency management at the microservice level.
As explained in 4.2, replication is often employed so that microservices can avoid coordination via cross-microservice queries. Microservices then queue or publish their own data items’ updates as asynchronous events, which are then eventually delivered to consumer microservices. However, expressing data replication through application-level mechanisms creates a great barrier for developers.
F6. The database system should offer effective abstractions for users to specify data replication across microservices, so that varied replication consistency models can be properly supported.
In 4.2 and 6, we observed the practice of retrieving data from several microservices to build a real-time view. However, microservice developers find challenges when implementing consistent cross-microservice queries, such as point-in-time views ( 5.3).
F7. The database system should provide native support for consistent cross-microservice online queries, but at the same time respect the data sovereignity principle of microservices.
F8. The database system could optimize cross-microservice queries through pre-processing the data, pre-joining the data, or proactively replicating the data owned by different microservices while fulfilling the consistency-isolation levels required by the application.
Why are Web-Scale Application Architectures not Enough?
Although microservice applications can be considered a realization of BASE, an approach where functional partitions are eventually consistent, the microservice paradigm introduces data management requirements not originally envisioned by Pritchett (Pritchett, 2008), including but not limited to application-level replication, consistent cross-microservice queries, cross-microservice validations, heterogeneous services, events cutting across several microservices, and interleaving of distinct, but correlated events that lead to data races. As prescribed by the BASE model, functional partitioning necessarily involves pushing constraint enforcement to the application level (Pritchett, 2008), which as we have seen often leads to consistency problems in microservice architectures ( 5). Lastly, BASE can be considered a set of principles for designing functionally partitioned applications, thus not a full-fledged data management solution.
Furthermore, there have been several recent works in Internet-scale database services. These systems provide high availability at a global scale, at the same time offering high throughput data processing and multi-tenancy by design. While some of them support strong consistency guarantees, such as Google Spanner (Corbett et al., 2012), Amazon Aurora (Verbitski et al., 2017), and Azure Cosmos DB (Azure, [n.d.]), others trade stronger consistency for performance, such as DynamoDB (DeCandia et al., 2007) and Astra DB (Stax, [n.d.]). We address the common shortcomings of these solutions in the context of the aforementioned features.
F1 & F2. In addition to the lack of atomic event queuing in Internet-scale databases ( 5.6), database systems are unaware of the complex interrelationships among events in microservice applications. Thus, it would be desirable for developers to be able to specify event-based constraints on their processing, with the enforcement being performed at the database layer ( 5.4).
Consider, for example, a scenario found in popular e-commerce microservice applications (Apps, [n.d.]; sczyh30, [n.d.]; Weaveworks, [n.d.]; kbastani, [n.d.]) where an order checkout is optimistically processed, i.e., the products in the order are sold without checking their availability in stock. Connecting with Figure 1(b), the Stock microservice might subscribe to checkout and payment events in order to determine whether the process of acquisition of new stock should be triggered. However, the Stock microservice should only take into account checkout events whose corresponding payment events have arrived, since the sale of the products is only finally confirmed after payment.
In the presented scenario, resorting to a single tenant in a multi-tenant database system would allow practitioners to write database triggers for the latter functionality; however, this approach introduces an undesirable dependence between the schemas of the various microservices. By contrast, practitioners could resort to multiple tenants, e.g., one per microservice for isolation. Unfortunately, in this case, the checking of the stock replenishment condition would need to be pushed to the application level.
F3. As pointed out above, it is not obvious how to employ a multi-tenant DBMS to store the states of distinct microservices. One solution is to employ one tenant per microservice, but this leads the states to be completely isolated. Another possibility is to use a single tenant for the states of multiple microservices; however, the latter implies complete sharing. For instance, Figure 1(b) depicts the case where Orders requires coordinating with Stock, Campaigns, and Discounts to safeguard that an order transaction is correctly placed. Coordinating these 4 microservices would require either resorting to application-level 2PC or ad-hoc application-level coordination mechanisms (O5, O6) – in case states were partitioned across tenants – or use of DBMS transactions – in case states were shared. In either case, the principles of loose-coupling and autonomy among microservices would be jeopardized. On the one hand, application-level 2PC would imply API coupling with high consistency, while ad-hoc application-level coordination would typically entail loose coupling at lower data consistency levels. On the other hand, sharing of the same database tenant would imply that microservices lose the ability to independently evolve and fail.
F4 & F5 & F6. In the same line of reasoning of F3, multi-tenant database systems (i) cannot enforce cross-microservice constraints, (ii) cannot reason about cache consistency over distinct microservices’ database states, and (iii) cannot provide consistent data replication across tenants without failing to meet the decentralized data management and data sovereignity principles prescribed by the microservice paradigm. Consider, for example, the case where the state of Payment microservice refers to the customers relation in the state of the Customer microservice. By resorting to a single tenant for both microservices, we could potentially express a foreign key across their respective schemas. However, that would create a schema and failure dependency between the microservices. By resorting to different tenants, there would be no way to express the foreign key constraint across the two microservice states.
F7 & F8. Consider a scenario where in the context of a customer’s complaint about an undelivered package, a front-end microservice may need to provide a real-time view over the customer’s interactions, discounts being applied, and the customer’s credit score to enable business users to decide whether the customer can receive an additional compensation for what happened or only be reimbursed. Such an operation necessitates querying data from several microservices’ states. However, it is unclear how existing database systems can support consistent cross-microservice queries.
Simply providing querying services over the private states of multiple tenants is insufficient to achieve consistent views. Rather, the DBMS needs to capture the interactions between the microservices so that it can analyze and support consistency semantics. Although a DBaaS vendor could potentially provide a holistic distributed state view across tenants (considering all microservice states are deployed as a tenant), such a feature would still be insufficient if not addressed in conjunction with all the prescribed features that necessarily require knowing more about the application, as we address next.
How to realize the features into a microservice-oriented DBMS?
Usually, data management tasks executed at the application level are a black-box to the database system (Yang et al., 2018). As a result, the database system is often unaware of the complex data management logic and constraints put forth by application developers (Bailis et al., 2015). This impedance mismatch is particularly worsened when it comes to microservices, since data is flowing outside of the database (Helland, 2020), that is, there are many data management tasks that cut across microservices. The DBMS is oblivious to these tasks. To realize the features described, we believe it is essential for the database system to better understand what is occurring at the application level. To bridge this gap, we envision two paths:
(A) Extending current abstractions for building stateful middle-tier applications can be seen as a fruitful solution. Laudable initial efforts on virtual actors (Bernstein et al., 2014) and stateful functions (Tai, 2020) have not yet incorporated the features required by microservice applications to a sufficient degree. For example, in Orleans, data durability functionality is explicitly managed by the developer. In addition, there is no way to specify event-based constraints and it is unknown whether all microservice applications can be modeled through virtual actors (Orleans, [n.d.]; Wang et al., 2019);
(B) Extending the frontiers of a database system by having database components living in the application-tier and communicating the application’s needs to the database system. With current database APIs, applications have no way to push down complex dataflows to the database system in a meaningful way, because interfaces abstract away the complex interaction and data exchanges happening outside the database. Although an initial proposition in this line has been presented in (Laigner et al., 2021a), no formalism and implementation has been provided so far.
F9. Appropriate abstractions should be formalized to allow the database system to gain knowledge of the data management logic carried out by the application and the complex interplay of microservices. These abstractions could help in achieving a proper division of tasks between the application and the database system, where data management tasks are pushed to the database system, thus alleviating the burden on developers.
8. Conclusion
In this paper, we observe that the lack of a holistic data management solution for microservices leads practitioners to resort to ad-hoc designs. These are characterized by weaving together heterogeneous services and resorting to application-level data management mechanisms, which lead to problems that cannot be resolved through code refactorings (Fowler et al., 1999). To tackle the data management requirements of microservices by design, we present a set of candidate features for database systems so that they can play a central role in this new paradigm.
Acknowledgements.
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie agreement No 801199 and Independent Research Fund Denmark grant No 9041-00368B. Dedicated to the memory of Maria José Teixeira .References
- (1)
- why ([n.d.]) Microsoft [n.d.]. Why Orleans Streams? Microsoft. https://dotnet.github.io/orleans/Documentation/streaming/streams_why.html
- Apps ([n.d.]) .NET Application Architecture Reference Apps. [n.d.]. eShopOnContainers. https://github.com/dotnet-architecture/eShopOnContainers
- Azevedo et al. (2019) Leonardo Guerreiro Azevedo, Rodrigo da Silva Ferreira, Viviane Torres da Silva, Maximillien de Bayser, Elton F. de S. Soares, and Raphael Melo Thiago. 2019. Geological Data Access on a Polyglot Database Using a Service Architecture. In Proceedings of the XIII Brazilian Symposium on Software Components, Architectures, and Reuse (Salvador, Brazil) (SBCARS ’19). ACM, New York, NY, USA, 103–112. https://doi.org/10.1145/3357141.3357603
- Azure ([n.d.]) Microsoft Azure. [n.d.]. A technical overview of Azure Cosmos DB. https://azure.microsoft.com/en-us/blog/a-technical-overview-of-azure-cosmos-db (Accessed on 2021-07-01).
- Bailis et al. (2015) Peter Bailis, Alan Fekete, Michael J. Franklin, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. 2015. Feral Concurrency Control: An Empirical Investigation of Modern Application Integrity. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data (Melbourne, Victoria, Australia) (SIGMOD ’15). Association for Computing Machinery, New York, NY, USA, 1327–1342. https://doi.org/10.1145/2723372.2737784
- Bailis et al. (2016) Peter Bailis, Alan Fekete, Ali Ghodsi, Joseph M Hellerstein, and Ion Stoica. 2016. Scalable atomic visibility with RAMP transactions. ACM Transactions on Database Systems (TODS) 41, 3 (2016), 1–45.
- Bailis and Ghodsi (2013) Peter Bailis and Ali Ghodsi. 2013. Eventual Consistency Today: Limitations, Extensions, and Beyond. Queue 11, 3 (March 2013), 20–32. https://doi.org/10.1145/2460276.2462076
- Bernstein et al. (2014) Philip A Bernstein, Sergey Bykov, Alan Geller, Gabriel Kliot, and Jorgen Thelin. 2014. Orleans: Distributed Virtual Actors for Programmability and Scalability. (2014).
- Calçado (2015) Phil Calçado. 2015. The Back-end for Front-end Pattern (BFF). Retrieved September 10, 2020 from https://philcalcado.com/2015/09/18/the_back_end_for_front_end_pattern_bff.html
- Carrasco et al. (2018) Andrés Carrasco, Brent van Bladel, and Serge Demeyer. 2018. Migrating towards Microservices: Migration and Architecture Smells. In Proceedings of the 2nd International Workshop on Refactoring (Montpellier, France) (IWoR 2018). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3242163.3242164
- Carvalho et al. (2019) Luiz Carvalho, Alessandro Garcia, Wesley KG Assunção, Rafael de Mello, and Maria Julia de Lima. 2019. Analysis of the criteria adopted in industry to extract microservices. In 2019 IEEE/ACM Joint 7th International Workshop on Conducting Empirical Studies in Industry (CESI) and 6th International Workshop on Software Engineering Research and Industrial Practice (SER&IP). IEEE, 22–29.
- Chen et al. (2014) Tse-Hsun Chen, Weiyi Shang, Zhen Ming Jiang, Ahmed E Hassan, Mohamed Nasser, and Parminder Flora. 2014. Detecting performance anti-patterns for applications developed using object-relational mapping. In Proceedings of the 36th International Conference on Software Engineering. 1001–1012.
- Ciavotta et al. (2017) Michele Ciavotta, Marino Alge, Silvia Menato, Diego Rovere, and Paolo Pedrazzoli. 2017. A microservice-based middleware for the digital factory. Procedia Manufacturing 11 (2017), 931–938.
- Corbett et al. (2012) James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, JJ Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, David Nagle, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Dale Woodford, Yasushi Saito, Christopher Taylor, Michal Szymaniak, and Ruth Wang. 2012. Spanner: Google’s Globally-Distributed Database. In OSDI.
- Cruz et al. (2019) P. Cruz, H. Astudillo, R. Hilliard, and M. Collado. 2019. Assessing Migration of a 20-Year-Old System to a Micro-Service Platform Using ATAM. In 2019 IEEE International Conference on Software Architecture Companion (ICSA-C). 174–181.
- DeCandia et al. (2007) Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, and Werner Vogels. 2007. Dynamo: Amazon’s highly available key-value store. ACM SIGOPS operating systems review 41, 6 (2007), 205–220.
- Documentation ([n.d.]) PostgreSQL 12 Documentation. [n.d.]. NOTIFY. Retrieved September 10, 2020 from https://www.postgresql.org/docs/current/sql-notify.html
- EdwinVW ([n.d.]) EdwinVW. [n.d.]. pitstop. https://github.com/EdwinVW/pitstop
- Fowler (2011) Martin Fowler. 2011. PolyglotPersistence. Retrieved September 20, 2020 from https://martinfowler.com/bliki/PolyglotPersistence.html
- Fowler (2014) Martin Fowler. 2014. Microservices a definition of this new architectural term. Retrieved July 7, 2020 from https://martinfowler.com/articles/microservices.html
- Fowler et al. (1999) M. Fowler, K. Beck, J. Brant, W. Opdyke, and D. Roberts. 1999. Refactoring: Improving the Design of Existing Code. Addison-Wesley Longman Publishing Co., Inc., USA.
- Francesco et al. (2017) Paolo Francesco, Ivano Malavolta, and Patricia Lago. 2017. Research on Architecting Microservices: Trends, Focus, and Potential for Industrial Adoption. In International Conference on Software Architecture. 21–30. https://doi.org/10.1109/ICSA.2017.24
- Gamma (1995) Erich Gamma. 1995. Design patterns: elements of reusable object-oriented software. Pearson Education India.
- Gan et al. (2019) Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayan Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, et al. 2019. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. 3–18.
- Garcia-Molina and Salem (1987) Hector Garcia-Molina and Kenneth Salem. 1987. Sagas. In SIGMOD (SIGMOD ’87), Vol. 16. Issue 3.
- Ghofrani and Lübke (2018) Javad Ghofrani and Daniel Lübke. 2018. Challenges of Microservices Architecture: A Survey on the State of the Practice.. In ZEUS. 1–8.
- Gouigoux and Tamzalit (2017) Jean-Philippe Gouigoux and Dalila Tamzalit. 2017. From Monolith to Microservices: Lessons Learned on an Industrial Migration to a Web Oriented Architecture. 2017 IEEE International Conference on Software Architecture Workshops (ICSAW) (2017), 62–65.
- Hasselbring and Steinacker (2017) Wilhelm Hasselbring and Guido Steinacker. 2017. Microservice Architectures for Scalability, Agility and Reliability in E-Commerce. In IEEE ICSA Workshops. 243–246.
- Helland (2017) Pat Helland. 2017. Life beyond distributed transactions. Commun. ACM 60, 2 (2017), 46–54. https://doi.org/10.1145/3009826
- Helland (2020) Pat Helland. 2020. Data on the Outside vs. Data on the Inside, Vol. 18. ACM Queue. Issue 3.
- Hohpe (2007) Gregor Hohpe. 2007. Let’s have a conversation. IEEE internet computing 11, 3 (2007), 78–81.
- Kafka ([n.d.]) Apache Kafka. [n.d.]. KAFKA STREAMS. Retrieved September 16, 2020 from https://kafka.apache.org/documentation/streams/
- Katsifodimos and Fragkoulis (2019) Asterios Katsifodimos and Marios Fragkoulis. 2019. Operational Stream Processing: Towards Scalable and Consistent Event-Driven Applications.. In EDBT. 682–685.
- kbastani ([n.d.]) kbastani. [n.d.]. event-stream-processing-microservices. https://github.com/kbastani/event-stream-processing-microservices
- Kleppmann et al. (2019) Martin Kleppmann, Alastair R. Beresford, and Boerge Svingen. 2019. Online Event Processing. In Distributed Development, Vol. 17. ACM Queue. Issue 1.
- Knoche and Hasselbring (2019) Holger Knoche and Wilhelm Hasselbring. 2019. Drivers and Barriers for Microservice Adoption - A Survey among Professionals in Germany. 14 (01 2019), 1–35. https://doi.org/10.18417/emisa.14.1
- Krylovskiy et al. (2015) A. Krylovskiy, M. Jahn, and E. Patti. 2015. Designing a Smart City Internet of Things Platform with Microservice Architecture. In 2015 3rd International Conference on Future Internet of Things and Cloud. 25–30.
- Laigner et al. (2020) Rodrigo Laigner, Marcos Kalinowski, Pedro Diniz, Leonardo Barros, Carlos Cassino, Melissa Lemos, Darlan Arruda, Sergio Lifschitz, and Yongluan Zhou. 2020. From a Monolithic Big Data System to a Microservices Event-Driven Architecture. In 46th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2020, Portorož, Slovenia, Aug 26-28. 1–8.
- Laigner et al. (2019) Rodrigo Laigner, Sérgio Lifschitz, Marcos Kalinowski, Marcus Poggi, and Marcos Antonio Vaz Salles. 2019. Towards a Technique for Extracting Relational Actors from Monolithic Applications. In Anais do XXXIV Simpósio Brasileiro de Banco de Dados (Fortaleza). SBC, Porto Alegre, RS, Brasil, 133–144. https://doi.org/10.5753/sbbd.2019.8814
- Laigner et al. (2021a) Rodrigo Laigner, Yongluan Zhou, and Marcos Antonio Vaz Salles. 2021a. A Distributed Database System for Event-based Microservices. In The 15th ACM International Conference on Distributed and Event-based Systems (DEBS ’21), June 28-July 2, 2021, Virtual Event, Italy (DEBS ’21). Association for Computing Machinery.
- Laigner et al. (2021b) Rodrigo Laigner, Yongluan Zhou, Marcos Antonio Vaz Salles, Yijian Liu, and Marcos Kalinowski. 2021b. Data Management in Microservices: State of the Practice, Challenges, and Research Directions (Extended Version). WorkingPaper. https://di.ku.dk/english/staff/?pure=en/publications/data-management-in-microservices-state-of-the-practice-challenges-and-research-directions(7811cbd3-49de-4663-a0bf-a2acd4b3c8b0).html
- Lotz et al. (2019) J. Lotz, A. Vogelsang, O. Benderius, and C. Berger. 2019. Microservice Architectures for Advanced Driver Assistance Systems: A Case-Study. In 2019 IEEE International Conference on Software Architecture Companion (ICSA-C). 45–52.
- Luz et al. (2018) Welder Luz, Everton Agilar, Marcos César de Oliveira, Carlos Eduardo R. de Melo, Gustavo Pinto, and Rodrigo Bonifácio. 2018. An Experience Report on the Adoption of Microservices in Three Brazilian Government Institutions. In Proceedings of the XXXII Brazilian Symposium on Software Engineering (Sao Carlos, Brazil) (SBES ’18). ACM, New York, NY, USA, 32–41. https://doi.org/10.1145/3266237.3266262
- M. Del Esposte et al. (2017) Arthur M. Del Esposte, Fabio Kon, Fabio M. Costa, and Nelson Lago. 2017. InterSCity: A Scalable Microservice-Based Open Source Platform for Smart Cities. In Proceedings of the 6th International Conference on Smart Cities and Green ICT Systems (Porto, Portugal) (SMARTGREENS 2017). SCITEPRESS - Science and Technology Publications, Lda, Setubal, PRT, 35–46. https://doi.org/10.5220/0006306200350046
- Mazzara et al. (2018) M. Mazzara, N. Dragoni, A. Bucchiarone, A. Giaretta, S. T. Larsen, and S. Dustdar. 2018. Microservices: Migration of a Mission Critical System. IEEE Transactions on Services Computing (2018), 1–1.
- Messina et al. (2016) Antonio Messina, Riccardo Rizzo, Pietro Storniolo, Mario Tripiciano, and Alfonso Urso. 2016. The Database-is-the-Service Pattern for Microservice Architectures, Vol. 9832. 223–233. https://doi.org/10.1007/978-3-319-43949-5_18
- Microservice-API-Patterns ([n.d.]) Microservice-API-Patterns. [n.d.]. LakesideMutual. https://github.com/Microservice-API-Patterns/LakesideMutual
- microservices patterns ([n.d.]) microservices patterns. [n.d.]. ftgo-application. https://github.com/microservices-patterns/ftgo-application
- Neri et al. (2019) Davide Neri, Jacopo Soldani, Olaf Zimmermann, and Antonio Brogi. 2019. Design principles, architectural smells and refactorings for microservices: a multivocal review. SICS Software-Intensive Cyber-Physical Systems 35, 1-2 (Sep 2019), 3–15. https://doi.org/10.1007/s00450-019-00407-8
- Newman (2015a) Sam Newman. 2015a. Building Microservices (1st ed.). O’Reilly Media, Inc.
- Newman (2015b) Sam Newman. 2015b. Pattern: Backends For Frontends. Retrieved July 6, 2021 from https://samnewman.io/patterns/architectural/bff/
- O’Neil (2018) Patrick O’Neil. 2018. Escrow Transactions. In Encyclopedia of Database Systems, Second Edition, Ling Liu and M. Tamer Özsu (Eds.). Springer. https://doi.org/10.1007/978-1-4614-8265-9_150
- Orleans ([n.d.]) Orleans. [n.d.]. Best Practices. https://dotnet.github.io/orleans/docs/resources/best_practices.html (Accessed on 2021-05-16).
- Pavlo (2017) Andrew Pavlo. 2017. What are we doing with our lives? Nobody cares about our concurrency control research. In Proceedings of the 2017 ACM International Conference on Management of Data. 3–3.
- Pigazzini et al. (2020) Ilaria Pigazzini, Francesca Arcelli Fontana, Valentina Lenarduzzi, and Davide Taibi. 2020. Towards Microservice Smells Detection. In Proceedings of the 3rd International Conference on Technical Debt (Seoul, Republic of Korea) (TechDebt ’20). Association for Computing Machinery, New York, NY, USA, 92–97. https://doi.org/10.1145/3387906.3388625
- Ports et al. (2010) Dan RK Ports, Austin T Clements, Irene Zhang, Samuel Madden, and Barbara Liskov. 2010. Transactional Consistency and Automatic Management in an Application Data Cache. In OSDI, Vol. 10. 1–15.
- Pritchett (2008) Dan Pritchett. 2008. Base: An Acid Alternative. In File Systems and Storage, Vol. 6. ACM Queue. Issue 3.
- Richardson ([n.d.]) Chris Richardson. [n.d.]. API gateway pattern. Retrieved September, 15 2020 from https://microservices.io/patterns/apigateway.html
- Richter et al. (2017) D. Richter, M. Konrad, K. Utecht, and A. Polze. 2017. Highly-Available Applications on Unreliable Infrastructure: Microservice Architectures in Practice. In 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C). 130–137.
- sczyh30 ([n.d.]) sczyh30. [n.d.]. vertx-blueprint-microservice. https://github.com/sczyh30/vertx-blueprint-microservice
- Sentilo ([n.d.]) Sentilo. [n.d.]. Sentilo Platform. https://github.com/sentilo/sentilo
- Shah (2017) Vivek Shah. 2017. Exploration of a Vision for Actor Database Systems. Ph.D. Dissertation. University of Copenhagen, Denmark.
- Soldani et al. (2018) Jacopo Soldani, Damian Andrew Tamburri, and Willem-Jan Van Den Heuvel. 2018. The pains and gains of microservices: A Systematic grey literature review. Journal of Systems and Software 146 (2018), 215–232.
- spring petclinic ([n.d.]) spring petclinic. [n.d.]. spring-petclinic-microservices. https://github.com/spring-petclinic/spring-petclinic-microservices
- Stax ([n.d.]) Data Stax. [n.d.]. Astra DB Multi-cloud DBaaS built on Apache Cassandra. https://www.datastax.com/products/datastax-astra (Accessed on 2021-07-01).
- Tai (2020) Tzu-Li (Gordon) Tai. 2020. Stateful Functions Internals: Behind the scenes of Stateful Serverless. https://flink.apache.org/news/2020/10/13/stateful-serverless-internals.html (Accessed on 2021-03-05).
- Taibi and Lenarduzzi (2018) D. Taibi and V. Lenarduzzi. 2018. On the Definition of Microservice Bad Smells. IEEE Software 35, 3 (2018), 56–62.
- Taibi et al. (2019) Davide Taibi, Valentina Lenarduzzi, and Claus Pahl. 2019. Microservices Anti-Patterns: A Taxonomy.
- Tanenbaum and van Steen (2016) Andrew S. Tanenbaum and Maarten van Steen. 2016. Distributed Systems: Principles and Paradigms (2nd ed.). CreateSpace Independent Publishing Platform.
- Tighilt et al. (2020) Rafik Tighilt, Manel Abdellatif, Naouel Moha, Hafedh Mili, Ghizlane El Boussaidi, Jean Privat, and Yann-Gaël Guéhéneuc. 2020. On the Study of Microservices Antipatterns: A Catalog Proposal. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3424771.3424812
- Vassiliadis (2009) Panos Vassiliadis. 2009. A survey of extract–transform–load technology. International Journal of Data Warehousing and Mining (IJDWM) 5, 3 (2009), 1–27.
- Veloso (2019) Miguel Veloso. 2019. BFF implementation. Retrieved July 6, 2021 from https://github.com/dotnet-architecture/eShopOnContainers/wiki/BFF-implementation
- Verbitski et al. (2017) Alexandre Verbitski, Anurag Gupta, Debanjan Saha, Murali Brahmadesam, Kamal Gupta, Raman Mittal, Sailesh Krishnamurthy, Sandor Maurice, Tengiz Kharatishvili, and Xiaofeng Bao. 2017. Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases. In Proceedings of the 2017 ACM International Conference on Management of Data (Chicago, Illinois, USA) (SIGMOD ’17). Association for Computing Machinery, New York, NY, USA, 1041–1052. https://doi.org/10.1145/3035918.3056101
- Viennot et al. (2015) Nicolas Viennot, Mathias Lécuyer, Jonathan Bell, Roxana Geambasu, and Jason Nieh. 2015. Synapse: A Microservices Architecture for Heterogeneous-Database Web Applications. In Proceedings of the Tenth European Conference on Computer Systems (Bordeaux, France) (EuroSys ’15). Association for Computing Machinery, New York, NY, USA, Article 21, 16 pages. https://doi.org/10.1145/2741948.2741975
- Viggiato et al. (2018) Markos Viggiato, Ricardo Terra, Henrique Rocha, Marco Tulio Valente, and Eduardo Figueiredo. 2018. Microservices in Practice: A Survey Study. arXiv e-prints, Article arXiv:1808.04836 (Aug. 2018), arXiv:1808.04836 pages. arXiv:1808.04836 [cs.SE]
- Wang et al. (2019) Yiwen Wang, Julio Cesar Dos Reis, Kasper Myrtue Borggren, Marcos Antonio Vaz Salles, Claudia Bauzer Medeiros, and Yongluan Zhou. 2019. Modeling and Building IoT Data Platforms with Actor-Oriented Databases.. In EDBT.
- Weaveworks ([n.d.]) Weaveworks. [n.d.]. Sock Shop : A Microservice Demo Application. https://github.com/microservices-demo/microservices-demo
- Winslett and Braganholo (2019) Marianne Winslett and Vanessa Braganholo. 2019. Peter Bailis Speaks Out on Building Tools Users Want to Use. SIGMOD Rec. 47, 3 (Feb. 2019), 29–31. https://doi.org/10.1145/3316416.3316423
- Yang et al. (2018) Junwen Yang, Cong Yan, Pranav Subramaniam, Shan Lu, and Alvin Cheung. 2018. PowerStation: Automatically Detecting and Fixing Inefficiencies of Database-Backed Web Applications in IDE. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Lake Buena Vista, FL, USA) (ESEC/FSE 2018). Association for Computing Machinery, New York, NY, USA, 884–887. https://doi.org/10.1145/3236024.3264589
- Zhang et al. (2019) H. Zhang, S. Li, Z. Jia, C. Zhong, and C. Zhang. 2019. Microservice Architecture in Reality: An Industrial Inquiry. In 2019 IEEE International Conference on Software Architecture (ICSA). 51–60.
- Zimmermann (2017) Olaf Zimmermann. 2017. Microservices Tenets. Comput. Sci. 32, 3-4 (July 2017), 301–310. https://doi.org/10.1007/s00450-016-0337-0