Cloud Cost Optimization: A Comprehensive Review of Strategies and Case Studies

One of the fundamental techniques for compute cost optimization is right-sizing. It involves aligning the allocated compute resources with the actual requirements of the workload. By accurately assessing the workload’s CPU, memory, and storage needs, enterprises can avoid overprovisioning and reduce unnecessary costs. For example, GCP’s n2-standard-8 instance costs $0.388472 per hour, while a n2-standard-16 instance costs $0.776944 per hour in GCP. If a workload only needs 8 vCPUs, then right-sizing to a n2-standard-8 instance can save $0.388472 per hour compared to an over-provisioned 16 vCPU instance. This savings can add up over time, especially for workloads that run for long periods of time. Cost optimization via right-sizing of the compute resources can be achieved by monitoring resource utilization, analyzing performance metrics, and leveraging cloud provider tools or third-party solutions.

Autoscaling is a dynamic resource management technique that adjusts the number of compute resources based on workload demand. By automatically scaling resources up or down, enterprises can match compute capacity with the fluctuating needs of their applications [21]. Autoscaling ensures efficient resource utilization, avoids overprovisioning during low-demand periods, and improves responsiveness during peak loads. It enables enterprises to pay for compute resources only when needed, leading to significant cost savings.

Spot instances offer a cost-effective approach for non-critical workloads. Cloud providers offer spare compute capacity at significantly discounted prices, allowing enterprises to bid for these instances [13]. Spot instances can provide substantial savings compared to on-demand or reserved instances. However, it’s important to note that spot instances can be interrupted if the spot price exceeds the bid price. Thus, they are suitable for fault-tolerant, flexible workloads that can withstand interruptions.

However, spot instances are not guaranteed to be available 100% of the time. If your instance is interrupted, you will be given a few minutes to terminate your applications. To reduce the risk of your instance being interrupted, spot fleets can be used. A spot fleet is a group of spot instances that are launched together.

Reserved instances provide a discounted pricing model for enterprises that commit to using specific compute resources for a specified duration [8] [9] [10]. By reserving instances in advance, organizations can secure a lower hourly rate compared to on-demand instances. Reserved instances are suitable for workloads with predictable and steady demand. Enterprises can choose from different reservation options, such as standard, convertible, or scheduled instances, based on their flexibility requirements.

Serverless computing eliminates the need for provisioning and managing servers. With serverless architectures, enterprises pay only for the actual compute time and resources used by their applications [22]. This model offers granular cost control, as organizations are billed based on the number of function executions and resource consumption. By leveraging serverless computing, enterprises can optimize costs for event-driven workloads, where compute resources are only utilized when triggered by specific events. However, serverless computing may not be the cost efficient alternative to the equivalent compute offering if the application is long running [23].

Containerization technologies, such as Docker and Kubernetes, provide efficient resource utilization by packaging applications and their dependencies into lightweight containers. Containerization enables enterprises to deploy applications consistently across different environments and scale them based on demand. By optimizing resource allocation and improving density, containerization can lead to cost savings by reducing the number of required compute instances. The table 6 showcases the comparison of the cost incurred by traditional compute and containerized offerings for the specification of 2 vCPUs, 8 GB memory on a GCE instance type e2-standard-2, Cloud Functions function with similar specifications, and Cloud Run on GCP. All the pricing mentioned in the table 6 is on-demand pricing in the region us-central1.

However, since Cloud Run is a serverless containerized offering from GCP, although it is the cheapest option to run containerized workloads, it can become quite expensive if the application is long-running and does not depend on event-driven architecture.

One effective cost optimization strategy in the cloud is to leverage ARM Graviton EC2 instances instead of Intel-based instances. ARM-based processors, such as the Graviton processors developed by AWS, offer a compelling alternative in terms of cost efficiency. These processors are designed to deliver high performance while consuming less power, leading to lower operational costs [24]. By utilizing ARM Graviton instances, enterprises can potentially achieve significant cost savings, especially for workloads that are compatible with ARM architecture. It’s important to evaluate the workload requirements and compatibility before considering the use of Graviton instances. However, for compatible workloads, migrating to ARM-based instances can provide a cost-effective solution without compromising performance. Additionally, AWS’s ongoing efforts to optimize ARM-based instances and expand the range of supported workloads further enhance the cost-saving potential for enterprises.

Another effective strategy to reduce costs in cloud computing environments is through the implementation of automated discovery and deprecation of idle instances. Idle instances refer to virtual machines or cloud resources that are not actively utilized, yet still incur costs. By implementing a systematic approach to identify and list idle instances, organizations can gain visibility into their cloud usage patterns and identify opportunities for cost optimization.

The process may commence by monitoring resource usage and analyzing the utilization patterns of virtual machines. Through the use of monitoring tools and cloud management platforms, administrators can identify instances that consistently exhibit low or no utilization over a specified period. Once identified, these idle instances can be listed, allowing administrators to evaluate their necessity and potential for termination. By taking a proactive approach to manage idle instances, organizations can achieve significant cost savings. When an idle instance is listed, administrators have the opportunity to review its purpose and determine whether it is essential for ongoing operations. If the instance is found to be unnecessary or redundant, it can be safely stopped or terminated, eliminating the associated costs.

Implementing idle instance listing and stopping practices requires careful consideration of factors such as business requirements, service level agreements, and potential impacts on system performance. However, with proper planning and monitoring, organizations can achieve substantial cost savings while maintaining the required level of service availability.

Data deduplication and compression techniques play a crucial role in optimizing costs in cloud storage environments. By eliminating redundant or unused data, organizations can significantly reduce storage requirements and associated expenses. Deduplication involves identifying and removing duplicate data segments, while compression reduces the size of data by encoding it using efficient algorithms. Together, these techniques offer substantial cost savings by minimizing storage capacity needs and mitigating the impact of data growth [25].

One of the key benefits of data deduplication is the elimination of redundant data copies. In many organizations, multiple users or applications may store identical or similar files, resulting in unnecessary data replication. By identifying and storing only unique data segments, deduplication reduces the overall storage footprint, leading to cost savings [26]. Additionally, data compression techniques further enhance storage efficiency by reducing the size of individual files or data blocks. By employing compression algorithms, organizations can achieve significant data size reduction without compromising data integrity or accessibility.

The cost savings achieved through data deduplication and compression extend beyond storage capacity reduction. By minimizing the storage footprint, organizations can lower data transfer costs when moving data between cloud storage tiers or across different regions. The reduced data size also contributes to faster data transfer speeds, optimizing overall system performance. Moreover, data deduplication and compression techniques can enhance backup and disaster recovery processes, as smaller data volumes facilitate faster backup and recovery operations, reducing downtime and associated costs.

Implementing data deduplication and compression techniques requires careful consideration of factors such as data access patterns, application requirements, and computational overhead. It is crucial to select appropriate deduplication and compression algorithms that strike a balance between storage savings and processing overhead. Additionally, organizations must evaluate the impact on data access times and consider trade-offs between storage cost savings and the computational resources required for data deduplication and compression operations.

As data ages or becomes less frequently accessed, it may not require the same level of performance or accessibility as newer or more frequently used data. With data lifecycle management policies, organizations can define rules and criteria for data migration. This enables the automatic movement of data to lower-cost storage tiers, such as archival or cold storage, without sacrificing data availability or integrity [27].

By migrating data to more cost-effective storage options as it ages or becomes less frequently accessed, organizations can optimize their storage costs. This approach allows them to take advantage of different storage tiers that offer varying levels of performance, durability, and cost. It ensures that data is stored in the most suitable storage option while minimizing unnecessary expenses associated with storing all data in high-performance storage throughout its lifecycle.

Implementing data lifecycle management policies can be achieved through a combination of automated processes, data classification, and intelligent data management solutions. These policies can be tailored to specific business requirements, compliance regulations, and data access patterns [28] [29]. By adopting such policies, organizations can achieve significant cost savings by aligning storage costs with the value and usage patterns of their data.

Another important tactic for cloud storage cost optimization is enacting a data archival and retention policies. Enterprises are often required to retain data for compliance which can result in expensive cloud storage costs, especially if that data is kept in high-performance storage tiers. Implementing data archiving and retention standards becomes crucial to overcoming this problem.

To achieve cost efficiency, organizations can leverage specific storage options tailored for long-term data retention, such as archive storage tiers. These tiers offer significantly lower storage costs compared to standard storage tiers, while still ensuring sufficient durability and availability. Based on the data governance policies, the data can be migrated from high-performance tier to archive tier. Additionally, setting a small retention value on the data would result in the smaller data storage footprint for an enterprise resulting in significant cost savings in data storage.

By harnessing intelligent data management techniques, organizations can identify and apply suitable retention periods to different data sets, ensuring compliance with legal requirements while optimizing storage costs.

In order to find chances for cost reductions, it is essential to examine the network traffic patterns inside a cloud environment. Enterprises may optimize their network infrastructure and cut expenses by looking at the amount and kind of data flows using network traffic analysis. Enterprises can use monitoring tools and analytics platforms offered by cloud providers or third-party solutions to do network traffic analysis. These tools give users insight into the patterns of network traffic, including data transfer rates, peak usage periods, and the types of data being transferred [30].

Enterprises can find potential areas for optimization with the use of this information. For instance, they can discover data-intensive programs or services that cause a lot of network traffic and look for ways to improve how data is transferred between them. This may entail putting data compression techniques into practice, leveraging content delivery networks (CDNs) or edge caching to shorten the distances between data transfers, or using data deduplication techniques to stop repeated transfers.

In order to maximize network efficiency, network traffic analysis can also help find opportunities for traffic rerouting or load balancing. Enterprises can lower bandwidth consumption and possibly lower data transfer costs by intelligently directing traffic through efficient channels or dispersing it across numerous network resources.

Content Delivery Networks (CDNs) play a key role in improving network performance and lowering costs. When using CDNs, content can be delivered from the edge location that is closest to the end users by utilizing edge servers that are dispersed across different locations. The usage of CDNs, edge caching, and traffic control strategies to save costs and boost overall network effectiveness are examined in this section. By caching and distributing content closer to end users, CDNs aim to lower latency and boost speed. CDNs reduce the distance that data must travel across the network by strategically distributing content in geographically dispersed edge servers. By serving content from edge locations rather than the origin server, this not only improves the user experience but also lowers network egress costs.

Frequently accessed content is stored at edge server locations as part of the edge caching approach. The content is delivered directly from the nearest edge cache when a user wants it, avoiding the need for data to travel across the entire network. Enterprises may drastically lower network egress costs and enhance end-user response times by utilizing edge caching. Additionally, by offloading the traffic from the origin server to the edge locations, enterprises can lessen the stress on the origin server, increase network bandwidth, and lower egress costs by dumping data closer to the end users. As a result, less expensive origin server resources and data transfer are not required for serving static content.

Finally, optimizing network utilization and cutting costs need effective traffic engineering. To achieve the best distribution of network traffic, a network engineer may use smart load balancing and intelligent routing algorithms. Enterprises can reduce network egress and ingress expenses by routing traffic through the most economical pathways in each of cloud regions. In order to increase performance and save costs, traffic engineering techniques can also be used to prioritize important traffic, reduce bottlenecks, and optimize bandwidth use [31].

Utilizing data compression techniques to limit pointless data transfer is one efficient strategy. Data compression can drastically reduce the quantity of data carried over the network, resulting in lower network bandwidth usage and cheaper expenses. The type of data being compressed and the chosen compression ratio will determine the optimal approach to compress the data before transmitting it over the network. For example, the best compression ratio will typically come from lossless compression, but it will also cause some delay. On the other hand, lossy compression will result in some quality loss but can offer a better compression ratio with less latency.

Twitter’s Parquet encoding standard is one such example of using compression techniques for data that is being backed up to the cloud. The capability of Parquet compression to compress data at the column level is one of its main benefits [32]. By utilizing similarities and redundancies among columns, this columnar storage strategy enables effective compression, producing higher compression ratios than row-based storage formats. Twitter has been able to improve storage expenses in their cloud environment by lowering the size of the data saved in Parquet files. The adoption of Parquet compression by Twitter emphasizes the significance of choosing compression methods that are suited to the data and use cases.

In large cloud environments, optimizing network configurations is a crucial part of reducing costs. Enterprises can maximize network performance, decrease data transfer, and cut associated expenses by carefully tuning the network settings for their compute infrastructure. One common approach is using load balancing and efficient routing algorithms to evenly divide network traffic across the available resources and prevent pointless data transfer. In addition, streamlining network protocols and configurations, like TCP/IP settings, can increase network effectiveness and cut down on bandwidth usage. Prioritizing vital network traffic and efficiently allocating bandwidth resources can be achieved by using traffic shaping and quality of service (QoS) regulations. Enterprises can also use network analytics and monitoring technologies to get insights into network usage trends and spot optimization opportunities. Therefore, it is possible to significantly reduce network expenses while retaining optimal performance and reliability by routinely analyzing and fine-tuning network configurations in accordance with shifting workload demands and cost optimization objectives.

Implementing efficient log filtering and sampling methods is the first step in managing logging costs effectively. Enterprises can reduce the amount of logs stored and transmitted by defining precise filters, concentrating only on pertinent data. For example, log severity, particular components or services, or user-defined criteria can all be used as filters. Similar to this, log sampling enables the collection of a representative subset of logs as opposed to storing each individual log entry. Enterprises can strike a balance between cost reduction and keeping a sufficient level of system visibility by carefully choosing the sampling rate. For example, let’s say that a company stores 100 GB of log data per month. If the company does not use log filtering, then all of this log data will be stored. However, if the company uses log filtering to only store logins, errors, and other important events, then the amount of log data that is stored can be reduced to 10 GB per month. This would save the company 90% on storage costs.

Utilizing data compression methods and picking the best log storage option can have a big impact on cost reduction. Different storage tiers, such as standard storage, cold storage, or archival storage, are available from cloud providers at various price points. Enterprises can choose which logs to store in the most economical storage tier by analyzing the frequency and urgency of log access. Additionally, as mentioned in the section of data compression, implementing compression techniques for the logging data can reduce log size and minimize the storage costs without compromising the logs. For example, Twitter’s use of LZO compression to compress Scribe event log data [33] and Facebook’s use of ZStandard library to compress live logging data [34] are the two prominent examples of how using compression techniques in log storage can optimize the cost for a company’s infrastructure [35].

Another major expense incurred by enterprises is retaining logs for longer periods of time. When storing logs for an extended period, especially less important or logs related to regulatory compliance, extra costs may arise. By leveraging a data lifecycle management policy, the process of archiving or deleting logs in accordance with predefined policies can be automated thereby reducing the human intervention. Therefore, logs can be retained for the exact amount of time that is required by setting retention periods based on compliance requirements, business needs, and industry best practices. Moreover, by combining the log retention policy with the techniques like data compression and choosing an appropriate storage tiers for compliance-related logs, enterprises can incur a substantial amount of cost savings. Additionally, based on the contracts between cloud provider and enterprises, enterprises should elect for the cheapest storage solution to store the logging data.

To effectively optimize costs, log usage and costs must be proactively tracked. To track log volume, storage usage, and associated costs, organizations should set up alerts and notifications. Organizations can spot any unexpected spikes or patterns in log volume and take prompt action to minimize costs by setting thresholds and proactive monitoring mechanisms. Cost effectiveness is maintained over time by periodically reviewing log usage patterns and modifying monitoring strategies in response to changing needs.

Cloud providers offer recommendations for utilizing reserved instances (RIs) and savings plans to optimize costs. These recommendations are usually focused on compute instances like Amazon AWS EC2. For example, AWS provides recommendations for utilizing Reserved Instances (RIs) and Savings Plans based on usage patterns and potential cost savings. Whereas, GCP provides recommendations for using the Commited Use Discounts (CUDs) to optimize costs for the long-term contracts and Azure offers Reserved VM Instances (RIs) and reservation recommendations for the same purposes, respectively. In addition to the long-term commitments, these compute instance recommendations also involve offering the list of idle instances, underutilized, or over-provisioned resources. For example, GCP offers a machine learning-based Recommenders that observe the enterprises’ virtual machine instances for 8 days and then offer the recommendations to right-size the instances to save costs [36].

Cloud service providers provide recommendations for maximizing storage usage and costs. These suggestions look at storage usage patterns, point out ineffective or unused storage resources, and make suggestions for the best storage configurations. Enterprises can reduce wasteful storage expenses, improve data placement, and take advantage of cost-efficient storage tiers by implementing these recommendations. All major cloud providers offer cloud stoarge recommendations to help save storage costs. For example, AWS offers S3 Storage Lens to analyze and optimize storage usage, along with Amazon S3 Intelligent-Tiering for automated data tiering recommendations [28]. On the other hand, GCP provides a feature called GCS Autoclass. Based on each object’s access pattern, the Autoclass feature automatically moves objects in the bucket to the proper storage classes [27]. This feature moves data that is frequently accessed to Standard storage to improve future accesses and moves data that is not accessed to colder storage classes facilitating automated data lifecycle management to save costs.

Cloud service providers also offer recommendations for improving database configurations and usage. These suggestions examine database usage patterns, query efficiency, and performance, and finally they offer optimization tactics. Organizations can optimize database costs, reduce wasteful resource use, and improve database performance by putting these recommendations into practice. Such examples of database recommendations can be found in AWS’ Amazon RDS Performance Insights and Database Query Monitoring for identifying and optimizing database performance issues, Azure’s SQL Database Advisor [37], and finally GCP’s CloudSQL Insights that also offers recommendations on idle disks as well as over-provisioned CloudSQL instances [38].

The cloud network recommendations provided by the cloud providers mainly focus on the idle resources that still have IP addresses attached to them. For example, GCP’s Idle Resource Recommender would identify resources like persistent disks (PDs), IP addresses, and custom disk images that aren’t used. Since the IP addresses are pay-per-use resources in the cloud, deleting an instance or releasing the IP address from that instance would save 100% of the cost associated with the IP addresses. On the other hand, Amazon VPC IP Address Manager (IPAM) offered by AWS aids in managing an organization’s IP inventory. Therefore, with the help from IPAM, an enterprise can identity the idle IP addresses and release those IP addresses from the compute resources to save costs [39].

Organizations can break down monolithic applications into more manageable, independent services by implementing a modularized and microservices architecture. This architectural strategy has several cost-cutting advantages. First, it enables granular scaling, which prevents overprovisioning of resources by only scaling the required services in accordance with demand. Additionally, by precisely allocating resources to each service, microservices encourage efficient resource utilization while lowering overall infrastructure costs. By decoupling services, organizations can also take advantage of different pricing models, such as serverless computing, paying only for actual usage and achieving cost optimization.

Monolithic architecture, on the other hand, may provide management simplicity and potential cost savings as showcased in [40]. Organizations with monolithic architectures have a single code base, which lessens the challenges of managing and coordinating numerous microservices. Less resources are needed for monitoring, testing, and maintaining a single application, which can result in lower development, deployment, and operational costs. Additionally, because the entire application can run on a limited number of servers or containers, monolithic architecture may require less infrastructure resources than a microservices architecture, which lowers hosting and scaling costs. However, it is important to assess the specific needs and goals of the organization, as well as the scalability and future growth considerations, before deciding on the most suitable architectural approach for cost optimization.

Containerization technologies like Docker, coupled with orchestration frameworks such as Kubernetes, offer cost optimization benefits by improving resource utilization and workload management. Containers provide lightweight and isolated environments for applications, reducing the overhead of running multiple virtual machines. Organizations can effectively manage the deployment, scaling, and monitoring of containers with container orchestration, maximizing resource utilization and cutting costs. Additionally, because multiple containers can be installed on a single virtual machine, containerization enables more effective use of cloud resources while reducing infrastructure and licensing costs.

Cost optimization calls for the capacity to scale resources automatically in response to demand. Organizations can dynamically modify their infrastructure to suit workload patterns by implementing autoscaling policies. By ensuring that resources are only provisioned when necessary, autoscaling helps to cut costs during times of low demand. Cloud providers offer various autoscaling mechanisms, such as scaling based on CPU utilization, network traffic, or custom metrics, allowing organizations to right-size their infrastructure and optimize costs.

In serverless computing, also referred to as Function-as-a-Service (FaaS), programmers concentrate on writing code for particular functions rather than managing or setting up servers. The expense and difficulty of managing unused or underutilized resources are eliminated by this paradigm shift. With serverless, businesses save a lot of money by only paying for the time that functions actually take to execute. Therefore, by leveraging auto-scaling capabilities provided by the cloud provider, organizations can effortlessly handle workload fluctuations without incurring additional costs associated with idle resources.

These examples show how re-architecting a system or infrastructure can lead to significant cost savings. Re-architecting a system, however, necessitates careful planning, in-depth research, and an in-depth understanding of the current infrastructure and business requirements. Organizations can achieve cost optimization while enhancing agility, scalability, and resilience in the cloud by utilizing the advantages of modularization, microservices, serverless computing, containerization, and autoscaling. The next section explores case studies of companies that have successfully implemented system re-architecture strategies to achieve significant cost savings in their cloud infrastructure.

The audio-video monitoring service at Prime Video was originally designed as a distributed microservices architecture. This architecture consisted of a number of independent services, each of which was responsible for monitoring a specific aspect of audio or video quality. For example, one service may be in charge of monitoring audio loudness, while another service may be in charge of monitoring video bitrate. The microservices architecture had several benefits. It was relatively simple to develop and deploy new services, and scaling the service by adding more instances of each service was simple. However, the microservices architecture had several drawbacks. It was difficult to manage the service because there were so many independent services to keep track of. Furthermore, the service was not very scalable because each service had to be scaled separately.

Therefore, Prime Video redesigned its audio-video monitoring service as a monolith to address these challenges. A monolith is a centralized service in charge of all aspects of audio or video quality monitoring. By consolidating all of the services into a single monolith, the audio-video monitoring service was made easier to scale and manage. This eliminated the need to duplicate data between different services, which made the service more efficient.

AWS Step Functions is a serverless orchestration service for coordinating the execution of multiple AWS services. It is a powerful tool, but it is not cheap. Prime Video’s audio-video monitoring service orchestrated the flow of data through the service using AWS Step Functions. his resulted in a major bottleneck issue.

Every second of the stream, the service went through multiple state transitions. As a result, account limits on Prime Video were quickly reached. Because AWS Step Functions charges users per state transition, the total cost of all the building blocks was too high for the solution to be adopted on a large scale. Additionally, Amazon S3 was being used to store video frames by Prime Video’s audio-video monitoring service and the service made a large number of Tier-1 requests to Amazon S3. Tier-1 calls are the most expensive type of Amazon S3 calls that can be made. As a result, Amazon S3 storage of video frames was costing Prime Video a lot of money. Furthermore, the number of video frames that must be stored can vary depending on traffic volume. As a consequence, scaling the number of Amazon S3 calls that the service could make was impossible.

Prime Video redesigned the architecture of its audio-video monitoring service to a monolith application to address bottlenecks and cost issues. Because all of the components were now running in a single process, AWS Step Functions and Amazon S3 were no longer required. As a result, Prime Video no longer had to pay for state transitions or Tier-1 calls, which resulted in significant cost savings. The monolith architecture also allowed the service to be scaled up and down as needed. This was critical because the service was expected to handle an increase in traffic. Finally, moving the solution to AWS EC2 and AWS ECS also enabled Prime Video to take advantage of AWS EC2’s long-term compute savings plans as well as EC2 features like autoscaling based on the traffic load, which helped drive costs even lower.

Prime Video was able to reduce the cost of running its audio-video monitoring service by 90% as a result of these changes. The service was also capable of handling significantly more traffic.

The following are some key takeaways from this case study of Amazon Prime Video:

• •

  Architectures based on distributed microservices can be costly and difficult to scale.

• •

  Monolithic architectures have the potential to be more cost-effective and scalable.

• •

  Compute saving plans can assist you in saving money on Amazon EC2 usage.

• •

  Spot instances, reserved instances, and autoscaling can all help you cut your Amazon EC2 costs even further.

Pinterest runs multiple Flink jobs in various sizes and importance across their production YARN clusters. These jobs do everything from compute engagement statistics to process long-tail data. However, managing these jobs in a multi-tenanted environment while ensuring efficiency, resource availability, and interference minimization presented significant challenges. Pinterest identified several key issues related to cluster configuration and resource utilization during their cost-cutting journey. The lack of CPU isolation was one of the major challenges, resulting in unstable load tests and CPU bursts from one job affecting others on the same host. Maladjusted VCore reservations and burst capacity allocation also had an impact on resource utilization and overall cost effectiveness.

Pinterest undertook a series of initiatives to address these challenges and further optimize their Flink data processing clusters, including the implementation of CGroups soft CPU limits, capacity reservations, and container placement optimization. However, one of the most important steps they took was to switch from AWS i3 instances to i4i instances. Pinterest discovered that the new i4i instances performed exceptionally well with their Flink jobs, resulting in a 40% reduction in CPU usage for a marginal 10% cost increase. This upgrade not only improved performance, but it also reduced their overall platform’s AWS spend by 10%. By leveraging these strategies, including the adoption of i4i instances, Pinterest aimed to achieve better stability, improve resource utilization, and cost savings.

The following sections will delve into the specific steps Pinterest took to overcome these challenges and optimize their Flink data processing clusters, emphasizing the impact of their efforts on cost savings and performance improvements.

Pinterest embarked on an attempt to reduce the cost of their Flink data processing clusters. They aimed to improve stability, improve resource utilization, and achieve significant cost savings through a variety of initiatives and strategies. This section delves into Pinterest’s key techniques and their impact on cost optimization.

Pinterest’s efforts to optimize their Flink data processing clusters produced impressive results, including significant cost savings and performance improvements. They achieved better stability and resource utilization by implementing various techniques such as CGroups soft CPU limits, capacity reservation fixes, and container placement optimizations, resulting in a 20% cost reduction. Furthermore, switching from AWS i3 instances to i4i instances resulted in a 40% reduction in CPU usage. They achieved a 60% reduction in cross-host network traffic and a 50% reduction in CPU needs by mitigating CPU banding issues through improved task placements and leveraging colocation constraints. These measures, when combined with data optimization efforts, resulted in a 35% cost reduction on the Stream Processing Platform.

: Pinterest’s cost-cutting efforts are focused primarily on Flink job management and resource allocation within their YARN clusters. They must balance the overall system’s efficiency, ensure resource availability for higher-tier jobs, and prevent job interference in a multi-tenant environment. This includes improving CPU reservations for jobs of varying importance and scale, as well as optimizing CPU utilization and addressing noisy neighbor issues. On the other hand, Prime Video focuses on optimizing their video quality analysis system and monitoring infrastructure. Their goal is to reduce infrastructure costs while increasing the capacity of their defect detection system to handle thousands of concurrent streams. They address issues such as scaling bottlenecks, high costs of distributed components, and orchestration management limitations.

: The optimization efforts of Prime Video are aimed at lowering infrastructure costs and improving scaling capabilities. They intend to seamlessly monitor thousands of streams while minimizing the costs associated with distributed components and orchestration management. The goal is to handle increasing loads efficiently while also providing their customers with a high-quality streaming experience. On the contrary, Pinterest’s optimization efforts are aimed at improving the overall efficiency of their system. They intend to ensure that higher-tier jobs have the resources they require, to prevent job interference in the multi-tenant YARN environment, and to improve the overall efficiency of their Flink jobs.

: Pinterest’s multi-tenant YARN clusters address a lack of CPU isolation and inefficient capacity reservations. They use CGroups soft CPU limits to enforce CPU limits proportional to requested capacity, ensuring that burst capacity is regulated. They improve multi-tenant stability and prevent noisy neighbor issues by leveraging CPU-aware scheduling and introducing guaranteed burst capacity reservations. Prime Video redesigns their system to move away from a distributed microservices approach and toward a monolithic application. They eliminate the need for intermediate storage and reduce data transfer costs by combining all components into a single process. They use a single instance to implement an orchestration layer, which improves scalability and simplifies control flow.

: Both case studies emphasize the importance of optimizing resource utilization. Prime Video achieves resource utilization improvements by consolidating their components into a monolithic application. They eliminate the need for expensive video frame storage and reduce computational overhead by transferring data within memory. This enables them to process and analyze streams more efficiently, resulting in cost savings and improved performance. Pinterest prioritizes resource utilization in their Flink jobs. They address CPU banding issues by optimizing task placement, reducing cross-host network traffic, and enforcing colocation constraints. This results in more balanced CPU utilization and efficient resource utilization, resulting in cost savings without sacrificing performance.

: Both companies achieve significant cost savings through their optimization efforts. Pinterest’s optimization efforts result in significant cost savings. They are able to reduce their cluster size by 20% by implementing CGroups soft CPU limits and optimizing container placement. Furthermore, hardware upgrades to AWS i4i instances increase the efficiency of Flink jobs by 40%, resulting in cost savings and better resource utilization. By switching to a monolithic architecture, Prime Video achieves significant cost savings. The consolidation of components and the reduction of data transfer costs result in infrastructure cost savings of more than 90%. Using Amazon EC2 and Amazon ECS instances optimizes their cost structure even further, and they can benefit from Amazon EC2 compute saving plans for even more cost savings.

Both Pinterest and Prime Video demonstrate the effectiveness of their cost optimization strategies in their respective systems by focusing on these optimization targets, implementing architectural changes, improving resource utilization, and achieving significant cost reductions.