Maximising Cost Optimization with AWS Best Practices
In today’s data-savvy world, businesses must rely heavily on cloud infrastructure.
Why?
To provide scalable and seamless services to their loyal customers, ofcourse!
One such cloud computing platform that has emerged on the top is Amazon Web Services (AWS).
This prominent platform offers customers extensive services, including functionality, performance, and flexibility. But at a high cost.
With organizations scrambling to outbid their competitors in the cloud game, organizations are pushing for expenses and cost optimization checks, especially regarding AWS.
Our client, Barbour ABI, faced the same issue. They wanted to maintain the platform on AWS cloud but to optimize the amount spent in a structured manner. So, let’s look at how we helped Barbour ABI reduce its overall expenses without compromising the quality of leads and data.
But who is Barbour ABI?
Let’s understand a bit about our client first before we discuss the problems and solutions.
Barbour ABI is a leading construction intelligence platform providing customers with quality construction market leads and data for over 80 years. One of their primary functions was to deliver verified project details and GDPR-compliant contacts.
But, for Bourbon ABI to achieve this, they must rely heavily on council websites.
This is because they collect, verify and refine data before sending it out to their customers via their portal. The management of data and projects is done in collaboration with our experts at Arrk, using multiple modules such as ML module, Platform, Planning Portal, Engines, and Harvest.
So, why did we look into Platform modules?
The idea of the Platform module is to serve as the backbone of the solution, and development is done using Python Flask (API) in the backend functionality and ReactJS for the frontend. Also, the primary databases used for storage and retrieval are Postgres and ElasticSearch.
Multiple engines, such as notification and export engines, also help the platform perform external tasks. This entire process is deployed on an AWS cloud that uses services such as Elasticache, EC2, Key Management, SQS, and more.
But, with great tasks comes greater costs!
Considering the number of programs used, the extent of data to be sorted through, and the platform’s overall security, it is no wonder why AWS is so expensive!
So this was our task – using AWS but at optimized costs!
Challenge Accepted!
We at Arrk were challenged with optimizing AWS’s cost structure by looking for ways to collect, refine and verify data collected on the platform. We also had to deliver strategies to help reduce the overall expenses without compromising the quality of market leads while steadfastly delivering the product to Barbour ABI’s client base.
And solve it we did!
How did we solve it?
Our expert team identified areas in Barbour ABI’s AWS cloud platform to highlight where significant costs can be saved. With a thorough analysis of the utility of infrastructure, the patterns of usage,and the total breakdown of costs incurred, we could showcase options for optimising the platform to the client.
With both our collaborative efforts, we were able to leverage tools such as AWS CloudWatch logs and AWS Cost Explorer to gain valuable insights into optimizing the cost structure and breaking down areas where spending can be minimized.
What did we find?
As per our analysis, the key areas for cost savings include:
- Right-sizing – Here, we were able to showcase to Barbour ABI how to reduce the legacy application’s size and instances for lower environments.
- Optimizing storage – This was achieved by reducing EFS storage size by changing the type from GP2 to GP3. Also, we changed S3 to Glacier for harvest documents and changed the S3 retainer period to 1 day.
- Auto-scaling – We implement auto-scaling for Web, API and also shut down of scripts for lower environments
- Containerization and Serverless – Migrating engines did this to Fargate and Docker by migrating API and Web to Fargate-based ECS clusters.
- Reserving capacity and saving plans – We introduced AWS saving plans to help cover computation and ML services and reserved instances for Harvest and RDS.
Knowing what we know, it was now time to formulate strategies to support our analysis.
How did we plan to save costs?
For our ‘AWS Cost Optimization’ project, we implemented a documentation and project management system.
How did we do this?
By using Confluence and Jira!
We used Confluence as a central documentation hub and maintained a dedicated space where all relevant details, plans, and progress reports were saved. These included details about cost-saving options and other technical information. This ensures that all team members have access to updated information and can collaborate with themselves easily.
Using Jira, we created a dedicated board to track and manage all the activities related to cost optimization. The board follows a Kanban method, allowing users to visualize the entire process in one go or part by part. Each task has clear objectives and deadlines and can be assigned to the appropriate team members. This was done to support efficient task management and ensure the project moves smoothly.
We created a collaborative knowledge-sharing space and smooth workflow using these two platforms.
In summary, the Barbour ABI partnership with us was the ideal occasion to demonstrate how a thorough and rigorous strategy may deliver efficient solutions to optimise costs in the ever-changing world of cloud computing.
To get crucial insights into the overall breakdown of expenses and pinpoint areas where optimisation may result, our committed team collaborated closely with Barbour ABI. We assisted Barbour ABI in delivering great market leads and information to their clients while upholding overall expenditure optimisation by developing tactics specifically tailored to Barbour ABI.
So, if you are looking to reduce cloud expenses and improve the services you provide?
Join forces with us right now!
