Cloud adoption at DVLA – the future
In the second post we talked about maturing our adoption of cloud and enforcing standards through our centralised platform team.
As we wanted to deliver innovative products faster, the centralised cloud platform team accelerated the deployment cadence of the product teams using AWS services, and a ‘paved road’ approach to DevOps. The centralised cloud platform team now offers operational excellence (maintenance, upgrades and governance) in a platform, while reducing duplication and inconsistencies that existed across multiple product teams. This allows our product teams to remain working on the customer-focused innovation.
The cloud platform team sped up the delivery of many of our projects, there was a well-documented path to get an application running in production and this became incredibly valuable at a time when we had to get new services live quickly to support our key workers during the COVID19 pandemic.
We continue to improve our in-house capability, with more than half of the team now holding AWS certifications at one or more of associate, professional or speciality level.
It’s amazing the pace at which cloud technologies change. Over the last few months we have continued to keep the platform up to date with the latest security patches and enhancements.
We’re migrating our self-managed Kubernetes cluster over to the AWS managed Kubernetes service (EKS). Moving to managed services has reduced the time engineers spend keeping things running. We’ve been able to add new features like the standard Platform CDN and autoscaling on both our Kubernetes clusters. Our build servers now scale the number of nodes out and in, ensuring that we aren’t wasting resource. Investment into new tooling for code scanning and automation testing has been made to reduce any code smells and vulnerabilities in the pipeline. The cloud platform has also started running the automation test packs within the continuous delivery pipeline giving us that critical fast feedback. All this will be rolled out to all projects both existing and new.
These platform initiatives, although successful, are challenging when scaling beyond 5-6 cross-functional teams. Greater demand is placed on the centralised platform team, whether asking for advice or configuring a new database or queue. This can result in delays for product engineering squads as they wait on the platform team to write code to provision infrastructure.
The platform team are building the infrastructure as a product to be consumed by each engineering squad via self-service, with full automation. A shared ‘self-service’ platform will continue to reduce overlap and redundancy, while building in security, governance and best practice.
There are a number of areas focused on as we introduce self-service:
Self-service portal – this is the main point of interaction for our engineering squads, it will host product spaces so the teams have a clear sense of ownership. The main advantage of investing in ‘paved road and standardisation is that it can now be fully automated across all our products. The ability to deploy and configure applications across multiple environments, with all the supporting services being created in the background is now relatively straight forward. The first protype has demonstrated that an application on a developer laptop can be running within the Cloud Platform production environment within minutes.
Service catalogue – there are a number of services that we consume within the cloud provider. These are all defined in code but it’s the platform team that has to do the deployment. Using a service catalogue allows teams to select a service and have it deployed with all the security and best practice applied.
Service mesh and service discovery– – as we continue to deploy microservices to our platform, the more complex they become to manage. A Service Mesh is an additional layer that helps us manage the service to service communication, adding circuit-breaking, latency-aware load balancing, retries and deadlines, without our engineering squads having to worry about this in our application code.
Serverless – the rise of serverless is becoming even more popular which is no surprise, as it removes a lot of complexities of running a platform. It does, however, give a different a set of challenges for a cloud platform. Moving from the 10s of accounts to managing 100s has to be thought through. Moving from a multi-tenant Kubernetes cluster maintained by a small group of cloud engineers to product aligned accounts, means we now need to deploy guard rails across all the accounts to ensure our cloud platform remains secure.
Recruiting Cloud Engineers from the market continues to be a challenge due to the demand for skills, but we have built a great team by retraining existing staff. We’ve recently launched a new Cloud Academy programme which attracted over 300 applications.
The Cloud Academy programme is a fantastic opportunity to undertake and learn at an entry level. You get the opportunity to put theory into practice on real world projects, whilst on the job with the cloud platform team. The Cloud Academy allows our new engineers to grow wider skills such as team work, resilience, communication and problem-solving skills, along with developing technical skills in software engineering, coding, infrastructure engineering, infrastructure testing, and security.
The first 10 ‘recruits’ have already started and are currently in a 12-week bootcamp, before starting work alongside our engineers. After the 12 weeks they will have a number of certifications in DevOps, and AWS. They will then work with cloud engineers in the real-world.
DVLA continues to build and invest in its staff and cloud platform, the systems we use and are developing rival some of the biggest technology services in the UK. Last year alone we reached a record breaking one billion interactions with our customers with over 90% online. Our Cloud platform is central to continuing that success.
If you’d like to hear more from me then I’ll be talking about our Cloud Adoption journey at Digital Leaders week alongside Matt Lewis, DVLA Chief Architect.