So many clients of mine tend to start with Kubernetes (K8s) whether it be in AWS, GCP, or some hand-baked thing on EC2s. Overall most startups don’t need a full-blown K8s solution. There is rarely a need to be multi-cloud and generally, it is something that just causes more unnecessary work and causes a bloated DevOps department.
Starting simple with something like Fly.IO, Fargate, or Cloud Run. These are all solutions that still run in the cloud and can reduce the amount of management that is necessary in order to run various services. This can allow for a highly resilient system to exist while not requiring a massive DevOps group to manage it and keep everything up to date.
Many times I have found that even as a company begins to scale this still tends to be a viable solution for them. This allows a very elastic scaling with very little in the way to prevent it from scaling.
Cost Differences
Overall there are some major cost differences that can happen. With a single node GKE cluster you are looking at between $100 and $250 per month in Cloud Cost just for the cluster. From there you then have to consider any maintenance cost that might have to happen.
With Fly.IO for a monolithic API and simple Postgres Database this same setup can fit within their Free Tier. This would allow the initial deployment to operate in a very cost-effective manner.
Management
Within Kubernetes there is the continual need to upgrade systems and ensure that any patches are applied. There are also regular updates to the Kubernetes layer that has to be applied.
Using something like Fly.IO, Fargate, or Cloud Run removes a lot of the system management component that exists. It allows you to focus on your app development rather than having to fix little small problems.
When to use Kubernetes
Ultimately being a very big fan of Kubernetes personally there are absolutely times that is should be used. If you are preferring to deploy into a Datacenter rack rather than the cloud this could be a fantastic opportunity to leverage K8s. The other side would be if there is a need to be more multi-cloud and wanting to generalize deployments rather than having to create totally separate deployments.
I completely agree that Kubernetes as a starting approach to a monolithic API needs to be considered carefully, especially in early stage startups. You could also consider solutions such as Elastic Beanstalk and Heroku.
I would, however, add a number of scenarios to the “When to use Kubernetes” section. If you have an existing org of sufficient size supporting an app that is currently running on bare cloud VMs (such as EC2) and load balancers or some other form of ingress, you may be able to achieve significant cost savings, and, more importantly, very fast scaling, by migrating to k8s. Kubernetes is also quite amenable to microservices.
Kubernetes maturity in the devops space gives it a leg up on newer solutions like fly.io, and its consistency among multiple cloud providers means you’ll always be able to hire competent engineers, something that would be harder if you chose a vendor locked solution like ECS on fargate, or cloud run.
While you’re still paying $72/mo for the k8s control plane (EKS) in AWS (not sure of GKE cluster cost for just the CP), you take advantage of Fargate to run your workloads, and get the best of both worlds, the maturity of running kubernetes, without the siloed knowledge prone to vendor locked solutions.
Thoughts?
Absolutely there are a few other items that make sense to use Kubernetes. I think that at the point where you are running at a sufficient size some of the benefits of using a more Fargate solution are less noticed.
Fargate and Cloud Run aren’t as vendor locked in my opinion. Both of them ultimately just deploy a docker container. The bigger vendor lock is going to come in with what services you choose to use.
Plus Cloud Run does seem to be slowing some as GKE AutoPilot is growing. It seems to be becoming a nice on-demand mixed with Kubernetes.
To clarify, you’re saying there isn’t typically a need for smaller shops to run their own K8S control plan or hosts, correct? Many of the smaller (and sometimes not so small) Dev teams I encounter are deploying to solutions like AWS Fargate while also using AWS ECS/EKS.
I also see a lot of teams using solutions like Heroku and ControlPlane.com and just not worrying about managing their own infrastructure at all.
There is certainly some benefit in using both. Leveraging EKS for some of the GPU items and things where a full machine is necessary is absolutely useful.
Heroku is an awesome solution that I have used plenty of times in the past. Fly.IO has just been my favorite lately purely for the ease of everything (and the Grafana dashboards).
@Kammerdiener What’s this about grafana dashboards with Fly.io? (I know, I can just google it) – but how has that feature helped you?
I’m struggling right now with multiple telemetry solutions at the same time – Grafana, prometheus, and Loki on our k8s clusters, and loggly and datadog for our application code. Since our platform runs on a combination of AWS Lambda and k8s, the k8s-native solutions aren’t really tenable.
Do you have any recommendations for an inexpensive catch-all tracing, logging, and metrics solution? You’ve mentioned Grafana Cloud in the past – is that still on your radar?
Fly.IO offers some solid dashboards to get you started with managing your app.
Inexpensive is the tough part. Grafana Cloud is awesome however depending on the amount of data you are ingesting self-hosting Grafana can still be cheaper. I have been able to help by hosting the Grafana stack for clients in the past. This has helped provide them with the solid observability they needed while providing a very consistent cost (and not having to worry about the day to day maintenance of it).
@Shane Regarding the control plane, I would take that a step further and assert that there’s almost* no need at all for anyone, except when running on their own hardware, to ever self-manage a kubernetes CP. I worked on a team for two years that did this, as the solution was built in the early kubernetes days, before the advent of tools like kops or kind, and before any managed solutions such as GKE and EKS were made available by cloud providers, and as I was leaving, we were transitioning away from the hand-rolled solution to using managed cloud k8s solutions exclusively. It was incredibly costly (labor and sanity) to self-manage the CP, and if not for the corporate inertia of an F500 company, would have swapped over to EKS much sooner.
As for the data plane (nodes/hosts/VMs), the flexibility you get from the managed solutions allows you to choose between the high degree of control in self-managed nodes, the ease of letting the cloud provider manage your node groups for you, or the very quickly scalable and flexible, but more expensive (at scale) “compute as a service” sort of options like Fargate, which blur the line between fully managed CAAS solutions like AWS Lambda and a standard kubernetes deployment.
* there are always exceptions