3 Things I wish I knew as DevOps Engineer

Lately I’ve been reflecting back on the last couple of years when I used to be in a DevOps focused role. As many of you may already know DevOps is more about the way we work as developers and operational folk rather than a job role. So when dealing with developers or operations people, I’ve always tried to teach and cultivate a DevOps mindset to help those people get things done efficiently. Also, try to make myself redundant so those people can focus on getting things done, rather than rely on me to provide a service. Doing this over the last couple of years made me realise that there are some key aspects to DevOps that I would like to share. Dealing with different people from different cultural backgrounds, I found similarities in how we all do things and I started noting down points which I’ll describe below in more detail.

When I started jotting down these “DevOps tips”, I was like “Damn! I wish I knew these things when I started as a Developer” (or even earlier!)

I’ve recently released a video on my YouTube channel covering the topic, so if you’re interested in watching a video version, check it out below. I also make various other content for DevOps \ Developers and technical folks.

The first thing I wish I knew when I started as a DevOps engineer is understanding how biased opinions can limit your effectiveness in DevOps.

“Biased DevOps”

Being biased about a specific technology is not always a bad thing but sometimes it can get in the way of you and your team since it can affect the decisions you make. It can also make you a little short sighted, since you become quick to make up your mind. It then leads to you potentially ignoring better possible alternatives.

I have my own biased opinions about technologies I like, but knowing and realising the strength and weaknesses of a tech stack, is a super power in the DevOps world.

This applies developers as well as operations. Let’s take a look at some examples here.

Use case: Programming Languages

I want to take programming languages as a first example. Having a fixation on a specific language just because you have invested many years in it limits your skills and ability to solve problems efficiently. It’s a powerful skill to be able to pick a language that’s the strongest for the task you are up against, without having to really learn everything about the language.

I used to write C# for many years. When I start a project, I would start coding in C# immediately. When working on a data driven project, I chose C# as it was my strong point. I realised after spending days on ingesting data, that the performance was suffering considerably and I spent so much time optimising for performance, that I could not focus on getting the business logic right.

I then decided to step back for a moment. Started reading blogs about data and realised that most are using Python. Looking at sample code, I tried a short proof of concept with the data I had and before I knew it, i solved my performance problem in just a few minutes. It made me realise that the libraries I had in Python was designed for the job at hand. My biased ways of working, got the best of me.

In my video I give examples of this using simple statically compiled binaries in golang, operating system distribution choices and open source technologies.

Step back for a moment

Sometimes it helps to re-evaluate a problem you are facing by stepping back and looking at the problem with a wide-angle lens. Also not thinking about a technology you’re wanting to use to solve the problem.

As a person, its normal to have biased opinions, but remember as a DevOps engineer, you’re going to be presented with problems by a wide audience, so being able to challenge problems with multiple solutions will help you get a better outcome. So always ask yourself, “Are there other ways to solve this problem?”

Enough of that, time for my second tip:

“Understanding the fundamentals”

One thing to take away from our programming challenge described above, is that we needed to know about the fundamentals of technologies. We needed to know the strength and weaknesses of Python or C#.

People often ask me, “How do you learn new things so fast? How do you keep up with new technologies?”

In theory it's quite simple. The trick is, when you want to learn about a technology, learn about its fundamentals, not always about how to use the technology. Here is an example, Let’s take a look at docker:

When you start to learn about docker, the focus is always set on the dockerfile as well as the docker run and docker build command. So basically you only learn how to use docker.

Now instead of just doing that, it helps to peel away some of the layers and understand the fundamentals on which docker is based. If you dive into Linux kernel features that allow for containerisation, it helps you understand how docker works. So instead ask yourself, “What is a container? What makes a container?”. Understand its anatomy. You’ll learn about cgroups and namespaces. There are tons of information on the topic.

Now where this strategy becomes powerful is when you want to move from Docker onto Kubernetes. Every fundamental aspect you’ve learnt about containers applies to Kubernetes. So now you are aware of Linux features that make up the container. So you’’ll quickly grasp and realise that Kubernetes is simply an automation tool focused around the Linux features that help schedule container process on machines. Kubernetes uses the built-in Linux scheduler, it uses built-in OOM killers, it relies on Linux networking. You’ll learn about the Linux TCP stack, iptables and much much more.

You’ll notice above that I take a very different strategy to learning things. If I had gone to a website, I would’ve ended up learning about YAML files instead, and not really learning about how Kubernetes really works. I’d be missing out on Linux fundamentals, networking, service meshes and sidecar patterns!

Abstractions — Good or Bad ?

So if I took the traditional approach and learnt about docker and Kubernetes, I would be missing out on a lot of fundamentals as you can see above. Many technologies like Docker and Kubernetes have simple interfaces that makes it easy to use. Developer experience is the central focus point for a lot of technologies and cloud providers. In my opinion, abstraction can help a programmer, who only writes code not to worry about things outside of programming. In modern distributed systems, things are not that simple. If you have that mindset of only writing code as a developer, you are limiting your career potential.

In DevOps, we’re engineers. If you want to be a successful engineer, you often have to dive into implementation details, and peel away the layers of abstraction.

Final tip of the day:

Keep it Simple

This is probably a statement you must have heard before but it’s something I constantly having to remind myself. We live in a world where things are constantly over-engineered and it’s so easy to do so. Cloud providers make it so easy to get full stacks of technology up and running in a few clicks, but when it comes to features you actually need, things get quite interesting.

A simple google search on “cloud overspend” will show you how millions of dollars are wasted by companies. Also a very high percent of cloud compute is wasted and not actually being used efficiently. Ask yourself, “how much idle workloads do you have right now in your company?”

Examples of overspend can go from building an enterprise big data solution all the way to hosting a simple HTML website on a very complex Kubernetes cluster. I cover many examples in my YouTube video, so be sure to check that out!

You don’t need a big data solution if all you are doing is storing a few JSON files. You don’t need a complex Kubernetes cluster when you can host an HTML file on an S3 bucket. Don’t ingest terabytes of metric data into your third-party monitoring system when you are only monitoring a handful of metrics. Sometimes we can take a step back, look at the problem and solve it with a simple alternative. Keep it simple, Thing simple.

Sometimes we fly with tunnel vision and we’re so focused on the problem and excited about our implementation that we ignore possible simpler alternatives. Therefore, taking a step back and looking at the problem with a wide angle lens helps create new possible outcomes.

It happens more often than not, that the problem is self inflicted and when talking to the right people, one actually realises that the problem perhaps did not exist in the first place, or there was simply a misunderstanding which may end up making the actual problem easier to solve… So keep it simple.

Expand your super powers with Open Source

Consider learning and trying out open source tools in your own spare time. Technologies today makes it super easy to spin up and evaluate open source technologies, whether it’s Terraform or Docker, you can create local environments with all the dependencies and architecture with relative ease. Where ever it is that you work, or whatever it is that you do, sometimes you have to take matters into your own hands and go out of your way to learn new things. If your career has you stuck doing one monotonous thing, maybe build your own website or blog where you experiment on a technology that may help your current career.

Maybe you are stuck servicing build system “A” and your current role does not allow you to focus on improvements. Well, create your own personal site and try out build system“B” on your own site. This will help you expand your knowledge and give you something to showcase to your team to prove out the benefits of moving from system “A” to “B”. This is just a simple example, but playing around in the open source space will allow you to learn about many new tools and technologies. Every tool you learn is like a super power that you gain, which not many others will have.

Focus on your weak points

Depending on your current skill set, you may feel like you’re either strong in development and weak in infrastructure, or the opposite, and that’s ok.

I’d encourage to get out of your comfort zone and learn about the areas where you’re weakest at. Bring up those weak points. I personally see DevOps as more of an engineering role that is able to cultivate and empower people to be efficient. If you want to be a great engineer, being able to write some code places you ahead of the pack. You could perhaps build your own lightweight tooling to empower teams by filling out gaps in their processes. Having a stronger infrastructure background will help you tune a system where developer’s code is running, and understand things like performance, observability and many other cool things.

In my video I give a couple more examples on how I follow this strategy

Hope these tips are useful and let me know what your strategies are and things you wish you knew when working in DevOps and IT in general

Be sure to pop over to my channel and leave a like :)

Peace!