There is plenty information on the web regarding the philosophy of DevOps but I’d like to share what I feel DevOps is in slightly more technical terms because after all, I’m not Socrates or Plato and this is the work I am actually performing while working in industry
Starting off with the primitives and this concept is important to get right
DevOps engineering is synonymous with platform engineering and site reliability engineering. All these terms mean exactly the same thing. At the core, they are all software development and all follow the exact same problem solving strategies and techniques
You need to know how software works and you’ll have to have a few languages under your belt. Always remember that all DevOps engineering work is in code, no exception. You’ll follow a well thought through branching strategy and issue pull-requests on your work
If you can’t describe the solution in code or you randomly start inputting commands everywhere, stop
DevOps engineering adopts the “builder” attitude. You’ll build tooling for developer productivity, automating routine tasks and keeping the application stack online even when issues occur at 2AM
DevOps engineering is not operational support. I don’t care that your VPN connection won’t connect, mine probably won’t connect either. I’ll only care about that email alert if you have adopted the approach that the email alert should be the result of the automation detecting a problem and it’s informing you of the result of solving the problem on it’s own
These are the main categories I feel define the DevOps engineering work I do (they extend the definition above):
This is s very big part of what I do and it encapsulates many aspects of my work
I analyse technical and business processes with the view of building software that makes the processes efficient, less error prone and entirely automated
Developing for the web makes it really easy to ship the apps. The apps can either be entirely written in web languages or simply wrappers around the data that the underpinning applications output
Manual work has seen it’s final days and if you or anyone on the team are performing the exact same work repeatedly then something is fundamentally wrong with with how you interpret DevOps engineering
I have hit some fairly big home-runs regarding process efficiency and this results in my apps saving what I feel is quite substantial amounts of money. It’s hard to quantify sometimes but the basic equation is:
Total savings = cumulative daily salary of the engineers involved . x amount of days they perform the work over . x amount of repetition the entire task requires
A task that would otherwise require two engineers and would take fourteen days’ to complete (...and has to be repeated every 3 months) now requires zero engineers and will take the app one minute to complete and the app will not make any mistakes and you’ll know exactly what it’s done
I usually hammer out my apps that are involved with process efficiency in 2-3 weeks but this can vary depending on how complicated the tasks that I’m automating are. This 2-3 weeks includes the problem analysis and it might involve a conversation or two with the people that have performed the task before. Sometimes, even small web applications can make a big difference
All infrastructure is in code. All configuration is in code. It’s all described declaratively and tracked by git (the SCM). I build useful shell scripts that bring the IaC and CM together (i.e.: Once the infrastructure is provisioned, a shell script is automatically invoked and initially prepares the server for the configuration management). I want the deployment sequencing to flow and not wait for human interaction
Getting this right will enable the blue-green deployment strategy within your CI/CD and not just the traditional “rolling” deployment strategy (blowing the PHP opcache here is no problem because the pipeline will send in some warm-up HTTP requests)
Adopting an imperative approach to writing the IaC or configuration management is acceptable but it’s always a good idea to build on top of other people's work and to always ensure you don’t have convoluted conditional statements everywhere. This is particularly true when it comes to IaC and configuration management. Focusing on output is the best approach here and it can also be self-describing to a large degree
The ultimate aim that I achieve with systems and infrastructure automation is being able to run a single command and the entire application stack (including the supporting infrastructure and services) is deployed automatically, requiring no human interaction in minutes
I have ran projects at multiple companies that have done exactly this
Designing secure, highly available distributed application architecture takes a lot of thought time and effort as does designing a distributed dataset (CAP theorem)
Considerations around subject matter like PCI DSS compliance, north-south and/or east-west HTTP request routing, data consistency, serverless and autoscaling are all very complex
It’s important to always maintain good documentation around how the various services work and how they interact with each other
I also try to keep up with what industry expects DevOps engineering to be regarding service architecture and how other DevOps engineers have tackled the considerations and subject matter above. This has resulted in performing R&D around Kubernetes and running with internal projects and projects in my personal time that will bring me up to speed with Kubernetes
Decisions as to whether use the public cloud or manage your own physical hardware is actually not a trivial concept at all (I know industry makes it sound like it is but it’s really not)
Neutral decisions around the cloud vs physical hardware:
Reasons to go cloud
Reasons to go physical hardware
The ultimate goal for the DevOps engineering team is to become the company's Platform as a Service (PaaS) and the day-to-day work and planned sprints are situated around improving the platform. Think of this as the company’s internal Heroku
This goal is often a bigger project but it’s one that is manageable, provided you hire the correct DevOps engineering staff and encourage this as the desired outcome for the team
I build individual APIs around each of the systems and infrastructure components of the application stack and provide a nice abstraction layer to the developers (a lot of developers I have spoken too have no interest in how infrastructure or servers work) and this enables the developers to focus on core development of the app without having to spend time figuring out how the services work
I also build service APIs that help accelerate development. These service APIs can range from image processing to wrappers around SMS gateways
The platform will be offered to the developers via a series of APIs and useful apps. These apps are built for the CLI as well as for the web. I also build the API clients that I ship as PHP composer packages. Some of the APIs I build are very useful utilities and can be included in the DevOps engineering toolset and are very reusable
What to watch out for in DevOps engineering:
Startups are great. They are fast moving, don’t really have any bureaucracy and you can make agile decisions very quickly but they do have a downside
It may make more financial sense for a startup to completely ship DevOps to Heroku or some other PaaS or hosted solution. Often, startups don’t operate at scale or are not subjected to strict security or compliance regulations. They are often looking for a market-fit for their product or they have customers breathing down their neck and that’s all they care about
It’s not to say that the core development team has immunity against this type of thinking, they too can be outsourced to something like Upwork or Fiverr. Business people love the fact that code is written around the clock and it’s super cheap
You’d ultimately want to work for a bigger company or enterprise and not a cash strapped/homebrew startup. The enterprise is definitely where you want to be in terms of exciting and interesting DevOps engineering projects
I’ve heard it said before: “you want to be the startup, not work for someone else's startup”. This is very true
People might push back on process efficiency and automation in general because they feel their job is at risk
I often try to reason by analogy with people–I know first-principles reasoning is more effective but it might be to complicated for certain people, and the best analogy to use is evolutionary biology: if you don’t evolve, you die
Much like when fruit started growing on fruit trees which were growing higher, you either standup on two feet and eat or stay on all fours and hope for the best
You really need to understand the base concept and how the task really works when you are automating a specific task. If you don’t get this correct, often the software you feel has improved processes really has not. It’s probably made things worse and now you have a mess to clean up
There will always be a more effective way to express the logic, structure the data or perform the calculations. Just hammer out the app and refine the software as you go along. Just ensure that you don’t make obvious or dumb mistakes