3. DevOps Essentials: Part 2

DevOps for Aspiring Engineers: A Comprehensive Guide

3. DevOps Essentials: Part 2

If you're aspiring to become a DevOps engineer, you're probably eager to learn about the tools, practices, and concepts that are essential to the field. In this article, I'll provide a thorough overview of DevOps and delve into some of the key ideas that you should know as you start your journey.

Different Types of Environments

In a DevOps workflow, it's common to use different environments for different stages of the development process. These can include:

  • Development environments: Individual developers use these environments to work on their code. These environments may be set up on a developer's local machine or in the cloud.

  • Testing environments: Quality assurance teams use these environments to test new features and ensure that they are working as intended. These environments may be set up to mimic the production environment as closely as possible, or they may be more simplified environments used for early testing.

  • Staging environments: These environments are used to simulate the production environment as closely as possible, allowing teams to test new features and ensure that they are ready for release. Staging environments are often used to catch any issues that may not have been detected in testing environments before the code is released to production.

  • Production environments: This is where the software is used by users. Production environments should be carefully managed and monitored to ensure the stability and availability of the software.

Bare Metal vs Virtualisation vs Containerisation

When it comes to deploying and managing software applications, several different approaches can be taken. These include:

  • Bare metal: In this approach, software applications are installed and run directly on physical servers. This can be a good option for applications that require a lot of resources or that need to be run on specialised hardware. However, it can be more complex and time-consuming to manage and update applications on bare metal servers.

  • Virtualisation: In this approach, multiple virtual machines are created on a single physical server, and each virtual machine runs its operating system and applications. This can be a good way to make better use of resources and reduce costs, but it can also add some overhead in terms of performance and management.

  • Containerisation: In this approach, applications are packaged with their dependencies into lightweight containers that can be easily deployed and managed. This can make it easier to build and deploy applications consistently, and it can also help teams more easily scale their applications up or down as needed. Some popular containerisation tools include Docker and Kubernetes.

Configuration Drift

Configuration drift is a term used to describe the difference between the desired state of an application or system and its actual state. This can occur when changes are made to a system without following proper procedures, or when changes are made in one environment but not properly propagated to other environments.

For example, imagine that I have a production environment with a set of servers that are all configured in a specific way. If one of the servers is updated with a new configuration, but this change is not properly documented or applied to the other servers, I may end up with a configuration drift problem. This can lead to inconsistencies between servers and can make it more difficult to manage and maintain the system.

To prevent configuration drift, it is important to use tools like version control systems and configuration management tools to track and manage changes to systems. This can help teams ensure that all systems are consistently configured and that changes are properly documented and propagated.

Ephemeral Instances in the Cloud

In the context of cloud computing, ephemeral instances are temporary instances of virtual machines or containers that are created and destroyed as needed. These instances can be used to run workloads on a short-term basis, and they are often used for tasks like batch processing or scaling applications up or down as needed.

Using ephemeral instances can be a cost-effective way to run workloads in the cloud since you only pay for the resources you use and you don't have to worry about the ongoing maintenance of physical servers. It can also make it easier to scale your workloads up or down as needed since you can simply create or destroy instances as needed.

Some examples of tools that can be used to manage ephemeral instances in the cloud include AWS EC2, Google Compute Engine, and Azure Virtual Machines.

Container Orchestration

Container orchestration involves the automation of the deployment, scaling, and management of containerized applications. This can help teams more easily deploy and manage large, complex applications that are composed of many different containers.

Some popular tools for container orchestration include Kubernetes, Docker Swarm, and Apache Mesos. These tools can help teams automate the deployment and management of containerized applications, and they can also provide features like self-healing and automatic scaling to help ensure the availability and reliability of applications.

Version Control Systems

Version control systems are tools that are used to track and manage changes to software code. These tools allow teams to track the history of code changes, roll back changes if needed, and collaborate with other developers on code projects.

One of the most popular version control systems is Git. Git is a distributed version control system that allows teams to track code changes and collaborate on code projects. Other popular version control systems include Mercurial, Subversion, and CVS.

Configuration Management Tools

Configuration management tools are used to automate the process of configuring and managing systems and infrastructure. These tools can help teams ensure that systems are consistently configured, and they can also help teams automate the deployment and management of systems and applications.

Some popular configuration management tools include Ansible, Puppet, and Chef. These tools allow teams to define the desired state of their systems and infrastructure as code, and they can then use automated processes to ensure that systems are configured correctly and that any changes are properly propagated.

Key Practices in DevOps

Several practices are central to DevOps, including:

  • Continuous Integration (CI): CI involves regularly merging code changes from multiple developers into a central repository, and then automatically building and testing the code to ensure that it is stable. By automating this process, teams can detect and fix problems more quickly, which can help reduce the risk of errors and downtime. For example, a team might use a CI tool like Jenkins to automatically build and test their code every time a developer pushes a change to the central repository.

  • Continuous Delivery (CD): CD involves automatically releasing code changes to production environments as soon as they are ready. This can help teams deploy new features and updates more quickly and safely. For example, a team might use a CD tool like Travis CI to automatically deploy code changes to a staging environment as soon as they pass the CI tests.

  • Continuous Deployment: Continuous deployment takes CD one step further, by automatically releasing all code changes to production environments as soon as they are ready. This can be a powerful way to accelerate the software delivery process, but it also requires a high degree of automation and a robust testing process to ensure that code changes are safe to release.

  • Infrastructure as Code: Infrastructure as code refers to the practice of using code to manage and provision infrastructure resources, such as servers, networks, and storage. By using code to manage infrastructure, teams can make it easier to automate the provisioning and management of these resources, which can help improve the speed and reliability of the software delivery process. For example, a team might use a tool like Terraform to define their infrastructure as code and use automated scripts to provision and manage resources.

  • Monitoring and Logging: Effective monitoring and logging are essential for ensuring the health and reliability of software systems. By using tools to monitor the performance and availability of systems, teams can quickly detect and diagnose problems, and by using logging tools to track the behaviour of systems, teams can better understand how systems are being used and identify opportunities for improvement. Some examples of monitoring and logging tools include Prometheus, ELK (Elasticsearch, Logstash, and Kibana), CloudWatch, and Loggly.

ToolArea of DevOpsDescriptionCompanyDifficulty (1-5)Open SourceWebsite
AgileProject ManagementA set of principles for software development that emphasizes iteration and flexibility2Yesagilemanifesto.org
GitVersion ControlTracks changes to code and facilitates collaborationGit3Yesgit-scm.com
MarkdownDocumentationA lightweight markup language for formatting textJohn Gruber1Yesdaringfireball.net/projects/markdown
MySQLDatabase ManagementA popular open-source relational database management systemDatabase3Yesmysql.com
PythonProgramming LanguageA high-level, interpreted language with a strong emphasis on readability and simplicityPython3Yespython.org
FlaskWeb FrameworkA microweb framework for PythonFlask3Yesflask.palletsprojects.com/en/2.1.x
LinuxOperating SystemAn open-source operating system based on the Linux kernelLinux4Yeslinux.org
JenkinsContinuous IntegrationAutomates the build, test, and deployment processJenkins3Yesjenkins.io
DockerContainerizationAllows you to package applications and their dependenciesDocker, Inc.4Yesdocker.com
Docker-ComposeContainerizationA tool for defining and running multi-container Docker applicationsDocker, Inc.3Yesdocs.docker.com/compose
Docker SwarmContainer OrchestrationA container orchestration tool for managing Docker containersDocker, Inc.3Yesdocs.docker.com/engine/swarm
NGINXWeb ServerA high-performance web server and reverse proxyNGINX, Inc.3Yesnginx.com
AnsibleConfiguration ManagementAutomates infrastructure and application deploymentsRed Hat3Yesansible.com
TerraformInfrastructure as CodeA tool for defining and deploying infrastructure as codeHashicorp3Yesterraform.io
KubernetesContainer OrchestrationManages a cluster of containerized applicationsGoogle4Yeskubernetes.io
AWSCloud ComputingA comprehensive cloud computing platformAmazon5Noaws.amazon.com


As you embark on your journey to become a DevOps engineer, it's important to focus on learning the tools and practices that are most relevant to your goals. Some key areas to focus on include:

  • Understanding the principles of continuous integration and delivery

  • Familiarising yourself with infrastructure as code and configuration management tools

  • Getting hands-on experience with containerisation and container orchestration

  • Learning about monitoring and logging tools and best practices

  • Gaining an understanding of different types of environments and how they are used in the development process

To get started, you may want to consider learning the basics of a programming language like Python and familiarising yourself with tools like Git, Docker, and Kubernetes. You might also want to consider joining a study group or finding a mentor to help guide your learning.

Finally, if you're looking for more personalized help and guidance as you learn DevOps, I offer one-on-one DevOps coaching to help aspiring engineers like you achieve your goals. If you're interested in learning more, feel free to book a free consultation to discuss your goals and how I can help.

Thanks for reading! If you found this article helpful, please consider following me on Twitter or signing up for my newsletter to stay up-to-date on the latest DevOps trends and best practices.

Did you find this article valuable?

Support Bilal Shafiq by becoming a sponsor. Any amount is appreciated!