Lerning DevOps
DevOps is a culture, movement or practice that emphasizes the collaboration and communication of both software developers and other information-technology (IT) professionals while automating the process of software delivery and infrastructure changes.
DevOps is an evolution of agile software development, which demands frequent releases and iterative feedback so that applications move steadily toward the business goal, nurtured by collaboration between developers and users.
How DevOps Emerged? Developer and Operations are two different roles. Developer develops an application. He is responsible for planning the application, including designing it, architecting it, developing it etc. At some point he needs infrastructure. So he goes to systems guys who immediately point out that he should have come to them earlier and now his infrastructure will probably be late. To developer this delay is unacceptable. How long does getting a bunch of servers take ? To infrastructure guys the developer is being unreasonable. What does he understand ? They all go to Operations guy who now has to find capacity to support this application. Why didn’t they come to him while the application was being planned ? And on and on it goes.
Role of infrastructure and operations was merged: To overcome these difficulties slowly the role of infrastructure and operations was merged. Also Developers started involving Operations early in projects. Operations in turn started providing tools like continuous integration and continuous Delivery to ease the developer load. Further automation lead to virtualization and more coöperation with containers. More trust developed between Developers and Operations and the roles started overlapping and thus the role of DevOps was formed. So beyond a co-operation between the development and operations team, a series of common tools are found at any organization that implements the DevOps culture.
What you should learn or in short which tools you should learn ?
- Source Code Management
- Scripting
- Continuous Integration Tools (CI)
- Virtualization
- Cloud
- Provisioning
- Monitoring
- Log Aggregation
1. SCM Source control management system (SCM) is software that provides coordination and services between members of a software development team. At the most basic level, it provides file management and version control so that team members don’t write over each other’s changes, and only the newest versions of files are identified for use in the workspace. But that’s only the beginning. SCMs also give developers the ability to work concurrently on files (in branches that may or may not converge), to merge changes with other developers’ changes, to track and audit changes that were requested and made, to track bug-fix status and to perform releases. In some cases, SCMs may include other components to assist in managing a software process throughout the entire lifecycle. The difference between source control management systems and application lifecycle management (ALM) systems is really a matter of semantics and reflects the completeness of the tools provided in the system.
Examples : i. GIT & GitHub "Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency"
"GitHub is a web-based Git repository hosting service, which offers all of the distributed revision control and source code management (SCM) functionality of Git as well as adding its own features." Git Hosting Service Providers : GitHub, GitLab Assembla, Beanstalk, Bitbucket, CloudForge, Codebase, Fog Creek Kiln,planio, Perforce, RhodeCode, Unfuddle
ii. Subversion iii. Mercurial iv. CVS v. ClearCase vi. Perforce a.k.a Helix
2.Scripting : For automation scripting one must learn scripting using Bash, Ruby, Python
3. Continuous Integration Tools : A technique designed and named by Grady Booch that continually merges source code updates from all developers on a team into a shared mainline. This continual merging prevents a developers local copy of a software project from drifting too far afield as new code is added by others, avoiding catastrophic merge conflicts. In practice, CI involves a centralized server that continually pulls in all new source code changes as developers commit them and builds the software application from scratch, notifying the team of any failures in the process. If a failure is seen, the development team is expected to refocus and fix the build before making any additional code changes.
CI servers (also known as build servers) automatically compile, build, and test every new version of code committed to the central team repository, ensures that the entire team is alerted any time the central code repository contains broken code. This severely limits the chance for catastrophic merge issues and loss of work built upon a broken codebase. In mature operations, the CI server may also automatically deploy the tested application to a quality assurance (QA) or staging environment, ensuring the Agile dream of a consistent working version of software.
Examples : Jenkins, Oracle’s Hudson, TeamCity, Bamboo, CircleCI, Travis CI, Codeship, Semaphore CI, Drone . io
4. Virtualization: Virtualization is especially useful in complex environments, i.e. when you want to have similar environments for Development, QA/Testing, and Production. So if you want to set up and test anything in advance before running your apps in production blindly, then you need to configure suitable environment (virtualization) that has everything required for your application or project to run properly (provisioning). The term “infrastructure as code” references to using tools to automate infrastructure configuration with all the benefits that brings.
Service virtualization allows developers and testers to access the exact test environments they need on demand. Organizations are able to accelerate the software development lifecycle and reduce risks with simulated test environments that promote earlier, faster, and more complete testing. Examples : Docker & Vagrant
5. Cloud: For DevOps you will mostly find hosting in clouds AWS provides a set of flexible services designed to enable companies to more rapidly and reliably build and deliver products using AWS and DevOps practices. These services simplify provisioning and managing infrastructure, deploying application code, automating software release processes, and monitoring your application and infrastructure performance.
With Microsoft Azure cloud platform, you can also use a host of popular open source DevOps tools such as Puppet, Chef, Terraform, etc
Open Shift, based on top of Docker containers and the Kubernetes container cluster manager, OpenShift 3 adds developer and operational centric tools to enable rapid application development, easy deployment and scaling, and long-term lifecycle maintenance for small and large teams and applications.
Walmart’s OneOps is a Application Lifecycle Management tool that abstract most of the complex task of packaging, deployment, provisioning in all environments and is meant to give developers control over their own deployments.
Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence.
6. Provisioning : Provisioning means automatically constructing software or server for a user to use without any manual steps. Provisioning is also used as an means Change Management. Examples : Various provisioning softwares available out there however for DevOps some are favorite like Chef and Ansible.
7. Monitoring : Nagios, Zabbix, AWS CloudWatch and StackDriver for Google.
8.Log Aggregation : All the servers and all the applications running on all the servers produce a lot of log files. It is prudent to deploy an log aggregation solution on all the server that not only channels all the logs to a central place but is also capable of providing good analysis of the log files. Example : Graylog, Splunk, Logstash
List of DevOps Tools for DevOps Engineers
- Chef
- Jenkins
- Puppet Enterprise
- Bluepill
- MCollective
- Capistrano
- Supervisor
- Bcfg2
- Graphite
- RabbitMQ
- Ant
- CFEngine
- Apache Maven
- Ansible