There has been a lot of inquiries about how to get started quickly with what is commonly referred as the hyperscalers. Let’s dive in for a super quick primer!
All of these quickstarts assume the reader has accounts in each service with the appropriate rights and in most cases the reader needs to have the client installed.
export NAME="$(whoami)-$RANDOM"
export AZURE_RESOURCE_GROUP="${NAME}-group"
az group create --name "${AZURE_RESOURCE_GROUP}" -l westus2
az aks create --resource-group "${AZURE_RESOURCE_GROUP}" --name "${NAME}"
az aks get-credentials --resource-group "${AZURE_RESOURCE_GROUP}" --name "${NAME}"
As you can see setting up these clusters is very simple. Now that you have a cluster what are you going to do with it? Ensure you’ve installed the tools needed to manage the cluster. You’ll want to get the credentials from each copy into ~/{user}/.kube/config (except with eksctl as it copies the kubeconfig to the appropriate place automagically). To manipulate the cluster, install kubectl with your favorite package manager and to install applications the easiest way is via helm.
As you can see the setup of a kubernetes cluster in one of the major hyperscalers is very easy. A few lines of code and you’re up and running. Add those lines into a shell script and standing up clusters can be a single command…just don’t forget to tear it down when you’re done!
Comments Off on Kubernetes quickstarts – AKS, EKS, GKE
Building efficient development pipelines in Kubernetes requires the right tooling, a lot of planning, and a good understanding of your development processes.
To quickly build and deliver robust products and benefit from automation and efficient collaboration, the software team relies on continuous integration/continuous delivery (CI/CD) pipelines. Implementing CI/CD for cloud native applications makes delivery cycles more robust while streamlining the development and deployment workflow.
Let’s talk about the key components of a CI/CD pipeline, how to optimize these pipelines and some recommended best practices and tools.
What Makes an efficient CI/CD Pipeline
The Kubernetes platform and CI/CD workflows both aim to improve software quality, as well as automate and boost development velocity. So companies benefit from having CI/CD pipelines to use with Kubernetes.
The following are some key components of a Kubernetes-based CI/CD pipeline:
Containers help achieve encapsulation of application components while enabling seamless integration through runtimes.
Operating clusters deploy the containers for your software build once the CI/CD tool approves the containers.
Configuration management stores all details related to the infrastructure setup and identifies any newly introduced change in the system.
A version control system (VCS) is a unified source code repository that maintains code changes. This generates the trigger for a CI/CD tool to start the pipeline whenever a new change is pushed into its repository.
Image registries store container images.
Security testing and audits maintain the equilibrium between rapid development and security of the application by ensuring the pipelines are free from potential security threats.
Continuous monitoring and observability allow developers to obtain actionable insights and metrics by providing complete visibility into the application life cycle.
Key Considerations to Make Your Pipeline Effective
CI/CD sits at the core of DevOps practice, enabling a sustainable model to streamline and accelerate production releases. A comprehensive understanding of the workflow is fundamental to building an effective CI/CD pipeline, along with evaluating the enterprise requirement to help choose the right framework.
Below are some key considerations for making your pipeline effective:
All-in-one CI/CD tool vs. case-specific solutions: Similar to the infrastructure setup, it is crucial to diligently assess the available CI/CD tools based on use cases, technical requirements and organizational goals.
On-premises vs. managed vs. hybrid CI/CD: Each CI/CD pipeline type has its own effectiveness, depending on your requirements and infrastructure. Factors that determine the type of CI/CD pipeline to choose include ease of use, ease of setup, infrastructure and operating system support.
Code testing and validation: An effective validation and automated testing framework is one of the core components of a CI/CD pipeline. This ensures a stable build with zero code-quality issues while highlighting potential failure scenarios.
Rollbacks: These help organizations redeploy the previous stable release of an application. Implementing a diligently planned rollback mechanism in CI/CD is vital to safeguarding the application in case of failure or security incidents.
Defining a Kubernetes-Based CI/CD Pipeline
While defining a Kubernetes-based CI/CD pipeline, you can go with one of the two major paradigms below.
Push-Based Pipeline
An external system like a CI pipeline generates build triggers to deploy the changes to the Kubernetes cluster following a commit to a version control system repository in a push-based pipeline. Kubernetes cluster credentials are exposed outside the domain of the cluster in such a model.
Pull-Based Pipeline
Kubernetes operators deploy the changes from inside a cluster whenever new images are pushed to the registry in a pull-based pipeline.
Some Best Practices
Here are some recommendations for building an effective Kubernetes CI/CD pipeline. These include some useful best practices.
Avoid Hardcoding Secrets and Configurations in Containers
You should store configurations in configmap and not hardcode them in the containers. This provides the flexibility of deploying the same container in different environments without making environment-specific changes to it.
It’s also recommended to keep secrets out of containers and encrypt and store them in Kubernetes Secrets. This prevents credentials from getting exposed through a version control system in a CI/CD pipeline.
Use Helm for Deployments
Use the Helm package manager for Kubernetes application deployments to keep track of releases or logical groupings.
Enable Git-Based Workflows
To allow for all infrastructure configurations to be stored within git, CI/CD pipelines should follow a GitOps methodology. It makes infrastructure code more accessible to developers, letting them review the changes before they’re deployed.
Git also provides a unified source repository and snapshots of the cluster. These are easy for developers to refer to as needed and recover the application to the last stable state in the case of failure.
Use Canary/Blue-Green Deployment Patterns
Parallel to the running production instances, running a blue-green pattern of instances lets you test changes and switch over traffic when testing is complete, eliminating the need for downtime during deployment.
Cache and Reuse Container Images
Use caching and reuse features of Docker container images to minimize container build times and reduce the risk of introducing defects into the newly built container image.
Tools for Kubernetes CI/CD Pipelines
All-in-One CI/CD Tools
GitHub Actions is an open source CI/CD tool that supports automated build, test and deployment pipelines. It is the preferred CI/CD platform when the source code repository is GitHub.
GitLab CI/CD facilitates the continuous build, test and deployment of software applications without the need for third-party integration. Checkout our article on implementing a Gitlab pipeline for your project.
Jenkins (including Jenkins X) is an open source, automation server that promotes CI and CD in varying levels of cluster complexity, enabling developers to automate application build, test and deployment processes seamlessly across hybrid/multicloud setups. Jenkins X is an upgraded version that facilitates automated CI/CD for cloud native containerized applications and orchestration tools like Kubernetes or Docker.
Rancher Fleet is fundamentally a set of Kubernetes custom resource definitions (CRDs) and controllers that manage GitOps for a single Kubernetes cluster or a large scale deployment of Kubernetes clusters. It is a distributed initialization system that makes it easy to customize applications and manage HA clusters from a single point.
CI Tools
Circle CI is a cloud-based CI tool that uses an API to facilitate automatic Kubernetes deployments. It is intensely focused on testing the new commit before deploying via various methods like unit testing, integration testing, etc. Because of its features for implementing complex pipelines with configurations like caching and resource classes, it is one of the most popular lightweight integration tools for a Kubernetes ecosystem.
Drone CI is an open source CI tool built entirely on Docker that uses a container-first approach. The plugins, components and pipeline stages of Drone are deployed and executed as Docker containers. The platform offers a wide range of flexibility for using different tools and environments for the build, but you have to integrate it with a git repository.
CD Tools
Spinnaker is an open source continuous delivery tool that integrates with multiple cloud providers. Since the platform does not rely on a GitOps model, config files can be stored in the cloud provider’s storage.
Argo CD is a declarative GitOps continuous delivery tool that is lightweight, easy to configure and purpose-built for Kubernetes. The platform considers git the source of truth, which enhances security, making access control and permission management easier to administer.
Automation and Infrastructure Configuration Tools
Terraform by Hashicorp is an open source Infrastructure as Code tool that facilitates DevOps teams’ ability to provision and manage infrastructure programmatically via configuration files.
Red Hat Ansible is an open source automation platform that enables automation for provisioning, configuration management and infrastructure deployment across cloud, virtual and on-premises environments.
Salt by Saltstack contains a robust and flexible configuration management framework, which is built on the remote execution core. This framework executes on the minions, allowing effortless, simultaneous configuration of tens of thousands of hosts, by rendering language specific state files. Unlike Ansible, Salt is agentless, relying instead on secure shell (SSH) connections to complete tasks. For a security architect, Salt is a gem!
Collaboration and Issue Management Tools
Jira is implemented by teams for software collaboration, defect tracking and work management. The tool offers customizable features like an intuitive dashboard, optimized workflows, efficient search, filtering and defect management. Jira is purpose-built to support various use cases of project management, such as capturing requirements, test case management and tracking tasks in real time.
Zendesk is a cloud-based customer support platform that enables an organization to engage with its client through different collaboration channels, including phone, email, chat and social media. Zendesk provides one easy-to-use platform for cross-functional collaboration and customer communications, thereby helping organizations to better manage customer queries and respond quickly.
Security
Open Policy Agent (OPA) is an open source policy engine that supports a high-level declarative language that lets developers specify Policy as Code. The platform is built to impose granular-level policies on different components, including CI/CD pipelines, microservices, Kubernetes clusters, etc.
Kubewarden is an open source policy engine simplifying the adoption of policy-as-code. It does not require any domain specific knowledge or new language constructs and can take existing policies, compile into WebAssembly and deploy into existing pipelines using existing processes.
Kube-bench is an open source tool used to run the CIS Kubernetes Benchmark test on Kubernetes clusters. This ensures that the Kubernetes cluster is secure and deployed according to the security recommendations in the benchmark document.
SUSE NeuVector is a fully open source end to end cloud native security platform to implement zero-trust security in containerized environments. With full support for Openshift, Kubernetes, and simple containerized workloads, SUSE NeuVector allows for complete visibility into your cloud-native network and will prevent any communication not explicitly required for an application or workload to function.
Monitoring Tools
Foresight is an observability product for CI pipelines and tests that enable secure, real-time monitoring of CI/CD pipelines. In addition to tracking metrics, traces and logs, the platform offers live debugging capabilities to facilitate quicker resolution of failures.
Prometheus/Grafana are open source, event-monitoring tools that implement a high-dimensional data model and store metrics along with timestamps in a time-series database. Prometheus ships with a flexible query language and is one of the most popular alerting systems for complex Kubernetes clusters. Based on metrics generated by Prometheus, Grafana offers built-in visualization support for efficient querying and analysis.
Summary
Delivering high-quality software at speed is not easy to sustain and scale. If you develop modern applications today, CI/CD sits at the heart of your software development process because it offers agility, reduces risks of production recessions and ensures quality. It is often considered critical to build an effective CI/CD pipeline for rapid workflow execution. Doing so requires diligent technical analysis, a generous amount of planning and choosing the right set of tools.
Comments Off on Using CI/CD pipelines in Kubernetes explained
Amazon’s EC2 has several options for running instances. On-demand instances is what would be used by most. Reserved instances are used by those who can do some level of usage prediction. Another option which can be a cost saver is using Spot instances. Amazon claims savings up to 90% off regular EC2 rates using Spot instances.
AWS operates like a utility company as such it has spare capacity at any given time. This spare capacity can be purchased through Spot instances. There’s a catch, though. With a 2 minute warning, Amazon can take back that “spare capacity” so using Spot instances needs to be carefully planned. When used correctly Spot instances can be a real cost-saver.
When to use Spot instances
There is a fairly broad set of use cases for using Spot instances. The general consensus is simply containerized, stateless workloads, but in reality there’s a lot more.
Distributed databases – think MongoDB or Cassandra or even Elasticsearch. These are distributed so losing one instance would not affect the data; simply start another one
Machine Learning – typically these are running training jobs and losing it would only mean the learning stops until another one is started. ML lends itself well to the Spot instance paradigm
CI/CD operations – this is a great one for Spot instances
Big Data operations – AWS EMR or Spark are also great use cases for Spot instances
Stateful workloads – even though these applications would need IP and data persistence, some (maybe even all) of these may be candidates for Spot instances especially if they are automated properly.
Be prepared for disruption
The primary practice for working in AWS in general, but also working with Spot instances is be prepared. Spot instances will be interrupted at some point when it’s least expected. It is critical to create your workload to handle failure. Take advantage of EC2 instance re-balance recommendations and Spot instance interruption notices.
The EC2 re-balance recommendation will notify of an elevated risk of Spot instance interruption in advance of the “2 minute warning”. Using the Capacity Rebalancing feature in Auto-scaling Groups and Spot fleet will provide the ability to be more proactive. Take a look at Capacity Rebalancing for more detail.
If the workloads are “time flexible” configure the Spot instances to stop or hibernate vs terminated when an interruption occurs. When the spare capacity returns the instance will be restarted.
Use the Spot instance interruption notice and the Capacity rebalance notice to your advantage by using the EventBridge to create rules to gracefully handle an interruption. One such example is outlined next.
Using Spot instances with ELB
In a lot of cases Elastic Load Balancer (ELB) is used. Instances are registered and de-registered to the ELB based on health check status. Problem with Spot instances is the instance do not de-register automatically so there may be some interruption if the situation is not handled properly.
The proper way would be to use the interruption notice as a trigger to de-register the instance from the ELB. By programmatically de-registering the Spot instance prior to termination traffic would not be routed to the instance and no traffic would be lost.
Easiest way is to use a Lambda function to trigger based on a Cloudwatch instance termination notice. The Lambda function simply retrieves the instance ID from the event and de-registers the instance from the ELB. As usual, Amazon Solution Architects showed how to do it on the AWS Compute Blog.
Keep your options open
The Spot capacity pool consists of a set of unused EC2 instances with the same instance type (t3.micro, m4.large, etc) and Availability Zone (us-west-1a). Avoid getting too specific on instance types and what zone they use. For instance, avoid specifically requesting c4.large if running the workload on a m5, c5, or m4 family would work the same. Keep specific needs in mind, vertically scaled workloads need more resources and horizontally scaled workloads would find more availability in older generation types as they are in less demand.
Amazon recommends being flexible across at least 10 instance types and there is never a need to limit Availability Zones. Ensure all AZs are enabled in your VPC for your instance to use.
Price and capacity optimized strategy
Take advantage of Auto Scaling groups as the allocation strategies will enable provisioning capacity automatically. The price-capacity-optimized strategy in Spot Fleet due to how the instance capacity is sourced from pools with optimal capacity. This strategy will reduce the possibility of having the Spot instance reclaimed. Dig into the Auto Scaling User Guide Spot Instances section for more detail. Also take a look at this section which describes when workloads have a high cost of interruption.
Think aggregate capacity
Instead of looking at individual instances, Spot enables a more holistic view across units such as vCPUs, network, memory, or storage. Using Spot Fleet with Auto Scaling Groups allows for a higher level view enabling the concept of “target capacity”. Automating the request for more resources to maintain the target capacity of a workload enables considerable flexibility.
Other options to consider
Amazon has a considerable number of services which can be integrated with Spot instances to manage compute costs. Used effectively these services will allow for more flexibility and automation eliminating the need to manage individual instances or fleets. Take a look at the EC2 Spot Workshops for some ideas and examples.
As an “operator” working in a DevOps role good tools are a necessity. Tools which allow for automating most everything is crucial to keeping up with the vast amount of changes and updates created in a Agile development environment.
Using the same tools your counterparts on the team use will allow for expediting the learning process. In a lot of cases developers use a IDE (Integrated Development Environment) of some sort. Visual Studio Code comes to the forefront, but some ‘hardcore’ or ‘old school’ developers still use Emacs or even Vim as their development tool of choice. There are many out there and each has its pros and cons. Along with a IDE there will be the need for extensions to make things simpler. Let’s outline a few and focus on Visual Studio Code as the tool of choice.
Visual Studio Code is available for most of the commonly used platforms. It has a ton of extensions, but as a “DevOps Engineer” you’ll need a few to make your life easier. First and foremost you’ll want extensions to make working with your favorite cloud provider easier. There are plugins for AWS, GKE, and AKS as well as plugins for yaml, Kubernetes, and Github.
Another extension necessary for container development is the Remote Development Extension Pack. This extension provides the Dev Containers extension allowing for the opening of files and folders inside a container. It also provides a SSH extension to simplify access to remote machines. The Dev Containers extension will want to use Docker Desktop, but a better alternative is Rancher Desktop.
Rancher Desktop is another superb tool for several reasons.
100% open source
Includes K3s as the Kubernetes distribution
Can use with dockerd (moby) or containerd
Basic dashboard
Easy to use
To get started with it, download Rancher Desktop and install on your favorite platform. Follow the installation instructions and once installed go to the preferences page and select “dockerd (moby)” as shown below.
Rancher Desktop Kubernetes Settings
Now that you have Rancher Desktop installed as well as Visual Studio Code with all of the extensions take some time to get familiar with it. Best to start with your github account and create or fork a repository to work with inside Visual Studio Code. Reading through the various getting started docs yields hours of things to try or work with to learn.
To get started with your Rancher Desktop cluster simply click on the Rancher Desktop icon. In most windowed environments there’s a icon in the “task bar”.
Click on the Dashboard link to get access to view the K3s cluster installed when Rancher Desktop started.
Another way to access the cluster is to use kubectl. A number of utilities were installed to ~/.rd/bin. Use kubectl get nodes to view the node(s) in your cluster or use kubectl get pods -A to view all of the pods in the cluster.
Many utilities exist to view/manage Kubernetes clusters. Great learning experiences come from experimentation.
A lot was accomplished in this post. From a bit of reading to manipulating a Kubernetes cluster there is a lot of information to absorb. Visual Studio Code will be the foundation for a lot of the work done in the DevOps world. Containers and Kubernetes will be the foundation for the execution of the work created. This post provided the building blocks to combine the Dev and the Ops with what’s needed to automate the process.
New to the devops scene? Want to get started with a career supporting application operations, managing Kubernetes, running docker, or just browsing around. This site is going to be designed to provide some interesting anecdotes, entertaining articles, and how-tos for getting started with a career in the field of “devops”.
Ah yes…what is this “devops”. Everyone has an opinion for sure. Some call it a “paradigm“. That word has negative connotations though. Some actually feel it is a “engineering position”. Ok. Acceptable. Others just call it what it is…an operator who supports the development efforts of an enterprise. Dev-Ops. No matter, the idea here is to provide info for every opinion.
Want to know more about a topic whether it is Kubernetes, Application Development, Devops, or other enterprise datacenter topic? Speak up. Comments are welcome.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
