Let's Talk DevOps

Real-World DevOps, Real Solutions

Tag: Platform Engineer

  • What’s missing in Kubernetes

    Kubernetes is an open-source container orchestration system that automates the deployment, scaling, and management of containerized applications. It is widely used for its ability to manage containers at scale and is the de facto standard for container orchestration. However, despite its broad adoption, there are still a few missing pieces that need to be addressed to make it fully functional.

    Network Setup

    One of the main missing pieces in Kubernetes is a proper network setup. Kubernetes allows for the creation of multiple clusters, each with its own set of nodes, but it requires a well-defined network setup to manage communication between these clusters.

    Without proper network setup, nodes in the same cluster may not be able to communicate with each other, or there may be issues with cross-cluster communication. This can result in application downtime, loss of data, and other issues that can impact business operations.

    One solution to this problem is to use a software-defined networking (SDN) approach that allows for the creation of a virtual network infrastructure. An SDN controller can be used to manage the virtual network infrastructure and provide network services such as load balancing, routing, and security. With SDN, Kubernetes clusters can be properly connected, and communication between clusters can be streamlined.

    Security

    Another missing piece in Kubernetes is security. Kubernetes provides some basic security features such as role-based access control (RBAC) and network policies, but these are not always enough to secure the entire system.

    Security is a critical aspect of any container orchestration system, and Kubernetes is no exception. Kubernetes clusters are complex systems with many components, and securing them requires a multi-layered approach.

    To enhance security, Kubernetes clusters should be set up with secure communication channels and encrypted data storage. Additionally, it is important to create and enforce security policies that prevent unauthorized access to the system. This includes implementing identity and access management (IAM) policies, network segmentation, and regular vulnerability scanning.

    Monitoring and Logging

    Kubernetes also lacks inbuilt monitoring and logging capabilities. While Kubernetes includes some basic monitoring features, such as health checks and resource usage metrics, it does not provide comprehensive monitoring and logging capabilities.

    In a production environment, it is essential to have comprehensive monitoring and logging capabilities to ensure the health and availability of the system. Kubernetes clusters should be set up with a logging and monitoring stack that can collect and analyze logs and metrics from all nodes in the cluster. This can provide insights into the health and performance of the system, as well as help identify and troubleshoot issues.

    Conclusion

    Kubernetes is a powerful container orchestration system, but there are still a few missing pieces that need to be addressed to make it fully functional. A well-defined network setup, enhanced security, and proper monitoring and logging are all essential components of a fully functional Kubernetes environment.

    With the increasing adoption of containers and cloud-native applications, Kubernetes is becoming more important than ever. As organizations continue to adopt Kubernetes, it is important to ensure that the missing pieces are addressed to provide a reliable and scalable platform for containerized applications. By addressing these missing pieces, Kubernetes can continue to evolve and improve, providing a robust and secure platform for developers and IT teams.

  • Securing cloud native containers

    Securing cloud native containers

    Security, in and of itself, is a broad topic. Container security adds yet another facet to the already nebulous subject of security. In a lot of enterprises today security is first and foremost and the process for securing applications continues to shift left, meaning security is moving to be integrated earlier into the development process. This post will focus on some of the high level tasks and automations developers and operators can implement to mitigate risk.

    The issues.

    Misconfiguration

    The #1 security risk in any cloud native environment is misconfiguration. How do operators know if what they are deploying is secured properly? In a lot of cases, deployments are left insecure for long periods of time without anyone noticing. This is a massive problem, especially for new technologies such as Kubernetes.

    Software Defects

    Another security risk is software bugs. Every day new vulnerabilities are found in software. Some of the vulnerabilities are minor, but increasingly the discoveries constitute a potentially critical issue when deploying software to a public facing system. Vulnerabilities are “what is known”. There is a signature for each of the known vulnerabilities which can be used to scan software.

    However, “you don’t know what you don’t know”. Keep in mind many defects exist which are not known. These are the zero-day vulnerabilities.

    Defense-in-depth

    Scanning

    Scanning software for known vulnerabilities is an absolute must have in any defense-in-depth strategy. However, even the best scanning tools have unknown vulnerabilities (or known limitations). The best defense is offense so creating a system where your container images go through multiple scanners is always a good strategy.

    It is important to scan at many different points in the development process and also continually when in production. Any change could potentially be a breach. It is also very important to have layers which would support the other layers if a layer is permeable. Impervious security requires layers and your goal as a security architect is to create impervious security. Read on for other layers.

    Network visualization

    “It starts and ends with the network”. Kubernetes, being the orchestrator of choice for cloud native deployments, attempts to keep things simple which has lead to a number of CNIs (container network interface) to give platform engineering many choices when deploying workloads. Having something to visualize the network is important, especially when you can act upon those connections. NeuVector provides these capabilities. Being able to quarantine a pod or take a packet capture is key to ensuring continuous protection against unknown attacks and for any required forensics.

    Data protection

    Many different regulations apply to data for enterprises. Being able to provide audit reports for specific data regulations is massively important for HIPAA or PCI DSS. Being able to provide reporting for SOC2 compliance may be important. If your tool cannot “see” into the packet before it traverses the kernel, then it cannot prevent data from crossing a “domain” or prevent sensitive information from being leaked.

    WAF

    A lot of cloud native security tools have the ability to block layer 3 packets. Very few have true layer 7 capabilities. Being able to manage traffic at layer 7 is critical to anyone who is running applications on Kubernetes. Layer 7 is where a lot of unknown vulnerabilities are stopped, but only if the tool can look into the application packet before it traverses the kernel. This is the important point. Once the packet crosses the kernel you are compromised. Use a tool which will learn your workloads behavior. This behavior is the workloads signature and would be the ONLY traffic allowed to traverse the network.

    Wrapping it up

    Security is the highest scoring word in buzzword bingo these days. Everyone wants to ensure their environments are secure and it takes specialized tools for specific platforms. Don’t use the perimeter firewall as a Kubernetes firewall…it simply will not suffice for complete security inside a Kubernetes cluster. Use a tool which can watch every packet and the data inside every packet ensuring only packets with your workloads signature and nothing else traverses the network. One that allows for visualization of the network along with the traditional scanning, admission control, and runtime security requirements every cloud native implementation requires.

  • Knowledge spew on GitOps

    In working with a handful of customers the concept of GitOps continues to resonate more and more. Let us dive into a brain dump of some of the conversations related to GitOps and how these customers tackled the task at hand.

    First thing to remember is these customers are not massive. They are rather common actually. A Gartner defined “medium-sized” enterprise. Keep in mind these customers have the same issues as the giant enterprises just at a different scale.

    In every case there was a user story. At a high level, a common theme was the need to roll out updates to a specific application regularly enough to find ways to entice the consumer to purchase a widget of some sort. Ok, A/B testing. Simple enough.

    Each of the customers were in different maturity levels when it came to development processes, kubernetes knowledge, and devops methods. However they all have one thing in common…the need to deliver an application to their customer base on a deadline and continuously improve the application based on user feedback. All three of them were successful in meeting their self imposed deadlines. How?

    Simple. Every one of them came together, ironed out a plan, and implemented the plan. The interesting part, every one of them already knew how to get the product to market. All they needed was a bit of guidance on how to overcome obstacles and get shit done. How?

    • Step one. Define the top of the mountain, the finish line, the end result.
    • Step two. The project leads built out a high level timeline from end to beginning.
    • Step three. All of the team members came together to build out the task teams.
    • Step four. Each of the teams built out their respective timeline for contribution.
    • Step five. Build.

    Now how does this relate to GitOps? GitOps was the pivotal methodology to get it done. The pipeline was built with all of the parts in mind. If you recall the DevOps “infinity loop“, the key is to use that and combine it with the OODA loop decision model. The combination creates a very powerful decision making framework facilitating agile development with constant improvement. Sound simple? It’s not. It is in theory, but the implementation is like a relationship. Everything is great when dating, but the hard work is when dating turns to marriage. Same goes for creating a product. Designing the product, what it needs to do, all of the moving parts is fun. The real work comes in when the first working build is complete.

    This is where GitOps shines. The developers build things, test locally, and commit. The pipelines move it through the process and all of the other teams contribute to each part in this machine. If one part breaks down, the other work stops to crowdsource the problem. The problem is fixed and the machine continues on. GitOps is the magical fairy dust. What about the technology?

    The technology is rather mundane actually. Git. A code repository. A CI/CD pipeline. A build system. A test harness. A deployment platform. Git…the tools of choice are Github or Gitlab. Github is pretty slick, but Gitlab will allow for running locally in small environments building closed source deliverables. Each has a pipeline mechanism or there are many other tools such as Texton, Argo, CircleCI and many others with various features depending on what was needed. For build systems, many exist and again each as features as needed. However the deployment platform consistently remains the same, Kubernetes.

    Building deploy-able applications at scale is hard. There are many other moving parts, processes, tools, etc. in play. However one thing stands out in all of these engagements…give the right people who have the will to succeed the skills needed to succeed and the execution part will look easy.

    It is always fun to be a part of something, but its most precious reward is being able to step away and watch the machine run on it’s own.

    That’s the end of this spew. It went everywhere…maybe it’s more like a sneeze.

    Peace out.

  • Becoming Successful in DevOps: A Beginners Guide

    Where to start

    As a platform engineer, you will be tasked with ensuring that software projects run smoothly and efficiently. This includes managing all of the various components needed to develop and launch an app or system.

    This may include choosing your development platform, setting up servers for hosting, finding vendors for integrations, and more. Beyond just technology, there are also time-management strategies to manage your day.

    As you can imagine, being a part of this team comes with its own set of challenges and responsibilities. Being a senior member of the team takes some work — but it is totally worth it!

    Being a senior developer means that you know what’s going on technologically, and you’re able to teach others that knowledge. You’ll have the opportunity to make an impact by helping other engineers get things done effectively.

    It’s also important to remember that not everyone has a career in tech like you do, so don’t feel like you need to take over every task.

    Plan your career

    7 things to know as a devops engineer

    As mentioned earlier, being a successful platform engineer is more than just knowing how to work in different environments having separate and distinctly different conversations with various people about what tools you use for each environment, and how to manage your time effectively between projects and departments. It also means planning your career ahead of time.

    Most experienced devs will tell you that it’s impossible to become a senior developer without first becoming a team lead or manager. This makes sense, because you need to establish relationships within the department and organization that depend on you for leadership, guidance, and resources.

    And since most tech companies are moving towards agile project management systems like Scrum, you need to be able to facilitate those meetings, talk about logistics, and make sure everyone has everything they need to complete their assignments on schedule.

    All of these things require someone who leads by example, sets appropriate priorities, and keeps people motivated so that they feel comfortable coming to you for help when needed. You can’t expect anyone lower up the chain-of-command to do this if they don’t see you putting in the effort into developing your leadership skills.

    It’s totally normal to want to move up the ladder, but making changes to your position requires proving that you’re capable of doing so while still keeping quality control over your tasks and setting realistic deadlines.

    Create a good work-life balance

    7 things to know as a devops engineer

    Being a platform engineer means having close relationships with other people in your organization that depend on you for success, so make sure you are not overworking yourself.

    It is important to enjoy what you do for a living, and staying motivated requires finding time to unwind and relax. You can’t expect to be passionate about your job if you don’t take care of yourself outside of work.

    You need to recognize that it takes a lot out of you, both physically and mentally, and give yourself breaks to keep up momentum. Take frequent vacations, organize social events outside of work, and try to limit how many hours you spend working every day.

    In addition to taking care of yourself, use the same energy you put into your career on your personal life to promote healthy relationships and friendships.

    Having friends who treat you well will help you feel more balanced and confident in your position, and they will help you stay happier at home. Try to meet someone new once a week, if possible, to maintain strong ties with family members and colleagues.

    Be a creative thinker

    7 things to know as a devops engineer

    As a platform engineer, you will be tasked with many different projects and responsibilities. This can make it hard to know what your next job is! As such, being able to think outside of the box and brainstorm new ideas and solutions to current issues is important.

    You will have to use your creativity to figure out how to best solve these problems.

    A great way to hone this skill is by taking part in various tech events or masterclasses. You may even get chance to present your own project or idea!

    By attending events like these, not only do they give you knowledge about the field, but also lots of opportunities to improve yourself professionally.

    Stay up-to-date with technology

    7 things to know as a devops engineer

    As a platform engineer, you will be tasked with keeping up with new technologies. This is not only important for your career, but also to satisfy users’ demands for quality services they can rely on.

    As more companies implement automation via software or robotics, there are always new tools being used for this process. You will need to know what these tools are and how to use them.

    You will also have to stay up-to-date with the ever-evolving practices in both development and operations. For example, when developers adopt newer agile methodologies like scrum, you must understand why that approach is better than the traditional waterfall model.

    Likewise, understanding lean startup methodology means knowing why it works and if it applies to your organization. These concepts help bring down the overhead of launching an app by reducing the number of steps needed to get things going.

    By staying informed about such developments, you will make sure apps launch as quickly as possible while still meeting standards.

    Be a good leader

    7 things to know as a devops engineer

    As a platform engineer, you will be in charge of many different departments within your organization. You can expect that not every department in your own company or others you work with have strong leadership qualities.

    If this is something that worries you, don’t worry! It’s totally normal.

    It takes time to develop leadership skills, but you are never too young or old to learn them.

    You can always look to those around you for examples of how to lead and learn from their mistakes. There are several books and courses available online and through educational institutions that focus on developing leadership abilities.

    There are also mentorship programs where experienced leaders share knowledge and resources with younger people. By being part of these groups, you could gain some valuable lessons yourself.

    Be a good team member

    7 things to know as a devops engineer

    As a platform engineer, your success will depend on how well you collaborate with others. You can’t have successful collaborations if one party is constantly trying to get their way or take over the process.

    This isn’t helpful for anyone involved in the project, and it won’t set well with leadership either. Make every effort to work with people around you using sound reasoning and evidence, and keep discussions focused on topics that are productive and positive.

    Avoid gossip unless you want to deal with a lot of resentment later, and be willing to go beyond what is practical or appropriate to help someone feel better about themselves or the situation.

  • Creating a pipeline in Gitlab

    Creating a pipeline in Gitlab is a great way to deploy applications to various environments. It allows you to automate the process of building, testing, and deploying your application. With Gitlab, you can define a pipeline that consists of a series of stages, which can include tasks such as building the application, running tests, and deploying the application. By leveraging the power of a pipeline, you can ensure that your application is deployed quickly, efficiently, and with minimal hassle. This guide will walk you through the steps to create a pipeline in Gitlab, with code examples.

    Step 1: Creating a New Project

    To start a new pipeline, you will first need to create a new project in Gitlab. To do this, you can go to Projects > New Project and fill out the form.

    Step 2: Creating the Pipeline

    Once the project is created, you can go to CI/CD > Pipelines and create a new pipeline. You will be able to define the stages and tasks for the pipeline, and even add code snippets to configure the pipeline. Here is an example of a simple pipeline:

    stages:
  - build
  - test
  - deploy

build:
  stage: build
  script:
    - npm install
    - npm build

test:
  stage: test
  script:
    - npm test

deploy:
  stage: deploy
  script:
    - npm deploy

    Step 3: Running the Pipeline

    Once you have created the pipeline, you can run it by going to CI/CD > Pipelines and clicking the “Run Pipeline” button. This will execute the pipeline, and you can check the progress of the pipeline in the pipeline view.

    Step 4: Troubleshooting

    If you encounter any issues while running the pipeline, you can click on the pipeline to open up the pipeline view. Here, you will be able to view the log output of each stage, and debug any issues that may be occurring.

    Conclusion

    Creating a pipeline in Gitlab is a great way to easily deploy applications to various environments. By leveraging the power of a pipeline, you can ensure that your application is deployed quickly, efficiently, and with minimal hassle. This guide has walked you through the steps to create a pipeline in Gitlab, with code examples.