return
In the past decade, there has been a lot of discussion about DevSecOps methodologies and how these automated processes can sanitize the entire infrastructure of our applications. This focus has evolve the way we analyze systems. We've moved away from reviewing and auditing applications from the last point (when they are in production) and shifted towards a dynamic and interactive approach that accompanies the classic secure software development lifecycle from the beginning.
It's clear that this change in perspective greatly benefits the way we approach projects and the architecture of applications. As a result, each development becomes increasingly robust in terms of security. Now, security analysts can access information in real-time and can generate early warnings to avoid incurring a higher cost when addressing a vulnerability during the development cycle.
While applications are now more robust, attackers are forced to plan new attack vectors associated with new resources. These resources are tied to the way we automate protection tasks and the architecture we use to design our DevSecOps approach. One of the most critical points here, are the Pipelines.
A Pipeline
A pipeline is a set of processes and tools used to gather raw data from multiple sources, analyze it, and present the results in an understandable format. In a DevSecOps flow, this is crucial as it's tied to the process of analysis and continuous improvement specifically related to the code. It's worth noting that pipelines are not only used for secure development flows but are also employed in various sectors related to data analysis.
Pipelines are powerful and useful tools, so processing or manipulating this information is often of great interest to attackers.
Pipeline CI/CD
(Continuous Integration/Continuous Development)
A CI/CD Pipeline is the fundamental component of automated software development. While the term has been used to describe many different aspects of computing, in much of the DevOps industry, we use "Pipelines" to illustrate the broad applications of behaviors and processes involved in continuous integration (CI).
Within these new functionalities, we find the following potential vulnerabilities that could be currently affecting pipelines that are deployed unattended:
Broken Authentication in CI/CD
Oversights regarding the design of authorization and authentication for CI and CD channels can occur when designing the role and permission scheme. Some of the following examples can be avoided by having the appropriate process that validates all these configurations. Among all the methods, we can find the following:
• Adding Unauthorized Approver using Admin Permissions
• In this scenario, an attacker with administrator permissions on the continuous integration (CI) channel adds an unauthorized user as an approver, possibly bypassing the necessary security checks..
Ej: ci-tool add-approver --pipeline pipeline-name --user unauthorized-user
• Weak Authentication:
• In this scenario, an attacker takes advantage of weak authentication mechanisms to gain administrator access to the continuous integration (CI) tool and manipulate the approver settings, possibly using a brute force attack.
Ej :ci-tool login --username admin --password weakpassword
• Token Manipulation:
Ej: ci-tool --api-token stolen-token add-approver --pipeline pipeline-name --user unauthorized-user
Poor Handling of Secrets
Pipelines constantly connect to resources associated with other applications, services, or specific credentials linked to vital productive resources. Proper secret management is essential to avoid the issues listed below:
• Incorrect Repository Authorization:
• In this scenario, an attacker gains access to a CI/CD pipeline that interacts with a secret manager and exploits weak permissions to retrieve sensitive credentials.
Ej: ci-tool get-secrets --repository malicious-repo --secret-manager secretmanager-nam
• Code Injection::
• The attacker submits manipulated malicious code to a legitimate repository. When executed in the pipeline, it captures secrets or exploits them to perform unauthorized actions.
Ej: echo "echo $SECRET_VARIABLE" >> main.py
• Retrieve Credentials from CI/CD Admin Console::
• Access AWS and secret keys from a CI/CD admin console and then pass them as environment variables:
Ej: export AWS_ACCESS_KEY_ID=$(aws ssm get-parameter --name /path/to/access_key - -with-decryption --query Parameter.Value --output text) export AWS_SECRET_ACCESS_KEY=$(aws ssm get-parameter --name /path/to/secret_key --with-decryption --query Parameter.Value --output text) pipeline-stage-command
• Retrieve a Secret from Hashicorp Vault and Pass it as a Variable to the Pipeline Stage:
Ej: export SECRET=$(vault kv get -format=json secret/path/to/secret | jq -r '.data.key') pipeline-stage-command
• Retrieve a Secret from Kubernetes from the Admin Console and Pass it as a Mounted Volume in the Pipeline Stage:
Ej: kubectl create secret generic my-secret --from-literal=username=<username> -- from-literal=password=<password>
kubectl create volume secret my-secret-volume --secret=my-secret
pipeline-stage-command --volume=my-secret-volume:/secrets
Production Environment Exposed Due to Poor Configuration
The same story that occurs in the applications these pipelines verify. Typically associated with a dragged error or a misguided focus on the sense of security applied to CI/CD pipeline configurations.
• Misconfigured Environment Variables:
• This example comes from identifying poorly configured variables and exploiting these variables to gain unauthorized access to various resources.
Ej: ci-tool get-secrets --repository legitimate-repo --secret-manager $MISCONFIGURED_VARIABLE
• Modification of Unauthorized Configuration:
• In this scenario, the attacker can access the pipeline and modify the production environment configuration.
Ej: # Modify the production environment configuration file echo "MALICIOUS_SETTING = True" >> production.config
• Misconfigured Scripts:
• The attacker can identify a poorly configured script and exploit it to modify the production environment.
Ej: # Exploit misconfigured deployment script to modify production configuration ci-tool run-deployment-script --script "sed -i 's/ALLOW_PUBLIC_ACCESS = False/ALLOW_PUBLIC_ACCESS = True/' production.config"
• Insecure Configuration Manager:
• Below, we can visualize a code example where this flaw can be found. In the displayed example, the deploy function is part of a script. The repo_url parameter is intended to be a valid git repository. However, due to improper input validation, an attacker could inject arbitrary commands by adding them to the repo_url. parameter. In this case, the attacker can add the command (rm -rf /) to delete the entire filesystem.
Code:
Potential Malicious Repository:
• Insecure Credential Storage::
• In the example shown in the capture, the list_buckets function uses the AWS SDK to interact with Amazon S3. The code relies on the SDK's default credential search behavior, which checks various configuration files for AWS credentials. However, the pipeline has a security flaw where the AWS credentials are stored insecurely. An attacker can execute a command (cat ~/.aws/credentials) to retrieve the AWS credentials and potentially gain unauthorized access to AWS resources.
All these points are important to avoid the effect known as "snowball", where the error's propagation occurs throughout all the development chains implemented within our CI/CD. That's why addressing these errors at their root is often very useful to prevent a bigger problem in the future.
Tools That Can Help Us
As we will see next, the use of external components continues to cause headaches if they don't undergo proper validation.
• Trufflehog:
• TruffleHog is a sensitive data scanner for Git repositories. It scans the specified Git repository URL (https://example.com/git-repo.git) using regular expression pattern matching and entropy analysis to identify potentially sensitive information, such as API keys, passwords, and other secrets stored in the repository.
• Circleci:
• This command uses the CircleCI CLI tool to simulate the execution of a CircleCI configuration file (config.yml) locally. It packages the configuration file using "circleci config pack" and then runs the packaged configuration file using "circleci local execute". This can help identify incorrect configurations, vulnerabilities, or insecure practices within the CI/CD pipeline defined in the CircleCI configuration file.
Ex: docker run -v $(pwd):/src -w /src -t circleci/circleci-cli:latest circleci config pack .circleci > config.yml && circleci local execute -c config.yml
• Snyk:
• Snyk is a popular security scanning tool, used to test all projects and subprojects in the current directory. Snyk scans the source code and its dependencies, checking for known vulnerabilities, insecure coding practices, and other security issues. This helps identify and address security weaknesses introduced by insecure coding practices.
Ex: snyk test --all-projects --all-sub-projects
• npm:
• The npm audit command, integrated into the Node Package Manager (npm), conducts a security audit of the dependencies in a Node.js project. This command checks the project's dependencies against the National Vulnerability Database (NVD) and generates a report highlighting any known vulnerabilities and insecure coding practices. This helps identify and address security issues introduced by insecurely implemented or outdated dependencies.
• Análisis de dependencias de terceros:
• OWASP Dependency-Check is a widely used tool to identify known vulnerabilities in project dependencies. It scans the project located at <path-to-project>, analyzes the dependencies, and generates an HTML report (--format HTML) highlighting any insecure dependencies and/or vulnerable components.
Ex: OWASP-Dependency-Check --scan <path-to-project> --format HTML
Conclusion
As we identified in this post, we have reviewed several interesting points when understanding that if we implement automated mechanisms to improve the security of our applications, this does not mean that we should pay less attention to the way we design our automation flow. Attention to security must remain constant in all aspects of our architecture and must be enhanced with the same frequency and attention as all our assets.
