A New Deadly Combination in Nginx

Recently a deadly combination of vulnerabilities emerged, posing a severe threat to Kubernetes clusters utilizing Ingress-Nginx. By exploiting three critical vulnerabilities:

• CVE-2022-4886: the path field sanitization can be bypassed and ingress-nginx  credentials can be obtained
• CVE-2023-5044: Code injection via ‘permanent-redirect’ annotation
• CVE-2023-5043: Ingress nginx annotation injection causes arbitrary command execution

attackers can execute arbitrary code and escalate privileges, all with access to the Nginx Annotation object. These vulnerabilities have been confirmed in both NGINX and Kubernetes/Ingress-Nginx, as reported by Google and various GitHub issues.

Ingress-Nginx Controller: A Cornerstone of Kubernetes Networking

The Ingress-Nginx controller is a component in Kubernetes, leveraging NGINX as a reverse proxy and load balancer. This controller streamlines the management of incoming traffic, directing it to the right services in your cluster. Ingress-Nginx enables secure communication. With the power of NGINX, Ingress-Nginx offers a strong foundation for managing ingress traffic in your Kubernetes setup.

Annotations: Customizing Ingress-Nginx Behavior

Annotations in Kubernetes are digital markers that add metadata to objects. They enhance configuration management and aid in ensuring seamless communication between application components. Annotations in the Ingress-Nginx controller allow you to customize its behavior. These notes, embedded in your configuration, act as guidelines that help the controller manage incoming web traffic and set security rules. While annotations offer clear benefits, they can also be difficult to secure, with misconfigurations posing significant security risks.

Nginx CVE Review

• CVE-2023-5044 and CVE-2023-5043: These vulnerabilities allow attackers to steal sensitive credentials from the cluster by manipulating the Ingress object configuration. By default, these secrets include highly privileged credentials for the Kubernetes API server. The combination of these vulnerabilities enables attackers to inject arbitrary code into the Ingress controller process using the configuration-snippet (CVE-2023-5043) or permanent-redirect (CVE-2023-5044) annotation fields. This injected code grants access to the Ingress controller’s service account token, associated with a ClusterRole allowing access to all Kubernetes secrets within the cluster.
• CVE-2022-4886: This vulnerability exposes a critical flaw in the way Ingress-Nginx handles the path field within Ingress routing definitions. When an attacker manipulates this vulnerable path field, they can redirect requests to an internal file containing the service account token, a client credential used for authentication against the Kubernetes API server.
  • By crafting a malicious path parameter, an attacker can orchestrate a route that points to the internal file housing the service account token.
  • Once the attacker gains control of the Ingress object, they can exploit this misconfiguration to steal the Kubernetes API credentials from the Ingress controller. With these credentials in hand, the attacker gains unauthorized access to all secrets within the cluster.

Upwind’s research team has discovered numerous instances where both NGINX and Ingress-Nginx configurations possessed permissions to extract sensitive secrets, posing a substantial risk to Upwind customers. In response, our research team meticulously analyzed these vulnerabilities and proactively notified Upwind users, ensuring they were informed and equipped to safeguard their systems against potential exploits.

Nginx Attack Path Scenarios

• In multi-tenant clusters, attackers exploit Ingress object vulnerabilities, compromising the entire cluster through escalated privileges.
• Compromised supply chains and tainted configurations allow malicious actors to inject code, endangering users deploying these setups.
• Tainted online examples from untrusted sources create deceptive entry points, necessitating heightened security in Kubernetes environments. Using configuration samples from the web or ChatGPT can expose users to tainted setups, elevating the risk of exploitation.
• Malicious Helm charts in supply chains enable unauthorized access, especially in multi-tenant clusters, where attackers manipulate Ingress objects within their namespaces.
• Insiders with configuration change rights but no direct cluster access pose threats, emphasizing the importance of securing internal permissions effectively.

How To Fix Nginx CVEs

This deadly combination requires immediate action. You can remediate Nginx CVEs by doing the following:

• If you are using the out-of-the-box Kubernetes Ingress-Nginx controller, upgrade to version 1.9 or above to patch all vulnerabilities.
• If you are utilizing Helm charts, update to version 4.8.3 to ensure protection against all three vulnerabilities.
• For users employing pure NGINX, upgrading to version 1.19 or above is essential to safeguard against potential attacks.

Additionally, In the ingress-nginx controller deployment consider using the following flag in the Args section to mitigate vulnerabilities:

apiVersion: v1
kind: Pod
metadata:
  labels:
    ...
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  containers:
  - args:
    - /nginx-ingress-controller
 ...
### Append the flag inside the args section ###
    - -–enable-annotation-validation
### 
...

If your rule includes pathType values Exact or Prefix enable the strict-validate-path-type option from nginx-ingress-controller version 1.18 and above.

Upwind’s Response to Nginx Vulnerabilities

Upon learning about the Nginx vulnerabilities, Upwind’s research team was able to rapidly research and identify where these toxic combinations exist in our customers’ environments and notify them for immediate remediation. 

Going beyond the research team, the Upwind platform also identifies toxic combinations like Nginx using the following:

• Vulnerability funnel: prioritize vulnerabilities by using context such as if they are in-use, actively loaded in memory or exposed to the Internet
• Topology map: automatically see which resources are receiving Internet ingress or egress communication and identify any abnormal resource communication
• Inventory management: easily search for image names or versions to rapidly identify vulnerable images in your environment and perform needed upgrades 

Find Nginx Vulnerabilities 

For further information on Nginx vulnerabilities or for assistance identifying critical vulnerability exposures in your environment within minutes, please ping us at [email protected].