During CVE research at Upwind, we encountered an unfamiliar Apache service known as Apache Pinot. Apache Pinot is a real-time distributed OnLine Analytical Processing (OLAP) datastore specifically designed to deliver low-latency responses to OLAP queries.

pinot_architecture-1024x723

The Apache Pinot architecture is comprised of four key components:

  1. Control Plane Services (Green): These services oversee and manage the functionality of the system. The broker component executes requests, while ZooKeeper and Helix handle cluster management and load balancing.
  2. Local Data Components (Purple): This segment encompasses various stages, including data ingestion, storage and local query processing with minimal runtime.
  3. Remote or Live Stream Data Sources (Blue): Queries directed at the real-time service interact with stream or remote sources integrated with Pinot.
  4. User Interface: Primarily serving as an interface, it enables query execution on port 9000.

Why Use Apache Pinot?

Organizations implement Apache Pinot to enhance their matching and ETA calculation system. Here’s how Pinot can address these challenges:

  1. Real-time Processing: Apache Pinot can ingest and query data in real-time, allowing organizations to receive real-time information.
  2. Scalability: Pinot’s distributed architecture allows organizations to scale horizontally as their user base and requests grow, ensuring they can handle increased load without compromising performance.
  3. Complex Queries: Pinot provides native support for complex queries and aggregations in real-time.

Does Apache Pinot Pose a Security Risk?

As we began observing increased usage of Apache Pinot in various environments, we recognized the importance of conducting a thorough examination of the service. 

Firstly, it is crucial to acknowledge that exposing an interface that permits external users to query internal data sources poses significant risks for a company. Typically, Pinot services are intended to serve as a back-office tool, granting access to data solely for company employees. Most companies opt for a customized user interface instead of exposing their data through a back-office service.

pinot-detection-1024x419

Secondly, the Pinot service offers a wide range of functionalities, including the ability to run Groovy functions and establish connections with internal data sources. However, it is essential to acknowledge that such extensive capabilities also present potential vulnerabilities. An attacker could exploit the exposed interface allowing him to manipulate production data leading the company to false insights, pull sensitive data out of the data source such as monitoring data or even customer data, and execute malicious code.

To assess the security posture of Pinot, we utilized Upwind’s vulnerability scanning service to examine the most popular Pinot image and we found a significant number of vulnerabilities. 

pinot-detection-list-1024x616

Apache Pinot Vulnerability Finding

Upwind’s vulnerability management tool discovered a wide range of CVE types, including Remote Code Execution (RCE), Denial of Service (DoS) and Data Exfiltration vulnerabilities within Pinot.

It is crucial to recognize that an attacker who gains access to the Pinot server can fully exploit its capabilities. If there is a misconfiguration or elevated permission assigned to the service, the attacker can assume control over it.

Imagine a scenario where an individual engages in a casual conversation with a friend on Discord, unknowingly becoming the unwitting victim of a cyberattack. This nefarious acquaintance successfully deploys a standard trojan, stealthily breaching security measures, and ultimately gaining access to the Pinot server. 

The attacker can leverage this cheat sheet or exploit any of the 900 known CVEs associated with the Pinot service. Since Pinot is connected to multiple data sources within the organization, each component or consumer becomes a potential avenue for attacking the service.

However, it is important to note that an internally-used Pinot service, exclusively exposed to company assets, poses a lesser but not negligible risk. 

 Should You Stop Using Apache Pinot?

Upwind’s approach prioritizes affirmative security, but places a strong emphasis on proceeding with caution and implementing necessary precautions – the Upwind way!

Our approach is always affirmative, albeit with a cautious perspective. We recommend employing Pinot in an “Upwind way,” which entails adopting stringent security measures, conducting regular vulnerability assessments and implementing proper configurations to mitigate potential risks. While Pinot presents certain security challenges, its benefits in terms of real-time distributed OLAP make it a valuable tool when approached with appropriate security measures in place.

How to Deploy Apache Pinot Securely

Deploying Apache Pinot securely is paramount in safeguarding your data and systems. Here’s how you can fortify your Pinot deployment:

  • Regularly upgrade versions: Staying up-to-date with the latest Apache Pinot version is your first line of defense. Regular upgrades ensure that you benefit from security patches and enhancements. Alternatively, opt for a secure base image if you’re deploying Pinot without a container image. This foundational step establishes a robust security posture for your deployment.
  • Disable unnecessary Groovy functionality: Apache Pinot’s flexibility can come at a cost. Disable unnecessary Groovy functionality to eliminate potential vulnerabilities. In order to do so, you must edit the Pinot server, broker and controller configuration files.
  1. Cluster:
# Run this command to limit access to internal components
curl --location 'http://<controller>:<port>/cluster/configs' \
--header 'Content-Type: text/plain' \
--data '{"queryConsoleOnlyView":"true"}'

Copied

2. Controller:

#Open the controller configuration file for editing by running the following command:
vim $(find / -name "pinot-controller.conf" 2>/dev/null | head -n 1)

#Disable Groovy inside the controller
controller.disable.ingestion.groovy=false

Copied

3. Broker:

#Open the broker configuration file for editing by running the following command:
vim $(find / -name "pinot-server.conf" 2>/dev/null | head -n 1)

#Disable Groovy inside the broker
pinot.broker.disable.query.groovy=true

Copied

  • Save the changes: Restart the Pinot server to apply the modifications. This step ensures that your Pinot deployment operates without any Groovy-related risks and won’t expose any unnecessary components.
  • Secure exposure:
  • Minimize External Communication: Guarantee that your data sources and connectors employ secure communication protocols. Secure channels ensure that data exchanges are encrypted and tamper-proof, thwarting eavesdropping attempts and preserving the confidentiality and integrity of your data.
image-1-1024x598

In short, while organizations certainly can continue to use Apache Pinot, there are a number of security concerns that they should take into consideration. Pinot contains a number of vulnerabilities that can be exploited, exposing organizational data. In order to safely use Pinot, organizations should only use the service internally, along with regularly upgrading versions, disabling unnecessary Groovy functionality and securing exposure. 

Organizations should conduct regular vulnerability assessments using runtime data, in order to assess exploitable CVEs and determine the real risk that Apache Pinot vulnerabilities pose to their organization. Solutions like Upwind’s Vulnerability Management are a necessary first step in order to identify vulnerabilities, prioritize remediation and determine if Pinot poses a real risk to your organization.