CDH Issues

Job History Server does not enforce ACLs when web authentication is enabled

The Job History Server does not enforce job ACLs when web authentication is enabled. This means that any user can see details of all jobs. This only affects users who are using MRv2/YARN with HTTP authentication enabled.

Products affected:
  • Hadoop
Releases affected:
  • All versions of CDH 4.5.x up to 4.5.0
  • All versions of CDH 4.4.x up to 4.4.0
  • All versions of CDH 4.3.x up to 4.3.1
  • All versions of CDH 4.2.x up to 4.2.2
  • All versions of CDH 4.1.x up to 4.1.5
  • All versions of CDH 4.0.x
  • CDH 5.0.0 Beta 1
Users affected:
  • Users of YARN who have web authentication enabled.

Date/time of detection: October 14, 2013

Severity: Low
  Note: YARN is an experimental feature in CDH 4; it is no longer experimental in CDH 5.

Impact: Low

CVE: CVE-2013-6446

Immediate action required:
  • None, if you are not using MRv2/YARN with HTTP authentication.
  • If you are using MRv2/YARN with HTTP authentication, upgrade to CDH 4.6.0 or CDH 5.0.0 Beta 2 or contact Cloudera for a patch.

ETA for resolution: Fixed in CDH 5.0.0 Beta 2 released on 2/10/2014 and CDH 4.6.0 released on 2/27/2014.

Addressed in release/refresh/patch: CDH 4.6.0 and CDH 5.0.0 Beta 2.

Verification:

This vulnerability affects the Job History Server Web Services; it does not affect the Job History Server Web UI.

  Important:

The vulnerability is exposed only when the Job History Server HTTP endpoint is configured with an authentication filter (such as Hadoop's built-in AuthenticationFilter or a custom filter) that populates the HttpServletRequest.getRemoteUser() that is propagated to the Job History Server. This configuration is independent of the Hadoop cluster being configured with Kerberos security.

To verify that the vulnerability has been fixed, do the following steps:
  1. Create two non-admin users: 'A' and 'B'
  2. Submit a MapReduce job as user 'A'. For example:
    $ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi 2 2
  3. From the output of the above submission, copy the job ID, for example: job_1389847214537_0001
  4. With a browser logged in to the Job History Server Web UI as user 'B', access the following URL:

    http://<JHS_HOST>:19888/ws/v1/history/mapreduce/jobs/job_1389847214537_0001

If the vulnerability has been fixed, you should get an HTTP UNAUTHORIZED response; if the vulnerability has not been fixed, you should get an XML output with basic information about the job.

Apache Hadoop and Apache HBase "Man-in-the-Middle" Vulnerability

The Apache Hadoop and HBase RPC protocols are intended to provide bi-directional authentication between clients and servers. However, a malicious server or network attacker can unilaterally disable these authentication checks. This allows for potential reduction in the configured quality of protection of the RPC traffic, and privilege escalation if authentication credentials are passed over RPC.

Products affected:
  • Hadoop
  • HBase
Releases affected:
  • All versions of CDH 4.3.x prior to 4.3.1
  • All versions of CDH 4.2.x prior to 4.2.2
  • All versions of CDH 4.1.x prior to 4.1.5
  • All versions of CDH 4.0.x
Users affected:
  • Users of HDFS who have enabled Hadoop Kerberos security features and HDFS data encryption features.
  • Users of MapReduce or YARN who have enabled Hadoop Kerberos security features.
  • Users of HBase who have enabled HBase Kerberos security features and who run HBase co-located on a cluster with MapReduce or YARN.

Date/time of detection: June 10th, 2013

Severity: Severe

Impact:

RPC traffic from Hadoop clients, potentially including authentication credentials, may be intercepted by any user who can submit jobs to Hadoop. RPC traffic from HBase clients to Region Servers may be intercepted by any user who can submit jobs to Hadoop.

CVE: CVE-2013-2192 (Hadoop) and CVE-2013-2193 (HBase)

Immediate action required:
  • Users of CDH 4.3.0 should immediately upgrade to CDH 4.3.1 or later.

  • Users of CDH 4.2.x should immediately upgrade to CDH 4.2.2 or later.

  • Users of CDH 4.1.x should immediately upgrade to CDH 4.1.5 or later.

ETA for resolution: August 23, 2013

Addressed in release/refresh/patch: CDH 4.1.5, CDH 4.2.2, and CDH 4.3.1.

Verification:

In order to verify that you are not affected by this vulnerability, you should ensure that you are running a version of CDH at or greater than the aforementioned versions. To verify that this is true, proceed as follows.

On all of the cluster nodes, run one of the following commands:
  • On RPM-based systems (RHEL, SLES) rpm -qi hadoop | grep -i version
  • On Debian-based systems
    dpkg -s hadoop | grep -i version
Make sure that the version listed is greater than or equal to 4.1.5 for the 4.1.x line, 4.2.2 for the 4.2.x line, and 4.3.1 for the 4.3.x line. The 4.4.x and later lines of releases are unaffected.

Several types of authentication tokens use a secret key of insufficient length

Products Affected: HDFS, MapReduce, YARN, Hive, HBase

Releases Affected: If you use MapReduce, HDFS, HBase, or YARN, CDH4.0.x and all CDH3 versions between CDH3 Beta 3 and CDH3u5 refresh 1.

Users Affected: Users who have enabled Hadoop Kerberos security features.

Date/Time of Announcement: 10/12/2012 2:00pm PDT (upstream)

Verification: Verified upstream

Severity: High

Impact: Malicious users may crack the secret keys used to sign security tokens, granting access to modify data stored in HDFS, HBase, or Hive without authorization. HDFS Transport Encryption may also be brute-forced.

Mechanism: This vulnerability impacts a piece of security infrastructure in Hadoop Common, which affects the security of authentication tokens used by HDFS, MapReduce, YARN, HBase, and Hive.

Several components in Hadoop issue authentication tokens to clients in order to authenticate and authorize later access to a secured resource. These tokens consist of an identifier and a signature generated using the well-known HMAC scheme. The HMAC algorithm is based on a secret key shared between multiple server-side components.

For example, the HDFS NameNode issues block access tokens, which authorize a client to access a particular block with either read or write access. These tokens are then verified using a rotating secret key, which is shared between the NameNode and DataNodes. Similarly, MapReduce issues job-specific tokens, which allow reducer tasks to retrieve map output. HBase similarly issues authentication tokens to MapReduce tasks, allowing those tasks to access HBase data. Hive uses the same token scheme to authenticate access from MapReduce tasks to the Hive metastore.

The HMAC scheme relies on a shared secret key unknown to the client. In currently released versions of Hadoop, this key was created with an insufficient length (20 bits), which allows an attacker to obtain the secret key by brute force. This may allow an attacker to perform several actions without authorization, including accessing other users' data.

Immediate action required: If Security is enabled, upgrade to the latest CDH release.

ETA for resolution: As of 10/12/2012, this is patched in CDH4.1.0 and CDH3u5 refresh 2. Both are available now.

Addressed in release/refresh/patch: CDH4.1.0 and CDH3u5 refresh 2

Details: CDH Downloads

DataNode Client Authentication Disabled After NameNode Restart or HA Enable

Products affected: HDFS

Releases affected: CDH 4.0.0

Users affected: Users of HDFS who have enabled HDFS Kerberos security features.

Date vulnerability discovered: June 26, 2012

Date vulnerability analysis and validation complete: June 29, 2012

Severity: Severe

Impact: Malicious clients may gain write access to data for which they have read-only permission, or gain read access to any data blocks whose IDs they can determine.

Mechanism: When Hadoop security features are enabled, clients authenticate to DataNodes using BlockTokens issued by the NameNode to the client. The DataNodes are able to verify the validity of a BlockToken, and will reject BlockTokens that were not issued by the NameNode. The DataNode determines whether or not it should check for BlockTokens when it registers with the NameNode.

Due to a bug in the DataNode/NameNode registration process, a DataNode which registers more than once for the same block pool will conclude that it thereafter no longer needs to check for BlockTokens sent by clients. That is, the client will continue to send BlockTokens as part of its communication with DataNodes, but the DataNodes will not check the validity of the tokens. A DataNode will register more than once for the same block pool whenever the NameNode restarts, or when HA is enabled.

Immediate action required:

  1. Understand the vulnerability introduced by restarting the NameNode, or enabling HA.
  2. Upgrade to CDH 4.0.1 as soon as it becomes available.

Resolution: July 6, 2012

Addressed in release/refresh/patch: CDH 4.0.1 This release addresses the vulnerability identified by CVE-2012-3376.

Verification: On the NameNode run one of the following:

  • yum list hadoop-hdfs-namenode on RPM-based systems
  • dpkg -l | hadoop-hdfs-namenode on Debian-based systems
  • zypper info hadoop-hdfs-namenode for SLES11

On all DataNodes run one of the following:

  • yum list hadoop-hdfs-datanode on RPM-based systems
  • dpkg -l | grep hadoop-hdfs-datanode on Debian-base
  • zypper info hadoop-hdfs-datanode for SLES11

The reported version should be >= 2.0.0+91-1.cdh4.0.1

MapReduce with Security

Products affected: MapReduce

Releases affected: Hadoop 1.0.1 and below, Hadoop 0.23, CDH3u0-CDH3u2, CDH3u3 containing the hadoop-0.20-sbin package, version 0.20.2+923.195 and below.

Users affected: Users who have enabled Hadoop Kerberos/MapReduce security features.

Severity: Critical

Impact: Vulnerability allows an authenticated malicious user to impersonate any other user on the cluster.

Immediate action required: Upgrade the hadoop-0.20-sbin package to version to 0.20.2+923.197 or higher on all TaskTrackers to address the vulnerability. Note that upgrading hadoop-0.20-sbin will cause upgrade of several related (but unchanged) hadoop packages. If using Cloudera Manager versions 3.7.3 and below, you will also need to upgrade to Cloudera Manager 3.7.4 or later before you can successfully run jobs with Kerberos enabled after upgrading the hadoop-0.20-sbin package.

Resolution: 3/21/2012

Addressed in release/refresh/patch: hadoop-0.20-sbin package, version 0.20.2+923.197 This release addresses the vulnerability identified by CVE-2012-1574.

Remediation verification: On all TaskTrackers run one of the following:

  • yum list hadoop-0.20-sbin on RPM-based systems
  • dpkg -l | grep hadoop-0.20-sbin on Debian-based systems
  • zypper info hadoop-0.20-sbin for SLES11

The reported version should be >= 0.20.2+923.197.

If you are a Cloudera Enterprise customer and have further questions or need assistance, log a ticket with Cloudera Support through http://support.cloudera.com.