New Features and Changes in Cloudera Manager 5
The following sections describe what’s new and changed in each Cloudera Manager 5 release.
What's New in Cloudera Manager 5
What's New in Cloudera Manager 5.0.0 Beta 2
- Service and Configuration Management
- HDFS NFS Gateway role
- Supports restoration of HDFS data from a snapshot
- YARN Resource Manager High Availability
- Resource pool scheduler
- Support for Spark service
- Support for service extensibility
- Support to set up Oozie server High Availability
- Granular configuration staleness UI
- Support for setting maximum file descriptors
- Support for monitoring the Cloudera Search/Solr service
- New "failed" and "killed" badges displayed for unsuccessful YARN applications
- More attributes available for filtering displays of YARN applications and Impala queries
- New Operational Reports added for MapReduce jobs, HBase, and Impala (available as part of Cloudera Enterprise)
- Support for creating user-defined triggers for metrics accessible via charts/tsquery
- Charting improvements
- New table chart type
- New options for displaying data and metadata from charts
- Support for exporting data from charts to CSV or JSON files
- Administrative Settings
- Added a new role type with limited administrator capabilities
- Cloudera Manager Server will create a heap dump if it runs out of memory
- Configure the location of the parcel directory and specify whether and when to remove old parcels from cluster hosts
What's New in Cloudera Manager 5.0.0 Beta 1
- CDH version
- Supports both CDH 4 and CDH 5
- CDH 4 to CDH 5 upgrade wizard
- Support for YARN as a production execution environment
- MapReduce (MRv1) to YARN (MRv2) configuration import
- YARN-based resource management for Impala 1.2
- JDK version - Cloudera Manager 5 supports and installs both JDK 6 and JDK 7.
- Resource Management
- Static and dynamic partitioning of resources: provides a wizard for configuring static partitioning of resources (cgroups) across core services (HBase, HDFS, MapReduce, Solr, YARN) and dynamic allocation of resources for YARN and Impala.
- Pool, resource group, and queue administration for YARN and Impala.
- Usage monitoring and trending
- YARN service monitoring
- YARN (MRv2) job monitoring
- Configurable histograms of Impala query and YARN job attributes that can be used to quickly filter query and application lists
- Scalable back-end database for monitoring metrics
- Charting improvements
- New chart types: histogram and heatmap
- New scale types: logarithmic and power
- Updates to tsquery language: new attribute values to support YARN and new functions to support new chart types
- Ability to manage both ISV applications and non-CDH services (for example, Accumulo, Spark, and so on)
- Working with select ISVs as part of Beta 1
- Single Sign-On - Support for SAML to enable single sign-on
- Dependency enforcement to ensure incompatible parcels are not used together
- Option to not cache downloaded parcels, to save disk space
- Improved error reporting for management operations
- Backup and Disaster Recovery (BDR)
- HBase and HDFS snapshots: Supports scheduling snapshots on a recurring basis.
- Support for YARN (MRv2): Replication jobs can now run using YARN (MRv2) instead of MRv1.
- Global replication page: All scheduled snapshots (HDFS and HBase) and replication jobs for either HDFS or Hive are shown on a single Replications page
- Global Search box
- Several usability improvements
- Comprehensive detection of configuration changes that require service restarts, refresh and redeployment of client configurations.
Incompatible Changes in Cloudera Manager 5
Incompatible Changes Introduced in Cloudera Manager 5.0.0 Beta 2
- Impala releases earlier than 1.2.1 are no longer supported.
Incompatible Changes Introduced in Cloudera Manager 5.0.0 Beta 1
- Impala - With Cloudera Manager 4.8 (released in late November 2013), only Impala 1.2.1 is supported, due to the introduction of the Impala Catalog Server. However, CDH 5.0.0 Beta 1 was released with Impala 1.2.0 (Beta). Therefore, if you upgrade from Cloudera Manager 4.8 (with Impala 1.2.1) to Cloudera Manager 5.0.0 Beta 1, and then upgrade your CDH to CDH 5.0.0 Beta 1, your version of Impala will be downgraded to Impala 1.2.0 from 1.2.1. This will result in some loss of functionality. See New Features in Impala for a list of the new features in Impala 1.2.1 that are not in Impala 1.2.0 (Beta).
- Hive - HiveServer2 is a mandatory role for Hive in CDH 5.
- Hue - In CDH 5, Hue no longer has a Beeswax Server role. Hue now submits queries to HiveServer2.
- HDFS - Cloudera Manager 5 does not support NFS-mounted shared edits directories for HDFS High Availability. It only supports the Quorum Journal method for shared edits. If you upgrade from Cloudera Manager 4 with a working CDH 4 High Availability configuration that uses NFS-mounted directories, your installation will continue to work until you disable High Availability. You will not be able to re-enable High Availability with NFS-mounted directories. Furthermore, you will not be able to upgrade to CDH 5 unless you disable High Availability, and you will need to use Quorum-based storage in order to re-enable High Availability after the upgrade.
- The YARN (MRv2) configuration mapreduce.job.userlog.retain.hours has been replaced by yarn.log-aggregation.retain-seconds. Any existing value in mapreduce.job.userlog.retain.hours will be lost. However, this configuration never had any effect, so no functionality is affected.
- The following configuration parameters were removed from YARN. These never had any effect, so no functionality is affected.
- mapreduce.jobtracker.handler.count (non-functional duplicate of yarn.resourcemanager.resource-tracker.client.thread-count)
- The following YARN configuration parameters were replaced. Only the YARN parameters were replaced. Old configurations will be lost, but they never had any effect so this does not affect functionality.
- mapreduce.jobtracker.restart.recover replaced by yarn.resourcemanager.recovery.enabled (changed from Gateway to ResourceManager)
- mapreduce.tasktracker.http.threads replaced by mapreduce.shuffle.max.connections
- mapreduce.jobtracker.staging.root.dir replaced by yarn.app.mapreduce.am.staging-dir
- Cloudera Manager 5 sets the default YARN Resource Scheduler to FairScheduler. If a cluster was previously running YARN with the FIFO scheduler, it will be changed to FairScheduler the next time YARN restarts. The FairScheduler is only supported with CDH 4.2.1 and later, and older clusters may hit failures and need to manually change the scheduler to FIFO or CapacityScheduler. See the Known Issues section of this Release Note for information on how to change the scheduler back to FIFO or CapacityScheduler.
Changed Features and Behaviors in Cloudera Manager 5
The following sections describe what’s changed in each Cloudera Manager 5 release.
: Rolling upgrade is not supported between CDH 4 and CDH 5. Rolling upgrade will also not be supported from CDH 5.0.0 Beta 2 to any later releases, and may not be supported between any future beta versions of CDH 5 and the General Availability release of CDH 5.
What's Changed in Cloudera Manager 5.0.0 Beta 2
- Cloudera Backup and Disaster Recovery (BDR) is now included with Cloudera Enterprise Basic Edition.
- Cloudera Standard has been renamed to Cloudera Express.
- OS and packaging
- The name of the Cloudera Manager embedded database package has changed from cloudera-manager-server-db to cloudera-manager-server-db-2. For details, read the upgrade and install topics for your OS.
- Support for Ubuntu 10.04 and Debian 6.0 is deprecated.
- HDFS - enabling High Availability automatically enables auto-failover, unlike in Cloudera Manager 4 where enable auto-failover was a separate command.
- In CDH 5 there is no HBase canary because HBase is now monitored by a watchdog process. In CDH 4, the HBase canary is still used.
- The RegionServer default heap size has been increased to 4GB
- Chart "Views" and actions related to views have been renamed to "Dashboard".
- Changes to how attribute filters are displayed in the Impala queries and YARN applications screens
- The outdated configuration indicator on the Home, service, and role pages has a new graphic and now has a tooltip that displays whether a cluster refresh or restart is required. There is a new indicator for changes that require redeploying client configurations. You can click an indicator to go to the new Stale Configurations page to view and resolve the conditions that gave rise to the indicator.
- To match the naming convention of tsquery metrics, multiword Impala query and YARN application attribute names have changed from camel case to using an underscore separator. For example queryType has changed to query_type. For backward compatibility, camel case names are still supported.
- The main navigation bar in Cloudera Manager Admin Console has been reorganized. The Services tab has been replaced by a Clusters tab that contains links to individual services, which were previously under the Services tab, Activities and Reports sections, which were removed from the main bar, and a new Manage Resources section, which contains links to the new resource pools and service pools features. The All Services page has been removed.
- The "Safety Valve" properties have been renamed "Advanced Configuration Snippet".
- The screen for specifying assignment of roles to hosts has been redesigned for improved scalability and usability
- The io.compression.codecs property has moved from MapReduce to HDFS
What's Changed in Cloudera Manager 5.0.0 Beta 1
- When CDH 5 is installed, YARN is installed by default, rather than MapReduce, and is the default execution environment. MapReduce is deprecated in CDH 5 but is fully supported for backward compatibility through CDH 5. In CDH 4, MapReduce is still the default.
- The setting for yarn.scheduler.maximum-allocation-mb has been increased to a default of 64GB.
- The minimum heap size for the Solr service has been increased to 200MB (from 50MB previously) to enable it to better handle collection creation.