Hadoop 2.4.0 was recently released with several enhancements to both HDFS and YARN. This includes support for Access Control Lists, Native support for Rolling upgrades, Full HTTPS support for HDFS, Automatic failover of YARN and other operational improvements
This is the second release in 2014 (first one being 2.3.0 in Feb). Below are brief details about the enhancements -
- Access Control Lists enhance the HDFS permission model to support file access for arbitrary combinations of users and groups. Before this, you could only choose from 3 pre-determined options - a single owner, a single group and all other users
- Rolling Upgrades make upgrading large Hadoop installations much easier to manage
- Full HTTPS Support for HDFS across all components - WebHDFS, HsFTP and web interfaces
- YARN Application History Server - to provide a general solution for recording history of applications that run on YARN
- A generic Application Timeline Server which provides better metric capabilities for applications that run on YARN for application specific metrics such as containers used or resources consumed
- Automatic Failover for YARN - applications can failover to a cold standby ResourceManager in case of operational issues such as hardware failures
The team also shared some upcoming features in the next release, 2.5.0 -
- Further improvements in rolling upgrades
- Admin-specified labels for servers in YARN for enhanced control and scheduling
- Allow applications to delegate resources to others in YARN and share YARN's workload-management capabilities as well
- Automatic sharing of artifacts in YARN distributed cache
You can download the latest release and also look at the release notes for more details.