Hi, some doubts about why my EMR clusters are failing. As you can see on the image below, I have a cluster which utilizes HDFS (but only with peaks of 5% of utilization), and while its running a job suddenly the nodes start to be marked as UNHEALTHY. ![Enter image description here](/media/postImages/original/IMk6cP43bFSny94ibCWIEQoA) Knowing that the cluster has a 95% of HDFS free space. The reason of having UNHEALTHY nodes is that they are running out of space on their Instance Storage volumes (local EBS)? (due to some temporary files, for example). Should I add/increase the EBSs volumes on the cluster's nodes?

EMR - UNHEALHTY nodes & HDFS utilization

Hi,

some doubts about why my EMR clusters are failing. As you can see on the image below, I have a cluster which utilizes HDFS (but only with peaks of 5% of utilization), and while its running a job suddenly the nodes start to be marked as UNHEALTHY.

Enter image description here

Knowing that the cluster has a 95% of HDFS free space. The reason of having UNHEALTHY nodes is that they are running out of space on their Instance Storage volumes (local EBS)? (due to some temporary files, for example). Should I add/increase the EBSs volumes on the cluster's nodes?

주제

분석 스토리지

태그

아마존 EMR 스토리지

언어

English

ahMarrone

질문됨 2년 전781회 조회

1개 답변