EMR Serverless Application Disk Space?
0
When configuring an EMR Serverless application, you can choose the disk size for your preinitialized capacity and a maximum disk limit for the application:
However, at least for Hive jobs, EMR Serverless stores both the scratch and warehouse directories on S3. So what is this configured disk space used for? I've searched the documentation and there's no mention of how this space is used, or why it's needed.
1 Risposta
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
1
EMR Serverless uses disk size for the following purposes.
- Storing the log files ( Hive, Tez AM logs, Tez Task logs etc)
- Shuffle data is stored on the local disk.
- Resources such as aux libraries, user jar etc.
Hope it answers your question.
Contenuto pertinente
AWS UFFICIALEAggiornata 3 anni fa- AWS UFFICIALEAggiornata 3 anni fa
- AWS UFFICIALEAggiornata un anno fa
- AWS UFFICIALEAggiornata 3 anni fa
This does answer the question, thanks. But we needed this answer 3 months ago. We've since moved our company's infrastructure to a new cloud provider.