Some Notes on Apache Spark Memory Management

Published: | Last Edited:
Category: tech

Important configurations:

Sample Calculations:

Consider an EMR cluster with 1 master - 25 slaves running c5.18xlarge instance. Each instance comes with 72vCPU, 144 GiB Memory.


Next Post: Donate Compute Power to Folding@Home to Find Cures For Covid-19
Previous Post: Better SSH Keys Management with AuthorizedKeysCommand