# Elastic MapReduce (EMR)

Source: /aws/services/emr/

## Introduction

Amazon Elastic MapReduce (EMR) is a fully managed big data processing service that allows developers to effortlessly create, deploy, and manage big data applications.
EMR supports various big data processing frameworks, including Hadoop MapReduce, Apache Spark, Apache Hive, and Apache Pig.
Developers can leverage these frameworks and their rich ecosystem of tools and libraries to perform complex data transformations, machine learning tasks, and real-time data processing.

LocalStack supports EMR and allows developers to run data analytics workloads locally.
EMR utilizes various tools in the [Hadoop](https://hadoop.apache.org/) and [Spark](https://spark.apache.org) ecosystem, and your EMR instance is automatically configured to connect seamlessly to LocalStack's S3 API.
LocalStack also supports EMR Serverless to create applications and job runs, to run your Spark/PySpark jobs locally.

The supported APIs are available on the API coverage section for [EMR](#api-coverage) and [EMR Serverless](#api-coverage-emr-serverless), which provides information on the extent of EMR's integration with LocalStack.

:::note
To utilize the EMR API, certain additional dependencies need to be downloaded from the network (including Hadoop, Hive, Spark, etc).
These dependencies are fetched automatically during service startup, hence it is important to ensure a reliable internet connection when retrieving the dependencies for the first time.
Alternatively, you can use one of our `*-bigdata` Docker image tags which already ship with the required libraries baked in and may provide better stability (see [here](/aws/customization/other-installations/docker-images/) for more details).
:::

## Getting started

This guide is designed for users new to EMR and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command.

Start your LocalStack container using your preferred method.
We will create a virtual EMR cluster using the AWS CLI.
To create an EMR cluster, run the following command:

```bash
lstk aws emr create-cluster \
          --release-label emr-5.9.0 \
          --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m4.large InstanceGroupType=CORE,InstanceCount=1,InstanceType=m4.large
```

You will see a response similar to the following:

```bash title="Output"
{
    "ClusterId": "j-A2KF3EKLAOWRI"
}
```

You can also specify startup commands using the `--steps=...` command line argument to the `CreateCluster` API.

## Examples

The following code snippets and sample applications provide practical examples of how to use EMR in LocalStack for various use cases:

- [Running data analytics jobs using EMR](https://github.com/localstack/localstack-pro-samples/tree/master/sample-archive/emr-hadoop-spark-jobs)
- [Running EMR Serverless Jobs with Java](https://github.com/localstack/localstack-pro-samples/tree/master/emr-serverless-sample)

## API Coverage


### Elastic MapReduce API coverage

Source service: `emr`. 33 of 65 tracked operations are implemented.

Service documentation: /aws/services/emr/
License availability: available starting with the Ultimate plan. See /aws/licensing/ for current plan details.

| Operation | Status |
| --- | --- |
| AddInstanceFleet | Implemented |
| AddInstanceGroups | Implemented |
| AddJobFlowSteps | Implemented |
| AddTags | Implemented |
| CancelSteps | Not implemented |
| CreatePersistentAppUI | Not implemented |
| CreateSecurityConfiguration | Implemented |
| CreateStudio | Not implemented |
| CreateStudioSessionMapping | Not implemented |
| DeleteSecurityConfiguration | Implemented |
| DeleteStudio | Not implemented |
| DeleteStudioSessionMapping | Not implemented |
| DescribeCluster | Implemented |
| DescribeJobFlows | Implemented |
| DescribeNotebookExecution | Not implemented |
| DescribePersistentAppUI | Not implemented |
| DescribeReleaseLabel | Not implemented |
| DescribeSecurityConfiguration | Implemented |
| DescribeStep | Implemented |
| DescribeStudio | Not implemented |
| GetAutoTerminationPolicy | Implemented |
| GetBlockPublicAccessConfiguration | Implemented |
| GetClusterSessionCredentials | Not implemented |
| GetManagedScalingPolicy | Not implemented |
| GetOnClusterAppUIPresignedURL | Not implemented |
| GetPersistentAppUIPresignedURL | Not implemented |
| GetSession | Not implemented |
| GetSessionEndpoint | Not implemented |
| GetStudioSessionMapping | Not implemented |
| ListBootstrapActions | Implemented |
| ListClusters | Implemented |
| ListInstanceFleets | Implemented |
| ListInstanceGroups | Implemented |
| ListInstances | Implemented |
| ListNotebookExecutions | Not implemented |
| ListReleaseLabels | Implemented |
| ListSecurityConfigurations | Not implemented |
| ListSessions | Not implemented |
| ListSteps | Implemented |
| ListStudioSessionMappings | Not implemented |
| ListStudios | Not implemented |
| ListSupportedInstanceTypes | Implemented |
| ModifyCluster | Implemented |
| ModifyInstanceFleet | Implemented |
| ModifyInstanceGroups | Implemented |
| PutAutoScalingPolicy | Implemented |
| PutAutoTerminationPolicy | Implemented |
| PutBlockPublicAccessConfiguration | Implemented |
| PutManagedScalingPolicy | Not implemented |
| RemoveAutoScalingPolicy | Implemented |
| RemoveAutoTerminationPolicy | Implemented |
| RemoveManagedScalingPolicy | Not implemented |
| RemoveTags | Implemented |
| RunJobFlow | Implemented |
| SetKeepJobFlowAliveWhenNoSteps | Not implemented |
| SetTerminationProtection | Implemented |
| SetUnhealthyNodeReplacement | Not implemented |
| SetVisibleToAllUsers | Implemented |
| StartNotebookExecution | Not implemented |
| StartSession | Not implemented |
| StopNotebookExecution | Not implemented |
| TerminateJobFlows | Implemented |
| TerminateSession | Not implemented |
| UpdateStudio | Not implemented |
| UpdateStudioSessionMapping | Not implemented |

## API Coverage (EMR Serverless)


### EMR Serverless API coverage

Source service: `emr-serverless`. 14 of 22 tracked operations are implemented.

Service documentation: /aws/services/emr/
License availability: available starting with the Ultimate plan. See /aws/licensing/ for current plan details.

| Operation | Status |
| --- | --- |
| CancelJobRun | Implemented |
| CreateApplication | Implemented |
| DeleteApplication | Implemented |
| GetApplication | Implemented |
| GetDashboardForJobRun | Not implemented |
| GetJobRun | Implemented |
| GetResourceDashboard | Not implemented |
| GetSession | Not implemented |
| GetSessionEndpoint | Not implemented |
| ListApplications | Implemented |
| ListJobRunAttempts | Not implemented |
| ListJobRuns | Implemented |
| ListSessions | Not implemented |
| ListTagsForResource | Implemented |
| StartApplication | Implemented |
| StartJobRun | Implemented |
| StartSession | Not implemented |
| StopApplication | Implemented |
| TagResource | Implemented |
| TerminateSession | Not implemented |
| UntagResource | Implemented |
| UpdateApplication | Implemented |