# Batch

Source: /aws/services/batch/

## Introduction

Batch is a cloud-based service provided by Amazon Web Services (AWS) that simplifies the process of running batch computing workloads on the AWS cloud infrastructure. Batch allows you to efficiently process large volumes of data and run batch jobs without the need to manage and provision underlying compute resources.

Under the hood, the local Docker engine is used to run the containers that simulate your Batch jobs.

LocalStack allows you to use the Batch APIs to automate and scale computational tasks in your local environment while handling batch workloads. Batch jobs are executed using the ECS runtime, allowing for support of managed compute environments and improved service compatibility.

The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Batch integration with LocalStack.

## Getting started

This guide is designed for users new to AWS Batch and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command.

Start your LocalStack container using your preferred method.
We will demonstrate how you create and run a Batch job by following these steps:

1. Creating a service role for the compute environment.
2. Creating the compute environment.
3. Creating a job queue using the compute environment.
4. Creating a job definition.
5. Submitting a job to the job queue.

### Create a service role

You can create a role using the [`CreateRole`](https://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html) API.
 
LocalStack requires the role to exist with a valid trust policy. When [enforcing IAM policies](/aws/developer-tools/security-testing/iam-policy-enforcement), ensure that the policy is valid and the role is properly attached.

Run the following command to create a role for ECS task execution:

```bash
lstk aws iam create-role \
    --role-name myrole \
    --assume-role-policy-document '{
  "Version": "2025-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}'
```

Then attach the ECS task execution policy:

```bash
lstk aws iam attach-role-policy \
    --role-name myrole \
    --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
```


### Create the compute environment

You can use the [`CreateComputeEnvironment`](https://docs.aws.amazon.com/cli/latest/reference/batch/create-compute-environment.html) API to create a compute environment.

Run the following command using the role ARN above (arn:aws:iam::000000000000:role/myrole) to create a managed compute environment with FARGATE:

```bash
lstk aws batch create-compute-environment \
    --compute-environment-name myenv \
    --type MANAGED \
    --state ENABLED \
    --compute-resources type=FARGATE,maxvCpus=128,subnets=subnet-12345678,securityGroupIds=sg-12345678 \
    --service-role arn:aws:iam::000000000000:role/myrole
```

:::note
While networking resources such as subnets and security groups are required as input, LocalStack does not create real cloud infrastructure. These values must still be present for the compute environment to be created.
:::


### Create a job queue

You can fetch the ARN using the [`DescribeComputeEnvironments`](https://docs.aws.amazon.com/cli/latest/reference/batch/describe-compute-environments.html) API.

Run the following command to fetch the ARN of the compute environment:

```bash
lstk aws batch describe-compute-environments --compute-environments myenv
```

```bash title="Output"
{
  "computeEnvironments": [
    {
      "computeEnvironmentName": "myenv",
      "computeEnvironmentArn": "arn:aws:batch:us-east-1:000000000000:compute-environment/myenv",
      "ecsClusterArn": "arn:aws:ecs:us-east-1:000000000000:cluster/OnDemand_Batch_abc123",
      "type": "MANAGED",
      "status": "VALID",
      "statusReason": "Compute environment is available",
      "serviceRole": "arn:aws:iam::000000000000:role/myrole"
    }
  ]
}
```

You can use the ARN to create the job queue using [`CreateJobQueue`](https://docs.aws.amazon.com/cli/latest/reference/batch/create-job-queue.html) API.

Run the following command to create the job queue:

```bash
lstk aws batch create-job-queue \
    --job-queue-name myqueue \
    --priority 1 \
    --compute-environment-order order=0,computeEnvironment=arn:aws:batch:us-east-1:000000000000:compute-environment/myenv \
    --state ENABLED
```

### Create a job definition

Now, you can define what occurs during a job run. In this example, you can execute the 'busybox' container from DockerHub and initiate the command: 'sleep 30'. It's important to note you can override this command when submitting the job. 

Run the following command to create the job definition using the [`RegisterJobDefinition`](https://docs.aws.amazon.com/cli/latest/reference/batch/register-job-definition.html) API:


```bash
lstk aws batch register-job-definition \
    --job-definition-name myjobdefn \
    --type container \
    --platform-capabilities FARGATE \
    --container-properties '{
        "image": "busybox",
        "resourceRequirements": [
            {"type": "VCPU", "value": "0.25"},
            {"type": "MEMORY", "value": "512"}
        ],
        "command": ["sleep", "30"],
        "networkConfiguration": {
            "assignPublicIp": "ENABLED"
        },
        "executionRoleArn": "arn:aws:iam::000000000000:role/myrole"
    }'
```

If you want to pass arguments to the command as [parameters](https://docs.aws.amazon.com/batch/latest/userguide/job_definition_parameters.html#parameters), you can use the `Ref::` declaration to set placeholders for parameter substitution.

This allows the dynamic passing of values at runtime for specific job definitions.

```bash
lstk aws batch register-job-definition \
    --job-definition-name myjobdefn \
    --type container \
    --parameters '{"time":"10"}' \
    --platform-capabilities FARGATE \
    --container-properties '{
        "image": "busybox",
        "resourceRequirements": [
            {"type": "VCPU", "value": "0.25"},
            {"type": "MEMORY", "value": "512"}
        ],
        "command": ["sleep", "Ref::time"],
        "networkConfiguration": {
            "assignPublicIp": "ENABLED"
        },
        "executionRoleArn": "arn:aws:iam::000000000000:role/myrole"
    }'
```

### Submit a job to the job queue

You can now run a compute job.
This command runs a job on the queue that you have set up previously, overriding the container command to run: `sh -c "sleep 5; pwd"`.
This command simulates work being done in the container.

Run the following command to submit a job to the job queue using the [`SubmitJob`](https://docs.aws.amazon.com/cli/latest/reference/batch/submit-job.html) API:

```bash
lstk aws batch submit-job \
    --job-name myjob \
    --job-queue myqueue \
    --job-definition myjobdefn \
    --container-overrides '{"command":["sh", "-c", "sleep 5; pwd"]}'
```

## Multi-node parallel jobs

LocalStack supports [AWS Batch multi-node parallel (MNP) jobs](https://docs.aws.amazon.com/batch/latest/userguide/multi-node-parallel-jobs.html), which run a single job across a main node and one or more worker nodes.
The main node starts first, and the workers follow once it is running. Each worker receives the main node's private IP so the nodes can communicate.

MNP jobs run on EC2-backed compute environments only. Fargate is not supported.

To run one, register a job definition with `--type multinode` and a `nodeProperties` object that sets the main node, the number of nodes, and a container per node range:

```bash
lstk aws batch register-job-definition \
    --job-definition-name mnp-jobdefn \
    --type multinode \
    --node-properties '{
        "mainNode": 0,
        "numNodes": 2,
        "nodeRangeProperties": [
            {
                "targetNodes": "0:1",
                "container": {
                    "image": "busybox",
                    "command": ["sh", "-c", "echo node $AWS_BATCH_JOB_NODE_INDEX; sleep 10"],
                    "resourceRequirements": [
                        {"type": "MEMORY", "value": "512"},
                        {"type": "VCPU", "value": "1"}
                    ]
                }
            }
        ]
    }'
```

Then submit it to an EC2-backed queue:

```bash
lstk aws batch submit-job \
    --job-name mnp-job \
    --job-queue mnp-queue \
    --job-definition mnp-jobdefn
```

The submitted job is the parent. Each node is addressable as a child job using the `<jobId>#<nodeIndex>` notation, which you can inspect with `describe-jobs`:

```bash
lstk aws batch describe-jobs --jobs "<jobId>#0" "<jobId>#1"
```

Each node also receives additional [environment variables](#environment-variables), such as `AWS_BATCH_JOB_NODE_INDEX` and `AWS_BATCH_JOB_MAIN_NODE_PRIVATE_IPV4_ADDRESS`, that let the nodes coordinate.

## Environment variables

LocalStack injects a subset of the Batch environment variables into each job container:

- `AWS_BATCH_CE_NAME`
- `AWS_BATCH_JOB_ARRAY_INDEX`
- `AWS_BATCH_JOB_ARRAY_SIZE`
- `AWS_BATCH_JOB_ATTEMPT`
- `AWS_BATCH_JOB_ID`
- `AWS_BATCH_JQ_NAME`

[Multi-node parallel jobs](#multi-node-parallel-jobs) receive the following additional variables on each node:

- `AWS_BATCH_JOB_NODE_INDEX` — the index of the current node.
- `AWS_BATCH_JOB_NUM_NODES` — the total number of nodes in the job.
- `AWS_BATCH_JOB_MAIN_NODE_INDEX` — the index of the main node.
- `AWS_BATCH_JOB_MAIN_NODE_PRIVATE_IPV4_ADDRESS` — the private IP of the main node, set on worker nodes so they can connect back to the main node.

## Current Limitations

LocalStack simulates the execution of ECS-based AWS Batch jobs using the local ECS runtime. No real infrastructure is created or managed.

Array jobs are supported in sequential mode only.

The configuration variable `ECS_DOCKER_FLAGS` can be used to pass additional Docker flags to the container runtime.

Setting `ECS_TASK_EXECUTOR=kubernetes` is supported as an alternative backend, though Kubernetes execution is experimental and may not support all features.

## API Coverage


### Batch API coverage

Source service: `batch`. 24 of 48 tracked operations are implemented.

Service documentation: /aws/services/batch/
License availability: available starting with the Ultimate plan. See /aws/licensing/ for current plan details.

| Operation | Status |
| --- | --- |
| CancelJob | Implemented |
| CancelJobs | Not implemented |
| CreateComputeEnvironment | Implemented |
| CreateConsumableResource | Not implemented |
| CreateJobQueue | Implemented |
| CreateQuotaShare | Not implemented |
| CreateSchedulingPolicy | Implemented |
| CreateServiceEnvironment | Not implemented |
| DeleteComputeEnvironment | Implemented |
| DeleteConsumableResource | Not implemented |
| DeleteJobQueue | Implemented |
| DeleteQuotaShare | Not implemented |
| DeleteSchedulingPolicy | Implemented |
| DeleteServiceEnvironment | Not implemented |
| DeregisterJobDefinition | Implemented |
| DescribeComputeEnvironments | Implemented |
| DescribeConsumableResource | Not implemented |
| DescribeJobDefinitions | Implemented |
| DescribeJobQueues | Implemented |
| DescribeJobs | Implemented |
| DescribeQuotaShare | Not implemented |
| DescribeSchedulingPolicies | Implemented |
| DescribeServiceEnvironments | Not implemented |
| DescribeServiceJob | Not implemented |
| GetJobQueueSnapshot | Not implemented |
| ListConsumableResources | Not implemented |
| ListJobs | Implemented |
| ListJobsByConsumableResource | Not implemented |
| ListQuotaShares | Not implemented |
| ListSchedulingPolicies | Implemented |
| ListServiceJobs | Not implemented |
| ListTagsForResource | Implemented |
| RegisterJobDefinition | Implemented |
| SubmitJob | Implemented |
| SubmitServiceJob | Not implemented |
| TagResource | Implemented |
| TerminateJob | Implemented |
| TerminateJobs | Not implemented |
| TerminateServiceJob | Not implemented |
| TerminateServiceJobs | Not implemented |
| UntagResource | Implemented |
| UpdateComputeEnvironment | Implemented |
| UpdateConsumableResource | Not implemented |
| UpdateJobQueue | Implemented |
| UpdateQuotaShare | Not implemented |
| UpdateSchedulingPolicy | Implemented |
| UpdateServiceEnvironment | Not implemented |
| UpdateServiceJob | Not implemented |