AWS Glue Cost Optimization in 2026: DPU Right-Sizing, Flex Execution, and Auto Scaling
Cut your AWS Glue bill 40-60% in 2026 with DPU right-sizing, Flex execution's 34% discount, Auto Scaling on Glue 3.0+, and job bookmarks. Working CLI examples included.
AWS Glue cost optimization comes down to four levers: right-size the worker type and DPU count away from the 10-DPU default, move eligible batch jobs to the Flex execution class for a 34% discount ($0.29 vs $0.44 per DPU-hour), enable Auto Scaling on Glue 3.0/4.0, and turn on job bookmarks so you stop reprocessing yesterday's partitions. Applied together, those four changes routinely cut a Glue bill by 40–60% without touching the pipeline logic. This guide walks through each lever with worked numbers, the exact aws glue update-job flags, and the traps I hit most often when reviewing customer Glue accounts.
Glue Spark jobs default to 10 DPUs whether the workload needs them or not. Most jobs I audit run fine on 3–5.
Flex execution costs $0.29 per DPU-hour vs $0.44 for Standard, a 34% discount. It fits nightly batch, backfills, and non-urgent ELT.
Auto Scaling (Glue 3.0+) cuts DPU-hours 30–50% on bursty workloads by adding workers only when Spark actually queues tasks.
G.2X is the correct default worker type. G.4X and G.8X are for genuinely memory-bound joins, and picking them by reflex doubles your DPU rate.
Job bookmarks + Parquet + partition pushdown reduce scanned bytes so DPU-minutes fall alongside the S3 GET bill.
The Data Catalog is free up to 1M objects and 1M requests/month. Runaway crawlers and unbounded partition growth push you past that quietly.
How AWS Glue pricing actually works in 2026
AWS Glue bills you in DPU-hours. A DPU (Data Processing Unit) is a fixed compute unit that provides 4 vCPUs and 16 GB of memory. Every worker type is defined as some multiple of a DPU, and every second your job runs, AWS charges you for the DPUs it has allocated, whether Spark is actively using them or not. That last part is the source of most bill shock. The meter runs on allocated capacity, not utilized capacity.
Here's the 2026 rate card for Glue for Apache Spark:
Execution class
Rate per DPU-hour
Best for
Standard (G workers)
$0.44
Time-sensitive, SLA-bound jobs
Flex (G workers)
$0.29 (34% off)
Nightly batch, backfills, non-urgent ELT
Memory-Optimized (R workers)
$0.52
Wide shuffles, memory-bound joins
Python Shell
$0.44 (min 0.0625 DPU)
Small orchestration scripts
Streaming (G.025X supported)
$0.44 Standard
24×7 continuous ingest
Glue 2.0 and later bill per second with a 1-minute minimum. Glue 1.0 still carries a punitive 10-minute floor, which is a good reason on its own to migrate. Spark jobs enforce a hard 2 DPU minimum, and the console will happily default new jobs to 10 DPUs. So every time you click "Create job" without editing the number, you have opted into 5x the minimum spend. The official Glue pricing page has the current per-region rates. Check it before running quotes because a handful of regions carry a 10–15% premium.
Two more line items catch teams off guard. First, the Data Catalog charges $1.00 per 100,000 objects stored above the 1M free tier per month, plus $1.00 per 1M requests above 1M. Second, crawlers bill at the same $0.44/DPU-hour as jobs and carry a 10-minute minimum. A crawler you scheduled every 15 minutes "just to keep the schema fresh" is a real bill.
Right-sizing DPUs and worker types (G.1X, G.2X, G.4X, G.8X)
The single biggest win in Glue is dropping the DPU count. AWS defaults every new Spark job to 10 workers of type G.1X (10 DPUs). In my experience reviewing customer accounts, most transformation jobs on datasets under ~50 GB run happily on 3–5 DPUs. The default triples what the workload actually needs and does it silently (there's no warning banner in the console when you leave the field alone).
Here's the worker type ladder in 2026:
G.1X: 1 DPU (4 vCPU, 16 GB RAM, 94 GB disk). Light transforms, CSV/JSON to Parquet, small aggregations.
G.2X: 2 DPU (8 vCPU, 32 GB, 138 GB disk). The correct default for standard Spark workloads.
G.12X, G.16X: 12 and 16 DPU. Rare, and usually a sign of a pipeline-design problem, not a sizing problem.
R.1X–R.8X: memory-optimized at $0.52/DPU-hr. Reach for these only when you have measured OOM errors and confirmed a shuffle-heavy stage.
Picking G.4X when G.2X would fit doubles your per-worker DPU consumption. Picking R.2X when G.2X would fit adds 18% on top of that. Both are common. Teams size for the worst-case backfill and never resize for steady state. Right-sizing here is the same discipline as EC2, so if you haven't already, our cloud compute right-sizing guide for EC2, Azure VMs, and GCE covers the analogous decision for VMs, and the same "measured, not guessed" logic applies to Glue.
Update a job's worker configuration with the AWS CLI:
Set Timeout on every job. The default is 2,880 minutes (48 hours). A hung Spark stage at 10 G.2X workers eats $84 before it hits that ceiling, and yes, I've seen that exact bill land at 3am.
How to know if you're oversized
Turn on job metrics (--enable-metrics true) and look at three numbers in CloudWatch:
glue.driver.aggregate.numCompletedTasks vs number of workers. If tasks per worker is under 4, you have too many workers.
glue.jvm.heap.usage: if peak heap stays under 60% on G.2X, you don't need G.4X.
glue.driver.ExecutorAllocationManager.executors.numberAllExecutors: on an Auto Scaling job, the peak here is what you'd set as NumberOfWorkers on a fixed-size version.
AWS Glue Flex execution: when the 34% discount is safe
Flex is the easiest single-flag saving in Glue. AWS added it in 2022 for Glue 3.0+; in 2026 it applies to G.1X–G.8X on Glue 3.0, 4.0, and 5.0. The mechanic: Flex jobs run on spare AWS capacity, which means start time isn't guaranteed (you may wait a few minutes for capacity) and a running Flex job can be reclaimed if AWS needs the capacity back. In that case Glue restarts the job on new capacity, checkpointing where it can.
The math on a realistic nightly job: 10 DPUs × 30 minutes = 5 DPU-hours. At Standard, that's $2.20 per run. At Flex, $1.45. Over 30 days, $66 vs $43.50. Scale that to 40 nightly jobs and Flex saves you $900/month for changing one property.
Don't put Flex in front of anything an SLA depends on. Concrete no-go list:
Jobs feeding a customer-facing dashboard with a delivery deadline.
Streaming jobs (Flex doesn't apply; streaming is Standard only).
Any workflow where a restart mid-run would violate an idempotency assumption. For example, a job that emits notifications on completion.
Jobs shorter than ~5 minutes, since the queuing overhead can dominate the savings.
Auto Scaling in Glue 3.0 and 4.0
Auto Scaling was the first serious improvement to Glue economics in years. Before Glue 3.0, you paid for the workers you allocated for the full duration of the job. With Auto Scaling on Glue 3.0+, Spark adds workers only when the DAG has queued tasks and removes them when stages finish. You pay for what the job actually used, plus the driver.
On bursty workloads (jobs whose Spark stages have wildly different parallelism), Auto Scaling routinely cuts DPU-hours 30–50%. On uniformly-shaped jobs (a single big shuffle end-to-end), the savings are smaller because Spark just holds the peak the whole time.
With Auto Scaling on, NumberOfWorkers becomes the upper bound, not a fixed allocation. Set it generously (20–40 for typical batch), since you only pay for what Spark asks for. The AWS Glue Auto Scaling documentation lists the currently supported worker types; in 2026 it covers G.1X through G.16X, R.1X through R.8X, and G.025X for streaming only. Standard DPUs (legacy) aren't supported, so migrate off them.
Job bookmarks, Parquet, and partition pushdown
Honestly, the cheapest DPU-hour is the one you never spend because you didn't reprocess data you already processed yesterday. Job bookmarks track which S3 files (or JDBC row watermarks) a job has already ingested and skip them on the next run. Enable them on any incremental workload from day one, because retrofitting bookmarks onto a job with 18 months of state is painful. (I hit this exact bug on a customer migration last year, and it took two engineers a week to unpick.)
# In the job definition
"DefaultArguments": {
"--job-bookmark-option": "job-bookmark-enable"
}
# In the PySpark script, commit the bookmark at end of run
from awsglue.job import Job
job = Job(glue_context)
job.init(args['JOB_NAME'], args)
# ... transformations ...
job.commit()
Bookmarks aren't free of gotchas. They key on S3 object timestamps and paths, so if your upstream rewrites files in place or you change the partition layout, bookmarks can silently reprocess or silently skip. Belt-and-suspenders: also partition the source in S3 by date so that even if the bookmark misfires, the job only scans one day.
Which brings us to the second lever: read less. Every byte Glue scans costs DPU-time to deserialize. The compounding wins:
Parquet or ORC, not CSV/JSON. Columnar with compression cuts scanned bytes 5–10x for typical analytical workloads.
Partition by the filter column. If your queries always filter by event_date, partition the table by it in S3 (s3://bucket/table/event_date=2026-08-23/).
Predicate pushdown on the DynamicFrame reader so partition pruning happens at the catalog, not in Spark:
These techniques mirror the ones I covered in the cloud storage cost optimization guide for S3, Azure Blob, and GCS. The savings ratio in Glue is smaller than in raw storage scans (because Glue's compute meter is DPU-time, not scanned-bytes), but the compounding effect (less data scanned means less shuffle means fewer workers needed) still yields double-digit percentage savings on most jobs.
Data Catalog storage, requests, and crawler cost
The Data Catalog is essentially free for small deployments and quietly expensive for large ones. Here's the pricing:
Storage: 1M objects/month free, then $1.00 per 100,000 objects. An "object" is a table, table version, partition, or index.
Requests: 1M requests/month free, then $1.00 per 1M requests.
Crawlers: $0.44 per DPU-hour, 10-minute minimum, typically 2 DPUs.
The trap is unbounded partition growth on IoT / log / clickstream tables. A daily-partitioned table with 3 years of history has ~1,100 partitions. A minutely-partitioned table has ~1.6M partitions in the same window, and if you keep 5 versions of each (the default), you're at 8M objects. That's $70/month for the Data Catalog alone before anyone queries anything.
Three cleanup levers:
Set --conf spark.sql.parquet.partitionOverwriteMode=dynamic and drop SkipArchive in your crawler config so you don't accumulate table versions.
Aggregate hourly or minutely partitions to daily for cold data (older than 30 days).
Delete crawlers you don't actually need. Schedule the ones you keep at a cadence that matches how often the schema changes, which is almost never hourly.
Streaming jobs and the G.025X worker
Glue Streaming jobs run continuously and bill at the same $0.44/DPU-hour as batch, but 24×7 × 730 hours/month. A 2-DPU streaming job costs $642/month. That's why the G.025X worker (0.25 DPU each) exists. For low-throughput streaming, you can run 2 G.025X workers for 0.5 DPUs total, or $160/month. That's a 75% cut for streams whose event rate is under a few thousand records/second.
G.025X is streaming-only. It's not available for batch, and it's not supported by DataBrew or Glue for Ray. Configure it explicitly:
For high-throughput streaming (>10k records/s), G.025X will backpressure. Move to G.1X or G.2X, and consider whether a serverless option like Lambda, Azure Functions, or Cloud Run for the ingestion path is a better fit. At high sustained throughput, alternative compute per-vCPU-hour pricing can undercut Glue Streaming.
Monitoring DPU consumption and job cost
You can't optimize what you can't see. Every Glue job should have three things turned on before you argue about tuning:
--enable-metrics true: publishes 30+ Spark metrics to CloudWatch.
--enable-observability-metrics true: publishes DPU-hour and cost-relevant metrics to CloudWatch.
--enable-continuous-cloudwatch-log true: streams driver and executor logs to CloudWatch Logs.
Then tag every job with cost allocation tags (Project, Environment, Team) and enable those tags in Cost Explorer. Glue supports both resource tags on jobs/triggers/crawlers and DefaultArguments-based cost tracking. Pair this with the discipline in our Kubernetes chargeback and showback pipeline for namespace cost attribution if you run Glue across multiple accounts (you want per-team DPU-hour breakdowns, not just a single "Glue" line item in the payer account bill).
Here's a CLI snippet to pull last-month DPU-hours by job for a quick audit:
Look for USE1-ETL-DPU-Hour, USE1-Flex-DPU-Hour, and USE1-CrawlerDPU-Hour line items. The ratio of Standard to Flex DPU-hours is a fast tell for how far along the optimization curve you are. A mature Glue account often runs 40–60% of its non-streaming DPU-hours on Flex.
For a fully productionized cost-per-job dashboard, AWS published a solid pattern on the Big Data blog covering worker-type scaling on Glue for Spark. Combine that with a simple Athena view over your Cost and Usage Report and you have the same visibility a paid FinOps tool offers, for the cost of a small S3 bucket.
Frequently Asked Questions
What is a DPU in AWS Glue?
A DPU (Data Processing Unit) is Glue's compute unit: 4 vCPUs and 16 GB of memory. Glue bills you per DPU-hour that a job or crawler is allocated, at $0.44 for Standard workers, $0.29 for Flex, and $0.52 for memory-optimized R workers.
How is AWS Glue billed?
Glue bills per second, rounded up, with a 1-minute minimum on Glue 2.0 and later. The meter runs on allocated DPUs, not utilized DPUs, so a job configured for 10 DPUs that runs 15 minutes bills 2.5 DPU-hours, or roughly $1.10 at Standard rates.
What is the difference between G.1X and G.2X workers?
G.1X provides 1 DPU per worker (4 vCPU, 16 GB RAM, 94 GB disk); G.2X provides 2 DPUs per worker (8 vCPU, 32 GB RAM, 138 GB disk). Both are billed at $0.44 per DPU-hour, so cost scales with total DPUs, not worker count. G.2X is the recommended default for most Spark workloads because the larger per-executor heap reduces GC pressure.
Does AWS Glue Flex execution save 34% on every job?
Flex applies a 34% discount to the DPU-hour rate on any eligible Spark job (Glue 3.0+, G.1X–G.8X, batch only). It isn't suitable for SLA-bound jobs, streaming, or workflows that can't tolerate a restart, because Flex runs on spare capacity that AWS can reclaim mid-run.
Does AWS Glue have Auto Scaling and how much does it save?
Yes, Glue 3.0 and later support Auto Scaling for G.1X–G.16X, R.1X–R.8X, and G.025X for streaming. Enable it with --enable-auto-scaling true. On bursty workloads it cuts DPU-hours 30–50%; on uniformly-shaped jobs the savings are smaller because Spark holds the peak worker count throughout.
How do I reduce AWS Glue costs quickly?
Fix four things in order: drop DPU count from the default 10 to what the job actually needs, switch eligible batch jobs to the Flex execution class, enable Auto Scaling and job bookmarks on Glue 3.0+, and convert source data to Parquet with date partitioning. Applied together these routinely cut a Glue bill 40–60% without touching pipeline logic.
Rule-based groupings that slice your consolidated AWS bill by team, product, or environment without editing a single tag. Covers regular, inherited, and dimensional rules, split charge allocation, Terraform, and CUR 2.0 integration.
A practical 2026 guide to cutting AWS API Gateway spend: HTTP API migration for a 71% per-request saving, cache right-sizing, CloudFront offload, log tuning, and WebSocket connection-minute fixes with Terraform and CLI examples.
EKS Auto Mode adds a 12% management fee to EC2 hours, but replaces the platform-team labor of running Karpenter, EBS CSI, and load balancer controllers yourself. Break-even math from real production fleets, plus hidden costs and a safe incremental migration path.