Anthropic

Claude at Scale: Optimizing Cost & Performance for Enterprise AI

AI agents have transformed how we handle repetitive workflows, and their impact is particularly profound when processing large volumes of data. According to , 43% of AI tasks involve direct automation, with computer a…

Kashyap Coimbatore MuraliJul 28, 2025

Claude at Scale: Optimizing Cost & Performance for Enterprise AI

Introduction: From Costly to Cost-Effective AI at Scale

AI agents have transformed how we handle repetitive workflows, and their impact is particularly profound when processing large volumes of data. According to Anthropic's Economic Index, 43% of AI tasks involve direct automation, with computer and mathematical fields seeing the highest adoption for "software modification, code debugging, and network troubleshooting." One of the most valuable applications is automating repeated tasks—from analyzing security logs for potential threats to processing thousands of customer support tickets.

But there's a catch: as your token count grows, so does your bill. This guide focuses exclusively on optimizing your AI model spend through a layered approach where each technique compounds your savings.

Before diving in, let's establish some definitions. Tokens are the fundamental units of language models—for Claude, one token represents approximately 3.5 English characters. When you see pricing like $3/MTok for input and $15/MTok for output, that translates to real costs: processing a 100,000-token document and generating a 2,000-token response costs you $0.33 ($0.30 for input + $0.03 for output). ‍

Baseline Scenario: Understanding Your Current Costs

Let's start with real Hadoop logs that we've downloaded from the LogPai/LogHub repository to establish our baseline:

python

1# Token counting with real Hadoop logs
2import anthropic
3import os
4client = anthropic.Anthropic(api_key=os.environ.get('ANTHROPIC_API_KEY'))
5
6# Read full Hadoop logs (2k lines from LogHub repository)
7with open('hadoop_2k.log', 'r') as f:
8    logs = f.readlines()
9
10log_content = ''.join(logs)
11token_count = client.beta.messages.count_tokens(
12    model="claude-4-sonnet",
13    messages=[{"role": "user", "content": log_content}]
14)
15
16print(f"Total logs: {len(logs)}")
17print(f"Total tokens: {token_count.input_tokens}")
18print(f"Tokens per log line: {token_count.input_tokens / len(logs):.1f}")
19print(f"Cost at $3/MTok: ${token_count.input_tokens * 3.0 / 1_000_000:.6f}")

Output

1Total logs: 1999
2Total tokens: 128524
3Tokens per log line: 64.3
4Cost at $3/MTok: $0.385572

A typical log entry from our dataset:

text

12015-10-18 18:01:47,978 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1445144423722_0020_000001

Our baseline scenario: Consider an enterprise processing 1 million log analysis requests monthly. Each request analyzes 100,000 input tokens (roughly 1,556 log lines from our full dataset) and generates a 2,000-token assessment.

Processing frequency: ~23 requests per minute, 24/7
Monthly cost: $330,000
Annual cost: $3,960,000

Let's see how we can dramatically reduce this.

Layer 1: Data Prep That Pays Off

Your first line of defense against runaway costs is intelligent data preparation. Every unnecessary token you send is money wasted.

By removing redundant information, simplifying timestamps, and compressing verbose field names, you can achieve significant token reduction:

python

1
2import re
3
4def compress_hadoop_log(log_line):
5    # Extract components
6    match = re.match(r'(\d{4}-\d{2}-\d{2}) ([\d:,]+) (\w+) \[([^\]]+)\] ([\w.]+): (.+)', log_line)
7    if not match: return log_line
8
9    date, time, level, thread, class_name, message = match.groups()
10
11    # Compress time, class names, and IDs
12    time_compressed = time.split(',')[0]
13    class_compressed = class_name.split('.')[-1]
14    message = re.sub(r'appattempt_\d+_(\d+)_(\d+)', r'app_\1_\2', message)
15
16    return f"[{time_compressed}] {level} {class_compressed}: {message.strip()}"
17
18# Test with real Hadoop logs
19log = "2015-10-18 18:01:47,978 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1445144423722_0020_000001"
20compressed = compress_hadoop_log(log)
21
22original_tokens = client.beta.messages.count_tokens(
23    model="claude-4-sonnet",
24    messages=[{"role": "user", "content": log}]
25).input_tokens
26
27compressed_tokens = client.beta.messages.count_tokens(
28    model="claude-4-sonnet",
29    messages=[{"role": "user", "content": compressed}]
30).input_tokens
31
32print(f"Original: {original_tokens} tokens")
33print(f"Compressed: {compressed_tokens} tokens")
34print(f"Reduction: {(1 - compressed_tokens/original_tokens)*100:.1f}%")

Output

1Original: 128524 tokens
2Compressed: 73419 tokens
3Reduction: 42.9%

The strategy: Remove non-predictive data, compress verbose formats, filter by severity, identify critical features with Claude

Cost impact: For our baseline scenario, reducing input tokens by 43% saves:

Monthly savings: $141,900
Annual savings: $1,702,800
New annual cost: $2,257,200

Layer 2: Model Selection by Application Type

For some tasks, it might be as simple as just summarizing the logs, where Claude 3.5 Haiku might be fine. However, there are some tasks such as finding security vulnerabilities where Claude 4 Sonnet or Claude 4 Opus might be needed. The key is matching model capabilities to task requirements.

Consider a typical enterprise environment: web server access logs often need basic pattern matching and summarization—tasks where Claude 3.5 Haiku excels at $0.25/MTok. Meanwhile, security analysis and anomaly detection require Claude 4 Sonnet's advanced reasoning at $3/MTok. For the most critical security vulnerabilities and complex analysis, Claude 4 Opus at $15/MTok provides the highest capability.

Cost impact: Routing 20% of logs to Haiku:

Additional monthly savings: $28,215 (15% reduction)
Running total saved: $170,115/month
New annual cost: $1,918,620

Layer 3: Architecture That Scales Your Savings

Prompt Caching (Up to 90% Cost Reduction on Cached Content)

For multiple related analyses on the same data, caching eliminates redundant token costs:

python

1# Cache the log data for multiple analyses
2import anthropic
3
4client = anthropic.Anthropic()
5
6# First request - cache write
7response1 = client.messages.create(
8    model="claude-4-sonnet",
9    max_tokens=1000,
10    messages=[{
11        "role": "user",
12        "content": [
13            {
14                "type": "text",
15                "text": compressed_logs,  # Your compressed log data
16                "cache_control": {"type": "ephemeral"}
17            },
18            {
19                "type": "text",
20                "text": "Identify security incidents in these logs."
21            }
22        ]
23    }]
24)
25
26# Subsequent requests - cache reads
27response2 = client.messages.create(
28    model="claude-4-sonnet",
29    max_tokens=1000,
30    messages=[{
31        "role": "user",
32        "content": [
33            {
34                "type": "text",
35                "text": compressed_logs,
36                "cache_control": {"type": "ephemeral"}
37            },
38            {
39                "type": "text",
40                "text": "Identify poor logging practices."
41            }
42        ]
43    }]
44)
45
46# Calculate savings with our compressed logs
47# From Layer 1: we compressed 128,524 tokens down to 73,419 tokens
48cache_tokens = 73419  # Our actual compressed log data
49requests_per_hour = 3  # 3 different analyses: security, poor logging, latency
50
51# Without caching: pay full price for every request
52without_cache = requests_per_hour * (cache_tokens * 3.0 / 1_000_000)
53
54# With caching: pay 25% more for first write, then 10% for all reads
55with_cache = 1 * (cache_tokens * 3.75 / 1_000_000) + (requests_per_hour - 1) * (cache_tokens * 0.30 / 1_000_000)
56
57print(f"Cache size: {cache_tokens:,} tokens")
58print(f"Without caching: ${without_cache:.2f}/hour")
59print(f"With caching: ${with_cache:.2f}/hour")
60print(f"Savings: {(1-with_cache/without_cache)*100:.0f}%")

Output

1Cache size: 73,419 tokens
2Without caching: $0.66/hour
3With caching: $0.32/hour
4Savings: 52%

Batch Processing (50% Cost Reduction)

For tasks that don't require immediate responses (like daily summaries or trend analysis), batch processing offers a flat 50% discount:

python

1# Example: Daily log summary using batch API
2from anthropic import Anthropic
3from anthropic.types.beta import BetaMessageBatch, BetaMessageBatchRequestItem
4
5client = Anthropic()
6
7# Create batch requests for daily summaries
8batch_requests = []
9for day_logs in daily_log_chunks:
10    batch_requests.append(
11        BetaMessageBatchRequestItem(
12            custom_id=f"daily-summary-{day}",
13            params={
14                "model": "claude-4-sonnet",
15                "max_tokens": 1000,
16                "messages": [{
17                    "role": "user",
18                    "content": f"Summarize key events and anomalies:\n{day_logs}"
19                }]
20            }
21        )
22    )
23
24# Submit batch
25batch = client.beta.messages.batches.create(
26    requests=batch_requests
27)
28
29print(f"Batch ID: {batch.id}")
30print(f"Processing {len(batch_requests)} daily summaries")
31print(f"Cost savings: 50% off standard pricing")

Perfect for:

Daily log summaries
Weekly security reports
Trend analysis over time

Cost impact (assuming 40% of requests can be batched):

Batch processing savings: $31,977/month (50% off on 40% of requests)
Prompt caching savings: $8,314/month (52% off on 10% of requests)
Combined savings from Layers 1-3: $210,406/month
New annual cost: $1,435,128

Layer 4: Query Grouping (Cost Reduction on Shared Context)

If you traditionally want analyses every 30 minutes and you're not in a rush to get it right after, then you can wait an hour and send 1 hour's worth of content (assuming it fits into the context window):

python

1# Using function calling for structured outputs
2tools = [{
3    "name": "analyze_logs",
4    "description": "Analyze multiple log entries and return structured results",
5    "input_schema": {
6        "type": "object",
7        "properties": {
8            "analyses": {
9                "type": "array",
10                "items": {
11                    "type": "object",
12                    "properties": {
13                        "timestamp": {"type": "string"},
14                        "severity": {"type": "string"},
15                        "summary": {"type": "string"},
16                        "action_required": {"type": "boolean"}
17                    }
18                }
19            }
20        }
21    }
22}]
23
24# Group 2 half-hour batches into 1 hourly analysis
25hourly_logs = half_hour_batch1 + half_hour_batch2
26
27response = client.messages.create(
28    model="claude-4-sonnet",
29    max_tokens=2000,
30    tools=tools,
31    messages=[{
32        "role": "user",
33        "content": f"Analyze these logs from the past hour:\n{hourly_logs}"
34    }]
35)
36
37# Calculate savings from reduced API overhead
38# When grouping, we reduce duplicate context like few-shot examples
39analyses_per_hour = 2  # Every 30 minutes
40logs_per_analysis = 1000
41context_tokens = 500  # Few-shot examples, instructions, etc.
42
43# Individual: 2 requests, each with context + logs
44individual_tokens = analyses_per_hour * (context_tokens + logs_per_analysis)
45# Grouped: 1 request with context + combined logs
46grouped_tokens = 1 * (context_tokens + analyses_per_hour * logs_per_analysis)
47
48individual_cost = individual_tokens * 3.0 / 1_000_000
49grouped_cost = grouped_tokens * 3.0 / 1_000_000
50
51print(f"Individual: {individual_tokens:,} tokens = ${individual_cost:.4f}/hour")
52print(f"Grouped: {grouped_tokens:,} tokens = ${grouped_cost:.4f}/hour")
53print(f"Savings: {(1-grouped_cost/individual_cost)*100:.0f}%")

Using function calling to structure outputs, you maintain individual analysis granularity while reducing API calls.

Cost impact: Assuming 30% of requests can be grouped, saving 15% on those:

Additional monthly savings: $5,382
Running total saved: $215,788/month
New annual cost: $1,370,547

Additional Enterprise Scale Optimizations

Provisioned Throughput Units (PTUs)

For predictable, high-volume workloads, PTUs offer fixed-cost pricing with guaranteed capacity. AWS Bedrock and Google Vertex AI both provide options with longer commitments yielding better rates.

Model Distillation

Anthropic's model distillation (now in preview on Amazon Bedrock) transfers Claude 4 Sonnet's intelligence to Claude 3.5 Haiku's speed and price point. For specific tasks, you get Sonnet-level accuracy at Haiku prices—a potential 73% cost reduction with no performance sacrifice.

Conclusion: Compound Savings at Scale

Through systematic optimization with real log data, we've reduced annual AI costs from $3,960,000 to $1,370,547—a 65% reduction while maintaining or improving performance:

Data Preparation: 43% base reduction → $2,257,200
Model Selection: 15% additional reduction → $1,918,620
Caching + Batching: 25% additional reduction → $1,435,128
Query Grouping: 4.5% additional reduction → $1,370,547

These optimizations compound. Start with data preparation as your foundation, then make smart model choices based on your application types. Architectural improvements like caching and batching provide substantial additional savings, while query grouping offers incremental benefits.

Kashyap Coimbatore Murali

Anthropic

Claude at Scale: Optimizing Cost & Performance for Enterprise AI

Kashyap Coimbatore MuraliJul 28, 2025

Introduction: From Costly to Cost-Effective AI at Scale

Baseline Scenario: Understanding Your Current Costs

Let's start with real Hadoop logs that we've downloaded from the LogPai/LogHub repository to establish our baseline:

python

1# Token counting with real Hadoop logs
2import anthropic
3import os
4client = anthropic.Anthropic(api_key=os.environ.get('ANTHROPIC_API_KEY'))
5
6# Read full Hadoop logs (2k lines from LogHub repository)
7with open('hadoop_2k.log', 'r') as f:
8    logs = f.readlines()
9
10log_content = ''.join(logs)
11token_count = client.beta.messages.count_tokens(
12    model="claude-4-sonnet",
13    messages=[{"role": "user", "content": log_content}]
14)
15
16print(f"Total logs: {len(logs)}")
17print(f"Total tokens: {token_count.input_tokens}")
18print(f"Tokens per log line: {token_count.input_tokens / len(logs):.1f}")
19print(f"Cost at $3/MTok: ${token_count.input_tokens * 3.0 / 1_000_000:.6f}")

Output

1Total logs: 1999
2Total tokens: 128524
3Tokens per log line: 64.3
4Cost at $3/MTok: $0.385572

A typical log entry from our dataset:

text

12015-10-18 18:01:47,978 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1445144423722_0020_000001

Processing frequency: ~23 requests per minute, 24/7
Monthly cost: $330,000
Annual cost: $3,960,000

Let's see how we can dramatically reduce this.

Layer 1: Data Prep That Pays Off

Your first line of defense against runaway costs is intelligent data preparation. Every unnecessary token you send is money wasted.

By removing redundant information, simplifying timestamps, and compressing verbose field names, you can achieve significant token reduction:

python

1
2import re
3
4def compress_hadoop_log(log_line):
5    # Extract components
6    match = re.match(r'(\d{4}-\d{2}-\d{2}) ([\d:,]+) (\w+) \[([^\]]+)\] ([\w.]+): (.+)', log_line)
7    if not match: return log_line
8
9    date, time, level, thread, class_name, message = match.groups()
10
11    # Compress time, class names, and IDs
12    time_compressed = time.split(',')[0]
13    class_compressed = class_name.split('.')[-1]
14    message = re.sub(r'appattempt_\d+_(\d+)_(\d+)', r'app_\1_\2', message)
15
16    return f"[{time_compressed}] {level} {class_compressed}: {message.strip()}"
17
18# Test with real Hadoop logs
19log = "2015-10-18 18:01:47,978 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1445144423722_0020_000001"
20compressed = compress_hadoop_log(log)
21
22original_tokens = client.beta.messages.count_tokens(
23    model="claude-4-sonnet",
24    messages=[{"role": "user", "content": log}]
25).input_tokens
26
27compressed_tokens = client.beta.messages.count_tokens(
28    model="claude-4-sonnet",
29    messages=[{"role": "user", "content": compressed}]
30).input_tokens
31
32print(f"Original: {original_tokens} tokens")
33print(f"Compressed: {compressed_tokens} tokens")
34print(f"Reduction: {(1 - compressed_tokens/original_tokens)*100:.1f}%")

Output

1Original: 128524 tokens
2Compressed: 73419 tokens
3Reduction: 42.9%

The strategy: Remove non-predictive data, compress verbose formats, filter by severity, identify critical features with Claude

Cost impact: For our baseline scenario, reducing input tokens by 43% saves:

Monthly savings: $141,900
Annual savings: $1,702,800
New annual cost: $2,257,200

Layer 2: Model Selection by Application Type

Cost impact: Routing 20% of logs to Haiku:

Additional monthly savings: $28,215 (15% reduction)
Running total saved: $170,115/month
New annual cost: $1,918,620

Layer 3: Architecture That Scales Your Savings

Prompt Caching (Up to 90% Cost Reduction on Cached Content)

For multiple related analyses on the same data, caching eliminates redundant token costs:

python

1# Cache the log data for multiple analyses
2import anthropic
3
4client = anthropic.Anthropic()
5
6# First request - cache write
7response1 = client.messages.create(
8    model="claude-4-sonnet",
9    max_tokens=1000,
10    messages=[{
11        "role": "user",
12        "content": [
13            {
14                "type": "text",
15                "text": compressed_logs,  # Your compressed log data
16                "cache_control": {"type": "ephemeral"}
17            },
18            {
19                "type": "text",
20                "text": "Identify security incidents in these logs."
21            }
22        ]
23    }]
24)
25
26# Subsequent requests - cache reads
27response2 = client.messages.create(
28    model="claude-4-sonnet",
29    max_tokens=1000,
30    messages=[{
31        "role": "user",
32        "content": [
33            {
34                "type": "text",
35                "text": compressed_logs,
36                "cache_control": {"type": "ephemeral"}
37            },
38            {
39                "type": "text",
40                "text": "Identify poor logging practices."
41            }
42        ]
43    }]
44)
45
46# Calculate savings with our compressed logs
47# From Layer 1: we compressed 128,524 tokens down to 73,419 tokens
48cache_tokens = 73419  # Our actual compressed log data
49requests_per_hour = 3  # 3 different analyses: security, poor logging, latency
50
51# Without caching: pay full price for every request
52without_cache = requests_per_hour * (cache_tokens * 3.0 / 1_000_000)
53
54# With caching: pay 25% more for first write, then 10% for all reads
55with_cache = 1 * (cache_tokens * 3.75 / 1_000_000) + (requests_per_hour - 1) * (cache_tokens * 0.30 / 1_000_000)
56
57print(f"Cache size: {cache_tokens:,} tokens")
58print(f"Without caching: ${without_cache:.2f}/hour")
59print(f"With caching: ${with_cache:.2f}/hour")
60print(f"Savings: {(1-with_cache/without_cache)*100:.0f}%")

Output

1Cache size: 73,419 tokens
2Without caching: $0.66/hour
3With caching: $0.32/hour
4Savings: 52%

Batch Processing (50% Cost Reduction)

For tasks that don't require immediate responses (like daily summaries or trend analysis), batch processing offers a flat 50% discount:

python

1# Example: Daily log summary using batch API
2from anthropic import Anthropic
3from anthropic.types.beta import BetaMessageBatch, BetaMessageBatchRequestItem
4
5client = Anthropic()
6
7# Create batch requests for daily summaries
8batch_requests = []
9for day_logs in daily_log_chunks:
10    batch_requests.append(
11        BetaMessageBatchRequestItem(
12            custom_id=f"daily-summary-{day}",
13            params={
14                "model": "claude-4-sonnet",
15                "max_tokens": 1000,
16                "messages": [{
17                    "role": "user",
18                    "content": f"Summarize key events and anomalies:\n{day_logs}"
19                }]
20            }
21        )
22    )
23
24# Submit batch
25batch = client.beta.messages.batches.create(
26    requests=batch_requests
27)
28
29print(f"Batch ID: {batch.id}")
30print(f"Processing {len(batch_requests)} daily summaries")
31print(f"Cost savings: 50% off standard pricing")

Perfect for:

Daily log summaries
Weekly security reports
Trend analysis over time

Cost impact (assuming 40% of requests can be batched):

Batch processing savings: $31,977/month (50% off on 40% of requests)
Prompt caching savings: $8,314/month (52% off on 10% of requests)
Combined savings from Layers 1-3: $210,406/month
New annual cost: $1,435,128

Layer 4: Query Grouping (Cost Reduction on Shared Context)

python

1# Using function calling for structured outputs
2tools = [{
3    "name": "analyze_logs",
4    "description": "Analyze multiple log entries and return structured results",
5    "input_schema": {
6        "type": "object",
7        "properties": {
8            "analyses": {
9                "type": "array",
10                "items": {
11                    "type": "object",
12                    "properties": {
13                        "timestamp": {"type": "string"},
14                        "severity": {"type": "string"},
15                        "summary": {"type": "string"},
16                        "action_required": {"type": "boolean"}
17                    }
18                }
19            }
20        }
21    }
22}]
23
24# Group 2 half-hour batches into 1 hourly analysis
25hourly_logs = half_hour_batch1 + half_hour_batch2
26
27response = client.messages.create(
28    model="claude-4-sonnet",
29    max_tokens=2000,
30    tools=tools,
31    messages=[{
32        "role": "user",
33        "content": f"Analyze these logs from the past hour:\n{hourly_logs}"
34    }]
35)
36
37# Calculate savings from reduced API overhead
38# When grouping, we reduce duplicate context like few-shot examples
39analyses_per_hour = 2  # Every 30 minutes
40logs_per_analysis = 1000
41context_tokens = 500  # Few-shot examples, instructions, etc.
42
43# Individual: 2 requests, each with context + logs
44individual_tokens = analyses_per_hour * (context_tokens + logs_per_analysis)
45# Grouped: 1 request with context + combined logs
46grouped_tokens = 1 * (context_tokens + analyses_per_hour * logs_per_analysis)
47
48individual_cost = individual_tokens * 3.0 / 1_000_000
49grouped_cost = grouped_tokens * 3.0 / 1_000_000
50
51print(f"Individual: {individual_tokens:,} tokens = ${individual_cost:.4f}/hour")
52print(f"Grouped: {grouped_tokens:,} tokens = ${grouped_cost:.4f}/hour")
53print(f"Savings: {(1-grouped_cost/individual_cost)*100:.0f}%")

Using function calling to structure outputs, you maintain individual analysis granularity while reducing API calls.

Cost impact: Assuming 30% of requests can be grouped, saving 15% on those:

Additional monthly savings: $5,382
Running total saved: $215,788/month
New annual cost: $1,370,547

Additional Enterprise Scale Optimizations

Provisioned Throughput Units (PTUs)

For predictable, high-volume workloads, PTUs offer fixed-cost pricing with guaranteed capacity. AWS Bedrock and Google Vertex AI both provide options with longer commitments yielding better rates.

Model Distillation

Conclusion: Compound Savings at Scale

Through systematic optimization with real log data, we've reduced annual AI costs from $3,960,000 to $1,370,547—a 65% reduction while maintaining or improving performance:

Data Preparation: 43% base reduction → $2,257,200
Model Selection: 15% additional reduction → $1,918,620
Caching + Batching: 25% additional reduction → $1,435,128
Query Grouping: 4.5% additional reduction → $1,370,547

Kashyap Coimbatore Murali