Back to InsightsTechnology

Building Low-Latency Systems on AWS

Devon Edwards
August 2024
12 min read

Can you build low-latency trading systems on AWS? The short answer is yes—with caveats. We've helped several clients achieve sub-millisecond performance on AWS for trading workloads that don't require the absolute lowest latency. Here's what we've learned.

Setting Expectations

Let's be clear about what "low latency" means in this context:

  • Ultra-Low Latency (sub-10 microseconds): Not achievable on public cloud. If you need this, you need co-located hardware.
  • Low Latency (10-100 microseconds): Achievable with AWS Outposts or dedicated hosts, but expensive.
  • Moderate Latency (100 microseconds - 1 millisecond): Achievable on standard AWS with proper architecture.
  • Standard Latency (1-10 milliseconds): Easily achievable, suitable for many trading use cases.

Most systematic trading strategies don't actually need sub-100 microsecond latency. If you're trading on a minutes-to-hours timeframe, spending millions on co-location infrastructure is wasted money.

Instance Selection

Not all EC2 instances are created equal for latency-sensitive workloads:

Recommended Instance Types

  • c6i/c7i instances: Best price/performance for compute-intensive workloads
  • Metal instances (*.metal): Eliminates hypervisor overhead, ~10-20% latency improvement
  • Placement groups: Cluster placement groups minimize network latency between instances

Networking

Enhanced Networking with ENA (Elastic Network Adapter) is essential. Key configurations:

  • Enable ENA Express for even lower latency
  • Use placement groups to ensure instances are physically close
  • Consider AWS PrivateLink for VPC endpoints to reduce hops

Operating System Tuning

The default Linux configuration is optimized for throughput, not latency. Key tuning parameters:

# Disable CPU frequency scaling

echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# Disable transparent huge pages

echo never > /sys/kernel/mm/transparent_hugepage/enabled

# Network stack tuning

sysctl -w net.core.busy_read=50

sysctl -w net.core.busy_poll=50

CPU Isolation

For the most latency-sensitive processes, isolate CPUs to prevent scheduler interference:

# Boot parameter

isolcpus=2,3,4,5

# Pin your application

taskset -c 2,3,4,5 ./your_trading_app

Application Architecture

Cloud-native doesn't have to mean slow. Key patterns for low-latency applications:

Memory Management

  • Pre-allocate all memory at startup
  • Use object pools to avoid runtime allocation
  • Lock memory pages to prevent swapping

Networking

  • Use kernel bypass where possible (DPDK on metal instances)
  • Batch network operations when latency allows
  • Consider UDP for internal communications where reliability can be handled at application layer

Data Structures

  • Cache-friendly data layouts (arrays over linked lists)
  • Lock-free data structures for inter-thread communication
  • Avoid dynamic dispatch in hot paths

Measuring Latency

You can't improve what you can't measure. Essential practices:

  • Use hardware timestamps (clock_gettime with CLOCK_MONOTONIC_RAW)
  • Measure at percentiles (p99 matters more than average)
  • Profile regularly and track latency over time
  • Account for AWS "noisy neighbor" effects with statistical analysis

Case Study: Market Data Pipeline

We recently built a market data normalization pipeline on AWS achieving these metrics:

  • Median latency: 150 microseconds
  • p99 latency: 450 microseconds
  • Throughput: 2 million messages/second
  • Monthly cost: ~$15,000 (vs. $80,000+ for equivalent co-lo setup)

For a systematic strategy trading on a 5-minute timeframe, this performance is more than sufficient—and the cloud deployment provides flexibility and cost savings that wouldn't be possible with dedicated infrastructure.

When Cloud Isn't Right

Don't use AWS for:

  • Market making or high-frequency strategies requiring single-digit microsecond latency
  • Strategies where you're competing directly with co-located competitors
  • Regulatory requirements mandating on-premise infrastructure

Conclusion

AWS can absolutely support low-latency trading workloads for the right use cases. The key is understanding your actual latency requirements and architecting appropriately. We've found that many firms over-invest in latency infrastructure when their strategies don't actually require it.

Ready to discuss your project?

Let's talk about how we can help you build better trading infrastructure.

Contact Us