Slashing CI Wait Times: How Pinterest Cut Android Testing Build Times by 36%

At Pinterest, our Android CI pipeline was suffering from slow, unpredictable build times. End-to-end tests were sharded by package name on Firebase Test Lab, creating severe load imbalances where the slowest shard gated the entire build. We set out to fix this — and ended up cutting E2E build times by over 36%.

The Problem

Package-based sharding distributed tests unevenly. Some shards finished in seconds while others ran for minutes, and the entire build waited on the slowest one. On top of that, Firebase Test Lab added setup overhead and limited our control over scheduling and environment configuration.

The Solution: PinTestLab + Runtime-Aware Sharding

We built PinTestLab, an in-house testing infrastructure running Android emulators on EC2 instances. This gave us full control over scheduling, environment setup, and runtime orchestration. Combined with that, we developed a runtime-aware sharding algorithm that uses historical test execution data from Metro (our internal test management system) to distribute tests by expected duration rather than count.

The sharding logic uses a min-heap data structure for greedy assignment, sorting tests by historical runtime and assigning each to the shard with the lowest cumulative time. The system includes a fallback to round-robin distribution if Metro becomes unavailable, ensuring reliability during infrastructure issues.

Results

The impact was significant: a 36% reduction in total E2E CI build times, a 55% reduction in the slowest shard’s runtime, and roughly 9 minutes saved per build. The gap between the fastest and slowest shards compressed from hundreds of seconds down to just a few dozen.

Read the Full Article

For the full deep-dive with architecture diagrams and implementation details, read the original post on the Pinterest Engineering Blog: Slashing CI Wait Times: How Pinterest Cut Android Testing Build Times by 36%+.