Optimal Traffic Allocation Under Heterogeneous Variant Cost
AI Summary: Cost-optimal traffic allocation in experiments can save 5-15% of the budget by weighing the precision gained from additional subjects against their costs, which vary across treatment and control arms. Traditional 50/50 traffic splits may not be optimal when treatment costs differ from control costs, as seen in applications of LLMs, discounts, or vouchers. The main blockers to adopting cost-optimal designs are the need for speed in experimentation and the convention of 50/50 traffic splits for variance reduction. Cost-optimal allocation is particularly relevant in industries with varying treatment costs, such as tech, healthcare, and economics.