Select a model using payment-conversion evaluation

Goal

Compare model variants on payment conversion in an isolated test experiment and select using prespecified windows, sample counts, and uncertainty rather than public benchmarks alone.

Steps

1. Fix the eligible population, model arms, offer conditions, and conversion window. [src-openai-select-model-using-payment-conversion-evaluation]
2. Record exposures and test-mode payment events in Stripe, linked to model versions. [src-openai-select-model-using-payment-conversion-evaluation]
3. Compare conversion rates and sample sizes, then document the selection and limits. [src-openai-select-model-using-payment-conversion-evaluation]

How to verify

Verify arms, exposure counts, conversion numerator/denominator, and time windows before reporting a winning model.

Troubleshooting

If traffic or offers differ, stop comparing raw conversion; fix randomization/attribution and rerun on the same population.