Elea McDonnell Feit

Elea McDonnell Feit

@eleafeit.bsky.social

Professor of Marketing at Drexel. Philadelphian. Bayesian.

506 followers309 following83 posts

View full thread
View full thread

This paper taught me that the competition between advertisers for the most desirable ad slots is really important.

Caio Waisman@caiowaisman.bsky.social

We have a beautiful literature on experiments with interference between units but what if there’s interference between experimenters? Estimands change and in a way we can understand and unpack. I hope you enjoy this one! 😊 pubsonline.informs.org/doi/10.1287/...

View full thread

Happy to report that this paper has been accepted at Management Science! (At least that was reported in the ISMS Newsletter, but I can't find the paper on the Management Science website.

Elea McDonnell Feit@eleafeit.bsky.social

Joo and Chiong’s recent working paper provides a Gaussian approximation to the regret function that you can use with ✨any✨ asymptotically normal estimator. This means you can use the minimax-regret criteria with your favorite treatment effect estimators: diff-in-diff, ML estimators.

Elea McDonnell Feit reposted

Replying to @eleafeit.bsky.social

Joo and Chiong’s recent working paper provides a Gaussian approximation to the regret function that you can use with ✨any✨ asymptotically normal estimator. This means you can use the minimax-regret criteria with your favorite treatment effect estimators: diff-in-diff, ML estimators.

Elea McDonnell Feit reposted

We are seeing some impact on reproducibility efforts with the transition to the new US administration. For now, I suggest that you make backups of any data that required you to actively interact with the Federal government, as there may be a delay in response times.... #openscience

Toying with the idea of using RStudio + #quarto + github to manage content for my doctoral seminar on causal inference. I'd like to be able to create short "explainer" documents that integrate math + code + hyperlinks. I don't want this to be a major project. Thoughts on design and workflow?

Regardless of where you stand politically, the separation of powers is a fundamental principle of American democracy. The President should pursue policy goals by working with Congress—not through unilateral actions that undermine the constitutional process and sow uncertainty.

View full thread

This is a big contrast to null hypothesis testing, which weights all type-I errors equally and totally ignores type-II errors. All of the above papers show that return-aware strategies use *much* smaller sample sizes than the hypothesis testing you learned in college.

View full thread

Azevedo et al. 2020 demonstrate that the prior can affect Bayes optimal strategies. So, I say, “if you’ve got priors, use them.” And if you’ve got data on past A/B tests, you use it to form priors.

Finally, if you really, really want to keep using p-values, this working paper by Sudijono, Ejdemyr, Lal and Tingley shows how to solve Azevedo et al.’s “A/B Testing Problem” b tuning the hypothesis testing confidence level based on the prior to replicate the Bayes optimal solution.

View full thread

Bayes regret or minimax regret, which should you use? Minimax regret has the convenient "feature" that it does not require a prior on the treatment effect. But that could be considered a "bug" for companies that have a large number of past A/B test results and can use them to set priors.

Which brings me to Azevedo et al. (2020), who ask: If you have many ideas to test, should you run many small tests or fewer large ones? Solving for Bayes regret, they show it depends on whether your prior has fat tails. If you expect many high-performing ideas, then you should run smaller tests.

View full thread

I recently came across Manski 2019 that provides an accessible introduction to the minimax regret approach developed in a series of papers by Manski and Tetenov. They focus on applications to clinical trials, but A/B tests are analogous. (Love _The American Statistican_ BTW.)

Joo and Chiong’s recent working paper provides a Gaussian approximation to the regret function that you can use with ✨any✨ asymptotically normal estimator. This means you can use the minimax-regret criteria with your favorite treatment effect estimators: diff-in-diff, ML estimators.

Some of my followers will know Feit and Berman 2019. (How has it been five years?). Ron and I describe A/B tests as a decision problem where the goal is to maximize profit (minimize Bayes regret) and derive the optimal sample size over priors which we estimate from collections of past A/B tests.

Description: This course teaches critical thinking skills to combat misinformation. You will learn to identify common forms of misinformation, evaluate evidence and arguments, and critically evaluate claims you encounter in your personal and professional lives.

In the last few years, I've worked with several doctoral students and junior faculty who did excellent data analysis in Stan, #R or #Python, but had trouble translating their code to math notation to write a paper. Why should they be forced to do this? #rejectmathsupremacy