Post

Unpopular opinion but if you're interested in the effects of a medical product, it's 'RCT or bust' for me. I can see the case for observational analyses when randomization isn't possible, ethical, or the condition is too rare or some other such thing, but there's no such excuse here.

Saloni@scientificdiscovery.dev

Worth noting that many (not all) of the observational studies so far have been funded by GSK, which sells the shingles vaccine. Vaccines are great and the shingles vaccine is also great (for shingles), but people have got to be a little more skeptical of observational studies funded by pharma IMHO.

12 reposts101 likes

That's fine, the question is for medical *services*. Stuff like custom mRNA vaccines and other things is held back both by cost, and by the fact that it's different every time by design, so designing and funding an RCT isn't so simple.

This is a medical product though, not a service; I mentioned the disease being too rare for an RCT already, though I think developers can (and have) run RCTs for multiple custom mRNA vaccines (rather than each one).

The first especially is not a reliable source, and we have no idea whether it truly worked or it is dumb luck and "AI psychosis", but it seem fair to say that this sort of stuff would be hard to get by RCTs

Conflicts of interest notwithstanding: I think you’re underestimating the difficulty of trial enrollment and follow up for such long term outcomes in the absence of an animal model or clearly defined mechanism of action

Would you expect it to be harder than for other Alzheimer's trials? I'm not sure it would be, given shingles vaccines have already shown efficacy and safety for another condition.

Ah sorry, yes I agree on that, I meant to say compared to other prevention drugs. GLP1s is one example but tbf the incentive to participate in that trial would've been higher since it's also a treatment.

Placebo doesn't mean withholding the treatment, but they wouldn't get it as part of the trial. In this case, unlike for many other medical products like drugs, it's possible for people to get vaccines through another route and are similar to OTC. (Plus not as fraught as vaccines for fatal diseases)

and maybe I am misunderstanding--they ARE using the natural experiment that some folks do not get the vaccine, through choice, lack of access, or simply didn't get around to it. Is your worry that there is a variable that correlates with lack of choosing to get this vaccine?

I think observational studies make many more assumptions than RCTs do, and it's harder to tell what can potentially cause issues. Generally, for observational studies on the effects of medicines and vaccines, differential selection into treatment is an important one.

But if it was still a concern, one could instead run a stepped wedge trial or another design in which all volunteers get the vaccine but at different time points.

Counterpoint: the absence of RCTs has been weaponized especially in gender affirming care for minors and is the load-bearing rationale for the forced detransition of thousands of kids. RCT is best, I agree, but not a necessity to show that a given therapy has benefits > harms.

Yes but a trial would have been very be hard here? So going to England after wales seems like a good move, ont you can do while you get a trial going anyway? That welsh effect size would have shown up in the English data (and the fact it doesn’t is very telling)..

why don’t you simply go and read the ethical rules governing medical trials instead of writing pointless posts and then argue when it is pointed out to you it is pointless? The trials you ask for are unethical.

Sounds sane to me! RCT for confirmation, observational (plus, and I know this sounds crazy, theory) for exploration of potential RCTs to design. Caveat: I am not a real scientist.

I agree, I don’t think you can ethically randomize people to no treatment here. And so if you insist on an RCT that means you never know the answer. Ironically the antivaxxers love taking this position for bad faith reasons

Not running one when you have the ability (and incentive) to - as a pharma company with a commercial product that's still on patent - is hard to excuse. Just saying.

Not ethical why? This would be an efficacy trial for its effect on dementia, not shingles, it's not universally recommended in most countries (though there's a case that it should be) and shingles vaccine uptake isn't very high anyway

If not universally recommended but researcher knows there is an established effect on shingles,is it really ethical to give a placebo without informing the participants they could benefit had they chosen to take the vaccine? I don't think so But you can't really randomise if you do provide this info

I think it's more of a cost/benefit analysis from a company perspective. A study is generally run to get an indication included in the label. The decision to spend the money to add an indication will depend on how much more drug will be sold with that indication. I'm not as familiar with dementia,

but I have worked on cardiovascular effects studies. I would imagine it would be similar in terms of the number of people and how long folks would need to be followed. This is a lot of money. And I think protective effects might not result in enough sales to justify it.

Very interesting there's already an RCT ongoing on this! It's quite strange though: unblinded, and the control group isn't asked to come in for tests or follow up. (But the whole population is in registry data.) I guess that would make it similar to a natural experiment, also unblinded.

minvaccination.dkDAN-ZOSTER – Shingles Study 65+We are seeking individuals aged 65 or older who wish to participate in the DAN-ZOSTER research study. The study investigates whether vaccination against shingles can reduce the risk of cardiovascular ...

I weight RCTs very heavily, too But wouldn’t you agree that most medical choices & decisions do not have RCT evidence (much as we’d like)? In many cases, we just have to do our best with bad evidence

I think I'd agree that for vaccines, RCTs are crucial. I'd also want some evidence that there is a biologically plausible metchanism - but you don't get to do RCTs of vaccines without that. I don't agree that RCTs are the best method for every research qn.

I’m an epidemiologist and I mostly agree with this - the majority of observational research you see doesn’t even have a question that’s asked clearly enough for the answer to be meaningful However - sometimes you can’t do the RCT, sometimes you can do good observational research, and -rarely- both

Especially when it comes to dementia / cognitive decline, I always think there are obvious confounders that would make it less likely to receive such a vaccine AND more likely to get dementia. Same btw for hearing aids