Post

i guess to make this more reasonable they should've given the non-ai group something else, maybe a basic calculatir. and then taken the tool away.

not only that, the AI group didn’t even fuck off! they get thrown when the AI first gets taken away (which could happen for all sorts of reasons), and then after that more-or-less return to the same levels as the controls

They changed the working conditions for the AI-group but not for the other group. That is what the study is identifying. Not dependence on AI-use.

Fair, and I realize we're all dealing with space constraints, etc. And definitely take your point about reports going well beyond what can be concluded from the study. My sense is that this is a common problem in extrapolating from ed psych experiments of this kind to real-world learning conditions.

You were arguing that they’d be demoralised / demotivated in ways the control group were not?

Seems more like, eg: Have people draw 15 circles as perfectly as they can. One group has only pencil and paper, the other has pencil and paper too but can also use design software. Remove access to design software after 12 circles. Compare results between groups of final three circles.

I think that you've nailed it here. There was no real incentive for the participants to actually engage with the task in the first place. If they were to have incentivized correct answers, it'd likely have significantly changed how people engaged with the task.

Blaming it on the press for headlines that simplify research is fine I guess. This summary, however, is from the official University page where one of the co-authors works. He was quoted for it

also, the press release really isn’t far off from the language in the conclusion of the paper?

Even if it's not their fault, it's their responsibility. So it's like final proofs and why they are so stressful. It's my responsibility to catch all the typos or mistakes there even if they were introduced after my last edits. It's so strange to blame the university press office!

I've posted about it here, I used to post about it on Twitter, I've written about it in our paper on Misinformation in and about Science, etc. The point is this: Authors have an obligation to collaborate with their university press office to ensure the accuracy of reports about their own work.

In a sane universe, it would be the author's fault. In practice, I've seen a lot of occasions where authors try desperately to get the press office not to say something, and the office does anyways. I have a lot of sympathy to the people whose results get garbled this way despite their best efforts.

The number of times I have interviewed the author of a paper and said; "The media release says X" [often followed by "that seems to contradict the paper" and the authors have said; "Oh did they say that, I didn't tell them that and it's not really true".

Is it though, while I mostly agree with you, I can count numerous times PR were inaccurate in reporting scientific papers…most recent example in my field is lake sturgeon in the GL are 400 yrs old…if you read the paper that is NOT what it says at all but that’s the headline 🤷‍♂️

Anecdotally, the intrinsically worst chess-players are those who are addicted to using engines. Play them without their engines, and they totally suck, why? they forgot how to think.

Now this is what discourse (as opposed to random posting) on social media should be like. Put your point of view and when someone comes in, politely, with more information share that as well. Doesn't strike me as hard but sure seems to be.

One factor could be externalising motivation. Getting a machine to do something switches your motivation from solving the problem to getting the solution. Intrinsic motivation generally is good for learning. And more fun.

This is not what anyone would call a high powered study.

Yeah, but how else are they going to rebuild the Russian data centres?

A more direct one might be: have two groups sum a column of values on a spreadsheet. Give one group Excel and the other group paper and pencil. Then replace Excel with paper and pencil for the first group. Does using Excel make us dumber?

More like giving one group shovels and the other a dog that you have to try to get to dig the hole in the right place while you sit and do nothing.

Doing fractions shouldn't be that hard though. It's the perception of effort that's exactly the question. Why does firing a few neurons to do something we learned in 3rd grade count as "too hard" -- it shouldn't be comparable to an activity that could give even a very fit person blisters.

Joe beat me to it.

I care more about how Experiment 2 does it, in more than one way.

bsky.app/profile/mich...

If they proceeded to perform more poorly w/ trowels than people who'd never used shovels, I think we could conclude that using shovels erodes your ability to use trowels.

I’m a software engineer who often runs out of AI credits before the end of the month because my employer is cheap about them. It’s very annoying/demoralizing and that my progress gets slowed the last week of the month or whatever. I didn’t get dumber though 🤷‍♂️

Asserting that AI robs people of focus and persistence in the way taking away any tool that does the job for them is pretty obvious. I’m not sure greater rewards would have sold the case harder, its not like folks re getting big rewards to do their own searches or write their own cover letters.

Using a shovel for ten minutes would rob someone of their motivation and willingness to persist in digging the same sized hole with a trowel. Tools make work easier or even do that work are rewarding; it’s a known fact bolstered by studies that removing a consistent reward robs people of motivation

Do you have any evidence that the results were distorted by what you claim is a lack of explanation for the withdrawal of “AI” assistance? That would indeed be a flaw, but that isn’t what’s described in what you’ve screenshot.

Did the professor use AI for ten minutes to come up this dumb study ?

That is, there is a 10% increase in skip rates for those questions and a 10% decrease in solve rates. If it's just skipping questions then the statement "The lower solve rate of participants in the AI condition demonstrates that AI assistance impairs unassisted performance" is not true.

It would be interesting to see the study replicated with calculators instead of A.I.