Post

mr. TIM
@timkellogg.me

it’s probably likely that his work was a candidate for model training openai may still have not trained on it, e.g. if they thought it was too low quality they could filter it out, or synthetically rephrase it still, openai is going to need better data provenance in order to combat allegations

3 reposts69 likes

Erik Westlund
@erikwestlund.bsky.social

This is a serious issue. I also think it will be a wake-up call to scientists and academics working on cutting edge problems that using subscriptions where training on conversations is explicitly enabled may be unwise. Corporations, e.g., have been wary of this issue from the beginnning, paying ...

Erik Westlund
@erikwestlund.bsky.social

orders of magnitudes more for API-based model access w/ a guarantee of no training so they can protect their IP. Scientists should now budget for this. I admittedly thought it was a little paranoid, but if you could have patents, field-defining discoveries, etc. at risk, it clearly seems worth it.

Erik Westlund
@erikwestlund.bsky.social

I agree this is a bigger issue than just attribution, however. The issue with companies touting discoveries that their training data made possible is evidently problematic. But now I'm thinking of people at conferences w/ less "brain power" hearing ideas and working on them with AI. Before, it...

abenomicon
@abenomicon.bsky.social

> openai is going to need better data provenance in order to combat allegations this increasingly feels like something all saas will need to do. more NPI is material these days imo, and it's more likely than ever to quietly leak via model training

mr. TIM
@timkellogg.me

a much simpler alternative is to a paid researcher tier where chatgpt conversations never gets trained on. i’m sure startups would love it too

mr. TIM
@timkellogg.me

no question it’s in their training corpus, but unclear if a model actually got trained on it they do a shit ton of filtering & transforming. like, these steps are arguably the major component of a competitive advantage but it also makes it harder to trace which documents actually made it in

mr. TIM
@timkellogg.me

his claim is that he turned it on after he started working i’m guessing he turned it on after creating a conversation, and just continued the same conversation, hence his unclarity on what happened also, he didn’t really have much evidence that’s what happened

ldj5000
@ldj5000.bsky.social

Yea I’ve been confused by this, I thought ChatGPT has a setting to opt your account out of being used for training. I have that with my Claude account. Is the accusation that they’re ignoring that setting and training on it anyway? Or did these researchers forget to turn that off?

mr. TIM
@timkellogg.me

solid evidence here that the mathematician is fabricating the story bsky.app/profile/pekk...

Pekka Lund@pekka.bsky.social

For all that I can see, Andreas Thom seems to be yet another mathematician who makes serious accusations against OpenAI with no evidence or reasonable basis. He says he emailed OpenAI (Bubeck and Sellke) "shortly after" OpenAIs non-sofic group announcement, which happened on August 1.

Timmy Tokyo
@timmytokyo.bsky.social

Doesn't seem very solid to me. I saw that person vociferously defending OpenAI against the earlier allegations -- with very little personal knowledge. They seem to have a bias.

Brandon Downey
@bdowney.bsky.social

I found this on the page for chatgpt for academic research: "Workspaces include business-grade privacy and security protections, and data is not used to train our models by default." My take: there are a LOT of ways prompts could leak. Start with 'accidentally uses personal ChatGPT one session'. :|

openai.comAccelerating scientific discovery with ChatGPT for Academic ResearchersOpenAI is giving 100,000 academic researchers free access to ChatGPT's most advanced AI models to accelerate scientific research, collaboration, and discovery.