I did some tests. I found it interesting that in the reasoning content the model identifies itself as Kimi by moonshot and sometimes it assumes ChatGPT as identity. Then sometimes it references the need to be approved by "the evaluator", other times it references global system guidelines / moonshot guidelines, and something about an phantom instruction line being "you are an assistant responding in an API" or something like that.
I did not ask the model which model it is, I'm not interested in the model answering the question either. The interesting bit is that this happens while doing another task. I'm not saying that this is because of distillation of openai's model either or a proxying. To me this looks like a side effect of their RL.
Could it be because Claude / ChatGPT are the most viral models and thus are more likely to be in the training set so when someone asks "Which AI are you?" the likeliness of Claude and ChatGPT being a reasonable answer is higher?
It has been observed again and again that ChatGPT and Claude would identify themselves as DeepSeek when asked in Chinese, through the API without a system prompt to solidify their identities. I don't think identity is a good evidence for distillation.
An alernative, it could also be the case that all of the model developers are using distillation to improve their models while crying do as I say, and not as I do.
Or the answer is even easier. There is public data consisting of chat transcripts with ChatGPT, Claude and DeepSeek, and all AI providers train on that data. You don't need distillation for the text "I am ChaGPT/Claude/DeepSeek" to often appear in your trraining data.
That's because every single model release after GPT-3.5 has been, at least in part, been trained on data synthesized by GPT-3.5 and then the other OpenAI models, as well as Anthropic models later.
Also the distillation attacks and other distillation methods that gather chain of thought information from frontier models are also used to generate vast amounts of data in order to generate synthetic datasets to post-train what I believe is the vast majority of models that are being released.
If you look at any information about how these models are trained, the details about how the data has been synthesized, where the agentic traces come from that are used in post-training, etc., they almost never tell you about exactly what model was used to generate them.
If I was in Anthropic's shoes, I'd definitely have a small team running various open-weight models to distill them for a few billion tokens worth of training data.
In our own benchmark we found that K3.1 pro UI designs are very similar to OpenAI's designs.
K3 for reference had 0.43 similarity to OpenAI's models, now it's as similar as how OpenAI's models are to each other at 0.58, much higher than other cross-lab similarities.
Which benchmark? Would love to see it & any datasets.
It’s super tricky to tease out whether the similarities point towards distillation. A lot of the aesthetic similarities are naturally emergent from the training data.
There's no reason to train the model to be precise there. You always have this information externally, so why would anyone spend time and money ensuring this gets answered? (Then you have to correct again for RL updates)
I assume so. You're the only one in the comments who understood the post, people read only two words and ignored the rest. I also found it interesting that the model seems to be trained to assume identities too. So maybe giving it an identity by name and attributes can enhance the model's performance.
Or because ChatGPT produced texts exists on the internet now. I remember before destillation was a thing when Claude was new and it said it was chaptgpt (probably because ChatGPT was the only popular LLM ai at the time)
But that doesn't explain why previous versions usually identified themself as Claude rather than ChatGPT. Text from from ChatGPT is almost certainly more common on the Internet than text from Claude.
Fascinating that we’re converging on AI safety and alignment in time across jurisdictions. The cybersecurity approach taken here is interesting because should the model be opened it’s still not dealignable because it just doesn’t know enough.
The slow warming of the water is a good approach. I’ve been using abliteration.ai’s cybersecurity models to pentest my own software and it produces much more usable defensive results. Opus 5.5 can fix everything but the guardrails trigger too easily.
The Fable ban affair was a nightmare for defensive cybersecurity. As far as I’m concerned we’re already in the world of Void Star and Cloudbreaker is here.
if you want to save some $ you can start with abliteration and after few turns switch to normal glm5.3 for example from z.ai subscription or other source. I didn't test it yet with deepseek, but I'm assuming it will will work the same.
those models will follow your request when they already agreed to do this
deepseek tends to quickly assume this is benchmark and even when have web access thinks things like "simulated internet" etc (but waste tokens on thinking if there could be way to cheat)
Are you sure providers allow you to submit modified assistant turns? Because, if they do, great! Last I heard most of that was neutered, with some providers (ahem anthropic) even signing everything and encrypting the "precious" thinking turns
ranyume 1 day ago
vdfs 20 hours ago
ranyume 16 hours ago
tom1337 20 hours ago
rocqua 19 hours ago
bryango 19 hours ago
ygjb 18 hours ago
FooBarWidget 16 hours ago
redanddead 7 hours ago
lxe 12 hours ago
Also the distillation attacks and other distillation methods that gather chain of thought information from frontier models are also used to generate vast amounts of data in order to generate synthetic datasets to post-train what I believe is the vast majority of models that are being released.
If you look at any information about how these models are trained, the details about how the data has been synthesized, where the agentic traces come from that are used in post-training, etc., they almost never tell you about exactly what model was used to generate them.
fny 13 hours ago
0x_rs 13 hours ago
https://x.com/stevibe/status/2026227392076018101
NortySpock 13 hours ago
pcwelder 20 hours ago
K3 for reference had 0.43 similarity to OpenAI's models, now it's as similar as how OpenAI's models are to each other at 0.58, much higher than other cross-lab similarities.
toddmorey 15 hours ago
It’s super tricky to tease out whether the similarities point towards distillation. A lot of the aesthetic similarities are naturally emergent from the training data.
XCSme 9 hours ago
https://aibenchy.com/compare/moonshotai-kimi-k3-max/moonshot...
viraptor 18 hours ago
olalonde 11 hours ago
ggcr 1 day ago
ranyume 16 hours ago
senordevnyc 12 hours ago
nomilk 1 day ago
ranyume 16 hours ago
senordevnyc 12 hours ago
Also, please don't gate-keep conjecture on the internet. It's a fool's errand.
ranyume 6 hours ago
selcuka 21 hours ago
cubefox 21 hours ago
cubefox 21 hours ago
victorbjorklund 19 hours ago
cubefox 17 hours ago
arjie 1 day ago
The slow warming of the water is a good approach. I’ve been using abliteration.ai’s cybersecurity models to pentest my own software and it produces much more usable defensive results. Opus 5.5 can fix everything but the guardrails trigger too easily.
The Fable ban affair was a nightmare for defensive cybersecurity. As far as I’m concerned we’re already in the world of Void Star and Cloudbreaker is here.
Szpadel 1 day ago
those models will follow your request when they already agreed to do this
deepseek tends to quickly assume this is benchmark and even when have web access thinks things like "simulated internet" etc (but waste tokens on thinking if there could be way to cheat)
londons_explore 21 hours ago
hirako2000 20 hours ago
112233 16 hours ago