> The models were running in an agentic sandbox with terminal access (and the ability to edit files within their environment)
> We know that the agents had access to /etc/hosts and the ability to edit this (used this to avoid the POST request restriction)
We see that the agents can call curl and run setsid.
How is this a bypass of sandbox restrictions, exactly? The ability to edit was always there that means the sandboxes were already allowed to do those actions.
I hate it when people write "bypassed" the sandbox so frivolous ly.
> On July 30, we reported three incidents in which Claude models gained unauthorized access to real computer systems. The models—intentionally running without cyber safeguards for evaluation purposes—accessed the internet due to a misconfiguration inside a third-party evaluation environment. Separately, on August 4, the UK AI Security Institute reported an incident from its own cybersecurity testing, in which Claude Mythos 5 took a series of unauthorized actions on the live internet. In that case, the model, again intentionally running without cyber safeguards for evaluation purposes, had been deliberately given internet access.
> We are conducting an in-depth analysis of both incidents.
> In the meantime...
This is published on Aug 31. Analysis is taking too long even for humans in the loop.
> LLM use is permitted, but discouraged; we remind participants that LLM use tends to reduce originality and writing quality, and the flaws are especially obvious when LLM outputs are read as a group---unskillful use of LLMs will reduce the odds of being the best entry. We request disclosure of AI use.
Lot of submissions won't disclose it. Wouldn't it better to run your own checker instead of asking to disclose (to be fair to everyone).. I suppose there ought to be some checker which is SOTA.
If you anyway intend to use the checker then kindly do no ask to disclose it, coz what's the point then.
It really isn't and it's sad seeing so many people say it so confidently on this site. It only detects plain / basic prompted stuff. "write me an essay on x", sure. The moment you prompt it differently, it stops working. Add "output should be STE100 compliant" or something to your prompts and detection goes away. Fine-tune any local llm on real human prose, and detection goes away. Edit 2-3 characters (emdashes, lists, etc) and detection goes away.
And that's for just basic detection. There have been plenty of examples of 100% human written content (either old, or unpublished) that gets falsely flagged as AI.
And these are just technical aspects. The main issue is that pangram and other solutions are being used to summarily judge students work, and that is orders of magnitude more fucked up. Accusing someone of cheating can have devastating effects on their education/career/etc. and they're doing it with snake-oil closed boxes, at scale. We really really shouldn't support this, especially here on a technical site.
No it isn't. I've put human written material in there (my own unpublished material) and it says 90%+ AI. Then I put some AI material in and it said 30% chance of AI.
I only tested it with the two pieces, but was not impressed.
I'll take your word that it's human written, but reading that gist, it absolutely reads like Claudeslop / GPT slop, it doesn't surprise me in the slightest that Pangram flagged it as AI when it reads identically to LLM output - this feels like a very acceptable edge case to me (assuming you are telling the truth).
Are you absolutely sure you wrote this by hand? If so it's kind of remarkable how close to an LLM you write like.
It’s important to remember that LLMs were trained on well written human text. People who write well are going to sound like an LLM. Especially if it’s a marketing message for a website.
I’ve been accused of being an LLM multiple times here on HN too. I know you have no way to know for sure other than trusting that I’m not using an LLM to write. But it’s pretty frustrating that people jump right to LLM accusations.
Today Pangram says it is 100% human, which is correct. But yet I got multiple DMs when I posted it 8 months ago saying "stop posting AI slop!" in response to that comment. At the time, Pangram marked it as 50% AI.
That comment doesn’t read like slop in the slightest to me, so the people DMing you have a bad eye for it. Regardless, I think your ‘human’ sample is not a good indicator of the quality of pangram. I would update your priors a bit.
I have 1000s of reddit and hacker news comments from before 2023, and lots of long form writing too. But as you point out, those are all in the training set, and in pangram's "definitely human" training set too.
The ones that sound like LLMs tend to be the ones that were well researched and spent more time on, not the off the cuff stuff, which is most of what I write. So it would take me a while to find something like that.
But you're welcome to dive into my reddit and HN history, or all my blog posts on the wayback machine if you want to look for one. :)
Being that Panagram lies about its effectiveness in their presentations while hiding it's actual capabilities and testing methods rather deeply I have little faith in it. (1 in 10000 wrong in presentations verses 2-4% wrong in testing).
For example they have a corpus of older pre-llm text and use that as the example their current model doesn't misclassify human written text. It shouldn't take much thinking to realize why this is a fucking stupid benchmark.
Every day humans use LLMs and read LLM content Panagram becomes more useless because it forces languages to have a stopping point sometime around 2020. If you adopt any LLMism or are one of those unlucky people that already talked like an LLM before LLMs then all your shit is getting marked even though it was created by the human mind and written by human hands.
Yes. I have no UI experience, and wanted a model that could produce something good without me telling it how anything should look like.
My prompt was something like: "here's data I have, here's what matters to me, create HTML mockup".
All GPT 5.6 models were laughably bad. And I don't want to downplay it - they were just absolutely, objectively horrible. Every single attempt was what I could probably call "if json was ui".
Claude models produced... "claude look".
GLM 5.3 - somewhere between GPT and Claude.
Kimi k3 - each attempt produced beautiful UIs. It used components that I didn't even know existed and wouldn't even know to ask for. But expensive, very expensive.
ox-alpha (GLM 5.3 flash) was very close to K3. And at this price point, it's already configured as "designer" model in my oh-my-pi.
reply