> I don't know why you state this as if it's evidence against the concerns lol. Someone being concerned about the incentives of a situation doesn't de facto make them immune to those incentives, obviously.
I think you're reading some subtext into my comment that I didn't intend. Knowing myself, I assume the scare quotes there are just a bit of casual irony re: the insanely high stakes here. The word "concerns" as used by previous commenters doesn't seem equal to the context.
> The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are.
You can, in fact, opt out. You can opt out and do your damndest to stop what's happening, throw every cent you have at it, bend any ear that will listen, make use of the fact that your voice (as Anthropic leadership) has some meaningful weight.
If you really believe that we are heading down a path that's likely to end poorly for most or all of humanity, and you are the kind of person who's inclined to favor saving billions of lives over saving your own skin when the stakes are still relatively distant, abstract, and generally unclear, opting out is obviously on the table as a grand gesture that burns your position in the race to show just how fucking serious you are. The sense of inevitability your comment shares with many others does not seem well founded---we have, for instance, not had a global nuclear war yet. Leaders in the 20th and 21st centuries have shown remarkable restraint.
If today's political and tech leaders are unable to think beyond this inevitability, for whatever reason, the worst outcomes essentially become a self-fulfilling prophecy to the extent that reality bears them out.
---
But yes, these people are acting the way they are for obvious reasons, obviously. My previous comment is reacting to the general disagreement over whether Anthropic actually believes what they say about safety, etc., or whether it's a marketing gimmick. The purpose of my comment is to explain that "it's hard not to be cynical" about actions taken by very rich and powerful people that are claimed to be in everybody's best interests but are indistinguishable from the actions they would take to maximize their future power and wealth. I think everyone ought to agree with that statement. It's not a value judgment; it's simply an observation of how it feels to be on a plane whose pilot appears to be robbing the passengers (including you) at gunpoint and is conspicuously wearing the only parachute on board.
It's not just America. The main secret is out of the bag. If it wasn't Anthropic, it would be another company/nation state. Sure they could obtain, and with not.money or leverage, complain about data centers at local rallies, or they can be in the game, and hopefully steer it. It's going to happen with or without any one company or country. The secret it out, and it's unstoppable without complete societal breakdown..So either you advocate for the end of civilization, or you hope that you can help steer the emergence of super intelligence into something not wholly terrible.
Personally, I don't see much hope, even if there wasn't such a thing as AI. The power to destroy is always easier than the power to create, and as our power grows, the differential grows, until at some point, containment is no longer possible
I'll mention again the nuclear analogy. It is, believe it or not, possible for great powers, and even adversary great powers, to agree to limit the development and proliferation of dangerous technologies.
> The main secret is out of the bag.
This is not something you can do in a shed with a handful of GPUs just because you know "the main secret". To build something like Mythos you need tens of billions of dollars, massive amounts of power, enormous buildings filled with specialized bleeding edge computer chips that are made by (optimistically) a handful of companies with deep government ties. You need free access to all the intellectual property that humans have created and posted openly on the internet. You need all of this at each step, and to take each next step you (or somebody) needs to have taken the previous one.
For now, there are a million ways for a government to pump the brakes on this cycle.
Yeah, I think the nuclear analogy fails, honestly. The bomb does one thing: Destroy. AI can build, selectively infect, selectively manipulate. It is a vastly useful tool.
What is the difference? It's easier to make money with the AI you get at each incremental step toward potentially destroying human civilization (though, of course, it's debatable whether these companies really are making money as such).
So what? You are implicitly arguing that human civilization will be unable to resist engaging in a large-scale, coordinated effort to destroy itself, just to make a few bucks along the way. Is this true? I don't know. The point of the nuclear analogy is that we have previously shown that we can, under certain conditions, put the eschaton back on the shelf for some period of time, despite very real pressure to take more incremental steps toward doom. "But AI can write code" is not a refutation of the possibility that we could take a more measured approach to AI development.
I never claimed to be precise, ha ha. But let's not lose sight that each of the great powers agreed to limits on the number of nukes only after making enough to collectively destroy each other several times over. They stopped exactly when building more did not gain them any more security or advantage over their adversaries. Not a moment before that.
There may or may not be such a point with AI: A point at which ever smarter machines provides no marginal benefit to security. If that happens, I do expect agreements, should any one of us still exist.
> It is, believe it or not, possible for great powers, and even adversary great powers, to agree to limit the development and proliferation of dangerous technologies.
You were literally just criticizing Anthropic as disingenuous for begging for this. Or is your position that people other than those near the front of the race can agree to limit development? And if so: provide evidence.
Note also a key ingredient that makes nuclear non-proliferation possible is that they're pretty much useless weapons. There is no smaller order nuke that's dramatically more useful than a large conventional weapon. That's not true of AI models, which appear to be monotonically useful as they become more powerful.
> You can, in fact, opt out. You can opt out and do your damndest to stop what's happening, throw every cent you have at it, bend any ear that will listen, make use of the fact that your voice (as Anthropic leadership) has some meaningful weight.
There are billions of people who have opted out of playing the game. Has the game stopped? Has any game stopped because the people not playing it decided that it ought to? Only with government intervention, which is exactly what you just criticized Anthropic for being disingenuous for requesting.
Is your position that they should just be smart bloggers asking for regulation, instead of the preeminent lab asking for regulation, and that would be either more ethical or more effective? If it's less effective, isn't it de facto less ethical?
What say you about the thousands of smart bloggers asking for regulation who are ignored every single day and have no tools besides their blogs to steer the outcome?
> burn your position in the race to show just how fucking serious you are.
This is incredibly naive. Literally no one who is unconvinced of AI doom would be convinced by this... because they already don't believe the premise. Such a gesture would be readily explained away as "you were losing the race," or "you got rich enough already." This is the attitude when any individual opts out of participation (see: Hinton) and it's ridiculous to assume it'd be different if an entire company did it.
Not to mention, that an entire company can't do it. These companies have boards of directors. They are accountable to shareholders. A CEO who wanted to do this would simply be fired and the company would carry on. This is one of the key components of the trap. Large companies are not under the control of people but of incentives. They are literally deliberately designed not to be under the control of individuals –– to be immune to exactly the type of behavior you think is possible.
And yes, nuclear weapons are analogous to AI in the arms race dynamic to create and proliferate them. They are probably not analogous in there exists a stable equilibrium in nuclear weapons due to "accidents" of their nature. There need not be a similar equilibrium among competitive AIs.
----
And yes, your comment lands in exactly the category I mentioned. You do not believe the AI doom fears, so the behavior looks one way. I do believe in the AI doom concerns, so the behavior looks another way. This applies equally to yesterday's actions, today's actions, and tomorrow's, including some hypothetical honorable self-immolation to slow progress: I would see that as concordant with AI doom concerns, you would not. You would find it "hard not to be cynical" about the fact they already earned a ton of money, maybe was losing a bit of ground in the race, so on and so forth. This is plainly obvious to anyone who has had to converse with no-doomers, who can only analyze other people's behaviors under their own belief system, so it won't happen.
The only variable of disagreement is around AI doom.
(I wrote a longer comment originally, but I think it would have fallen on deaf ears.)
> The only variable of disagreement is around AI doom.
The source of our disagreement seems to be your belief that somebody can either a) believe "AI doom" is inevitable, or b) not believe it's possible. This is an obvious false dichotomy that's stunting your ability to engage effectively with what I've written, and also stunting your ability to understand the broader landscape of the issue. You are projecting this dichotomy onto everybody involved and understanding their behavior in that way, which is leading you to make other reductive and honestly bizarre claims—like, for instance, the idea that a sudden change of course from Dario Amodei at this moment in time would be broadly perceived as somebody who was already losing the race cashing out his chips. If you really do believe that, I have to assume it's because you're modeling your hypothetical observers as falling into one of your two extreme mindsets and assuming Dario, being a smart guy who knows a lot, thinks the same way. I believe it is—yes, I'm ready for the dopamine hit—naive to assume all people or even most people fall into one of these two camps. Naive, at best. Yours is a self-limiting framework for thinking about this stuff.
I encourage you to broaden your thinking and engage in less projection and ad hominem stuff in discussions like this. I probably won't reply to whatever you post next unless you can do a better job writing a substantive reply to what I've written here.
Then surely you can articulate how specifically – in a way that doesn't require large numbers of people acting against their own incentives – that AI doom is possible but not guaranteed.
I actually already know how you'd perceive Dario's opting out of the race because I already know how you perceive Dario's requests for regulation, which is the milder version of the same logic, and is vulnerable to the same cynical allegations of self-serving, which you've already expressed.
I think you're reading some subtext into my comment that I didn't intend. Knowing myself, I assume the scare quotes there are just a bit of casual irony re: the insanely high stakes here. The word "concerns" as used by previous commenters doesn't seem equal to the context.
> The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are.
You can, in fact, opt out. You can opt out and do your damndest to stop what's happening, throw every cent you have at it, bend any ear that will listen, make use of the fact that your voice (as Anthropic leadership) has some meaningful weight.
If you really believe that we are heading down a path that's likely to end poorly for most or all of humanity, and you are the kind of person who's inclined to favor saving billions of lives over saving your own skin when the stakes are still relatively distant, abstract, and generally unclear, opting out is obviously on the table as a grand gesture that burns your position in the race to show just how fucking serious you are. The sense of inevitability your comment shares with many others does not seem well founded---we have, for instance, not had a global nuclear war yet. Leaders in the 20th and 21st centuries have shown remarkable restraint.
If today's political and tech leaders are unable to think beyond this inevitability, for whatever reason, the worst outcomes essentially become a self-fulfilling prophecy to the extent that reality bears them out.
---
But yes, these people are acting the way they are for obvious reasons, obviously. My previous comment is reacting to the general disagreement over whether Anthropic actually believes what they say about safety, etc., or whether it's a marketing gimmick. The purpose of my comment is to explain that "it's hard not to be cynical" about actions taken by very rich and powerful people that are claimed to be in everybody's best interests but are indistinguishable from the actions they would take to maximize their future power and wealth. I think everyone ought to agree with that statement. It's not a value judgment; it's simply an observation of how it feels to be on a plane whose pilot appears to be robbing the passengers (including you) at gunpoint and is conspicuously wearing the only parachute on board.