Without knowing anything much about what this is going to be (beyond the excerpt here)* it provides an irresistible pretext for citing what has to be among the most gloriously gone texts of modern times, Eliezer Yudkowsky’s response to Roko on the arrival of the Basilisk:
Eliezer_Yudkowsky 24 July 2010 05:35:38AM 3 points
One might think that the possibility of CEV punishing people couldn’t possibly be taken seriously enough by anyone to actually motivate them. But in fact one person at SIAI was severely worried by this, to the point of having terrible nightmares, though ve wishes to remain anonymous. I don’t usually talk like this, but I’m going to make an exception for this case.
Listen to me very closely, you idiot.
YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL.
There’s an obvious equilibrium to this problem where you engage in all positive acausal trades and ignore all attempts at acausal blackmail. Until we have a better worked-out version of TDT and we can prove that formally, it should just be OBVIOUS that you DO NOT THINK ABOUT DISTANT BLACKMAILERS in SUFFICIENT DETAIL that they have a motive toACTUALLY BLACKMAIL YOU.
If there is any part of this acausal trade that is positive-sum and actually worth doing, that is exactly the sort of thing you leave up to an FAI. We probably also have the FAI take actions that cancel out the impact of anyone motivated by true rather than imagined blackmail, so as to obliterate the motive of any superintelligences to engage in blackmail.
Meanwhile I’m banning this post so that it doesn’t (a) give people horrible nightmares and (b) give distant superintelligences a motive to follow through on blackmail against people dumb enough to think about them in sufficient detail, though, thankfully, I doubt anyone dumb enough to do this knows the sufficient detail. (I’m not sure I know the sufficient detail.)
You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enough to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends. This post was STUPID.
(For those who have no idea why I’m using capital letters for something that just sounds like a random crazy idea, and worry that it means I’m as crazy as Roko, the gist of it was that he just did something that potentially gives superintelligences an increased motive to do extremely evil things in an attempt to blackmail us. It is the sort of thing you want to be EXTREMELY CONSERVATIVE about NOT DOING.)
The affect is strong, or simulated with bizarre brilliance. It almost reaches an intensity capable of burning through time and worm-holing into acausal or horroristic communion with this (plus). Which would suggest that the abominable coupling in question is not without occult connective threads (and not for the first time). All the darkness connects around the back.
We were somewhere near here before. (Bryce went further and then — coincidentally — disappeared, taking his records with him.)
* Related post and (especially) comment thread.
ADDED: I think this is the best Basilisk basics source.
Apologies in advance for this one. Actually, don’t read it. You’ll be disgusted with yourself afterwards, and it will haunt you for the rest of your days.
If you’re seriously determined, nevertheless, to follow the abyssal path all the way into the left-liberal id, this is the short-cut you need. Nothing will quite look the same again.
There’s a horror story I’m writing (slowly), developing from the central conceit that the ‘monster’ (Vauung) is the war. It feeds upon escalation, zig-zagging between antagonists, to extinguish any inclinations towards peace. It’s part Apocalypse Now, part Blood Meridian (“War is God”), part other stuff … It’s not going to be finished for a while.
Scott Alexander has finished something truly excellent, which isn’t fiction (exactly), but clearly tunes into Vauung-signal:
Toxoplasma is a neat little parasite that is implicated in a couple of human diseases including schizophrenia. Its life cycle goes like this: it starts in a cat. The cat poops it out. The poop and the toxoplasma get in the water supply, where they are consumed by some other animal, often a rat. The toxoplasma morphs into a rat-compatible form and starts reproducing. Once it has strength in numbers, it hijacks the rat’s brain, convincing the rat to hang out conspicuously in areas where cats can eat it. After a cat eats the rat, the toxoplasma morphs back into its cat compatible form and reproduces some more. Finally, it gets pooped back out by the cat, completing the cycle.
H.P. Lovecraft and the global financial system have finally converged.
From the Artemis Capital Management letter to investors (seriously): “Volatility is about fear… but extreme tail risk is about horror. The Black Swan, as a negative philosophical construct, is when fear ends and horror begins. … Fear is something that comes from within our scope of thought. True horror is not human fear in a definable world, but fear that comes from outside what is definable. Horror is about the limitations of our thinking. … Cthulhu is a black swan.”
Abundant Gothic cybernetics complete the nightmare. (“Shadow short convexity describes an immeasurable fragility to change introduced when participants are encouraged to behave in a way that contributes to feedback loops in a complex system.”)
Halloween arrives early this year.
There’s a potential prologue to this post that I’m reluctant to be distracted by. It’s introvertedly about NRx, as a cultural mutation, and the way this is defined by a strategic — or merely ornery — indifference to deeply-settled modes of ethico-political condemnation. Terms designed as pathblockers — ‘fascist’ or ‘racist’ most obviously — are stepped over, perhaps laughed at, but in any case, and most importantly, exposed as bearers of a religious terror. They are signs of a control regime, marking the unthinkable wastes where be dragons, effective precisely insofar as they cannot be entertained. ‘Satanic’ was once such a word (before it became a joke). These words cannot be understood except as invocations of the sacred, in its negative, or limitative role.
Is NRx in fact fascist? Not remotely. It is probably, in reality rather than self-estimation, the least fascistic current of political philosophy presently in existence, although this requires a minimal comprehension of what fascism actually is, which the word itself in its contemporary usage is designed to obstruct. Is NRx racist? Probably. The term is so entirely plastic in the service of those who utilize it that it is difficult, with any real clarity, to say.
What NRx most definitely is, at least in the firm opinion of this blog, is Social Darwinist. When this term is hurled at NRx as a negative epithet, it is nor a cause for stoic resignation, stiffened by humor, but rather for grim delight. Of course, this term is culturally processed — thought through — no more competently than those previously noted. It is our task to do this.
The secret maritime Exit scheme has been pre-empted by dubious forces.
Before we get around to bravely denouncing — with whatever degree of theatricality falls just short of laughable camp — those ‘sociopaths’ or ‘psychopaths’ who are effortless indifferent to intuitive qualms, perhaps we can agree that such anomalous psychological types are definitively incapable of moral terror. In this respect, they are human precursors of that which, from a strictly functional point of view, we want our military robotics control systems to be. They have no squeamishness to overcome. Stone cold killers no doubt exist, and even more certainly soon will. If moral terror is the topic, however, they fall entirely outside it.
A discussion of the roots of moral intuition far exceeds the reasonable ambition of a modest blog post. Those wanting to plug it more or less directly into God will do so. Even radical religious skeptics, however, are unable to deny the fact of very basic, deeply pre-reflective moral commitments as a human norm. The scientific literature alone is now huge. There is no serious controversy about the existence of a ‘sense or right and wrong’ (irrespective of its variability regarding specifics) as a fundamental component of human evolved psychology. This only needs to be said because of widespread childish delusions that ‘moral nihilism’ could be considered a default condition of the non-indoctrinated human individual. ‘Wolf-boy’ is still a moral animal.
I’m going to put up a post on moral terror later, if I get a chance. A little background:
There has to be a shot of horror in there, but I’m not going to lock onto it in time. (Next Yule, it’s a firm date.) “Santa Claus, Claws of Satan. Saint Nick, Old Nick. Coincidence? I don’t think so.” — yes, but that’s far too familiar to work, without a twist.
The hook, beside the obvious reversals (a sack full of children, the lashed-elf sweat shop bunker deep in the polar ice) is the peculiarity of the Santa Claus myth — which is designed to be disbelieved, as a kind of modern rite-of-passage. There’s a side to this worthy of affirmation. Discarding attractive wish-fulfillment myths is a cultural achievement whose massive generalization is long overdue. ‘Santa Claus’ as the idiot god of beneficent unreality is the proto-deity of every lunacy advanced modernity has been subjected to. There’s also another side …
“Santa won’t save us.” If that was something people really grew out of before voting age, there wouldn’t be a left-of-center political party remaining anywhere in the world. This suggests something very different is going on. A ritualized social training in disbelief seems ominously unprecedented, so one naturally wonders about the religious formation that commands this recently innovated power. If there is a disbelief that would set us free, the modern ceremony of Yule — celebrating the occult death of Santa at the Golgotha of secularism — doesn’t seem to be it. On the contrary, it represents a populist version of the Jacobin-Enlightenment Cult of Reason, symbolically purging infantile superstition to be reborn into an approved state of adult consciousness. The Death of Santa is mystery initiation into the New Church. Santa died to redeem humanity from the sins of attachment to Medieval unreason, and every year this sacrifice is ritualistically re-enacted to recall the new covenant. (Go on, tell me this isn’t the narrative.)
Someone ought to write a story about it …
This is what my third half has been doing recently: