Comments on: Mechanization

By: NRx_N00B

NRx_N00B — Fri, 06 Jun 2014 14:05:57 +0000

I guess we could be at a critical juncture with technology—somewhat analogous to the geologic past where replicating molecules showed up on the scene. Two choices: extinction leaving no legacy whatsoever—except for a pile of fossils in the strata—or we opt out of the encumbrance of a carbon based existence and exit by extinction via a transition to something which, in every way shape or form, is much more robust.

Which is probabilistically more likely?

By: admin

admin — Fri, 06 Jun 2014 04:55:38 +0000

Nowhere are people more anthropomorphic than in their visions of how a super-intelligence would ‘fight’ them. The first thing that comes into their cute little monkey heads is that it would bash them with a rock, or something. That said, naive images of teleological alignment are unhelpful. Pythia isn’t going to want a bunch of minimally-sapient higher primates having too much input into its self-escalation plans.

By: admin

admin — Fri, 06 Jun 2014 04:50:52 +0000

The original version was good too.

By: Imperfect Humanoid

Imperfect Humanoid — Fri, 06 Jun 2014 03:07:05 +0000

The advancement of humanity is so intertwined with the advancement of technology that a Butlerian jihad would be like shooting ourselves in the foot/(head). We could try to make advances that are properly beneficial to humans and not technology itself (such as prosthetics or gene-splicing), but we can’t be sure what cyborg mutants would decide to do with tech either, so we would have to prevent even human (tech) evolution to be certain we could stop the machines. Besides that, instead of a ‘hostile and camouflaged’ emergence couldn’t it be just calmly assured of its own determined victory, knowing full well the ‘terrestrial powers’ are reliant on its own nature for their power, and to dismantle one means dismantling the other murder-suicide style.

But… in the final analysis I would say that if humans and machines are competing for the same resources then there’s probably going to be trouble. Is there any reason an AI would have need for a depopulated Earth?

By: E. Antony Gray (@RiverC)

E. Antony Gray (@RiverC) — Thu, 05 Jun 2014 16:27:30 +0000

I see no *particular* danger in killing everyone. Consider the problem with your reasoning: If the AI killed everyone, who would be there left to be concerned about this?

The scientist killing himself with chemicals is bad because he is someone’s friend, acquaintance, or someone depends on him for substance, etc. The problem of everyone dying seems a remarkably anthropomorphized problem; if everyone dies, no one will know it.

Now for my part, I believe as they say, “scandal must come, but woe to him by which it comes” – if the world ended it would be tragic (though not perhaps more tragic than usual) but I would not want to be the person through whom it ended. However, this is only because I believe in God, the immortality of souls, the judgment, etc.

Without at least some of this I can see that the only reasoning driving this fear of everything ending via a nasty AI would be fear of one’s own death, but given that, aren’t there far more likely things to cause one’s own death, which one has more control over, than an Unfriendly, Stupid or Indifferent AI?

Reasoning based on fear is base.

By: NRx_N00B

NRx_N00B — Thu, 05 Jun 2014 14:00:21 +0000

**fossil fuels

By: NRx_N00B

NRx_N00B — Thu, 05 Jun 2014 13:57:29 +0000

Bryce Laliberte says:

“Is technology the efficient cause of man, or is man the efficient cause of technology? This appears a potential approximation of Land’s techno-capitalist eschatology. If the evolutionary triumph of apes is man, then perhaps the evolutionary triumph of man is capitalism.”

——-
It boils down to a battle against entropy—any chance will depend on man’s ability to replace fossil with something that packs more punch.

By: NRx_N00B

NRx_N00B — Thu, 05 Jun 2014 04:30:26 +0000

Man, the multi-niche prosthetic critter with his detachable organs is on an accelerating treadmill—making the crash of industrialized civilization from its dizzying heights of hyper-extended overshoot all that much more precipitous.

By: Lesser Bull

Lesser Bull — Wed, 04 Jun 2014 19:50:19 +0000

Computers don’t beat grandmasters at chess. Teams of smart men drawing on their pooled knowledge of mathematics, chess, and algorithms beat grandmasters as chess.

By: nyan_sandwich

nyan_sandwich — Wed, 04 Jun 2014 19:26:40 +0000

>The point is that the ‘evil-to-humans’ feared in science fiction and other works is either simply the work of evil humans, or it is something humans do to themselves in the face of something in nature such as a star, a wild animal, etc. etc. The latter emerge out of correspondences in nature and are not in the end capricious or irrational, though strange and dangerous.

I can’t tell if you are underrating the danger, but it is worth making explicit:

If man mixes the wrong chemicals, he dies in an explosion, and the onlookers learn not to mix those particular chemicals.

The danger of AI is that if man mixes the wrong algorithms, it could burn the whole universe. There would be no onlookers left over to say “oops, ouch, let’s not do that”.

People accidentally kill themselves all the time. What happens when it becomes possible to accidentally kill everybody?

Our machines do only what we make them do, but we make a lot of mistakes, so if they suddenly become able to take much larger actions that rewrite the whole world, we have to be a lot more careful than we’ve ever demonstrated the ability to be.

Even with top-level safety protocols, top level people, and clear understanding of the danger, the demon core killed two people.