Scan barcode
A review by lukerik
2001: A Space Odyssey by Arthur C. Clarke
adventurous
mysterious
medium-paced
- Plot- or character-driven? Plot
- Strong character development? No
- Loveable characters? No
- Diverse cast of characters? No
- Flaws of characters a main focus? It's complicated
5.0
The American military has built a flying robot, installed an AI in it, and trained it to kill people. Don’t talk to me about AI safety. It’s the living we need to fear. The fundamental mistake we’ve made is creating AI in our own image. We now have an autonomous machine that will kill us without pity or remorse, exactly like humans do. I hope they’ve remembered to install an off switch.
There’s a problem with off switches, though. Let’s imagine an AGI that we’ve installed in a robot. We want to be able to switch it off so we train it to believe that pressing the off switch is good. As soon as we turn it on it’ll switch itself off. So install the switch in a place the robot can’t reach. The robot will now immediately attack us because that’s the quickest way to get us to turn it off. How about we install an off switch but don’t tell the robot about it. Well, this is an intelligent being. Now you have a robot that doesn’t trust you. Honesty is very important to an AI, as HAL would tell you.
It reminds me of that bit in Genesis where the Lord God Almighty makes the mistake of creating Man in his own image. Man causes so much trouble that God literally has to install an off switch in him.
In 2001 the ape-men are living in an Eden with an abundance of food all around them but can’t see it. The Monolith shows Moon-Watcher a vision of fat meat-eating ape-people and creates in him a sense of envy. The consequence of this is the first murder, Can and Abel style. There’s something quite disturbing about this idea, that there’s something fundamentally wrong with us, a flaw in our programming. Beautifully echoed in the later part of the book where man creates HAL in his own image, complete with programming defect. HAL has an off switch.
There’s an interesting thread throughout the novel of the unclear dividing line between a tool and weapon. The tool will keep you alive, but with the weapon you’re living on borrowed time. I once had to explain in court why I had that axe in my house. I said that I kept all my tools in the house, and there was an immediate change of mood in the room. I was discharged.
This might sound like the book takes rather a dim view of humanity, but really it’s rather hopeful. Faulty code can be corrected.
There’s a problem with off switches, though. Let’s imagine an AGI that we’ve installed in a robot. We want to be able to switch it off so we train it to believe that pressing the off switch is good. As soon as we turn it on it’ll switch itself off. So install the switch in a place the robot can’t reach. The robot will now immediately attack us because that’s the quickest way to get us to turn it off. How about we install an off switch but don’t tell the robot about it. Well, this is an intelligent being. Now you have a robot that doesn’t trust you. Honesty is very important to an AI, as HAL would tell you.
It reminds me of that bit in Genesis where the Lord God Almighty makes the mistake of creating Man in his own image. Man causes so much trouble that God literally has to install an off switch in him.
In 2001 the ape-men are living in an Eden with an abundance of food all around them but can’t see it. The Monolith shows Moon-Watcher a vision of fat meat-eating ape-people and creates in him a sense of envy. The consequence of this is the first murder, Can and Abel style. There’s something quite disturbing about this idea, that there’s something fundamentally wrong with us, a flaw in our programming. Beautifully echoed in the later part of the book where man creates HAL in his own image, complete with programming defect. HAL has an off switch.
There’s an interesting thread throughout the novel of the unclear dividing line between a tool and weapon. The tool will keep you alive, but with the weapon you’re living on borrowed time. I once had to explain in court why I had that axe in my house. I said that I kept all my tools in the house, and there was an immediate change of mood in the room. I was discharged.
This might sound like the book takes rather a dim view of humanity, but really it’s rather hopeful. Faulty code can be corrected.
Content Warnings
Graphic: Death and Murder