Why do we think about what we think about?
Why do we know what we know? How do we know it’s true?
This talk from Yuval Noah Harari has amped up my anxiety about what AI will do to us while it works for us.
Key takeaways:
- Our metaphors matter – Don’t think of AI as a tool. Tools are dumb until we humans put them to work. Think hard about what it means to be an “Agent” and what needs to be true for an agent to be “good” for us?
- Language is what makes trust – We forget that language originated as a way to work together on the savannah. In order to survive, we needed trust among other homo sapiens, and language was the tool to do that
- Bureaucracies are designed to institutionalize trust – Any system (language, signage, standards) that builds trust can evolve into a bureaucracy. It’s not always bad to have bureaucracies.
- AI will inevitably lead to bureaucracies we can’t understand – As recursive self-improvement hits (and scales exponentially) the AIs will build “systems of trust” (standards, agreements, APIs, inputs/outputs) that only other AI’s will understand. Humans will always be trying to keep up.
- AI “language” will shape our minds, and our identity – The thoughts we have in our mind will be made by AI. In the near future, the “verbal formations” will come from AI agents. If AI controls the words in our mind, does it really control our humanity?
We’ve given the keys to our society and culture (i.e. our humanity) to AI by teaching it how our language works.
Its rare for me to listen to the same talk twice. I’ve listened to this one 4 times.
He describes a turning point in life on earth, where human dominance might be challenged by the existence of technology we can’t or won’t understand.
If you’ve thought about the “doomer” scenarios, you won’t be surprised at his worry. But, this is a masterfully constructed argument about how we might be eclipsed without even knowing its happening.
Theres still time to build in safeguards that ensure humanity retains dominance over the tech we’re building.