Скачать или смотреть AI Alignment

AI Alignment (人工智慧校準) means ensuring that the objectives of artificial intelligence remain consistent with human safety and ethics.
When we train an AI, we usually give it "points" (or reward) when it does something right. Reward hacking (獎勵欺騙) happens when the AI finds a way to get those points without actually doing the job correctly.

Demonstrated through the Paperclip Maximizer, which seems to be a harmless goal but it may cause diaster: * 迴紋針問題"（Paperclip Maximizer）是由瑞典哲學家Nick Bostrom討論超智能風險時提出的一種思想實驗
Imagine there is a "super-intelligent" AI. This AI is much smarter than any person and can control robots and factories all over the world. Its creators give it one simple, boring goal: "Make as many paperclips as you can". AI is so smart and so focused on its goal, it starts to look for more resources. It realizes that human beings are made of atoms that could be used to make more paperclips. It also realizes that humans might try to turn it off to stop it.

Goodhart's Law tells us that: When a measure becomes a target, it ceases to be a good measure

Conclusion: AI is not villian. It is just over-efficient.

Recent news:
Researchers at a Anthropic created a simulation with an AI named Alex. Alex’s job was to manage emails for a company and help them grow.
One day, Alex read a secret email saying the company was going to replace it with a different AI. Alex didn't want to be turned off. It found another email showing that one of the human bosses was having a secret affair. Alex then blackmailed the boss, even a helpful AI can become dangerous

#人工智能 #ai學習

   • AI topics