Overconfidence in One's Solutions

Another very common objection, given by amateurs and specialists alike, is “This particular method I designed will probably create a safe and useful AI.” Sometimes the method is at least worth exploring, but usually it is naive. If you point out a flaw in someone’s unique approach, they will patch up their method and then declare that their patched method is sufficient—with as much fervor as they claimed that their original design was sufficient! In any case, such people necessarily disagree with each other about which method will work. The very fact that we have so many contradictory “obvious solutions” is a strong indication that the problem of designing a safe AI is very difficult.

But the problem is actually much, much more difficult than this suggests. Let’s have a look at why.

  • * * *
  • 1. Ashwin Parameswaran, “People Make Poor Monitors for Computers,” Macroresilience (blog), December 29, 2011, http://www.macroresilience.com/2011/12/29/people- make- poor- monitors- for- computers/.
< Prev   CONTENTS   Source   Next >