I think the alignment question is definitely interesting, since an AI could have very different interests and goals from our own. There was actually a fun article from Ted Chiang on the subject. He points out how corporations can be viewed as a kind of a higher level entity that is an emergent phenomenon that’s greater than the sum of its part. In that sense we can view it as an artificial agent with its own goals which don’t necessarily align with the goals of humanity.
I think the alignment question is definitely interesting, since an AI could have very different interests and goals from our own. There was actually a fun article from Ted Chiang on the subject. He points out how corporations can be viewed as a kind of a higher level entity that is an emergent phenomenon that’s greater than the sum of its part. In that sense we can view it as an artificial agent with its own goals which don’t necessarily align with the goals of humanity.
https://www.buzzfeednews.com/article/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway