[Liberator]

Superior ability breeds superior ambition

I don't think this premise is sound.

This, it's usually the other way around actually.  The ones with the destructive levels of ambition are usually the ones that have a modicum of ability who have been told they are special or think they are smarter than everyone else and they get the big head.  I don't care who you are, there is always, ALWAYS, someone bigger, faster, stronger or smarter than you and you should behave accordingly.

[Mars]

I just don't see computers having ambition unless they're programmed to. Yes, we can give them self-preservation, yes we can make them fight our wars, but the fact is, they only have the drives we program into them.

[Topgun]

I just don't see computers having ambition unless they're programmed to. Yes, we can give them self-preservation, yes we can make them fight our wars, but the fact is, they only have the drives we program into them.

Mutation + natural selection will eventually take over.

[Liberator]

In a digital environment?  Where run exactly as intended unless programmed not to?  Computers can't mutate.  They have no mechanism for it.

[iamzack]

Unless we give them one.

[Herra Tohtori]

In a digital environment?  Where run exactly as intended unless programmed not to?  Computers can't mutate.  They have no mechanism for it.

Software can mutate if it's part of what it does.

http://en.wikipedia.org/wiki/Evolutionary_algorithm

Usually, evolutionary algorithms are applied by a program to find some optimized solution to a problem. Applying evolutionary algorithm to a program itself just requires that the program has a possibility to

a) reproduction
b) mutations and
c) selective process based on the success of each program in performing its task.

Reproduction can be either asexual or sexual process. Asexual reproduction means the program just copies itself over and over, while each copy may or may not get minute mutations to its code, and the mutated copies are then evaluated for their capability, and if deemed capable of performing their task, they are allowed to reproduce further, if not (a lethal mutation), that line "dies". Sexual reproduction is a much more optimized evolutionary algorithm, since it not only measures both the capability to perform the given task, but also its efficiency - the best programs would swap parts of their codome, resulting in faster and further optimization of the program routines.

Hell, you can even evolve simple programs out of scratch as long as you give the required building blocks and define the task that you want done.

For further information, watch this educational video:
Blind Clockmaker

 

[MR_T3D]

Three laws of robotics. Problem solved.

how did that turn out, again?
they creatively interpret the laws to imprison humans for their own safety, which, logically, would conclude with the creation of a matrix that would upload everyone's mind to a virtual, safe world they control, unless stopped by some guy.

[General Battuta]

Three laws of robotics. Problem solved.

how did that turn out, again?
they creatively interpret the laws to imprison humans for their own safety, which, logically, would conclude with the creation of a matrix that would upload everyone's mind to a virtual, safe world they control, unless stopped by some guy.

That was the movie. The movie was silly and had nothing to do with the book.

Of course the laws are still silly themselves.

[NGTM-1R]

Repeat after me: Do Not Give The AI Means To Physically Interact With The World

[General Battuta]

Repeat after me: Do Not Give The AI Means To Physically Interact With The World

Pretty much. Though even if it could, assuming a strong AI, it has nothing resembling a motor or sensory paradigm. It might be able to brute-force one, I suppose, but it's a long(er) shot.

[qazwsx]

inb4geth

[General Battuta]

inb4geth

we are all geth

[Scotty]

So does that make your mom a Geth Colossus?

* Scotty hides

But srsly, wouldn't it be simple enough to simply not include an evolutionary algorithm in any hypothetical AI?  Then it can't mutate outside the bounds of what we intend for it.

[General Battuta]

Any true AI would be capable of self-modification.

[Ghostavo]

Let's not make any hasty statements.

[General Battuta]

Let's not make any hasty statements.

If it's a true AI (in the strong AI 'replicates human intelligence' sense) it can presumably do anything a human can, namely, write code.

[Ghostavo]

Let's not make any hasty statements.

If it's a true AI (in the strong AI 'replicates human intelligence' sense) it can presumably do anything a human can, namely, write code.

Define intelligence.  

One of the problems in the field is that there is no consensus on what intelligence means. We have some idea of characteristics intelligent agents have, but there is no definition.

Another thing is what is a true AI? Any decision making program can be considered an AI. Are you saying some AIs are "truer" than others? Is Albert truer than Rybka?

Yet another problem, is with replicating human intelligence, aren't those terms are somewhat contradictory. Isn't rationality the supposed hallmark of an AI? Are humans rational? All open (or not so open) questions.

[General Battuta]

You're telling me things I already know and asking for a definition already presented. As a cognitive scientist I doubt designed-and-built strong AI is possible simply because of all the open questions you cited. However right in the post you quoted I postulated a strong AI that replicates human intelligence as a hypothetical.

[Ghostavo]

But not all learning programs use genetic algorithms, and either way, in the end they can't veer out of their goal.

A strong AI would most probably be very inhuman-like.

[General Battuta]

I didn't mention genetic algorithms. If the AI meets the criteria of strong AI, which are, I quote

artificial intelligence that matches or exceeds human intelligence — the intelligence of a machine that can successfully perform any intellectual task that a human being can

and if the AI was devised by humans, in most (but I'll concede not all) hypothetical designs for the AI the AI would then be able to modify itself.