Talk:Progress in artificial intelligence

Excuse me, but why are all of these board games listed? There is no explanation for what they are supposed to represent. It is useless information for the article. Contralya (talk) 17:33, 28 April 2008 (UTC)

The article shows problems that have been solved to varying degrees by artificial intelligence researchers and indicates the level of performance one can achieve with an AI algorithm. Board games are well-defined problems and so are easier to tackle than more open-ended problems (generally speaking). While the results might appear useless to some they have required advances in the theory and practice of AI and are important in that respect. You have to walk before you can run. Pgr94 (talk) 17:46, 28 April 2008 (UTC)

Machine translation

How should machine translation be classified? People knowing a second language often perform better than machine translation systems but the average person might be less capable. Also, if we take into account multiple languages, machine translation systems like google translate would significantly outperform most humans. How many people can translate from arabic -> polish and french -> swahili and armenian -> vietnamese. Over several languages machines win hands-down. pgr94 (talk) 22:22, 26 May 2010 (UTC)

I think it should stay where it is might it could use a comment noting machines higher versitality in the field as you noted? —Preceding unsigned comment added by 83.250.36.199 (talk) 19:09, 1 August 2010 (UTC)

I added a comment to the article as suggested. Feel free to improve on it. pgr94 (talk) 22:27, 1 August 2010 (UTC)

Rubics Cube

There has been some back and fourth editing regarding rubics cube and there seems to be a disperancy betwen users regarding what solving rubics cube constitutes to.

Since the optimal solution, aka the one requireing the least ammount of moves from every starting position is not yet known, i would list the game as strong super human but not optimal, as currently is. —Preceding unsigned comment added by 83.250.36.199 (talk) 19:02, 1 August 2010 (UTC)

In Optimal solutions for Rubik's Cube it says: "In 1997 Richard Korf[5] announced an algorithm with which he had optimally solved random instances of the cube." Are you suggesting that this is wrong or that it is optimal only for some cube starting positions? pgr94 (talk) 22:15, 1 August 2010 (UTC)

Classification

In an edit comment Charles Gillingham asked who developed the 5-level classification:

optimal: it is not possible to perform better

strong super-human: performs better than all humans

super-human: performs better than most humans

par-human: performs similarly to most humans

sub-human: performs worse than most humans

The short answer is, I did. However, the levels are explicitly mentioned in the various articles cited. I suspect we won't find a single reference that mentions them all, but they are there in the literature. My edits may be running a bit close to WP:SYN, so I'm open to other opinions on the matter. pgr94 (talk) 09:57, 30 January 2011 (UTC)

I suppose you could argue that it's commonsense. It's certainly not some egregious violation of WP:SYN, in my view anyway. Thanks for being so straight forward about it. I only noticed it because I'm going through the AI article and redoing the citations and along the way I ran into a few topics that were missing a citation and this was one of them. I'll leave the tag in AI and remove it here. ---- CharlesGillingham (talk) 05:48, 31 January 2011 (UTC)

Game shows

Today Watson beat the two Jeopardy champions. It is well above merely "par-human" —Preceding unsigned comment added by 173.88.249.116 (talk) 01:03, 17 February 2011 (UTC)

Watson was also beaten by a U.S. congressman: http://www.datacenterknowledge.com/archives/2011/03/01/congressman-defeats-ibms-watson-in-jeopardy/. It also doesn't do speech recognition, so it plays with an unfair advantage against human players. Admittedly, the congressman is a former Jeopardy champion. But should we be citing a couple Watson victories to support Watson's playing at superhuman level? I figure the claim is probably right, so I'm not removing it, but it would be nice to see a source that talks explicitly about fair comparisons between Watson's play and most human play. However, here are two other bases for doubt: (1) I've heard (only by word of mouth) that Watson can be beaten by an average human who knows how to exploit a couple weaknesses in Watson's play; (2) Watson depends heavily on human prepping, so some claim it might be better to call it a supplement to human intelligence and not really artificial intelligence. Wikipedia really shouldn't be deciding this question. —Ben Kovitz (talk) 22:49, 7 August 2011 (UTC)

Go citation

The citation for computers being par-human at go is wikipedia itself. Is this acceptable? (could not find the wikipedia guideline on this)

No, such a citation is not acceptable: WP:CIRCULAR. I just removed it. The claim is clearly true, but I wasn't able to (quickly) find a source that supports it directly. Can you find one? —Ben Kovitz (talk) 22:27, 7 August 2011 (UTC)

Driving cars

At the moment driverless cars are listed both in the section "Super-human" ("Driving a car: super-human. Google driverless cars are safer and smoother when steering themselves than when a human takes the wheel.") and "Sub-human" ("Autonomous driverless cars"). One of the two has to be wrong? 91.229.57.240 (talk) 18:05, 2 December 2013 (UTC)

I'm pretty sure anyone familiar with the state of the art would agree that autonomous cars are sub-human at this point. For instance, they are not yet able to drive safely in inclement weather. And the fact is that Google's cars still have to hand over the controls to humans when they're not sure what to do. They do by design drive much more conservatively than most humans, which is enough to explain the claims in the citation. Yak314 (talk) 13:51, 11 February 2016 (UTC)

I moved Safely driving a car from sub human to super human. Waymo has driven more than a million miles since January 2018 without causing a single accident. The average driver has caused many more accidents in a million miles. --Mschribr (talk) 18:41, 26 May 2018 (UTC)

Driving a million miles without causing an accident isn't superhuman, especially if we're not counting accidents the driver didn't cause. I moved it to par-human for now and marked caution around this claim as it has not yet been definitively proven. We need a reliable source will state that a particular car is safer on average than human driving in some domain to move it to "high human"; forbes sites blogs aren't generally considered reliable sources on Wikipedia. Rolf H Nelson (talk) 01:42, 31 May 2018 (UTC)

We can include accidents the driver did not cause. Since May 2017, Waymo has driven 4 million miles. That will be going up by more than 1 million every 2 months. In that time Waymo has not caused a single accident. Waymo has been involved in 5 accidents. How many humans have driven 4 million miles? Of those how many have been in 5 or less accidents? Find 1 person. Humans cannot match the driving record of Waymo. Waymo is superhuman at driving safely. --Mschribr (talk) 16:25, 19 June 2018 (UTC)

185 crashes per 100 million VMT

Human crash rates in 2011 were 185 crashes per 100 million miles according to [1], so 5 crashes in 4 million miles isn't even far from statistical error of that range. Driverless cars get a lot of coverage; I'm sure once evidence shows that driverless cars are safer than human drivers in some specific domain the media will cover that. Rolf H Nelson (talk) 18:41, 14 July 2018 (UTC)

The 5 crashes involving Waymo are not in the same class as the human crashes. Humans caused the 5 Waymo crashes. Wayno drove 4 million miles and caused zero crashes. In 2011 humans drove 100 million miles and humans caused 185 crashes. So Waymo is not in the statistical error range of any human driver. Waymo is much better than any human driver. The evidence is here. Waymo driving is better than humans driving. The media prefers accidents over safe driving, so Waymo may not be reported. --Mschribr (talk) 07:15, 15 July 2018 (UTC)

Waymo has driven another million miles in the last month without an accident. Since May 2017, Waymo has driven 5 million miles. Since it has not had an accident in 5 million miles I will move Waymo from par human to high human. — Preceding unsigned comment added by Mschribr (talk • contribs) 11:58, 27 July 2018 (UTC)

Most everyday tasks performed by humans

What are "Most everyday tasks performed by humans"? This is vague. Instead, can you name the everyday tasks performed by humans that an artificial intelligence does at a sub-human level? --Mschribr (talk) 14:22, 11 January 2015 (UTC)

Category names

I suggest changing the heading "Super-human" to "World class" and bringing the title "Strong super-human" back to super-human".

Having 5 classes rather than the original 4 is fine, but in most of "super-human" category, computers are not even undisputed world champions. e.g. in Bridge it's certainly world class but not the champion. If there isn't any good reason why not, I'll make the change. Chris55 (talk) 22:08, 7 January 2017 (UTC)

I'm not sure if that's a good idea. "World class" is not really compatible with the preexisting names. Here is how I interpret these definitions:

• optimal: it is not possible to perform better (theoretical best)
• strong super-human: performs better than all humans (better than 100% of humans)
• super-human: performs better than most humans (better than 80% but not 100% of humans)
• par-human: performs similarly to most humans (better than 20% but not 80% of humans)
• sub-human: performs worse than most humans (no better than 20% of humans)

--Acyclic (talk) 22:58, 7 January 2017 (UTC)

Those are your own interpretations. The problem is that "super-human" in normal parlance is defined by "Superhuman qualities are qualities that exceed those found in humans." (It's permissable to quote WP on a talk page I think :) Your definition doesn't meet that. Is it sensible to say "All Mensa members have superhuman intelligence"? And they are supposed to be cleverer than 98% of humans.

The "strong super-human" is a recent addition (6 months) and this article has been exaggerating the performance of computers in many fields. Where a computer is not an undisputed world champion it is incorrect to put it as "super-human". What do you suggest as a better alternative? Chris55 (talk) 17:17, 11 January 2017 (UTC)

The best adjective I can think of is "high". Does that work? Chris55 (talk) 19:39, 11 January 2017 (UTC)

Categories must be relative to human performance, except in the case of optimal performance. I suppose you're welcome to correct any incorrect categorical classifications. But it would be a tall order to have new categories that don't reflect their relation to human performance. --Acyclic (talk) 02:00, 12 January 2017 (UTC)

Ok, call it high-human. No worse a phrase than par-human and if the human isn't assumed it's necessary. Have you seen that cute video on you-tube where a chimpanzee comprehensively wipes the floor with humans on a short-term memory test? We can't assume that humans are cleverest in all areas! Chris55 (talk) 11:06, 12 January 2017 (UTC)

I'm not inclined to agree with the recent change to the "high human" category. The article has used the category "strong super-human" since 2009 when the material was taken from the artificial intelligence article, so I'll revert. The categorisation is not sourced, so if anyone has a reliable source that would be the best way forward. pgr94 (talk) 13:45, 4 February 2017 (UTC)

Where is the source for this dubious claim? Please give it. It is demonstrated that the so-called "super-human" is being used to be better than 80% of humans. That is a clear abuse of the term. The "strong super-human" section was only introduced in 2016. Chris55 (talk) 12:28, 5 February 2017 (UTC)

A google search of "strong super-human" links to this book: [2][3][4]. Also, there's

Rajani, S. (2011). Artificial intelligence: Man or machıne? International Journal of Information Technology and Knowledge Management, 4(1), 173–176.

Admittedly, not great sources. But is there any evidence for "high-human"? pgr94 (talk) 19:02, 5 February 2017 (UTC)

Rajani is then cited in an Artificial Intelligence Review article [5]. pgr94 (talk) 19:08, 5 February 2017 (UTC)

Clear evidence of copying in both cases, see Talk:Artificial_intelligence#Super-human?. In addition, two of the extra references you have supplied are direct quotations from Wikipedia. The other is a huge plagiarisation from the same source. Chris55 (talk) 09:57, 6 February 2017 (UTC)

Speech Recognition

The reference to near human performance in speech recognition (Harvard Business of AI) is flawed, yes in good conditions there's near human performance, but when you have multiple speakers in a single channel, recoded audio, background noise, technical dialog and so on the 2018 performance is still ~30% wer

--Sgt101 (talk) 14:01, 10 February 2018 (UTC)

The HBR source states "Voice recognition, too, even in noisy environments, is now nearly equal to human performance". Feel free to provide additional sources with their own assessment. Rolf H Nelson (talk) 18:59, 10 February 2018 (UTC)