Talk:Analytic hierarchy process

Same ol', same ol'

There was an unsigned change that reintroduced some very old content in this article and it previously caused a lot of problems. I changed it back to the version last edited by DCLawyer. I have a feeling this will be another slow-paced edite war so let me just state the problems with this proposed version of the article. 1) It makes an ad hominem argument by simply saying that the criticisms are a lack of understanding of AHP by the critics. Example: "AHP has had challenges to its theoretical and practical shortcomings from those who may not fully understand it." and "This criticism illustrates a lack of understanding for several reasons." The critics are numerous and the rank reversal problem has attracted quite a lot of attention. A review of the citations show that the critics are quite qualified and the criticisms are based on a mathematical analysis of the problem. I think, however, it shows a lack of understanding of the mathematical level of the issue by the editor who insists this is the correct version 2) This editor is again making the claim that AHP could be used to "solve" problems that already have formally proven solutions. Curiously, the issue of whether proponents of AHP claimed it could be used to solve problems such as actuarial calculations and optimal wing designs came up earlier in this discussion page. The article previously pointed out that proponents actually do make such claims. But, it was concluded that it could not be stated in the article that proponents make such claims because this information was only based on the claims made in previous version of this article. And yet here, againm, this editor is saying exactly that. It seems that the burden should be on this editor to show that AHP has been used on such problems and that it actually produces the same answer as the known mathematical optimization methods. 3) The language gets very flowery and POV. Example: "One of the amazing things about the AHP..." 4) I suspect this is a COI problem. The Saaty's have previously been warned about making these changes themselves. This could also be one of the software vendors that keep posting their products here.

The editor (if he/she is the same as the one who made these edits before) who keeps reintroducing this material refuses to even discuss it on the talk page even though they have been encouraged to discuss these issues in detail. Again, I make the offer to discuss the criticisms of AHP here in the discussion page.Hubbardaie 23:11, 3 September 2007 (UTC)

In my opinion, you were right to revert the edits. The anonymous person who made them didn't provide any sources for his/her claims (except for maybe one, IIRC). He/she may not know that Wikipedia requires sources for all claims, especially those that are subject to challenge or question.

For example, if you're gonna say that AHP has been challenged, you need to cite one or more reliable sources where such a challenge has been made. Otherwise, people will remove your claim. For subjects like AHP, books or peer reviewed journals are the best reliable source. Somebody's web site, probably not.

The same goes for claims that AHP is wonderful, or amazing, or better than sliced bread, or whatever. To say that stuff yourself is original research, and is not allowed. To quote or accurately paraphrase a reliable source that said it is acceptable, provided it is done in an evenhanded way. ("A study at the University of Foo found that shoppers preferred AHP to sliced bread by three to one," followed by a citation of the study's publication, preferably in a non-U of Foo place, would probably be OK.)

It would also be helpful if this anonymous editor got an account so people could discuss these things with him/her on their talk page. DCLawyer 23:45, 3 September 2007 (UTC)

Weasel words / citations needed

I deleted a paragraph that had some weasel words and a citation request that had not been acted on after more than two weeks. Ohio Mailman 23:42, 9 September 2007 (UTC)

No Citation

I deleted a sentence where no citation had been provided over a month after it was requested, plus a sentence that depended on the first one. Ohio Mailman 22:38, 30 September 2007 (UTC)

Misleading Reference

I removed the material about Egon Brunswik's Lens model. The reference provided to support it has nothing to do with scoring methods or the track record of Brunswik's model vs. the Analytic Hierarchy Process. Nothing else in the Kleindorfer book supports the claim of other methods being superior to AHP, or even discusses a comparison. The book contains only one brief reference to AHP, on page 224. It is a favorable mention, with the context that AHP "...enhance(s) the information-processing resources available to the group through the intelligent use of information technology." Ohio Mailman 15:01, 4 October 2007 (UTC)

New section: Uses and applications

I've added this section and wikified it. I hope everybody likes it. I'm working on some additional material to describe AHP more fully. Most of it is available for online examination and comment. If you want to look at it, let me know. Lou Sander 21:02, 23 October 2007 (UTC)

The first sentence of the "uses and applications" section gives BhusBhushan, Navneet and Kanwal Rai book as a source of the claim about what AHP is "best" used for. But I reviewed this source and while the book does make such assertions, I see no emprical evidence in the book supporting this claim. Of course, I'm not including examples of where it was used and where people were pleased with the results as evidence that deicisons were improved. I think all once can fairly claim from this source is that "proponents believe...". I added those two words. But if there is some other explicit and conclusive empirical basis for these claims then that source should be used and the "proponents believe..." qualification can be removed.Hubbardaie 02:19, 28 October 2007 (UTC)

I'm going to go ahead and to remove the words "Proponents believe" from this section. They cast doubt on a well-sourced fact that is extensively supported by other literature on AHP. It's not up to us editors to analyze the content of verifiable sources, or to include our personal research into their validity. 136.142.73.116 15:44, 30 October 2007 (UTC)

I agree with your edit, but I kind of wish you'd get a user name. This article has had its share of bad edits by anonymous users. DCLawyer 00:45, 31 October 2007 (UTC)

There are many "well-sourced" claims that are nonetheless controversial and not at all universally held by specialists in the field. If there are sources of specific empirical evidence of this claim and not just the claims of an AHP evangalist, then those sources should be added before the "proponents claim" qualifier is removed. I'll put it back. Hubbardaie 17:47, 5 November 2007 (UTC)

The anon is right. I reverted the original research. Cleome 08:03, 6 November 2007 (UTC)

I realize you believe the anon to be correct. The "original research" would be not to include this qualifier since "proponents believe" is all that one can derive from the source given. Since the authors of the source mentioned no substantial empirical evidence (only anecdotal cases), the original research would be to claim more than can honestly be claimed from that source. In the spirit of avoiding edit wars, I'll wait a few days for a response before I revert back to the proposed qualifier. Perhaps someone can source some published research that can justify the removal of "proponents believe". Hubbardaie 21:25, 6 November 2007 (UTC)

By the way, I'm getting the impression that I'm the only one who has read the single source provided for this claim. I don't think anyone who read it would call it "well sourced" since the source provides no evidence for the claim. I encourage those who want to insist the the claim is well sourced to read the source and point to the specific page where empirical evidence (not a mere case study) of the claim is provided. I'll wait a couple more days to revert to what I wrote just in case someone can come forth with this request. Hubbardaie 13:25, 7 November 2007 (UTC)

It seems like you are trying to twist or analyze the content of a reliable source. It isn't up to us to do that. See WP:V, which says "The threshold for inclusion in Wikipedia is verifiability, not truth. "Verifiable" in this context means that any reader should be able to check that material added to Wikipedia has already been published by a reliable source." Good Cop 08:32, 8 November 2007 (UTC)

No, no. You have it exactly backwards. To say that "the authors/proponents say X" is the undisputed fact among us. To presume that the authors have proven X by asserting X to be true without qualification, is the attempt to "twist or analyze the content" of the source. Yes, it is verifiable that these authors say that - and to say any more than that is editorializing. But I tell you what I'll do...I'll add other verifiable sources that claim exactly the opposite. Then we'll have to deal with how to combine contradictory claims in the article. Of course (like many other articles), we will find that the best way to combine these is through the acknowlegement of the controversy. More to come. By the way, how many "verifiable sources" do you think there are for the claim "astrology works" if we don't worry at all about the other side of the argument in that claim? Hubbardaie 12:59, 8 November 2007 (UTC)

I added another source and another point in criticisms. This is a respected peer-reviewed journal, not a single case study written by proponents. I also removed a claim that was half uncsourced (the part that said the criticisms are "mostly theoretical or speculative" and half redundant (the same case was already and more appropriately listed in applications). To assert that the criticims are merely "theoretical or speculative" in the face of these sources would, itself, be editorializing. Now, before anyone removes or qualifies the comment further, remember I'm using the same rule used by Good Cop above. The source only needs to be verifiable and whether there is controversy is irrelevant. According to Good Cop, as long as some verifiable sources states it, it can simply be asserted without further qualification. Hubbardaie 13:33, 8 November 2007 (UTC)

Skyrocket, after a well-thought-out pause, managed to muster "please just leave it alone" as the sum total of his/her response. Try to scrape up something a bit more articulate. We are trying to construct an ecyclopedic article, here. Think through the response and respond to specific comments I made. I know you can do it. Hubbardaie 06:43, 10 November 2007 (UTC)

New lead

I've replaced the old lead with a new one and wikified it. Points that were in the old lead but not this one will be added to subsequent sections. Lou Sander 16:40, 24 October 2007 (UTC)

I replaced the above "new" wikified lead with a newer one that is more complete and accurate, and probably better written. Lou Sander 02:53, 8 November 2007 (UTC)

Criticisms section

I reverted the removal of properly-sourced material. The reverted edit also altered the meaning of the paragraph and included redundant mention of the ten year old Schenkerman article. Ohio Mailman 20:52, 8 November 2007 (UT

No the source does not support the first part of the claim that the flaws are "mostly theorectical or speculative". It is only a source that it was applied in the case specified, and that case is already cited in the applications and uses. The mention of one particular case study simply doesnt belong in this section, anyway. And regarding the claim that the flaws are "mostly theoretical or speculative" you will have to provide a source specifically for that claim (which is not the source you provided). The age of the Schenkerman article is only relevant if you can cite a specific article that addresses Schenkerman's objections. I've researched this and I doubt you will find one. I think what you will find is that other simply drive on and generate new applications of AHP without ever addressing (or probably even being aware of) the earlier objections. Finally, I list three other critics besides Schenkerman. I could have easily added many more from peer reviewed journals (not merely a self-promoting book written by AHP proponents, like the source you quote). You will need to specifically cite a source that supports the claim that the criticisms are mostly theoretical or speculative before you should re-insert it. Hubbardaie 21:55, 8 November 2007 (UTC)

Also, if you insist that any "properly sourced" material cannot be removed, then you must apply that same standard to the Schenkerman citation in the first paragraph. I think the age is less relevant than the fact that the Schenkerman article is in a respected peer reviewed journal and the source you reinserted is not. But that dispute is irrelevant if the criterion for leaving in a claim is simply that it be "properly sourced". You cannot remove this citation and insist the other cannot be removed. It's been made clear in here that parsing a citation any further than that it exists is editorilizing and OR, so we should apply the same rules to all the citations. Hubbardaie 22:25, 8 November 2007 (UTC)

I have no problem with that source, but its already mentioned in the applications & uses and I'm not sure why that one alone (a watershed mgt case) is the one citation that should be used to make the point that it is popular. Seems like an one arbitrary choice out of the list of case studies. The applications & uses section already discusses its popularity. The "despite these concerns..." sentence also sounded partially redundant with the first sentence in the paragraph. Since the first sentence already mentions it is widespread, I added that it is popular - so that should cover both points adequately enough for a section that is supposed to be about criticisms. If we think we need to add such qualifying statements to the criticisms section, then it should make sense to add similar qualifiers to the (very flowery editorial) introduction. For example, we should introduce it as the "popular but controversial" method. I just think that a balanced article would either mention contradictory views in both the introduction and criticisms sections, or neither. If you insist that the sentence should be reinserted, then I will also add the "controversial" qualification in the introduction to balance out both sections. Hubbardaie 02:19, 9 November 2007 (UTC)

Someone keeps removing the properly-sourced sentence "The resulting decision errors cause AHP users to be "seriously misled".[17]". When I previously attempted to remove a claim I felt was editorializing, others insisted that it was "well sourced" and should not be debated as long as source can be verified. Now I have a well sourced claim (unlike the other source, this is a respected peer reviewed journal) and it keeps being removed. We need to have some consistency here. If this claim can be removed, then so can others, regardless of whether some think it is well-sourced. Hubbardaie 03:02, 9 November 2007 (UTC)

Good Cop has deleted a properly sourced statement in a respected peer-reviewed journal. He has now contradicted his previous position that as long as a statement is well-sourced it is editorializing to remove it. He is applying different rules for statements he agrees than the statements he disagrees with. The source of the statement he chooses to replace this one with, however, is not a peer reviewed journal and on that basis alone it should not trump a peer reviewed journal. If he insists that it is false, contrary to a citation of a respected journal, then that is OR.Hubbardaie 04:43, 9 November 2007 (UTC)

Good Cop insists I'm 1) distorting the meaning of the first paragraph and 2) distorting what Schenkerman said. First, I would like to hear why Good Cop thinks he has a special authority to decide the the meaning of paragrapsh should be. This is the criticisms section, and the first paragraph should introduce the criticisms - not just reiterate the flowery, non-judgemental language of previous sections. Secondly, Schenkerman's entire quote is provided a few lines later for all to see. Any reasonalbe reviewer would have to conclude that I provided an exact quote and the tone is entirely consistent with the tone of the entire quote in context. Good Cop previously insisted that well-sourced statements must not be removed and to do so was editorializing. He is now doing exactly that. Clearly, Good Cop thinks that "balanced" is to offer no challanges to the claims of AHP in the introduction but to offer challenges to the challenges in the criticisms section. Here is what I propose. I will add the qualifiers to the introduction section that the method is controversial. Let's see if Good Cop will apply the same rules to both sections. Hubbardaie 12:25, 9 November 2007 (UTC)

Good Cop is right. Ohio Mailman 14:38, 9 November 2007 (UTC)

You think Good Cop is right about him having special authority to decide the meanings of paragraphs? Or do you think he is right about me twisting Schenkerman? Or is it something else you haven't articulated? If the second one, see my detailed response below.Hubbardaie 23:18, 9 November 2007 (UTC)

I strongly agree with Ohio Mailman. Skyrocket654 16:24, 9 November 2007 (UTC)

Read what I wrote to Ohio Mailman and what I wrote below. You might not understand what you are strongly agreeing with.Hubbardaie 23:18, 9 November 2007 (UTC)

"Strong agreements" aside ("strongly [more important]" is 5 points on the AHP scale, so that's a lot, right?), Schenkerman's quote did not specify only situations where the erros occur so, of course, he would say "users can be seriously mislead". I made it clear that I was already specifically talking about those situations where this errors occured in which case, the users WILL be mislead (by definition of "error"). But since you found it confusing I replaced it. Again, this is the criticisms section. One cannot honestly read the criticisms sourced as then continue to presume that AHP is "proven" to work and make unchallenged, flowery claims about it in the rest of the article. That's why an honest and truly evenhanded approach would be to point this out. I moved the watershed management case citation to a "Proponent Responses" sub-section. I still don't know why you think that particular case study is the necessary and sufficient evidence of that point, but you may leave there if you like. For me, that was more of a point of basic logical article construction rather than a problem with that particular citation. On another note, if Good Cop would have specified his problem with "can be" vs. "will" in the first place (he mentioned that on my talk page, not in this discussion for some reason) instead of just deleting the whole sentence then we could have made a simple change and saved us both some time. I also suggest Good Cop review the WP:AGF (I’ve told him about this before in his behavior), WP:3RR and WP:OWN (he complained if me “destroying the work of other people” in a section I mostly wrote). Hubbardaie 23:13, 9 November 2007 (UTC)

i agree with hubbardaie Sweet Death (talk) 01:50, 7 February 2008 (UTC)

Hierarchies subsection

I've added some material about AHP hierarchies. Lou Sander 18:43, 14 November 2007 (UTC)

Development dates

Most of the initial AHP development was in the 1970s, not the 1980s. The seeds of it were sown from 1963-1969 when Dr. Saaty worked for the State Department (See his "Fundamentals of Decision Making and Priority Theory," page ix.) The first book on it was published in 1980 (Saaty's "The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation.") Lou Sander (talk) 15:09, 3 February 2008 (UTC)

Irrelevant

what i deleted was irrelevent. article is imbalanced. Sweet Death (talk) 01:48, 7 February 2008 (UTC)

Improving the criticisms section

There are some criticisms of this analytical process, but they have not found wide support. This section does not present the situation fairly --

• Undue weight is given to 18 year old debates among professors, including extensive quoting from one side.
• The claim that "the use of arbitrary scales and certain internal inconsistencies and theoretical flaws have been discussed extensively in the literature" is supported by only two (2) citations, both over ten (10) years old. Nobody in this century ever pays attention to them, let alone repeats them.
• The use of Wikipedia subheadings gives these matters undue importance in the article's Table Of Contents. They are of historical interest only.

How can these problems be fixed? Skyrocket654 (talk) 18:59, 10 February 2008 (UTC)

It seems Skyrocket654 has a point here. Because I am rather new at being a Wikipedia editor I am reluctant to make any changes myself.MathDame (talk) 17:03, 11 February 2008 (UTC)

I have also been bothered by the many TOC entries for this small section. I am fixing the problem by replacing the headings with bolded letters. Hope it's O.K. Cleome (talk) 18:38, 11 February 2008 (UTC)

I noted ERosa's remarks. 1) If all we have is 18-year-old stuff, we shouldn't say a lot about it. Maybe mention that there were debates 18 years ago, and provide the references. 2) If the critics are still arguing that their charges weren't answered, we should include current references. 3) We shouldn't care very much about proponents or opponents or what they might think, or how many there are, and so forth. Let them publish their stuff. We can include it if it's appropriate to do so. DCLawyer (talk) 02:06, 12 February 2008 (UTC)

I have been bothered for months by the quotations from those long ago debates. There were at least eight or nine articles in the series. It is not our job to summarize them, especially by arbitrarily selecting quotations. But the debates did take place, and there is no reason not to mention them. Readers who are interested in them can get a complete picture by reading the articles in the references.

The references should be kept but the quotations should be deleted. All this can be fixed by deleting everything between "In these debates,..." and "...good basis from which to develop." The two references that appear there are duplicates of those in the "A series of debates..." sentence. Ohio Mailman (talk) 16:17, 13 February 2008 (UTC)

There are some famous theorems in geometry that are over 2500 years old, and they are still good. The problem is that the criticisms were sound 18 years ago and the proponents never answered them in a way that is conclusive proof that the initial criticism is wrong. Contrary to what DCLawyer said, the burden of proof should be on the side that has not provided citations for how the critics were answered. If these criticisms are "out of date" then provide the citation that conclusively disproved them. Otherwise, assume they are still valid. To simply label them old without specific citation amounts to original research. Let's avoid breaking the NOR rule. I could point out that the criticisms of areas such as phrenology, astrology, the ptolemaic system, and certain aspects of Aristotelian physics are also quite old. If we use DCLawyers logic, should we now conclude that the criticisms have worn off and it is the burden of the critics to cite new criticisms? No,until the old ones are answered conclusively, they are still good.ERosa (talk) 02:43, 12 February 2008 (UTC)

The debates started 18 years ago but the critics argue that their charges were never adequately answered. No empirical findings to date have overturned these criticsms. If anyone has information to the contrary, please cite the specific source of the controlled experiment (not just testimonials from a case study) or the mathematical proof of the error in the criticism. I realize AHP proponents are almost religious in their support of AHP and prefer not to admit to any current debate on the topic. But the debate is real and is still real after 18 years. Furthermore, you should compare the relative emphasis given to criticisms sections in other methodologies an I think you will find that this is entirely consitent with how criticisms are presented in peer-reviewed literature. Finally, I'm not sure what Sweet Death meant by not having wide support, but among academics I find very few pro-AHP researchers. The one exception I found as a student of Saaty who started an AHP software company. On the other hand, if you feel that you have empirical evidence of how wide or narrow support for AHP is among objective researchers (not, of course, among AHP vendors and managers who bet their careers on buying the AHP tools) then you should cite that in the article. ERosa (talk) 18:56, 11 February 2008 (UTC)

I also just noticed that the history of the article included a point about rank reversal. Proponents argue that this has been addressed with the "ideal process mode". But this is still of interest at least historically because it was a key criticism for many years. Its also interesting that Saaty's initial argument was that when rank reversal happens, it actually *should* be happening and is not, as the critics charges, evidence of a fundamental logical flaw in the process. It was only much later that Saaty apparently agreed that rank reversals are often (I would say always) an absurd result and then developed the Ideal Process mode to correct it. My concern is that when a method has such a fundamental flaw, one has to be concerned about the entire approach and that just fixing the outcome after the fact is putting lipstick on a pig. But, since I don't want to be accused of original research, I'll see if some other source has already responded to Saaty's fix to rank reversal and write a section that cites it. In the mean time, I think rank reversal is important enough to at least be mentioned as part of the historical evolution of AHP.ERosa (talk) 21:15, 11 February 2008 (UTC)