Template talk:Optimization algorithms

Welcome!

Constrained nonlinear optimization

Most of the nonlinear topics include only unconstrained methods. It would be good to have a group for constrained optimization: Sequential unconstrained minimization techniques (SUMT), sequential quadratic programming, augmented Lagrangian methods, Proximal point methods; Successive linear programming (like MS Excel's solver), Gradient projection methods (Lancelot), Michael Saunders's MINOS; Filter methods (Fletcher), etc. Kiefer.Wolfowitz (talk) 23:17, 8 November 2010 (UTC)[reply]

Heuristics

Previously, there were 3 lines of heuristics, out of proportion to the treatment of heuristics in optimization textbooks and surveys. There were two other problems, imho.

First, many of the heuristics that were listed have severe problems with notability, just to be listed in Wikipedia, and some have problems with single-purpose accounts (often anonymous IPs) being the main and nearly sole author: such editing is often associated with conflicts of interest, especially self-promotion. I removed all the heuristics with such problems with notability (with even having a WP article).

Second, many of the remaining heuristics were perhaps notable enough to have a Wikidia article, although the concerns with self-promotion and COI remain with a few of them (which seem to share the same editors). However, regardless of their merits as heuristics, many of those heuristics are not described in optimization textbooks and so fail notability to be in the footer for computational optimization. I removed the heuristics that are not discussed by optimization textbooks from this optimization footer.

(It might be useful to create a heuristic footer with them, because they do share a lot of commonalities.)

Finally, I think that my edits are consensus in optimization, because our optimization articles ignore the heuristics that I deleted, which makes it really weird to devote the header to them. I did ask for second opinions at the Wikiprojects in computer science, mathematics, and systems (operations research).

Sincerely, Kiefer.Wolfowitz (talk) 11:03, 30 November 2010 (UTC)[reply]

I don't think branch and bound belongs in the heuristics category as it is/can be complete, unlike the other algorithms in that category. I'd put it in a category on integer linear programming or search algorithms. —Ruud 19:34, 30 November 2010 (UTC)[reply]

Agreed, because the WP article is about branch-and-bound in integer programming. (There are also branch-and-branch methods in global optimization.) Most of this template concerns continuous optimization. It might be worthwhile to consider either moving the name to "continuous optimization methods" or to create at least one line about combinatorial optimization. Kiefer.Wolfowitz (talk) 22:51, 30 November 2010 (UTC)[reply]

I have noticed this template popping up on pages for optimization methods lately. I don't quite see its purpose since the categories already have comprehensive lists of optimizers. Nevertheless, wikipedia is a group effort so it is not for me to dictate whether it should be here or not. I was, however, curious about the choice of which optimizers were included and studying this discussion page as well as the history of changes I see that someone has removed e.g. CMA-ES, particle swarm optimization and differential evolution claiming they are not notable. This is clearly a fallacy. PSO and DE have been used and cited thousands of times (see e.g. google scholar), and CMA-ES is considered by many researchers to be amongst the very best (meta-)heuristics in existence today. It is a bit annoying that people take it upon themselves to make large edits/deletions in areas where they have no expertise. Just yesterday I had to save local unimodal sampling from deletion because an editor with no expertise had tagged it for deletion.

Optimering (talk) 06:46, 14 December 2010 (UTC)[reply]

It's fine that the mentioned "metaheuristic" (sic., "heuristic) topics have their own articles, because they seem to be notable. However, the optimization footer is very selective, and features topics that are found in optimization textbooks. As I suggested before, it would be a good project for you or others to create one (optimization) heuristics footer: Using the three rows that were in the optimization template! Kiefer.Wolfowitz (talk) 12:09, 14 December 2010 (UTC)[reply]

In my opinion it is not a good selection criteria whether something has been published in a textbook. Textbooks are usually slow to incorporate the state of the art. Besides, the mentioned optimizers (PSO, DE, CMA-ES) have actually been the topic of numerous textbooks, handbooks, etc. I haven't seen any papers using Rosenbrock optimizers. Golden section search is only for 1-dimensional unimodal functions, etc. etc. So it is a strange selection we have here. I also don't see the relevance of linking to e.g. combinatorial optimizers from pages on real-valued optimization, and vice versa. I suppose I could make a template/footer dedicated to (meta-)heuristics, but I still don't see the purpose. Most articles have 'see also' sections for closely related optimizers and there are wiki categories for broadly related optimizers. Why do we need this template? I think we should instead consider deleting this template altogether? Optimering (talk) 12:41, 14 December 2010 (UTC)[reply]

Six hours ago, you suggested restoring the three lines of (meta)heuristics. Within an hour of my reply, you suggest deleting the whole template. This seems to be an unconstructive conclusion.

on WP, "See also" sections are for articles that are not linked in the main article. A footer might be better than a list of see also articles, including much besides (meta)heuristics.

I agree that the some items on the function-evaluation line have little value, and wouldn't object to your deleting weak items. I would suggest that you keep "Powell's interpolation method" (and consider adding line search there and for the Wolfe conditions for the gradient-line). Kiefer.Wolfowitz (talk) 13:18, 14 December 2010 (UTC)[reply]

Combinatorial optimization algorithms

I created a subgrouping for combinatorial optimization algorithms. I was tempted to include mathematical structures important in combinatorial optimization (networks, graphs, matroids, greedoids, etc.) but then this template would not have been focused on algorithms. I did try to include the principal combinatorial algorithms presented in optimization and CS algorithm textbooks. Kiefer.Wolfowitz (talk) 11:43, 2 December 2010 (UTC)[reply]

Computational statistics

I removed the line on statistical methods. Like the minor heuristics that were removed earlier, the statistical methods are rarely discussed in optimization textbooks (although they are discussed in statistical textbooks, e.g. in system identification). Kiefer.Wolfowitz (talk) 11:46, 2 December 2010 (UTC)[reply]