ID3 algorithm

ID3 is algorithm used to generate Decision tree.

Main idea is based on Okcham's razor - world is as simple as it should be. To count exactly 'as simple as possible' we count Entropy, described as follows:

$I_{E}(i)=-\sum _{j=1}^{m}f(i,j)\log f(i,j)$

Tree making is described as follows:

• Take all unused attributes and count their entropy concerning test samples
• Choose attribute which entropy is smallest
• Make node containing that attribute

External links

• Seminars - http://www2.cs.uregina.ca