Action model learning

Action model learning (sometimes abbreviated action learning) is an area of machine learning concerned with creation and modification of software agent's knowledge about effects and preconditions of the actions that can be executed within its environment. This knowledge is usually represented in logic-based action description language and used as the input for automated planners. Action model learning is a form of inductive reasoning, where new knowledge is generated based on agent's observations.

Given a training set $E$ consisting of examples $e=(s,a,s')$, where $s,s'$ are observations of a world state from two consecutive time steps $t,t'$ and $a$ is an action instance observed in time step $t$, the goal of action model learning in general is to construct an action model $\langle D,P\rangle$, where $D$ is a description of domain dynamics in action description formalism like STRIPS, ADL or PDDL and $P$ is a probability function defined over the elements of $D$ (Certicky, 2013). However, many state of the art action learning methods assume determinism and do not induce $P$. In addition to determinism, current methods differ in how they deal with other attributes of domain (e.g. partial observability or sensoric noise).

Usual motivation for action model learning is the fact that manual specification of action models is often a difficult, time consuming, and error-prone task (especially in complex environments).

Action model learning differs from standard supervised learning in that correct input/output pairs are never presented, nor imprecise action models explicitly corrected.

Research

Literature

Most action learning research papers are published in journals and conferences focused on artificial intelligence in general (e.g. Journal of Artificial Intelligence Research (JAIR), Artificial Intelligence, Applied Artificial Intelligence (AAI) or AAAI conferences). Despite mutual relevance of the topics, action model learning is usually not adressed on planning conferences like ICAPS.

References

• Certicky, Michal (2013). "Action Models and their Induction". Organon F, International Journal of Analytic Philosophy. 20. Institute of Philosophy of the Slovak Academy of Sciences: 206–215.
• Certicky, Michal (2014). "Real-Time Action Model Learning with Online Algorithm 3SG". Applied Artificial Intelligence. 28. Taylor & Francis: 690–711. doi:10.1080/08839514.2014.927692.