Nonlinear system

This article is about "nonlinearity" in mathematics, physics and other sciences. For video and film editing, see Non-linear editing system. For other uses, see Nonlinearity (disambiguation). "Nonlinear dynamics" redirects here. For the journal, see Nonlinear Dynamics (journal).

In mathematics and science, a nonlinear system (or a non-linear system) is a system in which the change of the output is not proportional to the change of the input. Nonlinear problems are of interest to engineers, biologists, physicists, mathematicians, and many other scientists since most systems are inherently nonlinear in nature. Nonlinear dynamical systems, describing changes in variables over time, may appear chaotic, unpredictable, or counterintuitive, contrasting with much simpler linear systems.

Typically, the behavior of a nonlinear system is described in mathematics by a nonlinear system of equations, which is a set of simultaneous equations in which the unknowns (or the unknown functions in the case of differential equations) appear as variables of a polynomial of degree higher than one or in the argument of a function which is not a polynomial of degree one. In other words, in a nonlinear system of equations, the equation(s) to be solved cannot be written as a linear combination of the unknown variables or functions that appear in them. Systems can be defined as nonlinear, regardless of whether known linear functions appear in the equations. In particular, a differential equation is linear if it is linear in terms of the unknown function and its derivatives, even if nonlinear in terms of the other variables appearing in it.

As nonlinear dynamical equations are difficult to solve, nonlinear systems are commonly approximated by linear equations (linearization). This works well up to some accuracy and some range for the input values, but some interesting phenomena such as solitons, chaos, and singularities are hidden by linearization. It follows that some aspects of the dynamic behavior of a nonlinear system can appear to be counterintuitive, unpredictable or even chaotic. Although such chaotic behavior may resemble random behavior, it is in fact not random. For example, some aspects of the weather are seen to be chaotic, where simple changes in one part of the system produce complex effects throughout. This nonlinearity is one of the reasons why accurate long-term forecasts are impossible with current technology.

Some authors use the term nonlinear science for the study of nonlinear systems. This term is disputed by others:

Using a term like nonlinear science is like referring to the bulk of zoology as the study of non-elephant animals.

— Stanisław Ulam

Definition

In mathematics, a linear map (or linear function) $f(x)$ is one which satisfies both of the following properties:

• Additivity or superposition principle: $\textstyle f(x+y)=f(x)+f(y);$
• Homogeneity: $\textstyle f(\alpha x)=\alpha f(x).$

Additivity implies homogeneity for any rational α, and, for continuous functions, for any real α. For a complex α, homogeneity does not follow from additivity. For example, an antilinear map is additive but not homogeneous. The conditions of additivity and homogeneity are often combined in the superposition principle

$f(\alpha x+\beta y)=\alpha f(x)+\beta f(y)$

An equation written as

$f(x)=C$

is called linear if $f(x)$ is a linear map (as defined above) and nonlinear otherwise. The equation is called homogeneous if $C=0$ and $f(x)$ is a homogeneous function.

The definition $f(x)=C$ is very general in that $x$ can be any sensible mathematical object (number, vector, function, etc.), and the function $f(x)$ can literally be any mapping, including integration or differentiation with associated constraints (such as boundary values). If $f(x)$ contains differentiation with respect to $x$, the result will be a differential equation.

Nonlinear algebraic equations

Main articles: Algebraic equation and System of polynomial equations

Nonlinear algebraic equations, which are also called polynomial equations, are defined by equating polynomials (of degree greater than one) to zero. For example,

$x^{2}+x-1=0\,.$

For a single polynomial equation, root-finding algorithms can be used to find solutions to the equation (i.e., sets of values for the variables that satisfy the equation). However, systems of algebraic equations are more complicated; their study is one motivation for the field of algebraic geometry, a difficult branch of modern mathematics. It is even difficult to decide whether a given algebraic system has complex solutions (see Hilbert's Nullstellensatz). Nevertheless, in the case of the systems with a finite number of complex solutions, these systems of polynomial equations are now well understood and efficient methods exist for solving them.

Nonlinear recurrence relations

A nonlinear recurrence relation defines successive terms of a sequence as a nonlinear function of preceding terms. Examples of nonlinear recurrence relations are the logistic map and the relations that define the various Hofstadter sequences. Nonlinear discrete models that represent a wide class of nonlinear recurrence relationships include the NARMAX (Nonlinear Autoregressive Moving Average with eXogenous inputs) model and the related nonlinear system identification and analysis procedures. These approaches can be used to study a wide class of complex nonlinear behaviors in the time, frequency, and spatio-temporal domains.

Nonlinear differential equations

A system of differential equations is said to be nonlinear if it is not a system of linear equations. Problems involving nonlinear differential equations are extremely diverse, and methods of solution or analysis are problem dependent. Examples of nonlinear differential equations are the Navier–Stokes equations in fluid dynamics and the Lotka–Volterra equations in biology.

One of the greatest difficulties of nonlinear problems is that it is not generally possible to combine known solutions into new solutions. In linear problems, for example, a family of linearly independent solutions can be used to construct general solutions through the superposition principle. A good example of this is one-dimensional heat transport with Dirichlet boundary conditions, the solution of which can be written as a time-dependent linear combination of sinusoids of differing frequencies; this makes solutions very flexible. It is often possible to find several very specific solutions to nonlinear equations, however the lack of a superposition principle prevents the construction of new solutions.

Ordinary differential equations

First order ordinary differential equations are often exactly solvable by separation of variables, especially for autonomous equations. For example, the nonlinear equation

${\frac {du}{dx}}=-u^{2}$

has $u={\frac {1}{x+C}}$ as a general solution (and also the special solution $u=0,$ corresponding to the limit of the general solution when C tends to infinity). The equation is nonlinear because it may be written as

${\frac {du}{dx}}+u^{2}=0$

and the left-hand side of the equation is not a linear function of $u$ and its derivatives. Note that if the $u^{2}$ term were replaced with $u$, the problem would be linear (the exponential decay problem).

Second and higher order ordinary differential equations (more generally, systems of nonlinear equations) rarely yield closed-form solutions, though implicit solutions and solutions involving nonelementary integrals are encountered.

Common methods for the qualitative analysis of nonlinear ordinary differential equations include:

• Examination of any conserved quantities, especially in Hamiltonian systems
• Examination of dissipative quantities (see Lyapunov function) analogous to conserved quantities
• Linearization via Taylor expansion
• Change of variables into something easier to study
• Bifurcation theory
• Perturbation methods (can be applied to algebraic equations too)
• Existence of solutions of Finite-Duration, which can happen under specific conditions for some non-linear ordinary differential equations.

Partial differential equations

Main article: Nonlinear partial differential equation

See also: List of nonlinear partial differential equations

The most common basic approach to studying nonlinear partial differential equations is to change the variables (or otherwise transform the problem) so that the resulting problem is simpler (possibly linear). Sometimes, the equation may be transformed into one or more ordinary differential equations, as seen in separation of variables, which is always useful whether or not the resulting ordinary differential equation(s) is solvable.

Another common (though less mathematical) tactic, often exploited in fluid and heat mechanics, is to use scale analysis to simplify a general, natural equation in a certain specific boundary value problem. For example, the (very) nonlinear Navier-Stokes equations can be simplified into one linear partial differential equation in the case of transient, laminar, one dimensional flow in a circular pipe; the scale analysis provides conditions under which the flow is laminar and one dimensional and also yields the simplified equation.

Other methods include examining the characteristics and using the methods outlined above for ordinary differential equations.

Pendula

Main article: Pendulum (mathematics)

Illustration of a pendulum

Linearizations of a pendulum

A classic, extensively studied nonlinear problem is the dynamics of a frictionless pendulum under the influence of gravity. Using Lagrangian mechanics, it may be shown that the motion of a pendulum can be described by the dimensionless nonlinear equation

${\frac {d^{2}\theta }{dt^{2}}}+\sin(\theta )=0$

where gravity points "downwards" and $\theta$ is the angle the pendulum forms with its rest position, as shown in the figure at right. One approach to "solving" this equation is to use $d\theta /dt$ as an integrating factor, which would eventually yield

$\int {\frac {d\theta }{\sqrt {C_{0}+2\cos(\theta )}}}=t+C_{1}$

which is an implicit solution involving an elliptic integral. This "solution" generally does not have many uses because most of the nature of the solution is hidden in the nonelementary integral (nonelementary unless $C_{0}=2$).

Another way to approach the problem is to linearize any nonlinearity (the sine function term in this case) at the various points of interest through Taylor expansions. For example, the linearization at $\theta =0$, called the small angle approximation, is

${\frac {d^{2}\theta }{dt^{2}}}+\theta =0$

since $\sin(\theta )\approx \theta$ for $\theta \approx 0$. This is a simple harmonic oscillator corresponding to oscillations of the pendulum near the bottom of its path. Another linearization would be at $\theta =\pi$, corresponding to the pendulum being straight up:

${\frac {d^{2}\theta }{dt^{2}}}+\pi -\theta =0$

since $\sin(\theta )\approx \pi -\theta$ for $\theta \approx \pi$. The solution to this problem involves hyperbolic sinusoids, and note that unlike the small angle approximation, this approximation is unstable, meaning that $|\theta |$ will usually grow without limit, though bounded solutions are possible. This corresponds to the difficulty of balancing a pendulum upright, it is literally an unstable state.

One more interesting linearization is possible around $\theta =\pi /2$, around which $\sin(\theta )\approx 1$:

${\frac {d^{2}\theta }{dt^{2}}}+1=0.$

This corresponds to a free fall problem. A very useful qualitative picture of the pendulum's dynamics may be obtained by piecing together such linearizations, as seen in the figure at right. Other techniques may be used to find (exact) phase portraits and approximate periods.

Newton's method for solving systems of nonlinear equations

When multiple nonlinear equations need to be solved for more than one variable, Newton's Method for Systems of Equations may be used to solve the equations simultaneously for the solution vector. The process is very similar to solving Newton's method for one variable, except the single nonlinear equation is replaced with a system of nonlinear equations, the derivative is replaced with a Jacobian matrix of partial derivatives, and the subtraction is replaced with a vector subtraction. Newton';s method for systems of equations reduces to Newton's method for nonlinear equations when the system of equations includes only one equation.

Procedure

For single equations, Newton's method consists of the following iterations until the iterations no longer produce any changes to $x$ of significance,

$x_{k+1}=x_{k}-{\frac {f(x_{k})-y_{k}}{f'(x_{k})}}$

For systems of equations, the same is true, but for vector $X$ instead of scalar $x$,

${\begin{aligned}&X_{k+1}=X_{k}-C(X_{k})\\\end{aligned}}$

Where $C(X_{k})$ is the solution vector to $J(X_{k})C(X_{k})=(F(X_{k})-Y)$

$J(X_{k})$ is the jacobian matrix for $X_{k}$

$Y$is a vector of known values

Note that these definitions can be redefined with $Y=0$ and in terms of $J^{-1}(X_{k})$ to conform to obtaining zeros in the definition defined in the Newton's method page.

Simple example

For example, the following set of equations needs to be solved for vector of points $(x_{1},x_{2})$, given the vector of known values, (2,3).

${\begin{array}{lcr}5x_{1}^{2}+x_{1}x_{2}+sin^{2}(2x_{2})&=2\\e^{2x_{1}-x_{2}}+5x_{2}&=3\end{array}}$

the function vector, $F(X_{k})$, and Jacobian Matrix, $J(X_{k})$ for iteration k, and the vector of known values, $Y$, are defined below.

${\begin{aligned}&F(X_{k})={\begin{bmatrix}{\begin{aligned}&5x_{1}^{2}+x_{1}x_{2}^{2}+sin^{2}(2x_{2})\\&e^{2x_{1}-x_{2}}+4x_{2}\end{aligned}}\end{bmatrix}}_{k}\\&J(X_{k})={\begin{bmatrix}{\frac {\partial {f(x_{1})}}{\partial {x_{1}}}}&{\frac {\partial {f(x_{1})}}{\partial {x_{2}}}}\\{\frac {\partial {f(x_{2})}}{\partial {x_{1}}}}&{\frac {\partial {f(x_{2})}}{\partial {x_{2}}}}\end{bmatrix}}_{k}={\begin{bmatrix}{\begin{aligned}&10x_{1}+x_{2}^{2}&&2x_{1}x_{2}+4sin(2x_{2})cos(2x_{2})\\&2e^{2x_{1}-x_{2}}&&-e^{2x_{1}-x_{2}}+4\end{aligned}}\end{bmatrix}}_{k}\\&Y={\begin{bmatrix}2\\3\end{bmatrix}}\end{aligned}}$

The equation to solve for each iteration is

${\begin{aligned}{\begin{bmatrix}{\begin{aligned}&10x_{1}+x_{2}^{2}&&2x_{1}x_{2}+4sin(2x_{2})cos(2x_{2})\\&2e^{2x_{1}-x_{2}}&&-e^{2x_{1}-x_{2}}+4\end{aligned}}\end{bmatrix}}_{k}{\begin{bmatrix}c_{1}\\c_{2}\end{bmatrix}}_{k+1}={\begin{bmatrix}f_{1}-2\\f_{2}-3\end{bmatrix}}_{k}\end{aligned}}$

and

$X_{k+1}=X_{k}-C_{k+1}$

The iterations should be repeated until ${\bigg [}\sum _{k=1}^{k=2}|f(x)_{k}-y_{k}|{\bigg ]}<E$, where $E$ is a value acceptably small enough to meet application requirements.

If vector $X_{0}$ is initially chosen to be ${\begin{bmatrix}1&1\end{bmatrix}}$, that is, $x_{1}=1{\text{ and }}x_{2}=1$, and $E$ is chosen to be 1.e-03, then the example converges after four iterations to a value of $X_{4}=(0.567297,-0.309442)$.

Iterations

The following iterations were made during the course of the solution.

Iteration Convergence Sequence

Iteration	Variable	Variable Contents
0	X	${\begin{bmatrix}1&1\end{bmatrix}}$
	F(X)	${\begin{bmatrix}6.82682&6.71828\end{bmatrix}}$
1	J	${\begin{bmatrix}11&0.486395\\5.43656&1.28172\end{bmatrix}}$
	C	${\begin{bmatrix}0.382211&1.27982\end{bmatrix}}$
	X	${\begin{bmatrix}0.617789&-0.279818\end{bmatrix}}$
	F(X)	${\begin{bmatrix}2.23852&3.43195\end{bmatrix}}$
2	J	${\begin{bmatrix}6.25618&-2.1453\\9.10244&-0.551218\end{bmatrix}}$
	C	${\begin{bmatrix}0.0494549&0.0330411\end{bmatrix}}$
	X	${\begin{bmatrix}0.568334&-0.312859\end{bmatrix}}$
	F(X)	${\begin{bmatrix}2.01366&3.00966\end{bmatrix}}$
3	J	${\begin{bmatrix}5.78122&-2.25449\\8.52219&-0.261095\end{bmatrix}}$
	C	${\begin{bmatrix}0.00102862&-0.00342339\end{bmatrix}}$
	X	${\begin{bmatrix}0.567305&-0.309435\end{bmatrix}}$
	F(X)	${\begin{bmatrix}2.00003&3.00006\end{bmatrix}}$
4	J	${\begin{bmatrix}5.7688&-2.24118\\8.47561&-0.237805\end{bmatrix}}$
	C	${\begin{bmatrix}7.73132E-06&6.93265E-06\end{bmatrix}}$
	X	${\begin{bmatrix}0.567297&-0.309442\end{bmatrix}}$
	F(X)	${\begin{bmatrix}2&3\end{bmatrix}}$

Practical considerations

Newton's Method for systems of equations especially large sets of equations, can be finickier than for single equations. Care should be taken to insure a solution is found within a reasonable number of iterations.

Singular matrices

The solution to the linear set of equations that must be solved may not necessarily result in a usable non-singular solution. This can be because the set of equations has no solution, or because of a poorly chosen starting vector for $X_{0}$. For example, had the initial $X_{0}$been chosen to be (0,0), the first iteration should have resulted in a singular matrix and the convergence would have failed. Care should be taken to choose a valid starting $X_{0}$that is known to produce a non-singular first iteration.

Asymptotic divergence to infinity

Even though a system of equations is known to have a solution, the iterations may asymptotically diverge to infinity. This is especially more likely to happen with large number of equations. However, this condition can be mitigated or prevented by selecting a good starting point for $X_{0}$. In addition, the following steps may further mitigate divergence.

• Limit the range of the linear solution, the C vector, to a small range, but large enough to converge in a reasonable number of iterations.
• Limit the $X$vector iterations to known limits for each $x_{k}$ entry.
• Scale down the C vector entries slightly to slow down the convergence. Slow convergences are less likely to go divergent.

Slow convergence

In time sensitive applications, convergence speed is importance in that slow convergence can have detrimental affects on the application that is being supported. Convergence time can be minimized through the following steps.

• Good selection of the initial $X_{0}$starting point is very importance in minimizing the number of iterations required for an acceptable convergence error. For example, the example above, had the initial $X_{0}$ been chosen to be (2,2) instead of (1,1), then seven iterations would have been required instead of four.
• Limit the $X$ vector iterations to known limits for each $x_{k}$ entry. The $X$ vector does not have diverge to slow things down. If no limit has been placed on the vector, or the limit is too big, the iterations may spend too much time recomputing large values instead of converging.
• Scale down the C vector entries slightly to slow down the convergence. This may help prevent the iterations from jumping around and taking too long to converge.
• Select a conference error point as large as possible that still meets the application requirements. For example, had an error of 1.e-15 been chosen for the example above, six iterations should have been required, as opposed to only four needed to converge to an error of 1,e-03. The additional two iterations may be acceptable for high precision applications, but would be a waste for applications that only need light precision.

Insure a solution does exist

It is much easier to determine that a known solution exists or does not exist with single equations. For example, $X^{2}=4$ has an obvious known solution (2), while $X^{2}=-4$ is obvious that no solution exists in the set of real number. With sets of equations, especially large sets, it is far more difficult to determine that a solution exists or does not exist. If a solution does not exists, the iterations will certainly fail, but if a solution does exist, the iterations may still fail. Upfront work may be required to determine that a solution does or does not exist before making conclusions.

Multiple solutions

It is easier to determine that multiple solutions exist with single equations. The $X^{2}=4$ from the preceding paragraph, for example, has a solution of $X=2{\text{ and }}X=-2$. For sets of equations, especially large sets, it may not be so obvious, and even if it is obvious, it may be more difficult to insure convergence takes place at the desired solution. Care should taken to start with an $X_{0}$ as near as possible to the desired solution., and that limits are installed on the individual $X$ entries to move the iterations away from undesirable solutions and toward the desirable solutions(s). It should be noted that many solutions exist in the example used above.

Digital verses continuous derivatives

Derivatives should be calculated using continuous functions whenever possible to maximize accuracy and minimize convergence problems. If the function is unknown and not possible to calculate continuous derivatives, digital derivatives may be used, but care should be taken to maximize accuracy. Use double samples close together, if possible. If not possible, such as in a string of data, cubic interpolations are preferred due to the cubic iterations retention of a defined first derivative. If accuracy is not an issue, linear interpolations may be used, while keeping in mind that the first directives are not defined at the data point, requiring that the next or prior linear segment be used to estimate the derivative.

Types of nonlinear dynamic behaviors

• Amplitude death – any oscillations present in the system cease due to some kind of interaction with other system or feedback by the same system
• Chaos – values of a system cannot be predicted indefinitely far into the future, and fluctuations are aperiodic
• Multistability – the presence of two or more stable states
• Solitons – self-reinforcing solitary waves
• Limit cycles – asymptotic periodic orbits to which destabilized fixed points are attracted.
• Self-oscillations – feedback oscillations taking place in open dissipative physical systems.

Examples of nonlinear equations

• Algebraic Riccati equation
• Ball and beam system
• Bellman equation for optimal policy
• Boltzmann equation
• Colebrook equation
• General relativity
• Ginzburg–Landau theory
• Ishimori equation
• Kadomtsev–Petviashvili equation
• Korteweg–de Vries equation
• Landau–Lifshitz–Gilbert equation
• Liénard equation
• Navier–Stokes equations of fluid dynamics
• Nonlinear optics
• Nonlinear Schrödinger equation
• Power-flow study
• Richards equation for unsaturated water flow
• Self-balancing unicycle
• Sine-Gordon equation
• Van der Pol oscillator
• Vlasov equation

See also

• Aleksandr Mikhailovich Lyapunov
• Dynamical system
• Feedback
• Initial condition
• Linear system
• Mode coupling
• Vector soliton
• Volterra series

Further reading

• Diederich Hinrichsen and Anthony J. Pritchard (2005). Mathematical Systems Theory I - Modelling, State Space Analysis, Stability and Robustness. Springer Verlag. ISBN 9783540441250.
• Jordan, D. W.; Smith, P. (2007). Nonlinear Ordinary Differential Equations (fourth ed.). Oxford University Press. ISBN 978-0-19-920824-1.
• Khalil, Hassan K. (2001). Nonlinear Systems. Prentice Hall. ISBN 978-0-13-067389-3.
• Kreyszig, Erwin (1998). Advanced Engineering Mathematics. Wiley. ISBN 978-0-471-15496-9.
• Sontag, Eduardo (1998). Mathematical Control Theory: Deterministic Finite Dimensional Systems. Second Edition. Springer. ISBN 978-0-387-98489-6.

External links

• Command and Control Research Program (CCRP)
• New England Complex Systems Institute: Concepts in Complex Systems
• Nonlinear Dynamics I: Chaos at MIT's OpenCourseWare
• Nonlinear Model Library – (in MATLAB) a Database of Physical Systems
• The Center for Nonlinear Studies at Los Alamos National Laboratory