Control theory has two main roots: regulation and trajectory optimization. The first, regulation, is the more important and engineering oriented one. The second, trajectory optimization, is mathematics based. However, as we shall see, these roots have to a large extent merged in the second half of the twentieth century.
The problem of regulation is to design mechanisms that keep certain to-be controlled variables at constant values against external disturbances that act on the plant that is being regulated, or changes in its properties. The system that is being controlled is usually referred to as the plant, a passé part out term that can mean a physical or a chemical system, for example. It could also be an economic or a biological system, but one would not use the engineering term “plant” in that case.
Examples of regulation problems from our immediate environment abound. Houses are regulated by thermostats so that the inside temperature remains constant, notwithstanding variations in the outside weather conditions or changes in the situation in the house: doors that may be open or closed, the number of persons present in a room, activity in the kitchen, etc. Motors in washing machines, in dryers, and in many other household appliances are controlled to run at a fixed speed, independent of the load. Modern automobiles have dozens of devices that regulate various variables. It is, in fact, possible to view also the suspension of an automobile as a regulatory device that absorbs the irregularities of the road so as to improve the comfort and safety of the passengers. Regulation is indeed a very important aspect of modern technology. For many reasons, such as efficiency, quality control, safety, and reliability, industrial production processes require regulation in order to guarantee that certain key variables (temperatures, mixtures, pressures, etc.) be kept at appropriate values. Factors that inhibit these desired values from being achieved are external disturbances, as for example the properties of raw materials and loading levels or changes in the properties of the plant, for example due to aging of the equipment or to failure of some devices. Regulation problems also occur in other areas, such as economics and biology.
One of the central concepts in control is feedback: the value of one variable in the plant is measured and used (fed back) in order to take appropriate action through a control variable at another point in the plant. A good example of a feedback regulator is a thermostat: it senses the room temperature, compares it with the set point (the desired temperature), and feeds back the result to the boiler, which then starts or shuts off depending on whether the temperature is too low or too high. Man has been devising control devices ever since the beginning of civilization, as can be expected from the prevalence of regulation problems. Control historians attribute the first conscious design of a regulatory feedback mechanism in the West to the Dutch inventor Cornelis Drebbel (1572–1633). Drebbel designed a clever contraption combining thermal and mechanical effects in order to keep the temperature of an oven at a constant temperature.
Figure 1.1 [Fly ball governor].
Being an alchemist as well as an inventor, Drebbel believed that his oven, the Athanor, would turn lead into gold. Needless to say, he did not meet with much success in this endeavour, notwithstanding the inventiveness of his temperature control mechanism. Later in the seventeenth century, Christiaan Huygens (1629–1695) invented a flywheel device for speed control of windmills. This idea was the basis of the centrifugal fly-ball governor (see Figure P.1) used by James Watt (1736–1819), the inventor of the steam engine. The centrifugal governor regulated the speed of a steam engine. It was a very successful device used in all steam engines during the industrial revolution, and it became the first mass-produced control mechanism in existence. Many control laboratories have therefore taken Watt’s fly-ball governor as their favourite icon. The control problem for steam engine speed occurred in a very natural way. During the nineteenth century, prime movers driven by steam engines were running throughout the grim factories of the industrial revolution. It was clearly important to avoid the speed changes that would naturally occur in the prime mover when there was a change in the load, which occurred, for example, when a machine was disconnected from the prime mover. Watt’s fly-ball governor achieved this goal by letting more steam into the engine when the speed decreased and less steam when the speed increased, thus achieving a speed that tends to be insensitive to load variations. It was soon realized that this adjustment should be done cautiously, since by overreacting (called overcompensation), an all too enthusiastic governor could bring the steam engine into oscillatory motion. Because of the characteristic sound that accompanied it, this phenomenon was called hunting. Nowadays, we recognize this as an instability due to high gain control. The problem of tuning centrifugal governors that achieved fast regulation but avoided hunting was propounded to James Clerk Maxwell (1831–1870) (the discoverer of the equations for electromagnetic fields) who reduced the question to one about the stability of differential equations. His paper “On Governors,” published in 1868 in the Proceedings of the Royal Society of London, can be viewed as the first mathematical paper on control theory viewed from the perspective of regulation. Maxwell’s problem and its solution are discussed in Chapter 7 of this book, under the heading of the Routh-Hurwitz problem.
The field of control viewed as regulation remained mainly technology driven during the first half of the twentieth century. There were two very important developments in this period, both of which had a lasting influence on the field. First, there was the invention of the Proportional–Integral–Differential (PID) controller. The PID controller produces a control signal that consists of the weighted sum of three terms (a PID controller is therefore often called a three-term controller). The P-term produces a signal that is proportional to the error between the actual and the desired value of the to-be-controlled variable. It achieves the basic feedback compensation control, leading to a control input whose purpose is to make the to-be-controlled variable in crease when it is too low and decrease when it is too high. The I-term feeds back the integral of the error. This term results in a very large correction signal whenever this error does not converge to zero. For the error there hence holds, Go to zero or bust! When properly tuned, this term achieves robustness, good performance not only for the nominal plant but also for plants that are close to it, since the I-term tends to force the error to zero for a wide range of the plant parameters. The D-term acts on the derivative of the error. It results in a control correction signal as soon as the error starts increasing or decreasing, and it can thus be expected that this anticipatory action results in a fast response. The PID controller had, and still has, a very large technological impact, particularly in the area of chemical process control.
Figure 1.2 [Feedback amplifier].
A second important event that stimulated the development of regulation in the first half of the twentieth century was the invention in the 1930s of the feedback amplifier by Black. The feedback amplifier (see Figure P.2) was an impressive technological development: it permitted signals to be amplified in a reliable way, insensitive to the parameter changes inherent in vacuum-tube (and also solid-state) amplifiers. (See also Exercise 9.3.) The key idea of Black’s negative feedback amplifier is subtle but simple. Assume that we have an electronic amplifier that amplifies its input voltage 
to 
= 
. Now use a voltage divider and feed back 
to the amplifier input, so that when subtracted (whence the term negative feedback amplifier) from the input voltage Vin to the feedback amplifier, the input voltage to the amplifier itself equals 
. Combining these two relations yields the crucial formula
(1) 
This equation, simple as it may seem, carries an important message, see Exercise 9.3. What’s the big deal with this formula? Well, the value of the gain K of an electronic amplifier is typically large, but also very unstable, as a consequence of sensitivity to aging, temperature, loading, etc. The voltage divider, on the other hand, can be implemented by means of passive resistors, which results in a very stable value for μ. Now, for large (although uncertain) Ks, there holds 
, and so somehow Black’s magic circuitry results in an amplifier with a stable amplification gain 
based on an amplifier that has an inherent uncertain gain 
.
The invention of the negative feedback amplifier had far-reaching applications to telephone technology and other areas of communication, since long-distance communication was very hampered by the annoying drifting of the gains of the amplifiers used in repeater stations. Pursuing the above analysis in more detail shows also that the larger the amplifier gain , the more insensitive the overall gain 
of the feedback amplifier becomes. However, at high gains, the above circuit could become dynamically unstable because of dynamic effects in the amplifier. For amplifiers, this phenomenon is called singing, again because of the characteristic noise produced by the resistors that accompanies this instability. Nyquist, a colleague of Black at Bell Laboratories, analyzed this stability issue and came up with the celebrated Nyquist stability criterion. By pursuing these ideas further, various techniques were developed for setting the gains of feedback controllers. The sum total of these design methods was termed classical control theory and comprised such things as the Nyquist stability test, Bode plots, gain and phase margins, techniques for tuning PID regulators, lead–lag compensation, and root–locus methods.
This account of the history of control brings us to the 1950s. We will now backtrack and follow the other historical root of control, trajectory optimization. The problem of trajectory transfer is the question of determining the paths of a dynamical system that transfer the system from a given initial to a prescribed terminal state. Often paths are sought that are optimal in some sense. A beautiful example of such a problem is the brachystochrone problem that was posed by Johann Bernoulli in 1696, very soon after the discovery of differential calculus. At that time he was professor at the University of Groningen, where he taught from 1695 to 1705. The brachystochrone problem consists in finding the path between two given points A and B along which a body falling under its own weight moves in the shortest possible time. In 1696 Johann Bernoulli posed this problem as a public challenge to his contemporaries. Six eminent mathematicians (and not just any six!) solved the problem: Johann himself, his elder brother Jakob, Leibniz, de l’Hˆopital, Tschirnhaus, and Newton. Newton submitted his solution anonymously, but Johann Bernoulli recognized the culprit, since, as he put it, ex ungue leonem: you can tell the lion by its claws. The brachystochrone turned out to be the cycloid traced by a point on the circle that rolls without slipping on the horizontal line through A and passes through A and B. It is easy to see that this defines the cycloid uniquely (see Figures P.3 and P.4). The brachystochrone problem led to the development of the Calculus of Variations, of crucial importance in a number of areas of applied mathematics, above all in the attempts to express the laws of mechanics in terms of variational principles. Indeed, to the amazement of its discoverers, it was observed that the possible trajectories of a mechanical system are precisely those that minimize a suitable action integral. In the words of Legendre, Ours is the best of all possible worlds. Thus the calculus of variations had far-reaching applications beyond that of finding optimal paths: in certain applications, it could also tell us what paths are physically possible. Out of these developments came the Euler–Lagrange and Hamilton equations as conditions for the vanishing of the first variation. Later, Legendre and Weierstrass added conditions for the nonpositivity of the second variation, thus obtaining conditions for trajectories to be local minima.
Figure 1.3 [Brachystochrone].
Figure 1.4 [Cycloid].
The problem of finding optimal trajectories in the above sense, while extremely important for the development of mathematics and mathematical physics, was not viewed as a control problem until the second half of the twentieth century. However, this changed in 1956 with the publication of Pontryagin’s maximum principle. The maximum principle consists of a very general set of necessary conditions that a control input that generates an optimal path has to satisfy. This result is an important generalization of the classical problems in the calculus of variations. Not only does it allow a much larger class of problems to be tackled, but importantly, it brought forward the problem of optimal input selection (in contrast to optimal path selection) as the central issue of trajectory optimization.
Around the same time that the maximum principle appeared, it was realized that the (optimal) input could also be implemented as a function of the state. That is, rather than looking for a control input as a function of time, it is possible to choose the (optimal) input as a feedback function of the state. This idea is the basis for dynamic programming, which was formulated by Bellman in the late 1950s and which was promptly published in many of the applied mathematics journals in existence. With the insight obtained by dynamic programming, the distinction between (feedback based) regulation and the (input selection based) trajectory optimization became blurred. Of course, the distinction is more subtle than the above suggests, particularly because it may not be possible to measure the whole state accurately; but we do not enter into this issue here. Out of all these developments, both in the areas of regulation and of trajectory planning, the picture of Figure P.5 emerged as the central one in control theory. The basic aim of control as it is generally perceived is the design of the feedback processor in Figure P.5. It emphasizes feedback as the basic principle of control: the controller accepts the measured outputs of the plant as its own inputs, and from there, it computes the desired control inputs to the plant. In this setup, we consider the plant as a black box that is driven by inputs and that produces outputs. The controller functions as follows. From the sensor outputs, information is obtained about the disturbances, about the actual dynamics of the plant if these are poorly understood, of unknown parameters, and of the internal state of the plant. Based on these sensor observations, and on the control objectives, the feedback processor computes what control input to apply.
Via the actuators, appropriate influence is thus exerted on the plant.
Figure 1.5 [Intelligent Control].
Often, the aim of the control action is to steer the to-be-controlled outputs back to their desired equilibria. This is called stabilization, and will be studied in Chapters 9 and 10 of this book. However, the goal of the controller may also be disturbance attenuation: making sure that the disturbance inputs have limited effect on the to-be-controlled outputs; or it may be tracking: making sure that the plant can follow exogenous inputs. Or the design question may be robustness: the controller should be so designed that the controlled system should meet its specs (that is, that it should achieve the design specifications, as stability, tracking, or a degree of disturbance attenuation) for a wide range of plant parameters. The mathematical techniques used to model the plant, to analyze it, and to synthesize controllers took a major shift in the late 1950s and early 1960s with the introduction of state space ideas. The classical way of viewing a system is in terms of the transfer function from inputs to outputs. By specifying the way in which exponential inputs transform into exponential outputs, one obtains (at least for linear time-invariant systems) an insightful specification of a dynamical system. The mathematics underlying these ideas are Fourier and Laplace transforms, and these very much dominated control theory until the early 1960s. In the early sixties, the prevalent models used shifted from transfer function to state space models. Instead of viewing a system simply as a relation between inputs and outputs, state space models consider this transformation as taking place via the transformation of the internal state of the system. When state models came into vogue, differential equations became the dominant mathematical framework needed. State space models have many advantages indeed. They are more akin to the classical mathematical models used in physics, chemistry, and economics. They provide a more versatile language, especially because it is much easier to incorporate nonlinear effects. They are also more adapted to computations. Under the impetus of this new way of looking at systems, the field expanded enormously. Important new concepts were introduced, notably (among many others) those of controllability and observability, which became of central importance in control theory. These concepts are discussed in Chapter 5.
Three important theoretical developments in control, all using state space models, characterized the late 1950s: the maximum principle, dynamic programming, and the Linear–Quadratic–Gaussian (LQG) problem . As already mentioned, the maximum principle can be seen as the culmination of a long, 300-year historical development related to trajectory optimization. Dynamic programming provided algorithms for computing optimal trajectories in feedback form, and it merged the feedback control picture of Figure P.5 with the optimal path selection problems of the calculus of variations. The LQG problem, finally, was a true feedback control result: it showed how to compute the feedback control processor of Figure P.5 in order to achieve optimal disturbance attenuation. In this result the plant is assumed to be linear, the optimality criterion involves an integral of a quadratic expression in the system variables, and the disturbances are modeled as Gaussian stochastic processes. Whence the terminology LQG problem. The LQG problem, unfortunately, falls beyond the scope of this introductory book. In addition to being impressive theoretical results in their own right, these developments had a deep and lasting influence on the mathematical outlook taken in control theory. In order to emphasize this, it is customary to refer to the state space theory as modern control theory to distinguish it from the classical control theory described earlier.
Unfortunately, this paradigm shift had its downsides as well. Rather than aiming for a good balance between mathematics and engineering, the field of systems and control became mainly mathematics driven. In particular, mathematical modeling was not given the central place in systems theory that it deserves. Robustness, i.e., the integrity of the control action against plant variations, was not given the central place in control theory that it deserved. Fortunately, this situation changed with the recent formulation and the solution of what is called the 
problem. The problem gives a method for designing a feedback processor as in Figure P.5 that is optimally robust in some well-defined sense.