chapter 8 future developments - university of...

Chapter 8Future developments

“The most fruitful areas for the growth of the sciences were those which had beenneglected as a no-man’s land between the various established fields”

- Norbert Wiener, Cybernetics: Or Control and Communication in the Animal andthe Machine, 1948

The future of machine learning control is bright, bolstered by concrete successstories and fueled by steady advances in hardware and methodology. As pointedout by N. Wiener, there is particularly fertile ground between two well-developedfields [275], and it is likely that there will be many exciting new technological de-velopments between the fields of machine learning and control theory. This fieldwill remain vibrant as long as

(1) it addresses problems that are interesting, important and challenging,(2) improving hardware facilitates increasingly ambitious demonstrations, and(3) investments in fundamental research yield new paradigms and

mathematical frameworks.

Machine learning methods are beginning to enter the control of complex systemswith numerous demonstrations in closed-loop turbulence control. These machinelearning methods already pervade technology and information infrastructure, withsmartphones providing compelling examples. Potential applications in engineeringand industry may have an equally transformative impact. Examples include dragreduction of cars, trucks, trains, ships, airplanes and virtually any ground-, air- andwater-born transport vehicle, lift increase of airfoils, aerodynamic force control mit-igating wind gusts, internal fluid transport in ventilation systems and oil pipelines,efficient operation of wind turbines, greener and stable combustion and efficientchemical processes, just to name a few. Other applications concern more broadly anyfluid flows or networks with well defined inputs (actuators) and outputs (sensors),

171

172 8 Future developments

like water and electricity supply in civil engineering, drug dosage and interactionsfor medical treatments, or financial trading.

In this chapter, we outline promising methodological advances that are poised totransform machine learning control (Sec. 8.1). Section 8.2 is dedicated to promisingenablers for systems with high-dimensional inputs and outputs. We also highlight anumber of exciting future applications that we believe will be heavily impacted byMLC, resulting in innovative solutions to scientific, engineering, and technologicalchallenges of the 21st century [43] (Sec. 8.3). These directions are not exhaustive,but merely offer a glimpse into the exciting future developments of machine learningin the control of complex nonlinear systems. This chapter concludes with an inter-view of Professor Belinda Batten (Sec. 8.5). Prof. Batten is an internally renownedscholar in reduced-order modeling and control and pushes the frontiers of renewablewind and water energy.

8.1 Methodological advances of MLC

One of the many benefits of MLC using genetic programming is its simplicity andgenerality, requiring little or no knowledge of the system being controlled. How-ever, there are a number of targeted improvements that may expand the power andapplicability of genetic programming for MLC to a wider range of problems withlittle added complexity.

Exploiting closeness of controllers: A major goal of future efforts is to reducethe training time required for MLC using GP. Genetic programming is an ex-tremely flexible framework for building complex input–output functions, al-though these representations are not unique, and it is often unclear whether ornot two GP controllers are similar in function. Improving this notion of closenessof controllers is critical to avoid redundant testing and to reduce training time. Inaddition, having an induced metric on the space of GP controllers may result ineffective controller reduction, so that complicated, high-performance controllerscan be reduced to their simplest form. One exciting development in this directionis related to embedding the space of GP controllers in a simple metric space usinghash inputs. The function of a GP controller can be uniquely identified based onits response to a random input sequence, likely sampled on the attractor. Thesehash functions then define an embedding which is much lower dimensional andEuclidean; moreover, if the sampling is truly random and incoherent with re-spect to the structure of the controllers, then it is likely that the embedding willpreserve the controller geometry. This is closely related to compressed sensingtheory (see below) and the Johnson-Lindenstrauss theorem [147].

Multi-dimensional scaling: The progress of an evolutionary algorithm may bevisualized in a two-dimensional plane with a suitable metric of the control lawsand multi-dimensional scaling. Thus, close (distant) control laws in the metricspace are close (distant) in the visualization plane. Each control law is associatedwith a value of a cost function. Thus, the visualization plane may indicate the

8.1 Methodological advances of MLC 173

topology of the cost function, e.g. the number of populated minima, maximaor saddle points. An example of such a visualization of an ensemble of controllaws has been provided in Section 7.4.1. Multi-dimensional scaling may also beused to illustrate MIMO control laws. These visualizations may provide valuableinformation for on-line decisions during a control experiment, e.g. tuning thelevels of exploration versus exploitation.

Library building and dictionary learning: Another straightforward modificationto GP for MLC is the implementation of library building and dictionary learning.Throughout the GP training process, a tremendous range of control laws are ex-plored, and typically, only information about the final controller performance andthe GP function representation are used in future iterations. Although it is neces-sary to obtain a statistically averaged measure of performance, or fitness, for eachcontroller, it is likely that data from individual control experiments will providerich information about attractor control. In particular, there are transient events inturbulence control, and it is likely that controllers are locally exploring favorableregions of phase space. Understanding controller individual histories at a highertime resolution may provide higher performance, multi-resolution analysis andcontrol.

Multiple operating conditions: In addition, library building may result in adap-tive GP for MLC for systems with multiple operating conditions. In cases wherethere are slowly varying parameters that change the operating conditions of asystem, such as temperature, altitude, chemical composition of feed stock, etc.,there may be multiple optimal control strategies. Library building and dictionarylearning provide a robust strategy to handle these cases with multiple attractors.First, the system parameters are characterized by comparing against a library ofpreviously identified attractors; sparse identification algorithms are particularlypromising, and will be discussed below [278, 34, 107, 46, 42]. After the parame-ters are roughly characterized, the controller jumps to the best GP controller thatfits the situation; we refer to this as fast feedforward control. Subsequently, ad-ditional slow feedback control can be included to add robustness to un-modeleddynamics and disturbances. However, it is likely that the inherent feedback builtinto GP controllers will be sufficient for many situations.

Learning robustness from control theory: Finally, there is an opportunity tobridge the gap between the excellent performance obtained using MLC and therigorous theoretical foundations provided by classical control theory. Investigat-ing issues related to robust performance of MLC will continue to be importantto provide theoretical guarantees. These guarantees are particularly importantin many aerospace applications, such as boundary layer control on a wing orcombustion control in a jet engine, since they allow for controller certification.Although MLC already provides impressive robustness to varying environmentalconditions in practice, it may be possible to augment robustness using techniquesfrom classical control theory. It will also be interesting to see how the introduc-tion of filters and dynamic estimators into the MLC framework will impact thecontrol of real engineering-relevant systems. Lastly, there is great interest in thecommunity for insight gained from effective MLC.


Genetic programming is a powerful and flexible regression technique, but ma-chine learning has many more techniques that may be used for MLC. There are anumber of more general methodological advances that will likely lead to new in-novations in machine learning control in the future. The fields of machine learning,control theory, and data science in general, are rapidly growing and steady progressis being made on many fronts. Additionally, innovations in sparse sensing and uncer-tainty quantification are resulting in improved sensor and actuator placement, whichwill undoubtedly impact MLC strategies. Here, we provide a high-level overview ofsome of these exciting recent techniques.

Machine learning applications: The field of machine learning is developing atan incredible pace, and these advances are impacting nearly every branch ofthe physical, biological, and engineering sciences [190, 92, 30, 194, 168]. Thisprogress is fueled by a combination of factors, including concrete success sto-ries on real-world problems, significant investment of resources by large corpo-rations and governments, and a grass-roots enthusiasm in the domain sciences.Surrounding this movement is the promise of a better future enabled by datascience.

Machine learning of the plant: In a sense, aspects of machine learning have al-ready been in use in classical engineering design and control for decades in theform of system identification [150, 174]. The goal of machine learning, like sys-tem identification, is to train a model on data so that the model generalizes tonew test data, ideally capturing phenomena that have not yet been observed.Recent techniques in machine learning range from the extremely simple andgeneral to highly sophisticated methods. Genetic programming has been usedto identify nonlinear dynamical systems from data [32, 239, 219], and sparsevariants exist [274]. The sparse identification of nonlinear dynamics (SINDY)algorithm [44] uses sparse regression [265] for nonlinear system identification.

Neural network based control: Machine learning has already been used in anumber of control schemes for decades. In fluid dynamics, neural networks areused to identify accurate input–output maps to describe phenomena such as thegrowth of structures in a boundary layer and reduce skin-friction drag [171] as anadd-on to opposition control [66]. Other examples where neural networks havebeen used to model and control turbulence include [193, 189, 95]. Neural net-works constitute a set of bio-inspired algorithms to model input–output behaviorusing a coupled network of individual computational units, or “neurons". Thesealgorithms were widely celebrated in the 1990s, but fell out of mainstream use,when it became clear that they were unable to solve many challenging problems.For example, although neural networks may be tuned to approximate nearly anyinput–output function, they are prone to overfitting and local minima in opti-mization. However, recently, the advent of deep convolutional neural networks(CNNs) has brought these algorithms back into the forefront of research [78].In particular, deep neural networks are being constructed to identify complexphrases to describe scenery in unstructured images [69] and process natural lan-guage [132]. Perhaps more famously, these networks have resulted in the Googledeep dream.

8.1 Methodological advances of MLC 175

Genetic algorithm based control: Genetic algorithms [137, 76, 122] have alsobeen widely used to identify parameters of both models and controllers for com-plex systems, such as fluids. Unlike genetic programming, which identifies boththe structure and parameters of a model, genetic algorithms only identifies pa-rameters of a model with a fixed structure. However, both genetic algorithms andgenetic programming are evolutionary algorithms, relying on advancing gener-ations of individuals and evaluating their fitness. Genetic algorithms have beensuccessful in many applications where the structure is known.

Clustering: In addition to algorithms that identify input–output maps, there isalso wide use of machine learning algorithms for classification or clustering data.For example, many complex systems are non-stationary and may be character-ized by multiple attractors in different parameter regimes. In these systems, it isoften not necessary or even helpful to estimate the full high-dimensional stateof the system, but instead it may be sufficient to characterize the coarse systembehavior. In this case, classification algorithms [279] (K-means, K-nearest neigh-bors, LDA, QDA, SVM [252, 241, 258], Decision Trees, Random Forests, etc.)are particularly useful. These classification algorithms are generally categorizedinto supervised algorithms, where the training data is labeled, and unsupervised,where the relevant clusters are unknown ahead of time. Classification is espe-cially important in control, as each classification may lead to a control decision.For example, in ultrafast laser systems with multiple operating regimes, sparseclassification algorithms have been used to rapidly identify the operating condi-tions, after which the controller jumps to a pre-determined near-optimal controlsetting and a slower adaptive control algorithm adds stability and disturbancerejection [107, 42]. In fluids, classification of operating regimes is also beingexplored [34, 46, 14]

Reinforcement learning: In robotics, the combination of machine learning andcontrol has been proceeding steadily for at least a decade. Techniques for au-tonomous learning and reinforcement learning [257] have been widely adoptedto control robots, both autonomous and tethered. Iterative learning control is usedfor tracking control, for example of robot arm position [35]. Similar algorithmsare also used in brain-machine-interface problems, such as to train prostheticdevices.

The field of machine learning, and data science more generally, relies on a num-ber of key steps: 1) data scrubbing, 2) feature extraction, mining, or engineering,and 3) model building. Each stage is crucial. The argument that machine learningwill replace highly skilled domain scientists and engineers is somewhat facetious,considering that these algorithms do not work without a notion of what is importantin a problem. For example, determining which labels should be used for trainingand engineering relevant features to distinguish various aspects of the data remainhighly creative and critical human tasks in many disciplines.


8.2 System-reduction techniques for MLC — Coping withhigh-dimensional input and output

This textbook contains examples multi-input multi-output (MIMO) control with fewinputs and outputs. The learning of control laws can be expected to take longeras the number of inputs and outputs increases. This is particularly true for high-dimensional actuation like surface motion or high-dimensional sensing like real-time particle image velocimetry, like in Section 6.1, or image-based feedback. Inthis section, we outline approaches coping with such high-dimensional input andoutput.

System reduction: Dynamical systems and control strategies have long bene-fited from dimensionality reduction, relying on the fact that even complex high-dimensional systems typically exhibit low-dimensional patterns that are rele-vant for models and control [125, 138]. More generally, it has been observedfor decades that nearly all natural data signals, such as audio, images, videos,laboratory measurements, etc., are inherently low-dimensional in an appropriatecoordinate system, such as Fourier or Wavelets. In this new coordinate system,many of the coordinates of the data will be negligibly small, so that the originalsignal can be accurately represented by a sparse vector, containing mostly zeros,in the new basis. This inherent sparsity of natural signals is the foundation fordata compression, such as MP3 for audio and JPEG for images. PCA provides atailored basis for optimal low-rank representation of data.

Compressed sensing: A recent mathematical breakthrough, called compressedsensing [50, 84, 18, 266, 52], has upended the traditional paradigm of collectingand analyzing data. Instead of painstakingly measuring every variable in a sys-tem, only to compress and discard the majority of the negligible information in atransform coordinate system, it is now possible to measure significantly less dataat the onset and infer the relevant terms in the sparse transform basis.In the past, solving for these relevant terms from subsampled measurementdata would amount to a brute-force combinatorial search. This class of non-polynomial-time, or NP-hard, problem does not scale favorably with the sizeof the data, and Moore’s law of exponentially growing computer power does notscale fast enough to help. A major breakthrough in compressed sensing is an al-ternative algorithm based on convex optimization using the sparsity-promoting`1 norm: kxk1 = ÂN

k=1 |xk|. It has been shown that under certain reasonable con-ditions, solving for the vector x with the smallest `1 norm will approximate thevector with fewest nonzero entries. The measure of nonzero entries is often calledthe `0 “norm", although it does not satisfy the properties of a norm; it is alsoequivalent to the Hamming distance of the vector from the zero vector, as in in-formation theory. This convex optimization does scale favorably with Moore’slaw, providing a clear path to solve increasingly large problems in the future.Another innovation surrounding compressed sensing is the establishment of clearconditions on when the theory will work. First, there must be sufficient measure-ments to estimate the nonzero transform coordinates; this minimum number of

8.2 System-reduction techniques for MLC — Coping with high-dimensional input and output177

measurements depends on the original size of the vector, the expected sparsity ofthe vector in the transform basis, and the level of noise on the signal. Next, thesamples must be sufficiently incoherent with respect to the vectors that definethe sparsifying basis; in other words, measurements should be as orthogonal, aspossible, to all of the basis directions. It was shown that random projection mea-surements of the state (i.e., taking the inner product of the state vector x with avector of identical independently distributed Gaussian elements) provide nearlyoptimal compressed measurements, regardless of the sparsifying basis. This istruly profound, as this enables a universal sparse sampling strategy. However,random projections of the state are not necessarily physical measurements, sincethey still require having access to all of the information in x. A more useful en-gineering measurement is often the spatially localized point measurement, corre-sponding to a single physical sensor. Fortunately, single point measurements areincoherent with respect to the Fourier transform basis, so that many signals maybe reconstructed from few point measurements.Regardless of our interest in full signal reconstruction, there are huge implica-tions of sparsity promoting techniques and compressed sensing in engineering.Many of the concepts are applicable more generally to machine learning, mod-eling and control, including measurement incoherence, working on compressedsubspaces of the data, and the structure and sparsity of data in general. For ex-ample, if a categorical or a control decision is desired, as opposed to a full-statereconstruction, it is often possible to use randomly sampled measurements andstill achieve high classification performance. It is also useful to consider the effectof subsampling or projecting data onto a low-dimensional measurement space.The Johnson-Lindenstrauss theorem [147] and the restricted isometry property(RIP) [51] provide powerful quantification of the distortion of high-dimensionalinner products after the data is compressed. These concepts are closely relatedto unitarity, and they may be used to embed high-dimensional data, or functions,as in the case of genetic programming, in a low-dimensional metric space. Hashfunction embedding is already being used in fluid flow control [183].

Sparse sensor and actuator placement: The placement of sensors and actuatorsis a critically important stage in control design, as it affects nearly every down-stream control decision. However, determining the optimal placement of sensorsand actuators is an NP-hard problem, which does not have an elegant solution,but rather involves a brute-force combinatorial search. In particular, this strategydoes not scale well to even moderately large problems of interest. To compoundthe difficulty, often sensors may be quite expensive, as in the case of putting trac-ers in the ocean or human vaccinators in a foreign country. Even if sensors areinexpensive, processing the vast streams of data from a sensor network may beintractable, especially for mobile applications which are power and computation-ally limited.Recent advances in compressed sensing (see above) are poised to revolutionizesensor and actuator placement for control problems. From an engineering per-spective, the ability to solve NP-hard optimization problems with convex opti-mization routines is transformative. Fortunately, the sensor placement problem


may be thought of as a compressed sensing problem under certain conditions, aswe are trying to find the few best locations to maximize a well-defined objec-tive function. Sparse sensor optimization using compressed sensing techniqueshas already been implemented for categorical decision making in complex sys-tems [39, 14]. Extending these methods to optimize sensors and actuators fora control objective is an important area of current research. Finally, extendingthe compressed sensing framework to dynamic point measurements, such as La-grangian tracer elements, has huge potential for the fields of oceanographic andatmospheric sampling.

8.3 Future applications of MLC

MLC can be expected to address many applications from everyday life to industrialproduction. Many control problems have a well-defined cost function, a finite num-ber of sensor signals (outputs) which monitor the state and a finite number of actu-ation commands (inputs) which shall improve the system performance. In short, wehave a multiple-input multiple-output (MIMO) plant and search for a control logicwhich optimizes a cost function.

Most factory processes fall in the category of a MIMO control problem. Åströmand Murray [222] provide an excellent introduction in the world of control prob-lems. The key strength of MLC comes into play when the plant behavior departsfrom linear dynamics and eludes current methods of model-based control.

Feedback control of turbulence is a grand challenge problem for control design asthe plant incorporates three key difficulties: (1) high dimension, (2) nonlinearity, and(3) potentially large time-delays (see Chapter 1). Turbulence increases the powerrequired to pump gas and oil in pipe-lines and is a major source of drag of ground,airborne, and maritime transport. Wind turbulence increases maintenance costs ofwind turbines via gusts and creates dangerous situations for ground- and airbornetransport. In contrast, turbulence has desirable mixing properties in heat exchangers,combustion and chemical processes.

Effective turbulence control can contribute to a key challenge of modern society:renewable energy production and reduced energy consumption. The world’s primaryenergy supply has more than doubled from 6,106 Mtoe1 in 1973 to 13,371 Mtoe in2012 [3]. The consumption in 2012 was 1,555 ⇥ 1017 Joule corresponding to anaverage rate of 17.75 TW or 2.5 kW per person on this planet. This corresponds toone powerful heater per person operating day and night. On the downside, the envi-ronmental cost of energy production is an increasing challenge. Part of this cost isimmediately felt as smog in the city or as noise near streets, railroads and airports.Other costs are indirect long-term consequences: Coal and fuel consumption pro-vided 60.4 % of the 2012 energy supply [3] and are particularly problematic becausethe CO2 emissions affect our climate. Our very existence is threatened by man-

1 Mtoe means million ton of oil or equivalent (11.63 TWh).

8.3 Future applications of MLC 179

made global warming. The Nobel committee has appreciated this fact by awardingthe 2007 Peace Nobel Price to the International Panel on Climate Change and AlGore “for their efforts to build up and disseminate greater knowledge about man-made climate change, and to lay the foundations for the measures that are needed tocounteract such change” [1].

On the demand side, 20% of the worldwide energy consumption is used for trans-port, mostly by oil [82]. This demand corresponds to a cube of oil with each sidemeasuring nearly 14 km. In 2014, the USA used 28% of its energy consumptionfor transport with 92% provided by fuel. The economic price of world transportamounts to 10% of the world gross domestic product (GDP) in 2011 [82].

The relevance of sustainable traffic may be measured by the fact that the Euro-pean Research Council supports over 11,000 research grants on this topic. In fol-lowing, we describe several transport-related opportunities for MLC:

Drag reduction of trucks: About one third of the operating costs of truckingcompanies is for fuel. The average profit margin is 3–4% in the USA. Thus, a5% reduction on fuel costs, for instance by active flow control, spells a substan-tial increase of the profit. Passive means, like vanes for the rear side are alreadyfor sale. Active control solutions have already been tested on the road and haveproven to save 5% fuel already under real-world traffic conditions. Experimentswith small-scale vehicles demonstrate the possibility of a 20% drag reduction[211, 19] at a fraction of the energy cost.

Drag reduction of cars: Personal cars are less subject to economic considera-tions as in the trucking industry. However, the European union has identifiedthe development of sustainable transport as a cornerstone challenge for the nextdecades. To encourage efforts in that directions, European union legislation [202]has set mandatory emission reduction targets for the car industry. The new stan-dards impose a reduction of 30% of greenhouse gas emissions and correspondingfuel consumption for new cars by 2020. Meanwhile, the transportation industryis a strategic economic sector in Europe accounting for 4.5% of the total employ-ment (10 million people). However, the market for new vehicles in Europeancountries is declining, whereas that of emerging countries is rapidly growing.To remain competitive in this market, European car manufacturers must developdisruptive technology concepts to produce safer and more sustainable vehicles.Improving the aerodynamic performance of road vehicles by flow control canhelp fullfill these requirements, in particular at highway speeds. At highwayspeeds, overcoming aerodynamic drag represents over 65% of the total powerexpense [140, 184]. This explains significant aerodynamic improvements on au-tomobiles during the last decades. Most existing flow control approaches to re-duce vehicle drag on commercial road vehicles are passive [118]. They rely onaerodynamic shaping, such as well-rounded contours (especially at the front),rear slope angle, and add-on devices [70]. Such passive approaches, however, arerestricted by design and practical considerations and cannot be ‘turned off’ whennot needed or modified to adapt to changing conditions.Recently, significant research has been focused on active flow control (AFC)solutions. Cattafesta & Shelpack [54] give an extensive overview of possible ac-


tuation mechanisms, whereas [65] present the most common AFC approacheson bluff bodies. Of the many available approaches, a large subset is consideredas an academic exercise, such as rotary [81, 27], streamwise [58], and trans-verse [53, 31] oscillation of a bluff body. Of the practical and realistic AFCmechanisms, many were investigated on the Ahmed body [4], such as syntheticjets [207], pulsed pneumatic actuation [149, 225], microjects with steady blow-ing [10], and Coanda blowing [104, 116]. AFC on heavy truck vehicles were alsoinvestiged, such as synthetic jet actuators [96], and Coanda blowing [97, 98].A drag reduction of 23% has been achieved on a truck model with steady Coandablowing [211]. The saved towing power was 3 times the invested actuation en-ergy. A better actuation efficiency of 7 has been achieved with high-frequencyCoanda blowing for a blunt-edged Ahmed body by [20] resulting in a 18% dragreduction. Both studies were based on a working open-loop control. The imple-mentation of MLC is the logical next step after a working AFC. Machine learningcontrol has recently been shown to improve existing control of an Ahmed bodyand a small-scale Citroën model in OpenLab PPRIME/PSA, France.

Safety of cars and trucks under wind gusts: Side-wind stability is important forpassenger comfort and safety of all ground vehicles. It is particularly criticaland a safety issue for vehicles with large projected side area, such as trucksand buses. Accidents linked to crosswind have been reported by several govern-mental agencies [48]. Side-winds and side-gusts originate from different sourcessuch as weather, surrounding traffic, or the topology of the terrain next to theroad. Several numerical [126, 145] and experimental [118, 59] studies were con-ducted to understand the dynamics of such flows. Both steady [139, 15] andunsteady [59, 267] crosswind investigations have been researched. The findingsof these aforementioned studies and others can be summarized as follows:

• The driving stability decreases with increasing speed.• A reduction of the lateral projected area in the back will reduce the rear side

force and thereby the yaw moment. However, this relation is not valid forwell-rounded rear-end configurations (C- and D-pillars).

• A-pillar radiusing has a large influence on the yaw moment.• The center of pressure of the aerodynamic forces has a large impact on the

vehicle stability.• Directional stability depends more on yaw moment than on the overall side

force.• For a wide range of vehicles, the yaw moment increases approximately lin-

early with the yaw angle up to 20�.

Despite the numerous studies on side-wind effects and driving stability, very fewhave tackled the issue using AFC. To our knowledge, only Englar [97, 98] andPfeiffer et al. [211, 212] applied Coanda blowing on the 4 rear edges of a truckto reduce drag and to control the yaw moment. Whereas Englar [97, 98] onlyimplemented AFC as an open-loop, Pfeiffer et al. [211, 212] successfully ap-plied it as a closed-loop control. The latter were able to achieve a leading 23%


drag reduction and a complete authority over the yaw moment. However, furtherimprovements on the previous results can be achieved:

• Distributed actuation comprising all locations with demonstrated effect ondrag and yaw;

• Distributed sensing providing real-time flow information;• Employing very general, nonlinear and robust control laws comprising peri-

odic, multi-frequency, and adaptive in-time closed-loop control;• Reduction in actuation power for both drag minimization and yaw control.

All these new possible improvements have two themes in common, safety andefficiency. MLC can optimize open-loop, adaptive and closed-loop control byusing harmonic functions as additional arguments of the control laws.

High-lift configuration of passenger aircraft: Civil aircraft are highly optimizedfor cruise conditions. Typical lift-to-drag ratios are around 17. This means 1Newtons of thrust lifts 17 Newton of weight at a cruise speed around 850 km/h.During landing the main goal is not a good lift-to-drag ratio but a steep descentat low velocity to reduce the noise signature on ground. This is achieved with ahigh-lift configuration through a reeled-out flap. Active control can prevent earlyseparation and reduce the size of these flaps for a given lift. Thus the weight ofthe aircraft is reduced during cruise resulting in lower propulsion power and thusreduced fuel consumption. This opportunity is being pursued by the two majorpassenger aircraft producers, Airbus and Boeing, and is the subject of intenseresearch.The economic aspect of reduced fuel consumption may be appreciated from thefollowing data: Average profit margins are between 1–2 %. Fuel is with a 28%contribution the most important operating cost of passenger airlines (2014, USairlines). Fuel may represent around 40% of the maximum take-off weight. Theairline not only saves fuel costs with active flow control. More importantly, it mayreplace 80 kg fuel by a paying passenger. In a 100 passenger airplane one pas-senger amounts to 1% of the income and thus accounts for a significant changein the profit margin.

Nacelle of a passenger aircraft: During take-off, the engines produce about 6times the thrust during cruise and can lift around 30% of the passenger aircraft.The large nacelle prevents the recirculation bubble from extending to the com-pressor, which would result in a dangerous loss of thrust. The large nacelle isonly needed for about one minute of maximum thrust during take-off. Activeflow control at the inlet may reduce size the nacelle and thus reduce weight. Thisis another opportunity to save fuel.

Cavity of aircraft and trains: Wheel cavities of aircraft are a major noise sourceduring landing. Similarly, cavities between train wagons produce noise. Thisnoise can be significantly mitigated by feedback flow control [231]. The cavitynoise reduction is a subject of intense research.

Drag reduction of ships: All moving ships create a wave pattern on the water.These waves require energy to be created and cause additional drag on the ship.For small ships, wave drag may be 80% of the total drag! One may conceive ac-


tive vanes at the front of the ship to mitigate part of the wave drag. For full-scaletankers, the wave drag reduces to 20% of the total drag but is still substantial.An expected 5% reduction of drag is considered as a threshold value for engag-ing in a new expensive ship design. The remaining portion of the drag is due toskin-friction.

Skin friction reduction: About 80% of the propulsion power of tankers andocean liners are due to skin friction drag. Future ships may profit from dragreduction due to hydrophobic surfaces. Similarly, skin friction is the major dragsource of passenger airplanes. Riblets have been shown to reduce skin frictiondrag by up to 11% and are actively pursued by aircraft manufacturers. Skin fric-tion may also be mitigated with distributed local suction and blowing. Drag re-ductions of 25% have been reported in low Reynolds number simulations [66]. Atlarger Reynolds numbers the reduction decreases. The distributed suction/blow-ing acts on plus unit scale to fight sweeps and ejections. An experimental realiza-tion would require myriad of actuators and sensors and will not be feasible for along time. A more realistic active control is oscillatory wall motion. Numericalsimulations [153] with high frequency oscillation show a 40% drag reduction.In experiments at RWTH Aachen, a 5% drag reduction was obtained. Feedbackcontrol with wall motion has hardly been explored.

We briefly mention other opportunities of feedback turbulence control and henceMLC.

Rockets: Solid fuel rockets may develop acoustic instabilities in the interior bodywhich may destroy the rocket. Evidently, this is an opportunity for closed-loopcontrol.

Wind turbines: Wind gusts create sudden loads on the rotor hub and bearingsof a wind turbine. These forces reduce the mean-time between failure and thusmake the maintenance effort more costly. Feedback controlled flaps at the trailingedge of the airfoil can reduce the unequal loading. The development of suitablehardware solutions and corresponding control logic is an active area of research.

Water turbines: Many underwater flows, such as in a river, may produce a moresteady loading. Preliminary results of the authors show that accelerating and de-celerating a cross-flow turbine to excite unsteady fluid forces can increase theenergy efficiency by 80 % [255].

Gas turbines: Lean combustion reduces COX and NOX emission. However, inthe lean limit, the danger of blow out or combustion instabilities increases.Closed-loop mitigation of combustion instabilities via the change of fuel sup-ply or fluid mechanic actuators are a demonstrated opportunity towards greenerenergy production [114].

Combustion engines: The internal combustion engine is ubiquitous in manymodes of transportation, providing propulsion for most motorbikes, cars andtrucks and convert chemical energy through combustion in mechanical energyvia a periodic piston operation. Despite the periodic piston movement, the flowexhibits large cycle-to-cycle variations. Some cycles are good for combustion,particularly if the fuel is well mixed in the piston by turbulent mixing. Some cy-


cles are less efficient. An ongoing effort of all engine producers is to reduce thesecycle-to-cycle variations and to stabilize a uniformly good mixing. Fuel injectionor mechanical actuation may be elements of a feedback stabilization for a betteroperation.

Pipe-lines: The power needed to pump oil through pipe-lines can be signifi-cantly reduced by the addition of polymers. One polymer molecule per 1 mil-lion oil molecules may reduce drag by 40%. Closed-loop turbulence control canhardly compete with this efficiency. However, there are still large opportunitesfor closed-loop control for gas-liquid-solid mixtures close to an oil plant.

Air conditioning: The goal of air conditioning is increased comfort. Ventila-tion shall refresh the air at appropriate temperature without unpleasantly felt airstreams. MLC may learn an optimal air stream management based on sensorinformation.Air conditioning causes other problems in hospitals. Airborne viruses need to beneutralized at active surfaces in the ducts. Such surfaces cause additional pressurelosses and require thus a larger ventilation power. The mixing problem may beformulated as follows: a fluid particle with a potential virus must be sufficientlyclose to the active surface at least once with near certainty. However, multipleencounters with the active surface would lead to unnecessary pressure losses.This is an exciting mixing problem for closed-loop turbulence control.

Pharmaceutical and chemical processing: The industrial production of food,e.g. chocolate, may happen in vessels with diameters up to 5 meters. Differentconstituents need to be uniformly distributed in these vessels. The mixing maybe closed-loop controlled by the inlets and mechanical mixing and monitored bycameras. Numerous production processes require a good mixing in vessels or inpipes. This is another exciting closed-loop mixing problem for which MLC ispredestined. The effect of mixing is highly deterministic yet hardly predictable.Mixing has largely eluded an intuitive understanding.

The previous examples have focused on turbulence control. The application ofMLC requires only a finite number of actuators and sensors. In addition, the costfunction needs to be strongly related to the control law. In other words, the controlledplant exhibits statistically stationary behavior. Many complex systems fall in thiscategory. Examples include

Predictions for solar and wind energy: The operation of an electricity networkshall satisfy the unsteady demand and prevent overproduction as the storagecapacities are limited. Hence, the production of renewable energy needs to bepredicted from weather and other data to identify potential supply-and-demandproblems early in time. This is an estimation problem which can also be solvedwith genetic programming [219]. Here, the input is the weather and other data,the output the energy production and the cost function the difference betweenestimated and actual production.

Short-term trading at stock markets: Here, the plant is the stock market, theinput the trading action of one player, the output the supply and demand curve,the control logic the automated trading, and the objective function the daily


profit. Long-term trading is far more challenging for automated learning as theassumption of statistical stationarity is not satisfied. The Dow Jones index, forinstance, has the long-term tendency to increase. Companies may be founded,new economies may appear, or old economies may disappear, etc.

Dog training: All dogs, in fact all animals, are trained with rewards (and punish-ment). Intriguingly, maximum animal performance is not obtained with a mono-tonic performance-reward curve. There exist many education concepts how thereward should depend on animal performance. MLC might yield novel concepts.

These examples show, pars pro toto, the many MLC opportunities for MIMOplants which are of large relevance in industry or everyday life. As a disclaimer,we note that there also exist many problems which are more suitable for a model-based control strategy. For instance, the effect of control surfaces on the motion ofan aircraft is well represented by a linear model for the usual operating envelope.Control design based on these models reliably cover many operating conditions.Thus, the robustness of control laws may be estimated. Continuing to establish adeeper connection between MLC and traditional control analysis will be essentialto ensure broad adoption of these methods, as discussed in Chapter 4.

The flow around the airplane is turbulent but the time-averaged effect of myriadof vortices yields a nearly linear relationship between motion of the control sur-face and aerodynamic force. Taking the time scale of the vortices as reference, themodel-based operation of the control surface can be considered as slow adaptivecontrol. Mathematically, the situation might be compared to statistical thermody-namics where myriad of gas molecule collisions, i.e. strongly nonlinear events, canstill lead to a linear relation between pressure and temperature in a finite volume bystatistical averaging.

8.4 Exercises

Exercise 8–1: Investigate each of the following machine learning clustering algo-rithms and think about a representative data set that works well with the methodand a representative data set that does not work with the method: K-means, lin-ear discriminant analysis (LDA), support vector machines (SVM), decision trees.

Exercise 8–2: Consider the following signal, which is obtained as the sum of twosine waves:

f (t) = sin(114p Hz t)+ sin(1042p Hz t).

The Shannon-Nyquist sampling theorem [200, 247] indicates that one must mea-sure at 1042 samples per second to accurately reconstruct this signal. However,because the signal is sparse in the Fourier domain (i.e., only two Fourier modesare active at 57 Hz and 521 Hz), we may reconstruct this signal from dramaticallyunder-sampled measurements.

8.4 Exercises 185

In this exercise, create a time-resolved signal f by sampling at 1042 samples persecond for 10 seconds. Now sample this signal randomly for 10 seconds with anaverage sampling rate of 128 samples per second. You may think of the sampledvector fs as the output after applying a measurement matrix C to the full time-resolved signal:

fs = Cf,

where f is a time-resolved vector of the signal: f =⇥

f (D t) f (2D t) · · · f (ND t)⇤T ,

and C contains random rows of the N ⇥N identify matrix.Finally, we may solve for the active Fourier modes in the compressed sensingproblem:

fs = CY f̂,

where Y is the inverse discrete Fourier transform. Use a convex optimizationroutine, such as cvx in Matlab®, to solve for the sparsest vector f̂ that solves theunderdetermined system above.

Exercise 8–3: Consider the LQR stabilization task from Exercise 4–1. Here, wewill define a metric on the space of controller functions in an attempt to improvethe convergence time of genetic programming control. First, create one hundredrandom states a where each component a1 and a2 are Gaussian distributed abouta1 = a2 = 0 with unit variance. Now, for each controller, b = K(a), evaluate thecontrol law at these same one hundred random states, and store the value in a100 ⇥ 1 vector bhash. It is now possible to use these vectors to construct a proxydistance between two controllers b = K1(a) and b = K2(a):

d(K1,K2) , kbhash,1 �bhash,2k2. (8.1)

Using this induced metric, modify your genetic programming search strategy toimprove the convergence rate by reducing the number of redundant controllerfunctions tested. For instance, when mutating, you might check if a new individ-ual is sufficiently similar to individuals from a previous generation, and imposeadditional distance criteria to improve exploration and exploitation.

Exercise 8–4: Pick a future application of MLC from Sec. 8.3 and explore thecurrent control approaches being applied tho this system. What are the challengeswith applying MLC to this problem?


8.5 Interview with Professor Belinda Batten

Belinda Batten is Professor of MechanicalEngineering at Oregon State University, OR,USA. Professor Batten is the Director of theNorthwest National Marine Renewable En-ergy Center (NNMREC), a collaboration be-tween Oregon State University, the Universityof Washington, and the University of AlaskaFairbanks. NNMREC supports wave, tidal,offshore wind, and in-river energy harvest-ing through research and testing. The con-sortium was established by the U.S. Depart-ment of Energy to facilitate the developmentof marine renewable energy technologies viaresearch, education, and outreach.

Professor Batten is internationally renownedfor her research in modeling and control ofdistributed parameter systems, especially for her development of computational al-gorithms for reduced-order controllers. Her current research projects include math-ematical modeling and control of autonomous vehicles and wave energy devices.She has raised significant funding to support her research from DARPA, DoD, NSF,and DOE.

Professor Batten has been a Program Manager for dynamics and control at theAir Force Office of Scientific Research, after which she was elected member of theScientific Advisory Board for the U.S. Air Force. She was also a professor of math-ematics at Virginia Tech, and served as Department Head for Mechanical Engi-neering from 2003-2007 and as Head of the School of Mechanical, Industrial andManufacturing Engineering from 2007-2011 at OSU. Her research was honored bynational and international awards, for instance by the prestigious Alexander vonHumboldt fellowship.

Authors: You are a leader in the field of marine renewable energy and have de-veloped numerous innovative technologies and control solutions. What were themain trends in marine renewable energy, especially related to fluid modeling andcontrol, in the past decade?

Prof. Batten: I will focus my comments on wave energy, as that is where myprimary expertise lies. Prior to this past decade, there has been a good amountof research on developing models of wave energy converters, and some work oncontrolling them. The modeling work typically leveraged the great volume of lit-erature on hydrodynamics of ocean structures. The work on control has typicallybeen model-based optimal control, and often has ignored the “control costs" thatarise through actuation. That is, the results on the amount of energy producedthrough active control were often highly theoretical.

8.5 Interview with Professor Belinda Batten 187

In the last ten years, more work has been done in experimental validation of com-putational models. Computational codes for simulating wave energy converterscontinue to be developed. Recently, codes are being developed for design of ma-rine energy arrays, both wave and tidal current energy converters.Some researchers are working on high fidelity fluid-structure interaction modelsfor wave and tidal energy converters. These codes are not currently suitable forcontrol design and computation as they require millions of state variables. Ascomputational capabilities continue to progress, these codes may be more usefulin the control regime, but at this point, model reduction of some type is requiredto use them for purposes other than simulation, e.g., for control or optimization.

Authors: You were an early advocate of merging control theory with fluid me-chanics. Do you see any key similarities or differences in the current efforts tobring machine learning into the fluid dynamics community?

Prof. Batten: One of the challenges in merging control theory with fluid dynam-ics in the early years was learning each others’ language. Basic terms like “con-trol" were used differently by control theorists and by fluid dynamicists. I remem-ber giving a kick-off talk at a meeting of the two groups, laying the foundation forthe discussion – including the typical nomenclature for mathematically describ-ing a fluid system with a control. One of the fluid dynamicists responded, “youmean my entire life’s work in actuator modeling is reduced to a B matrix?" So,part of getting the two groups to work together was facilitating a common under-standing of what problems were viewed as easy, hard, unanswered, unanswer-able, tractable, etc. Once these communities began to interact with each other,great progress was made, and a lot was learned.I hope that bringing machine learning into the community is a little easier be-cause some of the same people in the controls and fluid dynamics communitiesare involved, and collaborative work across disciplines starts with relationships.That said, I remember years ago when a seminar speaker talked about using neu-ral networks to develop a model for the Navier-Stokes equation; several in theaudience muttered about why anyone would throw out the physics.And that illustrates the crux of the issue. There are some really hard problems forwhich developing high fidelity physics based models for control is not feasible—at least not with today’s computational limitations. The fluid-structure interactionmodeling of a wave energy converter in the ocean is one of these problems. So,to develop machine learning approaches that we can test and verify on smallertractable problems can lead to solutions and insights on the more complex onesfor which developing physical models for control is infeasible.

Authors: Could you comment on the societal impact of marine renewable en-ergy, and how you see fluid mechanics and control helping to address the keychallenges?

Prof. Batten: The reality is that the world cannot continue to rely on fossil fuelsto meet energy needs. The supply of fossil fuels will eventually be exhausted, and


in the meantime, the impact of fossil fuels on our environment is increasingly de-structive. While people may debate about climate change, the acidification of ouroceans is measureable, and the consequences of delaying the move to renewableenergy are sobering.In my opinion, replacing fossil fuels will require a portfolio of renewables thataddress particular needs of place. Marine renewables could be an important ad-dition to this portfolio; this energy source is highly predictable, always present,and located within 50 miles of 50 percent the world’s human population. Tidesare predictable 100 years in the future; waves are predictable 84 hours out. Thosehorizons make it simple for a utility to manage electricity coming onto the grid,and one can control marine energy converter arrays to deliver what is needed.Unlike wind and solar, these energy sources are always present.The key challenge for marine renewable energy is becoming cost competitivewith other renewable energy sources. Control of marine energy converters hastypically been directed toward maximizing energy extraction. Maximizing en-ergy produced indeed contributes to a lower cost. However focusing solely onthat objective ignores the reduction in life expectancy of a wave energy converteror increase in operations and maintenance costs due to fatigue or failure of con-verters under extreme events. It may be that for certain converters, control shouldalso be used to minimize fatigue. At Oregon State University, my colleague TedBrekken has been working with his students to develop life extending control toaddress this concept.If computationally tractable models of fluid structure interactions of wave en-ergy converters—especially under extreme wave conditions—could be devel-oped, such models could be used to better predict the reliability and survivabilityof devices. Alternatively this topic could be fertile research ground for machinelearning. Better understanding reliability and survivability of devices could beleveraged to understand how to lower the cost of energy of the wave energy con-verters, thus contributing to adoption of marine energy.

Authors: In other fields, such as astronomy and particle physics, there are well-funded grand-challenge problems that the community tackles in a coordinatedeffort. Is there a need for similar larger scale collaborative efforts in fluid flowcontrol? If so, what do we need to do in order to get fluid dynamics to a similarposition in terms of funding and support?

Prof. Batten: I think great progress has been made in fluid flow control since theearly 2000s when I was challenging the communities to come together and workcollaboratively on these problems. A lot has been learned about what controlcan do and what kinds of models and computational codes are useful. I thinkthe big challenge problem for fluid dynamics related to marine energy is thefluid-structure interaction models, suitable for control design and optimization.A workshop that pulls community experts together to define other such problemsmight be an important next step.

8.5 Interview with Professor Belinda Batten 189

Authors: As the director of the Northwest National Marine Renewable EnergyCenter (NNMREC), could you comment on some of the grand challenges inthe marine renewable energy sector, and how they may be impacted by machinelearning methods in the coming years?

Prof. Batten: If there is a fundamental grand challenge in marine renewable en-ergy, it is how to lower the cost of energy of these technologies to make themcompetitive. While there are many subsystems within a marine energy array withhighly technical underpinnings and each contributes to the overall cost, it is nec-essary to take a system level viewpoint of this problem. Applying machine learn-ing to a system level performance of a marine energy array could have interestingoutcomes.

Authors: This chapter considers future developments and extensions to machinelearning control. If you were giving advice to a new student, what direction wouldyou point them in for important research problems, and what tools would youwant them to be equipped with?

Prof. Batten: I’ll leave the advice about new research directions in machinelearning to the experts in that area. Regarding the tools that I want students toknow, I believe that it is important for students interested in controls, be it learn-ing based methods or model based methods, to understand the value and limita-tions of the various approaches. When it comes down to it, any control approachis a tool, and it’s important to know when to use a screw driver and when to usea hammer. There are times when you could use either, but one is probably betterfor the job than the other.

Authors: We look forward to your continued breakthroughs in control and re-newable energy, and we thank you for this interview!

chapter 8 future developments - university of...

Documents