<abstract xmlns="http://www.w3.org/1999/xhtml"><p>The linear programming (LP) approach to solve the Bellman equation in dynamic programming is a well-known option for finite state and input spaces to obtain an exact solution. However, with function approximation or continuous state spaces, refinements are necessary. This paper presents a methodology to make approximate dynamic programming via LP work in practical control applications with continuous state and input spaces. There are some guidelines on data and regressor choices needed to obtain meaningful and well-conditioned value function estimates. The work discusses the introduction of terminal ingredients and computation of lower and upper bounds of the value function. An experimental inverted-pendulum application will be used to illustrate the proposal and carry out a suitable comparative analysis with alternative options in the literature.</p></abstract>

The linear programming (LP) approach to solve the Bellman equation in dynamic programming is a well-known option for finite state and input spaces to obtain an exact solution. However, with function approximation or continuous state spaces, refinements are necessary. This paper presents a methodology to make approximate dynamic programming via LP work in practical control applications with continuous state and input spaces. There are some guidelines on data and regressor choices needed to obtain meaningful and well-conditioned value function estimates. The work discusses the introduction of terminal ingredients and computation of lower and upper bounds of the value function. An experimental inverted-pendulum application will be used to illustrate the proposal and carry out a suitable comparative analysis with alternative options in the literature.

A linear programming methodology for approximate dynamic programming

University Institute of Industrial Control Systems and Computing, Polytechnic University of Valencia

International Journal of Applied Mathematics and Computer Science

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

{"article-title":"A linear programming methodology for approximate dynamic programming"}

The linear programming (LP) approach to solve the Bellman equation in dynamic programming is a well-known option for finite state and input spaces to obtain an...

A linear programming methodology for approximate dynamic programming

Publicado en línea: 04 jul 2020

Páginas: 363 - 375

Recibido: 26 oct 2019

Aceptado: 02 mar 2020

DOI: https://doi.org/10.34768/amcs-2020-0028

Palabras clave
linear programming, approximate dynamic programming, control applications, neural networks

© 2020 Henry Díaz et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

A linear programming methodology for approximate dynamic programming

Publicado en línea: 04 jul 2020

Páginas: 363 - 375

Recibido: 26 oct 2019

Aceptado: 02 mar 2020

DOI: https://doi.org/10.34768/amcs-2020-0028

Palabras clavelinear programming, approximate dynamic programming, control applications, neural networks

© 2020 Henry Díaz et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Palabras clave
linear programming, approximate dynamic programming, control applications, neural networks