Uneingeschränkter Zugang

Solving Finite-Horizon Discounted Non-Stationary MDPS

El Akraoui Bouchra

El Akraoui Bouchra

Slimane University, Laboratory of Information Processing and Decision SupportSultan Moulay, Morocco
Orcid-Profil
Suche nach diesem Autor auf
Sciendo|Google Scholar
Bouchra, El Akraoui
 und 
Cherki Daoui

Cherki Daoui

Slimane University, Laboratory of Information Processing and Decision SupportSultan Moulay, Morocco
Orcid-Profil
Suche nach diesem Autor auf
Sciendo|Google Scholar
Daoui, Cherki
  
09. Juni 2023
Solving Finite-Horizon Discounted Non-Stationary MDPS's Cover Image
Über diesen Artikel
Vorheriger Artikel
Nächster Artikel
Zitieren
COVER HERUNTERLADEN
Online veröffentlicht: 09. Juni 2023
Seitenbereich: 1 - 15
Akzeptiert: 26. Feb. 2023
DOI: https://doi.org/10.2478/foli-2023-0001
Schlüsselwörter
© 2023 El Akraoui Bouchra et al., published by Sciendo
This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.

Research background

Markov Decision Processes (MDPs) are a powerful framework for modeling many real-world problems with finite-horizons that maximize the reward given a sequence of actions. Although many problems such as investment and financial market problems where the value of a reward decreases exponentially with time, require the introduction of interest rates.

Purpose

This study investigates non-stationary finite-horizon MDPs with a discount factor to account for fluctuations in rewards over time.

Research methodology

To consider the fluctuations of rewards with time, the authors define new nonstationary finite-horizon MDPs with a discount factor. First, the existence of an optimal policy for the proposed finite-horizon discounted MDPs is proven. Next, a new Discounted Backward Induction (DBI) algorithm is presented to find it. To enhance the value of their proposal, a financial model is used as an example of a finite-horizon discounted MDP and an adaptive DBI algorithm is used to solve it.

Results

The proposed method calculates the optimal values of the investment to maximize its expected total return with consideration of the time value of money.

Novelty

No existing studies have before examined dynamic finite-horizon problems that account for temporal fluctuations in rewards.
Sprache:
Englisch
Zeitrahmen der Veröffentlichung:
2 Hefte pro Jahr
Fachgebiete der Zeitschrift:
Wirtschaftswissenschaften, Volkswirtschaft, Volkswirtschaft, andere
Zeitschrift RSS Feed