1. bookVolume 16 (2020): Issue 1 (May 2020)
Journal Details
License
Format
Journal
eISSN
1339-0015
ISSN
1336-9180
First Published
13 Aug 2012
Publication timeframe
2 times per year
Languages
English
access type Open Access

Geometry of the probability simplex and its connection to the maximum entropy method

Published Online: 09 Jul 2020
Page range: 25 - 35
Journal Details
License
Format
Journal
eISSN
1339-0015
ISSN
1336-9180
First Published
13 Aug 2012
Publication timeframe
2 times per year
Languages
English
Abstract

The use of geometrical methods in statistics has a long and rich history highlighting many different aspects. These methods are usually based on a Riemannian structure defined on the space of parameters that characterize a family of probabilities. In this paper, we consider the finite dimensional case but the basic ideas can be extended similarly to the infinite-dimensional case. Our aim is to understand exponential families of probabilities on a finite set from an intrinsic geometrical point of view and not through the parameters that characterize some given family of probabilities.

For that purpose, we consider a Riemannian geometry defined on the set of positive vectors in a finite-dimensional space. In this space, the probabilities on a finite set comprise a submanifold in which exponential families correspond to geodesic surfaces. We shall also obtain a geometric/dynamic interpretation of Jaynes’ method of maximum entropy.

Keywords

MSC 2010

[1] Arsigny, V., Fillard, P., Pennec, X. and Ayach, N. (2007). Geometric Means in a Novel Vector Space Structure on Symmetric positive definite matrices, SIAM J. Matrix Theory, 29, 328-347.Search in Google Scholar

[2] Amari, S.-i. “Differential Geometric Methods in Statistics”, Lecture Notes in Statistics, 28, Berlin (1985).10.1007/978-1-4612-5056-2Search in Google Scholar

[3] Amari, S.-i. “Information Geometry and its Applications”, Springer (2016).10.1007/978-4-431-55978-8Search in Google Scholar

[4] Amari, S.-i., Barndorff-Nielsen, O., Kass, R., Lauritzen, S. and Rao, C. “Differential Geometry in Statistical Inference” Institute of Mathematical Statistics Lecture Notes, Monograph Series, Vol. 10, Hayward, (1987).10.1214/lnms/1215467060Search in Google Scholar

[5] Barndorff-Nielsen, O. “Information and Exponential Families in Statistical Theory”, Chichester, Wiley (1978).Search in Google Scholar

[6] Brown, C.C. (1985) Entropy increase and measure theory, Proc. Am. Math. Soc., 95, 488-450.Search in Google Scholar

[7] Casalis, M. (1991) Familles exponentielles naturelles surRd invariantes par un groupe. International Statistical Review/Revue Internationale de Statistique, 241-262.Search in Google Scholar

[8] Calin, O. and Udriste, C. Geometric Modeling in Probability and Statistics, Springer Internl. Pub., Switzerland, (2010).Search in Google Scholar

[9] Efron, B. (1978) The geometry of exponential families. The Annals of Statistics, 6(2), 362-376.10.1214/aos/1176344130Search in Google Scholar

[10] Gzyl, H. and Recht, L. (2006) “A geometry in the space of probabilities II: Projective spaces and exponential families” Rev. Iberoamericana de Matemáticas, 22, 833-850.10.4171/RMI/475Search in Google Scholar

[11] Gzyl, H. and Recht, L. (2007) Intrinsic geometry on the class of probability densities and exponential families, Public. Mathematiques, 51, 309-322.Search in Google Scholar

[12] Gzyl, H. (2019) Best predictors in logarithmic distance between positive random variables. To appear Journal of Applied Mathematics, Statistics and Informatics, 15, 15-2810.2478/jamsi-2019-0006Search in Google Scholar

[13] Hotelling, H (1930) Spaces of statistical parameters. Bulletin of the American Mathematical Society (AMS), 36:191Search in Google Scholar

[14] Imparato, D. and Trivelato, B. Geometry of extended exponential models, in Algebraic and Geometric Methods in Statistics, Gibilisco, P., Riccomagno, E. Rogantin, M.P. and Wynn, H. eds., Cambridge Univ. Press, Cambridge, (2010).Search in Google Scholar

[15] Jaynes, E. T. (1957). Information theory and statistical mechanics. Physical review, 106(4), 620.10.1103/PhysRev.106.620Search in Google Scholar

[16] Klein, M. (1956) Entropy and the Ehrenfest urn model, Physica, 22, 569-575,10.1016/S0031-8914(56)90001-5Search in Google Scholar

[17] Lang, S. Math talks for undergraduates, Springer, New York, (1999).10.1007/978-1-4612-1476-2Search in Google Scholar

[18] Lawson, J.D. and Lim, Y. (2001) The Geometric mean, matrices, metrics and more, Amer. Math.,Monthly, 108, 797-812.10.1080/00029890.2001.11919815Search in Google Scholar

[19] Li, F., Zhang L. and Zhang Z. (2018) Lie Group Machine Learning, Walter de Gruyter GmbH & Co KG, ISBN9783110499506.10.1515/9783110499506Search in Google Scholar

[20] Moakher, M. (2005) A differential geometric approach to the geometric mean of symmetric positive definite matrices, SIAM. J. Matrix Anal. & Appl., 26, 735-747Search in Google Scholar

[21] Moran, P.A.P. (1960) Entropy, Markov processes and Boltzmann’s H-theorem, Proc. Camb. Phil. Soc., 57, 833-842.Search in Google Scholar

[22] Nielsen, F., and Sun, K. (2017). Clustering in Hilbert simplex geometry. preprint arXiv:1704.00454.Search in Google Scholar

[23] Nielsen, F., and Nock R. (2018) On the geometry of mixtures of prescribed distributions, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).10.1109/ICASSP.2018.8461869Search in Google Scholar

[24] Pistone, G. Algebraic varieties vs. differentiable manifolds, in Algebraic and Geometric Methods in Statistics, Gibilisco, P., Riccomagno, E. Rogantin, M.P. and Wynn, H. eds., Cambridge Univ. Press, Cambridge, (2010).Search in Google Scholar

[25] Pistone, G. and Rogantin, M.P. The exponential statistical manifold: mean parameters, orthogonality and space transformations” Bernoulli, 5 (1999), 721-760.10.2307/3318699Search in Google Scholar

[26] Pistone, G. and Sempi, C. “An infinite dimensional geometric structure in the space of all probability measures equivalent to a given one”. Ann. Statist., 23 (1995), 1543-1561.Search in Google Scholar

[27] Rao, C. R. (1992). Information and the accuracy attainable in the estimation of statistical parameters. In Breakthroughs in statistics (pp. 235-247). Springer, New York, NY.10.1007/978-1-4612-0919-5_16Search in Google Scholar

[28] Schwartzmazn, A. (2015) Lognormal distribution and geometric averages of positive definite matrices, Int. Stat. Rev., 84, 456-486.Search in Google Scholar

[29] Vajda, I. “Theory of statistical inference and information” Kluwer Acad., Dordrecht, (1989).Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo