This website concerns the theory and application of the maximum entropy (MaxEnt) method, for the analysis of probabilistic systems of all kinds. The method is based on the concept of entropy, one of the most fundamental discoveries of human thought. In the MaxEnt method, we maximize the (relative) entropy of a system, subject to its constraints, to infer the state of the system. Depending on the philosophical perspective adopted by the user, this can be interpreted variously as:
- (due to Jaynes 1957 and Shore & Johnson 1980) inferring the least informative state of the system, or
- (due to Boltzmann 1877 and Planck 1901) inferring the most probable state of the system.
The power of the MaxEnt method lies in its ability to infer the (probabilistic) state of a system which is under-constrained, i.e. for which no closed-form, deterministic solution can be obtained. Mathematically, it enables the user to construct a probability distribution or probability density function over the state space of the system, enabling a substantial reduction in model order. In thermodynamics – the first and still one of the foremost applications of the MaxEnt method – this enables a tremendous reduction in model order compared to the underlying molecular dynamical system, of approximately 23 orders of magnitude ! Over the past few decades, the MaxEnt method has been applied to many other fields of endeavour, including astronomy and astrophysics, communications theory, cosmology, climate science, earth science, ecology, engineering, fluid mechanics, genetics, geophysics, machine learning, material science, medical imaging, nanoscience, network science, thermodynamics (equilibrium and non-equilibrium), particle physics, plasma physics, quantum mechanics, robotics and the social sciences.
This website has been created to provide a central repository for information about the theory and applications of the MaxEnt method, as well as on interconnected approaches such as Bayesian inference. Links are provided to important scientific societies and companies in these two fields, as well as to major scientific conferences.