08 Statistics and machine learning

A prediction is only as useful as its interval.

Sparse chemical data make uncertainty the central problem, not a footnote. This chapter covers Bayesian inference, Gaussian processes, calibrated intervals, honest validation splits, graph neural networks and hybrid physics–machine-learning models, and how they combine into active learning.

Methodology Draws from the Gaussian-process posterior through six example points (Eqs. 8.2–8.3; ℓ = 0.15, σn = 0.05). Each curve is z1 cos θ + z2 sin θ in the latent Gaussian space, mapped through the posterior covariance, so every frame is an exact posterior sample. The spread shows where data are missing.

You will see

  • Bayesian inference and Markov chain Monte Carlo
  • Gaussian-process regression, equations and live
  • Conformal prediction and calibration
  • Validation splits that do not leak
  • Message-passing neural networks and hybrid models

8.1 Bayesian inference and MCMC

p(θ | D) = p(D | θ) p(θ)p(D),    α(θ → θ′) = min(1, p(θ′ | D) q(θ | θ′)p(θ | D) q(θ′ | θ))
(8.1)

The posterior combines what the data say with what was known before. Metropolis–Hastings samples it without computing p(D): propose θ′, accept with probability α. With a symmetric proposal the q terms cancel. Methods 03 runs this live on 18 published Henry’s-constant points and shows that the published correction lies far outside the test-set posterior.

8.2 Gaussian processes

A Gaussian process (Rasmussen and Williams, 2006) puts a prior directly on functions. With a squared-exponential kernel and noisy observations y at inputs X, the prediction at x* is Gaussian with:

k(x, x′) = σf2 exp(−‖x − x′‖22ℓ2)
(8.2)
μ(x*) = k*T(K + σn2I)−1y,    σ2(x*) = k(x*, x*) − k*T(K + σn2I)−1k*
(8.3)

ℓ: length scale; σf: signal amplitude; σn: noise. The variance grows away from the data, which is exactly the “know when it does not know” behavior a design system needs. Hyperparameters are usually set by maximizing the log marginal likelihood −½yT(K + σn2I)−1y − ½ ln|K + σn2I| − (n/2) ln 2π.

Lab 8.1 Methodology

Gaussian-process regression

Click or tap inside the plot to add observations; select a point to remove it. The band is the 95% predictive interval from Eq. 8.3 plus observation noise σn2. Watch it widen between and beyond the data.

σf = 1; inputs and outputs are dimensionless. The example points are arbitrary, for illustration.

8.3 Calibration and conformal prediction

An interval is calibrated if a stated 95% interval contains the truth about 95% of the time on new data. Model-based intervals, from a GP, an ensemble or a Bayesian neural network, are only as good as the model’s assumptions. Split conformal prediction (Vovk, Gammerman and Shafer, 2005) adds a distribution-free guarantee: compute absolute residuals on a held-out calibration set of size n, take

q̂ = the ⌈(n + 1)(1 − α)⌉-th smallest of |yi − ŷi|,    C(x) = [ŷ(x) − q̂,   ŷ(x) + q̂]
(8.4)

If calibration and test points are exchangeable, the interval covers the truth with probability at least 1 − α. Exchangeability is the catch: a new ion family is not exchangeable with the training families, which is why validation splits matter.

8.4 Validation splits that do not leak

Random splits put near-duplicates of test liquids in the training set and overstate accuracy. Harder splits estimate what a design campaign will actually face: hold out whole cation or anion families, split by time of publication, or, the only test that cannot leak, predict before measuring. The founder’s 2016 correlations held out 17 of 37 melting points and 83 of 131 viscosity points, chosen at random; LILM’s planned benchmark uses ion-family holdouts and frozen prospective tests (Platform). Planned

8.5 Graph neural networks

Message-passing neural networks (Gilmer et al., 2017) learn representations directly from molecular graphs. At each step, every atom gathers messages from its neighbors and updates its state; a readout pools atom states into a molecular prediction:

mvt+1 = Σw∈N(v) Mt(hvt, hwt, evw),    hvt+1 = Ut(hvt, mvt+1),    ŷ = R({hvT})
(8.5)

For ionic liquids, cation and anion are separate graphs whose representations must be combined with their interaction; the latent vectors of Chapter 3 are one way to do it. Graph models are data-hungry, and for many ionic-liquid endpoints (toxicity, decomposition, process performance) data cover only tens to hundreds of liquids, which is why LILM is designed around physics-first hybrids. Planned

8.6 Hybrid physics–machine-learning models

Two hybrid patterns recur. In residual learning, a physical model makes the prediction and a learned term corrects it. In parameter learning, the learned model supplies parameters to a physical equation, which keeps the equation’s thermodynamic consistency:

y = fphys(x; θphys) + gw(x)   (residual)      y = fphys(x; θ = gw(x))   (parameter)
(8.6)

The Henry’s-constant correction of the CO2 screen (2016), Hmod = 0.7403 Hpred + 7.9697, is the simplest hybrid: a linear recalibration of the COSMO-RS output, fitted on 50 measurements. Predicting NRTL τ parameters from descriptors is a parameter-learning example. The design rule behind LILM: use physics for what physics already knows, and learn only what physics and sparse data cannot supply.

8.7 Active learning

Active learning chooses the next experiment for the information it is expected to provide, not only for its predicted performance. Expected improvement (Chapter 4) balances the two; pure exploration picks the point of largest predictive variance. In a closed loop, every result, including failures, updates the model before the next choice. That loop, run on proprietary measurements, is where Avestiq’s planned defensibility comes from. Planned

References

  1. Rasmussen, C. E.; Williams, C. K. I. Gaussian Processes for Machine Learning. MIT Press, 2006. gaussianprocess.org/gpml
  2. Vovk, V.; Gammerman, A.; Shafer, G. Algorithmic Learning in a Random World. Springer, 2005. doi:10.1007/b106715
  3. Gilmer, J.; Schoenholz, S. S.; Riley, P. F.; Vinyals, O.; Dahl, G. E. Neural message passing for quantum chemistry. Proc. 34th ICML, PMLR 70, 2017, 1263–1272. proceedings.mlr.press
  4. Jones, D. R.; Schonlau, M.; Welch, W. J. Efficient global optimization of expensive black-box functions. J. Global Optim. 1998, 13, 455–492. doi:10.1023/A:1008306431147
  5. Mehrkesh, A.; Karunanithi, A. T. New quantum chemistry-based descriptors for better prediction of melting point and viscosity of ionic liquids. Fluid Phase Equilib. 2016, 427, 498–503 (held-out validation shares). doi:10.1016/j.fluid.2016.07.006

Privacy

Terms