concept

Bayesian Neural Networks

Also known as: Bayesian Neural Networks, Bayesian neural network

Facts (15)

Sources

A Comprehensive Review of Neuro-symbolic AI for Robustness ... link.springer.com Springer Dec 9, 2025 10 facts

procedureThe Laplace Approximation fits a Gaussian distribution centered at the maximum a posteriori estimate to approximate the posterior in Bayesian Neural Networks.

procedureVariational Inference approximates the posterior distribution in Bayesian Neural Networks by optimizing over a parameterized family of distributions q_φ(θ), where φ denotes variational parameters such as mean and variance.

referenceL.V. Jospin, H. Laga, F. Boussaid, W. Buntine, and M. Bennamoun published 'Hands-on Bayesian neural networks—a tutorial for deep learning users' in the IEEE Computational Intelligence Magazine in 2022.

claimThe posterior distribution p(θ | X, Y) in Bayesian Neural Networks is generally intractable and requires approximation methods such as Variational Inference, Laplace Approximation, or Markov Chain Monte Carlo (MCMC).

quoteBayesian neural networks not only provide accurate predictions but also quantify uncertainty in those predictions.

claimBayesian Neural Networks are a type of machine learning model that learn probability distributions over parameters rather than learning fixed parameters.

procedurePredictions for new data points in Bayesian Neural Networks are obtained by integrating over the posterior distribution of parameters, which captures uncertainty in the model’s weights.

referenceBayesian Neural Networks (BNNs) are neural networks that treat weights or activations as random variables with distributions, typically learned through variational inference or Markov Chain Monte Carlo (MCMC) methods.

formulaThe posterior over model parameters in Bayesian Neural Networks is expressed using Bayes’ theorem as p(θ | X, Y) = [p(Y | X, θ) * p(θ)] / p(Y | X), where θ represents the parameters of the neural network, X represents input features, Y represents target outputs, p(Y | X, θ) is the likelihood, p(θ) is the prior distribution, and p(Y | X) is the marginal likelihood.

claimA Bayesian Neural Network (BNN) flight controller can output both a control action and a measure of confidence, allowing the system to identify when it is unsure and defer to a safe policy.

Track: Poster Session 3 - aistats 2026 virtual.aistats.org Samuel Tesfazgi, Leonhard Sprandl, Sandra Hirche · AISTATS 5 facts

procedureMoule Lin, Shuhao Guan, Weipeng Jing, Goetz Botterweck, and Andrea Patane reinterpret weight-sharing quantization techniques from a stochastic perspective by using 2D-adaptive Gaussian distributions, Wasserstein distance estimations, and alpha-blending to encode the stochastic behavior of a Bayesian Neural Network (BNN) in a lower-dimensional, soft Gaussian representation.

claimDaniel Dold, Julius Kobialka, Nicolai Palm, Emanuel Sommer, David Rügamer, and Oliver Dürr improved subspace inference in Bayesian neural networks by identifying pitfalls and proposing a more natural prior that better guides the sampling procedure.

claimThe Deep Additive Kernel (DAK) model incorporates an additive structure for the last-layer Gaussian Process and induced prior approximation for each Gaussian Process unit, resulting in a last-layer Bayesian neural network (BNN) architecture.

claimThe stochastic weight sharing approach for Bayesian Neural Networks (BNNs) significantly reduces computational overhead by several orders of magnitude, enabling efficient Bayesian training of large-scale models such as ResNet-101 and Vision Transformer (VIT).

claimBayesian Neural Networks (BNNs) are constrained by increased computational requirements and convergence difficulties when training deep, state-of-the-art architectures.