Bayesian Optimisation

Highly specialised requests are first checked for feasibility from this information. Please do not send confidential data through the public form; describe it first and agree secure transfer afterwards.

1. 01Enquiry and requirement assessment
2. 02Technical scoping and proposal
3. 03Execution
4. 04Quality review and delivery, with an agreed round of revisions

Read the full project workflow.

In depth: how the work is done

1. Surrogate models: Gaussian processes and alternatives

   Gaussian processes provide a prediction and an uncertainty estimate at each point, which is what the acquisition function needs. The kernel encodes assumptions about smoothness and length scales, and its hyperparameters are estimated from data. For larger datasets, higher dimensions or categorical variables, alternatives such as random forests or Bayesian neural networks may be used. Model fit is checked, since a poor surrogate misleads the search.

2. Acquisition functions

   Expected improvement, probability of improvement, upper confidence bound and information-based criteria turn the surrogate's predictions into a decision about where to evaluate next. They embody different trade-offs between exploration and exploitation. Some are more robust to noise, others more aggressive. We choose according to the noise level, the evaluation budget and whether the goal is to find the best value or to learn the response.

3. Noise, constraints and multiple fidelities

   Experiments are noisy, and the surrogate must distinguish noise from structure, with noise levels either estimated or measured with replicates. Unknown constraints, for instance whether a setting causes a process to fail, can be modelled with additional surrogates. Multi-fidelity approaches use cheap approximate evaluations, such as coarse simulations, to guide expensive ones. These extensions increase complexity, so they are used only where justified.

4. Batch and sequential experiment planning

   Sequential optimisation evaluates one point at a time, which is efficient in evaluations but slow if each takes a long time. Batch methods propose several points to evaluate in parallel, for instance several furnace runs or simulation jobs at once, with strategies to keep the points diverse. We plan the batch size around the experimental or computing resources and explain how results will be fed back into the surrogate.

5. Initialisation, stopping and reporting

   A space-filling initial design provides the surrogate with a starting dataset. Stopping criteria combine budget, convergence of the best value and the remaining uncertainty. The final report presents the best configuration found, the evidence of its quality including repeated evaluations at the optimum, the surrogate's view of the response, and the number of evaluations. It notes that the optimum is conditional on the variable ranges and the objective chosen.

Common pitfalls this service helps avoid

• Too many variables. Standard methods degrade as dimension grows; reduce variables or use structure-exploiting methods.
• Ignoring noise. Treating noisy measurements as exact can mislead the surrogate and the search.
• Trusting a single best observation. In noisy problems the best observed value may be a lucky draw; confirmation runs are needed.
• Poor initial design. Starting points that do not cover the space can slow learning and miss good regions.

What good reporting looks like

A Bayesian optimisation report states the objective and variable ranges, the initial design, the surrogate and kernel, the acquisition function, noise and constraint handling, the batch strategy, the evaluation history and convergence, the best configuration with confirmation results, the surrogate model diagnostics and the limitations. It documents the software and settings. It does not claim to have found a global optimum, and it reports the number of evaluations used.

An illustrative scenario

Hypothetical example written for explanation. It is not a client case, a result or a testimonial.

Imagine a laboratory tuning four process parameters to maximise the yield of a synthesis. Each run takes most of a day and the yield measurements are noisy. After an initial space-filling design of eight runs, a Gaussian-process surrogate guides the next experiments one at a time. After twenty runs, the best setting is repeated three times to confirm the yield, and the surrogate shows that two parameters have little influence. The researchers report the confirmed yield and the explored region.

Questions to settle before you begin

• How many evaluations can I afford, and how noisy are they?
• How many variables do I really need to optimise?
• Are there settings that cause failures or violate constraints?
• Can I run several experiments in parallel?
• How will I confirm the best configuration?

Key terms

Surrogate model

A statistical model approximating the objective from data.

Acquisition function

A rule that selects the next evaluation point.

Gaussian process

A flexible probabilistic model giving predictions with uncertainty.

Exploration

Evaluating where the model is uncertain.

Exploitation

Evaluating where the model predicts good values.

Working together on Engineering Optimisation & Decision Science projects

Optimisation engagements begin with formulation: objectives, decision variables, constraints and the model or experiment used to evaluate designs. Much of the value is created at this stage, because poorly posed problems give unusable optima. We ask for the evaluation model or data, bounds on variables, constraints and any manufacturing or operational limits. Algorithms are chosen to suit evaluation cost and problem structure, and results are checked against independent evaluations and robustness tests. Reports present trade-offs rather than a single answer where objectives conflict, and they state assumptions behind the model that an optimiser could exploit.

Few projects need a single service in isolation. These are the services most often combined with this one.

• Engineering Optimisation

  Formulation and solution of optimisation problems in engineering design and operation: defining objectives, variables and constraints carefully, then choosing an algorithm suited to the problem structure.

• Response Surface Methodology (RSM)

  Sequential experimentation and modelling to relate responses to several factors and locate favourable operating conditions, using central composite or Box-Behnken style designs and fitted quadratic surfaces.

• Design of Experiments (DOE)

  Planning of efficient experimental programmes using factorial, fractional and optimal designs so that factor effects and interactions are estimated with the fewest runs the question allows.

• AI & Machine Learning for Engineering

  Machine-learning methods for engineering problems with physical data, designed around data quality, leakage-free validation and a clear account of when the model should not be trusted.