GpyOpt 2D dimensional acquisition plot problem

[lewisscola]

Hi,
I see an example in here: https://www.blopig.com/blog/wp-content/uploads/2019/10/GPyOpt-Tutorial1.html
The codes are:
#Import Modules

#GPyOpt - Cases are important, for some reason
import GPyOpt
from GPyOpt.methods import BayesianOptimization

#numpy
import numpy as np
from numpy.random import multivariate_normal #For later example

import pandas as pd

#Plotting tools
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
import numpy as np
from numpy.random import multivariate_normal

def obj_func_2d(x,y):
return((x2 + y2)*(np.sin(x)**2 - np.cos(y)))

fig = plt.figure(figsize=plt.figaspect(0.3))
fig.suptitle('Plots of our objective function')

Make data.

X = np.arange(0, 10, 0.25)
Y = np.arange(0, 10, 0.25)
X, Y = np.meshgrid(X, Y)
Z = obj_func_2d(X,Y)

First subplot

ax = fig.add_subplot(1, 2, 1)
ax.contour(X,Y,Z)

Second subplot

ax = fig.add_subplot(1, 2, 2, projection='3d')

Plot the surface.

surf = ax.plot_surface(X, Y, Z, cmap=cm.coolwarm,
linewidth=0, antialiased=False)

Customize the z axis.

ax.zaxis.set_major_locator(LinearLocator(10))
ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))

Add a color bar which maps values to colors.

fig.colorbar(surf, shrink=0.5, aspect=5)

plt.show()

def objfunc2d(x):
"""
x is a 2 dimensional vector.
"""
x1 = x[:, 0]
x2 = x[:, 1]
return((x12 + x22)*(np.sin(x1)**2 - np.cos(x2)))

bounds2d = [{'name': 'var_1', 'type': 'continuous', 'domain': (0,10)},
{'name': 'var_2', 'type': 'continuous', 'domain': (0,10)}]
maxiter = 50

myBopt_2d = GPyOpt.methods.BayesianOptimization(objfunc2d, domain=bounds2d)
myBopt_2d.run_optimization(max_iter = maxiter)
print("="*20)
print("Value of (x,y) that minimises the objective:"+str(myBopt_2d.x_opt))
print("Minimum value of the objective: "+str(myBopt_2d.fx_opt))
print("="*20)
myBopt_2d.plot_acquisition()

So I can see the objective function's value range is from 128.65 to 306.42,
But the posterior mean plot indicate the mean of the objective function is from -1.5 to 2.5(why posterior mean is not from128.65 to 306.42?)
I think the objective function data should correspond to the posterior mean. Why these 2 datasets have so large differences?

Could you help me explain that? Thank you so much for your help!

[apaleyes]

Pretty sure this is an effect of normalize_Y parameter, which is set to True by default: https://gpyopt.readthedocs.io/en/latest/GPyOpt.methods.html#module-GPyOpt.methods.bayesian_optimization

[lewisscola]

Thank you very much for your reply!
So if I choose normalize_Y=False, does the model performance will change?

[apaleyes]

it may. as always with such questions, it's best to just try it out

[lewisscola]

Thank you so much!
Can I ask you 2 additional questions

1. In my own experiments, I get the optimal value by 'run_optimization' and I get a different value by run'suggest_next_locations()'
   Could you explain why they are different and what is 'suggest_next_locations()' mean?
   
   

2. I plot the convergence plot with 3 different policies(EI,MPI,LCB).
   
   Can I plot them together like below?
   
   And is it possible for GpyOpt to plot opportunity cost ?

Thank you very much!

[apaleyes]

run_optimization is designed for functions that you can use in Python. here GpyOpt manages the whole optimization routine for you, and spits out the final result.

suggest_next_locations is for cases when you don't have an easy way of calling the objective, and thus have to manage the optimization loop yourself. Please refer to "External objective evaluation." notebook here: https://nbviewer.jupyter.org/github/SheffieldML/GPyOpt/blob/master/manual/index.ipynb

I don't recall any built in way to plot multiple methods on the same convergence plot. But the source code is available, so you could copy and tweak it.