SURROGATE BASED GLOBAL OPTIMIZATION OF COMPUTATIONALLY EXPENSIVE PROBLEMS: ALGORITHM DESIGN, CONVERGENCE ANALYSIS AND APPLICATIONS

Abstract

The optimization algorithms considered in the thesis are all aimed at multimodal, high dimensional, computationally expensive, and black-box objective functions. The first work proposes a novel global-local iterative framework to combine a global surrogate algorithm and a local trust-region search for continuous optimization problems. Extensive experiments demonstrate that the new algorithm can improve the solution accuracy. The application in the watershed model calibration problem validates its practical value. The second work develops a parallel surrogate optimization algorithm for mixed-integer non-linear problems. Extensive numerical comparisons with parallel and serial algorithms demonstrate its efficiency with up to 16 processors. In the application to hyperparameter optimization of deep learning models, the proposed algorithm achieves similar results as the benchmark algorithm within much less wall-clock time. The third work proposes a surrogate-based algorithm for constrained multi-objective problems. The proposed algorithm is tested in the benchmark problems and a mean-VaR portfolio optimization problem. Almost sure convergence is proven for all three algorithms.