Sequential adaptive design for jump regression estimation

Abstract

Selecting input variables or design points for statistical models has been of great interest in adaptive design and active learning. Motivated by two scientific examples, this article presents a strategy of selecting the design points for a regression model when the underlying regression function is discontinuous. The first example we undertook was to accelerate imaging speed in high-resolution material imaging, and the second was to use sequential design for mapping a chemical phase diagram. In both examples, the underlying regression functions have discontinuities, and thus many existing design optimization approaches cannot be used, as they assume a continuous regression function. Although some existing adaptive design strategies developed from the treed regression models can handle the discontinuities, the related Bayesian model estimation approaches come with computationally expensive Markov Chain Monte Carlo algorithms for posterior inferences and the subsequent design point selections, which may not be applicable for the first motivating example that requires the computation to be faster than the original imaging speed. In addition, the treed models are based on domain partitioning and are inefficient in cases when the discontinuities occur at complex sub-domain boundaries. In this article, we propose a simple and effective adaptive design strategy for regression analysis with discontinuities. After some statistical properties of the estimated regression model are derived in cases with a fixed design, a new criterion for sequentially selecting the design points is suggested. The suggested sequential design selection procedure is then evaluated using a comprehensive simulation study and demonstrated using two motivating examples.

Keywords:

• Active learning
• sequential adaptive design
• adaptive sensing
• discontinuous response surfaces

Data availability statement

The data that support the findings of this study are available from the corresponding author, Chiwoo Park ([email protected]), upon request.

Additional information

Funding

We acknowledge support for this work from the AFOSR (FA9550-18-1-0144), NSF (DMS-1914639) and Oak Ridge National Laboratory (4000152630).

Notes on contributors

Chiwoo Park

Chiwoo Park received his BS in industrial engineering at Seoul National University and a PhD degree in industrial engineering at Texas A&M University in 2011. He is currently an Associate Professor in the Department of Industrial and Manufacturing Engineering at Florida State University and a principal investigator at High Performance Materials Institute. His main research interest lies in machine learning and data science, focusing on object data (such as image, shape, motion, function and directional data) and surrogate modeling.

Peihua Qiu

Peihua Qiu is the founding chair of the Department of Biostatistics at the University of Florida. He was previously a professor in the School of Statistics at the University of Minnesota. He was the editor of Technometrics, an elected fellow of the American Statistical Association and the Institute of Mathematical Statistics, and an elected member of the International Statistical Institute. His research focuses on jump regression analysis, medical image analysis, statistical methods for monitoring processes, and patient survival data analysis.

Jennifer Carpena-Núñez

Jennifer Carpena-Núñez is a chemical physicist at Air Force Research Laboratory, Materials and Manufacturing Directorate at Wright-Patterson Air Force Base, OH. She earned a PhD degree in Physics from University of Puerto Rico. Her main research is the studies of carbon nanotubes: synthesis, characterization and structures.

Rahul Rao

Rahul Rao is a research physical scientist at Air Force Research Laboratory, Materials and Manufacturing Directorate at Wright-Patterson Air Force Base, OH. He has an MS in Materials Science and a PhD in Physics from Clemson University. The main focus of his research at AFRL was to conduct in-situ Raman spectroscopy studies during carbon nanotube growth, as well as studying novel catalyst systems, with the ultimate aim of understanding the fundamental principles behind carbon nanotube growth.

Michael Susner

Michael Susner earned BS in Chemistry (2005) from Michigan State University and received MS (2009) and PhD (2012) in Materials Science and Engineering from The Ohio State University. From 2014-2016 he was a Postdoctoral Research Fellow in the Correlated Electron Materials Research Group at Oak Ridge National Laboratory. He joined the Air Force Research Laboratory in 2017 as a NRC Fellow and worked in the Soft Matter Materials Branch in the Materials and Manufacturing Directorate as a UES Research Scientist from 2019-2020. He is currently a research materials engineer in the Photonic Materials Branch. He is interested in establishing structure-property correlations in functional materials, i.e., those evincing magnetic, ferroelectric, and superconducting behaviors, particularly in single crystal materials.

Benji Maruyama

Benji Maruyama is a Principal Materials Research Engineer in the Air Force Research Laboratory, Materials & Manufacturing Directorate, Autonomous Materials Lead & ACT3 (Autonomous Capabilities Team 3) Liaison lead. His focus area is the synthesis and processing science of carbon nanotubes using ARES which is the first fully Autonomous Research (ARES) Robot for materials. Dr. Maruyama’s interests include the research process itself, for which he promotes the \Moore’s Law for the speed of research” He is also the point of contact for carbon materials for the AFRL Materials & Manufacturing Directorate. His materials interests include carbon nanomaterials, energy storage, flexible-hybrid materials and processes, field emission, carbon, polymer and metal matrix composites, imaging of complex 3D microstructures and AI/Machine Learning. He is currently involved in the study of the origins of chiral growth for carbon nanotubes, defect engineering for low dimensional materials, catalysis and autonomous experimentation.