0
Research Papers

New Metrics for Validation of Data-Driven Random Process Models in Uncertainty Quantification

[+] Author and Article Information
Hongyi Xu, Zhen Jiang

Department of Mechanical Engineering,
Northwestern University,
Evanston, IL 60208

Daniel W. Apley

Department of Industrial Engineering and
Management Science,
Northwestern University,
Evanston, IL 60208

Wei Chen

Department of Mechanical Engineering,
Northwestern University,
Evanston, IL 60208
e-mail: weichen@northwestern.edu

1Corresponding author.

Manuscript received December 9, 2014; final manuscript received September 10, 2015; published online December 10, 2015. Assoc. Editor: Kevin Dowding.

J. Verif. Valid. Uncert 1(2), 021002 (Dec 10, 2015) (14 pages) Paper No: VVUQ-14-1007; doi: 10.1115/1.4031813 History: Received December 09, 2014; Revised September 10, 2015

Data-driven random process models have become increasingly important for uncertainty quantification (UQ) in science and engineering applications, due to their merit of capturing both the marginal distributions and the correlations of high-dimensional responses. However, the choice of a random process model is neither unique nor straightforward. To quantitatively validate the accuracy of random process UQ models, new metrics are needed to measure their capability in capturing the statistical information of high-dimensional data collected from simulations or experimental tests. In this work, two goodness-of-fit (GOF) metrics, namely, a statistical moment-based metric (SMM) and an M-margin U-pooling metric (MUPM), are proposed for comparing different stochastic models, taking into account their capabilities of capturing the marginal distributions and the correlations in spatial/temporal domains. This work demonstrates the effectiveness of the two proposed metrics by comparing the accuracies of four random process models (Gaussian process (GP), Gaussian copula, Hermite polynomial chaos expansion (PCE), and Karhunen–Loeve (K–L) expansion) in multiple numerical examples and an engineering example of stochastic analysis of microstructural materials properties. In addition to the new metrics, this paper provides insights into the pros and cons of various data-driven random process models in UQ.

FIGURES IN THIS ARTICLE
<>
Copyright © 2016 by ASME
Your Session has timed out. Please sign back in to continue.

References

Figures

Grahic Jump Location
Fig. 1

Random process model of high-dimensional data and its random realizations. This figure shows two examples of random process modeling. Left: spatial distribution of elastic modulus is modeled as a random process with respect to spatial location x. Right: stochastic constitutive law of microstructural materials, where the stress is modeled as a random process with respect to strain.

Grahic Jump Location
Fig. 2

Flowchart of testing the GOF of a random process model

Grahic Jump Location
Fig. 3

Illustration of u-pooling method. Right: u-values at three validation locations; left: area metric of mismatch between the empirical complementary CDF of u-values and the standard uniform distribution.

Grahic Jump Location
Fig. 4

SMM and MUPM's performances: (a) detection of the marginal skewness of random process data and (b) necessity of including high-order metric values. For illustration purpose, 50 out of 5000 samples of each dataset are plotted. Three orders of statistical information (M = 1, 2, 3) are compared.

Grahic Jump Location
Fig. 5

SMM and MUPM of five random process models in benchmark 2 (NSta-Sym): SMM on the left is low-order-dominant; MUPM on the right is high-order-dominant. Low metric value means high accuracy.

Grahic Jump Location
Fig. 6

Visual comparison and covariance matrix RMSE comparison of original NSta-Sym data and realizations by five different random process models. For visual comparison, 20 realizations are plotted in each figure. For covariance matrix comparison, RMSE between the original and the regenerated covariance matrices is plotted in the bar chart.

Grahic Jump Location
Fig. 7

SMM and MUPM of five random process models in benchmark 4 (NSta-NSym): SMM on the left is low-order-dominant; MUPM on the right is high-order-dominant. Low metric value means high accuracy.

Grahic Jump Location
Fig. 8

Visual comparison and covariance RMSE comparison of original NSta-NSym data and realizations by five different random process models. For visual comparison, 20 realizations are plotted in each figure. For covariance matrix comparison, RMSE between the original and the regenerated covariance matrices is plotted in the bar chart.

Grahic Jump Location
Fig. 9

Benchmark 4 with ten original samples: (a) original data, 20 randomly selected realizations for each random process model and covariance matrix RMSEs and (b) comparisons of SMM and MUPM

Grahic Jump Location
Fig. 10

A stochastic decomposition and reassembly strategy for material property analysis

Grahic Jump Location
Fig. 11

Comparison of original data and random process model realizations of G′, classes 2 and 8. The solid lines are the mean and the dashed lines mark the 5th and 95th percentile at each x location.

Grahic Jump Location
Fig. 12

Comparison of simulated mosaic RVE properties using: (1) original SVE properties data, (2) regenerated SVE properties data using GP, (3) regenerated SVE properties data using copula, and (4) regenerated SVE properties data using K–L. In the table, we compare the percent errors on L, P, and H, as well as the SSE of the three model predictions.

Grahic Jump Location
Fig. 13

SMM and MUPM comparisons of benchmarks 1 and 3 (Sta-NSym)

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In