0
Research Papers

Separability of Mesh Bias and Parametric Uncertainty for a Full System Thermal Analysis

[+] Author and Article Information
Benjamin B. Schroeder

V&V, UQ, Credibility Processes,
Sandia National Laboratories,
P.O. Box 5800, MS 0828,
Albuquerque, NM 87185-0828
e-mail: bbschro@sandia.gov

Humberto Silva, III

Thermal Sciences & Engineering,
Sandia National Laboratories,
P.O. Box 5800, MS 0828,
Albuquerque, NM 87185-0828

Kyle D. Smith

Thermal Sciences & Engineering,
Sandia National Laboratories,
P.O. Box 5800, MS 0828,
Albuquerque, NM 87185-0828

1Corresponding author.

Manuscript received June 22, 2018; final manuscript received February 4, 2019; published online February 22, 2019. Assoc. Editor: Tao Xing.

J. Verif. Valid. Uncert 3(3), 031006 (Feb 22, 2019) (12 pages) Paper No: VVUQ-18-1021; doi: 10.1115/1.4042815 History: Received June 22, 2018; Revised February 04, 2019

When making computational simulation predictions of multiphysics engineering systems, sources of uncertainty in the prediction need to be acknowledged and included in the analysis within the current paradigm of striving for simulation credibility. A thermal analysis of an aerospace geometry was performed at Sandia National Laboratories. For this analysis, a verification, validation, and uncertainty quantification (VVUQ) workflow provided structure for the analysis, resulting in the quantification of significant uncertainty sources including spatial numerical error and material property parametric uncertainty. It was hypothesized that the parametric uncertainty and numerical errors were independent and separable for this application. This hypothesis was supported by performing uncertainty quantification (UQ) simulations at multiple mesh resolutions, while being limited by resources to minimize the number of medium and high resolution simulations. Based on this supported hypothesis, a prediction including parametric uncertainty and a systematic mesh bias is used to make a margin assessment that avoids unnecessary uncertainty obscuring the results and optimizes use of computing resources.

FIGURES IN THIS ARTICLE
<>
Copyright © 2018 by ASME
Your Session has timed out. Please sign back in to continue.

References

Sierra Core Team, 2017, Sierra Thermal/Fluids Code, Sierra Core Team, Albuquerque, NM.
Helton, J. C. , Johnson, J. D. , and Oberkampf, W. L. , 2006, “ Probability of Loss of Assured Safety in Temperature Dependent Systems With Multiple Weak and Strong Links,” Reliab. Eng. Syst. Saf., 91, pp. 320–348. [CrossRef]
Helton, J. C. , Pilch, M. , and Sallaberry, C. J. , 2014, “ Probability of Loss of Assured Safety in Systems With Multiple Time-Dependent Failure Modes: Representations With Aleatory and Epistemic Uncertainty,” Reliab. Eng. Syst. Saf., 124, pp. 171–200. [CrossRef]
Oberkampf, W. L. , Pilch, M. , and Trucano, T. G. , 2007, “ Predictive Capability Maturity Model for Computational Modeling and Simulation,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND2007-5948. https://prod-ng.sandia.gov/techlib-noauth/access-control.cgi/2007/075948.pdf
VV/UQ, Credibility Processes, 2016, “ What is PIRT?,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND2016-6466.
VV/UQ, Credibility Processes, 2016, “ How to PIRT,” Technical Report, Sandia National Laboratories, Albuquerque, NM, Report No. SAND2016-6465.
Sankararaman, S. , and Mahadevan, S. , 2012, “ Comprehensive Framework for Integration of Calibration, Verification, and Validation,” 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, Honolulu, HI, Apr. 23–26, p. 1367.
Minana, M. A. , Turgeon, J. L. , Pilch, M. , and Hackney, P. , 2008, “ Sandia National Laboratories Advanced Simulation and Computing (ASC) Software Quality Plan—Part 1: ASC Software Quality Engineering Practices Version 3.0,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND2008-5517 https://prod-ng.sandia.gov/techlib-noauth/access-control.cgi/2008/085517.pdf.
Hills, R. G. , 2013, “ Roll-Up of Validation Results to a Target Application,” Technical Report, Sandia National Laboratories, Report No. SAND2013-7424.
Sargsyan, K. , Najm, H. N. , and Ghanem, R. , 2015, “ On the Statistical Calibration of Physical Models,” Int. J. Chem. Kinet., 47(4), pp. 246–276. [CrossRef]
Urbina, A. , Mahadevan, S. , and Paez, T. L. , 2012, “ A Bayes Network Approach to Uncertainty Quantification in Hierarchically Developed Computational Models,” Int. J. Uncertainty Quantif., 2(2), pp. 173–193. [CrossRef]
Oliver, T. A. , Malaya, N. , Ulerich, R. , and Moser, R. D. , 2014, “ Estimating Uncertainties in Statistics Computed From Direct Numerical Simulation,” Phys. Fluids, 26(3), p. 035101. [CrossRef]
Rebba, R. , Mahadevan, S. , and Huang, S. , 2006, “ Validation and Error Estimation of Computational Models,” Reliab. Eng. Syst. Saf., 91(10–11), pp. 1390–1297. [CrossRef]
Kennedy, M. C. , and O'Hagan, A. , 2001, “ Bayesian Calibration of Computer Models,” J. R. Stat. Soc.: Ser. B Stat. Methodol., 63(3), pp. 425–464. [CrossRef]
Mullins, J. , and Mahadevan, S. , 2016, “ Bayesian Uncertainty Integration for Model Calibration, Validation, and Prediction,” ASME J. Verif. Valid. Uncertainty Quantif., 1(1), p. 011006. [CrossRef]
VV 10, 2006, “ Guide for Verification and Validation in Computational Solid Mechanics,” The American Society of Mechanical Engineers, New York, Standard No. ASME V&V 10-2006.
VV 20, 2009, “ Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer,” The American Society of Mechanical Engineers, New York, Standard No. ASME V&V 20-2009.
SIERRA Thermal/Fluid Development Team, 2017, “ SIERRA Multimechanics Module: Aria User Manual—Version 4.44,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND2017-3776.
SIERRA Thermal/Fluid Development Team, 2016, “ SIERRA Multimechanics Module: Aria Theory Manual—Version 4.42,” Sandia National Laboratories, Technical Report No. SAND2016-10167.
Young, T. R. , 1980, “ CHEMEQ—A Subroutine for Solving Stiff Ordinary Differential Equations,” Naval Research Laboratory, Washington, DC, NRL Memorandum Report No. 4091.
Oberkampf, W. L. , and Roy, C. J. , 2010, Verification and Validation in Scientific Computing, Cambridge University Press, Cambridge, NY.
Richardson, L. F. , 1911, “ The Approximate Arithmetical Solution by Finite Differences of Physical Problems Involving Differential Equations, With an Application to the Stresses in a Masonary Dam,” Philos. Trans. R. Soc. London: Ser. A, 210(459–470), pp. 307–357. [CrossRef]
Roy, C. J. , 2005, “ Review of Code and Solution Verification Procedures for Computational Simulation,” J. Comput. Phys., 205(1), pp. 131–156. [CrossRef]
Roache, P. J. , 1998, Verification and Validation in Computational Science and Engineering, Hermosa Publishers, Albuquerque, NM.
Rider, W. J. , Kamm, J. R. , and Weirs, V. G. , 2016, “ Procedures for Calculation Verification: Simulation Credibility: Advances in Verification, Validation, and Uncertainty Quantification,” NASA/JANNAF, Moffett Field, CA, Report Nos. NASA/TP--2016-219422 and JANNAF/GL--2016-0001.
Helton, J. C. , Johnson, J. D. , and Oberkampf, W. L. , 2004, “ An Exploration of Alternative Approaches to the Representation of Uncertainty in Model Predictions,” Reliab. Eng. Syst. Saf., 85(1–3), pp. 39–71. [CrossRef]
Helton, J. C. , and Johnson, J. D. , 2011, “ Quantification of Margins and Uncertainties: Alternative Representations of Epistemic Uncertainty,” Reliab. Eng. Syst. Saf., 96, pp. 1034–1052. [CrossRef]
Helton, J. C. , and Sallaberry, C. J. , 2012, “ Uncertainty and Sensitivity Analysis: From Regulatory Requirements to Conceptual Structure and Computational Implementation,” IFIP Adv. Inf. Commun. Technol., 377, pp. 60–76. [CrossRef]
Eldred, M. S. , Swiler, L. P. , and Tang, G. , 2011, “ Mixed Aleatory-Epistemic Uncertainty Quantification With Stochastic Expansions and Optimization-Based Interval Estimation,” Reliab. Eng. Syst. Saf., 96(9), pp. 1092–1113. [CrossRef]
Helton, J. C. , and Davis, F. J. , 2003, “ Latin Hypercube Sampling and the Propagation of Uncertainty in Analyses of Complex Systems,” Reliab. Eng. Syst. Saf., 81(1), pp. 23–69. [CrossRef]
McKay, M. D. , Beckman, R. J. , and Conover, W. J. , 1979, “ Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code,” Technometrics, 21(2), pp. 239–245.
Iman, R. L. , and Conover, W.-J. , 1982, “ A Distribution-Free Approach to Inducing Rank Correlation Among Input Variables,” Commun. Stat.-Simul. Comput., 11(3), pp. 311–334. [CrossRef]
Sallaberry, C. J. , Helton, J. C. , and Hora, S. C. , 2008, “ Extension of Latin Hypercube Samples With Correlated Variables,” Reliab. Eng. Syst. Saf., 93(7), pp. 1047–1059. [CrossRef]
Iman, R. L. , 1981, “ Statistical Methods for Including Uncertainties Associated With the Geologic Isolation of Radioactive Waste Which Allow for a Comparison With Licensing Criteria,” Directorate of Technical Information and Document Control Proceedings of the Symposium on Uncertainties Associated With the Regulation of the Geologic Disposal of High-Level Radioactive Waste, Gatlinburg, TN, Mar. 9–13, pp. 145–158.
Adams, B. M. , Ebeidad, M. S. , Eldred, M. S. , Geraci, G. , Jakeman, J. D. , Maupin, K. A. , Monschke, J. A. , Swiler, L. P. , Stephens, J. A. , Vigil, D. N. , Wildey, T. M. , Bohnhoff, W. J. , Dalbey, K. R. , Eddy, J. P. , Hooper, R. W. , Hu, K. T. , Hough, P. D. , Ridgway, E. M. , and Rushdi, A. , 2016, “ Dakota, a Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.5 User's Manual,” Sandia National Laboratories, Technical Report No. SAND2014-4633.
Helton, J. C. , Johnson, J. D. , Sallaberry, C. J. , and Storlie, C. B. , 2006, “ Survey of Sampling-Based Methods for Uncertainty and Sensitivity Analysis,” Reliab. Eng. Syst. Saf., 91, pp. 1175–1209. [CrossRef]
Borgonovo, E. , and Plischke, E. , 2016, “ Sensitivity Analysis: A Review of Recent Advances,” Eur. J. Oper. Res., 248(3), pp. 869–887. [CrossRef]
Wei, P. , Lu, Z. , and Song, J. , 2015, “ Variable Importance Analysis: A Comprehensive Review,” Reliab. Eng. Syst. Saf., 142, pp. 399–432. [CrossRef]
Storlie, C. B. , Swiler, L. P. , Helton, J. C. , and Sallaberry, C. J. , 2009, “ Implementation and Evaluation of Nonparametric Regression Procedures for Sensitivity Analysis of Computationally Demanding Models,” Reliab. Eng. Syst. Saf., 94(11), pp. 1735–1763. [CrossRef]
Storlie, C. B. , and Helton, J. C. , 2008, “ Multiple Predictor Smoothing Methods for Sensitivity Analysis: Description of Techniques,” Reliab. Eng. Syst. Saf., 93(1), pp. 28–54. [CrossRef]
Saltelli, A. , Chan, K. , and Scott, E. M. , eds., 2000, Sensitivity Analysis, Vol. 1, Wiley, New York.
Saltelli, A. , Ratto, M. , Andres, T. , Campolongo, F. , Cariboni, J. , Gatelli, D. , Saisana, M. , and Tarantola, S. , 2008, Global Sensitivity Analysis: The Primer, Wiley, West Sussex, UK.
Tang, G. , Iaccarino, G. , and Eldred, M. S. , 2010, “ Global Sensitivity Analysis for Stochastic Collocation Expansion,” AIAA Paper No. 2010-2922.
Helton, J. C. , 2011, “ Quantification of Margins and Uncertainties: Conceptual and Computational Basis,” Reliab. Eng. Syst. Saf., 96, pp. 976–1013. [CrossRef]
Helton, J. C. , and Pilch, M. , 2011, “ Quantification of Margins and Uncertainties,” Reliab. Eng. Syst. Saf., 96(9), pp. 959–964. [CrossRef]
Newcomer, J. T. , 2012, “ A New Approach to Quantification of Margins and Uncertainties for Physical Simulation Data,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND2012-7912. https://prod-ng.sandia.gov/techlib-noauth/access-control.cgi/2012/127912.pdf
Gartling, D. K. , and Hogan, R. E. , 1994, “ COYOTE II—A Finite Element Computer Program for Nonlinear Heat Conduction Problems—Part I: Theoretical Background,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND94-1173.
Carslaw, H. S. , and Jaeger, J. C. , 1959, Conduction of Heat in Solids, 2nd ed., Clarendon Press, Oxford, UK.
Howell, J. R. , Siegel, R. , and Mengüç, M. P. , 2011, Thermal Radiation Heat Transfer, 5th ed., CRC Press, Boca Raton, FL.
Notz, P. K. , Subia, S. R. , Hopkins, M. M. , Moffat, H. K. , and Noble, D. R. , 2007, “ Aria 1.5: User Manual,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND2007-2734.
Mott, D. R. , Oran, E. S. , and van Leer, B. , 2000, “ A Quasi-Steady-State Solver for the Stiff Ordinary Differential Equations of Reaction Kinetics,” J. Comput. Phys., 164(2), pp. 407–428. [CrossRef]

Figures

Grahic Jump Location
Fig. 1

Concept of thermal race used for this analysis

Grahic Jump Location
Fig. 2

Typical VVUQ workflow applied to thermal analyses

Grahic Jump Location
Fig. 3

Conceptual illustration of margin assessment comparing predicted performance and a requirement. Margin is defined as distance from the edge of the prediction distribution to the requirement and uncertainty is defined as total prediction distribution spread.

Grahic Jump Location
Fig. 4

Application of Richardson extrapolation to UMR mesh convergence study. Predicted QoI values for different mesh refinements are shown as dots and × markers indicate Richardson extrapolations for possible additional uniform mesh refinements. Mesh refinements are shown as values normalized by the nominal mesh.

Grahic Jump Location
Fig. 5

Defining mesh bias as difference between QoI prediction at each mesh resolution and Richardson extrapolated QoI prediction. Bars are added to Fig. 4 to indicate the magnitude of mesh bias present for different mesh refinements.

Grahic Jump Location
Fig. 6

Histograms of QoI predictions based on 320 and 640 samples. Normal distributions fit to the sample mean and standard deviation are shown as distribution outlines.

Grahic Jump Location
Fig. 7

Top three main effect Sobol indices based on 320 (right bars) and 640 (left bars) iLHS samples

Grahic Jump Location
Fig. 8

Four metrics of sampling convergence: relative error value of the mean of the predicted QoI distribution (μ), the standard deviation of the predicted QoI distribution (σ), the distance between the minimum and maximum samples of the predicted QoI distribution (Spread), and the magnitude of the leading correlation coefficient (CorrMag)

Grahic Jump Location
Fig. 9

Impact of mesh size on QoI prediction for simulations that produced the nominal, left extreme, and right extreme values of the QoI prediction distribution for the nominal mesh. Circles indicate simulation results and triangles are Richardson extrapolated values.

Grahic Jump Location
Fig. 10

Richardson extrapolation applied to solution convergence of QoI prediction spread based on UMR samples. Dots indicate meshes used and × markers are based on Richardson extrapolation for possible additional UMR meshes. Mesh refinements are shown as values normalized by the nominal mesh.

Grahic Jump Location
Fig. 11

Shifting QoI histogram predicted using nominal mesh by the quantified mesh bias (gray arrow). Performance requirement (0.5) is shown as the dashed line.

Grahic Jump Location
Fig. 12

Percent of the prediction distribution below the performance requirement (0.5) as a function of mesh refinement. Percent values are based on the number of iLHS samples below the threshold. Dots correspond to actual meshes tested and the triangle is based on the Richardson extrapolation estimate of the true value. Mesh refinements are shown as values normalized by the nominal mesh.

Tables

Errata

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In