Research Papers

Dynamic Model Validation Metric Based on Wavelet Thresholded Signals

[+] Author and Article Information
Andrew D. Atkinson

Department of Operational Sciences,
Air Force Institute of Technology,
Wright-Patterson AFB, OH 45433
e-mail: andrew.atkinson@afit.edu

Raymond R. Hill

Department of Operational Sciences,
Air Force Institute of Technology,
Wright-Patterson AFB, OH 45433
e-mail: raymond.hill@afit.edu

Joseph J. Pignatiello, Jr.

Department of Operational Sciences,
Air Force Institute of Technology,
Wright-Patterson AFB, OH 45433
e-mail: joseph.pignatiello@afit.edu

G. Geoffrey Vining

Department of Statistics,
Virginia Tech,
Blacksburg, VA 24061
e-mail: vining@vt.edu

Edward D. White

Department of Mathematics and Statistics,
Air Force Institute of Technology,
Wright-Patterson AFB, OH 45433
e-mail: edward.white@afit.edu

Eric Chicken

Department of Statistics,
Florida State University,
Tallahassee, FL 32306
e-mail: chicken@stat.fsu.edu

Manuscript received December 22, 2016; final manuscript received May 24, 2017; published online June 14, 2017. Editor: Ashley F. Emery.This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States. Approved for public release; distribution is unlimited.

J. Verif. Valid. Uncert 2(2), 021002 (Jun 14, 2017) (10 pages) Paper No: VVUQ-16-1034; doi: 10.1115/1.4036965 History: Received December 22, 2016; Revised May 24, 2017

Model validation is a vital step in the simulation development process to ensure that a model is truly representative of the system that it is meant to model. One aspect of model validation that deserves special attention is when validation is required for the transient phase of a process. The transient phase may be characterized as the dynamic portion of a signal that exhibits nonstationary behavior. A specific concern associated with validating a model's transient phase is that the experimental system data are often contaminated with noise, due to the short duration and sharp variations in the data, thus hiding the underlying signal which models seek to replicate. This paper proposes a validation process that uses wavelet thresholding as an effective method for denoising the system and model data signals to properly validate the transient phase of a model. This paper utilizes wavelet thresholded signals to calculate a validation metric that incorporates shape, phase, and magnitude error. The paper compares this validation approach to an approach that uses wavelet decompositions to denoise the data signals. Finally, a simulation study and empirical data from an automobile crash study illustrates the advantages of our wavelet thresholding validation approach.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.


Sargent, R. G. , 2013, “ Verification and Validation of Simulation Models,” J. Simul., 7(1), pp. 12–24. [CrossRef]
Oberkampf, W. L. , and Trucano, T. G. , 2000, “ Validation Methodology in Computational Fluid Dynamics,” AIAA Paper No. 2000-2549.
ASME, 2009, “ Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer,” American Society of Mechanical Engineers, New York, Standard No. ASME VV 20-2009.
Jauregui, R. , Riu, P. J. , and Silva, F. , 2010, “ Transient FDTD Simulation Validation,” IEEE International Symposium on Electromagnetic Compatibility (EMC), Fort Lauderdale, FL, July 25–30, pp. 257–262.
Balci, O. , 2003, “ Verification, Validation, and Certification of Modeling and Simulation Applications,” IEEE Winter Simulation Conference (WSC), New Orleans, LA, Dec. 7–10, pp. 150–158.
Balci, O. , 2013, “ Verification, Validation, and Testing,” Encyclopedia of Operations Research and Management Science, 3rd ed., Springer, New York, pp. 1618–1627.
Kleijnen, J. P. , 1995, “ Verification and Validation of Simulation Models,” Eur. J. Oper. Res., 82(1), pp. 145–162. [CrossRef]
Law, A. M. , 2013, Simulation Modeling and Analysis, 5th ed., McGraw-Hill, New York.
Bendat, J. S. , and Piersol, A. G. , 1980, Engineering Applications of Correlation and Spectral Analysis, 1st ed., Wiley-Interscience, New York.
Box, G. E. , Jenkins, G. M. , and Reinsel, G. C. , 2008, Time Series Analysis: Forecasting and Control, 4th ed., Wiley, New York.
Naylor, T. H. , and Finger, J. M. , 1967, “ Verification of Computer Simulation Models,” Manage. Sci., 14(2), pp. B92–B101. [CrossRef]
Bayarri, M. J. , Berger, J. O. , Paulo, R. , Sacks, J. , Cafeo, J. A. , Cavendish, J. , Lin, C.-H. , and Tu, J. , 2012, “ A Framework for Validation of Computer Models,” J. Technometrics, 49(2), pp. 138–154.
Jiang, X. , and Mahadevan, S. , 2011, “ Wavelet Spectrum Analysis Approach to Model Validation of Dynamic Systems,” Mech. Syst. Signal Process., 25(2), pp. 575–590. [CrossRef]
Jiang, X. , and Mahadevan, S. , 2008, “ Bayesian Wavelet Method for Multivariate Model Assessment of Dynamic Systems,” J. Sound Vib., 312(4), pp. 694–712. [CrossRef]
ASME, 2006, “ Guide for Verification and Validation in Computational Solid Mechanics,” American Society of Mechanical Engineers, New York, Standard No. ASME VV 10-2006.
Oberkampf, W. L. , and Barone, M. F. , 2006, “ Measures of Agreement Between Computation and Experiment: Validation Metrics,” J. Comput. Phys., 217(1), pp. 5–36. [CrossRef]
Sprague, M. A. , and Geers, T. L. , 2004, “ A Spectral-Element Method for Modelling Cavitation in Transient Fluid-Structure Interaction,” Int. J. Numer. Methods Eng., 60(15), pp. 2467–2499. [CrossRef]
Russell, D. M. , 1997, “ Error Measures for Comparing Transient Data: Part 1: Development of a Comprehensive Error Measure,” 68th Shock and Vibration Symposium, Hunt Valley, MD, Nov. 3–6, pp. 175–184.
Whang, B. , Gilbert, W. E. , and Zilliacus, S. , 1994, “ Two Visually Meaningful Correlation Measures for Comparing Calculated and Measured Response Histories,” Shock Vib., 1(4), pp. 303–316. [CrossRef]
Schwer, L. E. , 2007, “ Validation Metrics for Response Histories: Perspectives and Case Studies,” Eng. Comput., 23(4), pp. 295–309. [CrossRef]
Sarin, H. , Kokkolaras, M. , Hulbert, G. , Papalambros, P. , Barbat, S. , and Yang, R.-J. , 2010, “ Comparing Time Histories for Validation of Simulation Models: Error Measures and Metrics,” ASME J. Dyn. Syst., Meas., Control, 132(6), p. 061401. [CrossRef]
Cheng, Z. , Pellettiere, J. , and Wright, N. , 2006, “ Wavelet-Based Test-Simulation Correlation Analysis for the Validation of Biodynamical Modeling,” 24th Conference and Exposition on Structural Dynamics, St. Louis, MO, Jan. 30–Feb. 2, pp. 2124–2132.
Ogden, T. , 1997, Essential Wavelets for Statistical Applications and Data Analysis, 1st ed., Birkhauser, Boston, MA.
Burrus, C. S. , Gopinath, R. A. , and Guo, H. , 1998, Introduction to Wavelets and Wavelet Transforms, 1st ed., Prentice Hall, Upper Saddle River, NJ.
Chui, C. K. , 1992, An Introduction to Wavelets, 1st ed., Academic Press, Boston, MA.
Misiti, M. , Misiti, Y. , Oppenheim, G. , and Poggi, J.-M. , 1997, Wavelet Toolbox Getting Started Guide, 1st ed., Mathworks, Natick, MA.
Donoho, D. L. , and Johnstone, J. M. , 1994, “ Ideal Spatial Adaptation by Wavelet Shrinkage,” Biometrika, 81(3), pp. 425–455. [CrossRef]
McGinnity, K. , Varbanov, R. , and Chicken, E. , 2017, “ Cross-Validated Wavelet Block Thresholding for Non-Gaussian Errors,” Comput. Stat. Data Anal., 106, pp. 127–137. [CrossRef]


Grahic Jump Location
Fig. 1

Decomposition of signal S into approximation and details [26]

Grahic Jump Location
Fig. 3

Decomposed signals (right-rear cross member)

Grahic Jump Location
Fig. 4

Thresholded signals (RRCM)

Grahic Jump Location
Fig. 5

Example data for follow-on study; system (blue) and model (red)



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In