Despite the rapid growth and widespread recognition of additive manufacturing (AM) technologies, the geometrical inaccuracy of the manufactured products remains a challenging issue and effective prediction of the geometric deviations in AM is critical for the solution of this issue. The layerwise characteristic of the AM process has motivated the investigation of deviation from in-plane and out-of-plane perspectives, the latter has been seldom studied and will be the focus of this paper. In this paper, an out-of-plane deviation modeling method will be proposed based on statistical modal analysis. Owing to the inconvenience in data acquisition, AM simulation is conducted to obtain the layer-level out-of-plane deviation on parts manufactured by the selective laser melting process. Discrete cosine transform is adopted to identify the major deviation modes from the data. The statistical relationship between mode coefficients and related part and process parameters is studied based on the Gaussian process model. To gain data for model training, experimental design is conducted to sample parameter combinations as simulation input. A case study is presented to demonstrate the proposed method and the effectiveness is validated on test data. The method can be applied in multiple domains of AM, such as quality control and tolerancing, to provide high-fidelity prediction of geometric deviations.