Damping is largely increasing with the vibration amplitude during nonlinear vibrations of rectangular plates. At the same time, soft materials present an increase of their stiffness with the vibration frequency. These two phenomena appear together and are both explained in the framework of the viscoelasticity. While the literature on nonlinear vibrations of plates is very large, these aspects are rarely addressed. The present study uses the fractional solid model to describe the viscoelastic material behaviour. This allows to capture at the same time (i) the increase in the storage modulus with the vibration frequency and (ii) the frequency-dependent nonlinear damping in nonlinear vibrations of rectangular plates. The solution of the nonlinear vibration problems is obtained through Lagrange equations by deriving the potential energy of the plate and the dissipated energy, both geometrically nonlinear and frequency-dependent. The model is then applied to a silicone rubber rectangular plate tested experimentally. The plate was glued to a metal frame and harmonically excited by stepped sine testing at different force levels and the vibration response was measured by a laser Doppler vibrometer. The comparison of numerical and experimental results was satisfactorily carried out for: (i) nonlinear vibration responses in the frequency and time domain at different excitation levels, (ii) dissipated energy versus excitation frequency and excitation force, (iii) storage energy and (iv) loss factor, which is particularly interesting to evaluate the plate dissipation versus frequency at different excitation levels. Finally, the linear and nonlinear damping terms are compared.