Tendon is a connective tissue that transmits loads from muscle to bone, while ligament is a similar tissue that stabilizes joint articulation by connecting bone to bone. Seventy to 90% of tendon and ligament's extracellular matrix (ECM) is composed of a hierarchical collagen structure that provides resistance to deformation primarily in the fiber direction, and the remaining fraction consists of a variety of non-collagenous proteins, proteoglycans, and glycosaminoglycans (GAGs) whose mechanical roles are not well characterized. ECM constituents such as elastin, the proteoglycans decorin, biglycan, lumican, fibromodulin, lubricin, and aggrecan and their associated GAGs, and cartilage oligomeric matrix protein (COMP) have been suggested to contribute to tendon and ligament's characteristic quasi-static and viscoelastic mechanical behavior in tension, shear, and compression. The purpose of this review is to summarize existing literature regarding the contribution of the non-collagenous ECM to tendon and ligament mechanics, and to highlight key gaps in knowledge that future studies may address. Using insights from theoretical mechanics and biology, we discuss the role of the non-collagenous ECM in quasi-static and viscoelastic tensile, compressive, and shear behavior in the fiber direction and orthogonal to the fiber direction. We also address the efficacy of tools that are commonly used to assess these relationships, including enzymatic degradation, mouse knockout models, and computational models. Further work in this field will foster a better understanding of tendon and ligament damage and healing as well as inform strategies for tissue repair and regeneration.