This article presents a novel approach to optimize geometric tolerances (flatness and cylindricity) by manipulating the rigidity among finishing and roughing cutting sequences during end milling of thin-walled components. The proposed approach considers the design configuration of the thin-walled component as an input and aims to determine semi-finished geometry such that the geometric tolerance parameters are optimized while performing a finish cutting sequence. The objective is accomplished by combining mechanistic force model, finite element (FE) analysis-based workpiece deflection model, and particle swarm optimization (PSO) technique to determine optimal disposition of material along the length of component thereby regulating rigidity. The algorithm has been validated by determining the rigidity-regulated semi-finished geometries for thin-walled components having straight, concave, and convex configurations. The outcomes of the proposed algorithm are substantiated further by conducting a set of end milling experiments for each of these cases. The results of the proposed strategy are compared with a traditional approach considering no change in the rigidity of component along length of the cut. It is demonstrated that the proposed approach can effectively optimize geometric tolerances for thin-walled components during end milling operation.