Vascular disorders, including hypertension, atherosclerosis and restenosis, arise from dysregulation of vascular smooth muscle cell (VSMC) differentiation, which can be controlled by regulatory factors. The present study investigated the regulatory mechanism of the phenotypic transformation of human VSMCs by NELIN in order to evaluate its potential as a preventive and therapeutic of vascular disorders. An in vitro model of NELIN‑overexpressing VSMCs was prepared by transfection with a lentiviral (LV) vector (NELIN‑VSMCs) and NELIN was slienced using an a lentiviral vector with small interfering (si)RNA in another group (LV‑NELIN‑siRNA‑VSMCs). The effects of NELIN overexpression or knockdown on the phenotypic transformation of human VSMCs were observed, and its regulatory mechanism was studied. Compared with the control group, cells in the NELIN‑VSMCs group presented a contractile phenotype with a significant increase of NELIN mRNA, NELIN protein, smooth muscle (SM)α‑actin and total Ras homolog gene family member A (RhoA) protein expression. The intra‑nuclear translocation of SMα‑actin‑serum response factor (SMα‑actin‑SRF) occurred in these cells simultaneously. Following exposure to Rho kinsase inhibitor Y‑27632, SRF and SMα‑actin expression decreased. However, cells in the LV‑NELIN‑siRNA‑VSMCs group presented a synthetic phenotype, and the expression of NELIN mRNA, NELIN protein, SMα‑actin protein and total RhoA protein was decreased. The occurrence of SRF extra‑nuclear translocation was observed. In conclusion, the present study suggested that NELIN was able to activate regulatory factors of SMα‑actin, RhoA and SRF successively in human VSMCs cultured in vitro. Furthermore, NELIN‑induced phenotypic transformation of human VSMCs was regulated via the RhoA/SRF signaling pathway. The results of the present study provide a foundation for the use of NELIN in preventive and therapeutic treatment of vascular remodeling diseases, including varicosity and atherosclerosis.