Loop closure detection problem is an essential issue in simultaneous localization and mapping (SLAM) problem. In particular, visual loop closure detection, which using a visual sensor, should be robust to environmental conditions like confusion caused by repeated structures. In this paper, we propose a robust visual loop closure detection algorithm through restrained repetitive features observed in repeating structures. The proposed algorithm aims to extract bag of visual words (BoVW) for each image frame with RootSIFT extraction, improve it by restrain dominantly repetitive features, calculates histogram similarity score with histogram comparing method and finally decides loop closure pair(s). Experimental results show that the proposed algorithm robustly performs loop closure detection.