Targeted at the harmful effects of garment image retrieval at present, a new approach of garment image retrieval featured in satisfactory performance is proposed. In this study, the Grab Cut auto segmentation algorithm is applied first to segment garment images and extract the image’s foreground. And then, the color coherence vector (CCV) and the dominant color method are adopted to extract the color features to conduct garment image retrieval. The experimental data show that the Grab Cut auto segmentation algorithm is capable of extracting the foreground of garment images with either simple or complex background. Meanwhile, the data also indicate that compared with the garment image retrieval by extracting color features using CCV, extracting color features by dominating color method shows both higher accuracy and recall rates.