It is feasible to implement the indoor localization techniques using wireless received signal strengths(RSSs) on the mobile devices since the RSSs indicate the information of distance between a transmitter and a receiver. However, the variation of RSSs severely reduces the localization accuracy due to the multipath effect and the unpredictable change of indoor environments. On this basis, a fingerprint-based indoor localization algorithm was designed leveraging the theory of sparse representation. In our approach, the variations were separated from the feature fingerprints using the sparse dictionary. Experiments were conducted for evaluating the performance of the algorithm in the real scenarios. Compared to traditional algorithms in indoor localization, the average localization error is reduced by 20%.