Abstract: A vision-based fingertip interactive projector-camera system was presented in this paper. The proposed method can detect touch operations by judging the fusion level of fingertip and its shadow. To improve the robustness of the proposed method, in image detection stage, finger and its shadow are extracted and segmented from the image separately. And then, a linear detection model was introduced to judge the fusing level of fingertip and its shadow in the image. While a touching event was detected, the touching location can be calculated from the extracted fingertip image coordinate. With the proposed method, users can realize finger-based operation on the projector screen without any accessory tools.