Abstract: This paper presents a hand posture detection method based on transform feature representation and hierarchical model. The hierarchical model comprises a series of appearance models and an overall discriminate model. Appearance model for each posture is composed of a general template as well as several sub-category templates. With all the sub-category templates as transition functions, the original gradient histogram features can be converted into a more discriminative representation form. This transform representation is used to construct the discriminative model in the hierarchy model to achieve further posture-background and posture-posture classification. Moreover, to boost the efficiency, a skin-filter is introduced to exclude a wide range of non-skin area. Experimental results show that the proposed algorithm can successfully cope with appearance variability caused by viewpoint changes, posture tilts and natural posture deformation with a detection speed up to 20 frames per second.