Abstract: In this paper, a real-time running targets detection method was investigated based on a patrol robot system. The convolutional neural network method was used as the classifier. Running targets with various poses under different camera viewpoints and backgrounds were collected for the training of the neural network. To discriminate the foreground target and the changing background caused by the robot motion, an optical flowbased method was applied. Optical flow of two successive frames taken by on-board camera was used to extract region of interest. To boost the detection efficiency and accuracy, both appearance and motion information of the target are used as input of the convolutional neural network. Experimental results show that under real outdoor scenarios, the detection accuracy can reach 85% with a running efficiency of 20 frames per second.