Wireless Capsule Endoscope (WCE) is a very promising tool for the examination of the gastrointestinal (GI) tract. However, there are some problems to be solved for the existed WCE, and one key problem is the accurate localization and tracking of the WCE. Among the possible localization methods, the magnet-based localization technique has its advantages: no need for power, not much space occupation, continuously tracking ability, and no negative effect. In this paper, we present the localization method for the magnet objective inside the WCE based on the magnetic sensor array outside the human body. Through the algorithm and system design we realize real time tracking of 3D position and 2D orientation of the magnet based on the magnetic dipole model. In order to overcome the interference of the human body movement, we propose the multi-magnets’ localization method; also, the 3D positioning and 3D orientation method is proposed, which can be used to make the 3D recovery of the GI tract and the accurate computation of the physiological tissue parameters. The real experiments show that the proposed localization system can run well and obtain the accuracy with 2~3mm for the magnet.