Abstract: Diffusion magnetic resonance imaging is a crucial non-invasive technique for probing the microstructure of the human brain in vivo. Traditional hand-crafted and model-based tissue microstructure reconstruction methods typically require extensive diffusion gradient sampling, which is time-consuming and limits their clinical applicability. Recent advances in deep learning have shown great potential for microstructure estimation; however, accurately inferring tissue microstructure from clinically feasible diffusion magnetic resonance imaging scans remains challenging without appropriate constraints. In this paper, we propose a scalable and flexible framework compatible with diverse network architectures. By integrating macro-scale anatomical priors and cross-parameter mutual information from multiple diffusion models, together with total variation regularization, our approach achieves high-fidelity and efficient diffusion microstructure imaging. Experimental results demonstrate that the method significantly reduces scan time while maintaining—or even improving—the accuracy of microstructure estimation.