In this paper, a thermal-dynamical consistent model for mass transfer across permeable moving interfaces is proposed by using the energy variation method. We consider a restricted diffusion problem where the flux across the interface depends on its conductance and the difference of the concentration on each side. The diffusive interface phase-field framework used here has several advantages over the sharp interface method. First of all, explicit tracking of the interface is no longer necessary. Secondly, the interfacial condition can be incorporated with a variable diffusion coefficient. A detailed asymptotic analysis confirms the diffusive interface model converges to the existing sharp interface model as the interface thickness goes to zero. A decoupled energy stable numerical scheme is developed to solve this system efficiently. Numerical simulations first illustrate the consistency of theoretical results on the sharp interface limit. Then a convergence study and energy decay test are conducted to ensure the efficiency and stability of the numerical scheme. To illustrate the effectiveness of our phase-field approach, several examples are provided, including a study of a two-phase mass transfer problem where drops with deformable interfaces are suspended in a moving fluid.