ADCME is a novel computational framework to solve inverse problems involving physical simulations and deep neural networks (DNNs). This paper benchmarks its capability to learn spatially-varying physical fields using DNNs. We demonstrate that our approach has superior accuracy compared to the discretization approach on a variety of problems, linear or nonlinear, static or dynamic. Technically, we formulate our inverse problem as a PDE-constrained optimization problem. We express both the numerical simulations and DNNs using computational graphs and therefore, we can calculate the gradients using reverse-mode automatic differentiation. We apply a physics constrained learning algorithm (PCL) to efficiently back-propagate gradients through iterative solvers for nonlinear equations. The open-source software which accompanies the present paper can be found at https://github.com/kailaix/ADCME.jl.