We study a multi-agent output regulation problem, where not all agents have access to the exosystem's dynamics. We propose a fully distributed controller that solves the problem for linear, heterogeneous, and uncertain agent dynamics as well as time-varying directed networks. The distributed controller consists of two parts: (1) an exosystem generator that locally estimates the exosystem dynamics, and (2) a dynamic compensator that, by locally approaching the internal model of the exosystem, achieves perfect output regulation. Moreover, we extend this distributed controller to solve an output synchronization problem where not all agents initially have the same internal model dynamics. Our approach leverages methods from internal model based controller synthesis and multi-agent consensus over time-varying directed networks; the derived result is a generalization of the (centralized) internal model principle to the distributed, networked setting.