In this paper, the theory of smooth action-dependent Lagrangian mechanics (also known as contact Lagrangians) is extended to a non-smooth context appropriate for collision problems. In particular, we develop a Herglotz variational principle for non-smooth action-dependent Lagrangians which leads to the preservation of energy and momentum at impacts. By defining appropriately a Legendre transform, we can obtain the Hamilton equations of motion for the corresponding non-smooth Hamiltonian system. We apply the result to a billiard problem in the presence of dissipation.