The paper addresses the model reduction problem by least squares moment matching for continuous-time, linear, time-invariant systems. The basic idea behind least squares moment matching is to approximate a transfer function by ensuring that the interpolation conditions imposed by moment matching are satisfied in a least squares sense. This idea is revisited using invariance equations and steady-state responses to provide a new time-domain characterization of least squares moment matching. The characterization, in turn, is then used to obtain a parameterized family of models achieving least squares moment matching. The theory is illustrated by a worked-out numerical example.