The weak Galerkin (WG) finite element method is an effective and flexible general numerical technique for solving partial differential equations. It is a natural extension of the classic conforming finite element method for discontinuous approximations, which maintains simple finite element formulation. Stabilizer free weak Galerkin methods further simplify the WG methods and reduce computational complexity. This paper explores the possibility of optimal combination of polynomial spaces that minimize the number of unknowns in the stabilizer free WG schemes without compromising the accuracy of the numerical approximation. A new stabilizer free weak Galerkin finite element method is proposed and analyzed with polynomial degree reduction. To achieve such a goal, a new definition of weak gradient is introduced. Error estimates of optimal order are established for the corresponding WG approximations in both a discrete $H^1$ norm and the standard $L^2$ norm. The numerical examples are tested on various meshes and confirm the theory.