Polytopal Element Methods (PEM) allow to solve differential equations on general polygonal and polyhedral grids, potentially offering great flexibility to mesh generation algorithms. Differently from classical finite element methods, where the relation between the geometric properties of the mesh and the performances of the solver are well known, the characterization of a good polytopal element is still subject to ongoing research. Current shape regularity criteria are quite restrictive, and greatly limit the set of valid meshes. Nevertheless, numerical experiments revealed that PEM solvers can perform well on meshes that are far outside the strict boundaries imposed by the current theory, suggesting that the real capabilities of these methods are much higher. In this work, we propose a benchmark to study the correlation between general 2D polygonal meshes and PEM solvers. The benchmark aims to explore the space of 2D polygonal meshes and polygonal quality metrics, in order to identify weaker shape-regularity criteria under which the considered methods can reliably work. The proposed tool is quite general, and can be potentially used to study any PEM solver. Besides discussing the basics of the benchmark, in the second part of the paper we demonstrate its application on a representative member of the PEM family, namely the Virtual Element Method, also discussing our findings.