We provide a new lower bound on the minimum distance of a family of quantum LDPC codes based on Cayley graphs proposed by MacKay, Mitchison and Shokrollahi. Our bound is exponential, improving on the quadratic bound of Couvreur, Delfosse and Z\'emor. This result is obtained by examining a family of subsets of the hypercube which locally satisfy some parity conditions.