Motivated by applications in machine learning and archival data storage, we introduce function-correcting codes, a new class of codes designed to protect a function evaluation on the data against errors. We show that function-correcting codes are equivalent to irregular distance codes, i.e., codes that obey some given distance requirement between each pair of codewords. Using these connections, we study irregular distance codes and derive general upper and lower bounds on their optimal redundancy. Since these bounds heavily depend on the specific function, we provide simplified, suboptimal bounds that are easier to evaluate. We further employ our general results to specific functions of interest and we show that function-correcting codes can achieve significantly less redundancy than standard error-correcting codes which protect the whole data.