The concept of $H$-sets as introduced by Collatz in 1956 was very useful in univariate Chebyshev approximation by polynomials or Chebyshev spaces. In the multivariate setting, the situation is much worse, because there is no alternation, and $H$-sets exist, but are only rarely accessible by mathematical arguments. However, in Reproducing Kernel Hilbert spaces, $H$-sets are shown here to have a rather simple and complete characterization. As a byproduct, the strong connection of $H$-sets to Linear Programming is studied. But on the downside, it is explained why $H$-sets have a very limited range of applicability in the times of large-scale computing.