Nice error bases have been introduced by Knill as a generalization of the Pauli basis. These bases are shown to be projective representations of finite groups. We classify all nice error bases of small degree, and all nice error bases with abelian index groups. We show that in general an index group of a nice error basis is necessarily solvable.