In the Selective Coloring problem, we are given an integer $k$, a graph $G$, and a partition of $V(G)$ into $p$ parts, and the goal is to decide whether or not we can pick exactly one vertex of each part and obtain a $k$-colorable induced subgraph of $G$. This generalization of Vertex Coloring has only recently begun to be studied by Demange et al. [Theoretical Computer Science, 2014], motivated by scheduling problems on distributed systems, with Guo et al. [TAMC, 2020] discussing the first results on the parameterized complexity of the problem. In this work, we study multiple structural parameterizations for Selective Coloring. We begin by revisiting the many hardness results of Demange et al. and show how they may be used to provide intractability proofs for widely used parameters such as pathwidth, distance to co-cluster, and max leaf number. Afterwards, we present fixed-parameter tractability algorithms when parameterizing by distance to cluster, or under the joint parameterizations treewidth and number of parts, and co-treewidth and number of parts. Our main contribution is a proof that, for every fixed $k \geq 1$, Selective Coloring does not admit a polynomial kernel when jointly parameterized by the vertex cover number and the number of parts, which implies that Multicolored Independent Set does not admit a polynomial kernel under the same parameterization.