In this work, we study an optimal control problem for a multi-agent system modeled by an undirected formation graph with nodes describing the kinematics of each agent, given by a left-invariant control system on a Lie group. The agents should avoid collision between them in the workspace. Such a task is done by introducing some potential functions into the cost function for the optimal control problem, corresponding to fictitious forces, induced by the formation constraint among agents, that break the symmetry of the individual agents and the cost functions, and rendering the optimal control problem partially invariant by a Lie group of symmetries. Reduced necessary conditions for the existence of normal extremals are obtained using techniques of variational calculus on manifolds. As an application, we study an optimal control problem for multiple unicycles.