In mobile edge computing (MEC) systems, edge service caching refers to pre-storing the necessary programs for executing computation tasks at MEC servers. At resource-constrained edge servers, service caching placement is in general a complicated problem that highly correlates to the offloading decisions of computation tasks. In this paper, we consider a single edge server that assists a mobile user (MU) in executing a sequence of computation tasks. In particular, the MU can run its customized programs at the edge server, while the server can selectively cache the previously generated programs for future service reuse. To minimize the computation delay and energy consumption of the MU, we formulate a mixed integer non-linear programming (MINLP) that jointly optimizes the service caching placement, computation offloading, and system resource allocation. We first derive the closed-form expressions of the optimal resource allocation, and subsequently transform the MINLP into an equivalent pure 0-1 integer linear programming (ILP). To further reduce the complexity in solving the ILP, we exploit the underlying structures in optimal solutions, and devise a reduced-complexity alternating minimization technique to update the caching placement and offloading decision alternately. Simulations show that the proposed techniques achieve substantial resource savings compared to other representative benchmark methods.