The number of pedestrian accidents continues to keep climbing. Distraction from smartphone is one of the biggest causes for pedestrian fatalities. In this paper, we develop SaferCross, a mobile system based on the embedded sensors of smartphone to improve pedestrian safety by preventing distraction from smartphone. SaferCross adopts a holistic approach by identifying and developing essential system components that are missing in existing systems and integrating the system components into a "fully-functioning" mobile system for pedestrian safety. Specifically, we create algorithms for improving the accuracy and energy efficiency of pedestrian positioning, effectiveness of phone activity detection, and real-time risk assessment. We demonstrate that SaferCross, through systematic integration of the developed algorithms, performs situation awareness effectively and provides a timely warning to the pedestrian based on the information obtained from smartphone sensors and Direct Wi-Fi-based peer-to-peer communication with approaching cars. Extensive experiments are conducted in a department parking lot for both component-level and integrated testing. The results demonstrate that the energy efficiency and positioning accuracy of SaferCross are improved by 52% and 72% on average compared with existing solutions with missing support for positioning accuracy and energy efficiency, and the phone-viewing event detection accuracy is over 90%. The integrated test results show that SaferCross alerts the pedestrian timely with an average error of 1.6sec in comparison with the ground truth data, which can be easily compensated by configuring the system to fire an alert message a couple of seconds early.