Many localization algorithms and systems have been developed by means of wireless sensor networks for both indoor and outdoor environments. To achieve higher localization accuracy, extra hardware equipments are utilized by most of the existing localization solutions, which increase the cost and considerably limit the location-based applications. The Internet of Things (IOT) integrates many technologies, such as Internet, Zigbee, Bluetooth, infrared, WiFi, GPRS, 3G, etc, which can enable different ways to obtain the location information of various objects. Location-based service is a primary service of the IOT, while localization accuracy is a key issue. In this paper, a higher accuracy localization scheme is proposed which can effectively satisfy diverse requirements for many indoor and outdoor location services. The proposed scheme composes of two phases: 1) partition phase, in which the target region is split into small grids; 2) localization refinement phase, in which a higher accuracy of localization can be obtained by applying an algorithm designed in the paper. A trial system is set up to verify correctness of the proposed scheme and furthermore to illustrate its feasibility and availability. The experimental results show that the proposed scheme can improve the localization accuracy.