Machine learning has become a major field of research in order to handle more and more complex image detection problems. Among the existing state-of-the-art CNN models, in this paper a region-based, fully convolutional network, for fast and accurate object detection has been proposed based on the experimental results. Among the region based networks, ResNet is regarded as the most recent CNN architecture which has obtained the best results at ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) in 2015. Deep residual networks (ResNets) can make the training process faster and attain more accuracy compared to their equivalent conventional neural networks. Being motivated with such unique attributes of ResNet, this paper evaluates the performance of fine-tuned ResNet for object classification of our weeds dataset. The dataset of farm land weeds detection is insufficient to train such deep CNN models. To overcome this shortcoming, we perform dropout techniques along with deep residual network for reducing over-fitting problem as well as applying data augmentation with the proposed ResNet to achieve a significant outperforming result from our weeds dataset. We achieved better object detection performance with Region-based Fully Convolutional Networks (R-FCN) technique which is latched with our proposed ResNet-101.