Agriculture plays a vital role to development of economy of India. Farmers have difficulty to select suitable fruit and vegetable crop. Disease management by manually is a challenging task. Most of the diseases are seen on the leaves or stems of the plant. Hence agriculturist needs to find out the efficient techniques. In this paper we have combined two techniques to predict the disease. The goal of proposed work to capture the leaf image from the camera. Apply preprocessing steps to remove noise from the image using Adaptive Median Filter Technique. K means segmentation is used to segment the affected part in the leaves. After segmentation Process extracts the features using color and Gabor and finally classify the disease based on Neural Network.