Imaging spectroscopy integrates conventional imaging and spectroscopy to attain both spatial and spectral information from an object. Processing and analysis of a hypercube can be a hard task. Robust methods for hyperspectral data classification are required, insensitive to imaging deficiencies and high input dimension. Mushrooms have a thin and porous epidermal structure, and are sensitive to handling and transportation practices. Mechanical damage triggers a browning process within the tissue changing its metabolic state. The objective is to quantify different levels of physical perturbation on the mushroom pilei, using near-infrared spectral images and machine learning approaches. An ANN classifier is implemented, whose input is a small set of vectors containing representative information, and output is the set of categorical labels that correspond to different levels of mechanical vibration. For obtaining a salient dataset for classifying the images, the Harris corner detection algorithm is employed. The advantage of using interest points is to replace an exhaustive search over the entire image space by a computation over a concise set of highly informative points. A frame-based classification approach is proposed and shown to produce an increase in the classification accuracy, since feature vectors regarded as single instances may not always carry sufficient discriminant information. Comparisons with statistical features computed from wavelet coefficients showed that interest points are more suitable in assessing mechanical perturbation. Comparisons on a classifier level with support vector machines showed that ANNs perform better for the specific application, implying a connection between the classification method and the underlying learning problem. Overall, the frame-based classification scheme reduced the misclassification rate. This approach is suited for challenging classification problems where the degree of class separation is variable, i.e., assessment of mechanical damage in mushrooms.