Nearest Neighbor Classifier

Traditional Nearest Neighbor Classifier (NNC) is one of the most commonly used and the simplest pattern classification methods yet devised [12]. It is a kind of statistic machine learning methods. We have N training sample pairs (x,, yi), i = 1, 2,...,N, where xi is a set of features and yi is class label. For a set of features x extracted from an image, it is desired to predict label y by utilizing the information contained in the set of training samples which are labeled correctly. A distance vector D = d, d2,...,dN is firstly calculated by di = V(x — x)(x — xi)'. The l-th sample is the nearest neighbor of x if di = min,' di. Therefore x is categorized into the class yl.

^-Nearest Neighbor algorithm (fcNN) is also a simple classifier as a variant of NNC. Based on distance vector D, k nearest neighbors of x with k smallest distances are obtained. Let the k nearest neighbors be {(xj, y'1),..., (x'k, y'k)}, x is assigned to class y according the majority voting among the labels y'1,... ,y'k .The only parameter is k which should be chosen carefully. Generally, larger values of k reduce the effect of noise for classification and improve the classification performance, but they make the classes less distinct.

 
Source
< Prev   CONTENTS   Source   Next >