Glossary · Rényi mRMR: which direction of Arimoto information?

The terms used across this project, A to Z. Each is also explained on hover where it first appears on a page.

Arimoto information also Arimoto’s mutual information, Arimoto mutual information: I_α(X;Y) = H_α(X) − H_α(X|Y), with Arimoto’s conditional entropy. Not symmetric for α ≠ 1, so I_α(f;C) and I_α(C;f) differ.
AUC: Area under the accuracy-versus-number-of-features curve, used here to compare selectors across all k at once.
blind zone: With unequal class sizes, class-side min-entropy relevance is 0 for words that favour the majority class, because the best guess of the class never changes.
class-side also class side: Relevance measured as I_α(C;f): how much seeing the feature tells you about the class.
feature-side also feature side: Relevance measured as I_α(f;C): how much knowing the class tells you about the feature.
index walk: When every remaining candidate scores exactly 0, the argmax returns the lowest index, so the selector picks features in file order.
lift also lifts: λ_c = q_c / q: how much more (or less) often a word appears in class c than overall.
min-entropy: Rényi entropy at α = ∞: minus the log of the probability of the single best guess.
mRMR also minimum redundancy maximum relevance: A greedy feature selector: at each step pick the feature with the most relevance to the class, minus its average redundancy with the features already picked.
naive Bayes: A simple classifier that assumes features are independent given the class; the evaluation used by Peng et al. and Fleuret.
Rényi entropy also Rényi order: A family of uncertainty measures H_α indexed by an order α. α = 1 is Shannon entropy; larger α weighs the most likely outcome more.