Multiple Classifier Systems

Morethanadecadeago,combiningmultipleclassi?erswasproposedasap- siblesolutiontotheproblemsposedbythetraditionalpatternclassi?cation approachwhichinvolvedselectingthebestclassi?erfromasetofcandidates basedontheirexperimentalevaluation. Asnoclassi?erisknowntobethebest forallcasesandtheselectionofthebestclassi?erforagivenpracticaltaskis verydi?cult,diverseresearchcommunities,includingMachineLearning,N- ralNetworks,PatternRecognition,andStatistics,addressedtheengineering problemofhowtoexploitthestrengthswhileavoidingtheweaknessesofd- ferentdesigns. Thisambitiousresearchtrendwasalsomotivatedbyempirical observationsaboutthecomplementarityofdi?erentclassi?erdesigns,natural requirementsofinformationfusionapplications,andintrinsicdi?cultiesasso- atedwiththeoptimalchoiceofsomeclassi?erdesignparameters,suchasthe architectureandtheinitialweightsforaneuralnetwork. Afteryearsofresearch, thecombinationofmultipleclassi?ershasbecomeawellestablishedandexciting researcharea,whichprovidese?ectivesolutionstodi?cultpatternrecognition problems. Aconsiderablebodyofempiricalevidencesupportsthemeritof- signingcombinedsystemswhoseaccuracyishigherthanthatofeachindividual classi?er,andvariousmethodsforthegenerationandthecombinationofm- tipleclassi?ershavebecomeavailable. However,despitetheprovedutilityof multipleclassi?ersystems,nogeneralanswertotheoriginalquestionaboutthe possibilityofexploitingthestrengthswhileavoidingtheweaknessesofdi?erent classi?erdesignshasyetemerged. Otherfundamentalissuesarealsoamatterof on-goingresearchindi?erentresearchcommunities. Theresultsachievedd- ingthepastyearsarealsospreadoverdi?erentresearchcommunities,andthis makesitdi?culttoexchangesuchresultsandpromotetheircross-fertilization. Theacknowledgmentofthefundamentalrolethatthecreationofacommon internationalforumforresearchersofthediversecommunitiescouldplayfor theadvancementofthisresearch?eldmotivatedthepresentseriesofwo- shopsonmultipleclassi?ersystems. Followingitspredecessors,MultipleCl- si?erSystems2000(SpringerISBN3-540-67704-6)and2001(SpringerISBN 3-540-42284-6),thisvolumecontainstheproceedingsoftheThirdInternational WorkshoponMultipleClassi?erSystems(MCS2002),heldattheGrandHotel ChiaLaguna,Cagliari,Italy,onJune24-26,2002. The29papersselectedby thescienti?ccommitteehavebeenorganizedinsessionsdealingwithbagging andboosting,ensemblelearningandneuralnetworks,combinationstrategies, designmethodologies,analysisandperformanceevaluation,andapplications. Theworkshopprogramandthisvolumeareenrichedwiththreeinvitedtalks givenbyJoydeepGhosh(UniversityofTexas,USA),TrevorHastie(Stanford University,USA),andSarunasRaudys(VilniusGediminasTechnicalUniversity, Lithuania). Papersweresubmittedfromresearchersofthefourdiversecom- nities,socon?rmingthatthisseriesofworkshopscanbecomeacommonforum VI Foreword forexchangingviewsandreportinglatestresearchresults. Asfortheprevious editions,thesigni?cantnumberofpapersdealingwithrealpatternrecognition applicationsareproofofthepracticalutilityofmultipleclassi?ersystems. This workshopwassupportedbytheUniversityofCagliari,Italy,theUniversityof Surrey,Guildford,UnitedKingdom,andtheDepartmentofElectricalandEl- tronicEngineeringoftheUniversityofCagliari. Allthesesupportsaregratefully acknowledged. WealsothanktheInternationalAssociationforPatternRecog- tionanditsTechnicalCommitteeTC1onStatisticalPatternRecognitionTe- niquesforsponsoringMCS2002. Wewishtoexpressourappreciationtoallthose whohelpedtoorganizeMCS2002. Firstofall,wewouldliketothankallthe membersoftheScienti?cCommitteewhoseprofessionalismwasinstrumental increatingaveryinterestingtechnicalprogram. Specialthanksareduetothe membersoftheOrganizingCommittee,GiorgioFumera,GiorgioGiacinto,and GianLucaMarcialisfortheirindispensablecontributionstotheMCS2002web sitemanagement,localorganization,andproceedingspreparation. April2002 FabioRoliandJosefKittler WorkshopChairs F. Roli(Univ. ofCagliari,Italy) J. Kittler(Univ. ofSurrey,UnitedKingdom) Scienti?cCommittee J. A. Benediktsson(Iceland) M. Kamel(Canada)

Includes supplementary material: sn.pub/extras

Klappentext

Theacknowledgmentofthefundamentalrolethatthecreationofacommon internationalforumforresearchersofthediversecommunitiescouldplayfor theadvancementofthisresearch?eldmotivatedthepresentseriesofwo- shopsonmultipleclassi?ersystems. Followingitspredecessors,MultipleCl- si?erSystems2000(SpringerISBN3-540-67704-6)and2001(SpringerISBN 3-540-42284-6),thisvolumecontainstheproceedingsoftheThirdInternational WorkshoponMultipleClassi?erSystems(MCS2002),heldattheGrandHotel ChiaLaguna,Cagliari,Italy,onJune24-26,2002. The29papersselectedby thescienti?ccommitteehavebeenorganizedinsessionsdealingwithbagging andboosting,ensemblelearningandneuralnetworks,combinationstrategies, designmethodologies,analysisandperformanceevaluation,andapplications. Theworkshopprogramandthisvolumeareenrichedwiththreeinvitedtalks givenbyJoydeepGhosh(UniversityofTexas,USA),TrevorHastie(Stanford University,USA),andSarunasRaudys(VilniusGediminasTechnicalUniversity, Lithuania). Papersweresubmittedfromresearchersofthefourdiversecom- nities,socon?rmingthatthisseriesofworkshopscanbecomeacommonforum VI Foreword forexchangingviewsandreportinglatestresearchresults. Asfortheprevious editions,thesigni?cantnumberofpapersdealingwithrealpatternrecognition applicationsareproofofthepracticalutilityofmultipleclassi?ersystems. This workshopwassupportedbytheUniversityofCagliari,Italy,theUniversityof Surrey,Guildford,UnitedKingdom,andtheDepartmentofElectricalandEl- tronicEngineeringoftheUniversityofCagliari. Allthesesupportsaregratefully acknowledged. WealsothanktheInternationalAssociationforPatternRecog- tionanditsTechnicalCommitteeTC1onStatisticalPatternRecognitionTe- niquesforsponsoringMCS2002. Wewishtoexpressourappreciationtoallthose whohelpedtoorganizeMCS2002. Firstofall,wewouldliketothankallthe membersoftheScienti?cCommitteewhoseprofessionalismwasinstrumental increatingaveryinterestingtechnicalprogram. Specialthanksareduetothe membersoftheOrganizingCommittee,GiorgioFumera,GiorgioGiacinto,and GianLucaMarcialisfortheir

Inhalt
Invited Papers.- Multiclassifier Systems: Back to the Future.- Support Vector Machines, Kernel Logistic Regression and Boosting.- Multiple Classification Systems in the Context of Feature Extraction and Selection.- Bagging and Boosting.- Boosted Tree Ensembles for Solving Multiclass Problems.- Distributed Pasting of Small Votes.- Bagging and Boosting for the Nearest Mean Classifier: Effects of Sample Size on Diversity and Accuracy.- Highlighting Hard Patterns via AdaBoost Weights Evolution.- Using Diversity with Three Variants of Boosting: Aggressive, Conservative, and Inverse.- Ensemble Learning and Neural Networks.- Multistage Neural Network Ensembles.- Forward and Backward Selection in Regression Hybrid Network.- Types of Multinet System.- Discriminant Analysis and Factorial Multiple Splits in Recursive Partitioning for Data Mining.- Design Methodologies.- New Measure of Classifier Dependency in Multiple Classifier Systems.- A Discussion on the Classifier Projection Space for Classifier Combining.- On the General Application of the Tomographic Classifier Fusion Methodology.- Post-processing of Classifier Outputs in Multiple Classifier Systems.- Combination Strategies.- Trainable Multiple Classifier Schemes for Handwritten Character Recognition.- Generating Classifier Ensembles from Multiple Prototypes and Its Application to Handwriting Recognition.- Adaptive Feature Spaces for Land Cover Classification with Limited Ground Truth Data.- Stacking with Multi-response Model Trees.- On Combining One-Class Classifiers for Image Database Retrieval.- Analysis and Performance Evaluation.- BiasVariance Analysis and Ensembles of SVM.- An Experimental Comparison of Fixed and Trained Fusion Rules for Crisp Classifier Outputs.- Reduction of the Boasting Bias of Linear Experts.-Analysis of Linear and Order Statistics Combiners for Fusion of Imbalanced Classifiers.- Applications.- Boosting and …