This book is based on the author's Ph.D. dissertation[56]. The the­ sis research was conducted while the author was a graduate student in the Department of Computer Science at Rutgers University. The book was pre­ pared at the University of Massachusetts at Amherst where the author is currently an Assistant Professor in the Department of Computer and Infor­ mation Science. Programs that learn concepts from examples are guided not only by the examples (and counterexamples) that they observe, but also by bias that determines which concept is to be considered as following best from the ob­ servations. Selection of a concept represents an inductive leap because the concept then indicates the classification of instances that have not yet been observed by the learning program. Learning programs that make undesir­ able inductive leaps do so due to undesirable bias. The research problem addressed here is to show how a learning program can learn a desirable inductive bias.

Inhalt

1 Introduction.- 1.1 Machine Learning.- 1.2 Learning Concepts from Examples.- 1.3 Role of Bias in Concept Learning.- 1.4 Kinds of Bias.- 1.5 Origin of Bias.- 1.6 Learning to Learn.- 1.7 The New-Term Problem.- 1.8 Guide to Remaining Chapters.- 2 Related Work.- 2.1 Learning Programs that use a Static Bias.- 2.1.1 Vere's Thoth without Counterfactuals.- 2.1.2 Vere's Thoth with Counterfactuals.- 2.1.3 Mitchell's Candidate Elimination.- 2.1.4 Michalski's STAR Algorithm.- 2.2 Learning Programs that use a Dynamic Bias.- 2.2.1 Waterman's Poker Player.- 2.2.2 Lenat's EURISKO.- 3 Searching for a Better Bias.- 3.1 Simplifications.- 3.1.1 Original Bias.- 3.1.2 Representation of Bias.- 3.1.3 Formalism for Description Language.- 3.1.4 Strength of Bias.- 3.1.5 When to Shift to a Weaker Bias.- 3.2 The RTA Method for Shifting Bias.- 3.2.1 Recommending New Descriptions for a Weaker Bias.- 3.2.2 Translating Recommendations into New Concept Descriptions.- 3.2.3 Assimilating New Concepts into the Hypothesis Space.- 4 LEX and STABB.- 4.1 LEX: A Program that Learns from Experimentation.- 4.1.1 Problem Solver.- 4.1.2 Critic.- 4.1.3 Generalizer.- 4.1.4 Problem Generator.- 4.1.5 Description Language.- 4.1.6 Matching Two Descriptions.- 4.1.7 Operator Language.- 4.2 STABB: a Program that Shifts Bias.- 5 Least Disjunction.- 5.1 Procedure.- 5.1.1 Recommend.- 5.1.2 Translate.- 5.1.3 Assimilate.- 5.2 Requirements.- 5.3 Experiments.- 5.3.1 Experiment #1.- 5.3.2 Experiment #2.- 5.4 Example Trace.- 5.5 Discussion.- 5.5.1 Language Shift and Version Spaces.- 5.5.2 Obsolete Descriptions: Strengthening Bias.- 5.5.3 Choosing Among Syntactic Methods.- 6 Constraint Back-Propagation.- 6.1 Procedure.- 6.1.1 Recommend.- 6.1.2 Translate.- 6.1.3 Assimilate.- 6.2 Requirements.- 6.3 Experiments.- 6.3.1 Experiment #1.- 6.3.2 Experiment #2.- 6.3.3 Experiment #3.- 6.4 Example Trace.- 6.5 Discussion.- 6.5.1 Knowledge Based Assimilation.- 6.5.2 Knowledge Based Set Equivalence.- 6.5.3 Bias in Formalism of Description Language.- 6.5.4 Interaction of Operator Language and Description Language.- 6.5.5 A Method for Computing a Strong and Correct Bias.- 6.5.6 Regressing Sub-Goals.- 7 Conclusion.- 7.1 Summary.- 7.2 Results.- 7.3 Issues.- 7.3.1 Role of Bias.- 7.3.2 Sources of Bias.- 7.3.3 When to Shift.- 7.3.4 Strength of Bias.- 7.3.5 How to Shift Bias.- 7.3.6 Recommending New Descriptions.- 7.3.7 Translating Recommendations.- 7.3.8 Assimilating New Descriptions.- 7.3.9 Side Effects.- 7.3.10 Multiple Uses of Concept Description Language.- 7.4 Further Work.- Appendix A: Lisp Code.- A.1 STABB.- A.2 Grammar.- A.3 Intersection.- A.4 Match.- A.5 Operators.- A.6 Utilities.