This book describes a method for building real-world problem solving systems such as medical diagnostic procedures and intelligent controllers for autonomous underwater vehicles (AUVs) and other robots. The approach taken is different from other work reported in the artificial intelligence literature in several respects: * It defines schema-based reasoning, in which schemas -- explicitly declared packets of related knowledge -- are used to control not only the reasoner's planning, but also all other facets of its behavior. * It is a kind of reactive reasoning that the author calls adaptive problem solving -- the reasoner maintains commitments to future goals but is able to change its focus of attention as the problem-solving situation requires. * It is a context-sensitive reasoning method. Every decision it makes relies on the use of contextual knowledge to be appropriate for the current problem-solving situation. Furthermore, context is represented explicitly; by always keeping a current representation of the context in mind, the reasoner's behavior is automatically sensitive to the context with very little work needed per decision. * Schema-based reasoning -- a generalization of case-based reasoning -- extends the usual idea of case-based reasoning to encompass all aspects of the reasoner's behavior, and it extends it to make use of generalized "cases" (i.e., schemas) rather than particular cases, thus saving effort needed to transfer knowledge from an old case to a new situation. Though the work originated in the domain of medical diagnostic problem solving, treating diagnosis as a planning task, it is even more appropriate for controlling autonomous systems. The author is currently extending the approach by creating a robust controller for long-range autonomous underwater vehicles that will be used to carry out ocean science missions.