School of Computer and Information Sciences Nova Southeastern University Certification of Authorship of Dissertation Work Submitted to:Dr. Steve Terrell, Dr. Jacques Levin, Dr. Rollins Guild and Dr. Ron Owston Submitted by: Name:James Gow User Code:gowj Day Telephone:305- Date of Submission:9/5/96 Purpose and Title of Submission: Replicate a previous study to see if any new hypothesis emerge in a case study environment and in a different domain. Student Development of an Expert System: A case study in middle school. Certification of Authorship: I certify that I am the author of this paper and that any assistance I received in its preparation is fully acknowledged and disclosed in the paper. I have also cited any sources from which I used data, ideas or words, either quoted directly or paraphrased. I also certify that this paper was prepared by me for this purpose. Student's Signature:______________________________________ James Gow Student Development of an Expert System: A case study in middle school by James Gow A Dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy School of Computer and Information Sciences Nova Southeastern University 1996 We hereby certify that this dissertation, submitted by James Gow, conforms to acceptable standards and is fully adequate in scope and quality to fultull the dissertation requrements of the degree of Doctor of Philosophy. _________________________________ ___________ Steven Terrell, Ph.D. Date Chairman of Dissertation Committee _________________________________ ___________ Jacques Levin, Ph. D. Date Dissertation Committee Member _________________________________ ___________ Rollins Guild Ph. D. Date Dissertation Committee Member _________________________________ ___________ Ron Owston Ph. D. Date Dissertation Committee Member Approved: _________________________________ ___________ Edward Lieblein, Ph. D. Date Dean, School of Computer and Information Sciences School of Computer and Informaiton Sciences Nova Southeastern University Certification Statement I hereby certify that this dissertation constitutes my own product and that the words or ideas of others used, are properly credited according to accepted standards for professional publications. Signed ___________________________________ James Gow An Abstract of a Dissertation Submitted to Nova Southeastern University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Student Development of an Expert System: A case study in middle school by James Gow December 1996 Students will participate in a three month replication of an Owsten and Wideman (1988) study with case study analysis and a different domain. This replication is designed to find if any new hypotheses emerge regarding the usefulness of novice knowledge engineering in middle school education to help students learn fractions. More computers will be used and fractions will be explored instead of biological classification of living things. Fractions have been a problem in public school math teaching for some time. Since hardware and procedural limitations have prevented the exploration of the middle school classroom with expert systems, this study represents a breakthrough regarding the use of novice knowledge engineering and the math curriculum. Acknowledgements I would like to thank Dr. Renate Lippert at the University of Pretoria, South Africa for her early support in my efforts to extend the work in novice knowledge engineering and Dennis Murphy for his help in teaching me about expert systems with his expert turtle (ET). I would like to thank Dr. Seymour Papert, Dr. Sherry Turkle (epiphenonimally) and the epistemology and learning lab at MIT for their support with materials that changed my viewpoint on constructionism. Most of all I would like to thank God, my family and church for standing by me through this difficult time that included a hurricane and the death of my son Peter. Table of Contents Abstract v List of Figures viii Chapter I. Introduction 1 Relevance, Significance and Need for the Study...........................1 Statement of Problem.......................................................................3 Barriers and Isssues........................................................................5 Elements and hypotheses.................................................................6 Limitations..........................................................................................7 Summary............................................................................................10 II. Review of the Literature Historical Overview Educational Software.........................................................................11 Theory and research specific to the topic Expert Systems...................................................................................16 Metacognition......................................................................................27 Control Groups and Refinement.........................................................30 Knowledge Base and Cognitive Model Building.................................................................................................31 Mental Imagery....................................................................................31 Programming Experience....................................................................32 CAI Research......................................................................................34 Fractions..............................................................................................38 Summary..............................................................................................44 vi III. Methodology Research Methods Employed Introduction............................................................................................47 Design....................................................................................................48 Procedures............................................................................................49 Data Collection......................................................................................55 Data Presentation.................................................................................57 Hypothesis and Measures of Success................................................59 Footnotes...............................................................................................62 Appendix A.............................................................................................................89 B.............................................................................................................90 C.............................................................................................................91 D.............................................................................................................92 E.............................................................................................................93 F.............................................................................................................95 G.............................................................................................................96 H..............................................................................................................97 References...........................................................................................62 Annotated Bibliography.........................................................................76 vii List of Figures Figure one Fraction Islands......................................................................................42 Figure Two Translation Models.................................................................................43 Figure Three Fraction Tree...........................................................................................53 Figure Four Piagetian Chart.......................................................................................58 viii  Chapter I Introduction Relevance and Significance Working with Expert Systems (ES) in learning is a new topic today. ES have been around for some time but their use in education for learning has been limited. Most ES learning programs have become Intelligent Computer Assisted Instruction (ICAI) or Intelligent Tutoring Systems (ITS) programs and have rarely left the laboratory (Dede, 1985; Wideman & Owston, 1993). ES offer an uncommon cross between tutors, tools, and tutees (Tutees are ES whose knowledge bases are created by students). Most of the research up to this point in the area of Novice Knowledge Engineering (NKE) has been with college level students on mainframes and PC's (Bouwman, 1995 and Knox-Quinn, 1995). In Lippert's dissertation (1988) she suggested the possibility of working with younger children, but she worked with college level students. Wideman and Owston, 1988 and Bigum, 1986 were the first to do formal research in the middle school area and opened the door to further exploration into the use of ES as tutees with children of this age. This research, despite hardware limitations, will prepare the way for the day when knowledge bases (KB) become a medium of exchange (Hayes-Roth, Lenate and Waterman 1983). Papert's first approach with children in Logo was with children in the seventh grade. The reason Papert chose the seventh grade was because it was just within the formal reasoning barrier defined by Piaget. What Papert was planning to do was work his way down the age level after work at the formal barrier of learning had been worked out. Today, work with younger children in Logo has been fruitful because of this early start. This pattern is repeating itself with NKE. Research is descending from this age level because of improved research techniques and availability of larger memory PC's in the public middle schools. This reduction in age range will result in applications of NKE that will be significant with lower age children. Dr. Owston (personal communication, August 11, 1994) feels sixth grade is the lowest grade level for NKE. A published paper at Eurologo (Gow, 1993) on NKE is with work at the seventh grade level. This area of NKE is a fruitful area of research and is providing insight into how children learn what they learn. Papert, in his early thinking about artificial minds, with Jean Piaget laid the groundwork for Logo. Since NKE deals in the artificial mind that spawned Logo there is no telling what innovations will come out of the work with NKE. In NKE, the object of construction is the knowledge that the student is in the process of acquiring. In the new model of instruction with ES, the object is the knowledge itself and provides a better understanding of how a student can acquire and put into use the knowledge they must learn. This understanding includes a metacognitive aspect since the student builds the knowledge they are building in their own minds. Anderson, in his widely quoted ACT* theory (1987), states that mental activity consists of production rules being activated. Metacognition is an important part of cognition. According to Eyler (1989) and Guernon (1989), students who have good metacognitive functioning are better problem solvers. By constructing knowledge bases (KB's), students can see their knowledge as they make it and get some understanding of how their mental processes work, improve their metacognitive thinking, and as a result, their problem-solving abilities (Swan, 1989; Eyler 1989; Guernon 1989). Statement of Problem Learning fractions has always been a problem for students (Harel, 1988). Even today in Dade County Public Schools, according to W.R. Thomas math teachers (Meeting, 1995), complaints about students ability to solve problems with fractions still persist. Because CAI (Computer Assisted Instruction) has not improved problem-solving skills in this area (Harel, 1988; Papert, 1983, 1993) and a growing perception that these skills needed to be improved (NCTM Standards, 1980, 1994), different methods have been tried. Logo was offered as a solution to the fraction solving problem. However, since it was found that skill gain was not always evident with Logo (Harel, 1988; Wideman et al., 1988 ), or unless explicitly taught (Swan, 1989), some other means of improving students ability to do problem-solving with fractions needed to be found. According to Means, Olson, and Singh (1995) most schools have few teachers interested in technology and students get frustrated because skills they learn are not used with other teachers or from year to year. An international study in twenty-two countries had similar findings (Plomp, T., Pelgrum, W. J. 1991). Plomp et al.'s international study found that the major problems were in the lack of enough software of high quality and insufficiently trained teacers. Caftori (1994) found that children do not use educational software as intended by its designers. Also Caftori found teachers did not read the documentation about the programs limiting their usefullness in directing educational activities. Hypermedia systems have problems because of the fact most hypermedia designs are based on the capabilities of the technology and not on the instructional theories or research findings. (Moore, D.M.; Yang, C. S., 1995). Too much learner control in hypermedia systems is granted for navagation without specifying learning objectives or pedagogic strategies. Many studies have failed to prove that learner control resulted in better learning effects (Moore, D.M.; Yang, C. S., 1995; Lookatch, R. P. 1995). According to Moore et al. 1995 learners may learn less from the oprtions they like most. Futhermore in a survey of technology using teachers and schools only 4% use multimedai and CD-ROM materials in the classroom (Liebowitz, J., 1996). Liebowitz (1996) says at the university level many faculty are unwilling to inject this technology into the classroom because they feel it is all hype and no content. Computer based expert systems sometimes called ITS (Intelligent Tutoring Systems) or ICAI (Intelligent Computer Assisted Instruction) have been found to develop problem-solving skills (Wideman et al., 1988). However because of their highly directive approach, students do not get an opportunity to develop their own original projects, a means of improving problem-solving skills (Wideman et al., 1988), Piaget (1971) and constructivist thinking says, "to understand is to invent." Without the abilbiity to invent, learning is inhibited. Recent research in the use of ES (Lippert, 1988; Lippert, 1989; Knox-quinn, 1991; Gow, 1993; Knox-Quinn, 1995; Bouwman, 1995) shows increases in problem-solving abilities among participating students and while developing original projects. Because an ES Knowledge Base Building (KBB) approach shows increases in problem-solving abilities, this approach was chosen for the teaching and learning of fractions. Not much is known about the strengths and limitations of student ES development in the middle school classroom as an educational activity. Most researchers have done studies with college level students (Starfield, 1983, 1986; Lippert, 1988; Lai, 1989; Knox-Quinn, 1991; Knox-Quinn, 1995; Bouwman, 1995). Most of the results of these college level studies have been positive but give little evidence on what the outcome on middle school students would be. In the middle school, studies of NKE have been limited with only a few observations (Knox-Quinn, 1991; Bigum, 1988) and few major studies (Wideman et al., 1988, 1991, 1993; Gow, 1993). A replication with case study analysis and a different domain of the Wideman et al. (1988) study is presented as a way of improving problem-solving problem in fractions. Additionally, a better understanding of what are the educational benefits to middle school children of expert system development and further clarification of the types of hypotheses about the strengths and weaknesses of the educational benefits to middle school students will be examined. Another interest is how ES fit into the educational software environment. Barriers and Issues The reason this research has not been done in the mathematics domain before is because it has been difficult to determine exactly how this age group and domain might be approached in Novice Knowledge Engineering (NKE). Also according to Reynolds (1988), hardware limitations have prevented research work in the public schools in the area of expert systems. Public middle schools in Dade County, Florida suffer tremdously from this problem (Gow, 1993). Even Wideman et al.'s research used only two computers and had them connected to a main frame via a telecommunications link. It has only been in the last three years that Miami, Florida, Dade County public middle schools have started to acquire the hardware capabilities necessary to run expert system software. Elementary schools in Dade County have consistently acquired more hardware. According to Piaget (1971) and Papert (1993), middle school students fall just within the operational stage of development necessary to do this activity. Up to this point, other researchers have not been sure how to approach this age group and domain. Hardware limitations have been so severe, it was considered impossible except for telecommunications links to use expert systems with existing public school computer systems. Wideman et al. (1988, 1993) have created some procedures in working with seventh graders that make this domain and age group possible. Because of Wideman et al. (1993) and Gow's (1993) preliminary research with seventh grade math students with expert systems, this domain is now more accessible. Public schools within the last year have upgraded their hardware base to include enough machines that could run expert systems with a class of twenty. By replicating Wideman et al.'s research with more computers, a different domain and video for "thicker" (Harel, 1990; Farber, 1990) data collection, it will be possible to see if any different hypothesis about the strengths and limitations of student expert systems development in the classroom as an educational activity emerge. Elements and Hypotheses The original hypothesis that emerged as a result of the Wideman et al. study were: 1. The experience of creating a knowledge base has the potential to deepen knowledge of the content domain while promoting the development of more mature cognitive strategies. Some additional hypotheses emerged as secondary and they are: 2. Groups will be able to complete tasks of greater cognitive complexity than is typically demanded of them in the curricula for their age level, and to do so with a good degree of enthusiasm. 3. Develop a deeper understanding of interrelatedness of taxonomic strata and keys, and the procedures of rule-based classification. 4. Force students to use rigorous and systematic thinking to succeed. 5. Project activities, promote students acquisition, practice, and extension of their cognitive and metacognitive skills. 6. After awareness of deficiencies in procedures is developed, students can rapidly assimilate the new strategies and use them as appropriate. 7. It may be likely that student development of knowledge bases could be undertaken in any subject domain that can be effectively presented by production rules, giving this activity a wide curricular scope. 8. Development of a simple, rule-based expert system can provide a valuable educational experience at a surprisingly early age, as long as the activity is properly structured and well supported. Precautions: (a) Group tasks are easily divisible, so that all members are challenged simultaneously and student time is not wasted. (b) Ample resource materials must be available so students do not have to compete for them. (c) The expert system shell should be introduced gradually so that students are comfortable with consulting it and developing simple prototypes before attempting their main tasks. The purpose of this study is to see if a new hypothesis emerges from this investigation as well as any new secondary hypotheses and how they differ from the ones above. Limitations of the study Wideman et al., (1988, 1993), collected data to form hypotheses about the strengths and limitations of student expert system development in the middle school classroom as an educational activity. The hypotheses formed are based on data collected from two sources: detailed notes and interviews based on procedures outlined by Bogdon and Biklen (1982). Bogdan and Bilken (1992) will be used in the updated study. However, to broaden the study and thicken the data collection, video tape will be taken of all events and activities based on Harel (1990). Fractions are a good area for the use of knowledge based systems because much of the learning is rule based and its teaching is difficult. Fractions are also a good area for rule based systems because design flaws in other teaching methods (Harel, 1998; Mcarthur, D., Lewis, M. W. & Bishay, M, 1995) have prevented the acceptance of their methods to teach this area. Better methods are needed to improve students' learning in this area since the old and new ones have proven ineffective. If more hypotheses are to emerge and a broader understanding of the educational benefits of expert system development with middle school children is to be found, it is hoped broadening of an existing study will show it. It is also hoped that this replication will show how expert systems can be used as educational software. In replicating Wideman et al.'s (1988) study, the students studied will remain constant at thirty-five, however, the domain engineered will be changed from biological classification of living things to addition of fractions with unlike denominators. Instead of relying solely on notes and post observation recall, video tape will be added to record all protocols. To overcome any limitations, Bogdan et al. suggest use of more detailed notes at the time of observation, video tape will be invaluable in this process since it will capture many things that may not seen in the classroom. In addition, the researcher will not study or teach his own class for whom he has academic responsibility. The class will be taught by another classroom teacher who will be trained by the researcher. The computers will increase from two PC computers in the Wideman et al. (1988) study to six Macintosh computers. Dr. Owston of Wideman et al. (1988) is available through e-mail so their help in replicating the study will be invaluable. Dr. Owston has agreed to be on the committee as an external advisor to this project. His influence and guidance will help guide this project to be a true replication and expansion of his previous study. Wideman et al. (1988, 1993) have specific procedures they followed in the teaching of students in learning how to operate expert system software and build knowledge bases. These procedures will be followed to reduce variables in the case study analysis and different domain of the study. The procedures consisted of an introductory phase of four stages, and a project phase that consisted of five stages towards the development of students' own expert system. According to Bogdan et al. (1992) access to a facility is important and often difficult to obtain. Since the researcher is a sixth grade math instructor, access to resources in the study site school are open. Computers in the numbers needed to broaden the study are available in the computer lab. Parental permissions (see Appendix C) for each individual student must be obtained for work in a Dade County Public School. The permission of the principal has already been obtained and all work will be done with students of another teacher who has agreed to cooperate. The subjects and the location of the study are readily available as well as the computer teacher's help and cooperation in using their children. All resources necessary for the implementation are available including the software. Software of a similar type has already been purchased from EXSYS (EXSYS, 1994) a forward and backward chaining expert system similar to the system used in the original study and used by Lippert. The version of EXSYS is a demo version recommended by Lippert (1989). The demo version is full featured but can save only 25 rules. According to Lippert (1988), 25 rules is enough for middle school children. EXSYS has been used in several commercial applications: CONVCTIV, used to predict weather and developed by NOAA at Colorado State University; Lung Cancer Information Retrieval Application, for obtaining staging, prognostic, and therapeutic information relevant to patients with lung cancer at Cedars-Sinai Medical Center, Los Angeles, California.; Metals and Alloys Identification Application, identifies common metals and alloys, developed by General Electric (Durkin, 1994). Summary Chapter I outlined the study about relevance, significance and need for the study and a new method to help students learn fractions. The statement of problem gave the reader a view of how solving fractions has been a problem for some time for middle school students. Despite several barriers (age and number of computers), the issues were disscussed and the hypotheses were reviewed regarding the study. Limitations of this study were outlined and any special terms were defined. Chapter II will review in more depth the literature in the field, and Chapter III will describe in more detail the method used in the study implementation as well as data collection methods. Chapter II Literature Review Educational Software There are many types of educational software available for teaching fractions. The most proliferate is CAI, however the problem with CAI has been for years is that it wastes the power of the computer and limits what the student can learn (Papert, 1980, 1990, 1994; Caftori, 1994). According to Means et al. (1995) many instructional uses of technology merely reinforce traditional didactic modes of instruction.. Educators need to recognize that it is one thing to use technology in isolated classrooms and another to make technology a potent force in transforming an entire school or an entire education system (Means et al. 1995). Most CAI types of software are drill and practice and seldom used as designed (Caftori, 1994). According to Caftori, 1994 children left on their own and wanting to have fun, will naturally divert their attention to the features that appeal to them most and bypass many of the programs learning opportunities. The problem persists even with ILS's (Integrated learning systems) although they have been tauted as a potental remedy for low test scores. ILS's include courseware and management software that run on networked hardware (Van Dusen and Blaine, 1995). In a national study Van Dusen et al. 1995 found that the ILS systems were being under-utilized accounting for the poor results in student improvement. According to Maddux and Willis, 1992: ...ILS's fit a behavioral, competency-based model of instruction that was more popular in the past than it is today. ILS's do not fit well with the cognitive/constructivist models of learning and teaching that have largely replaced behavioral theories. (pp. 56). What Van Dusen et al. 1995 found was that the systems were used sometimes for only ten minutes a week. According to West and Donald 1994, time on the ILS is directly related to achievement. If the student doesn't spend at least 40 percent of their time in the ILS, they will not show any significant achievement. According to Van Dusen et al. 1995 over 80 percent of the teachers using the ILS systems did not use the ILS reports although the ILS reports give feedback necessary for student achievement. Van Dusen et al. 1995 also found that teachers were not integrating the ILS's into the curriculum. What the teachers were doing was using the ILS software and curriculum as a supplement rather than a main part of the curriculum. The problem with authoring tools whether hypermidia like hypercard or linkway or courseware like Pilot is the teachers who have been trained with CAI, make more CAI type of software instead of improving on already flawed models. Liebowitz (1996) says that multimedia development is no trivial task, and one person does not have all the skills it takes to produce good multimedia programs. For the teacher who has taken two classes in computers to graduate from college, such an undertaking would be difficult. Like Harel, (1988) students might learn from making their own CAI software, however like in her experiment, the results were successful only when students spent more time on task than the other students (M. W. Lewis, Personal communication, April 7, 1992). Another problem with authoring tools like Pilot or hypermedia like linkway is that most teachers do not have the time to write software for their classes (Gow, 1985; Plomp et al. 1991). To get teachers to participate in technology classes they had to be paid $300.00 to participate and get their skills to the introductory level (Murphy, M. and MIller, A.,1996). Today's classroom is overloaded with students and the teacher spends most of their time grading and preparing for each day's classes. Today's computers and software have graphics and sound that the average teacher can't duplicate with their limited programming experience and the limited abilities of many authoring tools and for that reason most teachers reach to the shelf to find existing software to use with their students. Many teachers are not motivated to integrate technology into the classroom. Programming languages like Logo, MacSmarts and Scheme have been used to teach fractions and other skills. One problem with using these languages has been the ability of the student to transfer the skill to other domains (Clements & Gullo, 1984). Harel found that using Logo in a project environment improved performance, however, since the time on task was greater than the traditional group, a flaw in the method was detected (Harel, 1988). Swan (1988) found that problem-solving had to be explicitly taught when using Logo and for that reason it appears that using languages specifically to teach specific domains and problem-solving is not better than traditional methods. ICAI or ITS is an attempt to improve on the failures of CAI but since most researchers do not even agree on what good teaching is it is difficult for ICAI or ITS researchers to model good teaching in an artificailly intelligent teacher (Mark and Greer,1992). This puts many limitations on the artificial teacher making it at best highly experimental. There are many types of expert systems but only a few are available for educational use. Most of the research on knowledge base building has been done with forward and backward chaining systems (Lippert, 1988; Wideman et al., 1988; Knox-Quinn, 1991, 1995; Bouwman, 1995). ES like VPExpert were available for a limited time and are currently no longer supported (VPExpert, Personal communication, May 5, 1995). Many of the commercial systems do not support demo programs. The cost of demo systems is similar to CAI software contrary to Lippert's analysis (Lippert, 1987, 1990). About the only type of expert system shell that is complete and inexpensive are the home grown versions (Lippert, 1988; Gow, 1993). Using Durkin's book (1994),a demo version of an expert system called EXSYS was obtained at a reasonable cost ($30) however to include the manual the total cost was $75. Running the program without the manual would have been impossible. This cost is similar to those mentioned by Lippert (1987, 1990) but are a little higher when including the manual. $75 is competitive with today's educational software since the additional sound, graphics and video in multimedia increases the cost. The advantage of working with expert systems and building knowledge bases is that students work directly with the knowledge that they are required to learn. In the programming, authoring tool and hypermedia example, students must first learn the symbols and their manipulation to work with the skills they must learn. In building an expert system, students work directly with the knowledge they are to learn giving them a more direct route to acquiring the skills required in the school ciruiculum. One advantage of working directly with the knowledge students must learn is that they improve problem-solving and metacognitive skills not improved with other educational types of software (Lippert, 1988; Knox-Quinn, 1991; Wideman et al., 1993; Bouwman, 1995). A recent development in educational software is worth mentioning from the Rand Corporation (Mcarthur,D., Lewis, M. W. & Bishay, M, 1995). This initiative is called ESSCOTS (Educational Support Systems/Commercial off the Shelf) for learning initiative. What Mcarthur et al. have done is take commercial software off the shelf and put it into a generic shell that will allow what is normally used in industry to be used in a classroom. This is done by using programming languages that are included with the software for their expansion or interfacing with databases and using it to create menus for children to use. Some advantages of this software is that it already exists and does not have to be programmed from scratch. Some sophisticated software can be adapted for educational use at a reasonable cost. Mcarthur et al. report some success in an unscientific sample and list some of the limitations. The effort has produced no evaluation to measure what students learn using these tools and the work has not moved out of the lab and into a large classroom. There are other problems with this type of approach. There is a danger of overwhelming the student with possible issues and patterns to pursue because real search spaces are large ones (Mcarthur et al., 1995). Students using ESSCOTS have two spaces to search, the software itself and the subject they are studying doubling the complexity. No doubt some of the same problems will continue with teachers not reading documentation and using the software for what it is designed (Caftori 1994; Means et al., 1995). A big advantages of NKE over ESSCOTS is the direct manupulation of the information that is to be learned. Although ESSCOTS opens a new avenue for CAI, it still wrestles with some of the same fundamental problems in CAI, since it is CAI in a new wrapper. The student must first master the software they are using before they are introduced to the skills they are must learn. In NKE the student immediately deals with the knowledge they must learn since it is the subject of the NKE process. Expert Systems There are many types of expert systems on the market. Lippert in her NKE work at the University of Minnesota after reviewing several prototype systems decided to use an in-house program because of its simpicity rather than its wide commercial acceptance. Student development of expert systems (also known as NKE) has been in the literature since 1983 with Dr. Starfield's engineering students developing expert systems. Starfield, Butala, England and Smith (1983) report students were unusually enthusiastic about building their own expert systems, perhaps largely because they felt they were doing something worthwhile. However this is part of the problem. This reference is for college level students and is typical of most of the work in the field. Lippert (1988) built expert systems with college level physics students to teach projectile motion, but felt building of a Knowledge Base (KB) as an instructional strategy could also be used for a younger age group, and in a whole group context. Lippert called the student knowledge engineer Novice Knowledge Engineers and what students did while building knowledge bases for expert systems, Novice Knowledge Engineering (NKE). Lippert thought different domains were suitable for NKE and suggested in science, the revision of the classification of matter might be a good area for ES (Wideman et al., 1988). Although Lippert didn't work with middle school children she at least pointed out the possibility of it being done. However this was not a study done with middle school students and therefore still part of the problem. Bigum (1988) has found that students from year seven in high school to the final year of undergraduate study have little difficulty in working with some of the commercially available knowledge based system shells. Bigum however never developed any formal study or any method in working with his level seven students. Even in personal communication or any of his published papers did Bigum present any method of working with students of middle school age; he simply stated that they didn't have any trouble. Lipperts work pointed out two things: (a) The usefulness of NKE as instructional pedagogery; and (b) Students refined their knowledge as they built knowledge bases (KB's). Lippert's experiment was a case study of five students who built knowledge bases on physics problems in projectile motion for the ES. Student comments in Lippert's experiments were positive. They felt they were doing something useful and that the process of NKE was a good experience in their educational history. Although students produced an individual product and these products varied from ten rules to thirty, they consulted with each other to pool their expertise and sometimes did not finish their projects. Knox-Quinn (1991, 1995) also did a study of NKE with college level students and their learning of accounting principles. In preparation for her college level experiment, Knox-Quinn did a pilot study with gifted middle school students. In her middle school pilot study, Knox-Quinn concluded: "When students build expert systems in the classroom, it is not the quality of their final systems that is most important, it is the experience of the student as knowledge engineer that is of real value." (Knox-Quinn, 1991, p. 72). However, Knox-Quinn although closer to middle school students used gifted students who are cognitively more like college level students, the subjects for her final study. Knox-Quinn never presented any method for working with her middle school students or any formal research activities. The only researchers who have done experiments with middle school eighth grade students previously mentioned were Wideman et al. (1988) and Bigum 1986. In a newer study (Wideman et al., 1991), a PC-based expert system shell was used instead of a mainframe shell as in the 1988 study. The domain of the knowledge was weather. Their " . . . results provide partial support of the viability and efficacy of knowledge base creation as an instructional strategy for fostering cognitive development at the grade eight level" (Wideman et al., 1991, p. 43). Wideman et al. (1991) feel, "It would be premature to discount the value of expert systems activity for the average intermediate level student . . . " (p. 43). For this purpose, they recommend changes to the expert system shell making it easier for intermediate students to use and shorter projects with less complex knowledge bases. Seymour Papert (1993) in his book "The Children's Machine" reviews his thinking in the 60's when he was working with Jean Piaget. Papert says a few years earlier he had discussed with Piaget what would happen if children could play at " . . . building little artificial minds." Papert felt the essence of AI is to " . . . make theoretical psychology concrete." Since Papert felt children functioned well in the concrete area of thinking he reasoned that children could do some elementary form of AI. Papert says Piaget was intrigued by the idea of being able to take his aphorism of "to understand is to invent" into a new domain (AI). Papert says he and Piaget were carried away with images of children understanding thinking through playing with materials needed to invent " . . . a thinking machine, an intelligence." Although at the time Papert thought the discussions were nothing more than a "Gedankenexperiment" he says it formed the impetus for a real project later to become known as Logo. Papert says that when people do AI they select a piece of human mental activity, e.g. playing chess or seeing a cat, and then write a computer program that does something similar. After the program is written, the programmers discuss whether the program does what the human does with the activity. According to Papert (1993), these discussions produce insights into how people think or how they do not think. In his book, Papert says for him it seemed plausible to think that children doing elementary AI could benefit from these discussions that encouraged thinking about thinking. The way Papert envisioned children doing elementary AI was to study in groups a game called "twenty-one." In twenty-one students pick match sticks from a pile of match sticks one at a time. The person who takes the last stick loses. According to Papert (1993): The children's immediate goal is exactly that of people making what would later be called expert systems: Carefully watch someone engaged in the activity you want your program to imitate, and try to come up with rules you can put into a program to make the computer act similarly. (p. 170) Clancy (1993) in reviewing his work with Guidon-Manage (an ES, project), has come to the conclusion he must start with the user environment and not with computer science ideas. Clancy and Harel agree that they want the broadest possible understanding of how their work will fit within the larger socio-technical system. Since Harel and Lippert both developed their products in a classroom environment, a strictly quantitative approach will be difficult. In a preliminary study (Gow, 1993) while trying to teach fractions by building KB's and using a Logo-based ES shell, student knowledge gaps became apparent while building a knowledge base about fractions. Students were engineering the concept of one-half into a rule that would be put into the KB and later used to add to one-fourth. After all the concepts that represented one-half had been written on the chalk board and students had reached an impasse regarding their understanding of one-half (halfedness), a picture of the object being engineered needed to be presented to illustrate this gap. Students were having trouble with the idea of connectedness1. It was not until the above picture was drawn on the chalk board that students did not identify their gap. D. D. Murphy (personal communication, August 23, 1993) and P. M. Adams (personal communication, March 8, 1994) said it is not a gap but intuitive knowledge or expertise (truth without method) associated with side although students report appreciation for being made aware of this concept and their attitudes improved regarding the activity. This concept of a gap in the area of connectedness persists down to the sixth grade only in a reverse direction. Sixth graders, when asked the same questions had no explanation for a square shaped like a pound sign # where the sides are collapsed within the figure. In other words students were not including information regarding the connection of the sides at the corners (Gow, 1993). Since ES involve higher level thinking processes (Dede, 1985) and are easier to reprogram than computer assisted instruction (CAI) (Baur, 1987), this investigation will seek to determine if new hypotheses will emerge from the Wideman et al. (1988) framework about the educational value of NKE in the fractions domain with middle school students. Since CAI has proven to be of limited value in education utility (Battista, Clements & Nastasi, 1990; Clements & Nastasi, 1988; Frederiksen, 1984) and according to Papert (1990) a symptom of a way of thinking that puts emphasis on the smallest side of education and wastes the power of a computer, a replication with case study analysis and different domain may uncover some new hypotheses about the use of NKE in education with fractions. O'Neil (1987) says when knowledge embodied in a program is a result of a human expert elicitation, these systems are called expert systems (Durkin, 1994). Winston (1992) says the term expert system is a misnomer and what they should be called are "idiot savants." According to Winston the expert system lacks several attributues of a human expert: 1) They do not reason on multiple levels; 2) They do not use constraint-exposing models; 3)They do not look at problems from different perspectives; 4) They do not know how and when to break their own rules; 5)They do not have access to the reasoning behind their rules" (1992, pp.171-2) In writings by Nagy, Gault, and Nagy (1986), the expert system matches user input with the rules in its knowledge base, and, through a process of inference, provides the users with solutions to their problems. An ES is composed of three parts according to Ford and Macalister (1989), a knowledge base, inference engine, and a user interface or knowledge acquisition facility. In general, an expert system may have four components: 1) Knowledge base, analogous to Long Term Memory (LTM); 2) Inference Engine, the thinking part of the system; 3) Working Memory, transient facts, analogous to Short Term Memory (STM); and 4) Interface, connects user with the system. The knowledge base represents the rules of thumb (hueristics) and decision-making guides for using the data (Durkin, 1994; Hayes-Roth et al. 1983; Winston, 1992; Murphy, 1993a). In NKE, this is the student's role. S/he can design this knowledge for use in the expert system in association with the school curriculum. The inference engine is another term for the computer program and the user interface that allows non-experts to use the ES (Ford & Macalister, 1989). White (1987) states knowledge system programs have considerable potential for training and education and accordingly should become increasingly familiar to instructional specialists. An expert system is a computer program that attempts to duplicate the high performance of a human expert when that expert is working in his or her limited domain of expertise. Expertise is that significant activity most people cannot readily do. Expertise is also referred to as truth without method (Durkin, 1994; Hayes-Roth et al., 1983). The model of expertise can be broken down into four components: 1) Knowledge Base (domain-dependent knowledge in some subject area + some general background knowledge). There are two especially popular ways of representing knowledge in a computer: rules and frames ( Minsky, 1985). 2) Inference Engine: General search strategies that can deduce facts. The purpose of the inference engine is to search the knowledge base, deducing facts that can be added to the working memory (Durkin, 1994; Hayes-Roth et al., 1983; Winston, 1992). There are several schemes for arranging this search when the knowledge is represented as rules: frames, backward and forward chaining. When frames are used, inference engine search involves inheritance and daemon programs. 3) Working Memory: The collection of pertinent facts that have been established while working on some specific problem. 4) Interface: This is the I/O (input/output) system that obtains information from the world and reports back its findings. For humans, the I part corresponds to perception, while the O part normally corresponds to writing or speech (Murphy, 1993a). Rule-based expert systems require use of logical inference. The purpose of an expert system is to deduce facts about a world so the methods and terminology of logic provide a good starting place. Logic makes two distinctions about how the world of logic is divided: (1) statements (data), (2) inference methods (algorithms). The parenthesis is the programmer's view (Murphy, 1993a). Basic data of logic is captured in statements about a world. These statements can have only two values: True or False. These statements are called propositions, and there are three kinds: atomic, nonatomic (compound), and predicate (Durkin, 1994; Winston, 1992; Murphy, 1993a). The basic method of inference in logic is called deduction (going from the general to the specific). Induction is the reverse and example based. There are several ways of carrying out deduction: modus ponens, modus tolens, truth tables, and resolution. All systems try to establish facts from a given set of premises (facts and rules) (Murphy, 1993a; Durkin, 1994; Hayes-Roth et al., 1983). Expert systems are divided into two different types of systems, monotonic: and nonmonotonic. In a monotonic, system a fact can never be withdrawn. In this sense the system that is monotonic is one direction, forward or backward only (Durkin, 1994; Winston, 1992; Murphy, 1993a). These systems use propositions as the basic data of logic. Propositions are statements about some world. A proposition about a world can be evaluated as either true or false. Facts are propositions, e.g., the dog is brown, Washington was the first president, Pearl Harbor was a Japanese plot. For a proposition to be assigned a value, it must have meaning. The proposition the brown dog was a friend of Washington's Pearl Harbor makes no sense. Since it makes no sense, it cannot be assigned a value of true or false. Common sense reasoning since it can be assigned values of true or false, becomes a study of interest to artificial intelligence researchers (Minsky, 1986). Propositions can be further defined as atomic or nonatomic (Winston, 1992; Murphy, 1993a). Atomic propositions cannot be broken down into smaller propositions where some of its parts are propositions themselves. A nonatomic proposition can be broken down into more than one proposition. Parts of the nonatomic proposition can be propositions themselves, e.g., bill is a good guy and so is Tom. Here we have a logical connective (and) that joins the parts of the nonatomic proposition so that each part could be considered a proposition in itself. Bill is a good guy is true, and Tom is a good guy is also true. Each of the parts of the proposition are propositions themselves. Other logical connectives are OR (when used to join two propositions). The proposition is false only if both propositions that precede and follow the OR are false. XOR (exclusive or) is also a connective joining two propositions and is true if either the proposition preceding the XOR is true or the proposition following the XOR is true. NOT is another connective and reverses the value of the proposition. If a proposition is true not the proposition is false. IF...THEN is used in rules and are the most common connectives used in rule based expert systems like EXSYS (1994). Inference in expert systems is propositional calculus (Durkin, 1994; Murphy, 1993a). Here calculus means a method for calculating values. Here we are trying to determine the truth of propositions with two ideas in mind: 1) calculating the truth of compound propositions, and 2) deducing facts from given facts using syllogisms. In deductive reasoning we are interested in the form IF A THEN B since this is the form the rules are presented in rule-based systems. The most powerful method of deduction is called modus ponens (Durkin, 1994; Murphy, 1993a). In modus ponens, if you are given IF A THEN B and A is true then we must conclude B is true also. Since EXSYS uses choices (goals) in its backward chain, we could think of this as if we want to prove B is true then we must prove A is true (EXSYS, 1994). When the opposite of modus ponens is being used, we would say IF A THEN B and NOT B is true then we would say NOT A is true also. This form of reasoning is called modus tolens. From here we can see that deduction, whether we use modus ponens or modus tolens to get there, is a valid form of inference. Since Davis, Shrobe and Szolovits (1993) define inference as any way to get new expressions from old, we can see that these forms of deductive reasoning are doing just that, getting new from old. If A is old and true, then B is new and true. In a system like EXSYS that can do goal-directed rule-based backward chaining, we can give the system a goal (or choice) of NOT P and establish two rules (parameters): RULE 1: IF NOT Q THEN NOT P and RULE 2: IF NOT R THEN NOT Q. EXSYS will try to prove NOT P is true because it is a choice (goal). We will see how EXSYS would use these rules to prove by modus tolens the goal NOT P. This structure is a KB that could be created with a text editor in DOS and used in EXSYS (1994). CHOICE (goals) not p RULE 1 IF not q THEN not r RULE 2 IF not r THEN not p What happens in EXSYS to prove this goal is that EXSYS searches the knowledge base in the IF and THEN parts of the rules to find a rule that has information regarding the goal. The search is top down so Rule 1 is found first (EXSYS, 1994). EXSYS is looking for rules that have something to do with the choices (goals). The conclusion of Rule 1 will lead us to Rule 2 whose conclusion is our goal. The IF portion of Rule 1 (IF NOT Q) becomes a subgoal because since EXSYS does not know if NOT Q is true, the search continues top down. EXSYS finds Rule2 and is now trying to prove three goals NOT P (our original goal), NOT Q (a subgoal) and NOT R (also a subgoal). EXSYS makes Rule 2 NOT R into a subgoal (EXSYS, 1994). Since EXSYS checks facts after every subgoal, it now has something that it knows is true because the user answers Y (yes) to the question is NOT Q true? Since NOT Q is true, then EXSYS concludes by modus ponens that NOT P is true because NOT Q concludes NOT R is true and NOT R concludes NOT P. NOT Q is now added to working memory in an area of the program called facts. Facts will be displayed at the end of the run when EXSYS draws its conclusions (EXSYS, 1994). Since NOT R is in working memory because it was concluded after the user said NOT Q was true, it proves our goal of NOT P. NOT R however is not added to facts so it is not displayed in the final display of the results of the consultation. EXSYS asks the user if NOT R is true only on a forward chain. For EXSYS to do a forward chain, a special command must be given when the program is executed (See Appendix D). If the user does not answer Y (yes) to the question asked by EXSYS, is NOT R true, then the inference process will stop since there are only two rules but NOT R will be added to facts. The continuation of the inference process is dependent on the user answering the questions. Knowledge bases can be built to account for incorrect answers but are far more complex and are beyond the scope of this example. Also, beyond the scope are discussions on induction and abduction since they are considered not to be valid form of reasoning (Murphy, 1993a). Once the user answers true to NOT Q, then EXSYS follows the chain backwards to NOT P through NOT R. If more choices (goals) are added (NOT R and NOT Q) on a backward chain, the results will display values for NOT R and NOT P but not NOT Q. NOT Q will be listed as a fact since it is not a choice in a Then part of a rule. This example is true of knowledge-based systems. A knowledge base can be any collection of goals (choices) and rules (parameters) that can allow additional facts to be deduced (modus ponens goal-directed search) given some initial facts that the user must answer Y (yes) to when the knowledge-based system is running. Hayes-Roth et al. (1983) mention several rule-based systems that give directions for chemical spills, diagnose diseases and look for oil. These systems can have thousands of rules and even model human thinking processes and are referred to as learning theories (SOAR-Newell, 1990; ACT*-Anderson, 1986). Metacognition Metacognition is defined as a higher-level, superordinate executive process control process that governs the action of simpler cognitive skills and learning (Brown, 1978; Meichenbaum, Burland, Gruson & Cameron, 1985; Psotka, 1985; Schrader, 1988; Flavell, 1979; Shuell, 1986). Schrader (1988) says that the reason the literature defines metacognition as it does is because it is made up of studies of children who function at a lower level of metacognition called non-reflective. Metacognition is a developmental variable and is associated with age and educational level (Schrader & Thorpe, 1988). Schrader goes on to say that metacognition is not only thinking about thought processes such as how we know and what we know, it is also a diachronic process that operates, through reflection, on how we think about our thinking, having influence on development. Metacognition can be seen in activities described by Shuell (1986) as " . . . regulation and orchestration of the various activities that must be carried out in order for learning to be successful (planning, predicting what information is likely to be encountered, guessing, monitoring the learning process)." (p. 416). Clements and Gullo (1984) found planning strategies could be taught with Logo when they were specifically taught as part of the lesson. Since planning is metacognitive then we can say metacognition can be taught. Shuell describes metacognition, as a process concerned with what one does and does not know about the material being learned and the processes involved in learning it. These specific process include: (a) tasks - knowledge about the way in which the nature of the tasks influences performance on the tasks; (b) self - knowledge about one's own skills, strengths, and weaknesses; (c) strategies - knowledge regarding the differential value of alternative strategies for enhancing performance; and (d) interactions - knowledge of ways in which the preceding types of knowledge interact with one another to influence the outcome of some cognitive performance. (Shuell, 1986, p. 416) In examining the differences between Clements and Gullo (1984) and Cathcart's (1990) research, the method of teaching programming skills emerges as the main difference between the results in awareness of comprehension failure and reflectivity/impulsivity tests done by the two researchers. Since metacognition works through reflection (Schrader, 1988), improvements in reflectivity can show improvements in metacognition (Swan, 1988). Because Clements and Gullo (1984) had students first plan what they would program and draw the pictures with black markers on a piece of paper before they programmed their graphics the students scored higher on the Markham (1977) metacognitive tasks (Swan, 1988). Wideman et al. (1988) say that KB building promotes the " . . . acquisition, practice, and extension of cognitive and metacognitive skills" (p. 93). The charts drawn in Wideman et al. (1988) research are similar to the drawings in the Clements and Gullo (1984) studies and semantic nets1 accounting for the improvement in metacognition. No test of metacognition is used to measure the acquisition of metacognitive skills. Although Swan, (1988) equates reflectivity to metacognition, Cathcart did not do the Markham (1977) tasks for metacognition. These tasks involve students being presented with an incomprehensible problem. The point where they realized they do not know how to solve the problem is their score based on the questions number (1-11). In examining most of the results of metacognition testing, the data is between groups and not within groups after some treatment. It is not necessary to do pre and post tests to show significance, however what emerges from these researchers efforts is that metacognition can be taught and can be taught while using planning strategies. Thorpe (1988) found a weak link between attribution and metacognition suggesting metacognition was a factor making it possible for students to make attributions as they got older. Thorpe goes on to say, "It is proposed that the child's metacognitive knowledge provides the informational standpoint from which he views his learning. It will largely determine both what he perceives (learning outcome) and how it is explained (attribution)." (Thorpe, 1988 p. 77). Thorpe's final conclusions were that a developmental trend was evident with attribution to strategy increasing with age and metacognition. Thorpe also suggests classroom observation and perhaps ethnographic studies of individual children would present the information on the relationship between metacognition and attributional behavior in the complex classroom environment. Wideman et al.'s (1988) approach is qualitative and according to Bogdan et al. (1993), is similar to ethnographic procedures. Both Scrader and Thorpe find that metacognition is not unitary and call for a redefinition by Brown and Flavell. Bornholt, Crawford, and Summers, (1993) found that metacognition was not unitary even though they give a redefinition. Their redefinition of metacognition includes students talking aloud during tasks. Bornholt et al. (1993) base their redefinition on " . . . Idol, Jones and Mayer (1991) suggesting metacognition is shared behavior (thinking aloud), and it includes the learner's beliefs, judgements, attitudes, motivation, and self-concept. " (p. 4). Although this has implications for researchers collecting think aloud protocols, this observation will study students in shared behavior activities. Some new hypotheses regarding metacognition may emerge from this configuration. Hayes-Roth et al. (1983) and Durkin (1994) suggest metacognition can be put into the KB as metarules. Metarules give instructions about other rules in the KB. Control Groups and Refinement. Lippert (1988) in her study of NKE found knowledge refinement and the problems in using control groups with unlearned knowledge insurmountable. Lippert concludes using even a minimum control group (students taking only the pretest and postest) would have been unlikely to yield an informative comparison. Knowledge refinement can be identified by the following measures: 1) a decline in errors, 2) a decline in time on task, 3) a change in problem and volume representation (e.g. shift from two dimensional to three dimensional thinking 4) curtailment of reasoning (e.g. issues of efficiency and conciseness 5) conceptual differentiation and integration, 6) development of conditional knowledge, and a 7) refinement of procedural knowledge (e.g. pattern recognition and action-sequencing). (Lippert, 1988, p. 64) In examination of pre- and post-test data in pilot study one done by Gow (1990) in preparation for Gow, 1993, it was found students took less time to answer questions and had fewer errors on the post test after NKE activities, a criteria in refinement and evidence of compilation (Anderson, 1987). However, in the Wideman et al. (1988) study no pre or post tests were given. This makes the Wideman et al. study similar to the Lippert study where only qualitative data were collected. knowledge base and Cognitive Model Building Anderson's (1986) ACT production system is a cognitive model or learning theory that serves as an inspiration for building cognitive models (Gow, 1990) although SOAR (Newell, 1990) may be more complete. Because Knovac (1990) and Maxcy (1990) found concept mapping when used with other educational strategies leads to superior achievement, the method of building knowledge bases in pilot study one (Gow, 1990) in preparation for Gow, (1993) was through concept mapping, similar to bridging (Feuerstein's Instructional Enrichment Program in Maxcy, 1990) and clustering. Clustering is a method that engages the students in expressing their non-canonical knowledge regarding an object in a domain in a graphical representation or frame and mapping the canonical onto the non-canonical (Bussard 1991; Knovac, 1990; Maxcy, 1990). This process is analogical and can be the formalism to promote acquisition of problem-solving skills (Swan, 1989). Wideman et al. (1988) present production making from the decision tree. In Wideman et al.'s study he first made a decision tree and then made productions from the tree. EXSYS (1994) uses the same procedures to make productions in manual examples. Mental Imagery Cunniff and Taylor (1987) found that icons2 once learned brought back the meaning of programming constructs while their corresponding textual labels were lost. This suggests that if pictorial representations are used in the KB or in tree drawing when planning, it may help in the long term understanding of constructed knowledge. EXSYS can run external programs and depict graphics. In Gow, (1993), the Logo expert system shell had the same capability. However the original study done by Wideman et al. (1988) did not use any graphics in the KB. The underlying phenomenon of iconic recollection is reported by Finke (1989), in Pavio's dual coding hypothesis because it helps to explain why pictures, as a rule, are much easier to remember than words. Finke also points out some studies showing that pictures are often better recalled over time; an effect known as "hypermnesia," whereas words tend to be forgotten. In the development of Guidon-Watch Richer and William (1985) felt Guidon-Watch made the abstract diagnostic procedure used in NEOMYCIN more concrete and visible, while Moore, Nawrock, and Simutis (1979) found that graphics had an effect on performance in Computer Assisted Instruction. Clements and Gullo (1984) suggested it might be argued that the Logo treatment affected some errors directly through the development of visual discrimination ability in the context of graphic programming. With mental imagery, student's problem-solving abilities can also be improved because Finke (1989) and Frederiksen (1984) found that imagery can contribute to efficient problem-solving by mentally simulating physical events though it seems this must be done with metacognitive control (Guernon, 1989). The graphical representation is a decision tree and may have an effect on mental imagery promoting problem-solving. Programming Experience. Lippert (1988) regarding programming experience wrote, "Peter for example had extensive experience in computer programming in Assembly language and attributed his speed and success at constructing the logical inferences to this programming skill." (p. 206). Students in pilot study one (Gow, 1990), despite limited programming experience, remarked how similar NKE is to programming while Hayes-Roth et al. (1983) say their maxims for building ES's are similar to well-known guidelines for building other types of software. Swan (1989), Harel (1988), and DeCorte (1993) note that a review of the Logo/problem-solving literature presents a mixed picture of the usefulness of the language for the teaching and learning of problem-solving. Littlefield, Delclos, Bransford, Keith and Franks, (1989) present a reason for this when they state that data from their study suggest that previous reports concluding that Logo does not produce any generalizable learning may be premature because of their failure to consider the effect of mastery on the probability of transfer. The relevance of the Logo literature on programming has to do with its effect on the results programming ability has on knowledge base building. If programming, as Lippert says, makes it easier for students to build KB's but does not improve on their learning then it may not be a factor in emerging hypotheses. Some students might produce a KB faster because of the programming ability and not a better KB. Most students at middle school level have had some exposure to Logo. Since the Logo literature is relevant in the area of programming and its effect on the student's ability to build a KB important to this study, it has been included here. Mundy-Castle, Wilson, David, Sibanda, and Sibanda (1989) suggest programming in Logo appears to improve cognitive growth. However, when a control group was used, the results showed no differences between CAI and Logo groups (Wilson, Mundy-Castle, Sibanda, and Lavelle, 1990). Clemments, Battista, Nastasi, (1990) found there were cognitive improvements using Logo. It is important to note that Logo is a language (Bisaillon, 1989) and CAI a program, something many Logo critics fail to mention (Green, 1985). Clements and Gullo (1984) found there was no evidence that 12 weeks of programming experience affects cognitive development compared to 12 weeks of CAI experience, although they concluded that children in the Logo group may have increased their ability to produce original ideas and to produce creative ideas as compared to a normative group because the Logo programming eased divergent thinking within a figural context, as did Cathcart (1990). Swan (1989) noticed because programming sharpens a metacognitive focus, it can be an ideal medium for the teaching and learning of general problem-solving skills when such skills are explicitly taught and practiced. Harel (1988) found that a correlation exists between a child's level of understanding and involvement in Logo programming and his/her ability to understand and use different representational systems in fractions. CAI Research Since CAI has proven to be of limited value in improving problem-solving abilities with rational-numbers ( Clements & Nastasi, 1988; Harel, 1988; Battista, Clements & Nastasi, 1990; and Gow, 1990) and according to Seymour Papert (1990) a symptom of a way of thinking that puts emphasis on the smallest side of education and wastes the power of a computer, some other method needs to be found to help students improve their problem-solving abilities with fractions. Caftori (1994) found that children do not use CAI as intended by its designers and teachers did not read the documentation about the programs limiting their usefullness in directing educational activities. According to Means et al. (1995) few schools have teachers interested in technology and willing to use CAI to teach skills. One year a student has a teacher who uses CAI and students gain some proficiency in using a computer to learn, but the following year the teacher is not interested in using technology to teach the same skills on the next level. Students get frustrated because skills they learn are not used with other teachers or from year to year and because of this, develop few skills for learning on a computer. Students see the computer as a torture machine because if they don't catch on one year they won't have time to improve and be better at it the next time they use one. CAI research by itself legitimized CAI from the standpoint, if it didn't hurt then it couldn't be any worse than what has been used for thousands of years despite the difficulty in defining the methods (Manion 1985). Bozeman (1988) mentions a meta analysis by Kulik (1983), Bangert and Williams where computers reduce the time required for learning. In CAI there are studies like Pagliaro (1983) or Grandey (1970) making direct comparisons of traditional and Computer Assisted Instruction. In ICAI, there are studies that trace the history of CAI and ICAI (Dede, 1985) weaknesses of ICAI (Roberts, 1983), the adaptive nature of ICAI (Christensen, 1988), discuses formative and summative evaluation (O'Neil and Baker, 1987), and 14 year olds learning algebra (Sleeman, 1984) but none that makes a direct comparison between CAI and ICAI. Public domain CAI is available for schools at a low cost and therefore is the software of choice as opposed to knowledge based systems. A case in point is a recent development by the Rand Corporation is ESSCOTS (Educational Support Systems/Commercial off the Shelf) (Mcarthur,D., Lewis, M. W. & Bishay, M, 1995). This initiative takes commercial software off the shelf and puts it into a generic shell that will allow what is normally used in industry to be used in a classroom. This is a uncommon approach to CAI, allowing through programming languages, creation of menus for children to use. Advantages of this software is it already exists and does not have to be built from scratch. Sophisticated software can be adapted for educational use at a reasonable cost. Despite some success in an unscientific sample, the effort has produced no evaluation to measure what students learn from using these tools. The work is highly experimental and has not moved out of the lab and into a large classroom. Other concerns relate to the dangers of overwhelming the student with possible issues and patterns to pursue because real search spaces are large ones (Mcarthur et al., 1995). Students using ESSCOTS have two spaces to search, the software itself and the subject they are studying doubling the complexity. Manion (1985) says our new perspective would view the microcomputer and CAI primarily as learning tools that help students learn more effectively, efficiently, and easier than without them. Because of the increase of low cost CAI and the increased perception that computers are the 'future,' more and more schools are staking their educational survivability on the proliferation of microcomputers and the use of CAI. As more and more teachers are learning how to use computers, brought about by availability, an interest in ES for teaching has grown (Baur, 1987). A case in point is the statement by Jancura and Overby (1988) suggesting these technologies represent the application of artificial intelligence and ES to education and training methodologies. When the public demand for instruction on computers began, the public schools followed suit and began offering courses in microcomputers. Because public schools follow trends, computer instruction is the least certified area of the public schools in terms of teachers having credentials to teach computers according to a PESCO Union MIS survey (1987). Dede (1985) says after more than a quarter of a century of research and development, the field of artificial intelligence remains something of a mystery. Discussions about the educational uses of AI, in particular, have been afflicted with vague statements about the technology's potential, and with the hype that had once accompanied the introduction of educational television and the first wave of computer-assisted instruction. One possible explanation for this state of affairs is that so few educational applications of AI have moved beyond the research laboratory. What makes this project so interesting is that inexpensive knowledge based software can be used to do NKE to examine its effects on learning and the metacognitive processing of middle school students and apply AI to improve problem-solving abilities in fractions. White (1987) states: Improvements in computer hardware coupled with the development of micro-computer based knowledge system authoring tools have made the use of these programs appear much more feasible today. (p. 57) Graphics displays with a corresponding increase in memory size and speed, are a reason for the increase of computer power needed for other applications. In the development of Guidon-Watch Richer and William (1985) state Guidon-Watch makes the abstract diagnostic procedure used in NEOMYCIN more concrete and visible. One of those applications is artificial intelligence. It is fully expected that NKE will help students to improve their metacognitive processing and therefore their problem-solving abilities in fractions scoring better on post tests (Harel, 1988). After reviewing almost 300 abstracts, it was found research on ICAI or ITS's was not as complete as research on CAI, and ICAI or ITS's are a lot different from ES. This is a determining factor for examining an ES replication for studying students learning in fractions instead of ICAI or ITS's because of lack of research and availability of the program. White (1987) states although there have been extensive developments in industry, medicine, and the Department of Defense, there appears to be a comparative dearth of research efforts applying knowledge systems to the problems in public education. Doing the Owsten et al. replication and examining the effects of NKE on middle school students is an important undertaking both as White says to sharpen thinking and to be professionally enlightened and advance the body of knowledge regarding learning with ES in middle school environments and widening of different types of educational software. Fractions Idit Harel (1988) trained sixth grade students to design software with Logo that taught fifth grade students about fractions. Harel's work was done in an inner city school in Boston and used students from a lower Socio Economic Status (SES). Although Harel did a multi-comparative study with several different types of Logo groups her writing focuses on a student named Debbie in the software design group using qualitative analysis. By making Debbie the focus of her observations, Harel was better able to document changes in student behavior. Students spent four months on the Instructional Software Design Project (ISDP) with no additional learning about fractions. Debbie changed from an introverted, socially outcast student to a socially prominent outgoing individual in the Logo culture that developed in the ISDP. Harel compared pre- and post-curriculum based tests, Logo pencil and paper tests, and Logo programming tests with good success. Changes in fraction understanding was found by interviews with the students similar to Wideman et al. (1988). According to Groff (1996) traditional methods of teaching fractions should be abandoned. Kafai, (1993) and Selisky (1994) found students used strategies rarely found in books and advocated project oriented learning for fractions teaching. Although Groff advocates the postponing of fraction teaching as does Hoover, (1991), he points out that "...that, to be justifiable, the teaching of fractions must be based on experimental research findings, rather than on tradition and arbitrary opinions." (Groff, P. 1996, P. 178). In Harel's study (1988), one question that developed regarding the results was the length of time students spent on the project working on fractions and the comparison to another group that was not involved in the ISDP project. M. W. Lewis of the Rand Corporation (personal communication, April 7, 1992) felt there is a design flaw in Harel's study about the amount of time a student spent in the design group. Since students spent more time working on fractions, despite the software for learning approach, it is no wonder they learned more about fractions. When asked about this problem, Harel acknowledges there is some difficulty in creating identical groups for comparison (personal communication January 5, 1992) as does Lippert (personal communication, January 5, 1991). Harel (1988) and Groff (1996) documented the long-standing problems children have in learning fractions. Harel found that by project-oriented activities and design considerations (Perkins, 1986) students scored better on curriculum-based and rational-number tests in the domain of fractions. Since Harel (1988) and Groff (1996) found problems with fractions persist and several national assessments of children's mathematical achievements have found that children's learning and performance with fraction-ordering and computation were low and done with little understanding, new methods of teaching fractions needed to be discovered. Since recent research in learning with middle school children and learning while building knowledge bases for expert systems has proven to be successful, a replication of Wideman et al.'s (1988) using expert systems with case study analysis and different domain (fractions) might find new hypothesis. According to Groff (1996) and Harel " . . . the difficulty of fractions for elementary school children have been well documented (Carpenter et al., 1976; Behr et al., 1983; Peck & Jencks, 1981; Post, 1981; Poste et al., 1985; Kaput, 1987; Janvier, 1987; Tierney, 1987)." (Harel, 1988, p. 48). Fractions continues in the curriculum beyond elementary school on into middle school and is an important subject even in the eighth grade. The persistance of the teaching of fractions is due probably to the assessment tools used to measure childrens achievement. The SAT test has a large section of fractions as does the NCTM standards (Groff, 1996). Yet despite the continual teaching and learning of fractions beyond elementary school, many National assessments of children's mathematical achievements have found that children's learning and performance with fraction-ordering and computation were low and done with little understanding (Harel, 1988 and Groff, 1996). One thing that will be looked at in the interviews in the Wideman replication project will be children's learning and understanding of the system of representations of fractions and their ability to translate different representations of fractions. According to Harel (1988), the rational-number system is a sophisticated system familiar to elementary and middle school children. There are many rational-number sub-constructs perceived as islands of rational-number knowledge in the child's mind. Examples of these sub-constructs are: ratio, number-line, part-whole, operators, rate, decimals, and percentages. Harel (1988) says Lesh, Landau, and Hamilton and Behr, Lesh, Post and Silver found a child's rational-number conceptual model to contain: 1. Within sub-construct networks (or within-concept networks): when the child considers the sub-constructs of percentages and decimals. For example, he can compare their sizes and manipulate, translate, or operate on these quantities. 2. Between sub-construct systems (or between-concept systems): when the child considers the sub-constructs of percentages and decimals for example, he can compare their sizes and manipulate, translate, or operate on these quantities. Being able to see the whole rational-number conceptual system as the super-structure that includes several sub-constructs and the interrelations between these sub-constructs is being able to understand it. The between-concept systems have three components: a) different rational-number constructs (e.g., part-whole fractions, rationals, decimals, etc.) and their networks; b) links between those networks (i.e., the "sameness" or "distinctness" of rational-number sub-constructs); and c) operations that make it possible to transform a given rational-number into different forms. Harel (1988) says according to this theory, the relationship between the two systems (between-concepts and within-concepts) are reciprocal: the between-concept systems derive some of their meaning from within-concept networks and vice versa. For example, the translation of a simple fraction (1/2) into a percentage (50%) gives the fraction a meaning that includes the idea of proportion (one-half is like fifty to a hundred). Topology limits and simplifies the relationship a child perceives when thinking about a rational-number concept. When thinking about a number line, the child uses concepts of "betweenness" or "distance" in a subset of numbers. Children using a "ratio" will not use "betweenness" but think of the number in terms of rational-numbers. When children consider concepts on a rational-number "island," they will be sensitive of the properties of the rational-number "island." These within-sub-construct networks are shown to be intuitive, whereas some children might be more familiar with another conceptual network (e.g., part-whole) and less comfortable about another (decimals). Most children have a poorly organized and unevenly formalized between-sub-construct system associated with rational-numbers. The whole rational-number structure and its parts derive some meaning from each other. The role of the representational system and its parts also derive some meaning from each other. The role of the representational system is crucial in these relationships, since the networks and systems are manifested in different kinds of representations and translations among these representational systems. A child can represent 40% in a picture, or in numbers; he can also represent an operation such as 45*0.5=? by symbols, pictures, or beads (Harel, 1988). When we say a child understands the rational-number system we mean they can move from island to island without any delay or confusion and be able to differentiate between islands and understand characteristics and properties of each island (see Figure 1). Figure 1. Fraction islands. Each circle represents a different area of rational numbers. A student who knows the rational number system will move freely from island to island. A child who understands the rational-number system can illustrate a fraction using many different islands or representations from that island. This might mean the child will use circles, triangles, words, money, food or time to represent a concept. When a child can make these connections between islands this means they understand the rational-number system. A possible model of the translation process (moving from island to island) is in Figure 1. Harel says "I have reconstructed this diagram basing myself on the models of Lesh et al. (1983) and Behr et al. 1983." (Harel, 1988, p. 50). Harel says according to Lesh's research, manipulatives are just one way of presentation in the large representational system and that the other modes of representation (methods in Figure 1) also play an important role in different children's styles of thinking and no single manipulative aid was found to be the "best" for all children for all rational-number problems or for translating fractional representations. According to Harel (1988): Lesh et al. (1983) tested their subjects on different "translation modes" among representations of fractions. In their large-scale testing program (as part of their "RN PROJECT") they found that some translations were more difficult for children to process than others (they are listed below from the easiest for children, type number 1, to the most difficult, type number 9). (p. 51) The table Harel mentions here is listed is in Figure 2 below. Level 1. Translating word representation into word representation (for example, three-sixths equals six-twelveths: this was the easiest translation for the children in the RN Project conducted by Behr and Lesh (Harel, 1988). Level 2. Translating symbols into symbols (for example, 3/6= 6/12). Level 3. Translating symbols into words (for example, 3/6 equals six twelfths). Level 4. translating words into symbols (for example, three-sixths = 6/12). Level 5. Translating a picture into a picture, for example. Level 6. Translating words into a picture, for example. Level 7. Translating pictures into words, for example. Level 8. . Translating symbols into pictures, for example. Level 9. Translating pictures into symbols (the hardest for children of ages 9-14). Figure 2. Translation models. Each numbered level represents a different skill in the domain of fractions. Skills become increasingly more difficult as the level increases. Summary A replication of Wideman et al.'s (1988) work with case study analysis and different domain in math because of the preliminary results shown in Gow, (1990 & 1993 with seventh graders is feasible. Papert (1993) says his early work in the 1960's with Bolt, Beranek and Newman (BBN) in creating a new language for children (Logo) was tested on seventh graders. The reason for this was because of the Piagetian theory that children had reached the formal stage of reasoning near the seventh grade. Papert and the team felt they could descend the age range after they had developed techniques for teaching younger children and refined the language. A replication of Wideman et al.'s (1988) work with case study analysis and different domain is feasible and by adding more computers and a different domain, it is hoped the original hyppothsis will be proved and different hypotheses regarding NKE as an educational activity emerge. There may be some questions regarding new directions in emerging AI about rule-based systems, because according to Turkel and Papert (1990) emergent AI does not suggest that the computer be given rules to follow but tries to set up a system of independent elements within a computer from whose interactions intelligence is expected to emerge. However, because Turkle (1984) refers to a production-based system also as emerging it is unclear how emergent systems will replace production systems. As an example of emergent AI, Resnick (1991) simulated ant food gathering with high school students on the Connection Machine in *Logo at MIT. *Logo is uncommon because it has thousands of turtles that run independently because of the parallel processors in the Connection Machine. The connection machine is a massively parallel computer. With Resnick's *Logo language, the patterns of ants gathering food emerged from the programs that operated individual ants and environment areas called patches (Resnick, 1992) as the programs began to interact with each other. Resnick reports the opposition to emergent AI is strong and the tendency towards a centralized mindset prevalent even among experienced researchers. As an example, Resnick recounts an encounter with Marvin Minsky who thought that ant behavior would not emerge without being programmed. Because of a student's ability to build KB's about fractions with EXSYS and other ES, a merging of AI in ED is occuring, however constructionism is the direction for Ai-Ed researchers (Nic-aud, 1992). Overlooking NKE's development in AI will not be to the advantage of the emerging AI technologists because it may hold some promise for strengthening their own work as well as have implications for student modeling in ITS's (Sack, Soloway, & Weingrad, 1993). CHAPTER III Method And Procedures Introduction Something better has to be tried since the present methods of teaching fractions does not do the job. Expert systems provides a new approach from a direct knowledge, project-oriented perspective. Because a knowledge base has to be built that represents the knowledge the students are learning, they get an opportunity to work directly with the knowledge they must learn. Students thoughtfully study the concepts and try to organize them in a significant and rule bound way way so that they can be used in a purposeful manner. This intentionality makes the knowledge the object of study in a way that is reflexive and metacognitive as opposed to multimedia and CAI that are not being used as designed. In the old model students practiced exercises over and over even with computers and CAI. Most case studies of students learning with expert systems are at the University level and do not generally translate to middle school. In the new model, students create an object that must be used in an organized and significant way. The act of creating productions is focusing the student's thought on the knowledge itself, since a production is a unit of knowledge (Newell & Simon, 1972). In this systematic method of creating productions for the expert system, the student creates a product that embodies the knowledge they must learn. In the old model, projects are sometimes tried to use knowledge taught to create an object. This object is the use of the knowledge that the student must acquire. In the new model, the object is the knowledge itself, and provides a better understanding of how a student can acquire and put into use the knowledge they must learn. This way of learning is a more direct route and eliminates the design flaws in multimedia and CAI that allows children and teachers to use these programs in a way not designed. This understanding includes a metacognitive aspect since the student takes a look at what is going on in their mind when they see their KB being run. Anderson (1986) in his widely quoted ACT* theory states mental activity consists of production rules being activated. Design A qualitative study will be done with a sixth grade public middle school alternative education class of twenty students in Dade County of Miami, Florida. The students are 99% white, living in a middle class neighborhood being of 95% Hispanic origin. The middle school where the project will occur is the second largest middle school in Dade County, Florida, with 1,910 students in attendence. Students will be assigned randomly based on SAT scores to groups of two or three with a low, medium, and high SAT score in it. Where the group will contain only three students, the group will have a higher stanine, middle stanine and a lower stanine. A case study approach will be applied to a focus group chose at random within the classroom group, which will be studied in the context of the classroom and teaching the whole class (including the focus group) how to add fractions with unlike denominators using novice knowledge engineering (NKE) techniques. A video camera will be trained on the whole classroom and another on the focus group, as in the Harel (1988) study where only two video cameras were used. Audio tape will be used by the investigator while he interviews students during classroom activities. In the Harel (1988) study one camera was focused on the whole classroom and the other on Debbie as she did her work. Students will be interviewed before and after the project on their knowledge of fractions and computers. Piagetian interviews will be conducted during the project based on Piaget's et al. 1971 (see Appendix G). The classroom teacher will be trained in knowledge base creation and the steps required to teach students how to add fractions with unlike denominators using NKE techniques. Students will be told they are designing their kb's for 5ifth graders to help them with fractions similar to the Harel, 1988 study. Procedures Session One The classroom teacher will be introduced and her role and the reasearchers role will be explained to the students. Permission slips will already have been obtained from the students before the start of the project. Attitudes towards computers interviews (Appendix F) with questions like "Do you like computers? Do you know how to use a computer?" will be conducted. Initial in-depth interviews with the focus group regarding students knowledge of fractions will also be conducted. All students will be interviewed based on the Harel (1988) ten questions (See Appendix E) both at the beginning of the study and the end. Sample questions are: Question 1: Level 1. Translate three-sixths into an equivalent fraction with twelveths. Question 2: Level 2. Translate 3/6 in a number with twelveths. Question 3: Level 3. Translate 3/6 into words for twelveths. Question 4: Level 4. Translate three-sixths into numbers with twelveths. Question 5: Level 5. Translate a picture into a picture, for example. Piagetian questions (Appendix G) will be asked of all students while they are working designed to examine their knowledge of fractions. Sample questions may include: 1) What are you trying to do and what have you done? 2) How is the knowledge base different from real thinking? 3) How do your KB's show what you know about fractions? The Markham (1977) and Thorpe (1988) tests for metacognition will be performed in a group context to measure existing metacognition levels. Pretests for fraction problem solving ability will be done to measure any existing ability in adding fractions with unlike denominators. Session Two The classroom teacher will introduce students to a unit of adding fractions with unlike denominators with a discussion of fractions with unlike denominators in general. Regular middle school math books will be used. This will be a traditional lesson with expository teaching. Students will learn different types of fractions with unlike denominators and their addition . The instructor will work several examples on the board for the students to illustrate the problems with possible solutions. The instructor will show students how to create a decision tree to show the steps for adding fractions with unlike denominators. After the teacher is sure the students understand, they will then be asked to do some sample exercises in the book. While the teacher walks around the room to see how they are doing, the researcher will be carring a notepad taking notes on student interaction. The instructor will then focus in on the steps for identifying fractions with unlike denominators, finding common denominators and changing these fractions to fractions with like denominators and how that fits onto the decision tree. The introduction to addition of fractions with unlike denominators is not designed to be comprehensive, but enough for the students to get the notion that fractions with unlike denominators exist and that criteria or keys have been developed to identify these fractions and change them to fractions with like denominators and add them. Session Three The classroom teacher will show the students what an expert system does using a prototype system that classifies shapes like the one in Wideman et al. (1988). The EXSYS ES will be booted up in the computer and projected on the screen with a data view so students can see how the expert system works. Students will be treated to a demonstration of an existing KB geometric shape identification and will be shown what happens if something in the KB is changed. Students will be given the opportunity to interact with the teacher and make suggestions to change the program to see what happens when specific rules and choices are edited. Session Four The classroom teacher will take the students through the building of simple shape KB using EXSYS and identifying all the steps and decisions necessary for identifying different types of plane geometric shapes. For example, students will be shown how to use EXSYS with the selection of choices and parameters. Choices for identifying a circle would be the a) object is a circle b) the object might be a circle or c) the object is not a circle. The if parts of the rule would have a) the object is round b) the object contains 360 degrees c) the object contains 180 degrees. The then parts of the rules would contain a) the object might be a circle b) the object is a circle c) The object might is not a circle. The system determines if the shape is a square, triangle, or circle based on the properties of each figure. Session Five Next the class will break into their preassigned groups and each group will enter the simple shapes knowledge base, given to them by the classroom teacher, into the computer and debug it. Each group member will be assigned a role for their part in the KB building (KBB) activity. These roles will be assigned randomly but consist of a typist, a researcher, and a writer. The typist will sit at the computer and enter data, the researcher will get written materials for the group, and the writer will do the dittos and draw the trees necessary to make the KB. The roles can be changed for variety but must be maintained as well as group membership throughout the activity so that observations on the focus group can be maintained. For this knowledge base (KB), and for all later ones, students will use worksheets (which are actually screen prints of the system editor) to specify their choices and rules (see Appendix B). In session six, students will work in their groups to develop an expert system similar to the final project. For the similar project, a problem in adding fractions with unlike denominators was chosen (1/2 + 1/3). Session Six Students will then be assigned a project similar to the main project add 1/2 + 1/3. The object of this project is to develop an expert system for the addition of fractions with unlike denominators. This system will include modules for identifying fractions with unlike denominators, identifying methods of changing fractions to fractions with like denominators and methods for adding these changed fractions. Each group will work on the same project so they can compare KB's and discuss their work as it develops. Students will be asked to first develop a tree (see Figure 3) showing the steps a student might go through to identify the type of denominator, find a common denominator, and then add the fractions together. Once the tree is created, students will then define parameter names and choices and then write rules for their expert systems. Once this is done, students will go to the computers for building and debuging their systems. This may take several sessions. When the classroom teacher has determined that all goups have built their sample KB's with trees, then she will go to session seven. Figure 3. Fraction tree. Figure 3. Fraction tree. Each inverted V represents a decision point where the left branch is answered yes and the right branch is answered no. Session Seven Students will then be assigned their main project similar to the trial project add 1/2 + 1/3. The object of this project is to develop an expert system for the addition of fractions with unlike denominators. This system will include modules for identifying fractions with unlike denominators, identifying methods of changing fractions to fractions with like denominators and/ methods for adding these changed fractions as in session six. Each group will be given a project based on one of two different types of fractional problems: common multiples (1/4+1/5), divisibility (1/2 + 1/4), lowest common multiple (1/4 + 1/8). For example, in the common multiples problem, students will be asked to design a KB that identifies this type of fraction and then identify procedures that change these fractions to fractions with common denominators and then identify procedures that add these fractions. Students will be asked to first develop a tree (see figure 3) showing the steps a student might go through to identify the type of denominator, find a common denominator and then add the fractions together. Once the tree is created (see Session Six), students will then define parameter names and choices and then write rules for their expert systems. Once this is done, students will go to the computers for building and debugging their systems. Session Eight After the groups have an opportunity to study their assignment, then the classroom teacher will illustrate to the class the steps involved in a similar assignment, adding 1/2 + 1/5. This will involve how to identify fractions with unlike denominators, procedures of changing fractions to fractions with like denominators, and methods for adding the fractions once they have been changed to common form. The groups development of adding fractions with unlike denominators will be undertaken in a guided-discovery milieu. Appropriate reference material will be made available to students. The classroom teacher will work with the focus group to mediate their discussion of strategy, ask questions, offer hints, while the researcher video tapes group activities. The classroom teacher will work with the rest of the classroom groups to monitor their work also. The researcher's role is largely to promote the students own discovery of adding fractions with unlike denominators procedures and use of problem-solving strategies in building their systems. The classroom teacher will steer students toward using those procedures that problem-solving research suggests can be usefully taught: The use of questions to gain information, self-monitoring of progress, attempting solutions to resolve ambiguities, using information in the problem space (mental representation of problem) to restructure the problem, verbalizing and discussing different approaches (Frederiksen, 1984). Session Nine When the student's assignments are done, they will be given the opportunity to use their own and other groups' systems to show adding different fractions with different denominators and to experiment with their features, such as their ability to show different ways of identifying unlike denominators and their rules and to show why a particular denominator is identified in a certain way. Students will also work on bug fixes to fine tune their systems. Session Ten Follow-up interviews will conducted on the case study group to determine if any changes have occurred in their thinking on computers or fractions. All students will fill out the questionaires presented in the appendices on changes in fraction thinking and computers. Data Collection Two major forms of data collection were used in the Wideman et al. (1988) study to develop a "thick description" of students' experiences and work processes as they engage in their expert system projects (Denzin, 1970; Harel, 1988 and Turkel, 1983). These two same forms of data collection will be used in the replication, however a third will be added: video tape. In this replication with case analysis and different domain of the Wideman et al. 1988 video tape will be invaluable in analyzing changes in student behavior. First, during and after each session, the researcher will take detailed notes of the classroom interactions and activities following the procedures outlined by Bogdan et al. 1992. Bogdan et al. recommend immediate typing of experiences for the day to prevent forgetting of material. Bogdan et al. suggest this is tedious but necessary and recommend a sticktuitiveness to conquer boredom. These procedures include typing up notes and organizing them by similarities on topics in paragraphs. After notes are typed, they will be organized based on similarity. If two observations have similar themes, then these similar themes (cooperation) will be stored in a folder marked cooperation. Any later observations labeled cooperation will be put into this folder. This is called the folder method and will be used in this experiment. The folder method consists of putting similar comments into similar folders. By sorting ideas and concepts by folder similarities between subjects can be found along with reappearing themes. If one folder is fatter than another it might be an indication that the theme of the folder is significant. Participant observer notes are to be included in the sorting and the researcher is encouraged to express their own weaknesses whenever possible to avoid prejudices and bias. Although Bogdan et al. do not give many procedures for using video tape, researches like Turkel and Harel do. Video tapes will be taken wherever possible. Video will be invaluable to the researcher's environment and will serve as a backup for behavior that might be missed. When the students are working in their groups, the researcher will spend the session observing them to delineate as clearly as possible the planning and problem-solving activities engaged in by the group members, both on and off the computer. Second, data will also be collected throughout by informal, individual and group open-ended interviews of both students and classroom teacher. Student interviews will be designed to elicit students' affective and cognitive perceptions of the various phases of expert system development, the content or domain used, the group processes engaged in, and the teacher's role in project development. Since the interviews with the students are open-ended, the questions will be limited to questions regarding the knowledge bases and their building. This is similar to how Piaget (1975) evoked information from his subjects. If the student wants to expound on his/her information since the interviews will be open-ended, they will be allowed to. The data from these two sources will serve as the bases for developing working hypotheses about the strengths and limitations of student expert system development in the classroom as an educational activity. Qualitative analytic techniques will be used to articulate a portrayal of the various strategies used in planning and testing system development, how these change as the project progresses, and whether different groups will use different processes. Data Presentation Data will be presented as charts. Piaget (1971) presented his data in simple charts. An example is presented below from Piaget's book on mental imagery in the child. The chart depicts the successes for imagination of the movement of a snail on a track. Piaget examined 70 subjects aged four to seven years old with a two-dimensional technique. Piaget was trying to prove the existence of kinetic images and how children of different ages dealt with them. The chart below (Figure 4) shows that as the age increases the child is better at reproducing the images. What Piaget (1971) was able to determine from the charts is that : . . . it is easier for the child to imagine the position and orientation of a moving body in relation to a fixed object than in terms of the movement - which once again shows the difficulty of kinetic images. ( p. 89) Figure 4. Piagetian chart showing the success for imagination of the movement of a snail. Each column represents a different age level. After the data is analyzed we will be able to determine if the research was successful or not. The Hypotheses and Measures of Success section will address the success criteria and what must emerge to declare the project a success. Hypotheses And Measures of Success The original hypothesis that emerged as a result of the Wideman et al. (1988) study was: Would the experience of creating a knowledge base have the potential to deepen knowledge of the content domain while promoting the development of more mature cognitive strategies. In the Wideman et al. (1988) study several observations resulted apart from the main hypothesis and are listed below: 1) Development of a simple, rule-based expert system can provide a valuable educational experience at a surprisingly early age, as long as the activity is properly structured and well supported. 2) Groups will be able to complete tasks of greater cognitive complexity than is typically demanded of them in the curricula for their age level, and to do so with a good degree of enthusiasm. 3) Develop a deeper understanding of interrelatedness of taxonomic strata and keys, and the procedures of rule-based classification. 4) Forced students to use rigorous and systematic thinking to succeed. 5) Project activities promote students acquisition, practice, and extension of their cognitive and metacognitive skills. 6) After awareness of deficiencies in procedures is developed, students can rapidly assimilate the new strategies and use them as appropriate. 7) It may be likely that student development of knowledge bases could be undertaken in any subject domain that can be effectively presented by production rules, giving this activity a wide curricular scope. 8) Precautions: (a) group tasks are easily divisible, so that all members are challenged simultaneously and student time is not wasted. (b) Ample resource materials must be available so students do not have to compete for them. (c) The expert system shell should be introduced gradually so that students are comfortable with consulting it and developing simple prototypes before attempting their main tasks. Success criteria will be based on the extension or variation of the main hypothesis. The design of the experiment is to see if by varying the domain and increasing how many computers are used if this main hypothesis will not still emerge. If the original hypothesis does not emerge or the list of observations is not different or longer than the original list, then we might be able to say the experiment was a failure. Piaget said: One would be quite justified in considering a programme of psychological research a failure if it really said at the end what could have been assumed or deduced at the beginning. But the appearance of the unexpected in the final results might be taken as an indication of its success. ( Piaget , 1971, P. 351) Since it is impossible to assume what will happen in such an undertaking, it will be exciting to see what the unexpecteds will be. In the end, the unexpecteds will determine the success or failure of this research. Footnotes: 1 A semantic net as described in Winston (1992) is a chart like the one pictured with the labels on the nodes and subnodes (subgoals) of the tree. The top node is the root with the subnode being children and nodes without children leafs. Children are called parents if they have subnodes. Wideman et al. (1988) make no reference to artificial intelligence concepts as presented in Winston (1992). Winston (1992) page 20 has a more involved description of what a semantic net is. 2 Icon here is used as a symbol like picture that represents some concept. Like the picture of a cigarette with a circle around it and a line across the circle to represent no smoking. 3 Connectedness means relationship to real world application. Dade County Public Schools is trying to use the Dutch connected math curriculum to improve math scores. References Anderson, J. R. (1986). Cognitive modeling and intelligent tutoring. (ERIC Document Reproduction Service. No. ED 267 977). Anderson, J. R. (1987). Skill acquisition: Compilation of weak-method problem solutions. Psychological Review 94(3), 192-210. Bisaillon, D. A. (1989). Logo computer culture and children's development: The influence of socio-moral atmosphere. (Doctoral dissertation, Harvard University, 1989). Dissertation Abstracts International. Bigum, C. (1986a), Knowledge-based systems in education or artificial stupidity meets natural intelligence. Proceedings of the 1st Australian Artificial Intelligence Congress, Melbourne. 45-48. Bigum, C. J., Burton, D. & Gilding, T. (1988). Learners as novice knowledge engineers. Educational Research and Perspectives 14(1), 58-61. Educational Computing Research Group, School of Education, Deakin University, Victoria, Australia. Bogdan, R., & Biklen, S. K. (1992). Qualitative research for education: An Introduction to theory and methods. Boston: Allyn & Bacon. Bouwman, M. J. (1995). Student knowledge engineering in accounting: A case study in learning. Accounting Education 4(3), 231-248. Bozeman, W. C. (1988). Microcomputers in education: The second decade. T.H.E. Journal15(6), 82-86. Brown, A. L. (1978). Knowing when, where, and how to remember: A problem of metacognition. In R. Glaser (Ed.), Advances in Instructional Psychology. (pp. 100-110). New York: Halsted Press. Cathcart, G. W. (1990). Effects of logo instruction on cognitive style. Educational Technology Systems 6(12), 231-242. Christensen, D. L. (1988). Application of heuristic methods in the design of intelligent CAI. Proceedings Annual Meeting of the Association for Educational Communications and Technology. (pp. 25-28). New Orleans, LA. Clancy, W. J. (1992). Representations of knowing: In defense of cognitive apprenticeship. Journal of Artificial intelligence in Education 3, (2), 45-50. Clancy, W. J. (1993). Guidon-manage revisited: A socio-technical systems approach. Journal of Artificial intelligence in Education 4, (1). 5-34. Clements, D. H., & Gullo, D.F. (1984). Effects of computer programming on young children's cognition. Journal ofEducational Psychology, 76, 1051-1058. Clements, D. H. (1986, September 7). Testudinal testimony: Early studies on logo and mathematics. Logo Exchange 5, (1), 27-29. Clements, D. H. (1986a, November 30). Testudinal testimony: Research on logo and social development. Logo Exchange 7(1), 22-24. Clements, D. H., Nastasi, B. K. (1988). Social and cognitive interactions in educational computer environments. American Educational Research Journal 23, (1), 87-106. Clements, D. H., Battista, M. T., Nastasi, B. K. (1990).Social-Cognitive interactions, motivation, and cognitive growth in logo programming and cai problem-solving environments. Journal of Educational Psychology 2, 150-158. Cunniff, N. and Taylor, R. P. (1987). The graphical representation of programming: Research into alternatives for learners. [Summary] Proceedings National Educational Computing Conference, 168-173. Dede, Christopher J. (1985, May). Intelligent computer-assisted instruction: A review and assessment of ICAI research and its potential for education. Education Center. Cambridge, MA. 45-50. Durkin, J. (1994). Expert systems: Design and development. New York: MacMillan Publishing Company. Eyler, Charles Edward. (1989). The effects of metacognition on mathematical problem-solving. (Doctoral dissertation, Pennsylvania State University). Dissertation Abstracts International. UMI Dissertation Service. Ann Arbor, Michigan. EXSYS. (1994). EXSYS professional. EXSYS Inc. Albuquerque, New Mexico. Farber, N. G. (1990). Through the camera's lens: Video as a research tool. In I. Harel. (Ed.) Constructionist Learning: A 5th Anniversary Collection of Papers. Cambridge MA: MIT. Media Laboratory. (pp. 75-80). Fernandez, D. H. (Ed.). (1991). Addison wesley mathematics. New York: Addison-Wesley Publishing Company, Inc. Finke, R. A. (1989). Principles of mental imagery. Cambridge: MIT Press. Ford, R. C. and McAlister K. M. (1989). Tapping into expert systems. Personnel Administrator, 67-69. Frederiksen, N. (1984). Implications of cognitive theory for instruction in problem-solving. Review of Educational Research, 54, 363-407. Gow, J. M. (1985). CAI vs traditional instruction. Unpublished master's thesis, Ft. Lauderdale, FL, Nova Univeristy. Gow, J. M. (1990). Constructing knowledge bases with logo (CKB*) in seventh through ninth grade: Feasibility and theory. Unpublished Manuscript. Gow, J. M. (1992, April). Constructing cognitive models in Logo: A feasibility study with seventh graders. Poster session presented at East-West Conference On Emerging Technologies in Education. Moscow, Russia. Gow, J. M. (1992). Constructing cognitive models in Logo: A feasibility study with seventh graders. [Abstract]. Proceedings East-West Conference On Emerging Technologies in Education. Moscow, Russia: International Center for Scientific and Technical Information. Gow, J. M. (1993). Novice knowledge engineering with a Logo based expert system. Proceedings Fourth European Logo Conference Athens, Greece: University of Athens, Department of Informatics. (pp.269-273). Grandey, R. C. (1970). An investigation of the use of computer-aided-instruction in teaching students how to solve selected multistep general chemistry problems.Urbana, Illinois: University of Illinois, Urbana, Illinois. (ERIC Documentation Reproduction Service No. ED 056 515). Green, J. O. (1985, January). Logo under fire. Classroom Computer Learning. 28-59. Guernon, V. E. (1989). The effects of teaching heuristics within the context of a prescriptive metacognitive control system on problem-solving. (Doctoral dissertation, University of Connecticut). Dissertation Abstracts International. UMI Dissertation Service. Ann Arbor Michigan. Groff, P. (Jan, 1996). Is teaching fractions a waste of time? ERIC Document Reproduction Service. (No. ED 02759365).pp. 177-179. Harel, I. (1988). Software design for learning: Children's construction of meaning for fractions and Logo programming. (Doctoral Dissertation, Massachusetts Institute of Technology). Cambridge MA: MIT. Media Laboratory. 42-55. Harel, I. (1990). The silent observer and holistic note-taker: Using video for documenting a research project. In I. Harel (Ed.). Constructionist Learning: A 5th Anniversary Collection of Papers. (pp. 35-41), Cambridge MA: MIT Media Laboratory. Harel, I., & Papert, S. (1990). Software design as a learning environment project. In Harel, I. (Ed.) Constructionist Learning: A 5th Anniversary Collection of Papers. (pp. 85-92). Cambridge MA: MIT. Media Laboratory. Hayes-Roth, F., Waterman, D. A. & Lenat, D. B. (1983). Building expert systems. Reading Massachusetts. Addison-Wesley Publishing Company, Inc., Hoover, M. E. (1991) Fraction concept acquisition in middle grade learning-disabled and non-handicaped children (nonhandicapped). Dissertation Abstracts International. UMI Dissertation Service. Ann Arbor Michigan Jancura, E. G. & Overby, J. T. (Janurary, 1988). Expert systems training. The Woman CPA, January. Kafai, Y. B. (1993). Minds in play: Coomputer game design as a context for children's learning. Dissertation Abstracts International. UMI Dissertation Service. Ann Arbor Michigan Knox-Quinn, C. H. (November, 1988). A simple application and a powerful idea: Using expert system shells in the classroom. The Computing Teacher, 1, 12-15. Knox-Quinn, C. H. (1991). Student construction of expert systems: Impact on content area learning and problem-solving. (Doctoral Dissertation, University of Oregon) Dissertation Abstracts International. UMI Dissertation Service. Ann Arbor Michigan. Knox-Quinn, C. H. (1995). Student construction of expert systems in the classroom. Journal of Educational Computing Research 12(3), 243-262. Kulik, J. A. (September, 1983). Synthesis of research on computer-based instruction. Educational Leadership. 88-91. Lai, K. W. (1989). Acquiring expertise and cognitive skills in the process of constructing an expert system: A preliminary study. American Educational Research Association. San Fransisco, Ca. 98-101. Liebowitz, J. (1996, Februrary). KARMA: The knowledge acquisition reference multimedia aid. T.H.E. Journal. pp. 85-87. Lippert, R. C. (1987). Selecting expert system shells for classroom use: Some criteria and guidelines. Computers in Human Behavior, 3, 145-149. Lippert, R. C. (1988). Refinement of students' knowledge while developing expert systems. (Doctoral dissertation, University of Minnesota, 1988). Dissertation Abstracts International. UMI Dissertation Service. Ann Arbor Michigan. Lippert, R. C. (Winter, 1989). Expert systems: Tutors, tools, and tutees. Journal of Computer-Based Instruction 16, (1), 11-19. Lippert, R. C. (Spring, 1990). Teaching problem solving in mathematics and science with expert systems. Journal of Artificial Intelligence in Education, 1(3), 27-40. Littlefield, V. R., Delclos, V. R., Bransford, J. D., Keith, N. C. & Franks, J. J. (1989). Some prerequisites for teaching thinking: Methodological issues in the study of Logo programming. Cognition and Instruction, 6(4). 331-366. Lookatch, R. P. (1995, November). Technology for teaching? The research is flawed. Education Digest. pp. 4-8. Maddux, C. D. & Willis, J. W. (September, 1992). Integrated learning systems and their alternatives: Problems and cautions. Educational Technology, pp. 51-57. Manion, M. H. (January, 1985). CAI modes of delivery and interaction: New perspectives for expanding applications. Educational Technology. 34-39. Mark, M. A. & Greer, J. E. (1992). Methods for intelligent tutoring system evaluation. Proceedings East-West Conference On Emerging Technologies in Education. International Center for Scientific and Technical Information, Moscow, Russia. 89-95. Markham, E. M. (1977). Realizing that you do not understand: A preliminary investigation. Child Development, 48, 986-992. Maxcy, P. (1990). Combining a comprehensive thinking program, Feuerstein's instrumental enrichment, with computer thinking programs. International Journal of Cognitive Education and Mediated Learning, 1(1). 45-49. Mcarthur, D., Lewis, M.W. and Bishay, M. (1995) ESSCOTS for learning: Transforming commercial softwareinto powerful educational tools. Journal of Artificial Intelligence in Education, 6(1), 3-33. Means, B., Olson, K., & Singh, R. (1995, December) Transforming with technology: No "Silver Bullet." Phi Delta Kappan. (pp. 31-35). Meichenbaum, D., Burland, S., Gruson, L., & Cameron, R. (1985). Metacognitive assessment. In S. R. Yussen (Ed.), The Growth of Reflection in Children (pp. 3-30). Orlando, Florida: Academy Press, Inc. Minsky, M. (1985). A framework for representing knowledge. In R. J. Brachman & H. J. Levesque (Eds.), Readings in knowledge representation (pp. 245-262). Los Altos, California: Morgan Kaufman Publishers, Inc. Minsky, M. (1986). The society of mind. New York. Simon and Shuster. MIS Survey PESCO. (1986). Pesco. Dade County Public Schools. Schoolwide distribution. MIS Statistical Brief. (1986). State Department of Education . Tallahassee Florida. January. Moore, D. M., Yang, C S. (1995). Designing hypermidia systems for instruction. Journal of Educational Technology Systems, 24(1). 3-30 . Moore, M. V, Nawrocki, L. H., & Simutis, Z. M. (1979). The Instructional effectiveness of three levels of graphics displays for computer-assisted instruction. U.S. Army Research Institute for Behavioral and Social Sciences, Alexandria, Va. (ERIC Documentation Reproduction Service No. ED 178 057) Mundy-Castle, A. C., Wilson, D. J., Sibanda, P. S., Sibanda, J. S. (1989). Cognitive effects of Logo among Black and White Zimbabwean girls and boys. International Journal of Psychology, 24(5). 539-546. (Psych Info Dialog Abstracts. 1989. 11 Abstract No. 00747761) Murphy, D. D. (1993a). expert systems in the classroom for mathematics/science teachers. Ft. Lauderdale, FL: Nova Southeastern University, Graduate Education Module. Murphy, D. D. (1994). The little inference engine that could. Proceedings International Symposium On Mathematics/Science Education and Technology (pp. 118-122). San Diego, CA: AACE Publishers. Murphy, M. & Miller, A. (1996, March). Incentives pay off in technological literacy. Educational Leadership. pp. 54-56. Nagy, T., Nagy M., & Gault D. (1986). Building your first expert system. Culver, CA. Ashton-Tate. Newell, A., & Simon, H. (1972). Human problem-solving. Englewood Cliffs, N.J: Prentice Hall, Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press. Novak, J. D. (1990). Concept maps and vee diagrams: Two metacognitive tools to facilitate meaningful learning. Instructional Science. 19, 29-52. Dordrecht, Netherlands: Kluwer Academic, Publisher. O'Neil, H. F., & Baker, E L. (1987). Issues in intelligent computer-assisted instruction. Testing Study Group: The Impact of Advances in Artificial Intelligence on Test Development. California University, Los Angeles Center for the Study of Evaluation. Papert, S. (1980). Mindstorms. New York: Basic Books Inc. Papert, S. (1990). A critique of technocentrism in thinking about the school of the future. E & L Memo No. 2. Cambridge MA: MIT. Media Laboratory. Papert, S. (1990). Computer criticism vs. technocentric thinking. E & L Memo No. 1. Cambridge MA: MIT. Media Laboratory. Papert, S. (1993). The Children's Machine. New York: Basic Books Inc. Piaget, J. & Inhelder, I. (1971). Mental imagery in the child. New York: Basic Books Inc. Plomp, T.& Pelgrum, W. J. (1991). Introduction of computers in education: State of the art in eight countries. Computers in Education, 17(3). 249-258. Psotka, J. (1985). Reflections on computers and metacognition. (ERIC Document Reproduction Service No. ED 263 206). Resnick, M. (1991). Beyond the centralized mindset. Cambridge MA: MIT. Media Laboratory. Resnick, M. (1992). Beyond the centralized mindset: Explorations in massively-parallel microworlds. (Doctoral Dissertation, Massachusetts Institute of Technology). Cambridge MA: MIT. Media Laboratory. Reynolds, C. F. (1988). Introducing students to expert systems. Journal of Computer Assisted Learning, 4, 79-92. Richer, M. H. & William, J. (1985). Guidon-Watch: A graphics interface for viewing a knowledge-based system. (ERIC IR 012 0 022). Office of Naval Research, Arlington VA. Personnel and Training Research Program Office. Roberts, F. C. (December, 1983). Intelligent computer-assisted instruction: An explanation and overview. EducationalTechnology 23,(12), 7-11. Sack, W, Soloway, E., & Weingrad, P. (1993). Re: Writing cartesian student models. Journal of Artificial Intelligence in Education, 3(4). 100-105. Schrader, D. E. (1988). Exploring metacognition: A description of levels of metacognition and their relation to moral judgement. (Doct