ILS - Integrated Learning Systems Research

A Short Sample of Research Findings and Literature Review regarding Integrated Learning Systems:

• In a review of 100 studies of ILSes, Henry Jay Becker found that they “provided little evidence of ILS impact on student achievement. Where differences were found between the achievement of ILS users and comparable non-users, Becker concluded they were too small to have any educational significance.(Becker, 1992).

• Most scientific studies of ILSs have failed to demonstrate their impact on student outcomes (Van Dusen, Worthen, 1994).

• Skinneran operant conditioning learning theory provided the basis for ILS development (Mazyck, 2002). Instructional technologies must be optimized to support the cognitive requirements at hand, rather than only focusing on the behavioral model of learning theory. (Hooper et al., 1991)

• Independent studies of integrated learning system technologies have subsequently confirmed that learning discrete skills in isolation does little to support students in transferring knowledge to other domains of experience. This lack of transferability of skills from integrated learning system performance to other tasks is well-documented in the research literature (NCREL, 2002).

• Educators question the many evaluations of ILSs conducted by the ILS companies themselves. Most of the studies of ILSs were inconclusive and had many serious flaws (Mazyck, 2002).

• When ILSes are uses as adjunct activities or when teachers don’t chart progress and use results to complement curricular efforts, they have no positive effect on student achievement or attitudes (NCREL, 2002).

• Integrated learning systems have a long way to go before they can receive an unqualified endorsement (BECTa 1998, having spent $1.3 million on an extensive review of ILS vendors including Successmaker, Plato, and others).

• Although teachers generally like Successmaker as a system for enhancing basic skills, in most cases this was due to a general feeling about its positive or potential benefits rather than direct evidence.

• Students using Successmaker for more than a term had become less enthusiastic about the system.

• Successmaker won’t work if you use it for less than 3 times a week and just leave it to “get on with things on its own.” (Ian Hedley, Carter Community School, Successmaker Coordinator).

• ILS should abandon their “mindless adherence to the principle of individualized instruction.” (Becker, 1994)

• The Successmaker results relate to how they were publicized. Studies in reports from the vendor all show substantial positive effect sizes. Those from independent sources show modest or negligible effects (Bracey, 1991).

• Costs of ILSes are too great given its record of effectiveness (Levin, Destner, & Meister, 1986).

• Machines are not as effective as live teachers; ILS teaching is too mechanical, too impersonal; Pupils will find ILS instruction boring and repetitive, and thus can

lose their motivation to learn; ILS can teach routine skills but they cannot teacher higher order thinking skills or conceptual thinking (White, 1993).

• Computer based instruction is consistently more effective than traditional instruction, but the amount of improvement is low to moderate (Balajthy, 1989).

Selection

1. A team comprised of the participants—teachers, community members, administrators, students--should be in on the ILS selection (Hill, 1993).

2. Software and ILSes should be chosen by classroom teachers. Once selected, the system should be integrated into the local curriculum by the appropriate district curriculum staff. This role should not be usurped by computer teachers or the district computer coordinator or director (Wiberg, March 1992).

3. Schools looking at ILSes should not look at a company’s curriculum correlation charts, but sit down and look at the correlated lessons in-dept. Your curriculum has to be the baseline (Mageau, 1992).

Implementation & Staff

1. Hire a competent ILS systems manager. Most teachers need a support person who is the in-house ILS expert. These managers should be teachers who are knowledgeable about the curriculum and technical aspects of the system (Sherry, March 1992).

2. Plan site visits to talk about implementation, focus, funding, ongoing support and other relevant topics (Hill, 1993).

3. Comprehensive training for teachers and administrators on the ILS should begin PRIOR to its arrival and done by ILS consultants. In this way, the district can avoid the problem resulting from the lack of teacher support (Hill, 1993).

4. Get teachers to go to the lab with their students and circulate among them while they’re online. (Mageau, 1992).

5. The services of a SuccessMaker technician (lab manager) is desirable. This person is able to print out the reports and worksheets as requested by staff and able to provide necessary in-house support for teachers (Smith, 2002).

Professional Development

1. Give teachers time to preview lessons. Teachers should be given additional in-school time specifically for previewing ILS lessons. ILS curriculums are extensive; they can’t be taken home like other materials. Teachers should also have access to the ILS lab during after-school or weekend hours (Sherry, March 1992).

2. Most teachers never get the level of training necessary to use most of the ILS’s features (Sherry, 1992).

3. Teacher who is more involved with the ILS work of her students will likely produce the greatest learning gains

4. Teacher who had far and away the most successful effect size [substantial increase in test scores] knew the most about the system, knew the most about what the kids were doing in the lab, and went back to the classroom and made

decisions about what to do based on that information (Mageau, 1992).Use cooperative learning approaches to enhance the effectiveness of ILSs with diverse groups of students (Becker, 1992a; Brush, 1997a; Brush, 1997b; Brush, 1998; Hooper & Hannafin, 1991; Hooper, Temiyakam & Williams, 1993;Mevarech, 1994 as cited in Mazyck, 2002).

5. Cooperative learning paired with ILSs produces more willingness to take on difficult tasks and persist, long term retention of what is learned, higher level reasoning and metacognitive thought, creative thinking, transfer of learning, positive attitudes towards the task being completed and greater time on task. It also improves student achievement and attitudes in computer-based environments.

6. When pairs of students work cooperatively to complete exercises in an ILS, they OUTPERFORM their counterparts who use the system on an individual basis (NCREL, 2002).

7. Effort must be made to facilitate students’ transfer of knowledge to other domains of experience (NCREL, 1996). “Students may learn isolated skills and tools but they will still lack an understanding of how those various skills fit together to solve problems and complete tasks.”

8. Should be an integral part of the district’s educational strategy and guide professional development of teachers (Tingey, 2002).

Target Population

1. Students may be off-task if they are not supervised in an ILS implementation (this research referred specifically to Successmaker). (Underwood et al, 2002).

2. Some conclusions on computer-based instruction (CBI):

a. The lower the grade level or ability of the students, the more effective CBI is

b. CBI is consistently more effective than traditional instruction but the amount of improvement is low to moderate

c. Structured CBI, with emphasis on direct instruction, is more effect in producing achievement gains than unstructured CBI

3. The majority of computer at the secondary level was for introductions to the computer and computer programming. The secondary usage of computers was for low-level drill and practice with the typical student receiving such instruction only 15-20 minutes each week, hardly enough to gain proficiency in any skill being learned.

4. Low socio-economic status (SES) schools with predominately minority populations used their computers to administer drill and practice computer-based instruction.

5. Low SES schools with predominately white populations preferred to use computers with their higher-achieving students to teach programming and computer skills (Balajthy, 1989).

Reading Research

1. Computer materials are most effective when they allow the learner some degree of decision-making and control over the task.

2. Interactive materials are imperative.

3. When possible, learners should be able to self-check by looking back at previous work or calling up help features.

4. Preference should be given to programs that require the reader to make decisions about the progression or direction of the content, thus, promoting active involvement.

5. When viewing reading as a process of constructing meaning, the implications for computer-based instruction or the use of ILSs emerge as (Strickland, Feeley, and Webner, 1987):

a. Programs focusing solely on isolated skills, such as word recognition and word meanings are apt to be more effective when these skills are offered in context and in conjunction with one another. Isolating skills often fosters habits that are ineffective in authentic reading situations where understanding the message is essential.

b. CBI should be meaningful the reader, and it is unlikely that students will learn very much about reading from programs that defy all efforts to make sense of them.

References

Balajthy, E. (1989). Computers and Reading. New Jersey: Prentice Hall

Becker, H. J. (November, 1988). The impact of computers on children’s learning. Principal.

Bracey, G. (September, 1991). ILS research isn’t helpful. Electronic Learning 11 (1), p.16

Hill, M. (January, 1993). What’s new in ILSes? Electronic Learning, 12 (4), pg.12.

Hooper, S. & Hannafin, M. (1991) The effects of group composition on achievement, interaction, and learning efficiency during computer-based cooperative instruction. Educational Leadership.

King, J. B. (December, 2000). Use of integrated learning systems to enhance the proficiency of poor readers in middle school. Unpublished manuscript.

Mageau, T. (January, 1992). Integrating an ILS: Two teaching methods that work. Electronic Learning, 11 (4), pg.16

Mazyck, M. (March/April, 2002). Integrated Learning systems and students of color: Two decades of use in K-12 education. TechTrends 46(2), pg.33

O’Neal, S. (June, 2002). The impact of ESEA’s scientifically based research requirement on schools’ technology solutions. T. H. E. Journal, pg.64

Sherry, M. (March, 1992). Integrated Learning Systems: The ILS Mac Pack. Electronic Learning, 11 (6), pg.10; New York.

Smith, J. H. (1997). TCT ILS Project (Successmaker) Conclusions. Available online at http://atschool.eduweb.co.uk/gyhsmith/itnet/jhsmith1.htm on 7/1/2002.

Strickley, D. S., Feeley, J. T. & Webner, S. B. (1987). Using computers in the teaching of reading. New York, N.Y.: Teachers College Press

Tingey, B. & Thrall, A. (March/April, 2002). High stakes management. Multimedia Schools, 9(2), pg.S1-S7

Note: This article was written by the vice president of Successmaker.

Underwood, J., Cavendish, S., & Lawson, T. (2002) Integrated learning systems and educational outcomes. Available online at http://www.psychology.nottingham.ac.uk/research/credit/project/ils_evaluation/main.htm

L

Van Dusen, L. M. and Worthen, B. R. (1993). Factors that facilitate or impede implementation of Integrated Learning Systems. In G.D. Bailey (Ed.), Computer-based Integrated Learning Systems (p.115-119). New Jersey: Educational Technology publications, Inc.

White, M. (1993). Are ILSs good education? In G.D. Bailey (Ed.), Computer-based Integrated Learning Systems (p.115-119). New Jersey: Educational Technology publications, Inc.

Wiberg, E. A. (March, 1992). How to choose the right ILS. Electronic Learning, 11 (6), pg.10; New York.