Ranting & Raving on Instructional Design, Education & Technical Training

Silentium - Latin for "Shut Up & Pay Attention"

A recent study from researchers Louis Deslauriers, Ellen Schelew and Nobel Laureate Carl Wieman suggests that the Methuselah of instructional technologies, the venerable broadcast lecture, might finally be showing signs of going the way of geocentricity and the four humors. Applying methods taken from the theory of “deliberate practice” by psychologist Anders Ericsson, the research team introduced a more interactive, discussion-based and assessment-oriented approach to a physics class that strongly implies major improvements to science and engineering instruction in general.

The setting for the study involves two groups of electromagnetics students (control: 267; test: 271) wherein both were given the same learning objectives and enjoyed the same pedagogical approach (but not the same instructors) for the first 11 weeks of instruction. On week 12, Deslauriers and Schelew (both of whom have limited teaching experience) jumped into the fray and according to the BPS Research Digest lead the test group utilizing “…discussions in small groups, group tasks, quizzes on pre-class reading, clicker questions (each student answers questions using an electronic device that feeds their answers back to the teacher), and instructor feedback.” And, what is especially important to note here: there was no formal lecturing. According to the researchers the object of the game was:

“…to have the students spend all their time in class engaged in deliberate practice at ‘thinking scientifically’ in the form of making and testing predictions and arguments about the relevant topics, solving problems, and critiquing their own reasoning and that of others.”

In contrast to the test group, the control group went on learning the same material in the normal (typically passive) fashion epitomized by classroom lectures for probably the last 900 years. The students, however, apparently noticed a difference. As quoted in the BPS review:

“Student engagement (measured by trained observers) and attendance in the control group was unchanged in week 12 compared with earlier weeks. In the intervention group, attendance rose by 20 per cent and engagement nearly doubled.“

The critic or cynic might assert that the presenters were putting on a better show in the test case. What about student performance? On the first day of class after week 12 both groups were tested on what they had learned the previous week. In addition, as part of the preparation for the test, both groups were given all the materials used by the intervention group, i.e., the clicker questions, group activities and problem sets, and exercise solutions. The results are as striking as the jump in student engagement:

The non-lecture intervention group averaged 74 percent correct while the control group averaged 41 percent. Factoring out random guessing, the intervention group did twice as well as the traditional lecture students (the effect size being on the order of 2.5 standard deviations!). Not to be downplayed, student reviews rated the non-lecture approach very positively. Ninety percent said they enjoyed the process.

Jeffrey Mervis writing for the AAAS ScienceNow magazine quotes Wieman as saying:

“ ‘It’s almost certainly the case that lectures have been ineffective for centuries. But now we’ve figured out a better way to teach’ that makes students an active participant in the process, Wieman says. Cognitive scientists have found that ‘learning only happens when you have this intense engagement,’ he adds. ‘It seems to be a property of the human brain.’ ” – Jeffrey Mervis, A Better Way to Teach?

Given the novelty of the technique and the overt nature of the study there has been some criticism of the results based on the Hawthorne Effect. The research team discounts this criticism on the basis that the intervention only occupied a small percentage of the students’ overall daily learning activities. Drilling a little deeper, psychology professor Daniel Willingham (as recounted in Carey below) cautioned that the study might not have been designed well enough to discern which of the factors introduced in the new classroom style account for the gains in student performance and to what degree.

In what might be one of the clearest victories for proponents of the Inverted Classroom the research team is optimistic of the result and reckons it can be generalized to a wide range of post-secondary courses. No doubt further studies can be expected. The study in question is supported by a $12 million dollar program to investigate new methods to enhance science instruction using research-backed methods.

References.

Deslauriers, L., Schelew, E., and Wieman, C. (2011). Improved Learning in a Large-Enrollment Physics Class. Science, 332 (6031), 862-864 DOI: 10.1126/science.1201783

Carey, Benedict (2011). Less Talk, More Action: Improving Science Learning
http://www.nytimes.com/2011/05/13/science/13teach.html

Mervis, Jeffrey (2011). A Better Way to Teach?
http://news.sciencemag.org/sciencenow/2011/05/a-better-way-to-teach.html

Dwyer, Liz (2011). Research Proves College Lectures Need to Go the Way of the Dinosaur
http://www.good.is/post/research-proves-college-lectures-need-to-go-the-way-of-the-dinosaur/

Expert Performance and Deliberate Practice
http://www.psy.fsu.edu/faculty/ericsson/ericsson.exp.perf.html

The Inverted Classroom
http://www.hg2s.com/blog/2009/11/14/the-inverted-classroom/

Infocus 'What Not To Present' Contest

It’s not clear whether the presentation experts at InFocus Labs have opened a Pandora’s Box with this event, but their “What Not To Present“ contest apparently overwhelmed even their most staid judges in terms of popular response and the degree to which things can sink and still be considered acceptable. The response from the field was both daunting and gratifying:

“Our ‘What Not to Present’ contest was epic! Many thanks to all of you kind folks that submitted entries and spread the word about it. Many amazingly horrendous slides were sent in from all around the world. We laughed. We cried. We cringed.”

Naturally once the floodgates were opened the selection of a winner was not at all an easy task.

“We randomly chose our top 3 winners, but then quickly realized that we had to do more. So we are giving away ANOTHER projector to the slide we thought was the most horrendous. We passed the ugliness around the InFocus offices and to many of our partners pandering for votes – and we have a winner!”

Prizes generously include InFocus projectors and accessories.

Suspense mounting? Here’s the First Place winner from the random selection round:

See yourself in that slide? Me too. Kind of makes me cringe. Hopefully we all have slides like that locked securely in our pasts.

But that’s not the point of the “What Not to Present” competition. The good folks at InFocus must surely be sick and tired of their excellent products being associated with – one might even say equated to – the kind of visual flotsam that populated this contest. And they’d like it to stop. So, in an ongoing effort to assist in cultivating our design and presentation senses they are going to offer ongoing therapy to the readers of their blog wherein experts Garr Reynolds and Ellen Finkelstein will offer free advice on how to make presentations attract our attentions for all the right reasons. So, stay tuned. In the meantime, and capturing a feeling right at home on these pages, Ellen Finkelstein offers a few tips on how to avoid being submitted as a contestant in next year’s “What Not to Present” contest: Have Compassion on Your Audience!

Now, ready for this year’s Grand Prize Winner of the worldwide InFocus “What Not to Present” competition? Here you go.

Further Reading

PowerPoint Overload – Two Pounds of Sausage in a One Pound Bag

“The Cognitive Style of PowerPoint: Pitching Out Corrupts Within”

“PowerPoint Does Rocket Science–and Better Techniques for Technical Reports”

If you are the kind of designer who cannot tell the difference between Times and Helvetica, you’re in luck. A recent study by a team from Princeton and Indiana Universities shows that educational presentations that are hard for students to read may lead to improved memory performance. In the technical jargon of cognitive psychology the reason for this counter-intuitive result is due to the heightened “disfluency” caused by poor typography that leads to deeper processing (or encoding) in the brain.

Many classroom instructors and and instructional designers assume that clearer, easier to read, media reduce the “friction” of learning and act to promote and accelerate the transmission of new ideas and skills. Not so, say Connor Diemand-Yauman, Daniel Oppenhiemer and Erikka Vaughan who penned the study soon to be published in the journal Cognition. In some cases, they assert, making material harder to learn actually improves long-term memory. What’s worse, they have the control group data to prove it.

“Many educators believe that their ability to teach effectively relies on instinct and experience. However, research has shown that instinct can be deceiving and lead to educational strategies that are detrimental to learners.” – Diemand-Yauman, et al.

Two studies were undertaken to test the hypothesis that “desirable difficulties” can lead to enhanced learning. In the first, twenty-eight participants ranging in age from 18 to 40 were asked to learn fictional taxonomic data similar to that found in biology classes. The disfluent media presented the material in 12-point Comic Sans rendered in 60% grayscale or 12-point Bodoni MT also in 60% grayscale. The fluent media used 16-point Arial rendered in plain black. (It should be noted that the author knows more than one professional designer who considers Arial to be at least as disfluent as Comic Sans, grayscale notwithstanding.)

Participants were given 90 seconds to memorize their fictional taxonomic data. For example:

The norgletti

• Two feet tall
• Eats flower petals and pollen
• Has brown eyes

Each data set like the above was composed of three species of aliens, each with seven features, for a total of 21 items to be learned. After 90 seconds of study the participants were distracted for 15 minutes with another task after which their recall was tested (“What is the diet of the norgletti?”).

The results? Fluent learners successfully recalled 72.8% of their data. Disfluent learners scored higher: 86.5%! What’s more, differences between the two disfluent fonts were not found (probably because ugly is ugly).

“Similarly, many education researchers and practitioners believe that reducing extraneous cognitive load is always beneficial for the learner. In other words, if a student has a relatively easy time learning a new lesson or concept, both the student and instructor are likely to label the session as successful even if the student is unable to retrieve the information at a later time.” – Diemand-Yauman, et al.

Not wishing to hastily generalize their preliminary results to classroom conditions, Diemand-Yauman, Oppenhiemer and Vaughan arranged a study with 222 Ohio high school students (ages 15-18). In the high school study teacher-prepared instructional content (Powerpoint and worksheets) were reformatted (but not edited) using disfluent fonts or left unchanged. Different sections of the classes were randomly assigned to a disfluent or control group. Teachers were told that the study focused on the effects of different fonts in presentations to counteract the Pygmalion Effect. After the classes were presented in normal fashion exams were given along with a survey to assess whether disfluency affects motivation.

The results? Once again the disfluent group scored higher (m=0.164, sd=1.03; m=-0.295, sd=1.03; using Z-scores) and there was no difference between ugly fonts. Further, the survey revealed no motivational differences between fluent and disfluent presentations.

The authors warn that interpretation of the results and their subsequent application in the classroom be cautiously undertaken. First, the novelty and distinctiveness of the disfluent fonts might be a factor enhancing their “desirable difficulty.” Another issue is that the point at which a typeface changes from “desirably difficult” to “illegible” is not known.

The authors concede that there is a point at which “disfluent” pushed to its extreme becomes “impossible,” hindering learning altogether.

At present it seems as though the tonic effects of disfluency probably follow a U-shaped curve and that the exact parameters that affect the shape have to be teased out through further experiment.

Another question is whether this disfluent effect will be seen with other media as well. The authors of this study only considered typographic media, but one has to wonder if it is possible to obtain similar results with audio and video.

References.

Diemand-Yauman, C., et al. Fortune favors the Bold (and the Italicized): Effects of disfluency on educational outcomes. Cognition (2010), doi:10.1016/j.cognition.2010.09.012

McDaniel, M. A., Hines, R., & Guynn, M. (2000). When text difficulty benefits less-skilled readers. Journal of Memory and Language, 46(3), 544–561.

McNamara, D. S., Kintsch, E., Butler-Songer, N., & Kintsch, W. (1996). Are good texts always better? Text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1–43.

Oppenheimer, D. M. (2008). The secret life of fluency. Trends in Cognitive Sciences, 12(6), 237–241.

Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction, 12(3), 185–233.

The Bandwagon

If you are thinking of using social media in a class to help build useful collaborative connections, retire the fears of shy students and introduce the same engagement you see in sites like Facebook, think again.  A recent study by the Lab for Social Computing at Rochester Institute of Technology suggests that the use of social media in classrooms might yield little effect in improved communications and enhanced connections between students. The study into the effects of social media was conducted as part of a course on the use of social media and tools. It included contributions from online learning and course management systems and discussion groups that were proposed to enhance instruction, improve communication and facilitate connections between the students and course content. The results indicate that poor social acumen in the face-to-face interactions might be mirrored in the (more) virtual social medium. What’s more, echoing teacher and educational social media researcher Michael Wesch, the RIT study suggests that the educational use of social media may have to be learned:

“…the educational use of social media may not counteract poor social connections that are seen in face-to-face communication or elicit the same impacts seen in the use of social media sites such as MySpace and FaceBook.”

Researcher and team leader Susan Barnes comments on the hopes and goals of social media in the educational environment relative to her team’s findings:

“Many social media advocates have argued that the use of these tools in classroom settings could greatly enhance interaction and learning and assist shyer, more reserved students in becoming more involved, as has been seen in other online environments. However, our findings show that the incorporation of social media had no measurable impact on social connections, to the point that students did not consider other members of the class to be part of their social network.”

The RIT research team plans to expand the study to consider different educational formats and additional social media applications in an effort to determine the effects and differences of social media from traditional classrooms. The intent is to help educational planners and instructional designers better use social media in course development and delivery.

“The issues surrounding poor social network construction within online educational environments points to greater opportunities to examine how technology and mediated software can be better designed to suit the types of communication and interactions desired by our students.”  – Christopher Egert, co-author

References.
Jacobs, Stephen, Egert, Christopher A., Barnes, Susan B., “Social Media Theory and Practice: Lessons Learned for a Pioneering Course,” 39th ASEE/IEEE Frontiers in Education Conference, T4J-1, October 18 – 21, 2009, San Antonio, TX.

Study Examines Use of Social Media in the Classroom

There is an entertaining (and on-going) discussion at Edward Tufte‘s blog on the rate at which the human eye (specifically the retina) transfers information to the brain. The implications of the discussion point to the design of displays but the discussion has necessarily taken a turn in the direction of the likely question “What is the maximum amount of information (or data) that can be transferred from a PowerPoint slide to the brain?”

Issues of memory, interest and higher cognitive processing aside, preliminary research at the University of Pennsylvania and Princeton University suggests that the retina transmits data to the brain at 10 million bits per second – the rate of a basic 10Base-T Ethernet connection. Tufte sets the stage for the discussion by noting that viewing a PowerPoint slide is vastly different from viewing the world:

“Looking around the world is easier than analyzing evidence displays, and there may also be within-brain impediments to handling vast amounts of abstract data, but at least the narrow-band choke point for information resolution should not be the display itself.

The average PP slide contains 40 words, which take less 10 seconds to read. Call that 1000 bits per second, which comes to 1/10,000 of the routine human retina-brain data capacity.

Also most of our evidence displays are in flatland, which is a easier than 3D perceptual tasks. On the other hand, many serious data displays are not in the familiar 4D space/time coordinate system that our eye-brain knows so well.

Memory problems can be partly handled by high-resolution displays, so that key comparisons are made adjacent in space within the common eyespan. Spatial adjacency greatly reduces the memory problems associated with making comparisons of small amounts of information stacked in time (PP slides, for example).

– Edward Tufte, July 26, 2006″

The process from world to retina to brain seems sufficiently complex and multivariate that I am inclined to side with Tufte’s correspondent Niels Olson when he points out:

“While PowerPoint is surely a horrid way to transmit information, I’m not sure we can inject very abstract information into people at ethernet rates. 40 words in 10 seconds doesn’t translate to 1000 bits per second transmitted over the optic nerve, which connects the retina to the banks of the calcarine sulcus in the occipital lobe, via the optic chiasm and the lateral geniculate nucleus. At a minimum the data being transmitted would require an analysis of the typography’s geometry (edge detection being a basic function of the retina), the amount of the visual field taken up by the display, the location of the display’s image on the retina relative to the fovea, and the rates of change in the display and surrounding motion (the speaker, other audience members, etc).”

Interestingly Olsen picks up on a decidedly (Eric) McLuhanesque point when he comments on the 240-words-per-minute rate, a figure that roughly corresponds to both the average reading speed of sighted readers today (McLuhan) and the rate at which words in audio form (like podcasts) are transferred [Olsen comments on this in more detail in a later post]:

“Your guesstimate of 40 words in 10 seconds leads to a 240 word-per-minute reading speed. Like normal readers, braille readers can read at 200 to 400 words per minute. Is there any evidence that a person with an aquired partial nerve blindness also aquires an impaired ability to reason spatially? My classmates at Tulane Med found they preferred listening to the lecture audio I recorded at one-and-a-half speed, which also pushes close to 200 words per minute. Most people found twice-speed to be uncomfortably fast. This 200, 240, 400 word-per-minute rate may be a more accurate definition of the rate at which the human mind can receive and abstract information in word form, and this is likely driven by communication between Broca’s area and Wernicke’s area via the arcuate tract. Keep in mind, reading is a highly abstract function.”

The discussion has far from petered out. Combining the eye and the ear, The New York Times reported on research conducted at the University of California, San Diego, which calculated the average daily intake of data for a North American at 34 Gigabytes plus 100,000 words. What this means is that if you believe the estimate, our eyes and ears are busy handling that much data via all channels in a 24-hour period. According to the New York Times and the San Diego study the eye is still hard at work in the new media:

“Print media has declined consistently, but if you add up the amount of time people spend surfing the Web, they are actually reading more than ever.”

I leave it as an assignment to the interested reader to calculate the rate of information in Mbits/second of 34 Gigabytes per 24-hour period.

HMI Report/UC San Diego

References.

Penn researchers calculate how much the eye tells the brain

Kristin Koch, Judith McLean, Ronen Segev, Michael A. Freed, Michael J. Berry, Vijay Balasubramanian, Peter Sterling, “How Much the Eye Tells the Brain,” Current Biology 16 (July 25, 2006), 1428-1434.

The American Diet: 34 Gigabytes a Day

How Much Information?

Although distance and online learning have become staples in today’s colleges and corporate classrooms, they are not regarded as approaches without problems. Statistics for completion of online courses are typically quoted at around 30%, leading many to conclude that the means and methods of online instruction are unappealing to the learner and less than effective for the teacher. Furthering concerns about the overall effectiveness of online instruction, a 2007 study at the University of Missouri suggests that online learning (or e-learning) may not be a good match for some learners.

“Distance learning was designed to provide learners with more opportunity and flexibility for learning. Distance learning allows the learner to overcome traditional barriers to learning such as location, disabilities, time constraints, and familial obligations. However, not every learner will be successful in a distance learning environment.”

Comparing demographic (age, gender, ethnicity, employment) and affective (personality, motivation) issues that might form barriers to learning, researcher Shawna Strickland looked at what makes some people successful at online learning while others drop out. Strickland cites some common barriers to successful online learning as:

• Lack of institutional support
• Lack of free time
• Family constraints
• Financial limitations
• Poor time management skills
• Isolation
• Anxiety and stress
• Limited prior experience
• Previous academic failure

Although no correlation with learning style was found (p. 35), Strickland notes that individual motivation and the degree to which the student accepts personal responsibility for his/her learning act as a prime factors in distinguishing the successful from the unsuccessful learners.

“…the major difference between the distance and traditional learner is the motivational level of the distance learner. A possible reason for this increased motivational level is that the learner has accepted more responsibility for the educational experience. Although the authors [see Simonson et al.] have provided rationale for this key difference, they further state that, when comparing the individual attributes of the two types of learners, they are ‘not generally different from each other.’ “

Strickland also sees communication as key to a successful outcome:

“The success of distance learning is dependent on communication between the learner, his or her peers and instructor. To encourage success within distance learning, it is necessary to evaluate each individual’s needs on a case-by-case basis. While successful learners tend to display certain traits, any adult learner with the proper motivation and preparedness could be successful in a distance learning program.”

References.

Strickland, Shawna L., “Understanding Successful Characteristics of Adult Learners,” Respiratory Care Education Annual Volume 16, Fall 2007, pp. 31-38.

Furst-Bowe, J., Dittman W., “Identifying needs of adult women in distance learning programs,” Int J Instr Media (2001) 28(4), pp. 405-413.

Mupinga, D. M., Nora, R. T., Yaw, D. C., “The learning styles, expectations and needs of on-line students,” College Teaching (2006) 54(1), pp. 185-189.

Simonson, M., Smaldino, S., Albright, M., Teaching and learning at a distance: Foundations of distance education 2nd ed., Merrill Prentice Hall (2003)

When should simulations be used in class? Do they matter or are they eye candy, empty demonstrations and a waste of time? A recent study published in Medical Teacher suggests that medical students benefit significantly in both learning and retention when high fidelity simulations are used in training. Authors Corey Heitz, Ashley Brown, James E. Johnson & Michael T. Fitch of Wright State University and Wake Forest University School of Medicine, compared the educational effects of a 90-minute live simulation to a traditional lecture.

A team of physicians assisted the in the presentation by acting the roles emergency medical staff, nurses and even family members. A computerized Laerdal SimMan(tm) was programmed to represent the patient who displayed symptoms like nausea, mental confusion and vomiting. As reported in Heitz, et al. (2009) the students were immersed in a theatrical enactment of the medial crisis:

“A Laerdal SimManTM was transported from the simulation center to the medical school lecture hall where a prerecorded EMS radio call announced the arrival of the fully clothed simulation mannequin. Resident physician actors portrayed EMS provi- ders, nurses, and family members. Student volunteers ran the case as emergency physicians and patient management decisions were guided by class input. The clinical scenario was enhanced with group discussion of the relevant basic science mechanisms underlying the autonomic nervous system, neurotransmitters, receptors, and neuropharmacology.” – Heitz et al. (2009)

The authors note that one of the key differences in this trial was the size of the group – 112 students in two groups. The live simulation was based on a clinical scenario designed to bring out basic concepts in neuroscience already presented in a lecture several days earlier by a participant who was unaware of the study.

Results of the training were measured using four multiple-choice pre-tests and post-tests.

“The primary study outcome was this comparison of student performance on a pretest compared to a posttest administered immediately after the simulation session, and participants were significantly more likely to get all four posttest questions correct after experiencing the simulation.” Heitz et al. (2009)

A follow-up post-test was given to students eleven days later to assess retention. The researchers suspect that immersion in the simulation contributed to recall:

“The concepts presented during our simulation session improved student testing immediately and may have facilitated performance on an examination 11 days later.”  - Heitz et al. (2009)

The authors conclude that the use of simulations of this kind can be valuable in medical education:

“The students not only felt the simulation experience correlated well with basic science concepts, but also showed statistically significant improvement on the pre- and posttest examinations. Our results show that this type of learning exercise may provide an alternative for ‘‘typical’’ lecture-style education.” – Heitz et al. (2009)

References.

Wake Forest University Baptist Medical Center (2009, July 14). “Simulating Medical Situations Helps Students Learn, Retain Basic Science Concepts”. ScienceDaily. Retrieved April 22, 2010, from http://www.sciencedaily.com/releases/2009/07/090714085822.htm

Heitz, Corey , Brown, Ashley , Johnson, James E. and Fitch, Michael T.(2009) “Large group high-fidelity simulation enhances medical student learning”, Medical Teacher, 31: 5, e206 — e210

Emergency Simulations at Wake Forest University School of Medicine