ALNmagazinelogo Volume 2, Issue 2 - October 1998 Issue Table of Contents ISSN 1092-7131 RIPE: Rapid Instruction and Production Environment Bruce A. McHenry and Nishikant Sonwalkar Hypermedia Teaching Facility Massachusetts Institute of Technology Abstract WWW-based ALNs (Asynchronous Learning Networks) offer advantages, especially for distance learning, but the creation process generally relies on the batch conversion of source materials, in existing media, into "interactive" web pages. This process may be enhanced with a quality improvement methodology that makes the customers (i.e., students) part of the development process. Our system is designed to facilitate prototyping and to integrate testing. Instructors should be able to generate materials quickly and without having to undergo extensive training. RIPE proposes a "mini-studio" that supports the capture of "sketch and talk" presentations using a straightforward interface. In order to integrate testing, RIPE also supports threaded discussions between the teacher and the students. These originate from students' questions and are embedded in the originating web pages. Furthermore, RIPE provides a test-bed for the idea of having students rate every answer that they view. I. INTRODUCTION WWW-based ALNs (Asynchronous Learning Networks) offer advantages, especially for distance learning [1], [2], [3], but the creation process generally relies on the batch conversion of source materials, in existing media, into "interactive" web pages. This process may be enhanced with a quality improvement methodology that makes the customers (i.e., students) part of the development process. Figure 1 shows that prototyping and testing are integral parts of quality driven processes. Figure 1 is similar to the PDCA (Plan, Do, Check, Act) cycle advocated by Deming [4] and Shiba [5]. mchenry1.jpg (29920 bytes) Figure 1. Requisite Process for Quality Improvement. In current ALN technology, prototyping is weak because the process of making web pages is too slow or indirect to change content on the fly in response to an individual student's needs. In creating content using a batch process and without this feedback, content designers can too easily become concerned with "How does it look?" rather than "Does it say the right things, in the right places?" Moreover, when the content experts and the web page designers/creators are different people, a long delay between finding a problem and publishing the fix is all but ensured. While responsiveness to students' needs and problems is an essential part of ALN development, there is little support for collecting this data as part of testing. Threaded discussions, email and chat sessions do capture content specific feedback but this "testing" is not integral with the prototyping. Each of the authoring environments surveyed (Macromedia Authorware [6], Centra [7], CyberProf [8], LearningSpace [9], HITE [10], iFactory [11], WebCT [12]) has at least three of the following four limitations: 1. If the student asks a question about a web page, the question and answer are separated from the page. 2. If other students are allowed to answer questions, there is no mechanism to distinguish the best answers, or even the correct ones. 3. If the student adds a query to the thread, there is no automatic notification of a reply. 4. Replies can only use text, not diagrams, pictures or speech. II. APPROACH Our system is designed to facilitate prototyping and to integrate testing. Instructors should be able to generate materials quickly and without having to undergo extensive training. RIPE proposes a "mini-studio" that supports the capture of "sketch and talk" presentations using a straightforward interface. In order to integrate testing, RIPE also supports threaded discussions between the teacher and the students. These originate from students' questions and are embedded in the originating web pages. Furthermore, RIPE provides a test-bed for the idea of having students rate every answer that they view. We expect that the ratings will be useful for selecting the best answers within related sets of questions. The students' rating data will also contain information about individual preferences that may eventually be used to customize the presentation. A. Software Architecture: Design and Implementation HTML/JavaScript web pages are created on the fly with four different appearances depending on whether the user is a student or teacher, and whether the page is published or not. The largest of these modules ("ShowPage.pl"), is responsible for presenting a content page with its graphics, text and audio. "ShowPage.pl" displays different controls on the page depending on whether the user is a student or instructor. The flow of control is presented in Fig. 2. mchenry2.jpg (62196 bytes) Figure 2. Flow of Control. Most of the buttons on a content page call a control module ("Control.pl") which implements the majority of authoring and student functions (take/delete a picture, record audio, update the text, publish the page and notify the interested parties, jump to previous/next/contextual page, rate an answer). The other scripts provide supporting functions, such as allowing users to confirm their questions and email addresses, display the queued questions, send themselves bookmarks, add themselves to an answer's publication notification list, and show the table of contents. The system supports one author and many concurrent students. The choice of freezing past contributions vs. allowing them to be edited is a major trade-off with threaded discussions. In RIPE, answers can still be modified after publication so that mistakes can be fixed. Unlike most threaded discussions, RIPE's goal is to develop an educational product, not to maintain a record of how that was done. The instructor's work in progress is visible to the students. It could instead be hidden until published, but it was felt that there is little harm in allowing this access. The early design would have allowed the main presentation to advance automatically (i.e., the end of the audio track for each page would commence the download and playback of the next). However, this ideal of continuous playback gradually lost currency. Some pages will have no audio by which to set the display interval. But the main problem is that users need time to ponder the material and decide whether there are any important questions before advancing. Even if there are none, an obligatory interaction that requires minimal attentiveness helps to ensure that students, while they can still fall asleep during a presentation, won't miss much if they do. In order to encourage users to rate the answers, the act of voting also returns users to the origin of the question. Unfortunately, this does not always work because users expect to use the browser's "Back" button in order to return to the origin of the last hyperlink, or question in this case. (See trial results for an estimate of the frequency of occurrence.) There is no way to interdict the Netscape browsers' "Back" button. The browser toolbar can be turned off but then some users are in the habit of going back by using a menu off the alternate mouse button, or the "Alt+Left" key combination. However, it is possible to intercept the "Back" button in the CGI script and pointedly ask the user to either vote or explicitly abstain from voting. There are two kinds of situations where students have questions: 1) Students know that they don't understand, but are not sure why and 2) they have a question in mind. The first situation implies that ratings should be based upon replies to the question, "How useful was this answer?" The second situation needs an answer to, "Was the question answered?" The first situation is certainly not uncommon, and at least one professor has said that this kind of non-specific question is the most typical, at least in undergraduate courses. RIPE compromises by asking students to "Rate overall answer(s) to the question." The use of ratings has been the main source of disagreement between the authors. The main objection is the "too simplistic answer" problem, in which students favorably rate an attractive solution that actually fails in some critical but non-obvious way. If students ever realize that the answer is bogus, it is long after they have seen and reviewed the answer. Under a system using student ratings alone, there is no redress: the misleading but highly rated answer will remain highly rated, pulling students in and providing them with a false sense of satisfaction. As implemented, RIPE ameliorates this problem because it allows only one content creator and that person is also expected to be the content expert. As an expert, they are expected not to provide misleading answers. However, reliance on a sole authority can be limiting when the authority is not readily available. In this case, teaching assistants and even students themselves might author material. How is the material to be refereed then? One could ask the content expert to review the newly created materials. It takes less time to review than to create. There could be several options in the event of an incorrect answer. The first, which is supported by the current version of RIPE, would simply be to edit the error (and provide notification to the author and the misled students). Another approach would be to leave the material unmodified but to have a specially weighted rating for the content expert/editor. This instructor rating could, for instance, carry as much weight as all of the student ratings. B. Instructor Interface Fig. 3 shows the physical components of the system. The instructor's computer is connected to a camera focussed on a paper sketchpad, and a microphone. Audio is provided by Progressive Networks' RealAudio so students can use RIPE on virtually any platform. Picture and audio capture require an SGI Indy workstation that is also running a WWW server. mchenry3.jpg (43218 bytes) Figure 3. Physical Setup. The instructor's controls are easy to use (Fig. 4). image51.gif (25187 bytes) Figure 4. Instructor's Controls. The instructor can * take pictures of the sketchpad that are automatically displayed in a web page using the "Picture" and "No Pic" buttons * type into the web page and record an audio attachment ("Record", "Stop") that automatically plays when students open the page. "Play" invokes a separate audio playback control panel * Use the "Format Text" button to embed HTML commands into the text and preview the output. * Use the "Context" button links to the page where the question occurred (parent page) to look at any related questions, see the text and hear the voiceover before responding. The "Context" button is not needed to see the parent page's picture because it is automatically inherited on the new answer page. * Use the "Queued Questions" button to link to a page containing the remaining unanswered questions. To help select a question to answer, the questions are ordered oldest first, and each is followed by the list of students who want to see the reply (notification list). When finished with an answer, the instructor uses the "Publish" button. This takes the question off the queue and sends email to the notification list. The students' controls (Fig. 5) consist of buttons to page forward and backwards within an explanation ("Next Page", "Prev Page"), an audio control panel, links to the previously asked questions, a text box for asking a new question and a bookmark utility. Students are requested to rate every answer. In order to encourage this, the rating buttons serve both to vote and to go back from an answer to its question's context. image52.gif (184679 bytes) Figure 5. Students' Controls. III. METHOD Prof. Frank Feng taught the Spring 1997 version of 2.001, Mechanics of Materials, and he agreed to participate in the trial. Dr. Feng provided three sets of source materials during the week leading up to and the day immediately following the final exam: 1) Spring 1996 final exams with solutions, 2) Solutions to a sample final that had been handed out in class, 3) Solutions to the Spring 1997 final. All were scanned using Adobe Photoshop and cropped before being transferred onto web pages. The final exam with solutions was also rearranged so that the questions and solutions would appear on the same web page. The originals were not created with this use in mind and some of the smaller notations were difficult to read in their digitized form. There were no voiceovers. Links were added to the 2.001 area in HITE and Prof. Feng told students in class to look for the new materials there. The class was also notified by email. No special instructions were given to the class. The author showed Prof. Feng how to use a browser on his desktop Macintosh to enter RIPE with instructor privileges. Prof. Feng could reply from his office to the questions that only required text based answers. The experiment ran for approximately one week. IV. RESULTS Overall, there were only twelve questions and no sub-questions (see Table 1). Particularly for the Spring 1996 sample, most questions were about the overall content of the final rather than the material on the page. All but two of the questions were answered with a simple yes/no answer that did not lend itself to a rating. Therefore, it is not surprising that 78% of the users merely abstained or used the "Back" button instead of giving a rating. For the Spring 1996 final, about half of the 46 ratings were "A," while the other half were "F." Presumably the "F" ratings had more to do with whether the student liked the answer than whether the reply answered the question. The problem seems to be with the choice of content. The final exam solutions were self-contained and, as Prof. Feng noted, if a student couldn't understand the solutions, it would be too late to pass the course. Pages Hits Qs Q Hits Ratings Spring 1996 Final w/ Solutions 10 2036 6 205 46 Practice Exam Solutions 6 882 6 103 13 Spring 1997 Solutions 2 25 0 - - Table 1. Usage Summary From the Trial. Attention was also focused on the fact that this approach would put a heavy burden on the instructor if there were many questions. The effort could become comparable to writing a textbook. Whereas textbook publishers offer advances and royalties, there is no similar compensation in place for authors of new web content. There were no software faults during the trial. A. Educational Viability RIPE is really a methodology for rapidly creating content. Figure 6 shows two factors that, in addition to distance, favor RIPE: mchenry6.jpg (28050 bytes) Figure 6. Factors Influencing RIPE's Penetration Potential. The factors suggest that RIPE offers the fewest advantages when the students: * come from a relatively uniform background (e.g., because of admissions requirements or course prerequisites) * have similar objectives (e.g., to get a good grade, and graduate) * are taking fundamental courses that are invariant from one semester to the next (the instructor can predict the vast majority of the questions) The existing materials are already well matched to the typical student's needs. Accordingly, it is hardest to make the case for RIPE in a core subject like mechanics of materials that, according to Prof. Feng, "has been taught the same way for a hundred years" to uniformly prepared students who are already on campus and expect to take classes. On the other hand, if the subject is new and changing then there will be a need to develop content. Similarly, more heterogeneous classes will increase the need for content creation. In continuing education and graduate studies, students are more likely to come with specific issues, more diverse backgrounds and a greater variety of questions. The third major factor is distance--which makes all of the online approaches more attractive. B. Influence of Institutional Factors Many of the most vigorous adopters of distance learning have been second- and third-tier schools. Usually, these are regional schools with commuting students. Their products are commodities, so proximity is the main reason that students select them. Since these will probably be the first schools to face serious revenue shortfalls due to rising enrollment in less expensive distance learning options, we should not be surprised that so many are responding proactively by moving towards cyberspace. In entertainment, second rate local performances can no longer draw a crowd when the alternative is to see and hear nationally and internationally acclaimed artists at the movies, on CD and TV. Yet this "star performer" effect is little noticed in education. There are thousands of mediocre physics high school teachers who kill off intrinsic curiosity in tens of thousands of students every year. This occurs despite the fact that outstanding videotapes covering the same material have existed for some time. Why is the diffusion process so slow? Technological diffusion can be held up by "gatekeepers." In this case, the teachers are threatened by the innovation and, since they are also gatekeepers, they have the power to block it. A mediocre accounting 101 teacher at a community college may not want students to see how much better videotaped instruction could be, and may be reluctant to give up the role as leader of the class, as well as being unsure of how to redefine the job. The job anxiety is not altogether justified. Stand-up teachers can become coaches and mentors to students. It may be desirable to delete some of the content created by RIPE, just to let students "discover" the missing answers with guidance from other students and the teacher. If all of the questions and answers are too clearly in place, the student may become unmotivated. Established colleges will still face a dilemma. If they promote the use of courses developed by star teachers at the expense of their traditional classes, they will alienate faculty whose role is to "stand and deliver." As online learning matures, its products may begin to deliver better value. If this happens, students will gravitate towards programs that use them. Unless the non-star faculty members are able to provide value in some other way, they will create operating losses. Newer institutions that develop around distance learning approaches will not have the same high costs that are sunk into constructed classrooms and tenured faculty. This will provide them with an opportunity to compete in the top ranks to grant degrees that do not require residency for reasons such as research. Though distance learning can be a way to leverage the faculty, one paradoxical effect has been that the departments with the lowest enrollments tend to be the most aggressive adopters of distance learning. Whereas these departments hope to see revenue enhancement, heavily saturated departments at MIT see distance learning as a magnet for additional students that they do not want. Stanford has inverted this logic and turned its high demand courses (EECS) into sources of revenue with exceptional profit margins [13]. The money comes from the employers and students, who pay full tuition but place few extra demands on the campus infrastructure and faculty. Interdisciplinary studies are becoming increasingly important. Yet it is often difficult to cross over into other fields because of specific assumptions about the preparation of the students. Prerequisites are usually required in order to ensure uniform preparation for the class, but it is often impossible for students from other fields to take so many subjects. Usually, only portions of the prerequisites are actually needed for the higher level class. One approach is to break classes down into smaller modules so that the prerequisite portions can be isolated and taken individually on an as-needed basis. RIPE can also help to fill in the prerequisites for the students who need it, when they need it. This micro-modularization of the programs and the courses will be necessary in order to accommodate more interdisciplinary students and participation from non-matriculating persons, including alumni. C. Trickle Down from Corporate Desktops? Teaching systems, which distribute the knowledge of experts, are the first step towards modeling and supporting many other kinds of communications that are used in the real world. While RIPE's roots are in education, there is a much larger market for communication within and between corporations. Given the economies of scale in software development, it seems quite likely that systems designed for the broader market will trickle down to education rather than the educational systems being adopted and expanded for broader uses. Microsoft's Powerpoint, for example, already incorporates audio and is capable of generating presentations on the web. D. Planned Technical Improvements The premier item on the agenda is to facilitate instructor access and increase the number of faculty who could use RIPE. Regular office PCs, equipped with a suitable camera mount, camera and a microphone, should also be able to serve as mini-studios. In order to expedite answers to students' questions, other students should also be able to answer, at least with text. This will create possibilities for student-to-student interaction and could encourage students to compete to give the best answers. However, it will also introduce a higher rate of errors into the content. Although other students will probably be able to point out a significant percentage of these errors, an authority should eventually review all student replies. Good answers should receive a "seal of approval" that would be visible next to their respective questions. The authority would edit, delete or return the other answers for revision. It should be possible to import images from .gif-generating programs on any client. The question list should be re-ordered according to the usage of the individual questions. Within the list, there should be groups of related, competing questions; these should be ranked by rating, not number of uses. Questions could become "related" if they are spawned by disgruntled users of existing questions. Students who rate answers with a 'D' or 'F' would be asked if they would like to reenter the question. Detailed statistics generation will help to assess the performance of individual answers and provide scores that encourage intra-student competition and could be used to allocate credits. V. CONCLUSIONS So far, RIPE has shown: 1. The concept of a "mini-studio" enables "instant" web page creation by using the sketch and talk methods with which teachers are familiar. 2. An implementation of a content development system based on PERL scripts. 3. A trial without significant problems due to loading or software errors. 4. That recruiting teachers to use this new approach is difficult because RIPE cannot compete with the classroom approach and teachers currently have few incentives to use RIPE. A critical mass of users is needed to create sustaining interaction and this cannot be achieved when online systems are positioned as a supplement to classroom instruction [10]. 5. That the type of material used with RIPE should be carefully chosen because some materials do not elicit many questions. Referring back to the original goal of improving quality by speeding up the development cycle, RIPE has demonstrated a viable approach to prototyping but not yet to testing. We look forward to further testing with classes that are oriented towards distance learning and use of the web. REFERENCES 1. http://www.gartner.com/training/products/alpha.html 2. http://www.njit.edu/DL/sld002.htm 3. http://www.open.ac.uk 4. The New Economics for Industry, Government, Education, W. Edwards Deming MIT CAES 1994 5. A New American TQM: Four Practical Revolutions in Management, Shoji Shiba, Alan Graham, David Walden 1993 Center for Quality Management 6. Authorware -- http://www.macromedia.com 7. Centra -- Centra Software, Inc., Lexington, Massachusetts http://www.centra.com/ 8. UIUC Cyberprof -- http://cyber.ccsr.uiuc.edu/cyberprof/general/homepage/Newpage/toplevel/welcome.html 9. LearningSpace -- http://198.114.68.60/ 10. HITE -- Statistical Analysis of Hypermedia Teaching Facility Usage, Spring 1997, HTF Publication available from Dr. N. Sonwalkar, MIT Rm 9-230, http://curricula.mit.edu 11. Interactive Factory, Inc., Boston, Massachusetts -- http://www.ifactory.com/ 12. WebCT -- http://homebrew.cs.ubc.ca/webct/webct.html 13. http://scpd.stanford.edu/intro.html Privacy Policy | The Sloan Consortium © 2002