Reading Summaries and Reflections from
Multimedia for Learning: Methods and Development
by Alessi and Trollip
by Tom Cutshall
| Chapter 1 | Chapter 13 |
| Chapter 2 | Chapter 14 |
| Chapter 3 | Chapter 15 |
| Chapter 4 | Appendix A & B |
| Chapter 12 | References |
Chapter one is an introduction to the general principles of multimedia for learning. Alessi and Trollip offer a first rule for correctly developing computer based instruction--to use the computer as delivery method in situations where using the computer is likely to be beneficial. They point out that different situations call for different methods such as delivery by instructor, peers, books, or videos. Interactive computer delivered multimedia instruction seems the logical way to deliver online instruction on PsycINFO near the point of need of the online PsycINFO database.
Alessi and Trollip (2001, p.6-10) describe the four phases of instruction that should occur for learning to be effective and efficient:
It is good to see all the phases as a guideline for what should be included in the instructional resource I will be creating. I would have considered guiding and practicing two ends of the same continuum of scaffolding as the support starts high then is reduced as the learner gains more experience and fluency. In fact all four phases of the instruction model include the concept of scaffolding.
I found it interesting that in the section highlighting eight different methodologies for facilitating learning that Alessi and Trollip point out it is rare for a single lesson using any methodology to provide all four phases of instruction. They point out that tutorials and hypermedia (which is probably the type of methodology I will use) generally engage the first two phases of instruction (presenting information and guiding the learner). It sounds like I should also include a combined drill and game methodology or simulation segment to motivate learner practice for fluency and retention. That should be followed up by a test to assess learning and future instructional needs.
It is also interesting that Alessi and Trollip state the four phases of instruction usually occur over days or weeks, not a single instructional session, but I think we are trying to include all four phases of instruction in one short instructional session. The question I would ask then is: Can all four phases fit into one instructional session? To fit everything into one session should the guidance and the practice be merged?
In chapter two Alessi and Trollip summarize learning principles and approaches. They start with the principles of behavioral psychology including classical conditioning, operant conditioning, and how it lead to Skinner's basic behavioral rules.
To avoid extinction Skinner suggested using intermittent reinforcement for longer lasting behaviors that are more resistant to extinction. I was planning to use positive reinforcement for my instructional product but had not considered intermittent reinforcement. Given the fact that the instructional lesson will only be 30 minutes I plan to use positive reinforcement throughout rather than intermittently.
Instructional design is largely based on behavioral psychology but newer ID models contain cognitive and constructivist elements. Alessi and Trollip next turned to principles of cognitive psychology, which places emphasis on unobservable factors such as the mind, memory, attitudes, motivation, thinking, reflection, etc. They detailed several areas of cognitive theory that are important to design:
The discussion on perception and attention had parts very relevant to my project included making information easy to receive, the position of information on the screen, and the fact that differences and changes attract and maintain attention. The section on encoding was a reminder that learning is facilitated by dual coding such as complementary visual and auditory information.
Two methods given for enhancing memory were imposing organization on the information and repetition of information, with organization being a more powerful method of memory enhancement. I know that the principle of organization works from personal experience and metacognition and will make sure my instructional information is organized and grouped logically so that repetition will not need to be used too much.
The section on comprehension suggested that comprehension of rules and principles (such as online searching of PsycINFO in my tutorial's case) can be determined by having the learner apply the principles and demonstrate correct applications of the rules.
It will be important that my web based instruction be composed of active learning with interactivity that maintains the learners attention, helps them store new knowledge and skills, and facilitates learner comprehension.
Motivation is a necessary part of learning. Alessi and Trollip summarized Malone's Motivation Theory to enhance intrinsic motivation (motivation from within the person). The four parts are:
For my tutorial on PsycINFO I could use the "fantasy" motivation of having the learner imagine they have to find an article on topic X for their term paper so they are motivated by seeing why they should learn how to search PsycINFO I will probably also use the aspect of control through contingency--giving the student different feedback based on their actions, choice--letting them return to a section for review or go straight to a section most relevant to them. Perhaps sensory curiosity could be aroused by interesting graphics on the web site and cognitive curiosity aroused by the statement--"you can't find it all on the web!". But it also seems odd to me that it would be motivational to give information that is contradictory--I think that would just confuse and decrease motivation.
Alessi and Trollip also summarized Keller's ARCS Motivation Theory for ways to increase learner motivation:
I think that a big part of the motivation factor for my tutorial on PsycINFO will be emphasizing the relevance to the learner-- showing them what they are learning will be useful to them and using examples (obtained from Dr. Frick) that are similar to the type of research projects they will have. I will try to use relevancy and content variety to capture and hold learners attention. I can build the learners confidence through making expectations clear, giving the learner control of the pace and order and repeatability of the sections they cover, and making it so that any learner who wants to learn will be successful. Hopefully the satisfaction part of motivation will be obtained through having students do actual searches on PsycINFO while in the environment of the tutorial (using frames), and by giving positive consequences and encouragement.
The motivation theories presented gave me several ideas for improving motivation in the learners that will use my tutorial, but I will be careful to only use the motivational techniques that fit the learning situation and not go overboard trying to fit every type of motivation into the lesson.
The section on locus of control brought out the good point that a good balance is to give the learner control of some factors generating a perception of control but giving in reality only partial control. This is something I am wrestling with as I try to determine if the user should be able to pick and choose which sections of the lesson they will go through (giving them the motivating control of choice) or if they will need to complete all the sections of the lesson. Sometimes students may not think they need a section of a lesson which is in fact crucial to meeting learning objectives. This may be due to a lack of learner metacognition--awareness of their own knowledge and abilities when it comes to searching online databases. Perhaps the tutorial can overcome this by prompting the student to reflect on past successes and failures doing online searching.
Part of my tutorial will probably include a diagram to create a mental model of what is happening when the student uses the search terms "And" "All" "Or" "Any" "Phrase" "Not" to connect search concepts.
It is important that my tutorial provide real world skills that will enable the learner to have near transfer to using the PsycINFO database on their own and having far transfer to using other online databases from the same company (Ebsco) or other online databases on GALILEO. I will take the advice of Alessi and Trollip and make the instructional environment very similar to the environment in which the learner will use the knowledge gained (using database screen shots and at some points providing a live PsycINFO session).
In the section on individual differences Alessi and Trollip point out that the most important individual difference is motivation and different reinforcements are effective for different people. The fact that I plan to use several different motivational techniques will help meet individual differences. I am also contemplating the option of providing audio along with the reading portions of the tutorial to accommodate learning styles and preferences.
The section on constructivist psychology principles included useful reminders that my instruction should be situated learning that can transfer to the real world and anchored in real world goals and problems of the student researching using PsycINFO Part of the tutorial will have the learner build a search strategy and use the strategy in a live PsycINFO session to find results. Learner's printable search results will be a learning artifact. Constructivism also includes autonomy and choice for the learner as I described in the motivation section above. Another part of constructivism theory that I hope to incorporate in my tutorial is reflecting the complexity of the real world by not over simplifying the skills needed or using only clean cut search examples. To reflect the real world and the cognitive skills involved, I need to use examples that sometimes have obstacles to overcome or need a few rounds of refinement of the search to bring success. Often in the real world several different approaches and techniques need to be used to be successful in searching online databases.
But I disagree with the constructivist viewpoint that tutorials produce inert learning not easily transferred to new situations. I feel that as long as I take the approaches described above to make the instruction anchored, situated and motivational that the learning will be relevant and transfer to the real world. I think that letting learners explore information freely would not give many of the learners the structure needed for them to effectively learn about all the skills needed to be successful searching PsycINFO
I agree with Alessi and Trollip that even though people may learn constructively (constructing knowledge based on their own experience and interpretations), that does not mean that online instruction can't facilitate learning in other ways. To adapt my instruction to different learners, subjects, and situations I think it is best to use a variety of techniques from the behavioral, cognitive, and constructivist learning theories.
Question: It seems odd to me that it would be motivational to learners to receive information that is contradictory--I think that would just confuse and decrease motivation. Am I not fully understanding this concept?
In chapter three Alessi and Trollip discuss instructional factors relevant to all interactive multimedia instruction. In the section on the title page I found several useful tips for my instructional product including:
In the section on directions it was pointed out that directions should be specific to the operation of the program, simple and self evident, and not include procedures used much later in the program. It is best to start with a minimal amount of instructions then through user testing see if more detail is needed. I would add that a goal in creating the interface and user testing is to make the instructional product as intuitive as possible so that few instructions are needed. Context sensitive help should also be provided at relevant points of the instruction as needed. Rollover text over every icon or navigational element can provide further instruction about the operation of the program.
Alessi and Trollip in the section on learner control of the program point out that the most important learner control is sequence (moving forward, review, and where to go next) and the pace of the program. Since my web based tutorial will have hyperlinks the learner will be able to control the sequence and pace of the instruction. I also need to make sure that my interface contains consistent global controls such as help, the homepage, and menu options. While I will allow the learners early termination of my program by closing the browser, I don't think I can do the programming to allow the learner to return to where they left the program. I think there is some javascript I could copy into the tutorial which would allow learners to bookmark in their web browser the last page of the tutorial they use. Alessi and Trollip provide a good general rule to provide more learner control for problem solving and higher-order thinking and more program control for procedural learning and simpler skills. I think my tutorial will be primarily procedural (steps to searching) but will include some problem solving (creating and refining a search strategy). Given the audience of undergraduate students I will probably give them a rather high level of control.
Buttons with words or icons are a popular and user friendly method of control. Buttons can be always visible and thus remind learners of their options. Alessi and Trollip wisely suggest using a small rather than large number of buttons so as not to overwhelm the learners with too much information or take too much valuable page real estate from the content. I plan to use just a few buttons with clear meanings, rollover explanatory text, and graphical confirmation for major navigation control and help. If more control is needed I will provide a menu.
I will probably make use of a full screen menu immediately after the welcoming title page and perhaps at the beginning of each section. I will probably use a frame type menu on the left portion of each screen to provide navigational control and visual orientation to the learner. Although it will appear to be a frame to the learner, it will be a single page subdivided by a table. This will avoid the usability problem frames can create of not being able to bookmark or easily determine the URL of a page within a frame.
I plan to include obvious underlined hyperlinks to allow nonlinear navigation and links to popup windows with definitions in my tutorial. As suggested by Alessi and Trollip my hyperlinks will cause cursor change, provide rollover explanatory text (using alt tags), provide visual confirmation of being clicked, and will follow the conventions of followed links turning purple so the learner has a visual clue as to where they have been. I was surprised to see that Alessi and Trollip suggested avoiding use of hyperlinks for global controls such as "Next" or "Exit". But I guess a hyperlinked button would be more obvious for this function then hyperlinked words.
I plan to give learners control via the mouse. Most web browsers don't include keyboard equivalents and I doubt that learners would return to my instruction often enough to justify learning keyboard alternatives to clicking on navigational elements.
In the section of the chapter on presentation of information I strongly agree with Alessi and Trollip that consistency makes for a better learning environment. Learners spend less cognitive energy figuring out the navigation and more cognitive energy on learning content. I agree that it is good to have consistent functional areas allocating sections of the screen for a specific purpose (such as content or navigation). But I disagree with the suggestion that directions or control options should be at the bottom or right side as variety in browser window size (which is fully controlled by the learner) may keep the learner from seeing the directions or control options. I agree that functions such as "Next" should logically be at the end of the content but I would suggest the Next button be very close to the end of the screen content rather than all the way on the bottom of the screen. I will probably place most of my control functions in the top part of a left frame for easy viewing regardless of browser window size.
As a video is very useful for modeling a process I plan to include a Flash movie or alternative screen capture animation to model parts of the search and retrieval process on PsycINFO Perhaps I can add voice over explanation for dual coding. But I was surprised that Alessi and Trollip said simultaneously reading text and hearing identical speech may decrease learning. Perhaps that would be the case if the downloading of audio and video were at different rates causing video to lag behind audio. But if they are playing in synch I sometimes like to have text to read while viewing video and hearing audio to clarify some parts of the audio that may be unclear. But I would certainly give the learner control over receiving video with text and audio or just video and audio.
Text will be an important part of my tutorial so I will make use of standard conventions of left to right, top to bottom and spacing between paragraphs. I will also work hard to have very concise text instruction that is chunked on pages so learners don't have much (if any) scrolling. I will probably make use of the suggested methods of emphasizing text through boxes, arrows, larger letters and isolation.
I agree that it is very important for text to be lean--just enough words to explain what is needed and no more. It is my experience that learners will not read a lot of text on a web page--they skim for the highlights. If the text on the page is long they may not read it at all. So it is important that my pages be concise chunks of just the most important instruction. Alessi and Trollip make a good case for transition statements between sections to tell learners if they are learning more about the topic at hand or going onto the next topic. Of course I will also make sure to use clear language with consistent use of terminology, follow conventions of grammar, spelling, and punctuation, and test my prototype on real users to make sure I am writing at the correct reading level.
As Alessi and Trollip point out, pictures (especially animation) capture learner attention and is more memorable than text. In addition to concise text instructions I plan to make frequent use of screen shots and animation to illustrate the process of creating a search, linking to full text, and finding call numbers. In some parts of the instruction the graphics will be the primary information but text will also be included for dual coding purposes, for those who learn better with text, and learners with a visual impairment or slow Internet connection . At this point I don't think I will be using graphics as analogy but may use graphics for organization or as a cue. I plan to use graphics that are authentic screen shots or animation and perhaps a simple chart of the research process. It is true that drawings and charts should avoid too much detail which could confuse learners and keep them from focusing on the most important points. I will keep any flash movie or video limited to 20-30 seconds to keep the learner interest and decrease download time. A flash movie or video will be useful for demonstrating and modeling. I will try to include the option of audio or text that the learner can control, but for the demonstrations the flash movie or video will probably be the most useful for the learner.
I plan to use color to make the instruction more interesting, attract attention to important information, and to convey to the user when they are leaving one section of the instruction and going on to another section. But this use of color will be redundant to transition text so that learners with color or visual disabilities won't lose information. At the same time I will make sure to not use too many colors (more than four), maintain good background/foreground contrast for all my text, and be consistent in my use of color. I found it interesting that Alessi and Trollip suggest avoiding red/blue, blue/yellow, and blue/green combinations. I thought it was only the red/green combination that was a problem for people with color blindness.
I plan to include links to procedural and context specific help through a "Help" button consistently placed on each tutorial page and use explanatory rollover text (alt tags) for all graphics and clickable functions throughout the tutorial. I will probably also include links to popup windows with word definitions from any words that learners may need additional information.
In the section on ending a program Alessi and Trollip recommend always giving the learner a way to temporarily leave the program and restart where they left off. Of course the learner can always close the web browser to quit. If I can find the appropriate javascript I would like to create a button that would let the learner quickly bookmark the last page they use so they can return to where they left. But if the script does not work with most versions of Internet Explorer and Netscape I could just give quick directions in how to save as favorites/bookmark. If the learner clicks on the exit button I will probably take them to a screen which makes sure they want to exit and makes sure they saw the option to bookmark the last page visited.
I want to use the tutorial for initial learning, guide the learner through some live searching for transfer of the learning and end with a self test created with Course Builder to assess the learning and provide feedback and reinforcement. Once the learner completes the assessment I will guide them to a page that gives their score, gives the credit for the tutorial, clearly states that the tutorial is completed, and gives the learner links to begin the tutorial again or return to Research Central.
Question: I was surprised that Alessi and Trollip said simultaneously reading text and hearing identical speech may decrease learning. Why is that?
This chapter focused on the factors important to the design and evaluation of tutorials. I chose this chapter to read as my instructional product will be in a tutorial format. Alessi and Trollip state that a good tutorial should provide information presentation/modeling of skills and guidance. My flowchart reflects plans for information presentation, modeling of skills through animated demonstrations, and guidance of the learner using the skills in a live PsycINFO search. I also plan to include assessment (the fourth phase of instruction) at the end of certain sections and at the end of the tutorial.
The general structure of a tutorial is the introduction, presentation of information, question and learner response, judging of learner response, feedback or remediation, then the cycle of presentation and questions continues. After all the information has been presented and questions responded to the tutorial ends with a closing screen.
My introduction needs to include the objectives for my tutorial. I will use my objectives to motivate learners by clearly and concisely demonstrating the relevance of the tutorial to the needs of the learner. To relate new information to what learners already know, I will compare PsycINFO to searching on Google but then explain why PsycINFO is better for finding articles in peer reviewed scholarly journals.
I do not plan to use a pretest as it may decrease learner motivation and cause apprehension. The only prerequisites of my tutorial is knowledge of how to use an Internet browser and computer mouse. Most if not all of my users already have that knowledge. By allowing the adult learner to pick and choose what sections of my tutorial to use they can decide what information they already know and what sections they should go through.
Questions and interactions in tutorials help keep the learner attentive, provide practice, encourage deeper thinking and assess recall and comprehension. Questions and interactions will hopefully also serve to motivate learners selection of program sequencing--if they don't understand the concept enough to answer questions correctly or complete the live search interaction they will be motivated to review content.
By having a live search and questions interspersed throughout the tutorial (at the end of each section) I will be following the advice of Alessi and Trollip to keep sequences of information presentation short and interspersed with questions or interactivity. Each section will only have 2 to 3 short screens of content before the questions or interactions. Hopefully this will enhance comprehension and recall and increase attention and motivation.
There are two basic types of questions. Alternate response or objective questions require the learner to choose a correct answer from a list. This type of question includes true/false, multiple choice, matching, and marking. These type of questions are easy for the computer to judge and provide easier ways to design feedback. Constructed response questions require the learner to produce rather than select a response. This type of question in multimedia includes completion and short answer questions. These are much harder for the computer to judge and for design of feedback.
I plan to use multiple choice questions in my tutorial. I was pleased to see Alessi and Trollip provided guidelines to creating multiple choice questions that can asses comprehension and problem solving. I plan to use CourseBuilder to create the questions and let the learner use a mouse for responding. I will provide three or four alternatives and make sure the incorrect alternatives are plausible and not obviously incorrect by spelling, grammar, or length of answer. Alessi and Trollip have a great idea of first constructing the questions as a short answer test then using the most common incorrect answers as the foils or distracters (incorrect alternatives). This will reinforce the correct concepts and help clear up any misconceptions. But I was unclear why Alessi and Trollip suggest not using alternatives like "none of the above" or "all the above" as I thought both of these alternatives were sometimes used for variation and to reinforce subtle differences and similarities.
I may also include some true/false questions for variation. True/false questions are easy to judge and provide feedback and easy for learners to respond to. Since they are easy to respond to, a learner can complete several true/false questions in place of one multiple choice question and have about the same odds of being correct by guessing. I probably will not use matching questions that use drag and drop best suited for simple factual or verbal type learning.
I don't plan to use constructed response questions in this tutorial due to the programming involved, but if I do this type of question in the future I will follow the authors rules of thumb and only blank out key concepts for fill in the blank, not have too many blanks, and not place the blanks at the beginning of the question. For short answer I will keep the responses short to prevent typos and to facilitate judging.
To more accurately determine learner comprehension of the information I will strive to make the questions not just the tutorials text rephrased into a question--what the authors call a verbatim question. I will use Alessi and Trollip's good suggestion of creating paraphrase questions that use synonyms or new application questions where the learner applies a principle or rule to a new situation. The new application type of question will mean that the learner will take the principles learned from the instruction to search a different topic during the live search. I will probably not have categorical questions where the learner applies principles or rules to a subordinate or superordinate class. Through user testing I will make sure the questions are clear, free of jargon or abbreviations, and are at appropriate reading levels. I will also avoid negative words (such as "not") which may be overlooked by the learner. I plan to use pop up boxes to provide learners feedback on their responses to tutorial questions. This way the learner can see and reflect on the question, their response, and the feedback.
Another factor to consider about questions is response economy--minimizing the amount of typing or physical activity needed for the learner to respond. Multiple choice and true/false questions where the learner responds with a mouse click have high response economy. Short typed responses prevent typing errors and are easier to judge then long answers. A typing prompt should be below a question and clickable answers should be obvious so the learner does not waste cognitive energy trying to figure out how to answer.
Judging is evaluating a learner response to give feedback. Types of responses include correct response, response with expected error (common errors), response with unexpected error, partially correct response, and non judged response such as the learner inputting their name. The multiple choice and true/choice questions I plan to use will be single selection responses that will be easy to judge.
Feedback is the programs reaction to the learners response. Most frequently the feedback is to inform the learner about the correctness of their response. Reinforcement follows a correct response and positive correction with information to improved future performance should follow an incorrect response. I plan to use immediate feedback with text and hints or explanations if the answer selected was incorrect. Since I will only have three or four choices in my multiple choice questions the learner will eventually get the right answer after a few tries. I will offer the learner who needs remediation a review of sections of the tutorial.
Alessi and Trollip discuss four types of information organization--verbal information, concepts, rules and principles, and skills. The PsycINFO tutorial will be mainly focused on skills. The tutorial will give step by step instructions, demonstrations, modeling, help on the difficult steps, and provide for practice.
My interface design for the tutorial will let students go through the tutorial in a linear fashion or pick specific sections of interest and choose their own branching. The subject matter is suited to a tutorial format with content arranged in a clear and logical sequence.
I will make the page controls obvious with buttons and arrows for "Back" and "Next". Learners will control the direction and timing of page changes. The continuous left menu will provide quick links to specific sections. The pages will not require the learner to complete the optional interactions to proceed.
I plan to do usability testing on five undergraduates (hopefully all psychology majors). I will use this feedback to improve the resource and determine if additional help beyond the initial instructions are needed. I plan to use rollovers (alt tags) on all graphics to provide the user additional information. I also plan to incorporate hints and information when the learner selects the incorrect answer.
Question: I was unclear why Alessi and Trollip suggest not using multiple choice alternatives like "none of the above" or "all the above" as I thought both of these alternatives were sometimes used for variation and to reinforce subtle differences and similarities.
In this chapter Alessi and Trollip provide an overview of a very useful model for design and development that is similar to the Instructional Systems Design (ISD) approach. The model is standards based, iterative (drafting, evaluating and revising), focuses on planning and management, guided by principles of cognitive psychology, progresses from discussion to implementation, emphasizes creativity, and encourages a team based approach. The model has the three attributes of standards, ongoing evaluation, and project management that run throughout the three phases of planning, design, and development.
Standards are the foundation of a good product. The project team standards or my standards are combined with the specific project and client standards. The standards must then be consistently applied throughout the project. Ongoing evaluation must be used to determine if the standards are being applied. Ongoing evaluation is an iterative process of design, testing, evaluation and revision.
Project management is good management of resources, money, and time through planning, monitoring, and communication. Often the difference between a successful and unsuccessful project is project management so I will need to be a good project manager to successfully complete the project on time and up to standard.
Alessi and Trollip then summarized the three phases that will be more fully discussed in the chapters that follow. The discussion on planning included defining the scope (including setting learning goals), identifying learner characteristics, finding out constraints, planning documents and style manual, collecting resources, brainstorming, determining look and feel, and getting client sign off. This type of planning is what I have been primarily focused on up to this point.
Design is assembling the content and deciding how it will be presented from both an instructional and interactive perspective. This involves developing content ideas, content analysis, prototypes, flowcharts and story boards such as I am working on for this week.
The third phase of development is taking the design and creating an effective product. This will include preparing the text, graphics, any audio and video, programming, support materials, doing alpha testing and revisions, and beta testing and revisions, client signoff, and validating the program.
Alessi and Trollip emphasized establishing expectations by having a formal method of reviewing program quality, having a clear budget of time and cost, and client sign off at specific stages of the process. This involves ongoing formative evaluation throughout the planning, design and development of the product.
In the remainder of the chapter Alessi and Trollip present and discuss a very useful evaluation form that I plan to use to guide formative evaluation of my product. The form includes evaluation of the products subject mater, auxiliary information such as introduction and conclusion, motivation, interface, navigation, pedagogy, behind the scenes data, robustness, and supplementary materials.
Alessi and Trollip gave a good amount of wisdom and very practical advice for evaluation of the above areas. Some of the highlights that I thought were especially relevant to my project included:
Question: Alessi and Trollip state their model is an empirical approach based on cycles of drafting, evaluating and revising. Don't they mean an iterative approach with several iterations of drafting, evaluating and revising?
Chapter 13 is about planning so the client and development team understand what the project is about and all the constraints that will impact the project. The first part of planning is defining the scope of the content, including the desired outcomes of the project, who is to learn what, and to what level of competence. It is important that this be put in writing and signed off by the client to avoid any confusion or "project creep". Project creep is when the client asks for changes that were not originally part of the plan. By agreeing to the abstract of the project, goals, and objectives, my client signed off on the scope of the project.
Part of planning is identifying the characteristics of the learners who will use the product. I am creating this product for a rather limited group of learners (undergraduate UGA psychology students) with for the most part similar age, educational level and reading proficiency. It has been my experience working with UGA undergraduates at the reference desk and in classes that most undergraduates have at least basic computer and Internet browsing skills and are proficient using the computer mouse. I have also learned that undergraduates can have the full range in level of motivation from those highly self motivated to those doing just the bare minimum to scrape by. I will seek to create instruction that will be motivational and very relevant and practical to meet the needs of undergraduates needing to find articles.
It is also important that my client understand the scope of the project, what to expect about determining the look and feel and interactivity, and the need for timely client review and sign off. I have been pleased that thus far my client has always responded to my requests for input within a day or two.
The constraints of my project are that it will need to run on the computers and software available in the Libraries and in campus EITS labs. It must be possible to download and install quickly and freely any plug ins needed to run my program. My program must be compatible with different version of Netscape and Internet Explorer and function acceptably when being accessed via a 56k modem. I will test my program and try to make it compatible with both PC and Mac but my highest priority will be to make it PC compatible as there are more PCs than Macs in campus labs and only PCs in the Libraries. My timeline constraint is the timeline imposed by the class and by my responsibilities of working full time and being a husband and father.
The section on determining the cost of the project will be most applicable later when dealing with a paying clients but even in this situation is useful to determine the time and resources needed. The sample costing guide included time and cost for content acquisition, developing standards, scripts, screens, underlying models, graphics, video, audio, interactions, software testing, project management, etc.
In the section on figuring cost for graphics the authors brought out the importance of allocating a specific amount of development time to each graphic. This is important because it is easy to spend more time than I should creating or finding early graphics with greater richness then intended then running out of time and not being able to do the rest of the graphics at the same level of richness. By allocating the time to each graphic in advance it is easier to stay on track and manage the time spent.
Alessi and Trollip point out that learner interactions with the program such as answering questions and playing games take more time to plan and implement than simpler presentation screens and thus should have more time allocated.
It is important to manage the budget, time and personnel keeping track of how much time and resources were allocated to each part of the project and how much time and resources were used. In this project I will have to mainly focus on how much time I have to complete the parts of the product and how much time I am spending so that I can make modifications as needed to keep within the time available.
Alessi and Trollip discuss creating a style manual which would be of most use to a team working on a project. In my case it can be a list of decisions regarding look and feel, color, active/passive voice, present/past tense, etc.
In the area of collecting resources I already have access to the database, online database help, training in information retrieval, experience working with undergraduates, computer tools, and instructional design resources.
Alessi and Trollip emphasize the usefulness of brainstorming, a process in which a small group works rapidly to produce as many ideas as possible in a nonjudgmental way. The group produces as many ideas as possible, recording each idea without evaluating its merit until the groups runs out of new ideas. The more ideas generated, the more likely it is that at least one of the ideas will be very good. I am glad that I will be able to present my ideas at the on the board sessions and get valuable input, ideas, and desk crits. I have done brainstorming with design teams before and found it to be very valuable. But like Alessi and Trollip said, it is very important to avoid the temptation to criticize the ideas generated during the brainstorming session as this will impede the creative process.
Another important part of planning is agreement with the client on the look and feel of the product. To make sure that the client and designer have a similar idea in mind it is good to share early with the client a screen design or prototype that demonstrates the basic screen layout and function. That is what I will be sharing with my client this week as I create and share screen designs and a flowchart.
Question: I was surprised to read that more development businesses fail because of inability to control project creep than any other reason. I would have thought economic reasons, school funding, and the skills of the development team would have a larger impact on the success or failure of a development business.
The role of the instructional designer during the design phase is to structure information so it can be learned effectively and design interactivity to maximize interest, learning and retention by the target audience. The instructional designer also produces documents that communicate effectively these ideas to the client, project manager, content experts, teachers, and production staff. Alessi and Trollip point out the best situation is when the designer can collaborate with all the above in the production of the design documents for increased feedback and ownership by all parties.
The first step is to develop initial ideas for the content through brainstorming the content (what is to be learned) and learning approaches (how to facilitate learning). Next the ideas are narrowed down to the most effective ideas. The choice of ideas is made by evaluating the brainstorming ideas based on learner characteristics, how well the ideas fit the subject matter and goals, the time needed to learn the content, restrictions of the delivery system, and the ability of production staff (in the present case just me!).
The most effective ideas that remain should next be analyzed. Task analysis looks at what the learner must learn to do (skills and behaviors). Concept analysis looks at the actual content, the information the learner must understand. The content analysis that I did for my project was based on a mental task analysis of what the learner must learn to do.
The purpose of task analysis is to break complex skills into component skills and determine an effective learning sequence. The learning sequence should begin with skills that the learner already has (basic computer, mouse, and Internet browser skills) then proceed to add new skills (database searching and finding the article online or in paper format) to their existing skills to accomplish the instructional goals and objectives. I broke down the new skills needed into the components of the content analysis.
Concept analysis is often used for declarative knowledge such as principles, rules, and verbal information. Alessi and Trollip define concepts as "a group of objects, events, or relationships that all share a set of common features" (p. 497). To teach the concepts it is useful to provide learners with examples of clear instances, clear noninstances, unclear instances, and unclear noninstances of group members (example: which are telephones and which just have some similarities to telephones?).
Alessi and Trollip advocate creating a preliminary program description. This includes identifying types of learning based on Gagne's five types of learning to help determine methodologies and other instructional factors. My tutorial will primarily be intellectual skills of problem solving through using procedures to find solutions to database searching problems (such as not finding enough articles or finding articles that lack relevancy). My tutorial will provide presentation and guidance, opportunities for the learner to practice on a live or simulated PsycINFO search, and assessment at the end of certain sections and the conclusion of the instruction.
To communicate my design ideas to my client and classmates I created a flowchart and a prototype of the screen design. A prototype is a representation of how the program will look and work. I did a representation of the introductory page and one of the section pages to show how the navigational elements on the left frame will expand to show the parts of the section selected. I may still look into creating buttons rather than text for the left frame. I will present the prototype to my client for her review and comments.
I also created a flowchart, a birds eye view of the structure and sequence of my program. My flowchart reflects the flexibility of allowing learners to go directly to the section of interest to them or go sequentially through the sections. Given the amount of variability in the route learners could take, I decided a level two flowchart was adequate. The best way to communicate my design and organization is to view the flowchart and prototype screens together. The flowchart also lets me work through various dry runs of the program before it is built to make sure all will logically flow and work.
Another way to communicate my design to my client is through a storyboard that provides a drawing of my design and notes about content. Alessi and Trollip suggest that the first step of creating storyboards is typing and revising all the textual content, questions, and feedback without separating the text into individual screens. They state that this approach will save time since revision will reorganize the screens into new boundaries. I disagree with this approach as I have already used the flowchart to break the concepts into logical divisions and manageable screen size. Visually seeing how the text looks on the screen will provide me added incentive to make sure the text is concise and emphasizes what is important.
Next Alessi and Trollip suggest writing and revising secondary text such as directions, menus, transitions, and exit messages. Then the storyboards can be created. I agree with Alessi and Trollip that frequent breaks in the text will keep learners attention better than long sections of text. They also wisely point out that it is instructionally more beneficial to have half the screen text and the other half a learning interaction than it is to have the entire screen text. Since I will be creating a tutorial on how to use a database I plan to include frequent graphics of the database interface, circles, arrows, flash movies demonstrating certain features, and live or simulated learner interaction with the database.
Alessi and Trollip have a good point that doing the storyboards on a computer may lead to focusing too much on the details and not the overview. I will probably take their advice of sketching the storyboards on paper first and include descriptions about the graphics that will be included.
Alessi and Trollip also have some good guidelines for breaking the text into the individual screens.
Once the storyboards are complete I will review the storyboards and the flowchart to evaluate the sequence, style, completeness, and length. I will also be looking for relevant graphics, amount of learner interaction, transitions, and any places where text could be enhanced by graphics. Then I will share the storyboards with my client, classmates, and some end users for formative evaluation. Storyboards, flowcharts, and prototypes should be approved by the client before development starts to avoid surprises and project creep.
Question: Do I have to go with the Alessi and Trollip suggestion of starting storyboards by typing all content without separating the text into individual screens? I have already used the flowchart to break the concepts into logical divisions and manageable screen size. I think visually seeing how the text looks on the screen will provide me added incentive to make sure the text is concise and emphasizes what is important.
Development refers to the process of producing, evaluating, improving, completing and validating an educational program based on a product design. Like planning and design, development should include the attributes of standards, ongoing evaluation, and project management. Project management includes time management and budget management. Alessi and Trollip suggest creating a chart that starts at the project deadline and works backwards plugging in each deliverable. I have used this idea before and found it to be a very valuable tool. Now that our class web site has all of our deliverable dates I need to create a chart like this. It is a very useful tool to determine how much time can be spent on different parts of the project while still making the deadlines. I must admit I need to use this tool so that I don't spend too much time trying to make parts of the product "perfect" to the detriment of later parts of the product.
Development will also include preparing the text and doing web page html programming (Dreamweaver is a great time saver). As suggested by Alessi and Trollip I will test my web pages on different browsers. My graphics will largely be cropped and augmented screen shots of PsycINFO and Flash demonstrations of database functions. I am glad Dr. Hill told me about the Camtasia (Camstudio?) software that can convert a desktop demonstration to Flash easily. This will be useful and save a lot of time as I am just learning how to use Flash.
In the section on producing video Alessi and Trollip make the good point that irrelevant video or video of a talking head are not very motivational or effective. Rather than large file videos I plan to make flash movies focusing on demonstrating how to effectively use PsycINFO. Perhaps I will include additional animation, text, and an optional voice over to accompany the flash movie. This will depend in some part on how effective and motivational undergraduate students find the demonstrations. When I use video in future projects I will take the books advice and experiment with using shorter clips, a smaller window, lower frame per second rate and mono audio to make the video accessible over slower dial up connections.
I found it interesting that audio used with animation can almost give the impression of watching video. I also learned that audio is possible on almost every platform and can be supported by even slow Internet connections. I will make sure that any audio I add will be optional and repeatable. Any text I use on screen with the demonstration and audio would be just summary text to avoid channel interference.
When I work on team projects such as in EDIT6210 I will need to heed Alessi and Trollip's advice about version control.
Support materials can include learner, instructor, and technical manuals and adjunct instructional materials. I plan to put on my tutorial's introductory web page the purpose and goals for the tutorial--why the learner should use PsycINFO and what they can learn from the tutorial. I will include on the introduction page and help page a link to download Flash and any additional free plug ins needed. I will probably add links to popup definitions and context specific help on certain pages. It will also provide the learner a quick reference guide to PsycINFO that they can print out at the end. I think it is much more effective to make a web site intuitive and provide help at point of need than it is to create a manual that probably few if any would use. But I understand that some instructional resources may need to include an instructor manual to provide the instructor information that learners should not have access to.
Part of the evaluation of my tutorial will be an alpha test that I will conduct (and hopefully some of my classmates will also provide evaluation in the form of desk crits). The alpha test will evaluate the programs content, flow, interface, navigation, affective considerations (such as motivation), and robustness. I plan to make good use of the evaluation form in chapter 12. Part of testing for robustness will be to ensure the product works regardless of what the learner does to it. I will need to check all links and buttons for functionality and also try unexpected interactions to make sure the product is robust enough to work with most any user. As I make revisions based on the alpha testing I will make sure not to introduce new problems while making revisions. If major revisions are needed then I will conduct a second round of alpha testing. I will do thorough alpha testing to make sure all is functioning properly before delivering the product to the client for beta testing.
Beta testing is a full test of the product by the client. In addition to the clients feedback a major part of beta testing will be getting feedback from undergraduate students that are psychology majors or who have similar characteristics. Alessi and Trollip suggest having three learners test the product, one who is similar to the best potential learner, one similar to an average learner, and one similar to the slowest learner. It may be difficult to ascertain which students that would be available to test the program would fall into what category. I plan to take Jakob Nielsen's advice on sample size for testing "the best results come from testing no more than five users and running as many small tests as you can afford. As you add more and more users, you learn less and less because you will keep seeing the same things again and again. After the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new" (Nielsen, 2000). I will ask the students about their familiarity with the Internet, GALILEO database searching, and other information to try to get student testers who have limited or no experience using PsycINFO and are representative of the programs learners.
I will observe the students individually using the program and make it more authentic by not providing assistance. I disagree with Alessi and Trollip's suggestion of having the student note comments they have. I think it is more productive to ask the student to "think aloud" their thoughts, decisions they make, any confusion, their reactions to what happens, etc. while interacting with the product. With explanation and some prompting the think aloud method could produce more information and be more natural than asking the student to pause and note questions. I plan to video tape the observations to record the users comments, behavior, and body language and where in the program these responses occurred. I will also take notes and ask the user after the observation to explain any comments or behavior I did not understand. Immediately after using the product is also an excellent time to ask the learner for suggestions on improvements and to find out how they felt about the tutorial. Alessi and Trollip had an excellent suggestion of asking the student if they would have finished the tutorial if I was not observing, and if they would recommend the tutorial to a friend. I will also look at the learners scores on the multiple choice test at the end of the tutorial to asses their learning.
Based on the feedback from the client and student testers I will make revisions. If the revisions are major I will run the revised product past the client and some additional students. Being a perfectionist I will need to heed the advice of Alessi and Trollip to end the testing and revision process when the program meets its purpose, not continually revising the program trying for "perfection".
The final client sign off will signify that all parts of the product are acceptable. As Alessi and Trollip point out it is not uncommon for the client to not pay much attention to the product until beta testing. That is why it is important to document all client sign offs of project decisions throughout the project. The client should be willing to pay for any changes outside of the agreed upon project details.
Validation is testing to determine if the program meets its goals in the real world learning environment. This summative evaluation occurs when the program is used by a large number of learners in their natural learning setting. Alessi and Trollip suggest this summative evaluation be based on Kirkpatrick's four levels of evaluation. The first level is assessing learner and instructor reaction and attitude--determining how much they like the tutorial. I will probably determine this by using a short web-based survey at the end of my tutorial.
The second level of evaluation is assessing learning. I will probably determine this by the scores learners obtain on the self test. But I am not sure how to program my tutorial to record this information (I hope this function is part of Course Builder). Also, the instructor will probably make the self test optional so I may not get many results.
The third level is assessing behavior change in the intended environment--can learners use what they have learned, is there a true transfer of learned skills into the real world. In order to enhance the transfer of learning I plan to include live searching of PsycINFO as part of the tutorial. Although hard evidence of behavioral change would be hard to obtain, I am hopeful that anecdotal evidence of students not needing as much help using PsycINFO will help determine the level of transfer.
The fourth level is assessing results and return on investment (ROI). Return on investment measures if the money spent designing and developing the program was worth the investment and what impact it had on the organization. Again I think I will only have anecdotal evidence to use for this type of evaluation.
I hope to also use the computer for evaluation of the product. Depending on the capabilities of Course Builder I hope to obtain information on the scores of those who take the self test. Since my product will hopefully be placed on the UGA Libraries server I also will be able to obtain statistics on tutorial use and individual tutorial page use.
Question: As it will be hard to determine which students are fast learners, average learners, and slower learners without bringing them in for an interview and product use, may I just use five undergraduate students with learner characteristics similar to the target audience? It is likely that the differences in the five students tested will be similar to the differences in the target audience.
Appendix A has useful storyboard forms for program design.
Appendix B has suggested table of contents for instructor, learner, and technical manuals. I plan to make my instruction intuitive enough and capable of running on most browsers so these manuals will not be necessary for this project.
Alessi, S., & Trollip, S. (2001). Multimedia for Learning: Methods and development. Needham Heights, MA: Allyn & Bacon, 3rd ed.
Nielsen, J. (2000). Why you only need to test with five users. Jakob Nielsen's Alertbox, March 19, 2000. Retrieved June 1, 2001 from the World Wide Web: http://www.useit.com/alertbox/20000319.html