A Step-by-Step Approach to Designing an Online Course for an Educational Web Site

• Introduction
• Steps:
  • One: Gather Resources
  • Two: Determine Learners
  • Three: Determine Course Content
  • Four: Establish a Pedagogical Approach
  • Five: Develop Organizational Framework
  • Six: Create User Interface
  • Seven: Create Course Components
  • Eight: Program It
  • Nine: Pilot Test
  • Ten: Launch
• Lessons Learned

In the Fall of 2002, the project team for ETR Associates' ReCAPP web site began to conceptualize its first online interactive course. The ReCAPP project team was interested in developing online courses because they offer several advantages over more traditional training interactions.

One major advantage of online courses is that they give users the opportunity to participate in a structured learning environment in their own homes or offices. In addition, online courses allow learners to work at their own pace. Online courses are also more cost-effective because they eliminate the time and expense of traveling to a training site. And finally, they can enhance learning by allowing for multiple types of interaction.

The ReCAPP project team chose the Behavior-Determinant-Intervention (BDI) Logic Model as the topic for its first course. Developed by ETR's Senior Research Scientist, Douglas Kirby, PhD, the BDI Logic Model is a program planning tool especially designed for public health practitioners to help them develop or modify intervention programs. In essence, BDI Logic Models are like "road maps" in that they show the logical connections or "causal pathways" between a health goal and the interventions selected to achieve that goal.

Despite the fact that we were dealing with a complex topic, the team chose the BDI Logic Model for its first online course for several reasons. First, we received a very positive response to Dr. Kirby's paper on BDI Logic Models. In fact, when posted on ReCAPP, the paper was downloaded approximately 3,500 times. Second, ETR was receiving more and more requests for training on BDI Logic Models and had limited resources to deliver face-to face trainings. Third, logic models support program developers in using science, theory and evidence in the development of programs — a practice highly valued at ETR. And fourth, the BDI Logic Model lends itself to e-learning in that it can be presented in a relatively straightforward way.

This paper will review the 10-1/2 month process of developing "A Beginner's Course in Developing BDI Logic Models" and share lessons learned from our course development experience. This article is not intended to serve as "the model" for developing an online course, but rather to share our experience with others interested in e-learning and online course development.

Developing an online course requires the skills of several different individuals. ReCAPP's online course development team consisted of several members representing expertise in project management, e-learning, BDI Logic Models and online course software.

The development team consisted of three core members: 1) ReCAPP's Project Director took responsibility for coordinating the online course project and also served as writer and editor-in-chief; 2) an instructional design specialist "translated" the original BDI Logic Model paper into an online course format and wrote the content of the course's seven lessons; and 3) ReCAPP's webmaster formatted the course, added static and animated graphics and programmed the entire course using Macromedia's Flash application.

In addition to this core staff, the team also included the developer of the BDI Logic Model (subject matter expert), an e-learning software consultant, a graphic artist, non-professional voice talent (four individuals), a sound engineer, and course pilot testers (three individuals). Coordinating the expertise of a team of professionals just described is essential to delivering a high quality product.

In addition to organizing an effective online development team, it's critical that the necessary software is obtained and that appropriate team members are trained in its use. For the type of online course developed by our team, the following software was required: Microsoft Word or a similar word processing application, Macromedia Flash, Macromedia Dreamweaver or a similar html formatting program, a full version of Adobe Acrobat, a vector-based graphic program such as Macromedia Freehand or Adobe Illustrator, and sound editing software (which, in this case, was provided by the sound engineer hired for the project). Depending on the type of graphics used in the course, a photo application such as Adobe Photoshop may also be necessary.

One of the most basic rules of training is to know who you are training. ReCAPP staff conducted two training assessment activities to learn more about individuals who would use the online BDI Logic Model course. First, in January 2001, we conducted an online user-survey to assist us in better understanding who was using the web site, how they were using it, and how they might like us to improve it.

The survey was sent through ReCAPP's mailing list which, at the time, consisted of 1,300 members. Of the 1,300 members, 191 individuals responded to the survey. Although we cannot call this sample truly "representative" of all ReCAPP users, we estimate that approximately 35% of ReCAPP's users are health educators (community based and school-based) and 30% are program administrators/planners. (The remaining users can be categorized as researchers, policy makers, social workers, clinicians and advocates.) Given that the majority of our users are health educators and program administrators, we chose to write the course with them in mind.

Once we knew the types of professionals who would most likely use the online course, ReCAPP's Project Director spoke in-depth with approximately 20 health educators and project administrators about the kinds of information and skills they would like to obtain from an online course on BDI Logic Models. This information was summarized and used to shape the content, support materials and "look and feel" of the course. For example, inclusion of downloadable worksheets at the end of each lesson, use of real life examples, links to important support resources (e.g. articles, web sites), and inclusion of "fun" animated characters were a result of feedback we received from talking with potential users.

The content for ReCAPP's online course was based on the work of ETR's Senior Research Scientist, Douglas Kirby PhD and his paper titled BDI Logic Models: A Useful Tool for Designing, Strengthening and Evaluating Programs to Reduce Adolescent Sexual Risk-Taking, Pregnancy, HIV and Other STDs. The online writing team studied and analyzed this paper on four levels:

• Deliver Most Important Messages. Kirby's paper on BDI Logic Models was reviewed for its most important messages. What are the essential components of BDI Logic Models? What is the essential background information? What information from the paper could be better passed on to the learner as a web link or appendix?
• Organize Information. Once the major pieces of information were prioritized, the team thought about how this information would be organized into lessons for a course. What was most important to present first? What would the learning objectives of each lesson be? What would the sub-topics for each lesson be? Answering these questions helped the team develop a course outline.
• Provide Opportunities to Develop Skills. Part of what makes online learning different from reading a book or manual is the opportunity to develop and practice skills. Once the content for the online course was established, the team identified interactive opportunities to teach skills (e.g., writing clear goal statements) relevant to developing a logic model.
• Simplify Language. Dr. Kirby's BDI Logic Model paper is written in an academic style using complex sentences and public health jargon. While thinking about our learners (health educators and program directors), we reviewed our prioritized content, sentence by sentence, and found ways to simplify language, shorten sentences, add explanation/definitions when appropriate, and "chunk" information using bullets or lists. In addition, our team added written examples, diagrams, graphics, animation and/or voice to concepts that we thought would benefit from reinforcement.

The paper was eventually translated into seven lessons consisting of more than 220 screens of information in 86 Flash movies. An additional section of the course provides the learner with an orientation of how the course is designed rather than on the subject matter. The seven lessons are listed below.

Lesson 1: Overview and Benefits of the BDI Logic Model
Lesson 2: Getting Ready to Develop a BDI logic Model
Lesson 3: Step One - Establishing a Health Goal
Lesson 4: Step Two - Selecting Important Behaviors
Lesson 5: Step Three - Identifying Determinants of Behaviors
Lesson 6: Step Four - Identifying Intervention Activities
Lesson 7: Getting the Most from Your Logic Model

Consistent with any training designed for professionals, adult learning principles were used in the design of the online course. Below find some examples of how adult learning principles were incorporated into the course's pedagogical approach.

• Adults learn through repetition and reinforcement. Key messages are repeated throughout the online course. In addition, the major points of each lesson are repeated by one of the course's character's known as "Dr. Data" in a "Words of Wisdom" section that ends each lesson. Learners are also given an opportunity to check their learning at the end of each lesson through an interactive quiz.
• Adults learn best when learning is applied to real life examples. Plenty of examples and illustrations are used throughout the course to better communicate abstract ideas or concepts. Learners are also strongly encouraged to complete worksheets at the end of each lesson. Worksheets are designed to help the learner apply the BDI Logic Model to a program they are developing at home. A listserv was also established to provide a forum for course takers to exchange ideas and ask questions about developing a logic model. The listserv also allows learners to reflect on their learning with their peers and trained moderators.
• Adults learn best when they are in a comfortable learning environment. The online course was developed so that learners can complete the entire course in one sitting or in various stages. The learner proceeds at his/her own pace. The navigation designed for the site allows users to move through out the course with relative ease. Resources such as web sites, articles, papers, and other information are provided throughout the course with hyperlinks. The course was designed to be a "one-stop shopping" learning experience with all the information needed to complete the course available with no more than a few clicks.
  
  Finally, the humor brought into the course by periodic visits from the "BDI Bunch," a group of animated characters who reinforce learning, brings enjoyment to what might be considered an "academic" topic.
• Adults have different learning styles. A three-part learning process was applied to all the major concepts in the course — Teach, Apply and Feedback. After a particular concept is defined and explained, an example or illustration may be used to bring real world context to the idea. Learners then have the opportunity to apply what they have learned in two ways.
  
  Each lesson is accompanied by a "Plan It!" worksheet that prompts learners with questions on information presented in the lesson. The "Plan It!" worksheets are designed to take the learner through a step-by-step process that will eventually lead him/her to a completed logic model. Feedback on the work completed on the worksheets is available from ReCAPP through continuing education credits or through the course's listserv.
  
  The user is also given the opportunity to assess comprehension of the major points made in the lesson through an interactive quiz called "Got It" at the end of each lesson. Quizzes are designed in a true/false format. When a question is answered correctly, the learner immediately receives positive reinforcement on his/her answer. A learner's incorrect response prompts an immediate explanation of the correct answer.
• Adults learn best when complex information is broken down into basic and easily accessible steps. When writing and formatting the online course, the development team followed the principles of Information Mapping, which is based on research into how the human mind actually reads, processes, remembers and retrieves information. These principles include:
  • "Chunking" information according to Information Mapping's "7 plus or minus 2 rule." Research tells us that the human mind processes information best when there are no less than five and no more than nine "chunks" of information at a time. The online course was organized and presented accordingly. It was divided into an optimum of seven lessons. In addition, steps within each lesson were "chunked" according to the same formula.
  • Presenting information so it is clear and easily accessible. Given the technical nature of the topic, it was especially important to keep both content and layout simple. Care was taken to keep sentences and paragraphs short, use bullets when appropriate, select non-technical vocabulary (10th grade level or less), format with large, easy-to-read fonts, and leave sufficient white space whenever possible.

Determining a consistent approach to presenting the course content is important. Each of the seven lessons of the BDI Logic Model course is organized in a similar and predictable way making navigation and integration of information easier for the learner. Each of the seven lessons contains learning objectives that inform the learner what he/she will be responsible for learning. Each lesson provides a summary of major points in a "Words of Wisdom" section located toward end of each lesson. The worksheet associated with each lesson ("Plan It!"), as well as the assessment quiz ("Got It?") is found at the end of each lesson.

As soon as the general framework and structure of each lesson was determined, the development team created a storyboard for each section of the course. A story board is a self-explanatory graphic representation of the course. It serves as an important visual roadmap to aid the team when planning and writing.

User interface refers to the system that enables users to interact with the course — the section headings, layout of the menus, the general navigation plan and the assistance functions.

In keeping with learning theory and the principles of Information Mapping, technical staff developed a user friendly interface and navigational structure. Clear and consistent prompts and navigation buttons are incorporated throughout. These include: consistent color coding; easy-to-use instructions that can be skipped; flashing "next" buttons that tell the users when they may move on; a course menu on every page with submenus that pop up on mouse rollover; heads and/or subheads on every page so users always know where they are; and easy-to-download worksheets and templates.

Even before they begin the course, users are sent to a log-in page where they are given clear instructions on what software and hardware they will need (i.e., sound capabilities, recent web browser, Flash Player, Adobe Acrobat, fairly fast computer processor). Also, if they decide to access a link to another web site or resource page, the destination opens in a separate window so users can easily return to where they left off in the course.

Elements that allow for user flexibility were also incorporated. For example, users have the option to turn the sound off or on, go through or skip instructions, answer or bypass quizzes that test for comprehension, and, in general, easily move between sections and subsections as they wish. Added benefits include the fact that the action stops on each page so that users can print each page. Also, the need to scroll down a page has been eliminated.

Course components, or assets, refer to all content to be programmed into the course. The components for this course included text documents, artwork of the BDI Bunch, and audiotape of voice-over narration. We avoided components like video that require a lot of bandwidth and slow download time in order to accommodate users who work with low-end equipment (a concern when serving some non-profit organizations).

One important course component in our BDI Logic Model course is the voiceover narration, which reinforces the key messages and allows for auditory learning in online courses. We used voiceover narration in key course sections — at the beginning of each lesson to read the learning objectives of that lesson, at the end of each lesson to summarize key points and guide users to next steps, and during the comprehension quizzes after each lesson to reinforce the correct answer or explain why an answer is incorrect.

Audio was also used for the BDI Bunch — the animated cartoon characters who appear in each lesson to reinforce information and add humor. The audio was recorded in a quiet area of our office by a professional sound engineer who brought recording equipment and sound blankets. The budget did not allow for professional voice talent, so a group of ETR employees lent their voices to the project.

Because it required more expertise than was currently available in-house, the infrastructure of the course was developed by an expert Flash consultant. Working in close consultation with the project team, he developed templates for each lesson with an easy-to-follow system for inputting content as well as the animated introduction to the course. He also developed the self-study quizzes at the end of each lesson using templates available in Flash. After that, the webmaster with beginning to intermediate training in Flash spent about two months building the course.

The course was pilot tested three times with health educators employed by ETR who were not familiar with BDI Logic Models. Each pilot tester was asked to take the course from start to end in one sitting. However, they were not required to complete the "Plan it!" Worksheets at the end of each lesson. Each pilot test took approximately three hours. Pilot testers were asked to comment on navigational ease, readability/understandability, flow, graphics, animation, and voice as they took the course.

In addition, pilot tester learners were observed on how they navigated through the course, body language and timing. Several improvements were made to the course as a result of the feedback we received during pilot testing. Pilot testing is an important phase of developing an online course and should be given appropriate time and attention.

"A Beginner's Course in Developing BDI Logic Models" was launched on ReCAPP on June 17, 2003. The course was promoted through various national and international listservs for health educators and program administrators/planners working in various public health fields. The course's log-in page has given the project team the ability to track the number of people logging into the course and the organizations they represent.

To date, approximately 1,500 people have logged into the course, but since it's possible to bypass the log-in page, we estimate that more than 2,000 people have actually accessed the course. The course includes an optional ten-question feedback form that provides opportunities to learners to tell us what they liked about the course and ideas about improving it. To date, course feedback has been very positive with consistent messages about usefulness of content, navigational ease, understandability and practical/downloadable resources.

1. On-going teamwork is essential.
   The development team needs to work together from the beginning of an online course project in order to conceptualize the large picture of the course and then meet periodically for updates and problem solving. The needs of the different members of the team should be articulated and addressed on an ongoing basis to maintain effective and efficient planning.
   
   The work of each of these individuals is dependent on the other. For example, the course writer benefits from coordinating with the webmaster who will format and code her work for the web. The writer also benefits from working closely with the subject matter expert to make sure content is accurate and correct. And course text needs be synchronized with the voices recorded by a sound engineer.
2. Be conservative in your estimates of time needed for course development.
   If you have limited or no experience in developing online courses, it's not unrealistic to double or even triple initial time estimates. Because this was our first experience and learning curves were great for everyone on the team, the project took much longer than expected. The online course team worked together (part-time) over a period of 10-1/2 months. Approximately 1,200 hours of staff and consultant time was used to develop the course. Given what we have learned, we estimate that our next online course would require about 25% less staff time and financial resources.
3. Work as a team to establish course organization early.
   Making minor changes to course content such as rewording text or modifying graphics is relatively simple — no more difficult than rearranging the furniture in a room. Reorganizing a course after formatting is underway is like moving the room to the other end of the house. It's much more time consuming because it requires a considerable amount of backtracking. For this reason, it's essential that the entire team is involved in developing and approving a clear and workable structure to the course before actual construction begins. After that, the actual content can be reviewed and approved by the team as it is being written.
4. Be realistic about staff and consultant capabilities.
   When working with new technology, such as Flash software, it's a good idea to have an expert consultant on hand to problem solve. It's equally important to choose your consultants carefully. You need to find people who are not only knowledgeable but are a good philosophical and stylistic fit.
5. When developing a budget, be aware of possible hidden costs.
   If doing some technical work in-house, include problem-solving and possible training time by consultant(s) in the budget. If possible, purchase software for all aspects of the production (i.e., sound editing) so you're not relying on a consultant for last-minute changes. Save time by rehearsing video and audio before scheduling recording sessions and budget in money for audio and video retakes.
6. Remember that interactivity is what makes a course distinctive.
   An online course is not the same as an online presentation. As mentioned earlier, the course development team used a variety of adult learning principles to maximize learning. These included: building in repetition and reinforcement; including real life examples and opportunities to exchange ideas and ask questions; using humor to create a comfortable learning environment; responding to the fact that people have different learning styles by giving users choices and using a combination of text, graphics, animation and audio to present information in a variety of ways; and providing information in basic and easily accessible steps.
7. Pilot or beta testing is key to good usability.
   When you're part of the development team, it's difficult — if not impossible — to be objective about the usability of a course. You are simply too familiar with the material to properly judge if the navigation is user friendly or if the information is clear. Therefore, it's essential to conduct user testing. Observing how a member of your target audience interacts with the course can often reveal problems your team would never have considered. To minimize the possibility of substantial makeovers, it's a good idea to produce and pilot test one lesson before you sign off on templates or begin to build the rest of the course.

For more information about the Resource Center for Adolescent Pregnancy Prevention web site (ReCAPP) or "A Beginner's Course in Developing BDI Logic Models," contact ReCAPP's Webmaster, Pat Rex at patr@etr.org.