Presenters:
John Doherty — Instructional Designer, Northern Arizona University
Wally Nolan — Lead Instructional Designer, Northern Arizona University
Both presenters started out by stating that online learning should be all about the pedagogy and not about the technology. In other words, instructors should not become so enamored of one particular technology that they forget to apply sound pedagogical principles when presenting content and engaging their students.
Like Missouri S&T, everyone at Northern Arizona University (NAU) is given a Blackboard course every semester. However, not everyone uses them equally well (again, like Missouri S&T).
John and Wally have encouraged faculty to use more Blackboard resources
by integrating their course materials online with the campus library
resources, which can easily be included in a Blackboard course.
Blackboard can easily link to outside web resources, so instructors who
want to link to the S&T library resources can also do so.
The
presenters showed a simple Venn diagram illustrating the intersection
of the social, teaching, and cognitive presences, where learning takes
place. For real learning to take place, you need to have all three
factors available. Another way to interpret this is that learning is
best when there is student-student (social) interaction,
student-instructor (teacher) interaction, and student-content
(cognitive) interaction.
Backwards design, the real focus of their presentation, is a fairly
simple concept to understand, yet seems so counter-intuitive sometimes:
- Identify the desired results — Figure out the desired learning outcomes for the concepts in the course.
- Determine acceptable evidence — How will the students demonstrate mastery of the material?
- Plan learning experiences and instruction — These are the guideposts down the path towards achieving the learning outcomes.
Yes, as with most projects, designing a quality online learning
experience will take a great deal of effort, planning, and preparation,
but the payoff is almost always worth it. Students will get more benefit
if they know what the learning outcomes should be and are able to
substantially work towards accomplishing that goal.
To help instructors in backwards course design, John and Wally developed a series of online tutorials:
By using these web sites, instructors can get a good feel for the
process of backwards course design. EdTech will undoubtedly provide our
own tutorials in our CyberEd course in the not-too-distant future (I
hope!).
To sum up: every action in a class should tie directly to the learning
objectives for the course. The only role technology serves is as a
“force multiplier”, allowing instructors to reach more students in
diverse environments, expanding the options for quality content, and
enabling instructors to meet the individual learning needs of each
student most efficiently.
Peggy Brown — Director of Instructional Design / Adjunct Faculty, Syracuse University
Michael Fudge — Senior Systems and Network Support Administrator, Syracuse University
A few select members of EdTech had the opportunity to attend Blackboard World 2011 in Las Vegas (July 12-14). Bb World is a fairly large conference hosted by (of course) Blackboard. However, even though many of the sessions touched upon Blackboard, either directly or indirectly, others covered topics that are applicable to a wide range of teaching situations, where which LMS instructors use is not relevant to the discussion.
One such session was called: Go Rubrics! Creating A+ Rubrics.
The basic idea behind a rubric is that it is a way of evaluating and measuring student work according to some pre-established criteria. In other words, it is one more performance assessment, like tests or quizzes. However, unlike a multiple-choice test where students are either right or wrong, it is NOT simply a checklist and is often highly subjective. But it is still possible to use a rubric to set clear expectations between students and instructors. Students are often frustrated with instructors because they don’t know exactly what an instructor is looking for in a paper. Therefore, the feedback they receive on their first paper is used as the
“baseline” for future assignments. Using a rubric, especially one that is available for students to view, gives students a much better idea of the standards that will be used to gauge performance in the class.
Using a rubric also gives students the opportunity to judge their own
work, or, if peer-review workshops are part of the class, the work of
other students. Students can focus their efforts on relevance and
quality within the assignment, if they are able to see how much each
criteria is worth for the assignment. For instance, content and
relevance might be worth most of the points on the assignment, while
fewer points are assigned to grammar and style. Students will also
receive consistent feedback from one assignment to the next (if they are
weak on grammar/style, or don’t provide sufficient detail in their
responses, the instructor can note this in repeated assignments, giving
the student more opportunities to correct for their deficiencies in one
area while remaining strong in another).
From an instructor’s
point of view, using a rubric has many benefits. Instructors can use the
rubric to outline learning objectives in the course. Are the students
“getting it”? Rubrics also make it easier to evaluate students more
equitably by using an established guideline for quality. Finally,
rubrics allow for much quicker grading through formalized guidelines.
Blackboard 9.1 SP 6 (which is NOT released on campus at the time of this
blog post) includes a handy rubric feature that can be used in blogs,
wikis, and other assignments within Blackboard for very, very quick
evaluation and grading of student posts.
Creating a “good” rubric
is often an iterative process. Once the initial rubric is created, it
is then refined over time to become more effective. Student input often
helps in this process. Rubric criteria should be tied directly to the
learning outcomes for the assignment and for the course. Also, the
rubric should contain numerous definitions as well as some examples of
quality work to illustrate the expectations from the instructor.
Rubrics
have been criticized for a variety of reasons. Some instructors believe
they take too much effort to create. There is definitely some truth in
this in that a good rubric takes a lot of thought and energy. However,
as with many tasks associated with quality instruction, an initial
investment before a class starts can lead to significant savings in time
and energy later when it is time to grade assignments. Rubrics can be
difficult to get just right, but again, this is often an iterative
process that may take several tries and refinements. Another criticism
is that a rubric may shape student work into a certain format. This is
not necessarily a bad thing, especially if an instructor is looking for
students to grasp concepts in a certain way. The rubric should
illustrate the “baseline” for acceptable work in a class. Students who
consistently go above and beyond the confines of the rubric through
their own innovation and initiative should be rewarded for their hard
work.
There are two basic styles of rubrics: holistic and analytical.
A
holistic rubric is simple, one-dimensional, and has a single
description scale. For example, a holistic rubric for a discussion board
might have five criteria: relevance, conciseness, engagement, clarity,
and timeliness. Students must meet all five criteria to get full credit
on a discussion board posting. Otherwise, they only get credit for the
criteria they actually meet (a post might be timely and concise, but
lack relevance, be unengaging, and somewhat difficult to understand).
Holistic rubrics are fairly quick and easy to create, but lack the depth
needed to fully rate student performance. They are best used for things
like discussion board postings and other short assignments where the
instructor is just looking for some basic understanding from the
student.
Analytical rubrics are two-dimensional matrices with a
full range of criteria and performance levels. This is what we typically
think of when we talk about rubrics. Criteria for an analytical rubric
might include relevance, clarity, grammar/style, direction-following,
and so forth. It is important to define what a “0” means within the
context of this rubric as sometimes it is not very clear. Also, examples
of the different performance levels are often very helpful to students.
They do not have to be long and complex examples, but should be
representative of what the instructor is looking for on the assignment.
One of the nice things about an analytical rubric is that the different
criteria can be “weighted” so that students who meet the higher-weighted
criteria while missing some of the lower-weighted criteria can still
demonstrate proficiency in the learning outcomes. Analytical rubrics are
definitely more complicated to produce, but the benefits can absolutely
be worth it.
Rubrics are NOT the only way to measure academic performance in a
course, and may not lend themselves to every discipline. For instance, a
rubric doesn’t make a whole lot of sense for an introductory calculus
class where students spend most of their time solving math problems.
However, they can be a VERY effective tool in more subjective
disciplines where students must demonstrate mastery of concepts through
writing exercises and other activities (e.g. presentations). Students
generally respond very favorably to rubrics because they have a clearer
picture of the expectations for quality work in a course and can
therefore dedicate their time and energy to meeting the criteria.
Instructors who have used rubrics for a while also tend to like them as
they DO speed up the grading process considerably.
EdTech uses the Quality Matters rubric, which is a weighted analytical
rubric, to evaluate the quality of online courses. We can provide
detailed feedback for the critical items required to meet the QM
standard, so it definitely saves us time and energy.