Serendipity35

Blackboard's data science people have done a study on the data from all that student clicking in their learning management system and aggregated data from 70,000 courses at 927 colleges and universities in North America during the spring 2016 semester. That's big data.

But the results (reported this week on their blog) are not so surprising. In fact, their own blog post title on the results - "How successful students use LMS tools – confirming our hunches" - implies that we shouldn't be very surprised.

Let us look at the four most important LMS tools they found in predicting student grades. As someone who has taught online for fifteen years, it makes sense to me that the four tools are the ones most frequently used.

On top was the gradebook - not the actual grades, but that students who frequently check their grades throughout the semester tend to get better marks than do those who look less often. "The most successful students are those who access MyGrades most frequently; students doing poorly do not access their grades. Students who never access their grades are more likely to fail than students who access them at least once. There is a direct relationship at every quartile of use – and at the risk of spoiling results for the other tools, this is the only tool for which this direct trend exists. It appears that students in the middle range of grades aren’t impacted by their use of the tool."

Next was their use of course content. That makes sense. Actually, I would have thought it would be the number one predictor of success. Their data science group reports "An interesting result was that after the median, additional access is related to a decline in student grade; students spending more than the average amount of time actually have less likelihood of achieving a higher grade!" That's not so surprising. Students spending more time (slow or distracted readers; ones who skimmed and need to repeatedly return to material etc.) are probably having problems, rather than being more though. The student who spends an hour on a problem that should take 15 minutes is not showing grit.

This is followed by assessments (tests etc.) and assignments. "If students don’t complete quizzes or submit assignments for a course, they have lower grades than those who do so. This was not a surprising finding. What was surprising to me is that this wasn’t the strongest predictor of a student’s grade." Why is that surprising? Because it is what we use to evaluate and give those grades.Digging a bit deeper in that data, Blackboard concludes that time is a factor as a "...strong decline in grade for students who spend more than the average amount of time taking assessments. This is an intuitive result. Students who have mastered course material can quickly answer questions; those who ponder over questions are more likely to be students who are struggling with the material. The relationship is stronger in assessments than assignments because assessments measure all time spent in the assessment, whereas assignments doesn’t measure the offline time spent creating the material that is submitted. Regardless, this trend of average time spent as the most frequent behavior of successful students is consistent across both tools, and is a markedly different relationship than is found in other tools."

The fifth tool was discussion. I have personally found discussions to be very revealing of a student's engagement in the course. I also find that level of engagement/participation correlated to final grades, but that may be because I include discussions in the final grade. I know lots of instructors who do not require it or don't grade it or give it a small weight in the final grade.

An article on The Chronicle of Higher Education website is a bit unsure of all this big data's value. "But it’s hard to know what to make of the click patterns. Take the finding about grade-checking: Is it an existential victory for grade-grubbers, proving that obsessing over grades leads to high marks? Or does it simply confirm the common-sense notion that the best students are the most savvy at using things like course-management systems?"

And John Whitmer, director of learning analytics and research at Blackboard, says "I’m not saying anything that implies causality."

Should we be looking at the data from learning-management systems with an eye to increasing student engagement? Of course. Learning science is a new term and field and I don't think we are so far past the stage of collecting data that we have a clear learning path or solid course adjustments to recommend.

Measuring clicks on links in an LMS can easily be deceiving, as can measuring the time spent on a page or in the course. If you are brand new to the LMS, you might click twice as much as an experienced user. Spending 10 minutes on a page versus 5 minutes doesn't mean much either since we don't know if the time spent reading, rereading or going out to get a coffee.

It's a start, and I'm sure more will come from Blackboard, Canvas, MOOC providers (who will have even greater numbers, though in a very different setting) and others.

I came across this "Web Accessibility MOOC for Online Educators (WAMOE)." Although there are no upcoming dates for the next live offering of WAMOE in D2L Open Courses, the site at https://weba11ymooc.wordpress.com contains all course content and activities that have been used in the MOOC. 

I know that many online educators are interested in learning about how to make their courses more available to students who struggle with web accessibility issues, but may not know where to go for information. This free MOOC, sponsored by Portland Community College and D2L, provides a free professional development opportunity to help eLearning professionals meet the challenges of improving accessibility in online learning.

I believe that using this resource in conjunction with a F2F cohort of faculty on a campus would be an excellent way to approach the topics contained in the MOOC. As with any learners in a MOOC, a hybrid approach to using the online resources is often the best approach to increasing completion of the c"course" and eliminating some of the frustrations of learning on your own.

NOTE: Much of the course content was originally developed by Karen Sorensen and other staff at Portland Community College. PCC also freely shares their comprehensive document titled “Web Accessibility Guidelines.” All course content is open licensed by CC-BY, NonCommercial, Share-Alike 3.0 with attribution to Portland Community College and D2L Corporation.

I found it interesting that when The Chronicle of Higher Education assembled its list of national trends for its Trends Report, they included instructional design as one of them. It's odd to think of it as a "trend" since ID did not start in education, it has been around for several decades and it has a big role, especially in higher education, today.

Instructional design started during World War II with the armed forces. It came from a need to provide technical training to large numbers of people efficiently.

Having worked in instructional design formally since 2000, I have seen the field change during the past 15 years. I subscribe to a few of those job alert websites and every week I see more openings for designers. Some of those jobs are in academe and even more are in industry. Most major companies now use instructional designers to develop employee training materials.

In higher education, instructional design is likely to have started at a college as a way to prepare distance-learning and extension programs. Those programs initially appealed to non-traditional students with family and work obligations and often as a distance from the physical campus that made attending classes difficult.

As the proportion of those students increased and as the technology to deliver courses became more sophisticated, online learning became more popular. Its acceptance by faculty lagged behind its acceptance by students. Designers who worked with faculty helped gain acceptance as they learned what an ID could do to actually help design their course for online delivery.

The share of students taking online courses has gone from less than 10 percent in 2002 to 28 percent in 2014, according to the Babson Survey Research Group. Babson also found that the percentage of academic leaders who see online learning as critical to their institution’s long-term strategy went from about half to nearly two-thirds. And that is why the one of their ten trends is to say that there is increasing importance and visibility for instructional designers.

A professional ID is needs technical ability, design skills, pedagogical knowledge, and the interpersonal skills to work 1:1 with subject matter experts - SMEs, or in this case, faculty. 

In my early days of managing an ID department, we often met faculty who were told that they had to "teach my course online" and who fully expected to just digitize all their regular face-to-face materials. They would ask us to scan hundreds of pages of handouts and readings, create or convert PowerPoint slides, and they wanted to videotape their usual 90 or 180 minute lectures. It was a very big learning curve.

A few saw the opportunity to translate their in-person courses to be offered online as an opportunity to really rethink the course objectives. In those early days, all faculty had to learn technical skills, especially whatever the current course management system was that the college was using. (Those often changed, much to their dismay.) 

For me, the best outcome over the 16 years that I worked in instructional design was that we were viewed not as just "the people who do online courses" but also as a department that could help improve the quality of teaching, whether in online, in-person, or hybrid courses. 

Having myself been trained as a K-12 teacher and doing graduate work in pedagogy, I was initially surprised at the lack of knowledge that professors had in that area. I shouldn't have been surprised since they always told me that they never took an education course and tried to "do the things my best teachers did and avoid the things the bad ones did." Objectives versus goals, rubrics, Bloom's Taxonomy and almost all of the things I had been taught and used in my secondary school classroom were brand new to higher education faculty. My knowledge about pedagogy needed to be diplomatically transferred to professors, but the best ones were intrigued and eager to know not only how to teach online but why to teach in new ways.

Most parents and educators who are aware of Minecraft as a way to teach programming, critical thinking, design thinking and other skills, and that it is something many students love to use. I came across a presentation by Jarred White that he has delivered at security conferences on "Threat Modeling the Minecraft Way."

He talks about this unusual application of Minecraft as a tool for threat modeling and the parallels between information security and Minecraft's game mechanics, for example, asset protection. 

There is a video online and at about 14 minutes he compares Minecraft's "threat agents" to real-world security attacks: creeper monsters are very similar to denial-of-service attacks; skeletons shooting arrows are similar to remote code execution; zombies are similar to viruses. 

That means that players can also design passive security measures, such as architectural feature that can block spiders from climbing up walls.

Not only is this approach an interesting one for younger students to examine, and perhaps will be an introduction to the next generation of security workers. It can be also used at higher levels and ages to model the basics of cybersecurity, work in 3D spaces in a real-time simulator. 

This is Jarred's presentation from the Bsides 2016 information security conference