A couple weeks ago I was having coffee with my friend Rob Reynolds who works outside of OU but uses Reclaim Hosting for his education technology business projects. He asked me a really good question which was “How do you specifically measure the success of OU Create?” I’ve learned that this is a really tricky question in education technology. Yes, we need to measure what we are doing. Yes, we need to prove our value, so-to-speak. But what does this mean within technologies whose goals are not necessarily to “improve X” or “be adopted by X% of faculty”?
A danger I find in edtech is that there are mechanisms you can build into your technology in efforts to build whatever narrative out of the data that you want to build. It is believed that we can answer these types of questions by tracking clicks, surveilling our users, and then “discovering” the answers within the data. And this entire premise runs counter to what we believe is a core reason for why OU Create exists–to provide a space where users can learn about data, data ownership, data privacy, and data transferability.
Here’s what my answer was to Rob’s question and here’s how I hope to answer this question. With any new technology or technology paradigm, we need to be more like R&D than we do IT. To me that means that the more we are investigating new questions–to continue to be in a mode of inquiry–the better. This is something I’ve come to realize I’ve always believed, but have just recently started to learn how to articulate after spending time with good people like Kristen Eshleman and Amy Collier.
We're experimenting with assessment in R&D this year: 'Exploring Meaningful Measures of Accountability' https://t.co/bvYi805UvZ
— Kristen Eshleman (@kreshleman) August 23, 2016
What’s needed, according to Dave Snowden, are “ways of measuring success without knowing in advance what that success may be.”
“So how do these new questions arise?” you ask. I believe that you have to continue to have your ground to ear and be in constant communication with your community.
In nearly every talk that I gave on OU Create this year, I’ve said that the platform does not do exactly what we had expected it to do when we adopted it. In fact, it does much more. We started OU Create with the thesis that it could serve as a platform for students portfolios and open courses in a very similar manner to how University of Mary Washington originally did it. But it is, after doing this for a few years, indeed, much more nuanced than that. And if I was to make one specific goal for OU Create in 2017, it would be to, better communicate that.
This post, a sort of end-of-the-year wrap up, is a step towards trying to pull together a better aggregated story as to what has and is happening with OU Create–though it’s really only a start. Yes, it still leans on metrics, but please don’t get distracted by those too much. They are merely a tool that serves as an anchor. Below the numbers I’ve linked to relevant blog posts, which I’m personally more interested in. I’d rather build towards stories and relationships than numbers. But numbers do make it easy to build an infographic… So one of those is at the bottom too.
Big Bucket Metrics
We had 1,691 domain orders in 2016. In total, we’ve had 3,767 orders since August 2014, so 45% of our user base joined this year.
Many of our users install applications on top of their domain to run it. The top five applications are WordPress, Drupal, Omeka, DokuWiki, and OwnCloud. Those five are followed by Scalar, MediaWiki, Moodle, Joomla, and Grav. It’s important to note that not all of our users utilize the applications in Installatron. We have many users that are using static HTML sites.
12,017 of our 24,466 all-time blog posts came in 2016. In 2015, we had 9,323 blog posts and 3,126 from August to December of 2014. I’m curious as to the amount of active bloggers, but this appears to me like people are doing different activities than blogging or else we would have seen bigger blog numbers. This is one metric that tells me that Domains is more than blogging on WordPress.
But that brings to me something anybody who gets in this space has to learn. The answer to the question of “What is everybody doing?” is “We don’t know.” And we’re proud that it’s that decentralized. We can give insight to what we come across but we don’t always have easy ways of coming across it. As an example,within the last few days we’ve discovered an open course called American Gangster: From Jay Gatsby to Jay Z that utilizes Hypothes.is and someone utilizing OU Create to host their own self-organized ListServ.
The Creaties
In April , we held the first annual Creaties, an award show dedicated to celebrating and showcasing the very best work on OU Create. Winners took home awards in categories such as Best Portfolio, Best Course Site, Best Wiki, and Best Community. 102 sites and posts were nominated, 2,683 online fan votes helped produce 31 total winners.
Further readings:
Creaties Website
Blog post: Welcome to the Creaties
Blog post: Recapping the OU Creaties
Support
This year we focused on building more robust support for OU Create users. In January, revamped support documentation was rolled out using Documentor. This documentation is Creative Commons licensed and is currently being utilized by Middlebury College, Coventry University, and Georgia State University. In February, we began supporting users with Freshdesk, an online customer support and ticketing solution, which notifies support staff automatically through an integration with a CTE Slack Channel. This year we responded to 112 support tickets.
Further Readings:
OU Create Support Docs
Blog Post: Integrated Customer Support and Slack
This Week on OU Create
In 2015, we started a blog that curates what our team considers the “best of” that particular week. Anoop Bal, who originally started the blog, left us for grad school (which is totally awesome) and, after a brief pause in publication, we’ve continued the blog with his spirit. This year we wrote 19 blog posts and highlighted 80 different users.
Further Readings:
This Week on OU Create Blog
Blog Post: The New (and Improved!) OU Create
Blog Post: This Week on OU Create is Back.
Student Projects
Student projects are difficult to summarize because there are too many to make a coherent statement about them. Students used OU Create this semester to build photography blogs, promote their music, promote their businesses from OU’s Integrated Business Core, chronicle study abroad, publish a student-run newsmagazine, run for campus office, build digital exhibits, maintain open lab journals, build databases about campus amenities, build digital mapping projects, host secret dinner societies, write movie reviews, etc etc etc. One my favorite parts of students on OU Create is the way in which they’ll weave in and out of academic work. Some of it starts in a class and ends up in a very different space and that’s really fun to watch.
For the infographic, we’ve highlighted the course Making Modern America: Discovering the Great Depression and New Deal as well as the Global Engagement Fellows. The first, which we often refer to as simply New Deal, had students build out an Omeka site that contains 967 artifacts (or “items”), both historical and modern. Global Engagement Fellows blogged from 32 different countries this year.
We also highlighted a project in which we partnered with the Graduate College to do their first workshop series which focused on Digital Identity.
Further Readings:
New Deal Omeka Site
Global Engagement Fellows
OU Create Graduate Workshop
Blog post: Talking Digital Identity and Scholarship with Graduate Students
Blog post: Exploring My Own History via a Class Omeka Project
Faculty Development
This is another “bucket” which is hard to tell the complete story as nearly everything in which I’ve discussed previously in this post has some level of faculty development. We hosted four unique workshops for faculty that focused on OU Create and specific uses for them whether that was a way of teaching or, since we are currently migrating to Canvas, how to use OU Create inside of Canvas.
We also do classroom visits where we’ll show up to any classroom and either talk about OU Create or help students set it up. This year we did 34 of those.
We started doing more one-on-one consultations as a partnership with the Vice President of Research whose office has an initiative to assist faculty in build public presence of their identity, specifically as it related to Broader Impacts.
Tangentially, and not included on V1 on this infographic, John Stewart and Keegan Long-Wheeler ran two really excellent Faculty Learning Communities around gamification. The first, GOBLIN, is about gamification and is an OER available on OU Create. The second had faculty learn storytelling strategies and use Twine in conjunction with OU Create to build seven text-based games.
GOBLIN Faculty Learning Community
Experience Play (XP) Faculty Learning Community
Blog Post: Research and Support Sites in OU Create
Blog Post: Teaching Digital History Methods with OU Create
Blog Post: Creating a Digital Museum of the Tulsa Race Riots
Blog Post: Twine and Serious Gaming
Blog Post: GOBLIN as an OER
Blog Post: A Brief History of GOBLIN Faculty Development
Blog Post: Anatomy of Slaying GOBLINs
I’ve likely left out a lot in this post, but promise to continue building on top of this structure, so stick around for more. If you’re interested in chatting about Domains or OU Create, feel free to contact me. Also, considering coming to Domains 17, a conference we are hosting in OKC with Reclaim Hosting on June 5 and 6, 2017.
And here’s the fancy infographic (download high resolution version). All of the inspiration for this goes to Laura Gogia who has been doing some beautiful infographics on open dissertations. If you see ways to improve this or have questions, please let me know.
Infographic credits: Created with the free infographic vector kit from Medialoot and with icons from the Noun Project by Lil Squid, Wilson Joseph, and Joost Har.
Featured image: flickr photo shared by kurichan+ under a Creative Commons ( BY-NC-ND ) license
