Vancouver Trip Summary

I spent Thursday and Friday of last week with my lovely colleagues in Vancouver. Some things to note:

  • The Vancouver office is awesome, especially the art (h/t David Ascher’s wife)
  • Thanks to Jennie and the rest of the YVR team for making me feel welcome around the lunch table!
  • Luke promised to play guitar but he never did :(

Here’s how the two days went down:

  • Sabrina and I started off by having a morning meeting with Michelle via Vidyo. This produced several clarifying insights including the use of “portfolio” as the key metaphor for Clubs pages in the MLN Directory. This helped shaped our conversations during the rest of my visit.
  • Sabrina and I then reviewed what we already know about our audience, our programs and offerings, and value adds for the user.
  • We then sketched out a model for an engagement funnel

IMG_20150821_150739

    • Then we got to work on the MLN Directory model. We came up with streamlined sketches for the various content types, thinking in terms of mobile-first.
      • Member profile:
        • See field listing
        • Implied functionality: certain Leadership roles might be auto-applied (e.g. if the user owns an approved Club page, the system can apply the “Club Captain” role), while others might require an admin interface (e.g. Regional Coordinator, Hive Member). We’d like to allow for flexible Role names, to accommodate local flavor (i.e. Hive Chicago has specific role names they give to members).
      • Club and Hive pages:
        • Club page field listing
        • Hive page field listing
        • A key insight was that we should treat each distinct entity differently. That is, Club pages and Hive pages might be quite different, and we don’t need to try to force them into the same treatment. We also recognized that our MVP can simply address these two specific types of groups, since this is where our programs are focused.
        • We decided that focusing on Reporting for Clubs would be the highest value functionality, so we spec’ed out what that would look like (wireframes coming soon)
        • For Hive pages, we want to re-create the org listings and contact cards that the current Hive Directories have
  • We also met with Laura de Reynal and David Ascher to hash out plans for the audience research project. More on that soon, but you can see our “most important questions” at the top of this pad.
  • The issue of badges came up a few times. First, because we found that the plan for “Club Captain” and “Regional Coordinator” badges felt a little redundant given the concept of “roles.” Second, because we saw an opportunity to incentivize and reward participation by providing levels of badges (more like an “achievements” model). Seems like our colleagues were thinking along the same lines.

All in all, it was a really productive couple of days. We’ll be getting wireframes and then mockups out to various stakeholders over the next heartbeat, along with hashing out the technical issues with our engineering team.

Feel free to share any comments and questions.

Pledge to Teach Survey Results (first month)

At the very end of June, we added a “Pledge to Teach” action to teach.mozilla.org. After people complete the pledge, they are able to take a survey that allows us to find out more about their particular context for teaching the Web, and their needs.

I’d like to share results from the first month, during which 77 people completed the survey, out of 263 users who took the pledge (29% response rate).

First, we asked what people are interested in, in terms of teaching the Web, and provided some options for people to choose from (people could choose as many as they liked).

  • Starting a Mozilla Club: 57%
  • Getting access to Digital Literacy curriculum: 79%
  • Running learning events (Maker Party, Hive Pop-Ups, etc.): 61%
  • Professional Development to help me get better at teaching digital skills: 81%
  • Connecting with a network of peers: 77%
  • Exploring tools to make the Web: 66%

The results from this question align with our strategic plans to develop more curriculum, provide more professional development offerings, and build tools to help people connect with one another.

We asked about the age range of learners:

  • 6-13: 40%
  • 14-24: 78%
  • 25-34: 44%
  • 35-44: 26%
  • 45-54: 17%
  • 55+: 19%

These findings align with the age-range that most of our existing curriculum is optimized for (14-24). That said, we know our audience is broader, and that content can be adapted for different audiences. Certainly we have community members that work in the K-12 space and higher ed. Given the numbers for learners over 24, we may consider more intermediate/advanced web literacy content, and/or address this audience with more in-depth Teach Like Mozilla and MDN content.

We asked how many learners people expect to reach this year:

  • 0-50: 32%
  • 51-100: 22%
  • 101-200: 26%
  • 201-500: 5%
  • More than 500: 14%

This data speaks to the fact that survey participants are more likely individuals who have direct interactions with learners, vs larger partners with wider networks. The survey was intended to reach individual educators/mentors, but we might consider a similar survey directed to partners, too.

Note: we’ve since added a question to the survey that will allow us to know how many learners people reach at any one time. This will inform our curricular design process.

We asked about the contexts in which people teach (again, respondents were able to choose multiple answers):

  • At standalone events (for example, a one-day workshop, hackathon or Maker Party event):  51%
  • In a classroom during the school day: 52%
  • As part of an afterschool or summer program: 31%
  • With my friends and family at home: 56%
  • At professional meet-ups or training events with other adults/mentors/educators: 51%
  • At a college or university: 27%
  • At a library or other community space: 26%

Some of these results surprised us. For example, the responses for teaching at home and with friends is higher than we’d expected, as were the number of people teaching in professional meetups. If these trends continue, they will inform our curricular and professional development content offerings. We are also having a Web Literacy Training Fellow join us later in the year, and she will specifically address these contexts.

These findings also show that people are teaching across various contexts, which may speak to some leadership pathways (e.g. classroom teachers also hosting standalone events to reach more people).

Finally, we asked people about their motivations for teaching the Web. Here is a sample of those responses:

  • The Internet is a place where any information is available, and people ought to know how to access the best of the information they seek, and know how to protect themselves beyond anti-virus programs. I want the opportunity to teach and engage with learners and peers outside of the classroom. (Canada)
  • I always believe that I should never wait for opportunities to help other people rather I should let myself open doors to help others. I want to share a part of what I know to people who wants to learn more about the digital world. (Philippines)
  • I think technology especially the Web could be a wonderful facility to awaken and support children being creative and using free thought as a positive means of fully participating in communities, society and the world. (UK)
  • Am driven by the passion to make the world a better place. I want students from my school to have extra skills apart from the normal curriculum taught in school. (Kenya)

This is a very small sample so far, but we’ll look at results for the second month soon and see if trends continue.

In the meantime, the results of the survey will inform several of our next steps, including:

  • Consider iterations on site information hierarchy and calls-to-action
  • Create content strategy that reflects community needs—this includes everything from site copy to blog content to curricular content to social media
  • Advance partner strategy given these insights

We are also starting a new research effort with support from the Webmaker product team, to complement the survey. The project hasn’t been fully designed yet, but will likely help us dig deeper into questions about our community’s assets, needs, and contexts.

Quick update: engagement on the MLN Site

Pledge to Teach

In my last post, I mentioned that we had recently launched the Pledge to Teach the Web. Since we launched it three weeks ago, 240 people have taken the pledge.

Screen Shot 2015-07-24 at 3.25.11 PMScreen Shot 2015-07-24 at 3.26.32 PMScreen Shot 2015-07-24 at 3.27.10 PMOf those who’ve taken the pledge, about a quarter of them have also completed a survey that we sent as a follow-up. The survey is helping us gain a better understanding of our audience, their contexts for teaching, and their needs. We’ll share an analysis of the survey results next month.

Site Traffic

Since we launched teach.mozilla.org back in April, we haven’t been particularly focused on driving traffic to the site. That changed recently, as we began our Maker Party promotion efforts in earnest. We started promoting Maker Party on both beta.webmaker.org and on mozilla.org. Those two referrals, along with our email campaign, led to our most highly trafficked week on the site since launch, during the lead-up to Maker Party. Our highest day was July 13th, when we had over 11K sessions. Since the initial bump, traffic has dropped back down again to between 1200 and 2500 sessions per day.

Unsurprisingly, the Maker Party page is the most popular content, after the homepage. The Activities page is the next most popular.


What we’re doing next with regard to user engagement

  • Adding analytics tracking to several things so we can better measure conversion rates.
  • Experimenting on pledge flow to increase conversion rate. One possibility is to make the pledge the only CTA on the homepage.
  • Determining our post-Maker Party strategy for people who take the pledge. We’re discussing ideas here.
  • Experimenting with “Community” link to increase Discourse activity. This is a larger-than-the-site effort , though. We can be promoting Discourse across all of our work.

Whistler Wrap-up

What an amazing week!

Last week members of the Mozilla community met in beautiful Whistler, BC to celebrate, reflect, brainstorm, and plan (and eat snacks). While much of the week was spent in functional teams (that is, designers with designers and engineers with engineers), the Mozilla Learning Network website (known informally as “Teach”) team was able to convene for two meetings—one focused on our process, and the other on our roadmap and plans for the future.

Breakthroughs

From my perspective, the week inspired a few significant breakthroughs:

  1. The Mozilla Learning Network is one, unified team with several offerings. Those offerings can be summarized in this image: Networks, Groups, and Convenings.MLN ProgramsThe breakthrough was realizing that it’s urgent that the site reflects the full spectrum of offerings as soon as possible. We’ve adjusted our roadmap accordingly. First up: incorporate the Hive content in a way that makes sense to our audience, and provides clear pathways for engagement.
  2. Our Clubs pipeline is a bit off-balance. We have more interested Club Captains than our current (amazing) Regional Coordinators can support. This inspired an important conversation about changes to our strategy to better test out our model. We’ll be talking about how these changes are reflected on the site soon.
  3. The most important content to localize is our curriculum content. To be fair, we knew this before the work week, but it was definitely crystallized in Whistler. This gives useful shape to our localization plan.
  4. We also identified a few areas where we can begin the process of telling the full “Mozilla Learning” story. By that I mean the work that goes beyond what we call the Mozilla Learning Network—for example, we can highlight our Fellowship programs, curriculum from other teams (starting with Mozilla Science Lab!), and additional peer learning opportunities.
  5. Finally, we identified a few useful, targeted performance indicators that will help us gauge our success: 1) the # of curriculum hits, and 2) the % of site visitors who take the pledge to teach.

Site Updates

I also want to share a few site updates that have happened since I wrote last:

    • The flow for Clubs has been adjusted to reflect the “apply, connect, approve” model described in an earlier post.
    • We’ve added a Protect Your Data curriculum module with six great activities.
    • We added the “Pledge to Teach” action on the homepage. Visitors to the site can choose to take the pledge, and are then notified about an optional survey they can take. We’ll follow up with tailored offerings based on their survey responses.pledge

Questions? Ideas? Share ’em in the comments!

Medium-term roadmap

Earlier this week, I wrote about the short-term roadmap for teach.mozilla.org. Now I’d like to share a few details about what we envision a little farther out (Q3 and into Q4).

Badges

The bulk of the work here will be improving the user experience for both badge applicants and badge reviewers. We’ll also be rolling out some new badges that are aligned with our programmatic plans, and will recognize the key volunteer roles we’ve identified (i.e. Regional Coordinators and Club Captains).

Directory

I’m really excited about this project because it will transform the site from simply being a place to find resources to a community, and because we’ll be able to offer more customized experiences for users once we know more about them. The Mozilla Learning Network Directory will include rich mentor profiles and group pages (where “groups” include Clubs, Hives, and organizations), as well as the ability to search and browse. The initial build will also include a full integration of Discourse. (We’re drawing heavily on the Hive Directory for inspiration.)

Curriculum functionality

It’s been a long time coming, but soon we’ll begin designing a more permanent solution for making our curriculum content dynamic. This will include adding basic user interactions (“Likes,” ratings, comments), as well as dynamically facilitating the creation and display of remixes and translations. We’ll likely also have a tool for users to create and share their own playlists, and to submit curriculum for consideration.

Ongoing iteration on the engagement flow

We’ll continue to learn what works in terms of connecting people quickly to what they need, and we’ll likely continue to make changes as a result of those learnings. Our engagement strategy will get some serious power behind it as we move forward with the email tooling project that’s happening in parallel.

Thimble!

Finally, our team at CDOT are actively working on making improvements to Thimble, our open source code editor for teachers and learners. We wrote about those improvements a few weeks ago.

Short-term roadmap for teach.mozilla.org

Teach.mozilla.org was released into the wild at the end of April, and we spent part of the last month doing a round of iterations based on early feedback.

The most notable iteration is the homepage. Our goals were to clarify the three main actions a user can take, incorporate the blog, and use more plain language across the page. Do you think we succeeded?

homeI want to share an updated short-term roadmap, so you can know what’s coming up in the immediate future:

  • The next thing we’ll release is a revised workflow for new Clubs. The workflow reflects our evolving thinking about the organizing model. It will work like this: people will submit their Clubs on the site, they’ll then be connected with a Regional Coordinator who will work with the Club Captain to refine the description if necessary. Once approved, the Club will be visible to everyone who visits the site. For more on what a Regional Coordinator is, check out Michelle’s blog post.
  • Next up: Since the three main actions require a pretty heavy lift for users, we want to provide a lower-effort action. We’re going to test out a “Pledge to Teach” action that simply allows the user to, well, pledge to teach the web in their community.
  • Maker Party is right around the corner. We’ll be promoting it on the homepage, and making some changes to the Maker Party page itself, including adding case studies of successful parties, and linking to the recommended Maker Party activities for this year.
  • The Clubs page will get a makeover soon, too. The goal is to provide better scaffolding for people who are thinking about starting a Club, as well as to provide a more useful experience for people who are running Clubs.
  • We’ll keep on adding more curriculum as it’s tested and finalized!
  • We’re starting the localization effort! Check out this globalization ticket for an overview of all the different pieces of this project.

I’ll blog about the medium-term roadmap soon!

User Testing the Teach Site

We are soooooo close to releasing the new Teach site.

People seem to dig the bright colors and quirky illustrations throughout the site.

People seem to dig the bright colors and quirky illustrations throughout the site.

In advance of the release, I wanted to conduct some user tests to make sure we’re still on the right track. This week I conducted two user tests with members of the community (yay!). As is always the case with user testing, I learned a lot from observing users interact with the site.

You can see detailed notes here and read my recommendations below.

These recommendations are based on formal user tests with two users as well as feedback from people who’ve been involved or observing the process throughout.  Also, please note that I wasn’t able to test the primary functionality on the site (adding a Club to the map), so these recommendations are more about IA and other content issues.

Findings and Recommendations

Screen Shot 2015-04-10 at 2.48.07 PM

ACTIVITIES / CURRICULUM / RESOURCES

Findings:

  • People want to see more activities and resources.
  • People expect to be able to sort and filter.
  • Our internal distinction between the Clubs Curriculum (the official curriculum for Clubs; with a strong recommendation for following the prescribed path) and Teaching Activities (more “grab and go”-style) is not intuitive to users.
  • The Teach Like Mozilla content needs to be more integrated into common user flows.

Recommendations:

  • Continue with current plan for developing and publishing more approved curriculum and activities.
  • Continue brainstorming work around scalable presentation of curriculum begun in this heartbeat. The ideas discussed so far address sorting and filtering, and make good use of the Web Literacy Map as an organizing tool.
  • As part of that design work, we should also allow users to access all teaching materials from the same page, and provide specific views for “official Clubs curriculum.” I recommend we keep the Teaching Activities page, and remove the Clubs Curriculum sub-page. This content is one of our primary offerings so it belongs at the top level. /cc @iamjessklein
  • We need to offer a solution for sharing resources—e.g. maker tools, other curricula, programs. (Hello, Web Lit Mapper!)
  • We need to design a stronger connection between teaching activities and the Teach Like Mozilla content. A short-term solution might be to link to the TLM page from every individual activity page, but we should also be working towards a better longer-term solution. /cc @laurahilliger

Screen Shot 2015-04-10 at 2.50.46 PM

CLUBS

Findings:

  • The Clubs Toolkit is not findable, and needs to be supplemented with content targeted towards helping people “get started.”
  • We are not providing enough information for the use case of a person who is deciding in the moment whether to start a Club.

Recommendations:

  • Make the Clubs Toolkit more visible on the page.
  • Consider renaming the Clubs Toolkit something like “Getting Started Guide” or “A Club’s First Month” – and editing content to match. /cc @thornet
  • Based on my understanding of the expected pathways to starting Clubs, I do not think we need to make any significant changes to the Clubs page to address the use case of someone coming to the site and deciding in the moment whether or not to start a Club. As I understand it, our plan for growing Clubs makes use of the following scenarios:
    • 1) Someone is “groomed” by staff member, Regional Coordinator, or other community member. By the time they arrive at the site, they have the specific intent of adding their Club.
    • 2) Someone finds out about us through Maker Party, and through a series of communications learns about Clubs and decides to start one. They are coming to the site with the specific intent to add their Club.
    • 3) Someone with an existing program or group wants to be listed in the database. Again, they are coming to the site with the intent to add their Club.

In short, I don’t think we’ve yet seen a reason to have the site serve a “selling” or persuasive function. I *do* think the Clubs page is a natural first stop for someone who is looking to understand how to start a Club. I think the changes recommended in the bullet points above address that.

Screen Shot 2015-04-10 at 2.51.21 PM

EVENTS/MAKER PARTY

Findings:

  • The copy describing the Events page in the main navigation is misleading, since the content on the Events page is about Maker Party.
  • People may understand throwing a Maker Party as a “first step” to starting a Club, rather than a lower-bar option for people who do not want to start a Club (and perhaps never will).

Recommendations:

  • I think we should re-brand what is currently the Events landing page as “Maker Party.” We’ve already sort of done this in that, while the page is called “Events” in the nav, the h1 copy in the hero image is “Host a Maker Party.” I suggest we change the copy in the nav to “MAKER PARTY” and the teaser copy to “Our annual global campaign”. /cc @amirad

Screen Shot 2015-04-10 at 2.52.06 PM

INFORMATION ARCHITECTURE

Findings:

  • Users tend to ignore, not see, or misinterpret the CTAs at the bottom of every page
  • Users do not notice links to the sub-pages in the main navigation

Recommendation:

  • We need to design better, more intuitive pathways for viewing secondary pages

ADDITIONAL RECOMMENDATIONS

I’m going to keep banging this drum: We need to clarify our audience! I think we’ve made good progress in terms of clarifying that our “first line” audience includes educators and activists. But I think we have to take it a step further and clarify who those educators and activists are working with. There are at least two axes that I think are important to be clear about: first, the global nature of our work, and second, the specific age groups of what I’m calling the “end learners,” for lack of a better term.

I think we do a pretty decent job of conveying the global nature of the program through copy and imagery, though obviously implementing our l10n strategy is absolutely fundamental to this.

I think we are less clear when it comes to the age groups we’re targeting with our programs and materials.   For example, I think we ought to specify the appropriate age level for each activity. (And the images, activity titles, and copy should reflect the target audience.)

Questions, comments, disagreements wholeheartedly welcomed!