Benutzer:Benedikt Schäfer (WMDE)/new-editors-test/Key Findings and Results 2017-2018

Overview

Activities

Associated Research

Available Materials for Adoption

Go back to 'Key Findings and Results' overview page.

2017 Campaigns Overview

This page contains key findings and results from the first five campaigns from 2017 to early 2018 in English. For the overview of the campaign pages in German, please click here. (this page is work in progress)

2017 Motivation and Scope

Our motivation was to stop the downward trend of new editors (>10 edits). Therefore, we planned four campaigns throughout 2017, including the January campaign in 2018. With these campaigns, we tested the underlying assumption that online campaigning on Wikipedia can attract Wikipedia readers to get involved and become an editor (>10 edits).

As this was our first attempt to attract users with online campaigning in Wikipedia, we decided to use an iterative approach: We ran the first two campaigns on a low banner diet (between 5-20%) to learn more about the effects of campaigning.

Questions we were interested in:

• Which effect does online campaigning have on readers and editors of Wikipedia?
• What works and what does not work to attract new users and motivate them to edit Wikipedia (banner design, calls to action, user journeys, videos, approaches to “learn Wikipedia”, etc.)?
• Which means are there to measure campaigns?

After running the first campaign on a low banner diet, we used the findings to create a big campaign with an 80% banner diet. With that approach we made sure to gain crucial insights in the beginning and to use only the best practices on a big scale and therefore guarantee the most positive impact of campaigning.
Back to top

Key Findings and Results 2017-2018

• Use simple and actionable calls-to-action (CTA) for banners (e. g. You can improve the accuracy of Wikipedia! CTA: Learn how to improve articles).

• Use a factual and clean design for banners to attract new users. This is perceived as more accurate and professional than colorful and creative banner designs (and is more accepted by Wikipedia readers).

• Explain very briefly and clearly the next steps to edit Wikipedia on a landing page. Videos help people to understand and learn.

• Guided tours help people to get an overview of Wikipedia’s “buttons” before starting with the first edit.

• Campaign periods not during holidays. People use Wikipedia mostly on Mondays and least on weekends (at least in Germany).

• Get as much feedback as possible from the community and if possible from external organisations to test messages and designs before running a campaign.

• Implement daily monitoring of the key indicators to adjust at any given moment.

• User journeys with just one option work best: banner → landing page → registration
  

Back to top

Campaign results in detail

Initially we planned four campaigns with an ascending banner diet:

1. Thank You Campaign in January 2017 (100% → Please note: The main part of the banner is from the Fundraising team aimed to attract new members of our NGO. Only one small button directs the users to our landing page for new editors)
2. Spring Campaign in April 2017 (5-10%)
3. Autumn Campaign in October 2017 (80%)
4. Thank You Campaign in January 2018 (100% → Please note: The user first sees a banner from the Fundraising team. After four impressions the 'new editors banner' gets displayed.)

While conducting the first two campaigns, we found that we need more testing and information to execute a large-scale campaign in autumn with a banner diet of 80% or more. Therefore we developed an additional fifth campaign:

• Summer Campaign in July 2017

Below you can find the results of all campaigns in detail:

'Thank You' Campaign 2017

Banner

Brief Explanation:

Our first campaign to attract new editors was supplementary to the Fundraising team's 'Thank You' campaign. The original 'Thank You' banner said thank you to the donors of the fundraising campaign. The redesigned banner contained three Buttons:

• 'Thank You' text
• Become a member of Wikimedia Deutschland e. V.
• Get involved as an editor

With every campaign we wanted to answer a set of questions.

Questions:

• Do illustrations support the message / call to action?
• Are introductory videos videos helpful for new editors?
• Which of the videos is more attractive to users?

Campaign Parameters:

• Length: 6 days (January 1 - January 7, 2017)
• banner diet: 100%
• User journey: banner → landing page → registrations

Learnings:

• Illustrations attract people to get involved.
• Wikimedia Commons is not suitable as media source for campaigning. The number of clicks could only be measured with a complicated workaround. In the meantime, we solved that problem with a video tool. An additional problem was that the videos need to be rendered at different screen resolutions. That takes considerable time. Hence, the videos need to be produced at least one week before using them.
• Tracking of registrations was not possible – only clicks on the button on the landing page.
• There were too many options in the 'thank you' banner. It was impossible to direct as many people to the registration as we assumed.
• In addition, the landing page had too many options to get involved. The potential new users didn’t really know what to do next.

Landing page (German): https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/Mach_mit

Back to top

Spring Campaign 2017

Banner - correct mistakes

Brief Explanation:

With the second banner campaign in spring we wanted to attract users with one simple call to action: correcting errors in Wikipedia. In contrast to the first campaign we used a very simple and plain design.

Questions:

• How appealing is the design of the banner and the landing page for new editors?
• How does a simple call to action appeal to new editors?
• Does a guided tour help to get started in Wikipedia?

Campaign Parameters:

• length: 10 days (April 12 - April 22, 2017)
• banner diet: 5 - 10% (of guest users)
• user journey: banner → landing page → registration → guided tour

Learnings:

• A simple call to action works very well (significantly higher conversion rate).
• How to successfully implement tracking: to find a solution to that problem we needed know-how about SQL from our Software team and it took copious amounts of trial and error with the Campaigns Extension)
• The planned impression and registration figures were not reached. Idea for future campaigns: implementing live monitoring of impressions and registrations (creating the possibility of raising the banner diet while running the campaign).
• On a short term scale, the guided tour did not influence the number of new editors' edits. But numbers are not high enough to prove this finding.

Landing page (German): https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/Fehler_korrigieren

Back to top

Summer Campaign (additional) 2017

Banner 1 - factual approach

Banner 2 - emotional approach

Brief Explanation:

As described in the Motivation and Scope, we decided to use an iterative approach in order to learn about the aspects of a successful banner campaign with every campaign. Having conducted two campaigns did not equip us with all the information we needed to run an autumn campaign on maximum scale. We decided to add a summer campaign to obtain additional information (see Questions).

Questions:

• Which approach is more appealing to potential editors: emotional illustration and text or a factual look & feel?
• Does live monitoring in Wikipedia work?
• Which of our video tutorials works best?
• Does a guided tour really does not make a difference to the new editors edits?

Campaign Parameters:

• length: 8 days (July 11 - July 18, 2017)
• banner diet: 20 - 30%
• user journeys:
  • banner (emotional) → landing page (emotional & not wiki) → registration → guided tour
  • banner (factual) → landing page (factual & on-wiki) → registration → guided tour

Learnings:

• The emotional design does not cause significantly higher conversion rates compared to the factual design.
• Daily monitoring works.
• The message of the banner was not intelligible to users. Furthermore, the German Wikipedia community criticized the emotional message.
• The video on top of the page was the one most clicked.
• The click rates of videos were higher on the external (not on-wiki) landing page.
• The Tracking of external pages is difficult and not comparable to wiki page views.
• Guided tours have a slightly positive effect on new editors' edit numbers.

Landing page (German):

1 (factual) https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/Entdeckungen-sortieren

2 (emotional) https://entdecke.wikipedia.de

Back to top

Autumn Campaign 2017

Banner 1: More Precise Articles

Banner 2: Add Pictures

Banner 3: Add Sources

Banner 4: Get Involved

Shortly explained:

The previous three campaigns aimed to include all important learnings for the autumn campaign – as this was the campaign with the highest banner diet. In this campaign we used our findings and decided to test further with four different banners. We used a quite formal and factual design with three simple call to actions for each banner to get started:

• You can make Wikipedia more vivid! CTA: Learn how to add pictures to articles (German: Du kannst Wikipedia anschaulicher machen! CTA: So bebilderst du Artikel)
• You can improve the accuracy of Wikipedia! CTA: Learn how to improve articles (German: Du kannst Wikipedia genauer machen! CTA: So überarbeitest du Artikel)
• You can improve the reliability of Wikipedia! CTA: Learn how to add citations (German: Du kannst Wikipedia noch verlässlicher machen! CTA: So ergänzt du Belege)

Additionally to the three simple entry points we decided to just call for registration in the fourth banner.

Questions:

• Do the users need further information on a landing page to start editing Wikipedia?
• How does just a call for registration perform compared to the three different calls to action that have an additional landing page explaining next steps?
• Which of the three simple entry points works best to get people start editing?

Learnings:

• Currently a proper a/b testing is not possible with central notice options. Hence four (more than two) banners are shown randomly to probably the same users instead of showing only one banner repeatedly to only one reader.
• users provided with more detailed information and a concrete call to action are more likely to register and also to edit after registering
• the following call to action worked best to get people started: “You can improve the accuracy of Wikipedia! CTA: Learn how to improve articles” (German: Du kannst Wikipedia genauer machen! CTA: So überarbeitest du Artikel)
• feedback from the community (online and offline) and a pretest with an external company on the different messages and the design led to the best product possible

Link zur Landingpage: https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/JetztMitmachen

Back to top

'Thank You' Campaign 2018

Banner

Shortly explained:

Like our first campaign to attract new editors also this campaign was supplementary to the Fundraising Thank You campaign. The Thank You banner originally says thank you to the donors of the fundraising campaign. This time our banner were shown separately after the fundraising banner were shown to the readers. Thus there were very few users, that really saw a banner aimed to attract new editors.

With this limited conditions we focused our aims on finding out more about the training modules that we transferred from the english modules.

Questions:

• How does a Thank You banner attract to contribute as a new editor?
• Do the users complete the trainings modules, when they start them?
• How does the completion of training modules affect the editing behavior?

Learnings:

• The very low banner diet and also the different messages – saying thank you for funding und also asking for contribution with editing – resulted in half of the average conversion rate in page clicks
• The numbers of users completing a training module are very low (29 people) and thus cannot be transferred into common sense about the editing behavior that follows. Nevertheless, tendencies do not show any differences in people who did a training module or not.

Link to Landingpage: https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia

Back to top

Go back to 'Key Findings and Results' overview page.