Overview
Inlate 2014, Samara released an updated to the Member of Parliament Website Project, first conducted in 2013. This project seeks to better understand how MPs use their websites and the extent to which MPs take advantage of the opportunity the Internet provides to better connect with their constituents. The initial 2013 MP Website Project was profiled as Case Study #9.
The project analyzes websites for the ways a constituent can contact an MP, the work an MP does, and the ways MPs involve Canadians in the political process. The results are published in an easy-to-read infographic. In addition to the 14 criteria searched for in 2013, two new criteria were added – a privacy policy for information shared with an MP and some form of an expense report.
Data Collection
Data collection in 2014 mirrored 2013 practices. Between May and July 2014, Samara volunteers visited 299 Canadian MPs’ websites to see how many sites included criteria from the new 16-point checklist. We excluded the four party leaders from our analysis as well as five vacant ridings (by-elections pending at time of data collection).
Volunteers were trained and guided by Samara staff at Samara’s offices. Websites were coded in French where MPs’ language preference was French. MP websites were located either through a search engine or through information on www.parl.gc.ca.
Similar to 2013, the analysis focused on the content of MPs’ websites and did not take into account how visually appealing websites were, or how easy they were to navigate; Samara simply looked for evidence that the checklist criteria were present or not present. This reduced room for subjectivity on the part of the data collectors. To ensure accuracy, Samara staff double-checked 30 MP websites (10 percent of the sample) selected randomly.
All percentages reported were rounded to the nearest whole number. To receive a point for social media, the MP had to have at least two of: Facebook, Twitter, photo sharing (e.g. Flickr), or video sharing (e.g. Youtube). More information on Samara’s methodology is available here.
As with any study, there are limitations to the design and data collected. While websites are a key communication tool for MPs, websites do not necessarily capture all of their online activity. For example, though we looked for links to social media accounts on MP websites, we did not evaluate how often or how well MPs were using these other platforms.
Findings
The 2014 findings support the general trend found in 2013 — MPs are more inclined to share information about themselves and their work in Parliament or in their constituency than they are to try to begin a conversation with citizens online or encourage their participation in the political process by sharing information about how to get involved.
MPs’ websites, on average, include 9 out of 16 criteria on the checklist, only four websites checked off 14 criteria, and none checked off all 16. Three MPs did not have a locatable website, down from five in 2013.
Comparing 2013 to 2014 — nearly all of the indicators moved in a positive direction, which is good news. The leading areas of improvement were:
- 39 more MPs included their work in Parliament
- 30 more MPs had a space for discussion on
their websites
- 21 more MPs listed their hours of operation
For the two new criteria, we found that 77% of MPs included expense information and less than one-quarter had information about a privacy policy.
The results did not all move in a positive direction, however:
- 15
fewer MPs offered a newsletter sign-up
- 9
fewer MPs posted information about their constituency work
- 9
fewer MPs linked to the central party’s page
Samara celebrated the Top MPs again, raising the threshold (from 11 out of 14, to 13 out of 16) to accommodate the new criteria. With a higher threshold, twelve MPs (out of 299) made the list this year, compared to 17 (out of 300) last year. Had Samara used 14-point checklist from 2013, 20 MPs would have been a ‘top website’ in 2014 — another signal that MPs are adding to their sites. In fact, 11 items showed improvements of between 1% and 13% points.
Lessons Learned
There are several measures of success following the release of the second Website Project.
Media Coverage
- 2013: 15 national articles
- 2014: 2 national articles
- The
inaugural year received substantial media coverage, which added to the
project’s credibility and widely disseminated Samara’s findings. However, reduced
national media attention in 2014 was offset by social media sharing and
discussion on Twitter and Facebook, which also directly involved MPs.
MPs’ Response
- Many
MPs noted, either publicly online or to Samara directly, that the project
helped them to reconsider the contents of their websites. In fact, it was partly
MPs’ expressed interest that motivated Samara to conduct the project
again. Early in 2014, a handful of MPs
inquired with Samara if the project was to be repeated, suggesting that MPs and
their staff hoped re-designs would warrant a “Top Website” title.
Improved Websites
- Improved criteria and overall website scores in 2014 confirm
that updates and improvements to MPs’ websites are something that can be
accomplished fairly quickly and at a low cost to political leaders. This is a
positive message for MPs, constituents, and democracy.
Opportunities
The second edition of the project brought with it two new opportunities for Samara.
Positive Feedback Loop
- Public feedback in 2013 that led to the
additional criteria included in 2014 demonstrated Samara’s own openness to
improve and make our work more useful and relevant to stakeholders.
Maximize Organizational Bandwidth
- Many volunteers who contributed to the project in 2014 also worked on the project in 2013. This helped build institutional memory and improved the project’s accuracy since it took less time to train these volunteers. Samara was also able to use a similar design file from the original project, saving time and costs.
Critical Issues
In 2013, we faced three critical issues – handling lowest scoring websites, time between data collection and publication, and tracking the impact of Samara’s work. How did these critical issues play out in the second edition?
Low Scoring Website
- In both years, Samara made the decision to not share the lowest scoring MP websites (with the exception of MPs with no locatable websites) – a decision in line with Samara’s mandate to be constructive and positive.
- MPs
with top websites appreciate the recognition – and will tweet or display a ‘top
website’ badge on their website. The positive tone of the project ensures it’s
something MPs look forward to and generates a bar of excellence
Data Collection Timeframe
- The challenge between time of data collection and publication remains pertinent. Many MPs’ websites are updated from time of collection (May to July) to publication (December). This delay has lead to a new challenge; a few MPs have asked Samara to reevaluate their sites based on updates which occurred after the time frame of our data collection.
- To
accommodate this challenge, Samara has created a committee of volunteer coders to
handle these requests in order to provide up-to-date evaluations of MPs’
websites. This encourages MPs’ online improvement year round.
Samara’s Impact
- Tracking the impact of Samara’s work is a continual challenge. While typically reliant on national media coverage, less attention this time around encouraged Samara to measure impact among stakeholders – MPs and Samara followers, which was positive. This project has become one of Samara’s key stakeholder engagement exercises.
Additional Resources
2013 Checklist of tips for elected officials websites
2014 Checklist of tips for elected officials websites
Contact
Laura Anthony, Research Analyst, Samara ([email protected])