Organization: Samara
Project: The Member of Parliament Website Project
Country: Canada
Government Level: National

Overview:  

Samara’s Member of Parliament Website Project analyzed the websites of 300 members of the Canadian Parliament (MPs) to better understand how MPs use their websites and the extent to which MPs take advantage of the opportunity the Internet provides to allow them to better connect with their constituents. Samara created a list of 14 criteria that were analysed by volunteers for each website. Samara summarized the results in an infographic that can be shared across social media platforms. They also provide a list of the Top MPs websites and a checklist of tips for elected officials’ websites.

Overall, the analysis found that MPs are great at offering their constituents the basic information about themselves, but they perform poorly at offering ways for Canadians to engage with politics or have input on decisions and policy.

Background:

Overview: Samara is a non-partisan charity that works to improve political and civic participation in Canada, and is the country’s foremost organization dedicated to reconnecting citizens to politics. Formed in 2009 out of a belief in the importance of public service and leadership, it is named for a samara, the winged “helicopter” seed of the maple tree and an important symbol of Canada. Samara’s research and educational programming began with the initiation of Canada’s first-ever series of exit interviews with former MPs, which have been adapted into a book, Tragedy in the Commons, to be published by Random House in April 2014. The organization has a number of initiatives to increase citizen engagement, including:

• The Samara Democracy Reports series, which advance a non-partisan research agenda that shines new light on Canada’s democratic system and serves as a resource to increase Canadians’ understanding of politics. This will culminate in the Samara Index, an annual report on citizens and politics.
• The Samara in the Classroom programs, which adapt Samara’s research into materials that encourage new ways of teaching politics and active citizenship. Samara’s reports are used in university classrooms, have spurred new academic research and publications, and are being adapted into a grade ten textbook with McGraw Hill Ryerson.
• Democracy Talks, a national public outreach initiative that provides Canadians, including young people and newcomers, with a platform to discuss how politics can better reflect their daily lives.
• The Everyday Political Citizen project, which launched in 2013 to inspire Canadians from coast to coast with hundreds of nominations for citizens showing that positive politics is possible.

MP Website Project: Between August and September 2013, Samara visited Canadian MPs’ websites to see how many elements on a 14-point checklist could be located. These elements were designed to reflect the ways a constituent can contact an MP, the work an MP does, and the ways MPs involve Canadians in the political process. From this survey, Samara identifies the top MP websites (those who scored 11 out of 14 or higher) and provides links to all MP websites.

In addition to presenting information through reports, Samara uses infographics and other ways of conveying information to spark citizen interest. The elements represented in The Members of Parliament Websites Infographic reflect the interaction between MP and citizen: the ways a constituent can contact an MP, the work an MP does, and whether MPs involve Canadians in the political process. For example, the infographic demonstrates that while 98 percent of MPs websites tell citizens where their office is located, only 15 percent include office hours of operation.

Implementation:

For the Member of Parliament Website Project, Samara identified a 14 categories relevant to evaluating MP websites and examined 300 MPs websites, excluding party leaders and the four electoral districts (ridings) that were vacant at the time of data collection. Samara used various academic research on website content analyses to develop a ‘codebook’ that described in greater detail each checklist item and provide examples or guidelines to help train/guide the volunteer data collectors. The volunteers proceeded to 'code’ each website, recording the data (whether the checklist item was present or not present) into an excel template designed for the purpose.

Data collection occurred in August and September 2013 by nine Samara volunteers. Volunteers were trained and guided by Samara staff at Samara’s offices. Websites were coded in French where MPs’ language preference was French. MP websites were located either through a search engine or through information on www.parl.gc.ca.

The analysis focused on the content of MPs’ websites and did not take into account how visually appealing websites were, how easy they were to navigate, or how recently information had been updated; Samara simply looked for evidence that the elements were present or not present. This reduced room for subjectivity on the part of the data collectors. Accuracy was ensured by double-checking 11 percent of the MPs websites, selected randomly.

All percentages shared on the infographic were rounded to the nearest whole number. To receive a point for social media, the MP had to have at least two of: Facebook, Twitter, photo sharing (e.g. Flickr), or video sharing (e.g. Youtube). More information on Samara’s methodology is available here.

As with any study, there are limitations to the design and data collected. While websites are a key communication tool for MPs, their websites do not necessarily capture all their online activity. For example, though links to social media accounts on MP websites were counted, Samara did not evaluate how MPs were using these other platforms. Samara also recognizes that not all MP’s websites serve as a reliable measure of their work in Ottawa or the constituency. For example, some MPs have very simple websites, but are continually applauded for their hard work and tendency to return individual phone calls. In other words, they have chosen not to report on their work via their website.

Achievements:

The best measure of success for this project will be if MPs’ websites receive higher scores when this project is replicated in one to two years. In other words, will MPs websites contain more items off the checklist?

However, there have been other immediate indications of success.

First, the findings of this research project received substantial national media coverage, which adds credibility to Samara’s work, and helps spread Samara’s message by capturing the attention of constituents and MPs. This gives weight to the idea that MP websites matter to representation, and that MPs can make website improvements with relative ease (compared to other democratic reforms).

Second, there is evidence (indicated by Samara’s correspondence with MP offices and MPs’ Twitter comments) that MPs are responding to the project. Several MPs directly, or indirectly through their staff, indicated their websites will change to include one or more indicators from the checklist (for example, office hours). At least one MP quickly repaired their inactive website that was found to be ‘down’ for the months during the data collection phase and additional follow-up checks.

Third, Samara’s project engaged other organizations and individuals in this space. A group working to improve Canada’s Parliamentary website (www.parl.gc.ca) was interested to know whether Samara considered links to resources about Parliament or the House of Commons as a checklist item. Tech/Communications professionals suggested that mobile-friendly website be a need-to-have rather than a ‘bonus’ on the checklist for elected officials. This was constructive feedback and engagement with new stakeholders.  

Finally, Samara has heard informally that the tip sheet/checklist has been circulated at different levels of government. Though Samara remains focused on national politics in Canada, designing a tip sheet with implications for all elected representatives extended the impact of this research. Innovative civics teachers have also used the website checklist as a classroom activity to further students’ understanding of elected representatives roles.

Critical Issues:

Samara made a deliberate decision not to share the lowest scoring MP websites (with the exception of MPs with no locatable websites) in the same way the top websites were shared. For the first iteration of this project, the emphasis was on celebrating MPs who excelled. This reduced the risk of any backlash against MPs with low scores and shielded some from more overt public criticism. It was also in keeping with Samara’s efforts to be constructive and positive. However, this lack of “public shaming” may have limited the pressure on MPs with very low scores to improve their websites.

A different challenge is in minimizing the time between data collection and publication. There was a time-lapse between data collection and analysis (August-September), and publication of the infographic (December), which meant that some MPs had changed/updated their website, and likely would have received a different evaluation on the 14-point checklist. While data collection occurred in the summer, many MPs’ websites were likely updated with new information once they returned to the nation’s capital for the fall session. There is a tension between providing an accurate assessment of MPs websites while recognizing that there are periods when they are busier than others. Ideally, the time-lapse between data collection and publication should be minimized to ensure the research still reflects reality as accurately as possible. In response to this concern, Samara is proactive about providing a thorough methodology on all research projects.

Finally, with publicly-shared research, it remains an ongoing challenge for Samara to track and assess the full impact of the research and resources Samara generates.

Samara, in the interest of reputational independence, has relied primarily on media coverage of research, including this MP website project, to gain traction for Samara’s ideas. In the future, Samara’s impact may be magnified by also engaging directly with political actors, such as the national offices of political parties, who often provide guidelines and direction to MPs and their staff. The checklist will also become a part of Samara’s “Welcome to Parliament” package for new MPs that are sent out after elections.

Contacts: 

Jane Hilderman, Research Manager, Samara [email protected]

Samara’s general email account ([email protected]

Samara welcomes comments or suggestions at any time.

Additional Resources: 

Checklist of tips for elected officials websites

Tips for Elected Leaders’ Websites

Note: This post is part nine in a series of case studies on tools PMOs have used that can be replicated or serve as models for organizations in different contexts. To see all of the case studies, click here. To contribute a case study on a project that your organization has created, please fill out the template or email Dustin Palmer at [email protected].