Information for Units Planning to Launch a Survey
Why was the Survey Review Committee established?
In conducting institutionally mandated surveys, the Office of Institutional Analysis observed a very low response rate, far below other U15 institutions. In investigating one particular case, the National Survey of Student Engagement (NSSE), it was determined that another survey of students at the University of Manitoba had been launched on the date that the NSSE survey responses dropped off sharply. Surveys are an important source of information for many areas in the University. The Survey Review Committee was established to improve response rates by coordinating the timing of, and ensuring the quality of, future surveys.
Things to Consider
Plan ahead. Depending on the size and complexity of a survey, about one year ahead is recommended. Look at the schedule of approved surveys that is on the web site and note that surveys must avoid time slots already taken and time slots for University mandated (usually U15) surveys.
A decision to conduct a survey typically arises from the annual review of a unit’s strategic plans and associated activities. Advance planning is also needed to ensure that costs can be expensed within the desired fiscal year. This is particularly important when using external survey research firms. If the survey requires Ethics Review Board approval, time is required for that process. See their web site for details.
Privacy requirements under The Freedom of Information and Protection of Privacy Act (FIPPA) and The Personal Health Information (PHIA) must be addressed. This would include such things as a description of how the privacy of the subjects will be protected. Are responses anonymous or confidential? If confidential, consideration needs to be given to who will contact the population being surveyed, how this contact will be achieved, if the identity of the respondents will be connected to their responses, and who will have access to this information. Regarding confidentiality, the survey software should be housed in Canada and/or the data should be housed in Canada. Otherwise a legal document called a “data agreement” with the vendor may be needed. Applicants should work with the Access and Privacy Office to ensure that all requirements are met and are detailed in the application.
The most important step is to develop and have approved by stakeholders a statement of the specific objective or objectives. This is generally one of the most difficult and time consuming parts of doing survey research. A general statement such as “we would like to know what students think about methods of instruction in dept x” is not sufficient. A clearer statement would be “ the dept of x would like to determine to what extent students taking our courses are satisfied with the “y” types of instruction currently offered in our courses: face to face, project team work,…etc.” It is a good idea to consult widely within a unit as to what are the important questions to ask on a topic, as this may vary depending on the stakeholder group.
What are the other steps involved in survey research? In addition to developing objectives, determine:
- Who is assigned to manage the survey;
- Who is doing the technical aspects;
- Who is developing the questions;
- When will you want to conduct the survey;
- What software will be used if it is an online survey;
- What is the source of contact information for the survey recipients;
- Who is going to analyze the data;
- By what date will the analyses be available, and;
- What will the results be used for? See the point above regarding strategic planning.
Consider whether the survey development and/or administration would benefit from contracting with a professional survey research firm. As a guideline, the more complex the survey, and the more sensitive the data, the greater the likelihood of benefitting from dedicated expertise. “Opinion Surveys” as opposed to fact finding surveys would qualify under the “more complex” category, as would politically sensitive issues.
How do I go about finding a professional research firm? The University of Manitoba Purchasing Department can assist by indicating firms previously employed, and by providing a template for a “request for proposal” (RFP), if required. Allow sufficient time in your timetable for this process. Generally, the more complex the project, the longer it takes to select a service provider.
How do I compare the costs of doing it in-house to contracting it out? Make a list of all the steps, estimate the number of hours for each activity for each person, and multiply by their hourly salary rate including benefits. Add any expenses, such as software costs, license fees, etc. Think of it as “billable hours” – any and every 15 minutes dedicated to a project is a cost incurred by the project. Consider opportunity costs and timeliness of the results.
Review the application for approval form. Some but not all of the components have been covered in the above points. Consult with the Chair of the Committee if you have any questions. Please submit completed forms to Shannon.Mansfield@umanitoba.ca.
The guidelines above are also available in an MSWord document.