Here is a very basic guide on how to write a report from survey data. It's not intended for absolute beginners. It is more of a reminder for those who once learned statistics, but aren't sure how to convert a statistical printout (from software such as Epi Info or SPSS) into a written report. This is based on the format that Audience Dialogue normally uses. Following these instructions will produce a more thorough report than you normally see from commercial research companies. The purpose is to produce a report that's completely self-explanatory: one that can be passed on to any well-educated manager, who should be able to fully understand it without having to make further inquiries. When you've finished writing it, you should be able to hand it over and say "Here's the report. It tells you everything you need to know about the survey." Note, more information on research reporting (and usage) is available in the book Know your Audience - A Practical Guide to Media Research (revised 2007), which provides a comprehensive and usable guide to all aspects of audience research. This book is available as a 384 page PDF document for just 10 Euros, US $15 or AUD $15. Click here to contact us and place an order, or simply email john(@), removing the brackets when you type the address into your email program. We use PayPal, enabling you to pay easily and securely in different ways including by credit card. You scan the pages of the earlier edition here. 1. Introduction to the surveyThis introductory section will usually contain a contents page, a credits page, an executive summary, and background data on the survey.
If there is anything else that refers to the survey as a whole, it should be mentioned in this preliminary section. Next, you need to go through each question separately... 2. Write about each question in turnCover each question asked in the survey - about 2 pages per question is usually enough. Avoid covering one question across more than one page opening. Control variables (such as time, date, and place of survey) should be included as if they are questions. Questions should be covered in the most logical order for the hypotheses - often (but not necessarily) the same order as the questionnaire. For each question, write about each of these 5 points:
2.1. Who was asked? 2.2. How many people were asked? 2.3. What was the question? 2.4. What type of responses was gathered? If open-ended, were the answers then combined by coding? If "other" answers were sought in addition to the multiple-choice answers, list any "others" mentioned by at least 1% of the sample. 2.5. Summary of responses The verbal summary is like a headline, expressing the main finding from tha question. One sentence is enough. This is followed by a more detailed verbal explanation of the results, both describing them in words, and also commenting on them. For the frequency table - which can often be combined with a graph - the format depends on the type of answers that were given to this question. At the foot or top of the table, show the total raw numbers and the total % (usually 100%). For questions that allow multiple answers, show two percentages: % of respondents and % of answers. It's best to copy and paste a table from the statistical software into your word processor file, to avoid numeric errors - removing unnecessary elements later. If more than about 1% of respondents who were eligible to answer this question did not answer, here provide any reasons why this might have happened. There are three main types of question, and the responses to each are displayed differently. These are (1) multiple-choice (but single-answer) questions, (2) multiple-answer questions, and (3) open-ended questions. Now for some examples of each type... 2.5.1. Multiple-choice questionFor multiple-choice question: list the frequency distribution of answers for the question, showing both raw numbers and %. List the frequency distribution, showing both raw numbers and percentages. If the sample was less than about 200, list whole percentages, e.g. 15%. If more than about 200, give percentages with one decimal point, e.g. 15.1%.
Example 1: Single-answer question. Notice how the raw numbers are in a different type style from the percentages. Gender of respondent.
Example 2: Multiple-answer question. Note simple bar graph on right of percentages. Q14. Base: all respondents (n=181)
2.5.2. Numeric questions: group responses Example 3: This table has grouped ages that were gathered as individual years, compared the survey results with the census data, and shown the difference. The table was followed by a comment on the reasons for the difference, and how much those differences might affect the survey results. Because the main purpose of the table was to compare the sample with the population, it did not include raw numbers, which would have made the presentation confusing. Base = all people (606 of them answered)
2.5.3. Open-ended questions Example: 4 In this case, the hypothesis related to noticing live programs, hence the highlighting. Other responses were of less interest for the research. Notice that when a question allows multiple answers, the percentage is ambiguous (% of people or % of answers?) so you need to state what the percentage is based on.
2.5.4. Verbal summary When asked what changes thay had noticed on provincial radio in the last 6 months, 138 people (76% of the total sample) mentioned at least one type of change. 58% of these mentioned the introduction of live programs. 20% mentioned changes to programs, other than live programming, and 29% mentioned changes that were not program-related - such as broadcast hours or quality of reception. These percentages add to 107%, because a few people mentioned more than one type of change. Take care not to confuse frequency of response (what surveys measure) with othe factors. Thus in the above example it would be wrong to write "Live programs were the most popular change." Just because live programs were more noticeable doesn't mean they were more popular - which might imply more liked. After you've written a summary, get somebody else to read it and see if they can misinterpret it. If they can, rewrite it! If you have any other data (e.g. from a census or previous surveys) related to that question, mention it in the summary. Also discuss any problems that occurred with that question - e.g. ambiguous wording. 2.5.5. Graph the results The best graph format to use is generally the horizontal bar chart, with one line per frequency. Pie charts take too much space, and histograms don't have enough space for labels, if there are more than a few different answers. Don't use colour graphs if your report will be photocopied in black and white - or use patterns as well as colours to distinguish the bars in your graphs. 2.5.6. Significance testing 3. Cross-tabulate relevant pairs of questionsMost questions don't have much meaning on their own, so you need to cross-tabulate them with other questions. Each cross-tab will test a major or minor hypothesis. Don't try to cross-tab each question with ever other - only the ones that either (a) you have hypotheses for, or (b) you want to make a demographic comparison for. If the answers to a cross-tab are not statistically significant, report them only if this is an important hypothesis. Otherwise, just report that the different wasn't significant - e.g. "Question 3 produced no significant differences at the .05 level for different sexes, age groups, or geographical areas." When putting tables into a report, remember than you can only squeeze about 8 columns across a page, and about 50 rows down. If there are too many different answers, you'll need to recode the question to reduce the number of columns (or, sometimes, rows). Here's an over-simplified guide to which statistical test to use:
4. Groups of questionsSometimes, a group of questions can be combined - for either of two reasons:
4.1. Combined table and summary 4.2. One combined graph 4.3. Use scale averages to compare questions Make sure not to include missing values in scale averages - e.g. if respondents are using a 1 to 5 scale, and "don't know" is coded 6, exclude those 6s from the calculation of averages. 4.4. Correlations Example 6 - a table from a group of questions. This table combines the answers from 8 related questions, making it easy to compare which types of program are most and least listened to. Always include a Total column (usually 100%, as here) so that readers can see what the percentages mean, and in which direction they add up. The percentages are based on the number who answered each question. Notice that the sample sizes are in a different type, again to avoid confusion. Respondents who answered Yes to question 1 (148 of them) were asked question 6:
Example 7 - A cross-tab from a group of questions. This has one question in each column and one answer in each row -generally the easiest format to read. This time the totals are not shown, because they would make no sense. Sample sizes in each area are shown as "n=" to avoid confusion with the other figures, which are all percentages. Q.18 (Sample = all respondents )
5. Conclusions, maybe recommendations, and appendixesWhen all this is done, you'll have reported on each question separately, with a page or two per question. Then you'll have had reports on grouped questions. When somebody reads a report like this, they feel swamped with data. So you need to summarize it, and explain how each question fits with the others, and come to some overall conclusions. At the end you should write several pages of conclusions for people who've just read (or skimmed) the whole report, summarizing what you think are the main findings of the survey. If it’s relevant to add recommendations, they can go here. However, there’s no point in writing recommendations without giving reasons for them - otherwise the readers don’t act on them. Finally, there can be appendixes. It's good practice to include a copy of the questionnaire - even though you have already given the wording of each question individually. The appendixes may also include tables related to sample selection, instructions to interviewers, and so on. Sometimes there's so much appendix material that it can be produced as a separate volume - the existence of which should be mentioned in the first volume. The advantage of the two-volume report is that because fewer people are interested in the technical details, fewer copies of the second volume can be produced. Another way of saving paper is to produce both a full survey report (with few copies distributed) and a short summary report, with many copies distributed. The disadvantage of the latter approach is that it means writing two separate reports, which costs more, and takes longer. Thus writing a summary report is usually justifiable only if the expected readership is large - say 100 or more. If your readers all have internet connections, you can save paper (at your end) by emailing the report to them. A few suggestions, if you do this: If you are interested in writing research reports we suggest you seriously consider our book Know your Audience - A Practical Guide to Media Research (revised 2007), which provides a comprehensive and usable guide to all aspects of audience research including how to write and use research findings and reports. This book is available as a 384 page PDF document for just 10 Euros, US $15 or AUD $15. Click here to learn more about the book. |
|