Image shared under a creative commons license by Sally Muir.
Today I attended the second of Tabetha Newman’s workshops on Survey methodology – Analysing Survey Data. The aim of the session was to provide attendees with a greater understanding of analysis and to prove that it is not as dry a subject as many textbooks make it seem. My thoughts on the previous session – Designing and Running Successful Online Surveys – can be found here.
The workshop was broken up into two sections:
The Theory – focusing on managing data, checking for validity and an introduction to descriptive statistics and statistical analysis.
The Practice – looking at some practical examples of statistical analysis in the context of survey data and how these might be reported.
Tabetha distinguished the two sections of the days as Statistical literacy (understanding data is not just a mathematic skill – to critically analyse and evaluate we need a kind of literacy) and Statistical calculation (the more computational, testing side of statistics).
This portion of the workshop been with a consideration of the analyst’s mindset when beginning data analysis – are they undertaking exploratory data analysis, looking to see what patterns emerge, or are they hypothesis testing, looking for specific answers. From the definitions given, I would argue that the DigiLit project will be approaching its data with a mixture of these mindsets – we have variables that we are interested in, but no defined hypotheses related to them.
As with the previous workshop, Tabetha gave us a handy acronym for handling our raw data:
Collate your data – download if online, input if handwritten
Outliers – look for them, unusual data, often very high or low values
Data Management – best practice for storing and organising
Error Checking – could also include input of symbols or zeros that will affect data analysis tools
Record every edit – the importance of keeping track of changes during coding
We then looked at the key questions to ask yourself in order to consider the validity of your data and Tabetha provided a short but effective introduction to the concepts of data types, descriptive statistics and statistical analysis in the context of survey data.
After a much needed tea and biscuit break, we began looking at some examples of the kinds of data you might collect and the tests you might apply to them. We looked at likert scale quests data and also nominal (categories with no scale) data.
The most useful part of the session for me personally, was looking at different ways of presenting and reporting data. As a key element of the DigiLit Project lies in the summary reports which will be issued to schools, it was useful to get some tips on how to make these simple and effective in conveying the message of the data. Tabetha’s main points for presenting the data of a survey question are:
Question options – showing what was available
Summary table data- including any subgroups (e.g. Faculty)
Average per subgroup
Frequency histogram per subgroup
Results of stats analysis, plus a sentence or two in English explaining it!
We also looked at ways to represent data from free text questions and Tabetha introduced the idea of using info-graphics to represent survey data. Whilst this would not be of sufficient detail for our school level summary data – it may be interesting way to externally share the overarching city data we collect.
Tabetha’s key message throughout the day was that it’s important to get away from the numbers and remind yourself of the broader question (and context of the data).
Overall the session was incredibly useful (and well worth the journey)! Whilst I have studied data analysis in the past, I often find textbooks on the subject are dry and far too wordy – almost as if they are trying to make everyone believe that analysis is something to fear! Tabetha’s session was an easy to follow, easy to digest refresher for me and it was great to have the opportunity to think about analysis specifically in the context of surveys as I haven’t worked with qualitative data in this way in the past.
And I’m almost surprised to be writing this – but now I’m actually a little excited about getting to work with our data later in the project – you can quote me on that Tabetha 🙂