Open Data Kit + Audience Finder = Open Audience Finder?
I’ve worked on audience surveys for a long time and I’m always looking to improve my methods. Relatively recently the roll out of Audience Finder from the Audience Agency has done a lot of good in standardizing many basic questions and methods for a range of arts organizations. (see standard questions: link)
This makes a lot of people happy as they have a basis for comparing and contrasting their results with others. Most of the arts organizations I’ve worked with see the value, one way or another. Everyone has their own tweaks or preferences to suggest but some collaboration around some standard questions is generally a good thing…
Around the same sort of time, I started learning about and putting Open Data Kit into use, for various audience and other kinds of end-user/patient surveys. Hence the combination of the two, and in this post, I will be describing how you, with less technical expertise than you probably think, can use it too.
Skip to the technical stuff
Open Data Kit It is not the only Open Source survey/form designing set of tools out there (hey, Limesurvey), but it is the one I’ve come to rely on and the one I most recommend.
Often it seems that Open Source options are intimidatingly hard to start using. I should note I am not a web developer or programmer outside of being pretty good with Excel. I agree that this may be a bit of a niche skill but I could see most people being able to at least edit and tweak a survey design and from there it’s only a small step to do everything yourself. The documentation, guides and community are all excellent help.
People thought they couldn’t manage their own websites, or write e-bulletin/html formatted newsletters, or use “that clever spreadsheet that guy from Finances set up for us”. It’s all within the realms of feasibility and at it’s heart is based on spreadsheets (that you know how to navigate/edit/use) and mobile phones/tablets (that you know how to use).
Back in the day, I started off using paper forms in combination with a scanner and software called Keypoint by Speedwell, which I think now only exists for administering standardized exams. The dream was that it would make data entry from paper forms instantaneous, but the reality was there was always still quite a lot of manual tidying up to be done, especially with tightly packed forms or any kind of text / handwritten entries. It was also pretty expensive and we only had a license for one user at a time.
Then I was using SurveyMonkey but this was obviously limited to web-based surveys. Worked great for cases where only a web survey was required, but in audience research you generally do quite a lot better in cases where you have to get out and
harass engage people face to face. For many events and festivals, with no box office or audience database, face-to-face is really the only option they have. *Maybe* you can give people a link to an online survey and hope they complete it when they get home, much the same as you would have done with a pre-paid envelope, though I have never found this particularly effective.
Paper was a pain for one set of reasons, digital was a pain for another set of reasons. In some cases, you could try and use both; and why wouldn’t you want to make it as easy as possible for people to respond? In this case you also have the job of bringing the two sets together. Of course the human-readable questions are the same but I guarantee the raw data each method produces will be formatted differently. (There is another argument to made that you ideally shouldn’t mix these methods at all, but we’ll ignore that for now)
The holy grail seemed to be an affordable, mobile-based survey that wouldn’t be reliant on always having an internet connection or that required expensive subscriptions (inevitably the feature you want is on the next pricing tier) or that bogged you down with various proprietary solutions or require particular ways of working.
Something that would manage web surveys, be available offline on kiosk or tablet style computers, could of course deal with manual data entry if required and maybe even, at some point in the future, would also be able to manage some degree of OCR / OMR scanning of paper surveys. (like I used to do with Keypoint)
I have not managed to banish the paper survey yet, and it may always be around, but it is getting more feasible every day. Enter Open Data Kit!
Designing forms (questionnaire, survey, etc) is done in Excel (or other spreadsheet software) according to ‘XLSFORM’ standards. These can be edited and shared freely as basic .xls files. This ‘human readable’ spreadsheet is then converted into a ‘machine readable’ ‘XFORM’ which will run on the appropriate software (generally ODK Collect for mobiles/tablets and Enketo for web browsers). In theory you can edit the XFORM .xml file directly if you like with a text editor like Notepad++ but this is rarely advisable or required. The XLSFORM to XFORM step can be done both on and offline through other open source programs. At this step the form will also be ‘validated’ which will highlight any problems with the design: this question has no answer options, you’ve got two questions labelled the same etc etc. Very useful.
Over the last two years or so, I developed a lot of ODK forms (called XLSForm or XForms) that were based on or around the Audience Finder ‘standards’, which saved huge amounts of time in programming the same questions over and over again; and setting up a workflow to analyze the results. The next natural step seemed to be, in keeping with the Open Source ethos, to try to share as much of this work with you as possible! Once I realized I could give it a silly acronym “O.A.F” I just had to make it happen.
Particular thanks to Leicester Arts Festivals for providing the impetus to take a lot of this from ‘that would be nice’ theory to ‘well, let’s actually try it’ practice. Please note although I have chatted with a few staff, the Audience Agency do not formally support this, use at your own risk, discretion… etc. (Thanks Alice & Katie) Questions were up to date based on the 2016 template, and even then there’s a few variations or options to be aware of. If you want their view, speak to them directly.
For completions sake, I am not necessarily recommending that the Audience Finder template is the right approach for you / your organisation / your festival or event; but it is a great place to start if you want to borrow some ideas or haven’t even thought about this before.
In this post, I will share generic examples and templates for you to use as you like. All I ask is you drop me an email (email@example.com) to let me know if you’ve found it helpful and how you’ve used it; I am happy to answer questions. Maybe you are interested in arts audience surveys, or just in an example usage of Open Data Kit more generally. There are quite a few bits I am still working on and will update when I have the chance.
What’s the benefit?
Assuming you are sold on the idea of doing a survey in the first place…
– Use tablets, mobiles, e-readers, laptops to replace paper surveys thereby reducing or avoiding data entry and speeding up analysis.
– ODK Collect works on Android devices, most modern browsers can manage Enketo; both are part of the same ‘ODK ecosystem’ – program it once, works pretty much the same on both.
– Based on Open Source standards to modify and customize as you like, no subscription, no one off charge, forever
– All designed to work ‘natively’ offline avoiding patchy Wifi / mobile signal indoors or outdoors
– As many if not more features than commercial options; media playback and capture, GPS, barcodes, signatures, multiple languages. There is no ‘pro’ version you have to upgrade to, to get all the features, but you can buy hosting and help from third-parties.
– Active community of developers, end users and commercial providers, regular updates and new developments
(Example deployments/usage here) – (List of help for hire here, including yours truly of course)
– Various options for free, no install, starter plans, albeit with various limits and catches: Ona, KoboToolbox and SurveyCTO
Of course this does mean you have to learn things and do work yourself but as mentioned earlier, I am not a programmer and thanks to the kind efforts of many people, the barrier is getting lower every day.
A final particular note on money, where it’s commonly heard: “Cost should not be the main reason for going Open Source.” While in my experience, it has definitely become a timesaver and cheaper in the long run (Hello monthly $0.02 hosting fee) there was obviously a learning curve and the further you go with it, the more responsibility you could end up taking on in terms of keeping devices, software and servers up to date.
Overall, I’ve been able to do more work with the same, even fewer, resources; which has led to being able to work with a wider range of organizations. It’s helped some groups participate in collaborative projects where otherwise they wouldn’t have been able to. At least when there is a repeatable, reliable technical solution in place, the debate then becomes more about the shared objectives, ownership of the process, who asks what questions and so on. Less time worrying about collection, data entry, cleaning and ‘faff’ – more time analyzing, more data to be analyzed, happy happy days.
From a learners perspective, I found was that it was easy to do these steps bit by bit, somewhat interdependently and only get more advanced as I needed. New features and tweaks keep being added, including my most anticipated feature of paper OCR/OMR scanning. (ODK 2.0) However, if I was satisfied with the basics, there’s very little to stop you just sticking with that from now until doomsday. ODK 2.0 will most likely exist as an advanced version for those who specifically want those features, ODK 1.0 will continue to do the basics.
The templates are designed to replicate the ‘core’ Audience Finder questions. In each there are a couple of areas you will need to provide your own information but these are pretty clearly signposted. While a big part of the idea is to allow you to fiddle and change whatever you like, I would generally recommend trying it out ‘as is’ before you think about changing things up too much.
Most of this is probably easier if you are using Google Drive, however you can still download them as regular spreadsheets and the functions will stay the same.
Here also is a short summary of some mobile devices you might consider using:
tablet-options-for-odk-collect-and-other-purposes (Feb 2017)
Template Form (to collect responses)
Dashboard (to summarize results)
Convertor (optional; to convert raw data ready for upload to Audience Finder)
The last two will be in the same spreadsheet but have quite different roles to play.
(Last updated 27/02/17)
I want to use Google Forms:
1. Audience Finder template form (link)
– This is a fully editable link so please make your own copy before messing around with this one! Use the 3 dots in the top right corner to ‘Make a Copy’ to your own Google Drive.
2. Google Sheets dashboard and convertor (not started yet)
– Eventually, there will be a dashboard and convertor for collected responses that use the Google Forms. It will be pretty similar to the XLSForm version but is less of a priority at the moment for two reasons. 1. It already has some pretty decent built-in summary statistics and visualizations anyway. 2.The raw data that Google Forms produces seems to be pretty similar to that which Audience Finder currently requires, so it’s not as tricky to shuffle things around by hand if you must.
I want to use Open Data Kit and/or Enketo:
1. Audience Finder template form (on Google Drive)
– This is not editable but you can download a copy of it as a standalone spreadsheet
2. Google Sheets dashboard (complete) and convertor (in progress) (on Google Drive)
– I’ve completed the first few questions of the dashboard and will continue with this
– If some of the formulas seem convoluted, this is to hopefully ensure that it all works on comparatively older versions of Excel as well as newer ones. But I would appreciate any feedback from Excel people out there. (I will also test it on some open source spreadsheet software just to be 100%)
How do I use any/either of them?
Here’s a rough diagram and more detail below:
Edit: 08/03/2018 – I have only just realised you can use Google Drive/Google Sheets as a form server; you don’t necessarily need ODK Aggregate. There are pros and cons to each. For now I would assume you can’t use things like Enketo and end user management/accounts would not be as detailed.
Case 1: Using Google Forms
Estimated time for a tech-novice – a few hours or less
This is a completely different route to most of the others but is the quickest and easiest way of getting started.
1. Copy and edit the free Google Forms template to your own Google drive
2. Edit as needed, distribute and collect your responses
3. In the ‘Responses view’, create a spreadsheet for your responses
4. Add this spreadsheet to the dashboard and convertor file (…that hasn’t been developed yet)
– Yes, this has nothing to do with Open Data Kit that I’ve been talking about so far! But it is ‘free’ and easy to start playing around with.
– Only works online, maybe you only want to do a web survey, but paper copies could be input via the same. Any hey, maybe you are supremely confident in your wifi or mobile signal to manage this anyway.
– Google Forms is about as easy to edit and design as possible, though it is also restrictive with limited features in terms of logic and routing (displaying relevant options). Larger forms get harder/slower to edit.
– Google Forms is free to use, and is likely suitable for collecting hundreds if not thousands of responses. However as it is primarily licensed for personal usage, it is unsure whether heavy business or organisational usage would be blocked. Google Forms may add some kind of offline collection feature in future but this has been requested for a long time. Although it is free to use it is not truly open-source and Google may change the features or pricing at any particular time in the future.
– Appsheet could be a good way of expanding this to enable offline functions. (paid for option)
Case 2: Use ODK with Android devices – all offline
Estimated time for someone who isn’t terrified of spreadsheets – 3-4 hours
This allows you most of the benefits of ODK – offline collection with mobile devices and complete control over the design of the form. There are more steps but none are individually that complex. It will only work on Android devices and you will be transferring files on and off ‘manually’ via USB cable rather than via internet transfer.
1. Edit the OAFv1.xls file appropriately to your organisation or event: eg add your name
2. Use this bit of the ODK website to convert the .xls file into an .xml file
– You can also preview what it would look like (on Enketo)
– You could also download XLSForm Offline (developed by Nafundi, pay as you feel) and use that to manage the XLSForm to .xml file step.
3. Install ODK Collect on Android devices (website or Google Play store)
4. Transfer the .xml file, generally via USB cable, onto the Android devices, in the appropriate location: /sdcard/odk/forms
5. Collected responses
– You should find the form in ‘Fill Blank Form’ and collected responses will be tracked in ‘Send Finalized Form’
6. Download and use ODK Briefcase to transfer completed response data, generally via USB cable, from the devices in spreadsheet format
7. Copy this raw data into the dashboard spreadsheet for viewing
Case 3: Use ODK with Android devices – online capable as well
Estimated time for someone whose parents call them occasionally for tech support – maybe a day?
Adding ODK Aggregate to the picture then allows for designed ‘blank’ forms to be sent via the internet to Android devices and for completed forms/responses to sent back via the same. If you are only managing 1 or 2 different forms and a handful of devices, you can probably live with just Case 2 but being able to do things wirelessly is a bit more slick and this whole step is required to be able to use non-Android devices (Enketo)
Follow steps 1-3 above.
4. Set up ODK Aggregate on Google App Engine (GAE)
– This will require a gmail account if you don’t already have one
– By default, you can stick with the free-usage levels that Google allows and you will not have to enable billing for this account, however even when you do, the charges will be pretty low.
– Follow the step by step instructions here to set up App Engine ready for ODK Aggregate, download the latest installer from ODK and install this as a ‘project’ on your App Engine server.
– ODK Aggregate can be run on other server configurations or offline on a Virtual Machine, but this is going to be much more complicated for most users.
- And here is a video on how to install ODK Aggregate by yours truly:
5. Upload a designed form (.xml file) to Aggregate
6. Log in to your Aggregate server with your ODK Collect devices
7. You can now download blank forms from your server and upload completed responses
8. Export your responses to a spreadsheet file, then copy this into the dashboard
– You may also want to try ‘publishing’ data, this provides a ‘live’ output of data to Fusion tables or Google Sheets
Case 4: Use Enketo with any web browser
Estimated time for someone who has already set up an Aggregate server – a few hours
This requires the use of ODK Aggregate but the collection can now take place via any device running a suitably recent web browser – apple, windows, desktops, laptops, mobile and so on. Once the form is loaded in a browser, collection can take place offline, before reconnecting to the internet to upload completed responses.
1. Follow the above steps to create a form (.xml file) and set up an Aggregate server
2. It is simplest for most to buy this service from Enketo – starting at $6 per month for 1 form.
– You can do your own installation, there is a guide to setting up on Amazon EC2.
3. You will need to follow instructions for linking Aggregate to Enketo, this is a matter of copying a few keys and web addresses around, not too complicated.
4. From Aggregate, find the relevant form to launch with Enketo
5. You will now have a web address that serves your form with the responses going back to Aggregate
– Note there are a few specific features in ODK Collect that are not present in Enketo and vice versa, plus they each display and navigate forms slightly differently. By default ODK is primarily page-by-page, question-by-question, whereas Enketo is more browser oriented.
– Note that you will most probably need to learn how to make API calls, if you don’t already, at the least in order to delete a form once you’ve finished with it. (here is my post on the Enketo group about it)
That’s it for now. I will keep updating this page though.
Ask me questions! firstname.lastname@example.org
Have you got a working example I can show my colleagues/boss/collaborators?
I have loaned out a number of devices in the past and will probably be happy to do so again, providing I’m not using them at the time. Or to set up an Enketo link, just get in touch! I may to put up live examples here at some stage – where you can see the design, aggregate and dashboard bits all working together.
This page has a link that will take you to a test ODK Aggregate server, no login required, you can browse around. If you download ODK Collect on your Android device, by default it will connect to this test server, so you can see the two working together.
This shows an example of an Enketo form whilst this links to the spreadsheet that provided the design (XLSform)
This video is me messing around with an e-reader using the software
You could also try ODK Build to design a form, download it as a spreadsheet, XLSform then upload it for conversion whilst also being able to see a preview in Enketo.
This video is a short, good overview of the whole system but not related to arts/festivals/events specifically so you will have to use your imagination. There are quiet a few ODK guide videos popping up on Youtube.
Can I put ODK Collect on my iPad/iPhone/iDevice?
No, and the short version is you need to use Enketo instead (or some other platform that incorporates Enketo, like SurveyCTO or Ona). There is a long discussion about it here and in short people generally find using Enketo a better solution than developing and maintaining an iOS version of ODK Collect.
The longer version is that Android is much more open to open source developers than Apple/iOS and a large part of ODKs mission as such is ensuring that zero-budget, charities and development agencies can benefit from this technology. Reasons why Apple operates this way are detailed here (link) and for fairness there is equally long version about Google here (link). Although I’ve tried to espouse some of the great virtues of Open Source in general, it should be noted that Free Software is, specifically, something a bit different.
What devices do you recommend?
tablet-options-for-odk-collect-and-other-purposes (Feb 2017)
Virtually any android device. I have used Google Nexus 7 (1st gen) quite a bit and even had quite a lot of success with some rooted Nook E-readers. If you want to use the camera/video/gps etc, then obviously make sure your device has this. A removable micro-sd card is also pretty useful but unless you are capturing photos/video/audio the amount of storage required overall is pretty low. I would consider features that might be useful for you in other ways for example:
– An interactive exhibit guide with audio narration
– Having your promo videos and materials to hand or available for audiences to browse
– Health and safety checklists, forms or other weighty manuals it would be useful to have as .pdfs on a tablet.
– Gathering quick voxpops, comments or mailing list sign ups
– Guiding people around, showing onsite/indoor maps
– A fun photo booth or other social-media linked application
For Enketo: a web browser, in order of preference: Chrome, Safari, Firefox, Internet Explorer 10 or later
For ODK collect: in theory any device as early as Android version 1.6 ‘Donut’