Best practices for designing data collection forms

In the last blog post I looked at some key features to consider when looking for forms technology to help collect data.  However, regardless of which tool you choose, there are some basic usability issues to consider when designing your forms.  Following best practices for designing data collection forms can save your colleagues time when entering data and can also play a key role in ensuring that the data entered is accurate and of high quality.

Image from UX Movement

Image from UX Movement

So what are some of the things you should consider when designing a new form or re-designing an existing one?  How do you address the elusive concept of usability?  Luckily there is already a lot of research on this topic, so we can learn from mistakes that others have already made.

1. Don't limit your thinking to paper forms

You may already have paper forms and want to now create web or mobile based counterparts.  In this situation it’s easy to try and create a direct copy of your paper form.  However, paper and digital forms have different features.  What works well in one may not work well in the other.  Consider the following:

Layout

Paper forms are often designed to be as small as possible.  The page size is a crucial design constraint, which can lead to creative layouts to avoid a one page form becoming a two page form.  With digital forms you have no such constraints.  Consider breaking your form out into a series of sections or tabs with names that help explain what the content is for each section.  Think carefully about how best to organise the questions in your form sequentially.  What flow makes most sense when entering data?

Take the example of entering address information:

Consider your full data collection process

Before you re-design all your paper forms as digital forms consider carefully what your full data collection process will look like.  Let’s take two examples:

  • Switching from using paper forms to using mobile forms: Your field teams will enter data directly into mobile devices and it will be synchronised (manually or automatically) with a central computer or server.
  • Switching from using paper forms to using web forms: Your field teams will continue using paper forms for primary data collection, but will then enter that data into a web version of the form to help with analysis.

In the second scenario paper forms are still a part of your data collection process.  Hence, you should consider carefully any changes in layout or design that will complicate the process of data entry from paper to online forms.  If your fields are arranged in a different order on the paper and digital forms then the data entry process will be more complex and more likely to include transcription errors.

2. Personalise forms using conditional logic

Digital forms can include conditional (or skip-logic).  This makes it possible to show or hide some questions based on the responses to other questions.  This can greatly improve usability by helping show users only questions relevant to them.  You are in effect combining several personalised forms into one form.

3. Automate responses where possible

Paper forms may include fields for the person completing the form, the data or time it was completed and the location where it was completed.  As per the example below, digital forms may be able to complete all these fields automatically, saving time.

  • Name of person completing the form - Linked to user’s account on the system
  • Date and time - Entered automatically as the time the form was saved
  • Location - Taken from GPS coordinates with mobile forms

4. Use defaults and help text

Digital forms have two nice features that can help enhance usability.  First, you can normally define detailed help text that gives guidance on what is expected for each question.  Unlike paper forms where this must be permanently shown, on digital forms it is normally shown only when you click on the help icon for each field.  This avoids cluttering your form with lots of extra text that advanced users don’t need to see.

Some tools will also let you specify default answers to questions.  If you know from your analysis (or common sense) what the most common response is to a question then consider setting it as the default.  That way users wanting to give that response see that the answer is already supplied and can move on.  Others needing to give a different response simply over-write it.

5. Shorter is better

When designing your forms consider if you really need all the data that you are asking for.  Consider the chart below.  This shows the completion rate for forms based on the number of fields.  In circumstances where your colleagues or users are not required to complete each form (eg with surveys and organisations with optional reporting requirements) consider the impact of each additional field you add.

Chart and data sourced from Hubspot

Chart and data sourced from Hubspot

Is it really necessary to ask all the questions on your form?  We suggest you pose this question:

Do we really need this information to guide our decision-making for this project or programme?

If the answer is no (or not clear) then consider removing the question.  In my view this should also apply to data needed to monitor indicators.  If your programme indicators don’t link back to decision-making processes (management or learning) then why measure them?

A controversial position that donors may disagree with perhaps, but one that certainly links your data collection to improving programme quality.

6. Collect only what you need as you need it

Paper forms can be hard to distribute, collect and analyse.  Often questions relating to multiple stages of the project or programme are included in the same form.  Field staff complete part of the form at one point and then return later to fill out the rest. 

Digital forms - particularly mobile-based forms - are easy to distribute and collect.  Instead of designing one large form, consider designing several forms that are linked to specific stages of the project implementation process.  This approach makes more sense to field staff who can focus on collecting only data relevant to what they are doing at the time.

Platforms like BetterData use workflow to link these forms to stages of implementing a project.  This approach guides field staff as to which forms are needed at which stages of implementation.

7. Make careful use of field validations

Digital forms typically allow you to validate the responses given in a range of ways.  This could simply require that a response is given to a field before the form can be saved.  Or it could be more advanced to require a specific type of response to each field.  For example:

  • A number greater or lower than X is entered
  • A word with at least X characters is entered
  • A data before or after X is entered
  • A number or word entered follows a specific pattern (e.g. an email address pattern or phone number pattern)

Validations are a great way to improve your data quality.  When used well they can avoid users entering data that doesn’t make sense.  When used poorly they can block users from saving a form when they don’t have an answer to a required question or want to give an answer in a format different to the one that you were expecting.

Be sure to field test your forms carefully if you are using advanced validations.

8. Test and test again

Finally, make sure you allow sufficient time to field test your forms before rolling them out.  If time allows consider:

  • A pre-test by colleagues who you work with - they can help catch obvious mistakes and give feedback on the overall flow of the form
  • A field-test by a sample of people that you expect to use the form - they should ideally complete the form using real data in normal circumstances.  This often highlights any 'edge-cases' that you may not have considered.

If possible try to watch some of the testing.  You'd be surprised what you can learn by quietly watching over someone's shoulder as they puzzle over what you thought was a clearly worded question... 

Key reading on form design

If you want to read more deeply on this topic then take a look at the following resources.  

Sensible Forms: A Form Usability Checklist - A check-list of key usability issues to consider when designing forms.

Mobile Form Usability: Avoid Splitting Single Input Entities - When designing web based forms it often makes sense to split certain fields out.  This article suggests that this can actually introduce usability issues on mobile forms

Web Form Innovations on Mobile Devices - Luke Wroblewski is one of the pioneers of mobile first design.  In this article he explores some of the latest innovations that can help improve the design of mobile forms.

Why Users Fill Out Forms Faster with Top Aligned Labels - Detailed usability research comparing the time taken to complete forms with side versus top aligned labels.