Chapter 5 Questionnaire Components

5.1 Overview

Questionnaires are currently the most used items on Experimentum. They are made up of one or more individual components, available in 3 different types: mixed, radiopage and ranking.

Questionnaire Type	Response Type	DV
Mixed Questionnaire	Multiple types of questions can be combined in a single component.	Variable by question type. See each questionnaire type for more information.
Radiopage	Multiple questions asked across a single radio button response scale	Numeric value which is not seen by participants that corresponds to text labels which are seen by participants. Can incorporate automatic reverse-scoring.
Rank Order	A list of options which are dragged into relative rank order by participants.	Rank position assigned by participants of the various items.

5.2 New Questionnaire

To create a new questionnaire, navigate to the researcher’s section of the page by using the menu on the right and selecting the questionnaires option.

From here, you will be able to see the questionnaires for which you have ownership status (ones you have created yourself, or ones that someone else has shared with you). To create a new questionnaire, click the button on the top right “New Questionnaire”:

The following pop-up box will appear, allowing you to choose the type that you want to create:

The types that you choose will depend on what type of functionality your study requires. You should note that pencil and paper questionnaires will not always be able to be copied exactly onto Experimentum. If you make any changes to questionnaires to deliver them electronically you should discuss required changes with your supervisor to ensure their validity has not been compromised.

5.3 Save Questionnaire and Reset Buttons

Once you have entered information about the questionnaire you should save it. Saving regularly is good practice. Do this by pressing the save questionnaire button.

The reset button is akin to a “revert” button, where any changes made after the last save can be discarded and you return to the last saved version.

The JavaScript for saving the questionnaire occasionally stops working on some browsers, so we recommend saving the questionnaire after every 3 or so changes to minimise loss of work in the event of an error.

Please note that the site currently does not support version control, so once you save a change the previous version will be overwritten.

5.4 Accessing and Editing Previously Saved Questionnaires

To access your previously saved questionnaire component, navigate to the researcher’s area of the site by selecting “Researchers” from the menu on the right-hand side of the page.

You can access your questionnaires by accessing the questionnaires section of the page by using the button below:

Alternatively, if you remember the number of the questionnaire component, you can type it in to the grey rectangle in the experiments button. When you press return you will be taken straight to the information page of the questionnaires

To edit the questionnaire, press the “edit” button at the top of the page and you will be taken back into the editable version:

5.5 Questionnaire Information

All types of questionnaires will ask you to provide information about it, the only difference will be the Questionnaire Type, which will correspond to the type of questionnaire you have chosen on the previous pop-up box.

Name for Users: this is the name of this component that will be displayed to users. Please ensure that it is appropriate.
Name for Researchers: this is the name of this component for researchers. It will be displayed on lists and visible to you, your supervisor and admin staff. It should be informative and appropriate.
Question Order: this indicates how you want Experimentum to display the questions contained in this component only.
- Fixed will display the questions to the participants in the order you have put them on this page.
- Random will mix the order of question presentation up automatically. Order of presentation will be recorded in your data when you download it.
Limited to: limitations will only allow people in a certain age range and with a certain gender identity to complete this component of the study.
- If you are planning on allowing anonymous participation you should not set limitations as age and gender identity of these users will be unknown and may not be allowed to do these sections of the study if participants refuse this information.
Labnotes: this is a short blurb about this component of your study. You should always add lab notes so that you, your supervisor and the admins will be able to tell what this component is about. The following sections will describe each questionnaire type and what they can do.

5.6 Questionnaire Tab

The questionnaire tab on the lower half of the page is different for each questionnaire. Please see the appropriate sections below for more information on these.

5.7 Feedback Tab

In most cases you should leave this empty. The feedback for the top-most set is normally where you would put your debriefing information. You can see the later sections on projects and debriefing.

5.8 Adding/Deleting Questions

All questions must be given unique names within the component they are in. Components cannot be saved if you have questions with duplicate names.

This applies to questions added individually or in bulk.

You might find it easier to give each question across the entire study unique names.

5.8.1 Adding Individually

In order to add a new question, you must ensure that all editing options are closed on the existing questions by clicking the pencil to get rid of the yellow surround or the cog to close the menu.

Pressing the button at the bottom of the page will allow you to add a new question.

Experimentum will add a new question by directly duplicating the previous question, with all the same text and attributes. Please ensure that you set a unique name for the new question before you attempt to save the questionnaire as questions with duplicate names will be removed by the system.

5.8.2 Adding in Bulk

It is also possible to add some types of questions in bulk by using a spreadsheet. Unfortunately, not all types are covered, but the most basic and most frequently used types are. Spreadsheet templates are available by clicking “add from spreadsheet” and selecting the underlined type of questionnaire from the options just above the buttons.

Each of the column headers will relate to each of the question attributes you would use when setting the questions manually. For ranking and radiopage questionnaires not all of these headers will be applicable. Double click on the ones you do not need to remove them as I have done in the image below:

Fill the template spreadsheet in and save it onto your own storage, copy the cells that contain your question information. See the section below for more about question information. Double click on the section that says, “Double-click to paste data from a spreadsheet” and paste it when in the new entry box that appears:

If you do not use the template spreadsheet, but one you have made yourself, ensure that the options that you are specifying match. You can also drag the headers on the entry window from left to right to re-order them if your own spreadsheet does not match the headers here.

Column Header:	Refers to:	Options:
name	dv_name (in data = q_name)	Names must only contain letters, underscores and numbers (numbers must not be the first characters).
question	The question that you are asking the participant.	Whatever you want to ask the participant.
type	The type of question, akin to the options selected from the menu opened by the cog	* select: pulldown menu selectnum: numeric pulldown menu radio: radiobuttons radioanchor: radiobuttons with anchors datemenu: date menu text: short text
options	Where you would specify the response options available to the participants.	Applicable to select and radio types. Numerical code, followed by “:”, and then text option. Specifying multiple by separating options using a semi-colon (;)
maxlength	Where you would specify the length of the response	Applicable to radioanchor and text. Radioanchor: how many points in the range. Text: how many characters allowed.
low_anchor	Specifying the low anchor for the selected option.	Applicable to selectnum, radioanchor and datemenu. Selectnum specifies the lowest number in the range of the dropdown menu Radioanchor specifies the text for low anchor Datemenu specified the lower end of the year range.
high_anchor	Specifying the high anchor for the selected option.	Applicable to selectnum, radioanchor and datemenu. Selectnum specifies the highest number in the range of the dropdown menu Radioanchor specifies the text for high anchor Datemenu specified the higher end of the year range.

Each of the three types of questionnaire component has their own template spreadsheet available for download, which are linked in the dialogue box you get when you click to “add from spreadsheet”. If you intend to add questions in bulk you should consult these spreadsheets for guidance in the first instance as examples are given.

5.8.3 Re-Order Questions

If you wish to present your questions in a specific order, and this order is different than what you have in your questionnaire component, you can click on the arrows button between the dv_name and the question text to move the question up or down the order of presentation.

5.8.4 Deleting Questions

Questions can be erased by clicking the picture of the trash/bin next to the question you want to delete.

Please note that the site currently does not support version control, so once you save a change the previous version will be overwritten. This includes deleted questions- once you have saved the changes they are gone.

5.9 Questionnaire Types

5.9.1 Mixed Questionnaire

Allows you to present multiple question types and gather multiple response types in a single component. When you select a mixed questionnaire, you will be taken to this page:

As you can see you have the same questionnaire information section at the top, as you will have in all questionnaires. The differences lie in the types of questions that you can include. Mixed questionnaires can accommodate multiple types of questions on a single page.

Attributes

Instructions: when you click the part that says “Click here to set instructions” you will be able to enter text instructions to the participants for this entire component. You will also be able to enter Markdown and HTML code, which allows you to link to and embed additional media or format the text in specific ways.
- If you use HTML tags you must ensure that you have matching opening and closing tags, as unmatched ones will prevent the component from recording data.
dv_name: this is the name of the dependent variable, the name that will be assigned to the column q_name in the downloaded data indicating which question the participant has answered. You should set one which is unique to this component and perhaps across all components.
Question: clicking the part which says “click here to set your question” will allow you to enter the question text that the participants will see, the question that you want to ask them.

Changing Question and Response Type

Clicking the cog next to the question and then the arrow in the resulting menu will allow you to choose the type of question you want to set.

When you have selected the type of question, you can then edit the response types that can be made by the participants by clicking the pencil next to the cog. When this editing mode is active, the response section will be unfolded, and will be displayed with a yellow surround.

You must close this editing mode by clicking the pencil a second time, getting rid of the yellow surround, before you add a new question, click the cog or save the questionnaire.

Question Types

Pulldown Menu: pulldown menus give your participants a set of text option responses to choose from. Each text option has a numeric coding value associated with it, which will be the returned value in the downloaded data in place of the text. This is specified before the colon. The text options are specified after the colon, these are not returned in the data.
- Each text value should be associated with a unique numerical coding value, non-unique numerical codes will be deleted upon saving the questionnaire. You can add multiple options to the menu.
Numeric Pulldown Menu: this allows you to set numerical values to a pulldown menu, with no associated text label. The range that is set is the range that will be displayed to the user, as below.
- You should use whole numbers only.
Slider: sliders provide you with a way for participants to return a finer-grained numerical response on a spectrum between two anchor points. A slider scale will be displayed to participants, with the low end of the range associated with the low anchor, and the high end the high anchor. Participants will move a slider along the scale, and a numerical value associated with the position they put it in returned in the data. You set the scale limits and increments, in this example from 0 to 100 in increments of 1.
- You can also change the text anchors by clicking on them as shown here.
- Slider knobs do not appear until the participant has clicked somewhere on the scale.
Radio Buttons with Anchors: this option allows you to set a scale with a low anchor and a high anchor, which you can change by clicking the text. You can add or remove points to the scale by pressing the + and – buttons on either side of the response. On this scale example below, a number will be returned associated with the point on the scale the participant submitted, here 1 (lowest) to 5 (highest).
Date Menu: date menus allow the participants to choose a specific date. You can set the range of dates to choose from, in the example below the participant is allowed to go back 100 years from the current date (-100y) but cannot go further into the future than today (+0y). When participants click on the response, they will be given a calendar to pick from, instead of typing the date.
Time: This allows participants to enter a time, in 24 hour format, using hours and minutes. You can change the response options (from single minutes to increments of 15, for example), but it is not advised.
Countries: participants can choose from a list countries. Returned in the data as the two letter ISO-3166-1 alpha-2 code.
Short Text: allows participants to enter free-response text up to and including 255 characters. Returned in the data as character strings. The maximum number of characters can be changed by changing the number after maxlength:.
Long Text: allows participants to enter free-response text over 255 characters. Returned in the data as character strings. If you do not need more than 255 characters, please use the short text to minimise strain on the server.
Message: allows you to put a line of text into a questionnaire which does not require a response amongst other questions. Note that to ensure that this appears where it should that these questions should be presented in a fixed order.

5.9.2 Radiopage

Radiopage questionnaires contain the same questionnaire information section at the top of the page as a Mixed Questionnaire.

The format of this questionnaire allows the user to set a number of questions with radio button responses on a single page, with a single response scale for all questions.

Each point on the scale has its own text label, and an “option value” number associated with it, rather than only the two extremities of a multiple point scale as in the radio buttons with anchors in the mixed questionnaire.

Attributes

Instructions: when you click the part that says “Click here to set instructions” you will be able to enter text instructions to the participants for this entire component. You will also be able to enter Markdown and HTML code, which allows you to link to and embed additional media or format the text in specific ways. See stimuli for more information on this.
Option values: these are the numerical equivalents of the text labels for each of the responses a participant can make to any question on the page. This number is not displayed to participants. The number you set is largely arbitrary, but the numerical figure associates with the choice rendered by participants is returned in the data set upon download and text values are not. You will need to know what number corresponds to which text response in order to carry out and interpret your analyses. It is possible to have automatic reverse scoring- see here for more information.
Option text: this is the text label associated with each response the participant can make to any question on the page. The associated text is what the participant sees, and it is displayed in the construction page directly below the numerical value you set. The option text is not returned in the downloaded data set.
Adding/removing options: you can add or remove the number of response options on the scale you give to participants by pressing the + or – symbols in circles on the right of the scale at the top of the questionnaire.

dv_name: this is the name of the dependent variable, the name that will be assigned to the column q_name in the downloaded data indicating which question the participant has answered. You should set one which is unique within this component, or perhaps across all components in your study.
Question: clicking the part which says “click here to set your question” will allow you to enter the question text that the participants will see, the question that you want to ask them.

Forward and Reverse Scoring

In many questionnaires it is common to find items which should be “reverse-scored”. Here you can indicate if the question is forward or reverse scored, and the system will make the changes to the data for you, saving you from having to recode them later.

Example: your scale at the top indicates 1-5 as below. As you can see “much less than average” is associated option value of 1. The forward scored question is marked “fwd” in the pulldown menu on the right of the table and the reverse scored question is marked “rev”.

If a participant chooses “much less than average” on a forward scored question the dv in the output will be “1”. But what happens to the reverse scored question?

This is the data output here. As you can see where the participant chose the option that returned a “1” for the forward scored question, for the reverse scored question that same choice returns a “5” in the data.

5.9.3 Rank Order

Attributes

Ranking questionnaires present a list of text options to participants to put into a specific order, based on some judgement they are asked to make on them.

As you can see, there is a single response type associated with this questionnaire type. The DV in this case will be the rank position assigned by the participants. Setting the order to fixed will set the initial order up on the order you have specified on the questionnaire editing page and random will randomise it.

Each question will be an option in the list of items to rank order. Participants will be able to drag each text item to rank order them, with the number specified in the column next to the text. The final presentation to participants looks like this:

In the data, the question name is saved as q_name and the rank order assigned by the participant is saved as the dv in the downloaded data:

5.10 Using Stimuli with Questionnaires

In some cases you might wish to include some stimuli with a questionnaire. For example, you might want to present an image with a set of questions asking the user’s opinions about that image.

We would recommend that you upload your image as per the instructions given in the stimuli page and use HTML or Markdown in the instructions at the top of the questionnaire component that you want to display the stimuli on.

First, you will need to find the path of the stimuli you have uploaded to the server by going to “My Uploaded Stimuli” by navigating to “Researchers” > “Stimuli” > “Upload Stimuli” and looking in the folders at the bottom of the page. Find the image that you want to use and click on it in the folders. It will display on the right hand side and the file path will be above it. Copy the file path, seen circled here:

Next you will need to paste that into the introduction box using an appropriate image tag. Here you can see the HTML syntax to link to an image:

The code is:

<center><img src="file/path/here.jpg" width="500px"></center>

width=500px alters the image displayed by resizing the image to be 500 pixels wide whilst maintaining the aspect ratio.

This is OK for one or two images, but if you are doing this with many images we would recommend that you resize the stimuli before uploading them.

<center> ensures that the image stays in the middle of the page. </center> is a “closing tag” and must be put in place to ensure that data for the component is saved to the server.

When participants go through the questionnaire this is what they will see:

This method works for all types of compatible stimuli, although the code used to display them will vary.

For audio:

<center><audio controls>
  <source src="file/path/here.mp3" type="audio/mpeg">
Your browser does not support the audio element.
</audio></center>

Displayed as a small media player:

And for video:

<center><video width="320" height="240" controls>
  <source src="file/path/here.m4v" type="video/mp4">
Your browser does not support the video tag.
</video></center>

Whilst you can link to externally hosted stimuli we would recommend that if possible you upload your stimuli to our server. This will ensure uninterrupted access to your media.

We cannot always host all stimuli. Files larger than 3mb each should be uploaded elsewhere to suitable platforms, such as YouTube, as loading times are often too long for files which exceed this limit.

Embedding media from outside sources may require different code than provided above.

See the stimuli page for further information on preprocessing and uploading stimuli.