Monday, 17 January 2022, 8:56 PM
Site: Flinders Learning Online
Topic: FLO Staff Support (FLO_Staff_Support)
Glossary: How-to glossary
Andrea Rankin

#### Quiz - create a fill-in-the-blanks (missing words) question

This entry relates to the Quiz activity.

Want to see how this question type works?

#### Steps

1. In the quiz (Quiz > Quiz administration > Edit quiz) or question bank (Administration > Question bank), click Create a new question

2. Choose Select missing words and click Add

3. Give the question a category

4. Give the question a name (required field) (this is only seen by teachers – make it something meaningful so when you view the questions in a list, you know what they are about)

5. Enter some question text. Where you would like the student to choose the missing word, enter a number on double square brackets, for example [[1]]. Number each gap sequentially starting at [[1]] – that is, [[1]], [[2]], [[3]] etc (required field)

6. Give the question a default mark (required field)

7. Fill in the choices. Choice 1 is the correct answer for the gap indicated by [[1]], choice 2 is the correct answer for the gap indicated by [[2]], and so on.

Choices with the same value for Group appear in the same drop-down lists (Shuffle)

8. Fill in General feedback for the quiz question

9. Click Save changes

#### Quiz - create a matching question

This entry relates to the Quiz activity.

Want to see how this question type works?

#### Steps

For this question type, you must provide at least two questions and three answers. You can provide extra wrong answers by giving an answer with a blank question. Entries where both the question and the answer are blank will be ignored.

1. Under Topic administration > Question bank > Questions, click Create a new question

2. Select Matching and click Add

3. Give the question a Category

4. Give the question a Question Name (required field)

5. Enter question text (required field)

6. Decide on the default mark for the question (required field)

7. Fill in General feedback (students will see this once they have answered the question)

10. Continue adding text for questions and the answer. You can provide extra wrong answers by giving an answer with a blank question

11. Click Save changes

#### Quiz - create a multiple choice question (MCQ)

This entry relates to the Quiz activity.

#### Create an MCQ

Want to see how this question type works?

##### Steps
1. In the quiz (Quiz > Quiz administration > Edit quiz) or question bank (Administration > Question bank), click Create a new question

2. Select Multiple choice and click Add

3. Give the question a category

4. Give the question a name (this will only be shown to teachers) (required field)

5. Fill in question text (required field)

6. Give the question a default mark (required field)

7. Enter some general question feedback

8. Choose whether the question has one answer only or multiple correct answers. 'One answer only' means users will only be able to select one box; 'Multiple answers allowed' means users will be able to select more than one box (if there is a specific number of correct answers, you may want to say this in the stem/lead-in sentence: 'Please select two of the following.')

9. Select whether you want to shuffle the question answers

10. Under Answers, fill in the choices

If you selected multiple correct answers for the MCQ (see step 8 above), you are advised to give wrong answers negative scores, to discourage students from selecting all options (you should say that they will be penalised for wrong answers in the question stem)

11. For the correct answer, change Grade to be 100%. If there are multiple correct answers (see step 8 where you would need to have selected 'Multiple answers allowed'), allocate the 100% across all correct options; for example, if there are 2 correct responses allocate them 50% each. (Note: they do not need to have equal weightings)

Assign to a correct answer in a question with one correct answer

Assign to a correct answer in a question with multiple correct answers

12. Click Save settings

#### Tips for creating MCQs

Bloom's level/s of taxonomy, what the item is testing (descriptor), and distracter plausibility:

 Question number x Correct answer (key) Learning objective Bloom's level of taxonomy Descriptor What the item is testing Distracter plausibility A....B....C....
##### Question tips
• Use a consistent style
• Use clear and concise wording (can be complex but not complicated)
• Make the question 'stem' either a direct question or an incomplete statement
• Avoid double negatives

You may want to use a 'stimulus' prior to the question – this could include introductory text/context, an image etc.

• Make the key (right answer) and distracters (wrong answers) approximately the same length, and use similar wording to avoid obvious standouts
• Keep answers consistent in grammar and logical to the stem (question)
• 3, 4 or 5 options per answer? There is no set preference, but the more options the harder it is to think up a plausible 'distracter'
• Avoid using 'none of the above' or 'all of the above' (especially if answers are shuffled!)
• Use a logical pattern but don't always put the correct answer in the same position
• Provide good feedback for all options (where relevant)
• Where possible, ensure that wrong answers represent a common misconception
##### Open response items
• Provide information to markers on full credit, partial credit and no credit
• If not marking for grades, provide good feedback on likely responses
##### References and resources

The Centre for University Teaching ran two workshops in 2013 with invited speakers from the Australian Council for Educational Research (ACER):
MacKinnon, Philip, MCQ assessment workshop 24 June 2013
Khoo, Siek Toon, Introduction to measurement concepts and assessment in education
"Bloom" search results (eLearning literature and resources collection)

#### Quiz - create an embedded answers (Cloze) question

This entry relates to the Quiz activity.

Want to see how this question type works?

#### Steps

To set up this question type, you need to generate cloze text. This means that where an answer is required, you will need to generate bracketed instructions so the answer can be marked automatically, and feedback generated. For example:

This can be a complex process if you are not familiar with it, and it is easy to make a mistake. You could try the Moodle Cloze editor (external resource) which can generate cloze from your text. Once generated, you can copy and paste it into the Question text area in this question.

1. Open the quiz and either:
1. Edit the quiz, click on an Add link on the right side of the page, and select + a new question.
2. Open the question bank and click on the Create a new question button

3. Give the question a Category and Question name (required field)

4. Enter question text (this will include Cloze text – see note above) (required field)

5. Fill in General feedback (students will see this once they have answered the question)

6. Give a penalty and hint for each incorrect try

7. Click Save changes

#### Quiz - create a new question (question types)

This entry relates to the Quiz activity.

#### Create in the question bank or in the quiz

NOTE: Before creating questions, you should set up categories in the question bank. Then use one of the two methods below to create questions in those question banks.

##### 1. Create from within the question bank
1. Navigate to the question bank from your topic’s Topic management menu

2. Select the category you want the question to be in.

3. Click on the Create new question button and choose the desired question type

4. Enter the details of your question.

5. Save your question. You'll then be taken back to the question bank category where the question is

6. Click on the Edit link to the right of your question and select Preview to test if it is working as expected

Tip: If you want a series of similar questions, you can duplicate then edit questions

##### 2. Create from within the quiz
2. Click on the cog in the top right corner and select Edit quiz

3. Click on one of the Add links on right side of the page, then select a new question

4. Enter the details of your question, making sure you choose an appropriate question bank category* for your question

5. Save your question. You'll then be taken back to your edit quiz page.

6. Click on Edit to the right of your question and select Preview from the dropdown list to test if it is working as expected

*Note: We strongly recommend that you do not use the category "default for your quiz name" as this category is segregated from the question bank. Questions in that category can't be used in other quizzes and can't be viewed with the main question bank. Storing questions in this segregated category can lead to confusion.

#### Create a new question (from question types)

Choose a question type and follow the instructions for creating under a category:

#### Import/export questions

You could also import questions from a file, or export questions to a file – talk to your eLearning support team.

#### Duplicate (copy) a question

1. Open the question bank you want to duplicate a question in

2. Find the question you want to duplicate

3. Click on the Edit link next to the question and select Duplicate from the menu that appears.

4. The question will open in an editing screen, where you can make the changes you want (eg to the Question name) and continue editing

5. Click Save changes when you have finished editing the question duplicate

6. The duplicated question will now appear on your list of questions.

#### Quiz - create a numerical question

This entry relates to the Quiz activity.

If you are interested in this question type, you may also be interested in Calculated, Calculated multichoice and Calculated simple. To create one of these types, at step 2 (below) select one of these options instead of Numerical.

Want to see how this question type works?

#### Steps

1. Open the quiz and either:
• Edit the quiz, click on an Add link on the right side of the page, and select + a new question.
• Open the question bank and click on the Create a new question button

2. Select Numerical and click the Add button.

3. Give the question a Category

4. Give the question a Category Name (required field)

5. Enter question text (required field)

6. Decide on the default mark for the question (required field)

7. Fill in General feedback (students will see this once they have answered the question)

8. Select answer 1, the error for getting it wrong, and the grade. Do this for all other correct/incorrect answers

9. Provide feedback for the answer (whether right or wrong)

10. Under Unit handling and Units, decide on the configuration (this will only apply if the answer is not a whole number)

11. Click Save changes

12. Preview the question to make sure it works correctly

#### Quiz - create a quiz

This entry relates to the Quiz activity.

Now you can create the quiz, then add questions to it.

1. In the topic, click the Turn Edit On button

2. In the module where you want to add the quiz, click Add an activity or resource

3. Select Quiz from the Activities tab

### Quiz settings

#### General

Give the quiz a Name and Description

#### Timing

Under Timing, you can choose to give the quiz Open and Close dates. Tick the Enable box to the right of the date you wish to enter, then enter the date.

Timeline block: The 'Close the quiz' date will show to students in the Timeline block.

Depending on the purpose of your quiz, you may want to change the default settings (see below). You can also click the question markprompt next to some options to find out more about a setting.

You can make the quiz available at different times for different groups or users in the sections via the Actions menu.

• Before the opening time, the quiz will be unavailable to students – they will be able to view the quiz introduction but not the questions. Quizzes with start times in the future display both the open and close date to students.

• After the closing time, students will:
• not be able to start new attempts and any open attempts will be automatically submitted.
• still be able to see the quiz description and review their attempts. What they see will depend on the settings you choose for Review options (ticked boxes in the 'After the quiz has closed' column).
##### Time limit

If enabled, the time limit is stated on the initial quiz page and a countdown timer is displayed in the quiz navigation block (see image to the right). Once the timer runs out, the quiz is submitted automatically. If the quiz close time is reached before the time runs out, the quiz will be automatically submitted (ie student does not get full time). You can make the quiz timing different for different groups or users in the  Group overrides or User overrides sections via the Actions menu.

If you have categories in your gradebook (eg for a quiz set), select the category you wish the quiz to be in here.

##### Attempts allowed

Unlimited (default), 1-10

Your choice here depends on what and how you are testing (ie the purpose of your quiz):

• If a diagnostic test, then probably choose 1; if a formative test, then maybe more often as the student is in the learning process.
• If the quiz is shuffled (see Layout) or has random questions (see quiz - add questions), then the student will get a new version for each attempt, useful for practice purposes.

You can change the allowed number of attempts for different groups or users in the Group overrides or User overrides sections of the Administration > Quiz administration settings block.

This setting only applies if more than 1 attempt is allowed. If each question builds on the last, you might set it at Last attempt, as you want to see that students are gaining knowledge/understanding progressively.

#### Layout

• Question order: Shuffled randomly – if selected, the order of questions in the quiz will be randomly shuffled each time a student starts a new attempt at the quiz.
• New page: You may want to group questions on a particular topic together, but not too many on one page as this will require students to scroll.  It is recommended that you do not use the setting 'never, all questions on one page', as student activity within the quiz will be less frequently logged and less information will be captured regarding the attempt.

Note: You can change how many questions per page (Actions menu > Edit quiz). You can Repaginate (eg 2 per page instead of 1 per page), or select questions and move them to a particular page.

• Navigation method: leave as Free (default) unless the quiz is building knowledge (last question relates to next question etc).

#### Question behaviour

• Shuffle within questions: if Yes (default), the option must also be enabled in question settings (when you create a new question).
• How questions behave: Deferred feedback (default) – students answer each question and submit the entire quiz, before anything is graded or they get any feedback.

Note: CBM stands for 'certainty-based marking' – if you choose this option, students will need to say how sure they are that the answer is right and will be penalised accordingly. More information about CBM (MoodleDocs)

• Show more… > Each attempt builds on the last: if multiple attempts are allowed and you set this to Yes, then each new attempt contains the results of the previous attempt, so students can concentrate on just those questions they answered incorrectly last time. If you choose this option, each attempt by a particular student uses the same questions in the same order, independent of randomisation settings. To show a fresh quiz on every attempt, select 'No' for this setting.

#### Review options

Tick the options you would like to choose from the table. Default settings are After the quiz is closed (if a closing date for the quiz has been set).

For each of the above settings, you can determine the timeframe when the students will see them:
During the attempt Immediately after the attempt Later, while the quiz is still open After the quiz is closed
These settings are only relevant for some behaviours, like 'interactive with multiple tries', which may display feedback during the attempt. These settings apply for the first two minutes after students have clicked 'Submit all and finish'. These settings apply after students have clicked 'Submit all and finish' and before the quiz close date. These settings apply after the quiz close date has passed. If the quiz does not have a close date, this state is never reached.

#### Appearance

• Decimal places in grades: Default is 2. This option determines how many digits will be shown after the decimal point when the grade is displayed. A setting of 0 for example will display grades as integers (whole numbers).

Note: If you use CBM (certainty-based marking) which deducts marks for not being certain, being wrong about being certain etc, then it makes sense to show the decimal places in grades. Likewise for 'Decimal places in question grades'.

• Show blocks during quiz attempts: Default is No – this is about usability (better display, no distractions).

#### Extra restrictions on attempts

These settings may be useful if you are using a FLO quiz as a final exam.

• Require password: If you specify a password then participants must enter the same password before they can attempt on the quiz.
• Require network address: You can restrict access for a quiz to particular subnets on the LAN or internet by specifying a comma-separated list of partial or full IP address numbers (eg 192.168. , 231.54.211.0/20, 231.3.56.211). This is especially useful for an invigilated quiz/exam, where you want to be sure that only people in a certain room are able to access the quiz.

Note: The Flinders network is not configured to allow restricting access to a given room.

• Enforced delay between attempts: You can set a time (from seconds to weeks) between the first and second attempt on a quiz. You can also (or alternatively) set a time from seconds to weeks for subsequent attempts after the second attempt (eg you might allow a student to take the quiz twice immediately with no delay, but if they want to improve their score with a third attempt, they are forced to wait a week and use the time for extra revision).
• Show more…Browser security: the options in this section offer various ways to try to restrict how students may try to 'cheat' while attempting a quiz.

#### Overall feedback

Overall feedback is feedback shown to a student after they have completed an attempt at the quiz. You can set the grade boundary (eg 100%, 65%, 50%) and apply appropriate feedback according to the range the student falls within (eg if a student gets below 65% they are referred to more information). This is a useful feature in a formative quiz where students are informing themselves/the teacher what they know/understand.

#### Common module settings

These settings are only relevant if you have groups and groupings set up in your FLO site.

• Group mode: you can choose whether the quiz should be organised by group.
Note: This only affects the results screen and allows you to filter results based on group enrolment.

#### Restrict access

None (default), Activity completion, Date, Grade, User profile, Restriction set

For example, you might have a series of quizzes and they need to do one before another – this would be the activity completion option but you would first need to enable completion tracking in the topic settings and quiz activities.

#### Activity completion

Apply these settings if you have enabled completion tracking for the topic and this activity.

#### Quiz - create a quiz essay question

This entry relates to the Quiz activity.

The essay question in quiz requires manual grading (marking).

Want to see how this question type works?

#### Steps

1. Open the quiz and either:
• Edit the quiz, click on an Add link on the right side of the page, and select + a new question.
• Open the question bank and click on the Create a new question button

2. Select Essay and click Add

3. Give the question a name (this is seen by teachers only) and fill in the Question text. You will probably want to change the Default mark to a number higher than 1

4. Provide general question feedback

5. If you want students to upload a file, select Allow attachments and choose a number from the drop-down menu (Note: if students are required to write a long essay as part of a quiz, we strongly recommend they write it in Word and either upload the file or copy and paste into the response box).

You can also list specific file types (e.g. docx, pdf, etc.) if you want to restrict what types of files students can submit.

6. Click Save changes

#### Quiz - create a short answer question

This entry relates to the Quiz activity.

This question type can be tricky if there is more than a one-word answer (which may have spelling problems!), or variations on the answer. You can avoid this problem by:

• using wildcards (*) – see information below Steps
• putting in the question stem (Question text) a proviso such as 'Please separate words with a comma' (if a two-word answer) or other similar guidance
• anticipating the variations and adding those to the answers with 100% as the correct grade for each variation

It would be a good idea to test this question on several users before you try it with students in the real situation, especially if it is assessed.

You can manually mark a short-answer question to avoid these problems.

Want to see how this question type works?

#### Steps

1. Open the quiz and either:
• Edit the quiz, click on an Add link on the right side of the page, and select + a new question.
• Open the question bank and click on the Create a new question button

3. Give the question a Category

4. Give the question a Category Name (required field)

5. Enter question text (required field)

6. Decide on the default mark for the question (required field)

7. Fill in General feedback (students will see this once they have answered the question)

9. Provide feedback for the answer (whether right or wrong)

10. Click Save changes

#### Using wildcards

You can use the asterisk character (*) as a wildcard to match any series of characters. For example, use:

ran*ing
to match any word or phrase starting with ran and ending with ing

If you really do want to match an asterisk then use a backslash like this:

\*
If you want one question with the two answers fuel and oxygen, you ought to be able to limit the number of variants by writing:
fuel*oxygen
This would accept "fuel oxygen", "fuel, oxygen", "fuel; oxygen", "fuel and oxygen", "fuel & oxygen" "fuel oxygen", "fuel und oxygen" "fuel&&oxygen". It would even accept "fuel or oxygen", "fuel but not oxygen" "fuel|oxygen" .
##### Example

Here are some answers and scores for a question "What does a rocket burn?"

1. oxygen*fuel
with a score 100%
2. *fuel*
with a score 50%
3. *oxygen*
with a score 50%
4. *air*
with a score 40%
5. *
with a score of 0%

The order of the answers is important. The answers are evaluated from 1st to last. When a match is found the process stops. If no match is found, the question is scored wrong and the general response is used. It is a good practice to put a wildcard as the last answer so the evaluation process knows what to do when nothing above it matches.

If any answer is the right answer (eg a non-graded open-ended quiz), you could have as the only answer * (worth 100%). This means all answers are acceptable. Make the question worth 0, and if you make the quiz non-graded (i.e. worth 0) it will not appear in the Gradebook.

#### Quiz - create a true/false question

This entry relates to the Quiz activity.

Want to see how this question type works?

#### Steps

1. In the quiz (Quiz > Quiz administration > Edit quiz) or question bank (Administration > Question bank), click Create a new question

2. Select True/False and click Add

3. Give the question a category

4. Give the question a Name (required field)

5. Enter question text (required field)

6. Give the question a default mark (required field)

7. Fill in general question feedback

8. Select the correct answer (true or false)

9. Enter feedback for response 'true'

10. Enter feedback for response 'false'

11. Click Save changes

#### Quiz - edit quiz questions (and regrade quiz)

This entry relates to the Quiz activity.

#### Before the quiz opens

This action relates to the 'Plan' step for Quiz. If, before a quiz opens, you discover that a question requires editing or correction, it can be easily edited.

1. Click the Topic Management button (cog icon) on the main menu, then the Question bank link

2. Use the Select a category drop-down list and select the category that contains the question

3. The questions contained in the category will appear. Locate the relevant question, then select Edit question from the Edit drop-down menu. After making the required changes, click Save changes.

If you discover the error after student access has commenced, it is possible to regrade your quiz. When a quiz question is altered, all instances of that question will be altered (ie if the question is used in another quiz it will also be altered). Students will not be notified of the changes made.

#### After the quiz opens

There are two steps to this process – edit the quiz question/s and regrade the quiz.

#### Step 1. Edit the quiz question/s

To regrade a quiz attempt (once it has already opened and students have attempted it), you will first need to edit the quiz question/s that needs changing. You can either edit the question/s (change mark, add comment) for all students OR for selected students.
##### Edit a quiz question for all students

3. Click on the cog to edit the question

4. Make changes, click Save changes

##### Edit a quiz question for selected students

1. Click on the quiz link

3. Choose the student and question in the Grades screen (scroll down) and click on the mark link (in the example below, it is 0.25)

4. In the next screen, scroll down and click on the Make comment or override mark link

5. Add a comment or change the mark

6. Click Save

How you regrade the quiz is different depending on whether it has random questions or not.
##### Quiz with no random questions

If you change the mark for a question to 0, you don't need to regrade.

If you adjust an answer (eg in an MCQ), you will need to regrade. You cannot change the question type (eg in an MCQ, you will need to leave it at one answer only, and make two answers correct, both worth 100%, rather than change the MCQ to multiple answers allowed).

1. Click on the quiz

2. Click Attempts at the bottom of the screen

3. Next to Show only attempts tick the box 'that have been regraded/are marked as needed regrading'

4. Click Dry run a full regrade (to see the outcome before you actually regrade – it is not a required step)

5. Click on the Continue button once the 'Regrade completed successfully' screen has appeared

6. Once completed, the attempts that need regrading will appear in the regrade column with 'Needed' appearing

8. Click 'Continue' button once Regrade completed successfully appears

9. To complete the process click Regrade all

10. Click Continue button once 'Regrade completed successfully' appears. You will now notice that the regrade column disappears, indicating that your quiz has been regraded

##### Quiz with random questions

If you need to remark one question you will need to identify which students were given the question, as that particular question could have occurred in any order (eg question 1 for some students but question 4 for others).

Example: If you have a randomised quiz of 20 questions, the problem question could be given in any of the 20 possible positions within the quiz.

1. In the quiz, click the Actions menu cog and choose Manual grading under the Results link

2. Click the Also show questions that have been graded automatically link

3. Locate the problem question for each of the possible question slots and edit however many results there are by clicking on update grades in the Automatically graded column. (The random question has a round symbol next to the question name. ) In the case below, you will need to edit 2 attempts on this question

4. Once you have edited the question and saved the edits, you will see that this question is no longer automatically graded – you have manually regraded the question

#### Quiz - import questions

This entry relates to the Quiz activity.

You can import quiz questions into the question bank from a text editor (eg Microsoft Word). The instructions below relate to multiple-choice questions (MCQs) in text format only (ie no images etc).

If you need help, contact your local eLearning support team.

#### Steps

##### In a text editor

1. Create your MCQs in a text editor such as Word, using this template (Aiken format). Choose the format for your questions (Option 1 or Option 2) and follow the instructions

2. Save your MCQ file as a .txt file and save in a UTF-8 format

3. Create a category in the question bank for your questions (if you don't already have one) – this is where your imported questions will be located

4. From anywhere in the topic, go to the Topic management panel > Topic settings > Import

If you are inside the quiz, you can also use the cog icon in the top-right corner of the page and select import

5. In the 'Import questions from a file' screen, select Aiken format for the File format (suitable for these question types/file type)

6. Under General, select the
Import category. Leave the settings Match grades and Stop on error as the default settings ('Error if grade not listed' and 'Yes')

7. Under
Import questions from file, either Choose a file... or drag and drop in the box, and click Import

8. The next screen will show you the questions you have imported. Click
Continue

9. The next screen will show you the question bank category you have put your imported questions into. Check that your questions are there (you can also move them to another category if you want)

Learn how to add questions (including random questions to a quiz (edit quiz).

#### Quiz - main entry

Whether you are starting from scratch or working with a quiz already created, using the quiz activity in a topic ideally consists of 5 stages, in a looped process.

##### 1. Plan  |  2. Build  |  3. Test  |  4. Administer  |  5. Review  || Support

The Quiz activity enables a teacher to create out-of-class quizzes comprising questions of various types, including multiple-choice, matching, short-answer and numerical. See Active quiz if you wish to use a quiz in a face-to-face lecture/tutorial (ie in class), with a classroom of computers/tablets/phones (devices).

##### Accidentally deleted a quiz?
It's important that you don't use the recycle bin to restore a deleted quiz. Refer to the quiz troubleshooting page, I accidentally deleted a quiz and want to restore it, for further information.

#### Good practice guides and tip sheets

Good practice guides and tip sheets have been developed to support quality in both curriculum design and teaching practice. Good practice guides provide a pedagogical overview and tip sheets provide you with practical strategies and ideas for implementation. Links to quiz-related resources are provided below.

The better your planning, the less likely you are to run into a problem once students start doing the quiz. Therefore, it is advised that you have a process for creating your quiz. Your college may have policies or guidelines around quizzes. Whether you are starting from scratch or revising a current quiz, these concepts, planning questions and tips that may help:

• What is your quiz's purpose? The answer will determine the settings you choose when you create the quiz.
• What principle/knowledge are you testing – does the quiz item test it?
• Line up quiz questions against the learning outcomes (you could include this information in the quiz question)
• Connect the questions with Bloom's level/s of taxonomy (Wikipedia) – again, what is being tested?
• Be creative – you could use scenarios, embed videos/other resources, provide learning pathways (eg hints, tips, more information, think again)

You have planned your quiz – its purpose (which determines quiz settings), the category/ies and questions, the feedback you will provide. Now you are ready to set up the quiz.

1. Create a quiz
2. Add questions to a quiz (edit quiz) – including random questions
3. Edit quiz questions (and regrade quiz)
4. Calculate a topic total including the best X from Y grades – if you have multiple (eg weekly/two weekly) quizzes in a topic and want to count the best 8 out of 10 (for example)

You've created your quiz and added questions. If you are confident the quiz is ready, it is a good idea to preview it to see if it works before making it available to students. When you preview the quiz, you will experience it as a student.

While the quiz is open, you can view how students are progressing (submitted or not, how many attempts so far etc). Once the quiz has closed, you can finalise the grades.

Reviewing your quiz (question validity, student experience and learning outcomes, alignment with topic learning outcomes) entails collecting data, analysing the data, and acting on it. Your data may also include student feedback – verbally and electronically. This step feeds into future quiz planning (step 1).

Training and support

Troubleshooting

Training

Support

You may have one of the following issues:

#### Quiz - manually mark/grade essay (or other) questions

This entry relates to the Quiz activity.

Your quiz may have a range of question types, including essay and/or short answer. The Essay question is the only one designed for manual grading.

#### Manually grade an essay question

The Manual grading report enables markers to manually grade written submissions within a quiz. This grading method is automatic for the Essay question type (irrespective of what the quiz is set to).

1. In the topic, click on the Quiz activity
2. Click on the Actions menu cog in the top-right corner, then select Manual grading

4. Use the Options section to choose which questions to mark and the order in which they appear

5. Underneath the options, you will see a list of student responses to be marked.
The student's response appears in the blue box. Include your comments and a mark in the green box

6. Click Save and go to next page when you have finished marking all those displayed. When you have marked all of the questions, you will be prompted to return to the Questions that need grading screen, with the responses that you marked now appearing in the Already graded column.

If you want to manually grade questions other than the essay question (eg the Short answer question type), you will have to override the automatic mark for that question.

1. In the topic, click on the Quiz activity

2. Click on the Actions menu cog in the top right corner, then select Manual grading

3. Click on Also show questions that have been graded automatically (above the table of questions and grades)

5. Use the Options section to choose which questions to mark and the order in which they appear

6. Underneath the Options, you will see a list of student responses to be marked. The student's response appears in the blue box. Include your comments and a mark in the green box. Save the changes.

#### Quiz - provide quiz feedback at quiz and question level

This entry relates to the Quiz activity.

Providing feedback is an excellent way to increase student engagement when they attempt and review a quiz. Feedback can be customised to address particular student responses to a question, the question more generally, or the total score achieved by the student. It can be delivered during the quiz, after the attempt but before the quiz has closed, or after the quiz has closed. The amount and quality of feedback you provide will depend on the quiz's purpose.

• Inclusion and release of feedback can increase the complexity required in quiz set-up. For help, please contact your local eLearning support team.

• Release of feedback to students is controlled by the quiz Review settings in conjunction with the Question Behavior setting. Please see 'I can't unhide the Grade column for a quiz' (Quiz - troubleshooting entry) for further information.

#### Good practice guides and tip sheets

Good practice guides and tip sheets have been developed to support quality in both curriculum design and teaching practice. Good practice guides provide a pedagogical overview and tip sheets provide you with practical strategies and ideas for implementation. Links to quiz-related resources are provided below.

#### The three types of feedback – specific, general and overall

• Specific feedback is feedback specific to, and dependent upon, the response the student gave to a question. It is response-specific.
Specific feedback is set up at the question level – when creating the question or later by clicking the edit cog   beside the question

Student view:

• General feedback is feedback shown to the student after they have completed the question. The same feedback is shown for all students, regardless of the response they gave. You can use the general feedback to give students a fully worked answer and perhaps a link to more information they can use if they did not understand the question.
General Feedback is set up at the question level – when creating the question or later by clicking the edit cog     beside the question

Student view:

• Overall feedback is the text that is shown after a quiz has been attempted. By specifying additional grade boundaries (as a percentage or as a number), the text shown can depend on the grade obtained
Overall Feedback is set up in the quiz's settings
In the scenario below, students scoring between 100% and 80% will see the message 'Well done'.  Those who scored between 79.99% and 0% will see            the message 'Please study this week's work again' (100% and 0% do not need to be entered, as they are the default grade boundaries):

Student view:

#### Quiz - purpose and settings

This entry relates to the Quiz activity.

These quiz setting guides focus on ways you might want to use the Quiz activity. Where relevant, they link to related quiz entries for further details.

#### Good practice guides and tip sheets

Good practice guides and tip sheets have been developed to support quality in both curriculum design and teaching practice. Good practice guides provide a pedagogical overview and tip sheets provide you with practical strategies and ideas for implementation. Links to quiz-related resources are provided below.

Where there is no right or wrong answer

The quiz is not assessed. You and your students could use it as a learning tool to assess their knowledge and/or skills, or identify gaps. Students could also 'self-assess' and/or determine their self-efficacy or self-confidence. Feedback on their question answers could personalise support through linking to resources University-wide or providing resources specific to the topic. View the quiz example set up for this purpose.

##### Quiz settings

• Attempts allowed: Unlimited/1 (depending on purpose)
• How questions behave: Deferred feedback
• Review options: The attempt

##### Edit quiz

Change the maximum grade in the quiz itself to 0 (default is 10). The quiz will function as normal, but the quiz will not appear as a grade item in gradebook.

• Maximum grade: 0.00 (‘Editing quiz’ screen)

##### Question settings

• Each choice option (MCQ) worth 100%
• Question is worth 0 (default mark)

Topic Management > Topic Settings > Gradebook setup

Check the gradebook. To make a quiz item worth 0 in the gradebook, change the weighted mean of grades to 0 for that item (Gradebook setup > Setup > Weights), if required.

Where there are right and wrong answers

The purpose of the quiz is other than to grade. You might use it as a learning tool to give students a way to test their knowledge as the weeks progress. Providing feedback that they can action or understand at a deeper level will be helpful. It is also a teaching tool to see how students are going, whether there is a gap in understanding for one/many, whether a deeper level of engagement is required (e.g. more resources, more class time).

##### Quiz settings

• Grade to pass: X.00 (choose any number between 0 and 10) – if using activity completion and/or restrict access, otherwise leave blank
• Attempts allowed: Unlimited/1... (depending on purpose)
• How questions behave: Deferred feedback; interactive with multiple tries...
• Review options: The attempt; Whether correct, Specific feedback, General feedback, Overall feedback (suggestions only)

##### Edit quiz

Maximum grade: Change the maximum grade for the quiz to the total of all questions (e.g. for 4 questions worth 1, 2, 2 and 4 the maximum grade would be 9). FLO will calculate a final grade according to the weighting for the quiz in the gradebook. Both student and teacher will be able to see what they got for the quiz (x out of x).

##### Question settings

• Question is worth 1 (default mark) or whatever you think it is worth
• MCQ: With 1 correct answer (one answer only option), this answer must be worth 100%; with multiple correct answers (multiple answers allowed option), they must add up to 100% (e.g. if there are 2 correct answers, each would be worth 50%). See Create a multiple choice question (MCQ) for more details.

Topic Management > Topic Settings > Gradebook setup

Check the gradebook. To make a quiz item worth 0 in the gradebook, change the weighted mean of grades to 0 for that item (Gradebook setup > Setup > Weights).

Where students are given the opportunity to extend their knowledge/understanding through feedback and prompts

The purpose of the quiz may be to grade/not grade. If you do not intend to grade the quiz, refer to the settings for non-graded quizzes above.

Interactive with multiple tries

You can use the 'Interactive with multiple tries' settings to give students a rich experience with learning pathways – they can choose to extend their learning or correct their learning in an informed way, through feedback that provides links to resources or embeds resources such as videos or images. Hints are provided with this setting to enable students to try again (if more than one try is enabled). Rich feedback deters students from guessing. If grading this quiz, you can set a penalty for requiring a hint, if set to multiple tries.

##### Quiz settings

• Attempts allowed: Unlimited (if you want to restrict the number of attempts, the number of hints is one less than the number of attempts)
• How questions behave: Interactive with multiple tries
• Review options: During the attempt; Immediately after the attempt; Later, while the quiz is still open

##### User experience

If the quiz setting for Question behaviour > How questions behave is 'Interactive with multiple tries', the student can see a Check button once they have answered a question. They can reattempt the question by clicking 'Try again' and selecting another answer. You can provide both Specific feedback (answer choice) and General feedback (on the question as a whole) to help the student.

When the student finishes the quiz, they get a prompt to 'Return to attempt' or 'Submit all and finish'.

When the student finishes the quiz, they can also click on a 'Review' link (depending on the boxes ticked under Quiz settings > Review options) and see their answers to one/more attempts. They then get the option to 'Submit all and finish'.

Once they 'Submit all and finish', they receive the Overall feedback (quiz settings) if any is provided.

They can go back in later and review their quiz responses (Quiz settings > Review options). This is handy if they want to remember something about their responses but do not want to do the quiz again (they may not have this option anyway).

Where students are tested on their knowledge/understanding for grading purposes

A summative quiz is likely to test a body of work so could be set to occur near the end of the topic, or at the end of a module. It will be listed in the Statement of Assessment Methods as a grade item valued at a percentage of the final grade. Because it is testing the student, they have less options when they do the quiz. They may receive feedback for each question, but are unlikely to be able to do the question again. Overall feedback for the attempt is a key area to support the student's learning.

Deferred feedback

With this setting, students must enter an answer to each question and then submit the entire quiz, before anything is graded or they get any feedback. While all review options are editable, you would not choose the Review option 'Later, while the quiz is still open' for a graded quiz.

##### Quiz settings

• Attempts allowed: 1
• How questions behave: Deferred feedback
• Review options: After the quiz is closed (all options ticked by default). If there is a possibility that students may request extensions/overrides, you can leave all the review options turned off until you are confident all students have attempted the quiz. You may decide to only show the Marks once the quiz is closed.

In the quiz settings, you can decide on how much Overall feedback to give students at the end of the quiz. You could use grade boundaries to tailor feedback.

##### User experience

If the quiz setting for Question behaviour > How questions behave is Deferred feedback, this is what happens when the student attempts a question. When they attempt a question, they may get feedback for their response, depending on how you have set up the individual question, but will move through the quiz without reattempting a question.

When the student finishes the quiz, they get the prompt to 'Submit all and finish'.

Once students click 'Submit all and finish', they receive the Overall feedback (quiz settings).

Students can go back in later and review their quiz responses (Quiz settings > Review options). This is handy if they want to learn from their responses, perhaps for a final assessment item.

Where students are manually graded on their submitted text answers (< 300 words is a guide)

To set up this type of quiz (it may only be one question that requires manual grading), you need to create an essay question.

#### Quiz - troubleshooting

This entry relates to the Quiz activity.

#### I can't see questions from other quizzes

If you create a quiz, then add 'a new question' rather than add 'from question bank' (Actions menu > Edit quiz screen), you may stumble into this problem. If you leave the Category as 'Default for [topic name]' or choose another category that you have already created, there won't be a problem. But if you choose the category ‘Default for [quiz name]’ from the pull-down menu, the questions will belong to the quiz and you won't be able to see or use them for any another quiz in the topic (e.g. in a revision quiz, or an end-of-topic exam that draws from questions used in formative quizzes throughout the topic).

##### Solution to prevent this problem

Do one of the following:

• Option 1: Go to the Question bank from your topic homepage (Topic Management > Question bank). Create a category and questions first, before you create the quiz, then add questions into your quiz 'from question bank'

• Option 2: Create a category, then create the quiz and 'add a new question', making sure the new question is saved in the category you created

Note: We strongly advise option 1 as the best approach to creating quizzes. See Add a category to the question bank for more information.

##### Solution if you have this problem

If you already have this problem and need to use your questions in other quizzes:

2. Create a new category (probably best to make the parent category 'top')
3. Go to your existing quiz, then Actions menu > Question bank
4. Select all questions which are in the quiz's default category, then using the 'With selected' menu at the bottom of the page, move to your new category

Note: Questions can be moved to different categories even if they are already used in quizzes, though be careful if they are used as random questions in a quiz.

#### I can't unhide the grade column in gradebook for a quiz

Quiz results visibility settings are controlled within the settings of the quiz activity, not in the gradebook.

Check the Marks setting for Later, while the quiz is still open and/or After the quiz is closedPlease see the entry Create a quiz and scroll down to Review options for further information.

#### I can't add/delete questions in my quiz

Once a quiz has been attempted, no additional questions may be added to that quiz, and no questions can be deleted (removed).

The following message will display on the Editing quiz screen (Actions menu > Edit quiz).

You can, however, delete questions if no attempts have been made. To delete multiple questions (Actions menu > Edit quiz):

1. Click the Select multiple items button
2. Click the Select all button or select individual questions
3. Click the Delete selected button

4. You will be asked to confirm that you want to removed the selected questions, so check these are the questions you want deleted
5. Click the Yes button

#### I have assigned answers to a quiz question incorrectly

If, before a quiz opens, you discover that a question requires editing or correction, it can be easily edited. Enter the Question bank and locate the relevant question. Select Edit question from the Edit drop-down menu for the question. After making the required changes, click Save changes.

If you discover the error after student access has commenced, it is possible to regrade your quiz. To regrade a quiz attempt you will first need to edit the quiz question that needs changing. When a quiz question is changed, all instances of that question will be altered (e.g. if the question is used in another quiz, it will also be altered). Students will not be notified of the changes made. Refer to the Edit quiz questions (and regrade quiz) entry.

#### I accidentally deleted a quiz and want to restore it

##### What you should do

If you accidentally delete a quiz and want to restore it, you can either:

• recreate the quiz – deleting the quiz activity does not delete the questions from the question bank (the original questions are available to use)
• contact your local eLearning support team to discuss if it's possible to restore the quiz from a previous availability.
##### What you should not do

You may be aware that you can restore deleted activities from the recycle bin, however, this is not recommended for quizzes. Restoring a quiz from the recycle bin will duplicate the question bank (make a copy of each question). If quizzes in your topic use random questions, students may receive the same question more than once. If you've restored a quiz from the recycle bin you will need to delete the duplicate questions before any quiz with random questions opens. Duplicate questions have '(copy)' appended to the question name.

If you need help, contact your local eLearning support team.

#### Quiz - view results and reports

This entry relates to the Quiz activity.

The following quiz reports are available in FLO. Generating and interpreting these reports can help you refine your quiz and question bank for the next cohort/overtime.

• Grades report: Shows all the students' quiz attempts, the overall grade, and the grade for each question. There are links to review all the details of a student's attempt, just as the student would see it (student view)

Useful for
tracking student progress, identifying students at risk, identifying problem questions (either difficult or badly designed) or easy/difficult questions (by time taken)
• Responses report: Similar to the Grades report, except that it shows the responses the students gave, rather than the marks they earned. It is also possible to show the question text or the right answer, to compare with the student's response. This is helpful when the question is randomised

Useful for identifying student misunderstandings (individual and common), gaps in their knowledge
• Statistics report: Gives a statistical (psychometric) analysis of the quiz, and the questions within it. The front page gives a summary of the whole quiz. Links drill down to a detailed analysis of specific questions

Useful for analysing individual questions and question behaviours (e.g. random guess score) but you will need to understand statistical terminology
• Manual grading reportAllows the marker to manually grade questions in the quiz

Useful for online marking of essay questions (more a marking facility than a report) – these can't be automated like other formats for questions (eg multiple choice). You can also mark short answer questions if required

#### Access the Quiz reports

1. In the topic, click on the Quiz activity

2. Click the Actions menu (cog icon) in the top-right corner of the page
3. Under Results choose from the four report options: Grades | Responses  |Statistics | Manual grading

The Grades report tells you about student quiz attempts, displaying the overall grade, and a summary of each correct/incorrect answer. It also reports on the State (student progress through a quiz), the time the quiz was started/completed and the total time the student took to complete it. You can regrade quiz attempts by selecting Regrade selected attempts. A bar graph summarises the grade range and frequency.

What to include in the report settings

 Setting Description Attempts from Select from:enrolled users who have attempted the quiz (default)enrolled users who have not attempted the quizenrolled users who have, or have not, attempted the quizall users who have attempted the quiz Attempts that are Tick the checkboxes as required to include in the report attempts that are:In progressOverdueFinishedNever submittedIf the quiz has been set up to allow multiple attempts, you can choose to show at most one finished attempt per user (tick the checkbox). Show only attempts Tick the checkbox to show only attempts that have been regraded / are marked as needing regrading.

Display options settings

 Setting Description Page size Specify the number to user attempts to display per page Marks for each question Yes – show marks for each question in the reportNo – hide marks for each question in the report Show average marks Enables the display of question average marks in the last row of the report Graph marks distribution Enables the display of the marks distribution graph at the end of the report

After selecting the required options, click Show report

1. You can download the results in a variety of formats (Comma-separated values text file, Excel spreadsheet etc)

2. Checkboxes allow you to individually regrade or delete the selected attempts. Before regrading attempts, you would edit the quiz question(s) first.

3. Each question's marks is hyperlinked to allow you to view the student's response

4. To review responses made by a particular student, click Review attempt under that student's name in the table.

#### Responses report

The Responses report is visually similar to the Grades report, but it shows the responses students gave to quiz questions, not the marks they earned. You can view the question text/correct answer to compare students' responses.

1. Click the Quiz link > Actions menu (cog icon) > Results > Responses

2. Use the setting form (What to include in the report) at the top of the page to control what is displayed (enrolled users who have attempted the quiz; enrolled users who have not attempted the quiz etc)

3. Click Show report

4. You can download the results in a variety of formats (Comma-separated values text file, Excel spreadsheet etc)

5. Checkboxes allow you to delete the selected attempts

#### Statistics report

The Statistics report provides a statistical ('psychometric') analysis of the quiz and the questions. The report has three sections:

• a summary of the whole quiz
• an analysis showing all questions in table format
• a bar graph of the percentage of correct answers (the 'Facility index') and the 'Discriminative efficiency'.

To get the report:

1. Click the Quiz link> Actions menu (cog icon) > Results > Statistics

2. In the Preferences just for this page, choose whether you want to calculate statistics from the highest graded attempt, all attempts, first attempt or last attempt.

3. Click Show report

4. You can download the full report in a variety of formats (Comma-separated values (.csv) file, Microsoft Excel (.xlsx) spreadsheet etc)

6. The Quiz structure analysis section lists all the questions in the quiz with various statistics in a table format. For more information, please refer to https://docs.moodle.org/39/en/Quiz_statistics_report#Quiz_structure_analysis.

Click on a question in the Question name column. In the Preferences just for this page, choose whether you want to calculate statistics from the first attempt or all attempts, and click Show report to go to a detailed report for that question, as shown below.

7. The last section of this report is a bar graph of the percentage of correct answers (Facility index) and the Discriminative efficiency.

Facility index (F): The mean score of students on the question (see table below for interpretation)

Discrimination efficiency: This statistic attempts to estimate how good the discrimination index is relative to the difficulty of the question.

 F Interpretation 5 or less Extremely difficult or something wrong with the question. 6-10 Very difficult. 11-20 Difficult. 21-34 Moderately difficult. 35-65 About right for the average student. 66-80 Fairly easy. 81-89 Easy. 90-94 Very easy. 95-100 Extremely easy.

#### Good practice guides and tip sheets

Good practice guides and tip sheets have been developed to support quality in both curriculum design and teaching practice. Good practice guides provide a pedagogical overview and tip sheets provide you with practical strategies and ideas for implementation. Links to assignment-related resources are provided below.

Self and peer assessment are valuable learning experiences for students if scaffolded well. Discussion about the process (as well as rubrics or other guides) is helpful so that students are comfortable assessing their peers and supported to do so. These two types of assessment are examples of authentic learning – self-assessment helps students in their life post-study (particularly in the workplace, where reflection plays a strong role in the performance and professional development), and peer assessment helps students to critically but constructively examine the ideas and opinions of others (also workplace relevant). Peer review aligns with a critical review of academic and other educational resources.

#### FeedbackFruits Pilot

In Semester 1 2022 Flinders is piloting two tools from the FeedbackFruits suite:

• Group Member Evaluation
• Peer Review

Find out more about these tools and see if you would like to take part in the trial.

#### Self and Peer assessment activity in FLO

The self and peer assessment activity in FLO enables the collection, review and peer assessment of students' work. Students can submit any digital content (files), such as documents or spreadsheets, and can also type text directly into a field using the text editor.

Submissions are assessed using a multi-criteria assessment form defined by the teacher. Students are given the opportunity to assess one or more of their peers' submissions, and can also perform self-assessment. Submissions and reviewers may be anonymous if required.

Students obtain two grades in this activity – a grade for their submission and a grade for their assessment of their peers' submissions. Both grades are recorded in the gradebook.

#### How the self and peer assessment activity works

This short (18 mins) video provides an overview – workflow, setup, tips etc (Workshop module = Self and peer assessment activity):

Timeline block:
• The 'Submissions deadline' will show to students in the Timeline block, as 'deadline for submission'
• The 'deadline for assessment' will show to students in the Timeline block, as 'deadline for assessment'
• If a deadline date is not used but its corresponding opening date is used, the opening date will show instead.

#### Create a self and peer assessment activity

There are currently two ‘self and peer assessment’ activities available in FLO. We recommend that you use the Enhanced self and peer assessment activity as it has more features than the ‘Workshop (Self and peer assessment)’ activity. The former, developed at the University of New South Wales (UNSW), is a modified (enhanced) version of the latter.

Written documentation for the FLO Staff support is still in development. In the meantime, please refer to UNSW’s step-by-step instructions on creating a self and peer assessment activity (course home page = topic home page). The ‘Workshop (UNSW)’ activity is the Enhanced self and peer assessment activity in FLO:

#### Styles and layout - apply text styles

This entry relates to styles and layout, in particular the HTML editor.

For reasons of accessibility, usability, universal design and sustainability, you are advised to apply styles that are built into the HTML editor when working with text. Categories of style usage include headings, quotes and other text elements.

#### Font colours

The font colour range is limited making it easier to be consistent. These colours ensure the text is readable on the screen. To view this feature you will need to expand the HTML editor menu.

#### Paragraph styles

Using paragraph styles will ensure that your site looks 'clean' (ie no unnecessary formatting) and consistent. These tools save you time as you don't have to worry about formatting text yourself, and are accessible to users with a sight disability, as their screen reader will be able to make sense of the text. Just as you wouldn't want to read a book that had no chapter headings or subheadings, so web users like 'chunked' text that makes reading easier.

1. In your FLO activity/resource/module, place your cursor where you want the style (eg heading) to appear, and from the toolbar menu, select the Paragraph styles icon
2. Make a choice from the Paragraph styles menu

3. Click Save changes once you have finished editing the text

#### Styles

These styles can be used to highlight something important that you want students to see (eg guest lecturers, assessment items due, preparation for a workshop). Using an option in the Styles menu will make this information stand out in your FLO site.

Example:

Please prepare for our next workshop by reflecting on your experiences at placement.

1. In your FLO activity/resource/module, in the HTML editor toolbar click on the Show more buttons icon

2. Place your cursor where you want the style to appear, and from the toolbar menu select the Styles icon

3. Make a choice from the drop-down Styles menu

4. Click Save changes once you have finished editing the text