6.7. Test, Self-test, Questionnaire

The following section describes the course elements test, self-test, and questionnaire and gives further information on their purpose.

6.7.1. What is a test?

A test contains questions of any type plus relevant answers. Such a test is a course element of OLAT and based on the IMS-QTI standard (Instructional Management Systems-Question and Test Interoperability). Its results can be seen by tutors and/or course administrators; they can also be related to the corresponding student.

6.7.2. What is a self-test?

A self-test is one course element of OLAT offering different types of questions in IMS-QTI standard format (Instructional Management Systems-Question and Test Interoperability). In contrast to tests, the self-test`s results cannot be seen by tutors.

6.7.3. What is a questionnaire?

A questionnaire helps to collect statistic data regarding the teaching programme. Its questions will also be stored in IMS-QTI standard format (Instructional Management Systems-Question and Test Interoperability). The results of such a questionnaire will be saved anonymously.

6.7.4. Which format do I use for tests, self-tests, and questionnaires?

Tests, self-tests, and questionnaires can be created by means of the OLAT test and questionnaire editor in the IMS-QTI format explained below.

6.7.4.1. IMS-QTI

The IMS project (Instructional Management Systems Project) is a project of the National Learning Infrastructure Initiative by EDUCAUSE. It specifies different standards regarding the exchange of learning contents. The Question and Test Interoperability Specification (QTI) describes a unified format for tests and questionnaires in order to make them exchangeable for various teaching platform. At the time being, OLAT supports IMS-QTI version 1.2.

(Content Packaging--used for the making of learning contents-- (Why do I have to format my learning content as IMS Content Package (CP)?) is also a format defined by that IMS project.)

6.7.4.2. Where do I get information on IMS-QTI?

The IMS-QTI specification is described here.

6.7.4.3. How do I profit by these standardizing efforts?

Only after creating and saving teaching or test objects in standard format can those objects be used as elements in different OLAT courses or other Learning Management Systems.

6.7.5. How do I create a test, self-test or questionnaire?

Before offering a test in an OLAT course, that test has to be created as an independent learning resource. After logging in in OLAT as author, click on "Learning resources". Now you can create tests and questionnaires by means of the corresponding editor (action boxes on your right; choose "Add" and "Create"). You can also add tests or questionnaires from your personal folder or from your hard disk to learning resources.

Tests and questionnaires can be created or edited by means of the OLAT editor. This editor automatically translates your documents to the XML format and creates ZIP files.

6.7.5.1. Can I use external test editors?

You can also create your tests by means of an external editor supporting the IMS-QTI standard format (version 1.2) before importing them to OLAT.

6.7.5.2. Creating Tests or Questionnaires with the Test-and-Questionnaire Editor

When creating a test or questionnaire or downloading a test or questionnaire already existing in OLAT you have to be in the section "Learning resources".

  • In order to create a test or questionnaire by means of the editor click on the link "Test" or "Questionnaire" within the action box "Create".

  • In order to choose a document already existing from your folder or hard disk and download it to the editor click on "Test" or "Questionnaire" within the action box "Add".

  • As soon as the editor mode is on you can choose from the action box "Add" various elements (section, single or multiple choice, Kprim, gaps in a text or additionally free text in questionnaires) in order to create your document.

  • The "Preview" shows how your document or question presents itself to the course participant.

Tests and questionnaires are built hierarchically according to IMS-QTI standard: a test consists of one or several sections; a section consists of one or several questions; a question consists of the question itself as well as one or several possibilities to answer (depending on the question).

6.7.5.3. How to Edit an Existing Test or Questionnaire

After creating a new test or questionnaire or after downloading an existing document to your learning resources (storage folder) you may want to edit that file. In order to do this you search for your file in learning resources via "My Entries" and click on that learning resource's name to get to the detailed view.

This detailed view allows you to modify OLAT meta data such as title and description via "Modify description." It is also possible to replace the learning resource's picture. In addition, you can allow others to copy or download your test or questionnaire. Of course, you can also determine who will or will not get access to your test or questionnaire.

In order to edit the content of your test or questionnaire you will have to start the test and questionnaire editor first. Just select "Edit" from the action box "Learning resource" on your right after switching to the detailed view.

Attention: If a test or questionnaire is already used you will only be able to correct typing errors. In this case OLAT will provide you with a corresponding message.

The OLAT test and questionnaire editor will help you to assemble the different elements of your test (or questionnaire). Possible elements are: section, single or multiple choice, Kprim and gap text. In order to add an element to your test (or questionnaire) your have to click on the corresponding element within the action box "Add".

Each test (or questionnaire) consists of one or several sections; these sections consist of one ore several questions. A question consists of the question itself as well as one or more possibilities to answer (depending on the question).

The sections assist you in organizing your test or questionnaire, e.g., to sort out the questions according to subject or level of difficulty. Each test or questionnaire has to contain at least one section.

In OLAT there are five different kinds of questions containing the question itself as well as one or several possible answers (depending on the question): single choice, multiple choice, Kprim, gap text, and free text (within a questionnaire).

  • Single Choice: a single choice question consists of a question and two answers at least. Only one of the answers is correct; only one of the possible answers can be selected.

  • Multiple Choice: there are at least two answers per question. More than one possibility can be selected; more than one answer may be correct.

  • Kprim: a Kprim question always consists of one question and exactly four possibilities to answer. The one answering has to decide four times if the answer is right or wrong.

  • Gap text: terms of a certain text are replaced by input fields. The one answering has to fill in the correct terms in those gaps.

  • Free text: only available within a questionnaire. A question consists of the question itself and a text field into which an individual answer can be inserted.

6.7.5.4. Create or Edit Questions

The test and questionnaire editor helps you to assemble your test as well as to create or edit single questions.

When creating a question you are able to either make general comments on the question (the so-called meta data) or insert the text of your questions and possibilities to answer.

The general comments on your questions may be:

  • title and description

  • the answers' alignment

  • how many attempts per question are allowed

  • how much time per question is acceptable

  • the answers' order

  • if there should be hints

  • if the correct answer should be displayed.

Additionally you can determine

  • if the students should be given some feedback and

  • if the question should be linked to a HTML page or CP learning content (question of comprehension).

The text of a question or answer visible to students can contain pictures and movies as well.

In addition, you can determine how the points and how many of them should be distributed.

6.7.6. What can my test, self-test, or questionnaire contain?

6.7.6.1. How to Insert Pictures, Movies, and Flash Files

Pictures within a test, self-test, and questionnaire: the test editor helps you to insert pictures into questions and answers. Just use the so-called file upload from your test editor. You can also use pictures within questions of comprehension. If you want to do this you will have to store those pictures separately together with the Java script generating your question of comprehension before linking it to a Content Packaging or a single page. Do not forget to adapt the pictures' path within the Java script. When giving feedback no pictures can be used. You may also edit your test or questionnaire without the editor. Pictures are to be stored in the folder "Media"; references are to be set in the file "qti.xml".

Flash files within a test, self-test, and questionnaire: questions containing Flash files may be part of tests, self-tests, and questionnaires.

6.7.6.2. How do I insert a Flash question in a test?

A Flash question cannot be edited within the test and questionnaire editor. Therefore, you have to insert your QTI file by means of a text editor on your hard disk.

Preparation of your test: If you have already stored an existing test in learning resources, please open this test with the test and questionnaire editor before adding a question. Do not fill in that question! Then save your test in learning resources. That question serves as placeholder and provides you with the necessary IDs for your Flash question. Now download your test to your hard disk. The option "Download possible" has to be on in the detailed view of your entry "Test" in learning resources. In case you do not have a test yet just follow the instructions in order to get a framework for it:

  1. Learning Resources -> Create a Test

  2. Insert Test's Title and Description; Enable Download

  3. Save Test -> Your test will be stored in the storage folder

In order to download that test activate the detailed view of the entry "Test" in learning resources. Its "Tools" will provide you with a link to download your test.

Un-zip the qti.zip file you have received. If your test contains pictures there will be a folder called "Media" for those picture files. If your test does not yet contain pictures nor Flashs there will not be a media folder. Therefore, you have to create a new "Media" folder accoring to the qti.xml file's hierarchy.

If a Flash file is correctly inserted in a qti.xml file the QTI Engine will be able to deal with results (that part that displays questions and organizes the test's progress). Ordinary Flash movies will be implemented in XML with the <matvideo> tag of the type "application/x-shockwave-flash". These Flash documents cannot communicate with the QTI Engine. If you implement your Flash file with a <matapplet> tag the QTI Engine of that file will allow a reply that can be processed by the QTI Engine. The <matapplet> tag has to be set within a <response_str> tag. Your Flash file will then be able to reply in form of a single string.

Example of a Flash question; excerpt of a qti.xml file

							
     <item ident="QTIEDIT:FLA:1063841914107" title="A Flash Test">
        <presentation>
          <material>
            <mattext><![CDATA[What is the answer to life, the universe and everything?]]></mattext>
          </material>
          <response_str ident="1063841914115" rcardinality="Single" rtiming="No">
            <render_extension>
                <material>
                <matapplet uri="media/sendparam.swf" width="300" height="200"></matapplet>
                </material>
            </render_extension>
          </response_str>
        </presentation>
        <resprocessing>
          <outcomes>
            <decvar varname="SCORE" vartype="Decimal" defaultval="0" minvalue="0.0" maxvalue="1.0" cutvalue="1.0"/>
          </outcomes>
          <respcondition title="Mastery" continue="Yes">
            <conditionvar>
              <varequal respident="1063841914115" case="Yes">42</varequal>
            </conditionvar>
            <setvar varname="SCORE" action="Set">1.0</setvar>
          </respcondition>
          <respcondition title="Fail" continue="Yes">
            <conditionvar>
              <not>
                <varequal respident="1063841914115" case="Yes">42</varequal>
              </not>
            </conditionvar>
            <setvar varname="SCORE" action="Set">-1</setvar>
          </respcondition>
        </resprocessing>
      </item>

						

Please note that the attribute of the "item indent" tag has to begin with "QTIEDIT:FLA:". This signals the QTI Engine not to display any forms. Nevertheless, your Flash should contain a kind of "Send" button.

The following part of a code shows how the Flash file responds to the QTI Engine.

							
----- Sample Code -----

var oResponseURL:String = _root.oResponseURL;
var oResponseParam:String = _root.oResponseParam;

sendbutton.onRelease = function(){
  loadVars = new LoadVars();
  loadVars[oResponseParam] = inputfield.text;
  loadVars.send(oResponseURL, "_self", "POST");
}
----- Sample Code -----

						

Copy the example from above to an empty text file (dokumentname.txt). Open the qti.xml file by means of a text editor as well as your text file containing the example mentioned above. The question generated without any data from the file qti.xml provides you with the necessary IDs for your Flash question. Use these IDs for the example in your Flash question. The Flash file's reference (.swf) has to be adapted as well. Please indicate the score for the correct or wrong answer. Then delete the question used as place holder in the qti.xml file and copy your Flash question (the modified example question) to exactly the same position. The Flash file (.swf) should then be stored in the folder called "Media". Your text document containing the example from above will not be needed anymore.

Zip the qti.xml document together with the folder "Media". The new archive should be named qti.zip. Then import that zipped test into OLAT's learning resources.

Tip for Mac users: in case you create zip files by means of StuffIt Deluxe these files will be stored in a new folder generated by StuffIt. Such qti.zip files cannot be imported into OLAT. OSX will be able to assist you in creating a zip archive using the terminal interface (command: zip -r qti.zip media qti.xml).

When reading in data from external text files in your Flash file you will have to adapt the original Flash file (.fla). Flashs in OLAT do not upload files according to the paths they have been uploaded. Flash always takes the browser's URL as starting point. Therefore, you have to put first the path OLAT provides (oStaticsURL) whenever your Flash file uses relative paths (making the path absolute and relative).

Example:

							
		Instead of calling

		loadVariables("media/foo.txt", 0);

		use

		loadVariables(_root.oStaticsURL + "media/foo.txt", 0);	

						

6.7.6.3. Movies in Test, Self-test, and Questionnaire

Movies in Test, Self-test, and Questionnaire: The test editor assists you in including movies in questions and answers. Just use the test editor's file upload. Movies can also be included in questions of comprehension. However, those movies have to be stored separately together with the Java script generating that question of comprehension. Your movies will then only be listed in the question's form. Therefore, you should embed them in HTML. Instead of [ VIDEO: media/01491.mov (video/quicktime) ], you put <embed type="video/quicktime" pluginspage="http://www.apple.com/quicktime/download/" width="300" height="200" src="media/dokumentname.mov" autoplay="true" controller="true"></embed>.

6.7.7. Will participants have advantages when doing a test and having access to its source at the same time?

No, tests and self-tests in OLAT are safe. There is no possibility to find out the correct answers when having access to their source. The students' answers are sent to the OLAT server before interpreting them.

The same is not true for questions of comprehension. These questions are directly embedded in your learning content in order to give your students the possibility to check their knowledge while reading your learning content.The interpretation of the students' answers takes place in the client.

6.7.7.1. Processing a Test, Self-test, and Questionnaire on the Server's Side

Test and self-test are programmed to process the interpretation of the students' answers on the server's side. The students' input is directly sent from the client to the OLAT server where it will be checked and analyzed. Depending on the test's configuration your students will be able to see that analysis together with their answers already given.

The students' test results will be personalized and stored in a data base. The results of a self-test and its answers will be stored anonymously.

6.7.7.2. Processing a Question of Comprehension on the Client's Side

Questions of comprehension should give your students the possibility to check their knowledge while reading learning contents. Their purpose is self-control and not assessment.

Questions of comprehension are directly linked to a learning content, e.g., a HTML page. Whenever a user reads through learning contents in a browser questions and their answers will also be uploaded. This means that the answers to questions of comprehension can be looked up in their source (the opposite is true for tests and self-tests).

You can create questions of comprehension by means of the test and questionnaire editor. It will automatically generate the relevant Java Script Code that should be copied from the editor to your learning content, e.g., your HTML page.

6.7.8. How do I set up a test?

In order to set up a test or self-test you first have to collect your questions and answers before determining how your test should be presented and assessed. Additionally you can choose if your students should get any further information, e.g., hints, correct answers or some feedback, how often a single question or the entire test may be done, and how many points can be achieved.

6.7.8.1. Solution

If you want to show your users the correct answers you will be able to collect them by means of the test and questionnaire editor. As soon as your users give the wrong answer and click "Save" the correct solution will be displayed.

6.7.8.2. Feedback

You can also give your users feedback to be displayed as soon as an answer has been saved. You decide by means of the test and questionnaire editor if the answer given should be marked right or wrong or if an individual feedback regarding the wrong answer should be displayed.

6.7.8.3. Hints

If you want to provide hints regarding the correct answers to your test you will be able to do so by means of the test and questionnaire editor. As soon as students have saved the wrong answer your hint will be displayed.

6.7.8.4. Time

The test and questionnaire editor helps you to limit the time needed for answering a question (or an entire section) as well as the period during which the test has to be taken. If a question, a section or the whole test has a time limit it is indicated by the hourglass icon next to the question's or section's title (beneath the test's title, respectively). If you start the test or select a question the total time as well as the time limit will be indicated.

The indication of the test's duration is static which means that it remains unchanged during the test and when the time is up. Only when saving the test's results the student will receive a message that the time limit has been exceeded.

6.7.8.5. Repeatability of Questions and Tests

Time can be limited as well as the number of possible attempts to solve a problem per question by means of the test and questionnaire editor.

You can also determine how often the whole test may be taken. You do this using the course editor after including the complete test to your course.

6.7.8.6. Allocation of Points Regarding Questions, Sections, Tests

In the test and questionnaire editor you determine how many points should be allocated per question and how many points are necessary in order to pass your test. This passing score only takes effect after including the test in your course.

Multiple choice questions and gap texts require preparatory work. You have to decide first if you want to allocate points when all answers are correct or if students score for every right answer. Then you determine that score.

6.7.8.7. What results can be shown when taking a test or self-test?

If you want to include a complete test in your course (course editor) you can indicate what kind of information your students will receive during and after taking that test. You decide if your students should see how many points they have gained after answering each question. Furthermore, you can choose between three alternatives to be shown after taking the test: no results, concise results or detailed feedback.

"No Results" in a self-test means that nothing will be displayed, neither in the self-test window nor in the relevant course element. At the time being, within the test window there will be no score indicated, however, after closing the test the student's score will be visible in the test element as well as the current status (passed/not passed). "Concise Results" means that the student can see his score compared to the highest possible score without a list of the asked questions. The detailed feedback lists all the questions and their correct answers as well as the solutions given by the student. The concise and the detailed results will be displayed immediately after taking the test or self-test.

The display of your feedback, hints or correct answers that can be configurated in the test editor does not depend on the display of the score.

6.7.8.8. Test Presentation and Navigation Possibilities within a Test

If you include your test or self-test in your course by means of a course editor you can choose between different presentation options. You can or cannot allow the navigation via the menu. If you forbid navigation you can all the same have a display of the menu or the number of questions. If you do not want to give your students further information on your test's or self-test's content you will be able to do so. In this case the title of each question can be indicated nevertheless; it will appear in the question window and not in the menu.

In addition, you decide if your test can be interrupted or terminated. In case of a termination no data will be saved. In case of an interruption the current state will be saved and transferred when starting the test anew. Attention: If a test with time limit is interrupted you cannot regain lost time. In case of a longer interruption it may well be that the remaining questions cannot be answered anymore.

6.7.9. What kind of questions can I cover with the OLAT test editor?

In OLAT there are five different types of questions consisting of the question's text and one ore more possible answers (depending on the type of question):

  • Single Choice

  • Multiple Choice

  • Kprim

  • Gap Text

  • Free Text (Questionnaire)

In order to add questions to your test or questionnaire you have to click on the relevant type of question in the action box "Add".

6.7.9.1. Single Choice

A single choice question consists of a question and at least two answers. Only one of them is correct; only one possible answer can be selected.

The questions as well as the answers can contain text, pictures and movies. They can be combined in any order.

6.7.9.2. Multiple Choice

There are at least two answers per question. Several possibilities can be selected; more than one answer can be correct.

The questions as well as the answers can contain text, pictures, and movies. They can be combined in any order.

6.7.9.3. Kprim

A Kprim question always consists of one question and exactly four possibilities to answer. You have to decide for each of those four answers if it is right or wrong. Three correct answers gain half of the score; four correct answers gain the highest possible score. This type of question is suited for questions that require different elements in order to be solved.

The questions as well as the answers can contain text, pictures, and movies. They can be combined in any order.

6.7.9.4. Gap Text

Terms of a certain text are replaced by empty input fields. The one answering has to fill in the correct terms in those gaps.

Your text can contain pictures and movies as well. They can be combined in any order. The terms to be filled in have to be set (synonyms separated by ;). At the same time you have to determine if there should be case sensitivity or not. Please note that OLAT will only accept answers as being correct the way you have indicated.

6.7.9.5. Free Text

This type of question is only available in a questionnaire. A text field as part of a question is displayed wherein an individual answer can be entered. Such a free text question has to be analyzed manually.

6.7.10. Is there an OLAT question pool?

A question pool is a collection of different questions centrally stored in learning resources. When creating a test of self-test you have access to this pool. This way, a question must only be entered once, however, it can afterwards be used as often as you wish.

At the time being, there is no possibility to record question pools in OLAT. In order to use questions more than one time you have to copy the entire test. In this case modifications must be made separately within the original test as well as within all of its copies.