|
|
| (3 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| There are several ways to administer a multiple-choice assessment online:
| | [https://markbox.uwaterloo.ca/ Markbox] is developed and maintained by UWaterloo’s [https://uwaterloo.ca/science-computing/ Science Computing] and used for the creation and grading of multiple-choice exams. It allows UWaterloo instructors to build multi-layout multiple choice exam sheets, or even build fully scrambled custom tests where each student gets a unique document. |
| * Paper-based options include Akindi, CrowdMark, and Markbox.
| |
| * Online options include LEARN and Mobius.
| |
|
| |
|
| == Differences between the paper-based solutions ==
| |
| You can find a comparison of the above listed paper-based solutions at the link below:
| |
| * [https://www.math.uwaterloo.ca/~pkates/LT3/mc.html'''Comparison by P. Kates''']
| |
|
| |
|
| == Features of Markbox ==
| |
| === Classic Bubble Sheet ===
| |
| The Classic Bubble Sheet is a scannable coversheet that can be simply handed out or attached to the test.
| |
|
| |
|
| To configure, complete the following Actions by clicking them sequentially:
| | '''Centrally Supported:''' No |
|
| |
|
| '''1. Select Page Size''' | | '''Possible Alternatives:''' [[Akindi]], [[Crowdmark]]; [[LEARN]] and [[Mobius]] for online only. |
| # Choose the correct paper size to fit all questions on one page.
| |
| #* Letter (8.5 x 11inch): maximum of 180 questions
| |
| #* Legal (8.5x14inch): maximum of 264 questions
| |
| :: To ''only'' download the coversheet, click '''Download Coversheet''' after selecting a page size.
| |
|
| |
|
| '''2. Upload Student List''' | | '''Instructor Contact:''' Your [https://uwaterloo.ca/centre-for-teaching-excellence/profile-type/faculty-liaisons CTE Liaison] for pedagogical support. |
| # Upload a '''.csv''' of the classlist by clicking '''Choose File'''.
| |
| # If there are headings in the classlist table, click the checkbox next to "Skip the first row (remove headers)".
| |
| # Click '''Upload Students'''
| |
|
| |
|
| '''3. Answer Key and Test Versions''' | | '''University Support:''' [mailto:mvucicevich@uwaterloo.ca Mirko Vucicevich] |
| # Type your answer into the text field or click the correct option on the bubble sheet.
| |
| #* If an incorrect answer is inputted, re-type the correct choice in the text field or click on the correct choice on the bubble sheet.
| |
| # To create multiple versions of the same test, click '''Add Version'''. Up to 10 different versions of the test (A, B, C, D, E, F, G, H, I, and J) are available.
| |
| # To delete versions of the test, click '''Delete Version'''.
| |
| # To adjust the value of each question, click '''Edit Weighing'''. Then, type in the desired value within the text field next to it or by clicking the up / down arrows.
| |
| #* NOTE: Questions must have a minimum value of 0. The weight of test questions can also be adjusted later during the marking process.
| |
|
| |
|
| Once you are done with the configurations, click '''Save Versions'''.
| | '''Vendor Support:''' n/a |
|
| |
|
| Click '''Download Coversheet''' to download a PDF file of the coversheet.
| | '''Accessibility:''' Information not currently available. Please reach out to [mailto:mvucicevich@uwaterloo.ca Mirko Vucicevich] with any accessibility concerns in the meantime. |
| | |
| Once completed coversheets are collected from students, begin [https://wiki.math.uwaterloo.ca/DAG/index.php?title=Markbox#Marking_Your_Test scanning and marking them].
| |
| ----
| |
| | |
| === Peer-Marked Coversheet ===
| |
| The Peer-Marked Coversheet is a scannable coversheet intended for in-class peer-marking exercises.
| |
| | |
| To return results to the students via email, the marker's name and ID will be removed from the page.
| |
| | |
| To configure, complete the following Actions by clicking them sequentially:
| |
| | |
| '''1. Configure Coversheet'''
| |
| | |
| # Choose the correct number of columns to fit all questions on one page.
| |
| #* '''One Column''': maximum of 10 questions over one column, up to 15 points each.
| |
| #* '''Two Columns''': maximum of 20 questions over two columns, up to 5 points each.
| |
| # Choose the paper size by clicking on the drop down menu next to "Paper Size".
| |
| #* Letter (8.5x11inch)
| |
| #* Legal (8.5x14inch)
| |
| # Within the text field next to "Number of Questions", enter a number at or below the question limit.
| |
| # To set a default maximum score for all questions, enter a value between 0-15 (One Column) or 0-5 (Two Columns) into the text field next to "Quick-set Max Scores" or by clicking the up / down arrows.
| |
| | |
| * To add more questions, click '''Add Question'''.
| |
| * To delete questions, click the '''X''' under the "Delete" column.
| |
| | |
| Once you are done with the configurations, click '''Save Config''' to save.
| |
| | |
| '''2. Upload Student List'''
| |
| | |
| For Peer-Marked Coversheets, it is recommended to add a "unknown" student to the classlist to handle cases where the marker fails to input their information, or a student not in the classlist acts as a marker.
| |
| | |
| # Upload a '''.csv''' of the classlist by clicking '''Choose File'''.
| |
| # If there are headings in the classlist table, click the checkbox next to "Skip the first row (remove headers)".
| |
| # Click '''Upload Students'''.
| |
| | |
| '''3. Download Coversheet'''
| |
| # Click '''Download Coversheet''' from the "Actions" list to download a PDF file.
| |
| | |
| Once completed coversheets are collected from students, begin [https://wiki.math.uwaterloo.ca/DAG/index.php?title=Markbox#Marking_Your_Test scanning and marking them].
| |
| ----
| |
| | |
| === TestCreator / Scrambled Multiple Choice Test ===
| |
| | |
| The TestCreator creates a hand-crafted test which supports a variety of scrambling and feedback options. To create a test with the TestCreator, please review the [https://uwaterloo.ca/science-computing/markbox1-user-guide MarkBox 1.0 user guide].
| |
| | |
| To create a Scrambled Multiple Choice Test, the TestCreator tool must be used to generate a '''.test''' file from MarkBox 1.0.
| |
| | |
| To configure, complete the following Actions by clicking on them sequentially:
| |
| | |
| '''1. Upload student list'''
| |
| # Upload a '''.csv''' of the classlist by clicking '''Choose File'''.
| |
| # If there are headings in the classlist table, click the checkbox next to "Skip the first row (remove headers)".
| |
| # Click '''Upload Students'''.
| |
| | |
| '''2. Configure Questions and Answers'''
| |
| # To input the questions and answers, upload the '''.test''' file created in the TestCreator.
| |
| # To download the PDF file, click '''Download PDF''' to the right of "Test Preview". ''Only one'' copy of the test can be downloaded.
| |
| # Once satisfied with the configuration, click '''Confirm'''.
| |
| | |
| '''3. Printing Options'''
| |
| # Choose the printing options (download type, bubble sheet location, double sided, and number of tests).
| |
| | |
| '''4. Generate Sheets / Download Test'''
| |
| # Once satisfied, click '''Generate Sheets''' from the Action list. It will download the printable test file based on the '''Download type''' setting.
| |
| * Once you see the progress bar, the test will generate as a background process, so it is okay to close the browser.
| |
| | |
| Once completed coversheets are collected from students, begin [https://wiki.math.uwaterloo.ca/DAG/index.php?title=Markbox#Marking_Your_Test scanning and marking them].
| |
| ----
| |
| | |
| === TA-Marked Coversheet ===
| |
| The TA-Marked Coversheet is a scannable coversheet for use-cases where you'd like graders to fill out the bubbles.
| |
| | |
| Ex) Generate this cover sheets for a series of short-answer questions. Attach them to the front of the test / quiz and have your students only fill out their name and student ID.
| |
| | |
| To configure, complete the following Actions by clicking on them sequentially:
| |
| | |
| '''1. Configure Coversheet'''
| |
| | |
| # Choose the number of columns over which the test questions will be listed.
| |
| #* '''Two Columns''': maximum of 60 questions over two columns, up to 15 points each.
| |
| #* '''Three Columns''': maximum of 90 questions over three columns, up to 10 points each
| |
| # Choose the paper size by clicking the drop down menu next to "Paper Size".
| |
| #* Letter (8.5x11inch)
| |
| #* Legal (8.5x14inch)
| |
| # Within the text field next to "Number of Questions", enter a number up to the question limit for the chosen column settings.
| |
| # To set a default maximum score for all the questions, enter a value between 0-15 (Two Columns) or 0-10 (Three Columns) depending on the number of columns into the text field next to "Quick-set Max Scores" or by clicking the up / down arrows.
| |
| | |
| * To add more questions, click '''Add Question'''.
| |
| * To delete questions, click the '''X''' under the "Delete" column.
| |
| Once you are done with the configurations, click '''Save Config''' in the bottom right corner to save.
| |
| | |
| '''2. Upload Student List'''
| |
| # Upload a '''.csv''' of the classlist by clicking '''Choose File'''.
| |
| # If there are headings in the classlist table, click the checkbox next to "Skip the first row (remove headers)".
| |
| # Click '''Upload Students'''.
| |
| | |
| '''3. Download Coversheet'''
| |
| # Click '''Download Coversheet''' from the "Actions" list to download a PDF file.
| |
| Once completed coversheets are collected from students, begin [https://wiki.math.uwaterloo.ca/DAG/index.php?title=Markbox#Marking_Your_Test scanning and marking them].
| |
| | |
| == Using Markbox ==
| |
| Before getting started, you must use '''Google Chrome''' as your browser which allows cookies.
| |
| | |
| Science Computing has a user guide for Markbox 1.0: [https://uwaterloo.ca/science-computing/markbox1-user-guide MarkBox 1.0 user guide]
| |
| | |
| Login to MarkBox: https://markbox.uwaterloo.ca
| |
| | |
| === Creating New MarkSet ===
| |
| 1. Click '''Create New MarkSet''' in the center of the page, the left navigation bar, or next to "All MarkSets".
| |
| | |
| 2. Type in the title of the MarkSet under "MarkSet Title:". It is only used for self-reference purpose. It is generally a good idea to include the current term and course number (e.g. Chem 120 Final Fall 2020).
| |
| | |
| 3. Select one of the four types that will best suit the desired test:
| |
| # [https://wiki.math.uwaterloo.ca/DAG/index.php?title=Markbox#Classic_Bubble_Sheet '''Classic Bubble Sheet''']
| |
| # [https://wiki.math.uwaterloo.ca/DAG/index.php?title=Markbox#Peer-Marked_Coversheet '''Peer-Marked Coversheet''']
| |
| # [https://wiki.math.uwaterloo.ca/DAG/index.php?title=Markbox#TestCreator.2F_Scrambled_Multiple_Choice_Test '''Scrambled Multiple Choice Test''']
| |
| # [https://wiki.math.uwaterloo.ca/DAG/index.php?title=Markbox#TA-Marked_Coversheet '''TA Marked Coversheet''']
| |
| ----
| |
| | |
| === Deleting a MarkSet ===
| |
| 1. Open the MarkSet that needs to be deleted.
| |
| | |
| 2. Click '''Delete Markset''' on the left navigation bar.
| |
| | |
| '''Deleting a MarkSet will permanently delete all data associated with it''', which includes:
| |
| * Uploaded Scans
| |
| * Processed Images
| |
| * Error Correction
| |
| * Final Grades
| |
| * Annotated PDFs
| |
| ----
| |
| === Marking Your Test ===
| |
| After the test has been completed and collected, scan the completed pages. Ideally, test scans should be compiled as a single PDF file for uploading.
| |
| | |
| '''Only the bubblesheets need to be scanned for MarkBox to mark them.'''
| |
| | |
| To scan full tests, ensure that they are in order.
| |
| | |
| To mark tests, complete the following Actions by clicking on them sequentially.
| |
| | |
| * At any time during the marking process, it is possible to '''Adjust Scoring''' of the test questions and '''Re-Download Test'''.
| |
| | |
| * To '''View / Mark Bad Tests and Scans''', the scanned tests must first be uploaded and correlated.
| |
| | |
| '''1. Upload Scans'''
| |
| # Select whether '''Only Bubble Sheets''' or a '''Full Scan''' of the test will be uploaded.
| |
| #* It is generally recommended to ''only'' scan bubble sheets unless you want to return the full scanned test back to each student.
| |
| # Click '''Upload'''.
| |
| #* After uploading the test scans, the MarkBox automatically matches students from the student list to their test scan.
| |
| # If you scanned your entire test, select the '''Full Test''' option and Markbox will attempt to '''group''' each page into a test. Please ensure that your test is scanned in the correct order.
| |
| | |
| '''2. Correlate Scans'''
| |
| # Check "Un-correlated Students" table to see if students incorrectly marked their student ID, or if a student who is not in the class submitted the test.
| |
| # To match these students or dismiss any submissions, click '''Correlate Scans'''.
| |
| # An image of the hand-written student number will be displayed and can be typed into the field under "Student:" to find and correlate them. You can also match their test version.
| |
| # To remove a student not in the class, confirm and click '''Student not in class list'''.
| |
| # To remove a test submission from being marked, click '''Remove test from automated marking'''.
| |
| # Click '''Done Correlating'''.
| |
| * NOTE: All tests should be correlated or removed from manual marking at this stage.
| |
| | |
| '''3. Correct Errors'''
| |
| * MarkBox will process and highlight with a blue rectangle if it finds errors. These errors ''must'' be manually corrected to ensure that grades are distributed accurately.
| |
| ** markings outside of the bubbles
| |
| ** multiple answers for the same question
| |
| ** blank answers
| |
| ** failure to properly erase answers
| |
| | |
| # Check the left navigation bar to see a list of test errors with the question number and type of error.
| |
| # If multiple selections are highlighted with the blue rectangle, click on selections that need to be removed. A circle will appear to the right of the question.
| |
| #* If a student clearly made a selection and it was not recognized by MarkBox, click on the surrounding rectangle to accept it as an answer.
| |
| #* If there is no error but MarkBox highlights it as a problem, click '''Ignore''' under that specific question or click the red circle next to the question number. If all questions are mis-marked as test errors, click '''Ignore All'''.
| |
| #** To unignore, simply click '''un-ignore''' or click on the circle next to the question number.
| |
| #* To view solutions while marking multiple versions, use the '''Manual Marking'''. Simply enter the test version of interest into the text field to display the answers.
| |
| #* If a test needs to be removed from grading, click '''Remove from Marking''' at the bottom right of the page.
| |
| # Click '''Save and go to next test''' to move on to correcting the next test.
| |
| | |
| '''4. View Test Stats'''
| |
| | |
| The statistics shown will be:
| |
| * '''Answer Key Type''' - The version of the test
| |
| * '''Number of Tests''' - Number of tests generated
| |
| * '''Maximum Score''' - Total number of available points
| |
| * '''Average Score''' - Average number of points scored
| |
| * '''Standard Deviation''' - The extent of deviation for the class as a whole
| |
| * '''Distribution of Grades (percent-based)''' - The percentile grade distribution that students achieved
| |
| * '''Student Scores & Selections''' - The individual selections that each student made in their test, presented as a downloadable '''.csv''' table.
| |
| * '''Question Analysis''' - Each question, its value, correct answer, PtB, percentage of correct answers (%C), number of students who chose each possible answer, marked errors, and blank responses, presented as a downloadable '''.csv''' table.
| |
| | |
| '''5. Download Gradesheet / Finalize Test'''
| |
| # Click '''Download Gradesheet''' to generate and download a '''.csv''' of results.
| |
| # After everything has been completed, click '''Finalize Test'''.
| |
| #* By finalizing the test, the ability to do the following will be lost:
| |
| #** Upload Scans
| |
| #** Correlate Tests
| |
| #** Alter Bubble Sheets (Correct errors)
| |
| #* The following will remain available:
| |
| #** View Bad Scans
| |
| #** Access Test Results
| |
| #** Send Results to Students via Email (Students will only be able to see their own marked coversheet, their selections, and the MarkBox selections.)
| |
| #** Adjust scoring, with some limited functionality
| |
| # If certain students need to be excluded from emails, click the checkbox under "Exclude" next to the student's name.
| |
| ----
| |
| | |
| === MarkBox Tools ===
| |
| | |
| '''1. Re-download Test'''
| |
| At any time, you may download the original test to the default download location on the user's computer by clicking '''Re-Download Test'''.
| |
| | |
| '''2. View / Mark Bad Tests and Scans'''
| |
| After [Uploading Tests] and [Correlating Scans], there may be rare instances where scans are failed and cannot be marked by the System.
| |
| | |
| # To view and manually mark any bad tests and scans, click '''View / Mark Bad Tests and Scans'''.
| |
| # If there is only one version, Manual Marking will display the solution to the test. If there are multiple versions, type the test version number into the text field under '''Manual Marking''' and click '''Get Solutions'''.
| |
| # Find the bad tests and scans under '''Bad Tests / Mis-Scans''' and click on a test to view.
| |
| # Click '''Download All''' to download all tests or '''Download''' to download individual tests and then print them to mark them if desired.
| |
| | |
| * NOTE: These bad tests cannot be re-integrated into your MarkSet, it is only for archiving purposes here.
| |
| | |
| '''3. Adjust Scoring'''
| |
| | |
| You may change the correct answer, remove questions from marking, or award bonus points.
| |
| | |
| # Click '''Adjust Scoring''', which will open the '''Edit Test Answers''' page.
| |
| # Click '''Download Unscrambled Test''' to download the test in the order displayed or click '''Download Unscrambled Solutions''' to download the solutions to each question in order.
| |
| | |
| * To change the correct answer for a question, click on the correct letter to replace.
| |
| * To add bonus marks for a question, increase the value of '''Correct Points'''.
| |
| * To penalize an incorrect response, increase the value of '''Incorrect Points'''.
| |
| * To remove a question from marking, set the value of '''Correct Points''' and '''Incorrect Points''' to 0.
| |
| | |
| Once satisfied with any changes made, click '''Save Changes'''.
| |
| ----
| |
| | |
| === Contacts ===
| |
| | |
| '''General Support Requests'''
| |
| | |
| Mirko Vucicevich: [mailto:mvucicevich@uwaterloo.ca mvucicevich@uwaterloo.ca]
| |
| | |
| '''High Severity Fallback'''
| |
| | |
| Steve Weber: [mailto:s8weber@uwaterloo.ca s8weber@uwaterloo.ca]
| |