Talk:Computerized adaptive testing

CAT question

I'm proposing Computer Adaptive Tests for final exams in my University, but I find some coarse reaction. They argument that this kind of tests punishes the most intelligent people and unmotivate them. Has anyone had an experience at implementing CATs for academic courses? -- Mario Montoya (User:200.23.145.23)

In order to implement an adpative test, you must first have scaled the pool using item response theory. The sample size typically required for IRT scaling is at least 200 and preferrably several hundreds or thousands of people. As a consequence, it is uncommon and generally impractical for use in academic settings unless there is a dedicated staff who produce one or a few exams each year and who can manage the CAT. The way this usually evolves is to first administer the exam on computer for a period of time while the database of questions and answers accumulates. Then the scaling is performed and an adaptive version of the exam is introduced.

I haven't seen any studies that reinforce the notion that CAT's punish the most intelligent students although CAT introduces a new administration method that many students dislike. For example, item review and modification are generally disallowed in CAT because if you get an easier item, then you know that you got the previous item wrong. If you can backup and correct that answerm then your score will be biased. In fact, if item review is allowed then here's how you subvert the exam: Intentionally answer each item wrong until you finish the exam. The CAT will have served the easiest possible items to you. Now go back and answer all the easy items correctly. If you actually get 100% correct, then you should pass but if you don't then you can take them to court and have them explain to a judge why, with 100% correct, you did not pass their exam! (This is, from memory, Howard Wainer's argument against allowing item review.) Many students are frustrated that they cannot skip harder questions and return to them later.

Why would you want to introduce CAT exams in an academic setting? CAT is really only helpful if you need extremely reliable tests over a broad range of ability (e.g., for categorization or on a broad entrance exam like the GRE). CAT is too much work and, as Wainer shows, too much of a security risk for general usage. -- User:Amead

Citations

I think the writers of this article should not only cite the books they used in writing the article (which they seem to have done), but also that they cite which sentences/paragraphs came from which texts. As I understand, citing sources is not just listing the book, but also citing the source in context, that is, in the article text. For example, in APA style, if I were to say "Computer-adaptive tests were first used in 1980," I would follow the sentence by citing the name of the author and the year of publication, like so: "(Michels, 2000)." In this article, nothing comparable to that is done. If I question the factual accuracy of a sentence or paragraph, I don't know which source to look at in the reference list. This is why many articles, such as Normal curve equivalent, use a reference style that cites particular sources for the statements in question. That is why I added the {{references}} template to the top of the article. There may be references cited at the end of the article, but within the text. – Chris53516 ^(Talk) 16:38, 20 November 2006 (UTC)