Home Print this page Email this page
Users Online: 562
Home About us Editorial board Search Ahead of print Current issue Archives Submit article Instructions Subscribe Contacts Login 

 Table of Contents  
EXPERT STATUS REPORT
Year : 2014  |  Volume : 2  |  Issue : 3  |  Page : 123-125

Test blueprinting for multiple choice questions exams


Department of Medical Education and Postgraduate Studies, Saudi Commission for Health Specialties, Riyadh, Saudi Arabia

Date of Web Publication31-Jul-2014

Correspondence Address:
Imran Siddiqui
Department of Medical Education and Postgraduate Studies, Saudi Commission for Health Specialties, Riyadh
Saudi Arabia
Login to access the Email id


DOI: 10.4103/1658-600X.137888

Rights and Permissions

How to cite this article:
Siddiqui I, Ware J. Test blueprinting for multiple choice questions exams. J Health Spec 2014;2:123-5

How to cite this URL:
Siddiqui I, Ware J. Test blueprinting for multiple choice questions exams. J Health Spec [serial online] 2014 [cited 2019 Nov 21];2:123-5. Available from: http://www.thejhs.org/text.asp?2014/2/3/123/137888

The test development process includes an overall plan, content definition and test specification. The overall plan of an assessment will then include the identification of the target group, and the purpose of the exam such as selection, classification, placement, diagnosis or certification. Furthermore, the details of the test design, assembly and production, administration, scoring, standard setting, reporting results, item banking, post hoc analysis and logistic requirements must all be decided. [1] Once the purpose of the test is identified then the most crucial step in the development of a test is to ensure that the test should measure what it is supposed to measure, known as the test validity. Constructing a test blueprint enables stakeholders to have a bigger picture of the exam and determine relative weights of individual content. [2]

There are a number of important uses for a test blueprint, such as:

  • Reduce validity threats which occur due under or oversampling bias of the subject matter
  • Target item writers to give a clear message in which content area they need to write items to assess specific learning outcomes at the prescribed level of complexity
  • Guide examinees to focus on the important areas of the subject matter. Since the test blueprint is a tool that ensures content validity of the exam, the examinees will get a clear message that their exam is balanced
  • Define the acceptable evidence to infer the areas of the subject matter
  • Give constructive feedback to examinees and faculty


The blueprint is a general term that means an outline of the detailed plan or a plan of action. Originally, this term was introduced in the late 19 th century in architecture for structural designs on prints composed of white lines on a blue background or blue lines on a white background before building a house. [3] Test blueprints which are sometimes called the table of content specifications, define the assessment instrument of the exam items allocated to each learning objective or content, along with the specific item character (e.g., with or without a scenario) given a certain weight to be included in the exam. It serves as a tool to align learning objectives with assessment items of the subject matter. [4]

The test blueprint could be developed in three forms, (a) the overall high level blueprint in which graduate outcomes are laid down against assessment tools, (b) the year or phase blueprint which is designed to assess learning outcomes of a year or phase and hence suitable for promotion or graduation exams, (c) while the "single examination" blueprint is for licensing exams. The test blueprint should be constructed by a small group of educators or trainers who are directly involved in the teaching or training during the programme but ideally the people who constructed the programme should propose the test blueprint.

A basic test blueprint is made up of a table called the matrix. It is a two-dimensional matrix chart, with an X and Y axis, which describes the sample of items to be selected in the test. The preferable format uses a spreadsheet which allows easy calculations and fine tuning of the proportions or number of items for an exam. Learning targets that could be termed as sections are laid down in rows, section headings are entered on the Y axis, and the columns are used to write domains which are entered on the X axis with specific tasks across each of the sections [Table 1]. The learning targets or sections of a programme or course will depend upon its specific nature and philosophy of the subject matter. These are usually driven by the curricular content, learning objectives or in some cases from pre-determined common clinical presentations or problems.

The next thing to do is to make a list of tasks from the learning outcomes, which would be expected to be achieved by the examinees. These tasks or learning outcomes could be termed as domains. Despite the variations in courses, subject matter and learning objectives, the domains may be common to many different blueprints, like recall or application of knowledge in basic sciences, appreciation of pathophysiology, identification of appropriate investigations, making the correct diagnosis, managing the case appropriately, and so on. The correct identification of domains is as important as selecting the correct sections, therefore, it is usually desirable to make two separate lists of these two because examinees may need to react in the same way (make a diagnosis) to different sections of the subject matter and he or she may need to behave in different ways (diagnose, manage or order investigations) to the same section of the subject matter. [4] For instance, an examinee may show poor performance in the pathophysiology of various diseases or may show poor performance in the whole of one section.

One has to note that the content and learning objectives should be written down in greater detail for teaching purposes, but in designing the test blueprint they should be grouped in major categories. While creating the test blueprint it is preferable to have 8 - 14 sections and 3 - 7 domains. The simple way to work out the correct number of sections and domains is to first decide the number of test items in the exam. Ideally, the number of cells in the test blueprint which are created in the spreadsheet of the test blueprint should not exceed 70% of the proposed number of items for that exam. This means that if an exam paper shall contain 100 items and the number of sections is 15, the number of domains should be approximately four giving the total number of cells in the blueprint matrix of 60, 60% of the total number of questions required for the exam. This will allow an appropriate representation of the content as compared to the test blueprint having 25 sections and six domains which might result in the production of an exam with no representation of the various subject matter areas.

The next step should be assigning the relative weight to each subject matter sections. The number of questions in each cell of the spreadsheet could be determined by a number of factors such as how important each area is and how much time was devoted to each area during instruction. In medicine, one way of assigning relative weight in cells of the spreadsheet is by considering the importance of the area in terms of impact and frequency of related clinical conditions. For instance, the diagnosis of ischaemic heart disease or diabetes mellitus should be assigned more weight than the diagnosis of Gaucher's disease or Takayasu's arteritis. It is important to note that the decision about the total number of items in the exam should be made prior to assigning the relative weights in the cells. It has been reported that to ensure a higher reliability of the exam the number of items should be at least 80 - 100 with an average discrimination of > 0.3. [5]
Table 1: Part of a test blueprint, as an example with 3 sections and four domains. In each cell the proportions of items testing low and high cognition are given, K1:K2


Click here to view


Often all six levels of Bloom's taxonomy have been used for test blueprints to distinguish different levels of cognition. [6] However, this is too complex and experts seldom agree on which level of the taxonomy to assign an item. Therefore, we have called the lowest two levels of Bloom's taxonomy, recall of memorised information and its comprehension, K1. While, higher levels of cognition, including interpretation, analysis, evaluation and the application of knowledge are called K2. If the introduction of a level of complexity in the exam is desirable, then it would be advisable to introduce the levels of cognition (K1 and K2) into the blueprint. The decision about how many K1 or K2 questions should make up an exam could be a general decision for the whole exam, and given as a ratio, such as 30:70, K1:K2 respectively. But it would be more appropriate to assign the desired number of K1 and K2 in each cell of the test blueprint matrix. It should be noted that exams in medicine and allied health sciences should have a higher proportion of K2 items where decisions about fitness to practice will be made. [Table 1] shows an example, from a portion of a test blueprint, with its essential components.

If the test blueprint is correctly aligned with the various areas of the subject matter, the remaining part will be the alignment of the test items with the blueprint. To do this, each and every item has to be classified according to the test blueprint. If there is an existing question bank, those assessment items that match the blueprint can be retrieved, and where there is no question available, a clear instruction could be given to the item writers about how many extra items they should write in a specific area of the subject matter.

In conclusion, constructing a test blueprint includes the following steps:

  • Identify the content areas of the subject matter (sections) to be measured by the exam
  • Identify the learning outcomes (domains) across each section to be measured by the exam
  • Weight the sections and domains in terms of their relative importance
  • Construct a spreadsheet in accordance with relative weights by distributing the test items
  • Allocate the relevant cells of the spreadsheet with their level of cognition


 
  References Top

1.Hochlehnert A, Brass K, Möltner A, Schultz JH, Norcini J, Tekian A, et al. Good exams made easy: The item management system for multiple examination formats. BMC Med Educ 2012;12:63.  Back to cited text no. 1
    
2.Downing SM. Twelve Steps for Effective Test Development. In Downing SM and Haladyna TM. Eds.). Handbook of test development. Mahwah, NJ: Lawrence Erlbaum Associates 2006; p. 3-25.  Back to cited text no. 2
    
3."blueprint" Shorter English Dictionary. 6 th ed. Oxford University Press; 2007.  Back to cited text no. 3
    
4.Grönlund NE. Assessment of student achievement. Allyn & Bacon Publishing, Longwood Division, MA, USA, 1998.  Back to cited text no. 4
    
5.Ebel RL. Measuring educational achievement. Englewood Cliffs, NJ: Prentice-Hall, 1965.  Back to cited text no. 5
    
6.Bloom BS. Taxonomy of educational objectives. In: Cognitive domain. Vol. 1. New York: McKay; 1956.  Back to cited text no. 6
    



 
 
    Tables

  [Table 1]


This article has been cited by
1 Quality assurance of test blueprinting
Ghada Eweda,Zakeya Abdulbaqi Bukhary,Omayma Hamed
Journal of Professional Nursing. 2019;
[Pubmed] | [DOI]



 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
References
Article Tables

 Article Access Statistics
    Viewed5826    
    Printed88    
    Emailed0    
    PDF Downloaded522    
    Comments [Add]    
    Cited by others 1    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]