In rbcavanaugh/pnt: Computer adaptive forms of the Philadelphia Naming Test

Introduction

The Philadelphia Naming Test (PNT; Roach et al., 1996) is a 175 item picture naming test utilized to assess anomia in people with aphasia. It is made up of diverse stimuli varying in length, age of acquisition, and lexical frequency (Francis & Kucera, 1982), it has a well-defined scoring system for classifying anomic errors (Dell et al., 1997), and its total score correlates highly with aphasia severity (Walker & Schwartz, 2012). The PNT was developed in a classical test theory framework (Traub, 1997). There are two primary outcomes measures that can be utilized to score the original PNT. First, an overall accuracy score. Second, the total number of response types observed. The former scoring type provides an overall estimate of anomia severity, and the total number of response types can be utilized to make inference on the underlying lexical-semantic and lexical-phonological abilities.

Despite the widespread use of confrontation picture naming tests, their utility for quantifying anomia is limited due to their construction with classical test theory. For example, the standard error is considered constant across all items regardless of the items difficulty or the participants naming ability. This assumption ignores that measurement error varies as a function of item difficulty and naming ability. To illustrate this point, consider the following example. If a test were administered to a group of people with very mild anomia that only included very easy items they would all likely score perfectly. Therefore, rank ordering these participants based on their naming ability would not be possible. However, if a test of more difficult items were delivered to the same group of people with aphasia, it is less likely they would all perform perfectly and rank ordering the individuals would be possible.

Item Response Theory (IRT; De Ayala, 2009; Lord, 1980) is a modern psychometric theory that formalizes the probability of a correct response given an items difficulty and the participants ability. difficulty can be conceptualized as the ease or challenge of producing a correct response and ability is operationalized as the degree to which an individual possesses a certain skill. As it pertains to the assessment results from the PNT accessed through this shiny application (Chang et al., 2021) difficulty and ability reflect the degree of naming difficulty and degree of degree of naming impairment, respectively. In IRT applications the range of both difficulty and ability ranges from -4 to 4.

As it pertains to this application, our research group first estimated the 1-Parameter logistic (1-PL) model (a utility of IRT) with archival data (Mirman et al., 2010) from 251 participants with aphasia (Fergadiotis Gerasimos et al., 2015). The 1-PL model is expressed as follows:

[formula 1]

Where the probability of (P) of a correct response by examine j with a naming ability [THETA] is equal to a log transformation of the participants naming ability minus the item's difficulty [BETA]. Note that [ALPHA] is the item discrimination parameter, which is assumed to be equal for all items under the assumptions of the 1-pl model.

Item characteristic curves can then be estimated for each item for all possible ability levels. Item characteristic curves are sigmoidal in shape and are presented for the following items in Figure 1: Ambulance, ball, and microscope. Note that zero is considered the average naming ability estimate. As can be extracted from the figure, a participant with a naming ability has a probability of responding correctly to ambulance, ball, and microscope of roughly 0.33, 0.80, and 0.15 respectively For an interactive application that can be utilized to plot item characteristic curves for any item on the PNT see https://aswiderski.shinyapps.io/IRTapp/.

The amount of information that each item carries about a given naming ability is formalized by the item information function and information is maximized when the item's difficulty matches the participants ability. Information is formalized as:

[formula 2]

where the information for item i is the product of the probability of a correct response for an individual with an ability of [THETA] multiplied by the probability of an incorrect response for item i from an individual with an ability of [THETA]. In the IRT framework, the standard error is the inverse of the information function:

[formula 3]

Therefore, the more information an item provides the less measurement error is present.

Next sections: