Numerical Lameness score intra-rater reliability, inter-rater reliability, and validity in canine gait assessment

December 26, 2014
Tammy Wolfe, DPT, PT, CCRP, GCFP
Special thanks to Chelsea Wells for administrative assistance and collection of data.

Background and Purpose

Among the types of canine gait assessment tools available, use of the force plate is clearly the “gold standard.”  However, because of financial, time, and space constraints, most veterinary and animal physical therapy professionals do not have the luxury of utilizing a force plate to analyze the majority of their patients’ gait patterns.  In an attempt to create more objective and universal communication between medical providers and in an attempt to document progress in rehabilitation, several lameness scales have been created and utilized.  The visual analogue score (VAS), the 0 to 5 lameness scale and the 0 to 4 lameness scale are a few of the more frequently utilized scales in an  attempt to describe lameness and changes in gait of canine patients.  The purpose of this study was to evaluate the intra-reliability,  inter-reliability, and validity of veterinarians and physical therapists utilizing a numerical 0 to 5 lameness scale to assess lameness in dogs.

Previous reliability and validity studies have compared various aspects of lameness and various types of scales.  One study compared clinician and owner VAS to force plate analysis in nine dogs with a diagnosis of fragmented medial coronoid process and found that there was a minimal correlation between owner VAS and force plate analysis and no correlation between clinician VAS and force plate analysis when taken at one, two, six and 12 months post diagnosis. (Burton NJ 2009)  Another study assessed the validity of a visual analogue scale questionnaire for use in assessing pain and lameness in dogs.  Forty-eight dogs with mild to moderate lameness were assessed by the owners utilizing an analogue questionnaire with 39 questions.  Only nineteen of the questions showed moderate repeatability of >.6. (Hudson JT 2004)

A third study evaluated the agreement between numerical rating scales (NRS), VAS, and force plate gait analysis in dogs.  In this study, three veterinarians with orthopedic training rated lameness utilizing the NRS and VAS before surgery, at 4 weeks and at 8 weeks post surgery.  Inter-reliability was low with no significant relationships between any observer’s scores and force plate data except in extreme lameness. (Quinn MM 2007)  And a fourth study comparing numerical gait analysis with force plate analysis before and after induced lameness in normal dogs showed a low correlation between scores obtained from vets with orthopedic training and veterinary students and the force plate analysis. (Waxman AS 2008)

Methods

In this study, 19 consecutive dogs arriving at a canine physical therapy office on a randomly selected day were videotaped by a third year physical therapy student with no canine experience.  This was a double blind process, where neither the student nor the clients were aware of why they were being filmed.  Two videos were taped.  The first video was from a lateral viewpoint and the second video was from a cranial/caudal viewpoint.  Each video filmed the dogs walking 140 to 150 feet on a smooth surface.

Each video was numbered, transferred to a CD, and sent to 5 canine-certified physical therapists and 5 general practice veterinarians with the scale described as below:

0/5 = Normal gait pattern
1/5 = Mild lameness, needing a trained eye to see
2/5 = Moderate lameness with a normal stride length and partial weight bearing
3/5 = Moderate lameness with shorter stride length and partial weight bearing
4/5 = Severe lameness with toe touch weight bearing and minimal use of the limb
5/5 = Non weight bearing

The physical therapists and veterinarians returned the score sheet.  In 6 to 8 weeks, they received another CD with identical videos in a different order.  They scored each video again and returned the score sheets.

Results

Intra-rater reliability and inter-rater reliability were zero to moderate.  The highest intra-rater correlation was .84 and the lowest was .26.  And the median was .47 when all 10 evaluators were considered.  The physical therapist group intra-rater reliability was a moderate .58 and score correlation in the veterinarian group was a low .37 correlation.  Inter-rater reliability was based on the number of dogs that everyone in the group scored identically.  Among the physical therapists, the score correlation was a low .16, and among the veterinarians, the correlation was zero.

Discussion

In previous studies, several types of lameness scales were compared over a period of time and the lameness score of each dog in the study could have changed.  In an attempt to limit variables, this study did not compare different gait samples over a period of time and did not attempt to compare different types of lameness scales to each other.  The idea behind the study was that, even if lameness scoring varied from person to person, if the intra-rater reliability scores were high, at least documentation of a single medical professional would be reliable in showing changes in lameness over time.  This hypothesis proved to be false in this study.  It is possible that orthopedic-trained veterinarians may have scored higher in both intra-rater and inter-rater reliability than general practice veterinarians.  It is also possible that additional training in scoring may have increased the intra and inter-reliability scores for all participants.

In conclusion, the veterinary profession continues to be primarily subjective in analysis of canine gait and lameness.  The 0 to 5 scale does not appear to be a valid form of gait analysis for most individuals to utilize to show changes in gait or to use as an outcome measure.  It can be, however, moderately valid to utilize when charting progress of an individual dog by one practitioner.  A more valid universal, objective lameness analysis scale would be beneficial so that physical therapists and veterinary professionals can accurately document changes in lameness with confidence in the objectivity of their observations.

References

1. Burton NJ, Owen MR, Colborne GR, Toscano MJ. Can owners and clinicians assess outcome in dogs with fragmented medial coronoid process? Vet Comp Orthop Traumatol 2009:22:183-189.
2. Hudson JT, Slater MR, Taylor L, Scott, HM, Kerwin SC. Assessing repeatability and validity of a visual analogue scale questionnaire for use in assessing pain and lameness in dogs. American Journal of Veterinary Research 2004: 65:1634-1643.
3. Quinn MM, Keuler NS, Lu Y, Faria ML, Muir P, Markel, MD. Evaluation of Agreement Between Numerical Rating Scales, Visual Analogue Scoring Scales, and Force Plate Gait Analysis in Dogs. Veterinary Surgery 2007: 4:360-367.
4. Waxman AS, Robinson DA, Evans RB, Hulse DA, Innes JF and Conzemius MG. Relationship Between Objective and Subjective Assessment of Limb Function in Normal Dogs with an Experimentally Induced Lameness. Veterinary Surgery 2008 37: 3: 241-246.

Copyright © 2014 by Dr. Tammy Wolfe, DPT, PT, CCRP, GCFP. All rights reserved.