Data held by the Ministry of Justice on nearly half a million urine samples in its random mandatory drugs testing (rMDT) of prisoners could be more effectively used, according to the Royal Statistical Society.
 
Public accountability and methodological scrutiny would benefit if the data gathered from prisons in England and Wales were properly analysed and fully published, the Society says. In addition, prisons' related key performance indicator should be re-defined to differentiate between testing positive for opiates, such as heroin, and testing positive for cannabis.
 
The Royal Statistical Society Vice-President, Professor Sheila Bird states:
 
"The Ministry of Justice has data on nearly half a million urine samples from prisoners in England and Wales who have been subject to random mandatory drugs testing. These data have been costly to collect and are not analysed with sufficient rigour.
 
"A statistically sound analysis would allow the Government to learn more about the weekday pattern of prisoners' use of specific illicit drugs and to gauge prisons' success in offering substitution therapies such as methadone or buprenorphine. Full publication would ensure separation of the wheat (underlying yearly trends in prisoners' use of specific drug) from the chaff (variation in how rMDT is implemented by different prisons across the years). Proper scrutiny of results on tens of thousands of rMDTs per annum requires more than headline figures that some percentage of rMDTs were 'positive for drugs'.
 
"As presently worded, prisons' key performance indicator on rMDT leads to a muddled situation. Positive tests for different drugs, for example heroin use in the past 2-3 days and cannabis use in the past 2 weeks are essentially being treated as equal, although clearly the health implications are very different."
 
The Society is raising concerns that:
Independent research in 1997 (ref. 1 below) first demonstrated that the percentage of prisoners testing positive for opiates in random mandatory drugs testing underestimated by a factor of 1.5 to 3 the proportion who were inside-users of heroin.
Home Office-funded research (ref. 2 below) subsequently confirmed, for England and Wales, that prisoners' inside-use of heroin was around 3 times greater than the proportion testing positive for opiates in rMDTs (ref. 3 below).
Heroin stays in the urine for 2 to 3 days, cannabis for 2 to 3 weeks. It is methodologically unsound (as well as having the perverse potential of setting up an incentive for inside use of heroin) for prisons' rMDT performance indicator (see point 7 below) to equate testing positive for opiates (which measures recent use) and for cannabis.
For statistical validity in measuring yearly patterns and trends of drug misuse in prisons, opiate positive rates need to take account of the weekday of obtaining rMDT urine samples Mondays versus Thursdays versus Sundays, for example. Unless the weekday pattern of rMDTs has remained the same year-on-year, comparison of opiate positive rates will not be like-with-like.
For reliability, recent uptake of methadone prescriptions in prisons in England and Wales requires rMDT to differentiate i) between testing positive for heroin versus for methadone; and ii) between methadone positive prisoners for whom methadone had been prescribed versus non-prescribed.
If those who have received prescribed opiates in prisons are excluded from rMDT, then "% positive for drugs in rMDT" in 2007 would be testing a different sub-population of prisoners than in earlier years (prior to methadone uptake). Ministry of Justice needs to justify the like-for-like basis of its yearly comparisons.
In England and Wales, rMDT positive rate is a key performance indicator for prisons and the target is to: "improve on previous year". It is statistically unsmart to ignore the uncertainty in "previous year's performance" by giving it target status in this manner (ref. 4 below).
rMDT of prisoners fails several of the performance monitoring recommendations in the Royal Statistical Society's report "Performance Indicators: Good, Bad, and Ugly" (ref. 4 below):
The Society's recommendations are:
To have clearly specified objectives, and achieve them with methodological rigour;
To be so designed that counter-productive behaviour is discouraged (equating heroin positive with cannabis positive);
Cost-effectiveness should be given wider consideration in both the design and the evaluation of performance monitoring procedures (Scottish Prison Service has ceased rMDT);
Independent scrutiny of a performance monitoring procedure is needed as a safeguard of public accountability, methodological rigour, and of the individuals and/or institutions being monitored (analyse by weekday of sample and by drug of interest separately for different types of prison);
Several sources of variation essential (weekday) or systematic (type of prison) as well as random must be recognized in design, target setting and analysis;
The reporting of performance monitoring data should always include measures of uncertainty (provide both numerator and denominator, not mere percentage)
References:
1. Bird AG, Gore SM, Hutchinson SJ, Lewis SC, Cameron S, Burns S. Harm reduction measures and injecting inside prison versus mandatory drugs testing; results of a cross-sectional anonymous questionnaire survey. British Medical Journal 1997; 315: 21-24.
2. Singleton N, Pendry E, Simpson T, Goddard E, Farrell M, Marsden J, Taylor C. Home Office Online Report 03/05 2005 (http://www.homeoffice.gov.uk/rds/).
3. Bird SM. Random mandatory drugs testing of prisoners. Lancet 2005; 365; 1451-1452.
4. Royal Statistical Society Working Party on Performance Monitoring in the Public Services (chair: Bird SM.). Performance Indicators: Good, Bad, and Ugly. Journal of the Royal Statistical Society Series A 2005; 168(1): 1 27 (see also www.rss.org.uk).