Lies, Damned Lies, and Jeremy Hunt’s Statistics

The government’s argument in their attack on junior doctors’ pay and conditions has been that they had a manifesto commitment to introduce 7-day access to all aspects of health care and that this was necessary to reduce excess deaths among weekend hospital admissions. The government’s approach seems to amount to forcing junior doctors to work more at weekends for less pay. But, unless they also force them to work longer hours, this must reduce the number of doctors on weekdays. If the original problem of excess deaths was due to a lack of junior doctors at weekends, the result would be to equalise death rates by lowering death rates following weekend admissions and raising those following weekday admissions.

Health Secretary Jeremy Hunt was very keen to talk about the evidence of excess deaths to justify his actions and, of course, evidence is very important. He claimed “We now have seven independent studies showing mortality is higher for patients admitted at weekends.” Therefore, we will look at this evidence.

The Department of Health’s “evidence”

The DH says there is significant evidence of a “weekend effect” where patients admitted over the weekend have higher rates of mortality (or “morality,” in one amusing typo).1 The DH lists eight pieces of what they call research in support.

1 The major study cited by DH is from the British Medical Journal (Freemantle et al., 2015):2 one of its co-authors is Bruce Keogh, National Medical Director of NHS England. It found that death rates were higher for patients admitted on Fridays (2%), Saturdays (10%), Sundays (15%) and Mondays (5%) than on other days. Since the overall death rate within 30 days for all admissions is 1.8%, this means that the Sunday rate is 2.1% or 3 in 1000 “extra” deaths.

Of course, we need to understand why, and this is where it is important to look at how ill patients are on the day of admission. The study informs us that, while 29% of weekday admissions are emergencies, on Saturdays the figure is 50% and on Sunday 65%. Using another criterion, mortality risk from all factors except day of admission, while 20% of weekday admissions were in the highest category, 25% on Saturdays and 29% on Sundays were in this highest risk of dying group. On these bases, we would expect an increased death rate for weekend admissions of anywhere between 25% and 125%. The observed ‘excess’ of 15% on Sundays should be a cause for congratulation.

This paper is an update of the previous study by Freemantle et al. (2012)3 (see 5 below), also including Keogh. The findings were broadly similar except that the death rate on Saturdays and Sundays were very significantly lower than the average for weekdays. In the update this curious fact, which certainly needs discussion and explanation, is barely mentioned.

To summarise, death rates for admissions on Saturdays and Sundays are increased by 10 to 15% but death rates for those already in hospital are reduced by 5 to 8%. Thus, the main source of support for the government’s Seven Day NHS plans does not provide any evidence for it. The weekend death rates for all patients are in fact far lower than one would predict from the seriousness of their illness. Nevertheless, the authors try to explain what they persist in describing as an overly increased rate.

They cite: reduced or altered staffing and mix of skills at weekends; impact of shift system; fewer senior staff available; more staff who are unfamiliar with policies; and need for more prompt treatment than available. They give the example of treatment for hip fractures which should be very prompt but they admit that their figures showed no significant difference for this condition. Nothing about the need for more junior doctors at weekends, still less that Saturdays should be counted as part of the normal week and that junior doctors should have this part of their pay cut.

Freemantle et al. (2012)3 tentatively say that “It may be that…7-day access to all aspects of care could improve outcomes for higher risk patients…admitted at the weekend.” They then say that the economics need looking at to see if this is “an efficient use of scarce resources.” The update, Freemantle et al. (2015),2 states that “It is not possible to ascertain the extent to which these excess deaths may be preventable; to assume that they are avoidable would be rash and misleading.” They draw attention to the reduced level of support services at weekends and state that “There is evidence that junior hospital doctors feel clinically exposed during the weekend.” This does not mean there are too few junior doctors on duty but that the support services and senior staff are not sufficient. Nothing about cutting juniors’ pay and forcing them to do more weekend shifts! They signally fail, in fact, to address the question in their paper’s title: “a case for expanded seven day services?

2 Ruiz et al.4 shows that there is a “weekend effect” in other countries where “The participating hospitals represent varied models of service delivery.” Nothing special about the NHS, then!

3 The East Midlands 7 Day Services Project merely quotes the results of Freemantle et al. (2012).3 It provides no further evidence but discusses how services might be expanded. It talks of increasing access to diagnostic services but, apart from calling for a more speedy examination by consultants, makes no specific mention of doctors. It does identify a substantial need for more funding, not so far addressed by DH.

4 NHS Services, Seven Days a Week Forum also quotes Freemantle et al. (2012).3 It then discusses 7-day services wholly in terms of increases in the availability of consultants. Juniors are mentioned in the context of benefiting from the advice and supervision of more consultants, which would reduce the need for such a large number at weekends, while improving their medical education and training.

5 This is Freemantle et al. (2012)3 and is essentially the same as the first paper (above).

6 Seven Day Consultant Present Care5 calls for daily reviews of patients by consultants and consultant-recommended treatments to be available seven days a week. Support services should also be available seven days a week both in hospitals and in primary care (general practice etc.). No mention of juniors except that their training would benefit from the wider availability of consultants. No support for Hunt’s attack on junior doctors.

7 This (Aylin et al.)6 is research on death rates among emergency admissions. It predates the Freemantle papers and the latter incorporate its findings. It found a 10% higher death rate at weekends. It wasn’t able to rule out that emergency admissions at weekends (of which there were fewer than during the week) were different in some way. Freemantle et al. (2012; 2015)3,2 showed that they were different, with more in the highest risk category. In particular, they suggest that cancer patients in a terminal condition were more likely to be admitted to hospital at weekends because of a lack of community and primary care options then. This would skew the death rates of cancer patients in hospitals. Aylin et al. say that this “may be a whole health system problem.” Interestingly, they quote studies showing that there is no “weekend effect” in intensive care units, attributing this to the high level of consultant input. Nothing about junior doctors but more about lack of senior staff and services.

8 The final paper (Temple, 2010) refers to trainees (i.e., junior doctors) being unsupported and unsupervised (according to DH website – the paper itself seems no longer available – “404 page not found”). Nothing justifying pay cuts and increased weekend working for juniors.

Jeremy Hunt’s new clothes

The DH’s evidence is aimed at supporting the idea of a 7-day NHS but they themselves, in their introduction, only mention urgent and emergency care and consultant cover. There is nothing about junior doctors. Of the eight papers, only three provided data and these were updated samples of the same type of data; one showed that the NHS “weekend effect” was international; the other four called for more consultant support and hospital services at weekends and increased funding; none mentioned junior doctors apart from their need for more senior support. There is one very simple possible explanation for this – people are admitted at weekends because they have to be – they are much more ill. A recently-leaked DH report7 admits that changing to a 7-day NHS may have no effect on the supposed excess deaths; it “cannot evidence the mechanism by which increased consultant presence and diagnostic tests at weekends will translate into lower mortality and reduced length of stay.” Jeremy Hunt’s attack on junior doctors’ pay and conditions (like his support for homoeopathy) completely lacks evidence.

1 https://www.gov.uk/government/publications/research-into-the-weekend-effect-on-hospital-mortality/research-into-the-weekend-effect-on-patient-outcomes-and-mortality

2 Increased mortality associated with weekend hospital admission: a case for expanded seven day services? Freemantle et al. 2015 http://www.bmj.com/content/351/bmj.h4596

3 Weekend hospitalization and additional risk of death: An analysis of inpatient data. Freemantle et al. 2012 http://jrs.sagepub.com/content/105/2/74.full.pdf+html

4 The Global Comparators project: international comparison of 30-day in-hospital mortality by day of the week

5 from Academy of Medical Royal Colleges

6 Weekend mortality for emergency admissions. A large, multicentre study

7 http://www.theguardian.com/society/2016/feb/15/weekend-effect-on-hospital-deaths-not-proven-say-hunts-own-officials

Advertisements

One thought on “Lies, Damned Lies, and Jeremy Hunt’s Statistics

  1. Pingback: Hunt debunked: there is no “weekend effect in the NHS – Random Walk in Science

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s