Vaping hyperbole maestro Prof David Nutt

The two giant pillars underpinning vaping advocacy are that (1) vapes are essentially all but as safe as breathing clean air and that (2) vaping is now finally cemented at the highest level of evidence as “the most effective way” we have of quitting smoking. I’ve previously looked at the first of these claims here. This time I’ll look at the effectiveness argument.

Randomised controlled trials: the “gold standard of evidence”

Many smoking cessation advocates megaphone evidence obtained from systematic reviews of randomised controlled trials (RCTs) to argue for the superiority of different drugs or nicotine replacements (especially vapes), over largely unassisted quitting. Quitting cold turkey is routinely  cast into a purgatory dungeon as having the lowest level of effectiveness of all ways of quitting.  This is despite 70 years of evidence showing that quitting without drugs, aids or professional assistance is and always has been by far the way that most ex-smokers stop on their final quit attempt (see later in this blog). Apparently and preposterously, such quitting is not “evidence based”.

The double-blinded RCT has been elevated to the hallowed zenith of evidence-based medicine. But there are many important ways in which RCTs seriously differ from how therapeutics are used in real world settings, away from the close involvement of RCT researchers. Nowhere is this clearer than with smoking cessation.

When a vaping advocate wants to broadcast the wonderful evidence on vaping, they rarely qualify their claims about RCTs by highlighting or even noting any of these differences. So what does the RCT evidence say and what are these critical differences?

Cochrane evidence on smoking cessation RCTs

The Cochrane library is a global project to pool RCTs on treatments and diagnostics for a wide range of health issues, winnowing those with high quality evidence from those with significant flaws. It has a dedicated sub-section focussed on drugs, nicotine replacement, behavioural and policy interventions to stop tobacco use.

The most recent (10 Nov 2025) Cochrane review of the effect of vaping on smoking cessation measured at (at least) 6 months after the trials concluded that across 104 studies involving 30,366 adults:

“for every 100 people using e-cigarettes to stop smoking 8 to 11 might successfully stop, compared with only 6 in 100 people using NRT, 6 in 100 using e-cigarettes without nicotine or 4 of 100 people having no support or behavioural support [typically advice, counselling etc] only.”

For simplicity here, let’s be generous and take 10 in 100 as the outcome for the e-cigarette group, instead of the stated range 8-11. The Cochrane sentence describes in plain English what is known as the absolute difference in efficacy between vapes, NRT and no support/behavioural support when it comes to quitting. But vaping advocates seldom lead with statements of absolute effect sizes, preferring to use relative effect sizes. Here’s a quick explanation of the important difference between relative and absolute differences, very basic but often misleadingly used concepts at the heart of questions about smoking cessation effectiveness (effectiveness means how something works in the real world; efficacy means how it works in the artificial world of trials).

Absolute vs relative impact

An absolute difference means the exact mathematical difference in event rates (here quit rates) between groups. So if 100 people were to use a smoking cessation medication and 10 quit, while 100 took a placebo and 5 quit, the difference between the two groups at follow-up would be a 5 percentage-point absolute difference. Five more in 100 who used the medication quit (10 altogether) compared with the 5 who used a placebo.

But the relative difference between the two groups is that those on the medication did 100% better than those on placebo (10% being twice as much as 5% — or 100% better).  So, if a smoker heard that a particular quit smoking method worked twice as well  (ie: 100% better) they would understandably think that here  was a no brainer: a way of quitting which was very substantially better that a placebo (or a comparative drug).

So in other words with the latest Cochrane data described above, if you are in a RCT (notwithstanding all the serious caveats set out below) and you used a nicotine vape to try and quit, there would be an absolute positive difference of 4 percentage points higher than if you had used NRT or a nicotine free vape, and 6 percentage points higher than if you had no support to quit or (for example) called a quitline for some counselling. 

Flipping this around, this would also mean that a smoker using a nicotine vape would be 90% likely to be still smoking after at least 6 months if they used a nicotine vape; 94% if they used NRT or a nicotine free vape and 96% if they  tried to quit pretty much unaided.

So if you were to only express these percentages in relative improvement  terms, things will sound so much more encouraging! Now, the improvements from nicotine vaping leap to 66% better than NRT or nicotine free vapes (10/100 v 6/100, the 4/100 difference being 4/6 or 66% better), and 150% better than quitting virtually unaided! 

How much more impressive do relative improvements of 66% or 150% sound than the meagre absolute increases of 4 and 6 percentage points?

Veteran vape promoter Alex Wodak had no hesitation in using the heavy ordnance to talk vaping up here when celebrating the release of an earlier Cochrane report:

The Lancet’s policy is that when describing “risk changes or effect sizes … absolute values rather than relative changes” should be used in reports.

Ordinary people typically understand the concept of therapeutic effectiveness as being straightforward. They have many experiences of personally using drugs to alleviate or prevent fever, pain, itching, cramping, nausea, gut problems, contraception, hypertension and more in a very long list.

If they ask a doctor whether a drug being prescribed or recommended “works” they mean works very well: they have high expectations. Few would be keen to use a drug that had a 90% failure rate (as with vapes) when it came to any of these health problems. And they would see trying to quit unaided as well beyond hopeless.

But this is the very low, dismal bar of absolute “success” we find with quit smoking meds, NRT and vaping in RCTs that are widely promoted as the best evidence we can get.

Critical differences between RCTs and real world use

But when RCTs are compared with  “real world” studies of how successful vapes are in smoking cessation, the bad news has barely got out of bed.

In RCTs, it is de rigueur to randomly allocate people to the experimental intervention or to a control group and not tell them which group they have been allocated. Those in the experimental intervention group are assigned to receive the active drug or procedure of interest while those in the control group(s) get an inert substance or placebo (a ‘sugar pill’ or saline solution are common).  Sometimes there is an extra group being compared: a ‘usual care’ group who have the relevant condition (eg: insomnia, anxiety, migraine). This group are asked to simply continue using the drugs or strategies they normally use for the condition in focus.

With double blinding, the aim is to have neither the trial participants nor the trial research staff know which subjects are in either group. Here, independent third parties are charged with group random allocation and those running the trial are kept in the dark about who is in which group until after the trial has concluded and the data are being examined. Those in the ‘usual care’ group of course know that they are not in the experimental or control group because they are not getting anything.

Enhanced or amplified placebo effects

Allocation is blinded to avoid staff inadvertently letting the cat out of the bag and telling subjects that they are getting the active drug or procedure or the placebo.

This is of critical importance because this knowledge can powerfully influence the expectations of those in a trial. Knowing you are being asked to take a dummy pill (or ointment, patch, lozenge, spray etc) or an established treatment that has been around for decades is understandably likely to deflate expectations of experiencing a positive outcome. This may increase the likelihood of people dropping out of the study (“I know I’m not getting the real thing here, so why bother?”) or adopting other strategies to reduce the problem that inspired them to join the trial to fix the problem. Equally, knowing you really are on the active ingredient intervention may raise your expectations, mirroring those that occur in real world use of a drug.

The asymmetry between knowing you are getting the active drug and knowing you are not may well artificially widen the gap between the performance of the two groups, flattering the performance of the active intervention or drug. This is because of what is termed the enhanced or amplified placebo effect. Belief about whether you are taking an active drug or placebo has been shown to have a substantial impact on therapeutic outcome, with stronger positive expectations resulting in greater therapeutic improvement. This is especially the case when it comes to psychiatric outcomes.

Ideally, those allocated to both intervention and control (placebo) groups should have identical toss-of-a-coin probability of guessing to which group they have been allocated. But many RCTs don’t even test for this “blindness integrity” and when they do, those using the active drug guess correctly far more than by chance,

A 2004 study of 73 nicotine replacement therapy (NRT) trials found only 17 tested for blindness integrity and that in 12 of these 17 subjects (71%) accurately judged treatment assignment at a rate significantly above chance. In a varenicline RCT, those in the varenicline arm were >3 times as likely to believe they were taking varenicline, versus “not sure” and the belief that one had received varenicline was significantly associated with a 5x increase in cessation rate. There’s a similar story with bupropion: Compared to those who guessed “not sure” those guessing they were taking bupropion were more than twice as likely to have been randomized to bupropion.

With nicotine-delivery products (including NRT and nicotine vaping) those with nicotine dependence are thoroughly familiar with the physiological cues they get many times every day when their brain signals to them “feed me, feed me more nicotine please!” So if they have been allocated to placebo group and are still getting those strong sensations, it’s little wonder they often guess they are not getting active nicotine in the trial.

How did the latest Cochrane review handle the issue of blindness integrity?

The Cochrane authors judged all 9 studies in their e-cigarette vs NRT cessation comparison (Analysis 1.1) to be at low risk for “Blinding of participants and personnel (performance bias)”.

Here’s how they get there. The authors say that for randomised studies without blinding, they still rated this domain low risk “if the intervention was compared to an active control of similar intensity”, because they “judged performance bias to be unlikely in this circumstance.” No elaboration of their reasoning was given here.

The lack-of-blinding problem was basically waved through on the assumption that an active NRT comparator neutralises performance bias. But in a field where it is common for trial authors to have spent years publicly and forcefully barracking for e-cigarettes, trial staff enthusiasm and investigator attitudes can shape encouragement, troubleshooting, adherence support, follow-up energy, and participant expectations. So the low-risk performance bias rating for all 9 studies is surely highly questionable.

Combating trial drop out

When RCTs are being planned, calculations should always be made to ensure that sufficient  numbers of eligible participants are enrolled to enable enough statistical power to be available to reliably point to any meaningful differences in quit rates. Those running trials must also plan for inevitable dropouts who start a trial but who do not complete it for whatever reasons.

Trial managers are acutely sensitive to the serious threats to scientific integrity, budget blowout and delayed trial completion  posed by significant participant drop out. Accordingly both formal ‘tried and tested’ protocols and informal practices are involved in trying to minimise participant attrition. Trial managers use retention strategies which centre around building good relationships with participants to motivate them to remain in trials and maintain their data records. Staff qualities seen as influencing retention include “empathy, being approachable, having a good manner, making trial participants feel appreciated, as well as staff who chat, listen, and answer participant queries, and go ‘the extra mile” … “Having face-to-face contact, or regular follow-up calls with participants, is felt to facilitate building good rapport … Less participant contact possibly hinders retention, as participants do not have a relationship built with trial staff and consequently may not feel as obliged to complete follow-up”

The phenomenon of “research participation effects” (RPE) has largely replaced investigation of issues previously conceptualised as the Hawthorne effect. These constructs are acknowledged as introducing factors which can add important motivations for trial participants to want to please the researchers with whom they have formed positive relationships. Trials often include staff who are PhD candidates who are highly motivated to ensure good trial retention so that their PhD research papers cross that elementary standard of trial integrity and reduce the risk of rejection by journal editors.

Walsh’s 2008 review of 12 smoking cessation trials found subjects were contacted on average 7.1 times across these trials. This level of on-going contact with people who are reminding you and willing you on to keep using a drug for at least the minimum recommended time does not occur when a smoker buys a course of NRT from a supermarket or pharmacy, picks up a script for varenicline, bupropion or cytisine from a doctor or buys a vape.

And here’s the crunch. A review of 7,521  participants in 16 NRT cessation studies found 61% of those in RCTs adhered to their full course of NRT compared with only 26% doing so in population studies. Only 20% of participants in an Australian bupropion trial had completed their full course when followed up. These are radically different levels of adherence, demanding scepticism when RCT results are seamlessly extrapolated to real world relevance.

Smoking cessation treatments are free to trial participants

Again, unlike in real world use, trial participants are provided with free drugs, NRT or vapes. Neither do they pay doctors’ consultation fees where these are relevant. These savings may be particularly attractive to those on low incomes and may positively bias their volunteering in trials. With illegal vapes in Sydney today costing $70-90 for rechargeable products and $50-60 for non-rechargeables depending of the number of puffs obtainable, free vapes will motivate some smokers but represent an unrealistic factor.

RCT subjects are often paid

And again, unlike real world purchasers of quit smoking products, subjects in RCTs are almost always compensated for any expenses incurred in their participation. These includetravel expenses, parking, meals during study visits, accommodation if required and compensation from time away from work. “General inconvenience” payments are also common. Research ethics committees generally will not approve payments that might inordinately influence trial participation “just for the money”. But again, those on low incomes may be attracted to trials and more motivated to see them through even where “general inconvenience” honoraria may be only a few hundred dollars.

Eligibility criteria Ideally, those enrolled in trials should mirror as closely as possible the characteristics of smokers in the population. Some of these are very well established with over-representation in smokers of social disadvantage (low income, education) and higher mental health problem rates than in the general community. This data linkage study of a sample of 1.5m Danish people (1995-2018) found 80% of the population had a history of treatment for a mental health disorder in a hospital setting, from general practitioners or private psychiatrists.

---

But those running trials prune out participants prior to trial commencement who can be expected to have unacceptable likelihoods of dropping out of the trial, fail to follow the expected procedures and contacts with the researchers or who live in situations where they are hard to contact.

In this review of exclusions from smoking cessation trials, 66% of those with nicotine dependence were excluded and 59% of those motivated to quit were rejected. In the table below the two largest criteria were smoking less than 10 cigarettes a day and insufficient motivation to quit. This is in spite of recommendations for light smokers to use medications and efforts to promote medication use in unmotivated smokers as part of pre-quittng or “preparation to quit” interventions, developments often promoted by pharmaceutical companies.

This US study found 36% of 1206 smokers screened for smoking cessation trial were deemed ineligible for reasons that included serious mental illness, alcohol and drug issues. While 24% of Caucasians were ineligible, 42% of African Americans were excluded.

A review of reviews on 279 RCTs of smoking cessation using pharmacotherapies found 18.3% explicitly excluded participants with any MHDs (mental health disorders), 54.5% conditionally excluded based on certain MHD criteria and 27.2% provided insufficient information to ascertain either inclusion or exclusion.  The authors concluded that “smokers with MHDs are not sufficiently represented in RCTs examining the safety and effectiveness of smoking cessation medications.”

In summary, when we add the widespread culling of those less likely to fully participate throughout a smoking cessation trial to the known radically lower levels of treatment adherence as described above, we are looking at serious differences between those participating in trials and those independently using quit smoking aids in real world, totally unsupervised contexts.

Collectively, these differences are likely to significantly flatter the communicated success rates for vaping (and any quit smoking med) promoted from RCTs compared with real world outcomes.

Defenders of RCTs here counter that because trial outcomes are typically calculated using intention-to-treat (ITT) analysis, that RCT outcomes always take attrition into account. ITT analysis involves considering the outcomes of all participants “according to their original assigned treatment group, regardless of whether they actually received the treatment, or dropped out”.

But as discussed above, this does not consider the intense efforts undertaken in RCTs to retain trial participants and to initially screen them to preclude those with known propensity to drop out or with other factors known to be associated with poot outcomes. These collectively introduce important biases into RCTs that make those participating very different and have very different experiences to random cross-sections of smokers trying to quit in the real world.

Longitudinal cohort and population studies

My 2023 (open access) book Quit Smoking Weapons of Mass Distraction (see from pp 185-198) summarises  longitudinal population cohort studies on the success of vaping in quitting. It looks also at the question of whether nations with widespread, open-slather vape access have lower smoking prevalence than those where vapes are more restricted or even banned.

Longitudinal cohorts show a similar desultory performance to RCTs, with by far the most common experience of vaping by smokers  over  several years of follow-up being continuing smoking and vaping (dual use) and abandonment of vaping while smoking continues (see the above pages my book for details).

And when it comes to asking whether nations with now over a decade of widespread liberal access to vapes have plummeting smoking prevalence compared with those with stricter controls, the answer is similarly mostly negative. On this, our recent piece in Addiction shows that with the exception of New Zealand, there is very little difference in smoking prevalence between the five nations shown below nor any obvious relationship between the prevalence of vaping and that of smoking.

Moreover, globally, there are 45 nations which have smoking prevalence under 10%, with most of these nations having negligible vaping.

“A few quit smoking on their own. Good. But most don’t”

Earlier, I noted the denigration of unassisted quitting as the low water mark of useful ways of quitting. The Australian Department of Health notes that “While quitting cold turkey may not work for everybody, many smokers quit smoking this way.” This is quite an understatement (see here). Wodak’s statement is flagrantly, stonkingly incorrect and suggests he has little understanding of population-wide smoking cessation.

The reputation of unassisted quitting (cold turkey) for having a poor success rate derives from a research obsession with evaluating  the outcome of individual quit attempts, rather than taking a population perspective on how large numbers of former smokers stopped smoking on their final, successful attempt.

If we ask “which method of quitting has the best success rate per quitting attempt?” common assisted quitting methods perform better than unassisted attempts. But if we instead ask, “For every 1000 former smokers, what method of quitting has produced the most quitting numbers?” unassisted cessation always wins the prize. What this obsession with success rates rather than with success numbers camouflages is that population attributable quit volumes are a product of success rates multiplied by the number of smokers using various methods. And here, there is daylight between smoking trying to quit without assistance and any (and sometimes all combined) other quit strategy.

Other blogs in this series:

Vaping theology: 1 The Cancer Council Australia takes huge donations from
cigarette retailers. WordPress  30 Jul, 2020

Vaping theology: 2 Tobacco control advocates help Big Tobacco. WordPress 12 Aug, 2020

Vaping theology: 3 Australia’s prescribed vaping model “privileges” Big Tobacco WordPress Feb 15, 2020

Vaping theology: 4 Many in tobacco control do not support open access to vapes because they are just protecting their jobs. WordPress 27 Feb 2021

Vaping theology: 5 I take money from China and Bloomberg to conduct bogus studies. WordPress 6 Mar, 2021

Vaping theology: 6 There’s nicotine in potatoes and tomatoes so should we restrict or ban them too? WordPress 9 Mar, 2021

Vaping theology: 7 Vaping prohibitionists have been punished, hurt, suffered and damaged by Big Tobacco WordPress 2 Jun, 2021

Vaping theology: 8 I hide behind troll account. WordPress 29 Jun, 2021

Vaping theology: 9 “Won’t somebody please think of the children”. WordPress 6 Sep, 2021

Vaping theology: 10: Almost all young people who vape regularly are already smokers before they tried vaping. WordPress 10 Sep, 2021

Vaping theology: 11 The sky is about to fall in as nicotine vaping starts to require a prescription in Australia. WordPress 28 Sep, 2021

Vaping theology: 12 Nicotine is not very addictive WordPress 3 Jan 2022

Vaping theology 13: Kids who try vaping and then start smoking,would have started smoking regardless. WordPress 20 Jan, 2023

Vaping theology 14: Policies that strictly regulate vaping will drive huge
numbers of vapers back to smoking, causing many deaths. WordPress 13 Feb, 2023

Vaping theology 15: The government’s prescription vape access scheme has failed, so let’s regulate and reward illegal sellers for what they’ve been doing. WordPress 27 Mar 2023

Vaping theology 16: “Humans are not rats, so everybody calm down about nicotine being harmful to teenage brains”. WordPress 13 Jul, 2023

Vaping theology 17: “Vaping advocates need to be civil, polite and respectful” … oh wait. WordPress 3 Oct, 2023

Vaping theology 18: Vaping is a fatally disruptive “Kodak moment” for smoking. WordPress Oct 30, 2023

Vaping theology 19: Vaping explosions are rare and those who mention them are hypocrites. WordPress 17 Nov, 2023

Vaping theology 20 : Today’s smokers are hard core nicotine dependent who’ve tried everything and failed – so they need vapes. WordPress 14 Dec, 2023

Vaping theology 21: Australia’s prescription vapes policy failed and saw rises in underage vaping and smoking. WordPress 10 Jan, 2024

Vaping theology 22: “Prohibition has never worked at any point for any other illicit substance”. WordPress 17 Mar 2024

Vaping theology 23: “84% of the Australian public are opposed to the way the government will regulate vapes” WordPress 2 Apr, 2024

Vaping Theology 24: “Tobacco control advocates are responsible for vape retail store fire bombings and murders. WordPress 27 May, 2024

Vaping theology 25: Vaping is as harmless as breathing in stream. So everyone relax. 10 tenets of vaping harm denial.WordPress 10 Mar, 2026

Vaping theology 26: “If Australia allowed vapes to be sold openly, this would lower smoking prevalence and kill illegal tobacco stone dead” WordPress 5 Jun,2026