Www.cfpnet.ca
MANITOBA PHARMACIST INITIATED SMOKING CESSATION PILOT PROJECT May 18, 2015Authors: Shawn Bugden, BSc (Pharm), MSc, PharmDKevin Hamilton BSP, MSc.Brenna Shearer PhD, O.T. Reg. (MB)Kevin Friesen BSc (Pharm)Silvia Alessi-Severini PhDReport Submitted to:Manitoba Health, Healthy Living and SeniorsCanadian Foundation for PharmacyNeighborhood Pharmacy Association of CanadaMANITOBA PHARMACIST INITIATED SMOKING CESSATION PILOT PROJECTAcknowledgmentsWe wish to thank the pharmacists whose participation and dedication made this project possible. This pilot project would not have been possible without the support of the following partners and their ernment of ManitobaCanadian Foundation for PharmacyPharmacists ManitobaCollege of Pharmacists of ManitobaUniversity of Manitoba, College of Pharmacy, Faculty of Health SciencesManitoba Tobacco Reduction AllianceNeighborhood Pharmacies of CanadaTable of ContentsList of Tables and Figures Pg 4Executive Summary Pg 6Introduction Pg 8Statement of IntentPg 9MethodologyPg 10FundingPg 10Pharmacist RecruitmentPg 10Target PopulationPg 11Participant RecruitmentPg 12Smoking Cessation Assessment Forms Pg 12Participant ConsentPg 13Smoking Cessation Pilot Project Components Pg 13EvaluationPg 15Evaluation MethodsPg 15ResultsPg 16Study PopulationPg 16Smoking Cessation ResultsPg 18Product ChoicePg 23Geographic SettingPg 24Resource and Cost AnalysisPg 25Medication CostPg 25Pharmacist TimePg 27Counselling CostPg 30Total Costs / Total Benefits Pg 30Cost EffectivenessPg 30DiscussionPg 31ConclusionPg 34ReferencesPg 35Appendix A. Project Stakeholders and Steering CommitteePg 39Appendix B. Orientation and Smoking Cessation FormsPg 41Appendix C. Consent FormPg 62List of TablesTable 1 Comparison of publicly funded smoking cessation programs across CanadaTable 2 Smoking cessation timelines and assessment formsTable 3Baseline patient characteristicsTable 4Mean and median days abstinent from cigarette smokingTable 5Self-rated improvements in additional health outcomesTable 6Criteria for interpreting ICERsList of Figures Figure 1Fagerstrom test for nicotine dependenceFigure 2Kaplan Meier curve of the probability of successful abstinenceFigure 3 Kaplan Meier survival curve using last observation carried forward methodologyFigure 4Kaplan Meier curve using longest quit attempt at any time throughout the pilot projectFigure 5Plot comparing the number of days a patient remained within the pilot project and the number of years as a smoker excluding those who dropped out prior to their scheduled quit dateFigure 6Proportion of cessation aids usedFigure 7KM curve stratified by smoking cessation aidFigure 8Box Plot of medication costs for all patients enrolled in the pilot projectFigure 9Box Plot of medication costs for all patients using medicationFigure 10 Medication costs for smoking cessation project Figure 11 Box plot showing a breakdown of the time spent with the pharmacistFigure 12Number of patients who spend an average of < 1 hour, 1-3 hours, 3-5 hours, 5-7 hours, or >7 hours with the pharmacistFigure 13Average amount of time spent with the patient at each visit in minutesExecutive SummaryAlthough many factors affect a patient’s quality of life, smoking cessation can arguably provide the single greatest impact in both long and short term health benefits. Unfortunately, tobacco is highly addictive and relapse is common. Effective, individualized strategies are required to improve the likelihood of successful abstinence.This report focuses on the impact of a community pharmacist operated smoking cessation pilot project.?Under the recently proclaimed Pharmaceutical Act, Manitoba pharmacists with additional training who have received authorization from the College of Pharmacists of Manitoba can prescribe smoking cessation medications. This project was developed in an attempt to enhance the utilization of pharmacists in preventative health care delivery. The feasibility, impact and cost effectiveness of this pilot project was evaluated in this report.MethodologyAll Manitoba residents over the age of 18 who were covered under Manitoba Employment and Income Assistance were eligible for enrolment. Patients were enrolled into the project beginning January 2014. A total of 12 pharmacies participated in the project. Pharmacists who completed an approved smoking cessation training program enrolled patients interested in quitting smoking. Administrative and clerical support was provided by pharmacy assistants.Following enrollment, patients were asked to take home and complete smoking assessment forms and to log every cigarette consumed within a 24 hour period. The pharmacist assessed the patient’s responses and set a quit date during the initial counselling session. At that time, a decision on the best use of smoking cessation aids was made between the pharmacist and the patient. The available options included: nicotine replacement therapy (patch, gum, lozenge and inhaler), varenicline, or bupropion.Follow-up visits were scheduled on the quit date and 1 week later, then 1, 3 and 6 months following the quit date. In addition to the counselling sessions, the pharmacist could provide medications at no charge to the patients. Pharmacists were provided a professional fee of $300 per participant recruited. Patients did not receive any monetary compensation for participation in the project.ResultsOf the 119 patients enrolled in the project, 2 successfully quit smoking and 41 patients reduced the amount they were smoking by an average of 16 cigarettes per day. Improvements in cough, shortness of breath, phlegm production or cold extremities were also reported in 63% of the patients. Pharmacists spent an average of 2.5 hours counselling each patient during the pilot project. This project was shown to be a cost effective method for smoking cessation. The total cost of the project for smoking cessation products and professional counselling was $55 950.86 or an average of $470.18 per patient. The estimated cost effectiveness in quality adjusted life years (QALY) gained was calculated between $4239 and $8252 per QALY. Additionally, those patients that reduced the amount of smoking can be expected to benefit from an average of $266 per month in savings. This can represent a significant increase in disposable income for this population. ConclusionsThis report provides encouraging evidence for a viable community pharmacist-run smoking cessation program. Quit rates similar to other programs was demonstrated in Manitobans who were of low socioeconomic status. With minor modifications, this project could be expanded throughout the province to provide an additional access point for patients to receive effective smoking cessation counselling. With more than 400 pharmacies in Manitoba, this represents a considerable existing infrastructure to support a smoking cessation program. Successful abstinence from smoking can be a long journey with multiple relapses. This project reinforced that smoking cessation was difficult, especially in this hard to reach population. However, these difficulties highlight the importance of effective strategies to support patients through multiple quit attempts and the significant clinical service that pharmacists offer to expand the number of access points and level of individualized support.MANITOBA PHARMACIST INITIATED SMOKING CESSATION PILOT PROJECTIntroductionSmoking is a leading cause of death. Smoking and tobacco use is responsible for more than 50% of lung, respiratory, and oral cancers and a substantial factor in cardiovascular disease.1 While smoking at young ages leads to lifelong habits, smoking is a changeable lifestyle behavior.1 Smoking rates in Manitoba have decreased in the past decade; however 1 in 5 Manitobans (20.5%) are smokers.1,2 Slightly more males (23.3%) than females (17.9%) smoke in Manitoba2 and 7% of Manitoba seniors smoke daily.1 The range of smoking rates range from 14.3% in British Columbia to 21.1% in Saskatchewan.2 While decreases in smoking rates have been achieved in the past decade, Manitoba exceeds the national average of 17.3%, ranking the province with the third highest smoking prevalence.3 Manitoba’s current smoking rates warrant continued efforts to reduce smoking in youth and adults.1,4 Not only are the smoking rates higher in Manitoba, but this rate increases in individuals with an annual income less than $20,000.5 It is estimated that 33% of lower socioeconomic status Canadians are smokers. The recently published report from the Centers for Disease Control and Prevention, Best Practices for Comprehensive Tobacco Control Programs,6 recommend that smoking cessation programs should be targeted to populations that lack access to these services, such as the uninsured or the underinsured. This population may benefit to a greater degree compared to the average population, yet they cannot afford the upfront costs associated with these types of programs.In 2010, 60% of smokers considered quitting and 10% of those who reported attempting to quit were successful.2 Between 1999 and 2010 in Canada, the percentage of people who smoked and were seriously considering quitting increased among those 25 years of age and older.2 The most common forms of smoking cessation supports used by those who attempted to quit smoking included nicotine replacement therapy (39%), friend or family member co-quitting (25%), and help telephone line or office program (15%).2 Reducing the number of cigarettes smoked was identified as the most common smoking cessation method (69%).2 While these self-reported results are promising, program evaluation outcomes on smoking cessation behaviors are limited or lacking.7 The Government of Manitoba is committed to reducing the use of tobacco. Legislation that banned the supply of tobacco products in pharmacies and health-care facilities was a significant strategy to reduce the number of smokers and restrict access to youth.8 The pharmacist is one of the most knowledgeable health care professionals in regards to smoking cessation products and strategies; knowledge achieved through advanced education, training and certification.9-11 Pharmacists are the most accessible health care professional.9,10 Community pharmacists provide a range of clinical services and expertise and practice in neighborhoods, towns and cities across the province.10 The role of the pharmacist in smoking cessation strategies is an underutilized resource in Manitoba. Smoking cessation programs in Alberta, Ontario, Newfoundland, and Saskatchewan utilize pharmacists as the point of access to counsel, offer medication options, and provide follow up to those who want to stop smoking.7, 11-16 Most provinces across the country provide coverage for prescription medications such as bupropion and varenicline through their drug plans according to a variety of programs and reimbursement schedules. Some jurisdictions (e.g., BC, Quebec, PEI) also provide coverage for over-the-counter (OTC) nicotine replacement products (gums and patches).The role of the pharmacist can best be described as underutilized and often an under recognized resource in smoking cessation programs. For individuals who smoke, many find smoking cessation overwhelming and difficult without assistance, including smoking cessation drugs and ongoing support and counselling. Many pharmacists have received additional knowledge and training specifically for initiating and monitoring of smoking cessation for individual clients. During the fall of 2012, discussions ensued with the Minister of Healthy Living, Seniors and Consumer Affairs, Government of Manitoba. The intent was to open discussion on the role of the pharmacist for healthier communities and specifically pharmacist initiated smoking cessation.The College of Pharmacists of Manitoba, Pharmacists Manitoba, and College of Pharmacy, Faculty of Health Sciences, University of Manitoba developed a proposal for a Manitoba pharmacist initiated smoking cessation pilot project. In 2013, the project proposal was approved and financial support was provided by the Department of Healthy Living and Seniors, Government of Manitoba, Canadian Foundation for Pharmacy and the Neighborhood Pharmacy Association of Canada. A project Steering Committee was established to oversee the project operations and deliverables. The committee membership included representation from the Government of Manitoba, College of Pharmacists of Manitoba, Pharmacists Manitoba, University of Manitoba College of Pharmacy, and community stakeholders and pharmacists. A full list of stakeholders is listed in Appendix A. Ethics approval for the project was granted by the University of Manitoba Health Research Ethics Board on January 6, 2014. Statement of IntentSmoking cessation has been established as a funded service in a number of Canadian provinces (Table 1). Manitoba is the only Western province that does not fund this service. The goal and objective of the pilot project was to establish feasibility, impact and cost effectiveness of a pharmacist initiated smoking cessation project in Manitoba. The purpose of the pilot project was to enhance the role of the pharmacist in preventative health-care delivery as part of the primary health care team and to reduce the number of Manitobans that smoke. The primary outcome of the pilot project was the self-reported smoking quit rate and smoking reduction rate at 6-months.Objectives of the project:Establish a provincial pilot pharmacist initiated smoking cessation project Reduce the number of low-income Manitobans who smokeEnhance utilization of the pharmacist in preventative health care deliveryTable 1. Comparison of publicly funded smoking cessation programs across Canada15,16Pharmacy ServiceBCABSKONQCNBNSPEINFMBSmoking Cessation3xDispensing Fee + Mark UpIncluded in Medication Management$2.00 per minute to max $300.00$125.00$52.50MethodologyFundingThe pilot project received funds from the Government of Manitoba ($90,000), Canadian Foundation for Pharmacy ($50,000) and Neighborhood Pharmacy Association of Canada ($19,000). In addition, some smoking cessation product costs were covered as insured benefits through the Employment and Income Assistance Program. The total pilot project cost was $159,000. Considerable in-kind support was also provided by the College of Pharmacists of Manitoba, the College of Pharmacy, University of Manitoba and Pharmacists Manitoba. Pharmacist Recruitment Manitoba pharmacists were recruited through email notification by the College of Pharmacists of Manitoba and Pharmacists Manitoba. All Manitoba pharmacists were invited to complete an application form for consideration in the pilot project. Each application form consisted of a letter of introduction to explain the project and application process, a pharmacist manager application form, and pharmacist application form(s). All applicants were required to demonstrate completion of a recognized smoking cessation training program, prior experience providing smoking cessation counselling and services as well as knowledge and access to the pilot project target population. A subcommittee of representatives from the Steering Committee confidentially reviewed all application forms for completeness and selected 15 pharmacies to participate in the pilot project. All successful pharmacy/pharmacists received an orientation (Content: Appendix B) to the project on Tuesday, January 7, 2014.The following were the pharmacies accepted to participate in the project:Tache Pharmacy, 400 Tache Avenue, WinnipegLoblaw Pharmacy 1512, 1035 Gateway Road, WinnipegDauphin Clinic Pharmacy, 622 – 3rd Street SW, DauphinWest-Man Medical Centre Pharmacy, 146 – 6th Street, BrandonPharmacie Dufresne, 10 – 1321 Dawson Road, PO Box 229, LorretteShoppers Drug Mart 547, 43 Marion Street, WinnipegReavie’s Pharmacy, 243 Main Street North, RussellSafeway Pharmacy, 1612 Ness Avenue, WinnipegLoblaw Pharmacy 1509, 80 Bison Drive, WinnipegShoppers Drug Mart 546, 230 Main Street, SelkirkLoblaw Pharmacy 1506, 1578 Regent Avenue, WinnipegShopper’s Drug Mart 537, 2211 Pembina Highway, WinnipegShopper’s Drug Mart 2421, 1017 McPhillips Street, WinnipegLoblaw Pharmacy 1505, 2132 McPhillips Street, WinnipegShopper’s Drug Mart 2422, 777 Sherbrook Street, WinnipegTarget Population The pilot project target population was Manitoba residents currently active with Employment and Income Assistance, at least 18 years of age or older, and interested in reducing or quitting smoking. The pilot project goal was to recruit 100 participants within the first six weeks following the pharmacist orientation program in January 2014. Participant Recruitment Pharmacists actively recruited participants from within the pharmacies they were employed. Posters were provided to the pharmacies to provide awareness of the project, eligibility criteria, and promotion of a sponsored smoking cessation project. Potential participants who met the eligibility criteria either self-identified or were approached by the pharmacist. Potential participants were provided with an information sheet and consent form. Pharmacies identified staff members who were not involved in the pilot project as resources for potential participants to answer any questions and ensure understanding of the pilot project and participant and provider responsibilities. An initial goal to recruit 100 participants into the pilot project became difficult to achieve within the original 6 week time frame. The participant recruitment deadline was extended for an additional three weeks. In total, project recruitment occurred over an eight-week time frame, from January 7, 2014 through to February 28, 2014. Client participants received the following services that were covered by either Employment and Income Program assistance and/or the pilot project funding:smoking cessation products (prescription medication and OTC products)pharmacist support and counsellingSmoking Cessation Assessment FormsThe Smoking Cessation Pilot Project Assessment Forms were adapted from existing pharmacist-guided smoking cessation project forms. The following is a list of assessment forms: Quit LogSmoking AssessmentFagerstrom Tolerance TestWhy TestPatient Information FormSmoking Cessation Assessment FormQuit DiaryWhy Do I Want to Quit?Pre-Quit PlanningMy Reason to QuitSamples of all forms are available in Appendix B. Participant ConsentAll client participants completed an informed consent (Appendix C). The informing of the consent was conducted by other pharmacy staff rather than the pharmacists directly involved in the project. All signed consent forms were given a study number and secured in a locked cabinet to ensure the adherence to the Manitoba Personal Health Information Act. The consent forms were forwarded along with all documentation to the University of Manitoba for evaluative purposes. Anonymity was carefully adhered to.Each pharmacy securely and confidentially assigned each participant recruited into the pilot project a study number. Only the study number was used on each of the evaluation forms. The linkage between the study number and the participant’s clinical information was maintained by the pharmacy in a secure and locked system. Consent forms with the linkage between participant identifiable information and the study number were forwarded to the Principle Investigator at the College of Pharmacy, Faculty of Health Sciences, University of Manitoba by Canada Post Express post and maintained as per standard confidentiality protocols.Smoking Cessation Pilot Project ComponentsEach participant received smoking cessation products and counselling for a three month time period. All costs related to smoking cessation over-the-counter and prescription products were covered by the pilot project. All costs related to compensation for pharmacist professional counselling services were covered by the pilot project. Each interaction with the pharmacist coincided with specific time frames for collection of information regarding self-reported smoking behavior changes, health changes, concerns or areas of stress, and review of supports provided and available (Table 2).Participants were supported in the completion of the forms if required or provided the documents to complete at home to the best of their abilities. Pharmacists provided in person and telephone contact and consultation for the duration of the three months of smoking cessation support. Initially, a Pre-Quit Log was completed and returned to the pharmacist. A subsequent appointment/interview was established to assess a participant’s readiness to quit smoking and the Pre-Quit planning was completed. In consultation with their pharmacist, each participant was provided with the best method and/or medication to support their quit smoking plan. Once a quit date was established for the participant a Quit Day appointment was made with the participating pharmacist.Table 2. Smoking Cessation Timelines and Assessment Forms. Forms available in Appendix BVisitFormsRecruitmentInformed ConsentPre-Quit LogSmoking AssessmentWhy TestPatient InformationFagerstrom Tolerance ScaleAssessmentSmoking Cessation AssessmentHow Do I Want to QuitWhy Do I Want to QuitPre-Quit PlanningWhen Do I Want to QuitQuit DaySmoking Cessation AssessmentQuit Day/Week PlanQuit Diary1-Week Follow-upSmoking Cessation Assessment1-Month Follow-upSmoking Cessation Assessment3-Month Follow-upSmoking Cessation Assessment6-Month Follow-upSmoking Cessation AssessmentAt the Quit Day appointment the participant received a Quit-Day/Week Plan, Quit Diary and other supportive information. Each participant received a one week post quit follow-up, a one month post quit follow-up, and had a three month post quit follow-up appointment with their participating pharmacist. The counselling sessions provided for support, encouragement, and identification of challenges and ways to address these. And, equally as important, this provided an opportunity to celebrate the client’s success. Prescriptions were obtained from the participant’s physician when required. The participants received ongoing supportive counselling on the appropriate time to start taking their medication as well as ongoing medication management advice to enhance adherence to the products and adaptation to each participant’s lifestyle and needs.Smoking cessation products and professional counselling was provided for three months at no cost to the participants. Six months following the initial Quit Day session established between the participant and pharmacist, the pharmacist conducted a follow up evaluation to determine each participant’s smoking cessation or smoking reduction status. Participants who completed consent forms for the pilot project were considered to be recruited for the project. Pharmacists were required to continue to follow up with each client throughout the three month smoking cessation project as well as for the six month follow up. Clients who ceased to participate in the project at any stage of the pilot project were contacted or attempted to be contacted by their pharmacist during the duration of the pilot project. EvaluationEvaluation MethodsThe project evaluation was conducted at the University of Manitoba, College of Pharmacy, Faculty of Health Sciences. It included statistical analysis of quit smoking readiness, smoking history, attitudes toward smoking cessation products, smoking quit rate at three and six month intervals, attitudes toward pharmacist interventions, and overall behavioral factors impacting success or failure of smoking cessation initiatives. In addition, an assessment of the cost effectiveness of the project was conducted, which can provide the basis for a budget impact analysis of a potential province-wide program. All data presented was anonymized and aggregated. Demographic data were gathered and expressed numerically and as a percentage. A Kaplan Meier (KM) survival analysis was performed to evaluate continued abstinence rates throughout the pilot project for the primary outcome as well as the longest quit time allowing participants to start and stop smoking. The KM analysis was also stratified by treatment received. Subjects were censored upon leaving the pilot project on the date of last reporting. Subjects were marked as failures on the date of first reporting having smoked in the preceding period. Univariate analysis with Pearson, point-biserial correlation, χ2 and for non-parametric data, Spearman and Kendall’s tau-b test was used to identify significant variables. ResultsStudy PopulationThe baseline data of the population study are captured in Table 3. The average age of these patients was 43 years with 49% being female. The majority of patients enrolled were located within urban centres (66%). This breakdown is representative of the overall population of Manitoba (72% urban, 28% rural).17Many characteristics of the studied population are associated with a high risk of relapse. The Fagerstrom Test for Nicotine Dependence is a method for assessing nicotine dependence (Figure 1). Figure 1. Fagerstrom Test for Nicotine Dependence 121602530162500003016250 Score1-2= low dependence5-7= high dependence3-4= moderate dependence8+= very high dependence00 Score1-2= low dependence5-7= high dependence3-4= moderate dependence8+= very high dependenceThe information necessary to complete the Fagerstrom Test was collected from all pilot project participants, whom displayed an average value of 6.1, which is on the upper end of the scale with a high dependence on nicotine. They were long-term smokers (26 years) and smoked approximately a package per day on average. Mental disorders requiring the use of medications was observed in 57 (48%) of the participants. Patients drank an average coffee equivalence of 5 cups per day (taking into account caffeine from tea and cola), 37% drank at least 1 alcoholic Table 3. Baseline patient characteristicsDemographic DataAge (sd)43 (13)Sex (% female)49%Place of residence (% urban)66%Years smoking (sd)26 (13)Cigarettes per day1-1018 (20%)11-2033 (36%)21-3033 (36%)31-507 (8%)Average (sd)19 (8.8)Other smokers in the household (% yes)34%Average prior quit attempts4Average coffee equivalents5 cups per dayAlcohol consumptionNone53 (61%)1-2 drinks per day29 (33%)>2 drinks per day5 (6%)Fagerstrom score6.1Participation in an exercise programNo exercise43 (47%)Occasional37 (41%)Regular11 (12%)Pregnant or breast feeding4 (3%)Recreational drug useNone62 (75%)Occasional15 (18%)Regular6 (7%)beverage per day, and 25% used illicit drugs. Combined with the required lower socioeconomic status, this population would be expected to be hard to reach and difficult to support in the smoking cessation process. Smoking Cessation ResultsOf the 119 patients who expressed interest in quitting and signed the consent form, 45 (38%) dropped out prior to their scheduled quit date. By the pilot project close out, 98 (82%) patients had dropped out, 19 (16%) remained as smokers, and 2 (1.7%) had abstained from smoking for at least 6 months. The proportion of patients who remained abstinent for 1 and 3 months were 30% and 19% respectively (Figure 2). The mean number of days abstinent was 46 with a median of 9 days (Table 4).Figure 2. Kaplan Meier curve of the probability of successful abstinenceTable 4. Mean and median days abstinent from cigarette smokingMeanMedianEstimate95% CIEstimate95% CI46.3(26.2,66.5)9(4.9,13.1)Long term smoking was common in this population with an average history of 26 years and the longer a patient smoked, the less likely they were to successfully quit (Figure 4; r2 = -0.318, p = 0.012).The assumption used in this analysis is that individuals that were lost to follow-up were no longer considered in the follow-up. They were “censored” at the time of loss to follow-up. Other smoking cessation projects have used a different analysis technique called last observation carried forward (LOCF). In this type of analysis the last known observation of the patient is carried forward to the end of the study. Patients that have quit and are lost to follow-up are assumed to have continued to abstain from smoking. This is not a credible assumption and results in inaccurate and inflated quit rates. If LOCF is used to analyze this project the resulting quit rate is 21% (Figure 3). Figure 3. Kaplan Meier survival curve using last observation carried forward methodologyWhile only a small number of patients were able to fully discontinue smoking, a much larger number of patients were able to decrease harm by reducing the number of cigarettes they smoked. Using the data at the last available follow-up appointment, 41 patients were able to reduce the amount of cigarettes consumed per day by an average of 16, while 4 patients restarted smoking more than they were previously.This reduction in smoking seems to have produced some improvements in health. Improvements in areas such as reduced coughing, shortness of breath, phlegm production and cold extremities, were observed even in those who did not quit. Participants were asked to rate these symptoms at each appointment on a scale of 0 to 3, where 0 corresponds to no symptoms and 3 is the worst ever. The majority of patients (63%) reported improvement in at least one of these areas, with reductions in phlegm and cough being the most common (Table 5). Table 5. Self-rated improvements in additional health outcomesChange in symptom scaleCoughShortness of breath (at rest)Shortness of breath (during exertion)PhlegmCold extremities1 point18 (24%)12 (16%)14 (19%)20 (27%)10 (13%)2 points6 (8%)3 (4%)9 (12%)5 (7%)4 (5%) 3 points1 (1%)1 (1%)2 (3%)01 (1%)Nicotine is considered one of the most addictive substances and many patients attempt to quit numerous times. On average, patients tried to quit 4 times prior to entering this pilot project. Some tried to quit multiple times during the project. When taking into account the longest quit attempt during the project, the 1 and 3 month abstinence rates were 38% and 26% respectively (Figure 4). There was a relationship between abstinence duration and heath. Patients were more likely to report reductions in cough the longer they remained abstinent (tau-b = 0.230, p=0.014). Figure 4. Kaplan Meier curve using longest quit attempt at any time throughout the pilot projectIn examining the relationship of baseline characteristics with success at abstinence from smoking, only one variable was shown to be significant. As stated previously this was a population of long-term smokers with an average history of 26 years. It is not surprising that the longer a patient smoked, the less likely they were to successfully quit (Figure 5; r2 = -0.318, p = 0.012).Figure 5. Plot comparing the number of days a patient remained within the pilot project and the number of years as a smoker excluding those who dropped out prior to their scheduled quit date.38246051106695r2 = -0.318p = 0.01200r2 = -0.318p = 0.012Product ChoiceNon-prescription nicotine replacement therapies were the most commonly selected products by the pharmacist (Figure 6). The patch was the primary cessation aid used in 40 (48%) participants while varenicline was used by 29 (35%). This was not a randomized trial; therefore a variety of selection biases could be associated with product selection. For example, one might expect that individuals that select varenicline may be more likely to have already tried and failed on nicotine replacement and as such be more resistant to quitting. However, when stratified by smoking cessation product choice, those who used varenicline showed a trend towards a more favorable quit rate, although this was not significant (Figure 7).6985-359410Figure 7. KM curve stratified by smoking cessation aid00Figure 7. KM curve stratified by smoking cessation aidGeographic SettingThe majority of patients enrolled were located within urban centres (66%). Baseline patient characteristics between the two settings were very similar. Some differences did arise in the average amount of time the pharmacist spent with the patients (115 minutes versus 165 minutes). This is likely explained by the higher drop-out rate (45% versus 33%). Each of the two settings had 1 patient successfully quit and proportionally more patients in rural settings reported improvements in other health outcomes. For example, improvements of at least 1 point in cough or phlegm production was observed in 42% and 54% respectively compared to 33% of urban patients who reported improvements in both of these areas.Resource and Cost AnalysisThe direct costs of a smoking cessation project can be broken down into 2 primary components. The medications used to support patients who are in the process of quitting and the time of the health professionals who counsel the patients through the process. Medication CostThe average cost of medication paid for by the project was $176.09 with a median of $61.73. This data is presented in a box and whiskers plot (Figure 8).Figure 8. Box plot of medication costs for all patients enrolled in the pilot project.27559002031695Average = $176.09Median = $61.7325th percentile = $0.0075th percentile = $312.7300Average = $176.09Median = $61.7325th percentile = $0.0075th percentile = $312.73Values more than 1.5 times the interquartile range (75th percentile – 25th percentile) are considered outliers. We see 1 extreme positive value (>$800). It is also worth noting that the data is extremely skewed with both the minimum value and the 25th percentile falling at zero dollars. This is the result of the large number of patients that either did not use medications or dropped out of the pilot project before medication use could be considered. Given this result, it may be more informative to look at the data for only those patients that actually used medications.If only the patients using medication are considered the average cost was $302.25 with a median cost of $282.84 (Figure 9). 27686001907870Average = $302.25Median = $281.8425th percentile = $149.3275th percentile = $433.2200Average = $302.25Median = $281.8425th percentile = $149.3275th percentile = $433.22Figure 9. Box plot of medication costs for all patients using medicationThis more accurately reflects the actual cost of medication used by patients actively engaged in the quitting process. The interquartile range ($149.32 to $433.22) is expected to reflect a typical cost of medication per patient quitting. Looking at the overall distribution of costs of all patients in the project may also be informative (Figure 10). Figure 10. Medication costs for smoking cessation pilot project 160934414744700Patients incurring costs within the $251 to $500 band was most common. It is important to remember that these costs reflect the incremental costs of the project over and above the costs of the social assistance project. Products not covered by social assistance (nicotine replacement) were billed through the project. Other prescription products normally covered by social assistance (varenicline) were not billed through this projectam and cost information was not captured.Pharmacist TimePharmacists spent a total of 275.5 hours counselling patients over 483 appointments. An average of 149 minutes was spent with each patient throughout the pilot project with the majority spending between 60 and 180 minutes (Figure 11 and 12). Figure 11. Box plot showing a breakdown of the time spent with the pharmacist3668091219392575th percentile = 171.5Mean = 148.9 Median = 14525th percentile = 107.502000075th percentile = 171.5Mean = 148.9 Median = 14525th percentile = 107.5882484708025(min.)00(min.)Figure 12. Number of patients who spend an average of < 1 hour, 1-3 hours, 3-5 hours, 5-7 hours, or >7 hours with the pharmacist89839517525Number of Patients00Number of Patients16290792974340Average Duration with each Patient00Average Duration with each PatientAfter a longer initial assessment visit typically lasting 74 minutes, follow-up consultations usually lasted 15-30 minutes (Figure 13). There were 4 possible follow-up appointments (1 week, 1, 3, and 6 months) in addition to the quit day, assessment and pre-visit (project introduction and discussion of forms to be filled out) appointments. On average, patients met with the pharmacist 4 times. Some patients require additional support over and above the recommended 4 follow-up appointments; 12 patients scheduled 1 to 6 additional appointments with the pharmacist. It was also worth noting that these estimates are likely an under-representation because it does not included failed attempts to contact patients and pharmacists did not count any extra time that was spent with patients that was not a scheduled follow-up.Figure 13. Average amount of time spent with the patient at each visit in minutes-60191115060Time (min.)00Time (min.)24565113301365Type of Visit00Type of VisitCounselling CostThe time spent by pharmacists has been outlined in detail above. The approach taken to cognitive reimbursement was a flat fee ($300 per patient enrolled). This makes the assessment of costs for this service straightforward even though the range of counselling service demanded varied widely (< 1hour to > 7 hours). The distribution of counselling time (Figures 11, 12, 13) suggests that a more sophisticated approach to cognitive reimbursement may be required.Total Costs For the 119 patients enrolled in the pilot project the total medication cost under the project was $20250.86. Total eligible cognitive reimbursement was $35700 for a total cost of $55950.86 for the project, or $470.18 per patient. Total Benefits It is expected that many of the 119 patients that were unsuccessful on this attempt will have taken an important incremental step on their path to quitting on a future attempt. These benefits are difficult to quantify. We do know that for every quitter, we can expect 3.6 to 3.7 in life years gained and an increase of up to 6.6 in quality adjusted life years (QALYs).18-20Cost Effectiveness If we assume that our costs are incremental to the existing project and that our benefits are fully incremental to existing programs we can make a pragmatic estimate of the incremental cost effectiveness ratio (ICER) for our intervention. If E represents the current benefit with existing smoking cessation efforts and C represents the cost of current efforts then:ICER = {(C+$55 950)- C}/{(E+(6.6QALY*2))-E}= $55 950.86 / 13.2 QALYs= $4239/QALYThis is the most favourable estimate but even in a sensitivity analysis using the lowest estimate for the incremental QALY benefit for a quitter the estimated ICER was $8252/QALY. Likewise, we estimated the cost per life year gained at between $7361 - $7771 per life-year. These cost effectiveness estimates do not include the harm reduction benefits seen in the pilot project (Table 4). It is also clear that a sizeable number of patients (n=41) were able to reduce the amount of cigarettes they consumed per day. These benefits are also not captured in the economic assessment above. The average reduction of 16 cigarettes may have a significant impact on the lives of the individuals involved. If we assume a price of $13.88 for a package of 25 cigarettes this reduction may have a very positive impact on individuals with fixed incomes. This reduction in smoking (0.64 packages per day) represents a savings of $266 per month. This may represent between 11% and 40% of the income available to an individual on social assistance. This economic benefit may be sufficient to change lives over and above the health benefits that may result from these reductions.DiscussionPharmacists are trained to provide smoking cessation counselling including optimal medication therapy. Because pharmacists are accessible at the time of smoking cessation product purchase, they are ideally situated to provide individualized therapy. One of the objectives of this pilot was to test if these factors could be successfully used to deliver a smoking cessation program to a difficult to reach population. Manitoba pharmacists were able to recruit 119 patients on Employment and Income Assistance in a period of approximately 8 weeks. Due to timing issues across the multiple pharmacies involved the project was actually over-subscribed as the target was 100 patients. Based on the smoking history, Fagerstrom score, comorbid mental health conditions and other substance abuse, it is clear the project was successful in reaching the intended target population.Our results suggest that successful smoking cessation is difficult for Manitoba residents who are of lower socioeconomic status. However, our rate of 1.7% is similar to that observed in a UK study comparing pharmacist consultation with nicotine replacement to group behavioural supportADDIN CSL_CITATION { "citationItems" : [ { "id" : "ITEM-1", "itemData" : { "DOI" : "10.1093/ntr/ntq222", "ISBN" : "4401786467", "author" : [ { "dropping-particle" : "", "family" : "Bauld", "given" : "Linda", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Boyd", "given" : "Kathleen A", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Briggs", "given" : "Andrew H", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Phil", "given" : "D", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Chesterman", "given" : "John", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Ferguson", "given" : "Janet", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Judge", "given" : "Ken", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" }, { "dropping-particle" : "", "family" : "Hiscock", "given" : "Rosemary", "non-dropping-particle" : "", "parse-names" : false, "suffix" : "" } ], "container-title" : "Nicotine and Tobacco Research", "id" : "ITEM-1", "issue" : "2", "issued" : { "date-parts" : [ [ "2011" ] ] }, "page" : "135-145", "title" : "One-Year Outcomes and a Cost-Effectiveness Analysis for Smokers Accessing Group-Based and Pharmacy-Led Cessation Services", "type" : "article-journal", "volume" : "13" }, "uris" : [ "" ] } ], "mendeley" : { "formattedCitation" : "(Bauld et al., 2011)", "plainTextFormattedCitation" : "(Bauld et al., 2011)", "previouslyFormattedCitation" : "(Bauld et al., 2011)" }, "properties" : { "noteIndex" : 0 }, "schema" : "" }.21 Their success rate was 2.8% measured at 52 weeks and was based on up to 12 weeks of one-to-one support typically lasting between 5 and 15 minutes. Although the study did not provide baseline characteristics to compare to, the intervention was similar. This population represents a resistant, hard to reach group. Given the high smoking rate of people with low incomes, this is an important group to support in the quitting process. In Quebec, coverage of smoking cessation aids seems to have been of particular advantage to low income individuals and may have contributed to a reduction of the smoking rate in Quebec from 30% to 25%.22 With an average smoking history of 26 years in a highly dependent group, successful abstinence for extended periods of time is expected to be low. Other studies that focused on the socio-economically disadvantaged have failed to have a single patient sustain quitting longer than 12 weeks.23 It is well established that quitting smoking is difficult and often takes several attempts before a user is successful. Now that they are armed with more information, they may be more successful on their next attempt. Sustained efforts are required to ensure continued improvements in this difficult population. Other pharmacist intervention studies in more general populations have shown higher quit rates.24-27 High drop-out and lost to follow-up rates are common in addiction studies. How missing data is dealt with can determine the reported benefit. For example, a recent community pharmacist initiated smoking cessation study reported an average quit rate of 25% at 6 months.27 However, the authors used the LOCF method to account for missing data which can overestimate benefit since most patients that start smoking again never return for follow-up. This method counts these patients as still abstaining. We used a KM curve which censors patients that were lost to follow-up to account for this missing data without contributing uncertain data. We feel the KM analysis provides a better estimate of abstinence rates and accounts for some of the difference in success rates. When applying the same last observation carried forward method to our data, we can achieve similar rates (21%). Given the high rate of relapse in all smoking studies, last observation carried forward is inappropriate and produces an inflated estimate of the project’s efficacy. Success can be measured in ways other than the complete cessation of smoking. With an average reduction in daily cigarette consumption of 16, this project is still a successful harm reduction strategy. By cutting back by more than a half a pack per day, these patients can also enjoy the benefits of an increase in available disposable income. Given the fixed income of these patients, we have estimated a savings of $266 per month, which represents a sizable portion of the available monthly income.In addition, this smoking reduction has also lead to patient reported improvements in other health outcomes, such as cough and shortness of breath. With a possible dose-response curve associated with many adverse outcomes, any amount of reduction can decrease the risk of serious events such as lung and cardiovascular disease.28Smoking is the leading cause for preventable death and is responsible for 1 in 10 adult deaths. The associated morbidity represents a sizable strain on our health-care budget. The “price” of smoking is so high that almost all interventions to support smoking cessation have been shown to be cost effective. An average of $470.18 per patient was spent supporting smoking cessation in this project. The estimated cost effectiveness was $4239 to $8252 per quality adjusted life year gained or $7361 to $7771 per life year gained. These values are similar to those seen in other pharmacist smoking cessation programs. The UK program with a comparable success rate calculated a cost effectiveness of ?2,600 per quality-adjusted life years gained (approximately $4700).21 The cost of $4349-8352/QALY for this project needs to be evaluated in some context. There is, however, no official cost effectiveness threshold in use in Canada. Values from $20 000 to $100 000 are frequently used.29 The typical threshold of $50 000 is sometimes suggested but the origin and defensibility of this threshold has been questioned.30,31 The World Health Organization has suggested linking the threshold to per capita gross domestic product (Table 6).32 Manitoba has an estimated per capita GDP of >40 000 per year.33 It is clear that regardless of the threshold considered, at $4239 to $8252 /QALY this smoking cessation project falls well below all established thresholds for cost effectiveness used in the adoption of new medications and other healthcare technologies.Table 6. Criteria for Interpreting ICERs_______________________________________________ CU RatioInterpretation_______________________________________________Less than per capita GDP Highly Cost Effective1X to 3X per capita GDP Cost Effective>3X GDP Not Cost Effective_______________________________________________Despite the level of cost effectiveness, the results suggest some methods for improving the efficiency of the project. The flat-cognitive fee does not work well for a population where the majority does not follow through on their stated attempt to quit smoking. Alternative structures that compensate for this initial discussion but support pharmacists more evenly throughout the process might be more economically efficient. It is also clear that there will be a need to support patients through multiple quit attempts.ConclusionDespite the low quit rates, the Manitoba pharmacist initiated smoking cessation project was cost effective. The inclusion of the reduction in cigarettes consumed per day would improve the estimate to an even greater degree. Other health benefits that cannot be quantified in this analysis are important and valuable additional outcomes to these patients. Our results suggest that quit rates in the range of other smoking cessation programs can be achieved in this hard to reach population.There are many factors that increase the risk of relapse present in this population: low socioeconomic status, mental health, concomitant substance abuse, long history of smoking and current dependence on nicotine. This presents major barriers to a successful quit attempt. However, this also demonstrates that this population is in greater need of a program to assist in the transition from a smoker to a non-smoker.References1. Government of Manitoba. (2011). CPPHO: Report on the health status of Manitobans.Retrieved from .mb.ca/health/cppho/index.html2. Reid JL, Hammond D, Burkhalter R, Ahmed R. (2012) Tobacco use in Canada: Patternsand Trends, 2012 Edition. Ontario: University of Waterloo Propel Centre for PopulationHealth Impact, University of Waterloo. Retrieved from tobaccoreport.ca3. Health Canada. (2012). Canadian tobacco use monitoring survey (CTUMS) 2011.Retrieved from hc-sc-gc.ca/hc-tobac-tabac/research-recherceh/stat/ctumsesutc_2011-eng.php4. Canadian Cancer Society: Manitoba Division. (2012, May 9). Canadian Cancer Societyreport shows Canadian cancer death rate down. Canadian Cancer Society retrieved fromcancer.ca/~/media/Files/Canadian_Cancer_Statistics_2012_MB.ashx5. The Conference Board of Canada. (2013). Smoking Cessation and the Workplace Briefing 1 — Profile of Tobacco Smokers in Canada. Ottawa 6. Centers for Disease Control and Prevention. (2014). Best Practices for Comprehensive Tobacco Control Programs - 2014: Cessation Interventions (pp. 40–55). Atlanta.7. Schwartz R, Walsh B, Keller-Olaman S, Kang J, Patterson C. (2012). Building thecapacity of health professionals in tobacco control: A review with suggestions for movingforward in Canada Final Report. Ontario: Ontario Tobacco Research Unit.8. Government of Manitoba. (2012, May 01). Legislation would prohibit sale of tobaccoproducts in pharmacies, health-care facilities, vending machines. Government of Manitobaretrieved from .mb.ca/chc/press/top/2012-05-01-142400-13973.html9. The Alberta Pharmacists’ Association. (2005). Pharmacist fee guide. Alberta:Edmonton.10. Manitoba Society of Pharmacists. (2012). Reimbursement for professional pharmacyservices: A proposal prepared for Manitoba Health. Manitoba.11. O’Connor S., Philipneri A., Schwartz R., Krynen-Hill M., Di Sante, E. (2012). TIMS-SKUpdate July 2012. Ontario Tobacco Research Unit.12. Canadian Pharmacists Association. (2011). QUIT patient documentation for theOntario Government’s funded QUIT smoking program. Ontario: Ontario Ministry of Healthand Long-Term Care.13. Ontario Ministry of Health Promotion and Sport. (2011). Overview of the pharmacy basedsmoking cessation program. Presentation to the Ontario Pharmacy AssociationSeptember 2011.14. Saskatchewan Ministry of Health. (2009). Ministry of Health Policy: Partnership toassist with cessation of tobacco policy. Section: Claims payment – cognitive services.Received October 8, 2012 from Pharmacists’ Association of Saskatchewan.15. Canadian Foundation for Pharmacy. Fees and claims data for government-sponsored pharmacist services, by province (updated Oct 2014). Retrieved from: cfpnet.ca16. Manitoba Society of Pharmacists. Publicly funded pharmacy services and fees across Canada. (August 2014). 17. Statistics Canada. (2011). Population, urban and rural, by province and territory (Manitoba). Retrieved from . Tran, M.T., Holdford, D.A., Kennedy, D.T., & Small, R.E. (2002) Modeling the cost-effectiveness of a smoking-cessation program in a community pharmacy practice. Pharmacotherapy 22(12):1623-1631.19. Crealey G.E., McEinay, J.C., Maquire T.A. & O’Neill C. (1998) Cost and effects associated with a community pharmacy-based smoking cessation program. Pharmacoeconomics 14(3):323-333. 20. Fiscella K., & Franks, P. (1996) Cost-effectiveness of the transdermal nicotine patch as an adjunct to physicians’ smoking cessation counseling. JAMA 275(16):1247-1251.21. Bauld, L., Boyd, K. A., Briggs, A. H., Phil, D., Chesterman, J., Ferguson, J., … Hiscock, R. (2011). One-Year Outcomes and a Cost-Effectiveness Analysis for Smokers Accessing Group-Based and Pharmacy-Led Cessation Services. Nicotine and Tobacco Research, 13(2), 135–145. doi:10.1093/ntr/ntq22222. Tremblay, M., Payette, Y., & Montreuil, A. (2009). Use and Reimbursement Costs of Smoking Cessation Medication Under the Quebec Public Drug Insurance Plan. Canadian Journal of Public Health, 100(6), 417–42023. Roddy, E., Romilly, N., Challenger, A., Lewis, S., & Britton, J. (2006). Use of nicotine replacement therapy in socioeconomically deprived young smokers: a community-based pilot randomised controlled trial. Tobacco Control, 15(5), 373–6. doi:10.1136/tc.2005.01451424. Costello, M.J., Spoule, B., Victor, J.C., Leatherdale, S.T., Zawertailo, L., & Selby, P (2011) Effectiveness of pharmacist counseling combined with nicotine replacement therapy: A pragmatic randomized trial with 6987 smokers. Cancer Causes Control 22(2):167-180. 25. Dent L.A., Harris K.H., & Noonan C.W. (2009) Randomized trial assessing the effectiveness of a pharmacist-delivered program for smoking cessation. Ann Pharmacother 43:194-201.26. Jackson, M., Gaspic-Piskovic, M., & Cimino, S. (2008) Description of a Canadian employer-sponsored smoking cessation program utilizing community pharmacy-based cognitive services. Can J Pharm 141(4):234-240. 27. Khan, N., Anderson, J. R., Du, J., Tinker, D., Bachyrycz, A. M., & Namdar, R. (2012). Smoking cessation and its predictors: results from a community-based pharmacy tobacco cessation program in New Mexico. The Annals of Pharmacotherapy, 46(9), 1198–204. doi:10.1345/aph.1P14628. U.S. Department of Health and Human Services. (2010). How Tobacco Smoke Causes Disease: The Biology and Behavioral Basis for Smoking-Attributable Disease: A Report of the Surgeon General How Tobacco Smoke Causes Disease?: The Biology and Behavioral Basis for Smoking-Attributable Disease A Report of the Surge. Atlanta.29. Kaplan R., Bush J., (1982) Health-related quality of life measurement for the evaluation research and policy analysis. Health Psychology 1982 1(1):61-80.30. Laupacis A. (2002). Inclusion of drugs in provincial drug benefit program: who is making these decisions, and are they right ones? CMAJ 166(1):44-77. 31. Bridges, J.F.P., Onukwugha E, Mullins CD. (2010) Healthcare rationing by proxy: Cost effectiveness analysis and the misuse of the $50 000 threshold in the US. Pharamacoeconomics et al 2010, 28(3):175-184.32. World Health Organization. (2005). Cost-effectiveness thresholds. Retrieved from . Statistics Canada. (2013). Gross domestic product per capita, Canada, provinces and territories, 2005/2006 to 2009/2010 (in current dollars). Retrieved from A. Project Stakeholders and Steering CommitteePrincipal Investigators Organization Dr. Shawn Bugden Associate Professor College of Pharmacy, University of Manitoba Apotex Centre 750 McDermot Avenue Winnipeg, Manitoba R3E 0T5 Dr. Brenna Shearer Chief Executive Officer, Pharmacists Manitoba #202 - 90 Garry Street Winnipeg, MB ~ R3C 4H1 Steering Committee Composition Dr. Silvia Alessi-Severini (Co-investigator)Associate Professor College of Pharmacy, University of Manitoba Justin Bates Neighborhood Pharmacies Association of Canada Murray Gibson Executive Director Manitoba Tobacco Reduction Alliance (MANTRA) Andrew Loughead Manager, Tobacco Control and Cessation Unit Manitoba Health, Healthy Living and Seniors Debbie Nelson A/Executive Director, Healthy Living and Populations Manitoba Health, Healthy Living and Seniors Cheryl Osborne MHS, Consultant - Primary Health Care Manitoba Health, Healthy Living and SeniorsScott McFeetors President, MSP Director, Pharmacy Operations Loblaw Companies Limited Kim McIntosh Assistant Registrar – Professional Development College of Pharmacists of Manitoba Sheila Ng Chair Professional Relations, MSP Pharmacy Practice Instructor University of Manitoba – College of Pharmacy Kristine Petrasko Regional Pulmonary Educator, Regional Pulmonary Rehabilitation Program Winnipeg Regional Health Authority Gayle Romanetz Vice President, Pharmacy Operations Loblaw Companies Limited Trevor Shewfelt Pharmacist Dauphin Clinic Pharmacy Primary Institution University of Manitoba Address: Apotex Centre 750 McDermot Avenue Winnipeg, Manitoba R3E 0T5 -31305-232410Appendix B. Orientation and Smoking Cessation Forms020000Appendix B. Orientation and Smoking Cessation FormsMANITOBA PHARMACIST INITIATED SMOKING CESSATION PROJECT 2014ORIENTATION SESSIONTuesday, January 7th, 2014, 1:00 – 5:00 P.M.1:00 – 1:15 Welcome and introductions1:15 – 1:30Overview of ProjectProject OutlineProject Goals1:30 – 2:00Ethical Considerations/U of M Ethics Committee Submission2:00 – 2:30‘Building your Smoking Cessation Toolkit’Smoking Cessation ResourcesSmoking Cessation Counselling Strategies2:30 – 2:45 Health Break2:45 – 3:45Professional DevelopmentSmoking Cessation Forms and Process3:45 – 4:15LogisticsProject Structure and ProcessesRemuneration ProcessesProject Timelines4:15 – 5:00Questions and DiscussionAppendix C. Consent Form ................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.