Together, the phenomenon of overweight and obesity is the second leading cause of preventable death in the United States, and has increased dramatically over the past 20 years (3;Reference Ogden, Carroll and Curtin10). Excess weight contributes to cardiovascular disease (CVD) risk factors and ultimately to CVD itself. Although behavioral weight loss interventions consistently demonstrate short-term success, longer-term weight re-gain is common (Reference Wadden, Crerand and Brock15). A critical factor for sustained weight loss is continued intervention with frequent contacts (Reference Jeffery, Drewnowski and Epstein9;Reference Wadden, Crerand and Brock15). The Weight Loss Maintenance Trial (WLM) was a multicenter, randomized, controlled trial comparing the effects of two maintenance interventions to a self-directed control group, among overweight or obese individuals at high CVD risk. Both the personal contact (PC) condition, a program of monthly contacts with a trained interventionist, and the Internet-based (interactive technology, IT) condition, a sophisticated interactive program, were designed to maintain frequent contacts compared with usual care.
This study describes and compares the implementation costs of the WLM interventions. We also estimate the development costs of the Internet-based intervention. Understanding better the costs of implementing and operating these programs will help analysts conducting economic evaluations of similar weight loss and other lifestyle change programs. It will also facilitate feasibility studies of their dissemination, if they should eventually prove cost-effective.
TRIAL DESCRIPTION
The WLM trial was a four-center randomized trial comparing two alternative strategies for maintaining long-term weight loss among 1,032 overweight and obese (body mass index = 25–45 kg/m2) persons on medication for hypertension and/or dyslipidemia. Four clinical centers participated: Duke University (Durham, North Carolina), Johns Hopkins University (Baltimore, Maryland), Pennington Biomedical Research Center (Baton Rouge, Louisiana), and the Kaiser Permanente Center for Health Research (Portland, Oregon). During Phase I, 1,685 participants started an intensive weight loss intervention of twenty weekly group sessions over 6 months. Lifestyle interventions focused on behavioral strategies for implementing increased physical activity and reduced calorie intake in the context of a healthy diet (i.e., the Dietary Approaches to Stop Hypertension diet [DASH]). Phase I enrollment occurred between August 2003 and July 2004.
Phase I participants who lost >4 kg were eligible for Phase II and were randomized between February 2004 and December 2004 to one of three weight loss maintenance interventions: (i) PC included monthly contact with an interventionist by telephone or in person; (ii) IT featured unlimited access to an interactive Web site that provided ongoing Web-based support; or (iii) a self-directed (SD) control condition. Follow-up weight and other data were measured at 6-month intervals through 30 months of Phase II (i.e., 36 months from the beginning of Phase I). Final data collection ended in June 2007.
The primary outcome was weight change from the end of Phase I to the end of Phase II (30 months after randomization). Other outcomes included subgroup weight change, control and prevalence of CVD risk factors, and behavior change measures. A detailed trial design description has been published (Reference Brantley, Appel and Hollis2), and is summarized below.
Personal Contact (PC)
Study interventionists contacted PC participants monthly over the 30-month intervention period; annually, nine contacts were phone-based, supplemented by three face-to-face (FTF) contacts. PC contacts included reinforcement of calorie intake to maintain weight loss, a weekly physical activity goal of 225 minutes, and consumption of the DASH diet.
FTF contacts were approximately 45 minutes and had three components. Check-in (10–15 min) included a weigh-in, review of self-monitoring and participant goals, and feedback on last month's performance. Discussion and Training (20–30 min) consisted of brief topical discussions or hands-on training, involving methods for handling high-risk situations and possible lapses in diet and activity plans. FTF sessions focused on problem solving and relapse-prevention strategies in which interventionists were extensively trained, and which were also practiced in phone sessions described below. In Action Planning (5–10 min.), interventionists used motivational interviewing techniques and strategies to assist participants with identifying short-term diet and activity goals.
Phone contacts lasted 10–15 minutes. Standardized materials and procedures guided the topical discussion, and documented it for use in future calls. Interventionists established rapport, reviewed the previous month's diet and activity efforts, and recorded self-monitoring and physical activity data. Identified barriers or difficulties were discussed and problem-solving techniques agreed upon. Interventionists used case scenarios as examples when identifying common situations, barriers, and problem-solving approaches. Scenarios also triggered questions and supportive comments.
Various techniques were used to ensure completion of one-on-one contacts. All calls were prescheduled. Subsequent calls were scheduled during the current call. Daily phone logs were generated for each interventionist with call schedules and contact data; logs were also used to verify call completion and for rescheduling. A contact information database was continuously updated. Participants were contacted within 24 hours to reschedule missed calls; rescheduling efforts were tracked for early problem identification. A case conference mechanism helped interventionists with difficult situations and to develop approaches to nonadherence.
Interactive Technology (IT)
Core behavior-change components of the IT intervention were similar to those of PC, but used Internet and automated phone technology to enhance feedback frequency and timeliness. Web site features included weekly weight loss tips, a personal profile, links to reliable health and weight loss-related information, and printable materials (Reference Stevens, Funk and Brantley12).
Figure 1 is an overview of IT's content and process. Core elements included strategies to promote adherence to health behaviors for weight loss maintenance. After initial FTF orientation with an interventionist, participants used the Web site to construct a personal action plan that could be updated at any time. Collaborative goal setting and problem-solving techniques were used to identify action plans for perceived barriers and triggers. Tailored e-mail reminders, with a backup interactive voice response (IVR) phone system, prompted participants to visit the Web site for self-monitoring and motivational modules. Participants could also log on at any time to enter data on weight, food records, and physical activity, communicate with other participants through a monitored bulletin board for social support, or seek other information. Feedback reports were available on demand, tracking participants' weight, dietary pattern, and physical activity over time.
Figure 1. Overview of the interactive technology intervention. IVR, interactive voice response.
Trial Results
Primary trial outcomes have recently been published (Reference Svetkey, Stevens and Brantley13). Thirty-month follow-up rates were 93 percent for IT (323 of 348) and 94 percent for PC (321 of 342). One death occurred in each group during the study; most others were losses to follow-up. Mean Phase I weight loss was 8.5 kg, although weight regain occurred after randomization. Both interventions produced improved weight loss maintenance over SD during the first 24 months following randomization (IT: mean −0.9 kg difference in weight change, p = .045; PC: mean −2.0 kg difference in weight change, p <.001). However, only the PC intervention showed a statistically significant improvement over SD at 30 months (PC: mean −1.5 kg difference in weight change, p <.001). Overall, 71 percent of study participants remained below entry weight at 30 months.
METHODS
We define intervention costs as the value of resources used to implement and operate the subject interventions over the 30-month trial period and measure them from the perspective of a sponsoring healthcare system. Costs exclude participant investments of time, money, or other resources. SD participants received no intervention components.
The primary analysis is limited to implementation costs and is not a true economic evaluation comparing the costs and consequences of alternative courses of action (Reference Shiell, Donaldson and Mitton11). Research-related costs not integral to the intervention are excluded. The primary analysis also excludes development costs; the analytic perspective is a healthcare system considering adopting an extant intervention, rather than developing its own. We did not analyze the cost of developing PC, which is conventional with well-understood development activities. For WLM, much of PC was adapted from the earlier PREMIER weight loss trial (Reference Appel, Champagne and Cooper1;Reference Funk, Elmer and Stevens5). In contrast, few previous studies used Web sites similar to WLM's IT intervention, designed specifically to support long-term weight loss maintenance (Reference Glasgow, Nelson and Kearney6–Reference Harvey-Berino, Pintauro and Gold8;Reference Tate, Jackvony and Wing14;Reference Wing, Tate, Gorin, Raynor and Fava16). To our knowledge, none estimated the costs of such an intervention. Therefore, in a later section, we present estimated IT development costs. We also assume that the Web site is (developed and) implemented by an organization with significant resources for and experience in healthcare-related Web site delivery. This suggests multiple Web-based projects jointly share most equipment and nonlabor resources.
Project staff identified relevant cost components, classified as labor or nonlabor. When possible, intervention components were micro-costed, that is, participant-level data were collected on the exact number and type of each resource. Project-specific unit costs were applied to quantities consumed, and the results summed to obtain resource values. For example, intervention staff time was valued as the total intervention time of each staff member multiplied by the appropriate wage rate, including fringe benefits and a 30 percent “support rate” multiplier that accounts for nonfringe activities (e.g., department meetings, daily breaks) not reasonably assignable to one project. (A project site uses the 30 percent figure in budgeting.) Other costs included equipment or printed materials. Cost data were collected from project staff, IT staff, finance department staff, expense reports, or retrospective labor estimates. We applied Portland Oregon-based unit costs (e.g., wage rates) to quantities, but tested a wide range of costs in sensitivity analyses. For each intervention, total cost and cost per participant were estimated. All costs were in 2006 dollars using the Prospective Payment System Input Price Index (4). Costs were discounted at a 3 percent annual rate. Univariate and multivariate sensitivity analyses addressed cost variation between different implementation settings. We also performed a Monte Carlo simulation in DataPro (TreeAge Software, Inc.); parameter-specific distributions were resampled and analytic results recalculated 10,000 times, with the intent to produce an empirical distribution of model results.
Inputs: Personal Contact
Face-to-Face Visits. Costs include counseling time (plus administration and multiple reminder call attempts), facility space, phone charges, standard printed materials (discussion guides and case scenarios), and supplies.
Telephone Contacts. Costs include counseling time (plus administration and multiple contact attempts), daily phone logs and other materials, phone charges, case conference mechanism, and supplies.
Annual Training Session. Costs include trainer and trainee time (plus administration), printed materials, and a rental facility fee. Travel time is excluded because on-site training is assumed.
Quality Control. Costs include an intervention director's oversight activities, and a case conference mechanism to periodically review problem cases for assistance with modifying an action plan.
Inputs: Interactive Technology (Implementation)
IT-Specific Infrastructure. We distinguish between general fixed overhead charges that are assumed common to both interventions (e.g., utilities, janitorial), and IT-specific infrastructure. The latter resources—enhanced air conditioning, uninterruptible power supply, generator, network backbone, Ethernet switches, router, firewall, and Internet connectivity—refer to particular networking, security, and other resources needed to operate an enterprise capable of hosting a Web site, but which are unnecessary for a PC-type intervention.
Direct Labor. A content developer/maintainer assured that Web site content was current, complete, and “on message.” A system architect organized and oversaw Web site structure, based on user needs and cost and scheduling constraints. An application developer converted design instructions from the system architect into program code. A primary contact person provided Web site user support by phone and/or e-mail. A product manager provided project coordination and oversight.
Direct Nonlabor.
HARDWARE
A collection of three Web farm servers was maintained to meet needs beyond any one server. Web farms often allocate primary and backup servers to a single task. Two backup servers provided redundancy. A Structured Query Language (SQL) database server provided searchable information retrieval and data warehousing. An SQL reporting server provided interactive and printed reports. Two domain control servers responded to security authentication requests.
SOFTWARE
An SQL server functioned as a relational database management system. E-mail control software organized Web site e-mail components, for example, authentication, file attachments, and embedded images. Two enterprise-level server platforms, designed for high-volume, high-traffic networks, provided basic functionalities of the server hardware.
CONTRACTED SERVICES
Internet service provider charges were monthly fees paid for a more user-friendly Web site address than the hosting site's default address. Data backup charges provided disaster recovery and data retrieval operations. Two years of IVR extended support represented ongoing maintenance of IVR hardware and software.
Inputs: Interactive Technology (Development)
Direct Labor. The second category of Web site design involves a clearly communicated stepwise outline of the standardized development pathway each interactive module follows. The design team organized the IT development process by task: needs assessment, content development, design specification, graphic design, usability testing, programming, quality assurance, pilot testing, and project management. Because most IT-based personnel performed multiple functions during development, it is more informative to categorize IT development labor costs by function than to categorize them by individual.
Direct Nonlabor.
HARDWARE
Developer workstations were high-performance computers used for graphics, computer-assisted design, software development, and scientific applications. Servers were computer systems that provide services to other computer systems (clients) over a network. “Server” refers to hardware—a computer system, or software—SQL server. A development/staging server temporarily staged new or revised Web pages before they were made live. An e-mail server was used as messaging and collaborative software including electronic mail, shared calendars and tasks, and support for mobile and Web-based information access, supporting very large data storage.
SOFTWARE
Various software components, requiring an underlying productivity suite such as Microsoft Office, were also used. Diagramming software produced graphical diagrams. Source control software managed multiple revisions of the same information unit, such as a computer program. An integrated suite of applications was used in graphic design. An SQL server managed the overall relational database. An integrated software development program provided comprehensive facilities to Web programmers during development to maximize productivity. E-mail control software organized Web site e-mail components, for example, authentication, file attachments, embedded images. A (standard) server platform provided basic functionalities of the server hardware. A Web server accepted information requests from clients (Web browsers), and served them formatted responses along with optional data contents, usually Web pages, including documents and linked objects (images, etc.). An oversight program provided precoded solutions to common program needs, and managed specific program execution.
ADDITIONAL SERVICES
Internet service provider charges. Data backup services. Months 31–36 of IT-specific infrastructure were also included.
Interactive Voice Response (IVR)
Direct Labor. Script and algorithm design involved developing participant contact language and a response flow chart. Programming labor converted design instructions into program code. Testing labor provided oversight of pilot testing. IVR technical support was the primary contact for technical/operational questions.
Direct Nonlabor. Total IVR system installation cost included the server, data modem, dual telephony board, text-to-speech software, system software, training, licenses, and 1-year warranty. Two years of IVR extended support represented ongoing maintenance of IVR hardware and software.
RESULTS
Supplementary Table 1, which can be viewed online at www.journals.cambridge.org/thc, lists component baseline costs for PC, and Table 1 lists total estimated implementation costs. Total discounted 30-month implementation costs for the 342 PC participants were $537,242 (per-participant costs: $1,571 total; $628 annual). Nearly all costs were labor, approximately half of which could be classified as variable, driven by the number of participants. Most fixed costs arose from the intervention director's 0.70 full-time equivalent allocation.
Table 1. PC Intervention: Baseline costs (30 months)—Implementation
Supplementary Table 2, which can be viewed online at www.journals.cambridge.org/thc, lists baseline IT implementation parameter values, and Table 2 summarizes estimated IT implementation costs. Total estimated 30-month implementation costs were $214,879 (per-participant costs: $617 total; $247 annual). Labor represented 90 percent of total costs; note that user support absorbed 33 percent of total labor expense (30 percent of total expense). WLM adopted a tiered approach to Web site user support; support staff fielded initial inquiries with more complex inquiries referred to the product manager, and as appropriate, to the Web programmer. Because Web site contact was largely patient-initiated, we believe that no labor expense would be considered variable (in the sense of varying with the number of participants). In particular, WLM staff believe that IT user support could accommodate double or even triple the number of participants without difficulty. Three percent of total costs were attributed to IT-specific infrastructure.
Table 2. IT Intervention: Baseline costs (30 months)—Implementation
*Server, modem, dual telephony board, text-to-speech software, system software, training, licenses, and 1-year warranty.
Sensitivity Analyses
In univariate sensitivity analyses, no reasonable change in parameter values would lower the estimated 30-month implementation cost of PC below $400,000. Even assuming both an FTF counselor hourly wage of $25 and a 30-minute FTF session together lowered total cost only to $406,000. Increasing the allocation percentage for IT-specific infrastructure from 1 percent to 50 percent raised total IT implementation cost to $339,000. The probabilistic analysis was simply confirmatory; IT was cheaper to implement in virtually all iterations (parameter distributions and results available upon request).
Web Site Development Costs
For illustrative purposes, we estimated the 36-month costs of developing the IT Web site. Supplementary Table 3, which can be viewed online at www.journals.cambridge.org/thc, lists baseline development parameter values, and Table 3 lists estimated IT development costs. As mentioned earlier, we present labor expenses by function because most personnel performed multiple roles. Nonlabor expenses are predominantly hardware (e.g., various servers) and software (e.g., version control).
Table 3. IT Intervention: Total costs–Development
Total discounted development costs were $839,949 (per-participant costs: $2,414 total; $805 annual). Web site labor represented 85 percent of total costs; the largest proportions of labor expense were in programming (36 percent), project management (17 percent), content development (15 percent), and design specification (15 percent). One percent of total costs were attributed to IT-specific infrastructure. We included IVR installation ($90,000) in development costs.
Summing development and implementation costs across 66 months (36 development + 30 implementation), total discounted IT costs were $1,045,197 (per-participant costs: $3,003 total; $546 annual). Although not surprisingly, total IT development and implementation costs exceeded PC implementation costs alone, it is interesting to note that the annual per-participant cost of IT development and implementation ($546) was 13 percent less than the analogous cost under PC implementation ($628). Also, we conservatively assumed that development and implementation were completely distinct periods, while in reality these activities may overlap in time, shortening the total measurement period. For WLM, implementation began early in year 3 of development, which would shorten the total period from 66 to 54 months (5.5 to 4.5 years).
DISCUSSION
The effect of both interventions was admittedly modest (Reference Stevens, Funk and Brantley12), and may not currently justify their implementation. However, research into effective strategies for maintaining long-term weight loss is in its infancy, and the WLM strategies are in early development. As more effective treatment modalities are designed, their implementation will become increasingly feasible, and our analysis lays the groundwork for appropriate cost accounting.
It is perhaps unsurprising that the average per-participant cost of IT implementation was substantially lower ($954) than that of PC implementation. Note that this difference would accommodate considerable tailoring of an existing Web site to local circumstances. Also, from the health system perspective, at 348 participants the IT program would seem to operate in the range of the average long-run cost curve exhibiting economies of scale, that is, the average cost of production declines as output increases. This has practical significance because as we discuss below, implementing health systems are likely to have much larger participant populations than that of the WLM trial. (Note, however, output here is number of participants, not quality-adjusted life-years or other health outcomes. Also, from a societal perspective a higher proportion of costs would be variable because of the value of participant time.) We assumed most costs were fixed, and the marginal cost of an additional Web site user approached zero. In an operation such as WLM's host location, the benefit of sharing costs across multiple participants is amplified by sharing joint resources across multiple projects, driving average costs even lower.
It is important, however, to distinguish between economies of scale and scalability, an important dimension of Web site development and operation. In contrast to economies of scale (declining average cost), scalability, strictly speaking, refers to roughly constant average cost—throughput increases proportionately with increases in inputs. The concepts are closely related, but qualitatively scalability is meant to convey the ease with which, say, a new location could be added to the Web site infrastructure.
How many users could reasonably be added before technical constraints would likely arise? The existing system that hosted the WLM Web site (i.e., with resource redundancy and other “competing” Web sites) supported 348 trial participants over 30 months. Some extant Web-related projects at the host site have up to 5 percent of their populations using the project Web site concurrently; in WLM, however, concurrent use was typically just one or two participants (again, of 348).
Host site engineers estimate the theoretical maximum of the site's hardware (e.g., servers) is 350,000 participants. This assumes that all excess capacity is devoted to the WLM IT Web site, and assumes no growth in other Web sites sharing the resources. However, mild performance degradations—e.g., slower response time, sporadic signal drops—would begin to emerge as early as approximately 3,500 participants, a much smaller number (although ten times the WLM population, and a more representative estimate of an implementing health system in the real world). Around 3,500 participants, average participant cost would decline to approximately $65 because few modifications would be needed (≈ $100 at 2,300 participants). The first likely constraint is bandwidth, the rate of data transfer supported by a network, usually measured in bytes per second. Without expanded bandwidth, increased Web site traffic would eventually induce less responsiveness and more frequent service interruptions. Specific project needs can also induce lower performance. WLM, for example, had more complex reporting requirements than many other Web sites at the host site; this can degrade performance of the associated reporting software.
Software modifications can roughly optimize performance; however, limitations of these modifications would greatly reduce the true maximum number of users under the current system to approximately 30,000 (again, assuming that true concurrent users represent no more than roughly 5 percent). The cost of software modifications in acquisitions and maintenance can vary widely, but $150,000 over 30 months is a reasonable estimate. In this case, average 30-month participant cost declines still further to $12–$15 (≈ $50 at 7,500 participants). At approximately 30,000 participants and the large theoretical hardware capacity notwithstanding, service would degrade sufficiently that further software modifications would be inadequate, and one or more new servers would be required to expand capacity, while maintaining service quality. Of course, these limits would be reached much more quickly if the host site were already near capacity. Even doubling capacity, however, at participant levels of 100,000 and above, would still result in average 30-month participant cost of $10–$15. The clear implication is that cost is unlikely to significantly inhibit implementation of Web-based programs for weight loss maintenance for most health systems, if their effectiveness is established.
CONCLUSION
When hosted in a facility with substantial resources devoted to and expertise in Web site delivery, costs of implementing a Web-based intervention for weight loss maintenance are substantially less than those of a conventional program using phone and in-person contacts. In this situation, the per-participant Web site cost remains less than the personal contact intervention, even including development costs. If future trials demonstrate the effectiveness of these or similar interventions in maintaining initial weight loss, our results can inform the economic evaluations that will assess the true value for money produced by their implementation.
SUPPLEMENTARY MATERIALS
Supplementary Tables 1–3 (www.journals.cambridge.org/thc)
CONTACT INFORMATION
Richard T. Meenan, PhD, MPH, MBA (richard.meenan@kpchr.org), Science Director, Victor J. Stevens, PhD (victor.j.stevens@kpchr.org), Senior Investigator, Kristine Funk, MS, RD (kristine.funk@kpchr.org), Senior Research Associate, Alan Bauck, BS (alan.bauck@kpchr.org), Technical Director, Center for Health Research, Kaiser Permanente Northwest, 3800 N. Interstate Avenue, Portland, Oregon 97227
Gerald J. Jerome, PhD (gjerome@towson.edu), Assistant Professor, Department of Kinesiology, Towson University; Johns Hopkins University School of Medicine, Johns Hopkins University, Baltimore, Maryland 21205
Lillian F. Lien, MD (lien0002@mc.duke.edu), Assistant Professor, Department of Medicine, Duke University Medical Center, Box 3075, Durham, North Carolina 27710
Lawrence Appel, MD, MPH (lapel@jmhi.edu), Professor, Department of Medicine, Epidemiology, and International Health, Johns Hopkins Medical Institutions, 2024 East Monument St, Suite 2–618, Baltimore, Maryland 21205
Jack F. Hollis, PhD (jack.hollis@kpchr.org), Senior Investigator, Center for Health Research, Kaiser Permanente Northwest, 3800 N. Interstate Avenue, Portland, Oregon 97227
Phillip J. Brantley, PhD (brantlpj@pbrc.edu), Professor; Chief, Department of Behavioral Medicine, Pennington Biomedical Research Center, Louisiana State University, 6400 Perkins Road, Baton Rouge, Louisiana 70808
Laura P. Svetkey, MD (svetk001@mc.duke.edu), Professor, Department of Medicine, Duke University Medical Center, Box 3075, Durham, North Carolina 27710
