Home / sciense / Data On Drug Use Is Disappearing Just When We Need It Most

Data On Drug Use Is Disappearing Just When We Need It Most

Jun. 29, 2017
at
10:00 AM

Data On Drug Use Is Disappearing Just When We Need It Most

By Kathryn Casteel

Filed under Opioids

It’s no secret that heroin has become an epidemic in the United States. Heroin overdose deaths have risen more than sixfold in less than a decade and a half.1 Yet according to one of the most widely cited sources of data on drug use, the number of Americans using heroin has risen far more slowly, roughly doubling during the same time period.2

Most major researchers believe that source, the National Survey on Drug Use and Health, vastly understates the increase in heroin use. But many rely on the survey anyway for a simple reason: It’s the best data they have. Several other sources that researchers once relied on are no longer being updated or have become more difficult to access. The lack of data means researchers, policymakers and public health workers are facing the worst U.S. drug epidemic in a generation without essential information about the nature of the problem or its scale.

“We’re simply flying blind when it comes to data collection, and it’s costing lives,” said John Carnevale, a drug policy expert who served at the federal Office of National Drug Control Policy under both Republican and Democratic administrations. There is anecdotal evidence of how patterns of drug use are changing, Carnevale said, and special studies conducted in various localities are identifying populations of drug users. “But the national data sets we have in place now really don’t give us the answers that we need,” he said.


Among the key questions that researchers are struggling to answer: Is the recent spike in deaths primarily the result of increased heroin use, or is it also due to the increased potency of the drug, perhaps because of the addition of fentanyl, a synthetic opioid that can kill in small doses?

“Everyone thinks they know the answer” to whether fentanyl is behind the increase in deaths among heroin users, said Daniel Ciccarone, who’s a professor at the University of California, San Francisco’s medical school and studies heroin use. “Well, show me the data. … When you don’t have data that leads to rational analysis, then what you’re left with is confusion, and confusion leads to fear, and that will lead to irrational consequences.”

Researching illicit drug use has always posed challenges. One ever-present problem: Reliable information on illegal behavior is, almost by definition, difficult to collect. But researchers point to particular limitations in data sources that would help shed light on the heroin epidemic. And they say the problems are getting worse: Some data systems that were once used by government agencies to gather information on users, consumption and illegal markets have disappeared over the past several years. Other sources that are still available are becoming more difficult to access or don’t provide a clear picture of the problem.

The National Survey on Drug Use and Health, which is an annual household survey, is sponsored by the Substance Abuse and Mental Health Services Administration, a division of the federal Department of Health and Human Services. Through roughly 70,000 interviews, the survey collects information nationwide on the use of tobacco, alcohol and illegal drugs, as well as Americans’ mental health. Experts who study illicit drugs say the survey is an important source for estimating the number of those who use alcohol, tobacco and, increasingly, cannabis (because of the normalization of marijuana use). But many consider it inadequate for calculating the number of users of harder drugs such as heroin, cocaine and methamphetamine, which carry a greater stigma. Moreover, the survey excludes people without a fixed address, meaning people who are homeless or transient — a category that includes many of the heaviest drug users. These factors lead experts to believe the survey significantly underreports the number of users of hard drugs in the U.S. (A spokesman for SAMHSA said the survey doesn’t capture certain populations, including the homeless, and acknowledged that it faces other “limitations inherent in surveys.”)

Because of the well-known shortcomings of the National Survey on Drug Use and Health and other surveys that rely on self-reporting, experts often try to combine different sources to reach a more reliable estimate of total drug use. By looking at drug production and seizures, for example, they can estimate the supply of drugs that are reaching users. Fluctuations in the street price of drugs can give hints about changes in supply and demand. Each of those estimates carries its own challenges and caveats, but in theory, putting them together should give researchers a more complete picture of drug use — and the drug market more generally — than any one data point alone.

“If you want to understand something about drug popularity, drug consumption, how much of a drug is going to be consumed in a given year, then you look at the economics,” Ciccarone said.

These more comprehensive efforts tend to yield estimates of drug use that are much higher than those based on user surveys. A 2014 analysis by researchers from the RAND Corp. that was conducted for the Office of National Drug Control Policy, for example, estimated that there were roughly 1 million daily or near-daily heroin users in the U.S. in 2010, more than 15 times the 60,000 chronic users reported in the National Survey on Drug Use and Health for the same year.

In a related report to the White House, RAND provided information on how much money heroin users were spending on the drug, estimating that expenditures were roughly $27 billion in 2010, with much of the spending driven by daily and near-daily users. That kind of analysis has become more difficult to conduct in recent years, however. RAND relied in large part on the Arrestee Drug Abuse Monitoring system, a federal program that conducted interviews with male arrestees within 48 hours of their arrest to collect information on drug use, treatment and market activity, among other topics, and then validated usage with urinalysis testing for 10 substances. The early version of ADAM, from the early 2000s, covered 35 jurisdictions. But the program was cut for a couple of years in the mid-2000s and then was revived on a smaller scale in 2007 before being eliminated in 2013. The Office of National Drug Control Policy, which took responsibility for ADAM in 2007, declined to comment on the record for this story.

Experts who used ADAM said that interviewing and testing arrestees provided insights not available through other sources. ADAM was the only national data source on individual users’ expenditures and consumption, two important metrics for understanding drug markets. The loss of the program left a particular gap in researchers’ understanding of heroin use because it had provided data on a crucial and hard-to-study group: heavy users, who often drive the overall market. Beau Kilmer, co-director of the Drug Policy Research Center at RAND, said information collected on heavy users is important for evaluating enforcement efforts and making decisions about treatment availability.

“In order to generate these estimates for heavy users, there isn’t one data source you can use,” Kilmer said. “We think about how do we combine the insights for multiple data sets, and they have their flaws, but by far the most important data set we used was ADAM.”

Another source of data that is no longer being updated is the Drug Abuse Warning Network, which was once a project of SAMHSA. It monitored drug-related hospital emergency room visits and provided insight on drug use in metropolitan areas. Rosalie Liccardo Pacula, another co-director of RAND’s Drug Policy Research Center, said a crucial advantage of DAWN was that it provided data at the local level, allowing researchers to understand drug-use patterns in different parts of the country. In an email, a SAMHSA spokesman said the agency is working with the CDC to develop a new system for collecting data from emergency rooms that will replace DAWN and combine other data sets. President Trump’s proposed 2018 budget, however, doesn’t provide funding for the new program; the spokesman said the agency will continue to “explore the viability of this approach to collect these data in the future.” In the meantime, researchers say the loss of DAWN has left a big hole in their understanding of the opioid problem.

“We know the importance of emergency rooms right now with the opioid epidemic because that’s where people are showing up,” Carnevale said. “Right now, there’s no systematic collection by the federal government for emergency room data — again, another major loss in our knowledge base.”

Researchers pointed to a third government database that could help them examine drug trends — if they had better access to the information. The Drug Enforcement Administration collects information on the drugs it obtains through seizures and undercover buys by its agents in a database known as the System to Retrieve Information from Drug Evidence. The database isn’t designed for researchers — it is an internal inventory and can therefore be influenced by which drugs law-enforcement authorities are focused on at the time — but the data has nonetheless been valuable to researchers because it contains information on the price, purity and composition of seized drugs. In some cases, Ciccarone and other researchers say, the information has become more difficult to obtain.

Pacula, who has worked with the STRIDE database and DEA data extensively, said the agency a few years ago became more restrictive with data requests after becoming concerned that researchers were improperly sharing raw data. Pacula said researchers can still access the data they need as long as they are careful about their requests; others, however, were more critical.

“The DEA has basically substantially reduced access to their data, which taxpayers are paying for that collection of data,” Carnevale said.

When asked about the access issues that the researchers described, a DEA spokeswoman wrote in an email that data entry into STRIDE had stopped in 2014 and was replaced with the National Seizure System, which isn’t available to the public and requires a formal request under the Freedom of Information Act to access. She declined to comment further on the researchers’ concerns.

Experts argue that with the heroin epidemic showing no signs of slowing down, the government should bring back data collection systems like ADAM and DAWN and ideally expand them to include more areas and broader populations. (ADAM, for example, sampled only male arrestees in urban areas.) Researchers say the cost of these programs is relatively minor, pointing to ADAM’s peak annual cost of about $10 million. That’s about a fifth of what it costs to fund the National Survey on Drug Use and Health survey each year.

The lack of reliable national data is hindering efforts to tackle the spread of heroin, experts say. Some big cities such as New York have embarked on their own data-collection efforts, something the many smaller cities and towns ravaged by heroin overdoses likely can’t afford to do. Researchers say they need the federal government to help fund and coordinate efforts to collect data from local coroner’s offices, emergency rooms and crime labs so that local officials know where and how to direct their efforts. And, more broadly, they say they need a wide range of groups — researchers, public health workers, law enforcement officers, as well as federal, state and local governments — to work together to understand the heroin epidemic and to figure out how to stop it.

“We need to have public safety, public health partnerships,” Ciccarone said. “We need the government to be forthright. We need it to think that researchers and public health officials are on the same side as the people who also want to stop the drugs.”

Footnotes

  1. From around 2,000 in 2002 to roughly 13,000 in 2015, according to data from the Centers for Disease Control and Prevention. Data collected and analyzed by The New York Times suggests that deaths from all drug overdoses, not just heroin, could have risen to 65,000 in 2016.

  2. The Substance Abuse and Mental Health Services Administration defines “current users” as those who have used heroin in the past month.

Kathryn Casteel writes about economics and policy issues for FiveThirtyEight. @kathryncasteel

Filed under

Government Data (3 posts)
Opioids (2)
Heroin (1)

Thanks for subscribing!
A confirmation email is headed your way shortly. In the meantime, subscribe to FiveThirtyEight’s other newsletters.

Want more FiveThirtyEight? Get our weekly most popular stories newsletter.

Subscribe

All newsletters

Please enter a valid email address and try again.

Most Popular in Science & Health

Energy

We Use A Whole Bunch Of Energy Every Summer (And A Lot Of It Goes To Waste)

Most Popular in Sports


Contracts

The Bobby Bonilla Retirement Plan: Quit Baseball In 2001, Get Paid Until 2035

Thanks for subscribing!
A confirmation email is headed your way shortly. In the meantime, subscribe to FiveThirtyEight’s other newsletters.

Want more FiveThirtyEight? Get our weekly most popular stories newsletter.

Subscribe

All newsletters

Please enter a valid email address and try again.

Interactives


How The Internet* Talks


How The Internet* Talks

Most Popular in Science & Health

Energy

We Use A Whole Bunch Of Energy Every Summer (And A Lot Of It Goes To Waste)

source

About redox

Check Also

How Do We Know When A Hunk Of Rock Is Actually A Stone Tool?

The first time archaeologist John Shea looked at what might be the oldest stone tools ever found, he almost blew them off. “Are you kidding me?” he remembers asking Sonia Harmand, his colleague at Stony Brook University who found the tools in 2011 along the shores of Kenya’s Lake Turkana, at a site now known as Lomekwi 3. Harmand’s analysis suggested that the tools were 3.3 million years old — 700,000 years older than the previously known “oldest” tools. And they were huge. The mean weight of the pointy flakes — the cutting tools that are “flaked” off larger rock cores — was 2 pounds. In comparison, the next-largest group of ancient hominin tool flakes have a mean weight of 0.06 pounds. It seemed like a lot for the hands of a primate that was probably half the size of the average modern human. Maybe, this time, a rock was just a rock. But then Shea looked at the Lomekwi tools more closely, and he saw multiple fractures, all running in the same direction — a telltale sign that the flakes weren’t just the lucky product of one rock bumping into another as it tumbled off a cliff or rolled through a stream. Something had pounded one rock against another in the same place over and over and over … until a sharp piece broke off.The archaeologists who study lithic technologies — “stone tools” to us lay primates — are used to making these kinds of determinations. For generations they’ve been sorting out what is a tool, what isn’t, and what those tools mean, and they’ve been doing it largely by sight, by context of the objects and the site where they were found, and by experience. They learn the signs. They use dating techniques and look for evidence of how tools were being used. They test hypotheses by making their own stone tools and using them to see, for instance, how an ax might wear down after it’s hacked through a few deer femurs. “But here’s the problem,” Shea said. “Archaeologists don’t have measurements to answer these questions. The decision rests on visual assessment. I’ve made stone tools for 45 years, started as an 11-year-old Boy Scout. So is this an artifact or not? Truthfully, I have no idea. I can’t tell you based on measurements, and my opinion shouldn’t count for science.”Archaeological analysis always involves some amount of informed interpretation, and not all archaeologists are this skeptical of the traditional methodology. Harmand, for one, thinks that visual assessment is crucial. But only during the past decade or so has there been any real alternative. Called geometric morphometrics, it’s a suite of digital tools that can turn one of Lomekwi’s 4-inch-wide flakes of volcanic basalt into a 2-D or 3-D map of its curves, cracks, shape and size. These maps could be put into databases that could, eventually, allow scientists to compare the minute variations among tools from different periods and different locations. This technology is still in the early stages of adoption. Many tools have been scanned; far more have not. It’s primarily used to give archaeologists around the world access to artifacts they would otherwise have to travel long distances to see. People such as Shea hope that it will, one day, allow archaeologists to more definitively answer questions that today are addressed using circumstantial evidence and logical conjecture.The central issue is not necessarily whether scientists can tell a tool from a naturally broken rock. The more complex question is how to tell the difference between tools intentionally created (probably by humans or our ancient ancestors) and those that another primate made by accident. The difference matters. Once upon a time, we considered tool use the factor that separated animals and humans. As scientists discovered tool use among more and more animals — including chimpanzees and octopuses — the goal post shifted. Today, as far as anyone yet knows, humans and our ancestors are the only animals that create sharp-edged tools specifically for cutting things. Humans have taught captive bonobos to make stone cutting tools, but we have no evidence of any living primates intentionally doing this in the wild.But the line is awfully fuzzy. Especially when you’re looking at really ancient tools such as those at Lomekwi. Somebody somewhere had to invent stone tool making. And “probably the first tools ever made by a hominin were made by accident,” Harmand said. Imagine your less-upright ancestor going to town on a nut with a rock, trying to get at the good stuff inside. “Of course, you will sometimes hit the stone with your other stone and then detach something,” she said. And although hominins seem to be the only primate to think, “Hey, I’ll take that bit of stone and go butcher me a hog,” there’s evidence that other primates are still accidentally making “tools” — and that those tools are very difficult to distinguish from the ones archaeologists attribute to our predecessors.Last month, Oxford archaeologist Tomos Proffitt published a paper that documents wild capuchin monkeys in Brazil cheerfully screwing with a whole scientific field by intentionally picking up rocks and banging them against other rocks over and over and over in the same place … until sharp pieces broke off. There’s no evidence that the monkeys were trying to make tools, and they generally ignored the flakes. Their behavior could be an aggressive display or possibly a means of getting at lichens or trace minerals in quartz dust. Either way, the accidental “tools” they produce from intentional rock-banging are virtually indistinguishable from what you might find in 3-million-year-old East African soil. “If you brought this to me [and said it came from] East Africa, I would have told you, ‘You’ve got a new Oldowan. Where’d you find it?’” Harmand said. “I’m convinced many of my colleagues would confirm.”The context in which the tools or “tools” are found is really the only way to tell the difference. Where did they come from? How old are they? Were the sharp flakes being used for anything, like butchering, which often leaves cut marks on bones nearby?Geometric morphometrics could shed some light on this. For instance, what would Shea do about the capuchin monkey “tools” and their similarity to ancient hominin tools? “One of the claims is that these objects are indistinguishable from artifacts manufactured by humans. Well, let’s see. You could scan them, the ones we’ve watched be modified by capuchins. Then scan early human stone tools, and compare them. Do they differ in specific ways, and is that statistically significant?” he said. “We can argue about what it looks like, or we can measure it.”Proffitt, the scientist who wrote the capuchin paper, is one of the archaeologists who have used geometric morphometrics for research. One paper he worked on investigated whether different methods of making stone tools might leave different signatures in the rock. The freehand technique is similar to what the capuchin monkeys do: hold a rock in each hand and mash them together. In what’s called bipolar knapping, on the other hand, you put your stone tool core on a stationary anvil rock on the ground and then hit the core with another rock. That may not seem like a lot of difference, but, theoretically, bipolar knapping gives you more control. “It speaks to intelligence and cognitive processes going on,” Proffitt said. “If you have only freehand knapping for a million years, and then you have bipolar knapping, you [now] have more varied behavior.”The team digitized hundreds of flakes and examined their 2-D outlines — and found no statistically significant differences between the ones made with the freehand technique and those made with bipolar knapping. It’s possible that a 3-D model would show something more. But this paper highlights that you need more than just the tool — whether you’re eyeballing it or digitizing it — to know how it was made. That’s not a totally new conclusion, but it’s important, because it helps to clarify what morphometrics can and can’t do.Some of the most detailed work in morphometrics has been with younger North American tools. Here, the questions are less about whether something is a tool — with 12,000-year-old artifacts, their human-madeness is obvious — but, rather, what differences in the styles of tools can tell us about the people who made them.No two tools are identical, said Andy White, an archaeologist at the University of South Carolina. There wasn’t an Old Spear-Getting Factory churning out duplicate objects. But what we don’t know is how much of the difference between them can be accounted for by the vagaries of artisan work and how much represents real, cultural distinctions between groups of craftspeople. “You can take measurements in a computer that I can’t imagine how you’d ever take them with a pair of calipers. [You can] fit curves to things. Quantify things,” he said. “[I use] computer modeling and simulation to try and give myself some kind of credible avenue to interpretation.” White is compiling a database of projectile points, dating from 8,800 to 6,600 B.C., from across what is now the eastern United States.Ultimately, though, Shea thinks geometric morphometrics hasn’t really had its first major test yet. It won’t really become widely used or relied upon for interpretation until it answers some kind of flashpoint, high-stakes question in archaeology, he said. And that could be Lomekwi. Some are still skeptical of whether those are actual, intentional tools. If there were another site from around the same time with tools that were either more clearly hominin-made or more clearly monkey accident, then digitization might be able to reveal, for certain, where on that spectrum Lomekwi falls. “They have this one site,” Shea said. “Somebody needs to find something almost, or slightly, older, and they need to go at each other.”