How Science can Help Make, Assess, Amend Laws Better: ‘Legal Epidemiology’ Lessons

Senior advocate Anand Grover interviews Scott Burris, Professor of Law, Temple School of Law, Philadelphia, Pennsylvania, United States.

Scott, please tell us how you started working on legal epidemiology.

A: We began working on legal epidemiology as part of a strategy to increase the effective use of law for public health and to increase awareness of the ways in which law could be hurting public health.

In most places in the world, there has been relatively little research on the effects that law has, including its effects on health. This makes no sense, given that law has a pervasive influence on health.

We use law as an intervention, e.g., laws against smoking in public; laws setting speed limits or other driving rules. But we also use law to establish the powers and duties and jurisdictions of agencies in the government, which obviously has a lot to do with what that agency can do for public health.

In most places in the world, there has been relatively little research on the effects that law has, including its effects on health. This makes no sense, given that law has a pervasive influence on health.

Finally, the rest of laws, that may say nothing about health explicitly, also often have very powerful health effects, e.g., laws that regulate access to housing; or tax laws that shape the distribution of income and wealth in a society have very powerful effects on health.

They directly influence the social determinants of health so that the kind of law which we call incidental health law may be the most important kind of law to study of all.

Did you find that the effects of law were not being studied and, therefore, you ventured into it? How did you pick on these issues?

It is absolutely the case that law is studied far less than other phenomena that influence health. We did a study on the funding of research on law and health by the National Institutes of Health over a 25-year period and found that less than 0.01 percent of all grants supported any kind of research on the health effects of laws.

That makes no sense when you consider how frequently the law is used as a health intervention and when you think about the pervasive effects of laws on all aspects of social structure and behaviour.

One analogy that we often use is that law is like a pill; it is like treatment. Except that, unlike a pill, we treat billions of people at a time with laws and yet, unlike pills, we have little or no pretreatment evaluation of its effects. Often, we do not have any idea of what the law is supposed to do, what it is doing or whether it has negative side effects or not.

You said that less than one-tenth of 1 percent of the grants are spent studying the law and its impacts. So, were you really working in an uncharted territory? What did you find after you started working on the issue of health and the impact of law?

It is not quite right to say that we were working in an entirely new area. In fact, there was a 50-year tradition of very good scientific research on the health effects of laws. Indeed, there was a record of that kind of research having a strong influence on policy.

The problem was that a very small slice of law was being studied. We have very good evidence on tobacco control, on road safety, on alcohol control law. Those bodies of research showed that it was possible to study law in a rigorous scientific manner and that such research could positively influence policy in a way that improved public health.

It is not quite right to say that we were working in an entirely new area. In fact, there was a 50-year tradition of very good scientific research on the health effects of laws.

The problem was these bodies of research were in isolated areas and it had never been a general practice to investigate the health effects of laws after they were passed or indeed to consider the health effects or the possible health effects before the laws were passed.

We knew it was feasible. Rather than inventing something new, we took on the task of scaling up that kind of research.

Why call it legal epidemiology?

It came out of some colleagues at the Center for Disease Control. Throughout our work we have tried to use the language of science and the language of public health to break through the cultural barriers in the world of public health research when it comes to studying law.

It was never the case that there were no scientific ways to study the health effects of laws. It was only that people in public health did not think it was their job or did not believe that law could be properly studied.

In other words, we had a cultural rather than a practical or scientific barrier. ‘Legal epidemiology’ was a good way to talk to health researchers about legal evaluation.

When you expanded areas of research on the impacts of law on health what areas did you cover in health?

The field really began in 2009 when a United States Foundation gave my team US $20 million to launch a programme to expand the quantity and quality of research on the health effects of laws.

That US $20 million was not for us to keep. It was money that we could give out as grants. During the first five years of the field’s development, we were giving away a couple of US $1.5 million every year to people working on a wide range of topics. We covered research ranging from the effect of cell phone and driving laws to gun control laws. It cut pretty widely. We also worked on tools that people could use to do this research better.

We built a software system, now called MonQcle, that was specially designed for people to take the words of law and turn them into numeric data that can be used in large-scale quantitative studies of the health effects of laws.

Today, we are in the second generation of MonQcle and we are about to add an artificial intelligence (AI) assistant to the system to help make the research more efficient. We also built a website called Law Atlas, which is used to display legal data and to give it away at no cost to researchers who are interested in doing studies on the health effects of laws. Finally, the third thing we did was to develop a training manual and a textbook of legal epidemiology’s research methods.

Tell us more about this software MonQcle. What is so unique about it?

This is important to understand. We believe that a lot of the current policy research does not measure legal exposure— that is an independent variable. If you are studying the health effects of the law, it is necessary to make sure that whatever population you study is actually exposed to the law.

This means that the law has an effect in a particular place for a particular period. For that kind of research, you must go out into the world of law, do conventional legal research and create an exact map over time of which jurisdictions have what laws at what points in time.

But it is not enough just to find out if they have a law. For example, one might study the health effects of recreational marijuana laws. Well, two states can both have recreational marijuana laws, but their regulatory systems, that is, the rules for using and selling marijuana and, therefore, the availability of marijuana and the actual rate of use of marijuana may differ drastically.

‘Legal epidemiology’ was a good way to talk to health researchers about legal evaluation.

You cannot just say that you know that you have a database that tells us which states have medical marijuana laws. You must capture the specific elements of those laws, the rules about dispensaries and sales, hours of sales, the amounts of sales, the potency of the drug and everything else. You must also capture a great deal of nuanced data about the law to study it properly. Again, this was generally possible and had even been done in some studies but it was not the norm.

We decided that one of the first things we had to do in our efforts was to develop standard methods for that kind of legal measurement. But as we started to do that and develop the methods, we realised that we did not really have software that made it easy to use.

A lot of people use Excel. But Excel is very cumbersome for coding and is prone to clerical errors. We tried to use Google Docs, Google spreadsheets and Google forms and Microsoft Access but none of them were exactly right because none of them were designed for what we were trying to do.

Ultimately, we built our own software application, MonQcle. It performs the key functions one needs for this kind of research.

First, it populates various kinds of jurisdictions as separate records so that one can code them. If you are doing a study of, say, laws in 50 states in the United States, it will populate the worksheet with the 50 states. If you click on, say, Alabama, then you can do the coding for Alabama.

Secondly it allows one to create a uniform set of variables that one is going to probe into and get an answer for each of those states. On the left of the screen, you have your questions, and on the right you have your jurisdictions. As you go back and forth between them it is very easy to put in the answers.

Thirdly, within each record we upload the law. We have the actual legal text that we are going to measure, so that the person who is doing the work has the questions on one side and the legal text that they are going to measure on the other. They can run through the questions and answer them very conveniently.

MonQcle is very user-friendly. In the backend you get data tables like an Excel table or a CSV file but on the frontend it is just questions and answers that you get from the text.

Over time, it has allowed us to develop an AI assistant, which we are going to deploy next year. We can use various techniques which are now under provisional patent to link the text of the law to variables to answers from which the AI can learn to take over.

What is the specific role of AI as opposed to MonQcle?

If it works, AI will become like another person doing the work so that way you can properly measure the key elements of the law to create a database that is suitable for research.

There is a need to have a transparent and explicit protocol for doing the coding. As in all scientific research it must be replicable.

Secondly, there must be strong quality control. Typically, we have multiple people doing the same work. It becomes expensive. It might take us over US $100,000 to make a longitudinal dataset on a particular topic, e.g., 20 years of a state’s minimum wages or 20 years of cell phone and driving laws or whatever you want to look at. That kind of longitudinal data to draw causal inference requires a research design.

The field of legal epidemiology really began in 2009 when a United States Foundation gave my team US $20 million to launch a programme to expand the quantity and quality of research on the health effects of laws.

One can put a lot of time and effort into compiling that data and making sure it is sound. With the AI assistant that we hope to deploy, once the machine has learned that particular coding scheme, it can replace a person. Now, instead of comparing two people’s work, we compare the work of a human with that of a machine. That is cutting our labour in half.

What is the type of research have you done and can you tell us something about the results in the health field?

The most important research that we have been involved in has looked at law and its relationship to social determinants of health. We built 15 to 20 years of longitudinal datasets, covering key income support programmes in the United States, including minimum wages, earned income tax credit and unemployment compensation levels and so on in the states.

These are basic safety net rules that determine how much money people have in their pocket, which is basically a proxy for lots of different social determinants of health or a way for people to cope and get resources for health.

Our colleagues at Emory University, led by an epidemiologist, Kelli Komro, did very sophisticated analyses that showed that, by and large, these programmes had a direct positive impact on health.

For example, they looked at the effect of minimum wage on low birth weight and other neonatal complications and found that raising the minimum wage by US $1 had a huge effect on the number of birth complications that occurred.

Another study showed the effect on human immunodeficiency virus (HIV). Yet another showed an effect on heart disease mortality. This kind of research is incredibly important in demonstrating to the policymakers that if you increase the generosity of social welfare programmes, you get an immediate payoff in better health outcomes for the targeted group with fewer healthcare costs.

Other kinds of research look at more specific aspects of laws and health. For example, we are in the midst of a project that looks at the effect of laws against bullying in schools on the health of students. We just have a paper out that describes our data and shows how it can be used. For example, we calculated that more liberal states adopted more comprehensive anti-bullying laws sooner than more conservative states.

Our data has been used in hundreds of studies on topics such as the effect of drug policies on health and the effect of things like naloxone and drug overdose prevention laws. There is almost no area that we do not have some data on. Increasingly the results are showing the effects of the law.

Is this type of research only happening in the United States or is it now spreading out to other countries?

We are working very hard to spread this globally because there is nothing that restricts it to the United States either as a science or as a social good. We have worked a little with the World Health Organisation (WHO) on some pilot projects.

I teach and lecture any chance I get on this around the world. I have just come back from a trip to China where I lectured on this and where there is some interest in translating our book. MonQcle is set up to cover any jurisdiction in the world so it is completely suitable for foreign research. We still have to introduce automatic language translation and so on.

There are also some challenges to legal research in places where the law is not necessarily available electronically. Of course, there are some places such as Europe where the text of the law is more readily available than it is in the United States. I will give you an example of two global projects. 

One has been conducted at the O’Neill Center in Georgetown in their HIV policy lab. After 40 years of not having it, the Georgetown team has produced a pretty robust tracker of HIV policy around the world which we have long needed. They have done a fantastic job in collecting laws under difficult circumstances from many places and making it available through a website.

MonQcle is their backend. They compile all their legal data in MonQcle and then they publish it on a website that they have designed.

We built a software system, now called MonQcle, that was specially designed for people to take the words of law and turn them into numeric data that can be used in large-scale quantitative studies of the health effects of laws.

Another project of ours is with the WHO abortion group. It is to create a tracking system essentially for national consistency with the WHO abortion guidelines. For example, the abortion guidelines suggest that there should not be a waiting period for an abortion. We are going to identify how many countries have followed that advice and which ones have not.

Another example from the WHO abortion group is in respect of the requirements of parental notification. The WHO recommends against those. We will identify which countries have followed the recommendation and which ones still require parental notification for an abortion. That will eventually be on a website that will allow the world and the WHO to see who is following the guidelines and who is not and where more encouragement is required for following the guidelines.

Yet another example of an element of legal epidemiology that is very important is what we call policy surveillance. This is the ongoing tracking of laws of public health importance.

On any given topic in which we think the law is important, we build datasets tracking the characteristics of those laws over time and across jurisdictions so that people know which jurisdiction has got what law and ultimately where the law seems to be a model to follow and, therefore, which jurisdictions need to change their law.

We have used this in a project called cityhealth.org to incentivise change. City health departments identify a group of policies that all big US cities ought to have. We track the adoption and character of those policies across 75 jurisdictions. This has been proven to be a potent way to mobilise cities to change their municipal laws and improve them.

It is our basic premise that there is nothing different about law compared to other social phenomena that are frequently studied. 

We do quantitative legal mapping techniques and the projection of data onto maps and ranking algorithms.

We track the laws of the cities and then, depending on their adoptions of good laws, we give cities medals, like in the Olympics, ranging from gold and silver to bronze. We found that cities try very hard over time to get to gold medals by changing their laws.

Have you seen any movement in India or any other country to have similar systems and processes in place?

Well, the fact is our web designers are from and are based in India. You would be surprised to know that we have not seen any interest from India on this. But India is a place that is well positioned to use it not only because of the sophistication of the software designers but also because it is in many ways a federal system. There are a lot of important state-level and municipal laws. Therefore, there could be studies on comparative laws.

Remember, if we think of law as a pill, an exposure to different laws across a country like India means that people are exposed to different pills. When you have a difference, you have two key things. First, you can use quasi-experimental or natural experiment techniques to find out how well the different laws are working; what effect they are having on health or other outcomes that you care about measuring. Secondly, you could take what you have learned and use it to improve law and improve health in the places that have the laws that are harmful or ineffective.

Presumably, this can be applied in any field apart from health?

Absolutely. In fact, the way I think of this as a public health lawyer is that almost everything is health and almost everything is probably tied to law. If what we are really talking about is research to understand what law does, that means it could not just be a health outcome, it could be an economic outcome; it could be an outcome related to social justice or access to resources or anything else you care about.

MonQcle is very user-friendly. In the backend you get data tables like an Excel table or a CSV file but on the frontend it is just questions and answers that you get from the text.

The lack of research on the impact of law on health really applies to its impact on all sorts of other things as well. We use the law all the time. We treat it as a very important instrument. We can study its effect scientifically and we should because a smart adaptive society is always trying to learn what works and what does not.

In most democratic societies, a new law is treated as a solution to a problem and once legislators have weighed in with a solution, they are very happy to believe they have solved the problem. We tend to assume that they are correct. It could take a decade or more before anybody is willing, politically, to revisit that problem and find that the solution they have got is wrong. That means 10 to 20 years may have elapsed before people even realise that the solution did not work.

So what you are saying is that by scientifically examining which laws have worked or not worked, we can enact more efficient laws; we need not waste so much time, is that correct?

Exactly, and not just whether the law was doing what it was supposed to do but continuing with our pill analogy whether it was having negative side effects.

Take for example the war on drugs. In 1973, the President of the United States announced a ramp up of criminal law strategies to ostensibly reduce drug trafficking and drug use.

There was never really a thorough effort to evaluate how it was working. It took almost 50 years for there to be a cultural realisation that first it was having no positive effect on drug use and drug trafficking and second, it was having terrible social side-effects. We were having mass incarceration of black people, with the rise of a very militarised policing system with police looking more like soldiers with M-16s, bulletproof vests and helmets. That realisation took decades.

We track the laws of the cities and then, depending on their adoptions of good laws, we give cities medals, like in the Olympics, ranging from gold and silver to bronze. We found that cities try very hard over time to get to gold medals by changing their laws.

We should have seen this happening in real-time with research from the start. In a sensible world, after we pass a law, we should immediately begin evaluation research. We should study the implementation of a law and see what is happening; whether it is falling flat from the start.

After a year or two we can start to see health effects in individual jurisdictions and maybe within four or five or six years or a decade we can start to compare across multiple jurisdictions the kind of effects the law has had. That way, we can send back to policymakers and the public actual information about whether this so-called solution is a solution or whether it has negative side-effects. Consequently, a serious effort can be made to think of alternatives.

A decade seems like a long time?

Considering that we have had the failing war on drugs model for more than five decades and only now is there realisation of the serious adverse effects, a decade is not a long time.

You have come up with the book on legal epidemiology with some other authors. Can you tell us a bit about it?

Legal epidemiology, theory and methods, which I edited with Alexander Wagner, a noted traffic law and alcohol researcher, and Rosalie Liccardo Pacula, a leading researcher on drug policy, is actually the second edition of our legal epidemiology research methods textbook.

Its purpose is twofold. First it describes, we hope in understandable terms, how empirical research works for lawyers. We need lawyers to understand research and research processes so they can contribute their expertise about legal data and about how the law works and about what the law is to research teams that may include people who are experts on natural experiments but not particularly specialists in law.

Yet another example of an element of legal epidemiology that is very important is what we call policy surveillance. This is the ongoing tracking of laws of public health importance.

The first part of the book is to bring lawyers up to speed on how research works. The other aim is to show researchers how they can use their general skills in evaluation or implementation to study law.

It is our basic premise that there is nothing different about law compared to other social phenomena that are frequently studied. Law is a collection of institutions and texts and behaviors and beliefs that change things in the world and all those things can be observed scientifically and then studied to determine their effects. The book is a manual for the special tricks of studying law empirically.