Mike Maltz writes the following on ethnography and statistics:
I got interested in ethnographic studies because of a concern for people analyzing data without an understanding of its origins and the way it was collected. An ethnographer collects stories, and too many statisticians disparage them, calling them “anecdotes” instead of real data. But stories are important; although they only give you single data points, if you have a number of them you can see how different they seem to be. That helps you determine if there is more than one process generating the data. in fact, I’ve noted that the New York Times now often augments an article with a bunch of stories about how different people were affected by the focus of the article.
Here’s the way I [Maltz] described the benefit of ethnography in the introduction to a book (Doing Ethnography in Criminology, Springer 2018) that Steve Rice and I edited:
I’m best known for quantitative, not qualitative, research. An engineer by training, I had never taken any courses in social science when I began teaching in a criminal justice program in 1972. My entire criminal justice experience up to that point was based on my having been a staff member of the National Institute of Justice from 1969 to 1972, and I was hired by NIJ because of my engineering background and my experience in police communications—see below.
My introduction to social science and to the research techniques that were then used by its practitioners began when I joined the criminal justice faculty of the University of Illinois at Chicago. I was put to work teaching social science statistics and, not knowing much about it, used the books that others used before me. But even then I was mystified by the common practice of looking to achieve a low p-value as the be-all and end-all of such research. Untenured and with no experience in the field, I taught what others thought was important. But I soon wised up, and described my concern about the methods used, some 30 years ago (Maltz, 1984, p. 3):
“When I was an undergraduate in engineering school there was a saying: An engineer measures it with a micrometer, marks it with a piece of chalk, and cuts it with an axe. This expression described the imbalance in precision one sometimes sees in engineering projects. A similar phenomenon holds true for social scientists, although the imbalance is in the opposite direction. It sometimes seems that a social scientist measures it with a series of ambiguous questions, marks it with a bunch of inconsistent coders, and cuts it to within three decimal places. Some balance in precision is needed, from the initial measurement process to the final preparation of results.”
And I further expressed my concern about the focus on “statistical significance” in a subsequent article (Maltz, 1994). Ethnography is a welcome and much-needed departure from that type of research. It deals with individual and group behavior that doesn’t conform to statistical or spreadsheet analysis. Yes, an ethnography may just be a single data point, but it often serves as a marker of importance, an exploration of what additional factors should be considered beyond the usual statistics, or as a counterexample to some of the more positivist studies.
In this regard, three examples provide additional context to my strong belief in the need for a qualitative orientation. The first was my initial experience while consulting on police communication systems (true electrical engineering!) for the Boston Police Department from 1966 to 1969. To satisfy my curiosity about the ways of the police, I requested, and was granted, permission to conduct an “experiment” on police patrol. The number of patrol cars in one police district was doubled for a few weeks to see if it had any effect on crime. And it did: compared to the “control” district, which had no arrests, the “experimental” district had six arrests. Moreover, there were no arrests at all for the same time period in either district in the previous area, so I could calculate that p = 0.016, much less than 0.05. What a finding! Police patrol really works!
On debriefing one of the arresting officers, one of the first lessons I learned was that police officers are not fungible. There are no extra police officers hanging around the station that can be assigned to the experimental district: they have to be drawn from somewhere else. The additional officers, who made all of the arrests, were from the BPD’s Tactical Patrol Force—the Marines of the department—who were normally assigned to deal with known trouble spots, and the two districts selected for the study were generally low-crime areas.
In fact, the TPF officers already knew that a gang of car thieves/strippers was active in the experimental district and decided to take them out, which resulted in all of the arrests they made. They couldn’t wait to get back to working citywide, going after real crime, but took the opportunity to clean up what they considered to be a minor problem. So after that experience, I realized that you have to get under the numbers to see how they are generated or, as I used to explain to students, to “smell” the data.
Another example: Some years ago I was asked to be an expert (plaintiff’s) witness in a case in the Chicago suburbs, in which the defendant suburb’s police department was accused of targeting Latino drivers for DUI arrests to fill their arrest quotas. My job was to look at the statistical evidence prepared by another statistician (the suburb’s expert witness) and evaluate its merits. I was able to show that there were no merits to the analysis (the data set was hopelessly corrupted), and the case was settled before I had a chance to testify.
What struck me after the settlement, however, was the geography and timing of the arrests. Most of them occurred on weekend nights on the road between the bars where most of the Latinos went to drink and the areas where they lived. None were located on the roads near the Elks or Lions clubs, where the “good people” bent their elbows.
I blame myself on not seeing this immediately, but it helped me to see the necessity in going beyond the given data and looking for other clues and cues that motivate those actions that are officially recorded. While it may not be as necessary in some fields of study, in criminology it certainly is.
A third example was actually experienced by my wife, who carried out a long-term ethnographic study of Mexican families in Chicago (Farr, 2006) all of whom came from a small village in Michoacán, Mexico. Numerous studies, primarily based on surveys, had concluded that these people were by and large not literate. One Saturday morning in the early 1990s, she was in one of their homes when various children began to arrive, along with two high school students. One of the students then announced (in Spanish, of course), “Ok, let’s get to work on the doctrina (catechism),” and slid open the doors on the side of the coffee table, revealing workbooks and pencils, which she distributed to the kids.
On another occasion, my wife was drinking coffee in the kitchen when all of the women (mothers and daughters) suddenly gathered at the entrance to the kitchen as someone arrived with a plastic supermarket bag full of something—which turned out to be religious books (in Spanish) on topics such as Getting Engaged and After the Children Come Along. Each woman eagerly picked out a book, and one of them said, “I am going to read this with my daughter.”
Clearly these instances indicate that children in the catechism class and the women in the kitchen were literate. The then-current questionnaires that evaluated literacy practices, however, asked questions such as “Do you subscribe to a newspaper? Do you have a library card? Do you have to read material at work?” In other words, the questionnaires (rightly so) didn’t just ask people outright “Can you read?” but rather focused on the domains they thought required reading. Yet no questions dealt with religious literacy, since literacy researchers at the time did not include a focus on religion. The result? The literacy practices of these families were “invisible” to research.
These anecdotes are but three among many that turned me off the then-current methods of learning about social activity, in these cases via (unexamined) data and (impersonal) questionnaires. Perhaps this has to do with my engineering (rather than scientific) background, since engineers deal with reality and scientists propound theories. To translate to the current topic, it conditioned me to take into consideration the social context, a recognition that context matters and that not all attributes of a situation or person can be seen as quantifiable “variables.” This means, for example, that a crime should be characterized by more than just victim characteristics, offender characteristics, time of day, etc. and that an individual should be characterized by more than just age, race, ethnicity, education, etc. or “so-so” (same-old, same-old) statistics. These require a deeper understanding of the situation, which ethnography is best suited, albeit imperfectly, to do—to put oneself in the position, the mindset, of the persons whose actions are under study.