Like many people who grew up reading science fiction, I idolized Hari Seldon. For those unfamiliar with Isaac Asimov’s Foundation series, Seldon invents “psychohistory”, a science that allows him to predict the future of humanity through probabilistic analysis. These days, Seldon might be called a computational sociologist — or a data scientist.
Early in my own career as a data scientist, I made a connection between data science and the social sciences. I observed that, while many data scientists start off in physics or other natural sciences, the most successful ones seem to have a robust grounding in the social sciences. They may not be academically trained as social scientists. But whether through classes, reading, or simply paying attention to the world around them, they’ve acquired a working knowledge of economics, psychology, and sociology.
Like Seldon, good data scientists recognize that people are not particles. People have agency, which makes reasoning about human behavior — and the data generated by that behavior — quite different from reasoning about most other phenomena in the natural world. Predicting human behavior requires an ability to model motivations and decision-making processes. People may also become aware that they’re the subjects of observation or experimentation, and this awareness changes their behavior.
If you’re thinking about becoming a data scientist, I strongly encourage you to round yourself out as a social scientist if you haven’t done so already. Read a textbook on traditional economics as well as one on behavioral economics. Similarly, learn about historical and modern perspectives on psychology and sociology.
Conversely, if you’re hiring data scientists whose job is to understand and predict human behavior, look for people with that least a basic grounding in the social sciences.
We may not all be able to predict the fall of the Galactic Empire and plan for its rebirth. But we can all do better than treating people as particles.