How to Build a Data Science Team

Dean Jain

Senior Staff Software Engineer · Enterprise AI, Data & Cloud Architect

November 6, 2017 · 5 min read

Data ScienceLeadershipTeam Building

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart LR
    M["🧭 DS Manager<br/>recruit · prioritize · interface up"]:::gov
    M --> S["🔬 Data Scientist<br/>pull · clean · model · communicate"]:::server
    M --> E["🛠️ Data Engineer<br/>ingest · scale · productionize"]:::obs
    S <-->|"hand-off both ways"| E
    classDef gov fill:#E0D6F5,stroke:#9B7EDE,stroke-width:2px,color:#0F172A
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A
    classDef obs fill:#AED6F1,stroke:#2E86C1,stroke-width:2px,color:#0F172A

Figure 1: Three roles, one loop. The scientist finds the insight, the engineer makes it real, the manager clears the path.

Most “data science team” failures aren’t talent problems. They’re composition problems. A cluster of brilliant scientists with nobody to productionize their work, or an engineering team with nobody asking the right questions. A team that ships is built from three distinct roles, hired through a process that tests the actual job, against a definition of success you wrote down before you started. Here’s how to assemble one.

TL;DR

Three roles, not one. Data Engineer (scale + productionize), Data Scientist (analyze + model + communicate), Data Science Manager (recruit, prioritize, interface upward). This is the split I use; missing any one creates a predictable failure mode.
Hire by doing, not by quizzing. A real data challenge on real-ish data predicts performance far better than algorithm trivia.
Communication is a core skill, not a bonus. A model nobody can act on is worthless, so screen for the ability to explain findings to non-experts.
Define success before you begin. Specific metrics over vague ones, and prove you followed a sound process not just that the number moved.
Good metrics are consistent, cheap, and quick. If you can’t measure results within a week for free, you’re probably tracking the wrong thing.

1. The three roles

A functioning team covers the full lifecycle from raw data to a decision that sticks and no single person does all of it well. Three complementary roles:

Data Engineer brings scale. Owns database ingest and extraction, builds the pipelines, and productionizes ML algorithms so a notebook becomes a system. Heavy on infrastructure skills. Without this role, insights never leave the laptop.
Data Scientist brings insight. Pulls and cleans data, runs the statistics and experiments, builds the models, and communicates the results. Without this role, you have plumbing and no answers.
Data Science Manager brings momentum. Builds and recruits the team, ensures the scientist and engineer hand off cleanly, keeps work prioritized and moving, and interfaces with management. Without this role, a talented team stalls on friction and unclear priorities.

The interesting failures live in the gaps between roles. Scientists who can’t hand work to engineers ship nothing. Engineers without scientists optimize the wrong things. Both without a manager drift from what the business actually needs. When you hire, you’re not collecting individual stars you’re making sure the loop in Figure 1 is closed.

2. Hire by doing, not by quizzing

Whiteboard algorithm trivia tells you almost nothing about whether someone can wring a decision out of messy data. The most predictive interview process makes candidates do the actual job on realistic data. A proven shape (popularized by Airbnb’s data team):

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart LR
    A["📞 Phone screen<br/>data-driven experience"]:::obs --> B["📊 Basic data challenge<br/>validates the resume"]:::gate
    B --> C["🧪 In-house data challenge<br/>real data, broad question, a day"]:::server
    C --> D["🗣️ Present findings<br/>detail · rigor · action · clarity"]:::warn
    D --> E["🤝 Partner &amp; values interviews"]:::good
    classDef obs fill:#AED6F1,stroke:#2E86C1,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef server fill:#A8E6D0,stroke:#2FA37C,stroke-width:2px,color:#0F172A
    classDef warn fill:#FFE6A8,stroke:#E0A106,stroke-width:2px,color:#0F172A
    classDef good fill:#BFEFC8,stroke:#3FA34D,stroke-width:2px,color:#0F172A

Figure 2: A hiring loop that tests the job, culminating in a real data challenge rather than algorithm trivia.

The heart of it is the in-house data challenge. Sit the candidate with the team, give them real data and a broad question, and a day to attack it however they like.

Questions encouraged, lunch included, and periodic check-ins so they don’t stall on something trivial. It’s transparent in both directions: they see what working with you is actually like, and you see how they really think.

When they present, four signals matter more than any clever technique:

Eye for detail did they investigate the data they relied on, or trust it blindly?
Rigor if they built a model, are the results sound?
Action-orientation what would you actually do with what they found?
Communication can they make it land with people who weren’t in the weeds?

That last one isn’t a soft add-on. A finding nobody can act on is a non-finding. The ability to explain it clearly is part of the core competency, so screen for it as hard as you screen for modelling.

3. Define success before you start

A team without a definition of success will always find a number that looks good. Set the target first:

Specific metrics over vague ones. “Improve engagement” is a wish. “Lift 7-day retention by 2 points” is a target you can be wrong about.
Judge the process, not just the outcome. Did you follow a sound method to get there, or get lucky? A good outcome from a broken process won’t repeat.
Good metrics are consistent, cheap and quick to collect. A simple rule of thumb: if you can’t measure results within a week, for free, and replicate the process, you’re prioritizing the wrong ones.

---
config:
  theme: dark
  fontSize: 17
  themeVariables:
    fontFamily: "Comic Sans MS, Comic Neue, Chalkboard SE, cursive"
---
flowchart LR
    M["📐 Candidate metric"]:::gov --> A{"Consistent?"}:::gate
    A -->|"yes"| B{"Cheap to collect?"}:::gate
    B -->|"yes"| C{"Quick (&lt; 1 week)?"}:::gate
    C -->|"yes"| G["✅ Keep it"]:::good
    A -->|"no"| X["❌ Drop it"]:::danger
    B -->|"no"| X
    C -->|"no"| X
    classDef gov fill:#E0D6F5,stroke:#9B7EDE,stroke-width:2px,color:#0F172A
    classDef gate fill:#D7C3F2,stroke:#8E5BD0,stroke-width:2px,color:#0F172A
    classDef good fill:#BFEFC8,stroke:#3FA34D,stroke-width:2px,color:#0F172A
    classDef danger fill:#FFB3B3,stroke:#D14545,stroke-width:2px,color:#0F172A

Figure 3: A filter for success metrics. If it isn't consistent, cheap and quick, drop it.

This discipline is what keeps a data science team from drifting into expensive, unfalsifiable work. The definition of success is the leash that keeps the team pointed at decisions that matter.

Why it matters: building a data science team is a composition problem, a hiring-signal problem, and a measurement problem in that order. Get the three roles in balance, hire people by watching them do the real work, and write down what winning looks like before you begin. Do that and the modeling the part everyone obsesses over mostly takes care of itself.

TL;DR

1. The three roles

2. Hire by doing, not by quizzing

3. Define success before you start

Further reading