The different types of MLEs
I've spent a lot of time figuring out what exactly MLEs (Machine Learning Engineers) do, even though I've been doing MLE work for almost 2 years now. I've come to realize that there's no cookie-cutter answer—every day is different, no matter what your specialty is. For the curious software engineer though, here are the 4 types of MLEs I've observed, trying to fit this ambiguous role into some sort of template:
The four different types of MLEs (from my experience)
1. Data focused MLE
If you're working on an ML service, you're likely dealing with big data. While you might not be responsible for populating tables (data teams usually handle that), you definitely know how to navigate data sources and understand how the data aligns with business requirements.
Almost every MLE needs to know how to ingest, clean, and prepare data for ML services.
The data focused MLE works on:
- building and maintaining data pipelines
- ensuring data freshness and quality in their ML pipeline
- writing ETL jobs (think spark, airflow, step functions, etc.)
2. Infra focused MLE
Without the proper infrastructure, your ML system will not survive production traffic unless you are working on a toy project with no users.
Depending on your business requirements, you will need to make key decisions about the infrastructure for your ML model that directly affects how your customers experience your application and how your model performs predictions.
For advertising for example, latency is very important as ads should be available in near real time and any delays in ad data freshness will lead to "stale" impressions where the relevance of the ads are low for users. In this case, you need to build a real time inference service that holds your trained ML model that is long running. This leads to you deciding to use a long running ECS with a large instance size so that you can load things on memory in order to perform inference very fast with any artifacts being readily available on disk.
For a streaming TV service you might only need to have fresh recommendations based on user activity every few days or even weeks. This calls for a batch inference service which calls for reliable offline jobs that run on some recurring schedule but gets rid of the need for a long running service.
This is the focus that I've been in for a while and I truly love it. It requires strong distributed compute knowledge and although you don't technically need to know the science specifics of the actual models, the strongest infra MLEs have strong foundational ML knowledge.
3. Model focused MLE (applied scientist)
In the big tech world you will often call MLEs that work on core modeling "applied scientists" and these are the MLEs that most people think about when they think of people working in ML.
In reality, applied scientists are yet just another subgroup of people working in ML that also have a strong grounding on software engineering skills with applied ML knowledge.
They may work on novel papers for their teams, but for the most part the true goal of an applied scientist is to create revenue with their ML models.
I've seen most model focused MLEs not really work on "productionizing" ML services and instead focus on doing offline analysis on their experiments in order to maximize revenue or other important metrics (like CTR, relevance, etc.).
These MLEs are often VERY up to date on the latest research being done in the topic of their team's specialty so if you're on a robotics team then expect to be really up to date with DRL while if you're on an advertising team be ready to be at the edge of NLP and LLM research.
The applied scientist might work on:
- offline analysis in notebooks (sagemaker, jupyter, etc.)
- prototyping quickly and often without always writing production grade code
- sometimes contributing to papers for the company
- driving revenue gains with their carefully constructed models
- frequent experiments and sharing results with shareholders
4. Full-stack MLE
Finally, this is a new term I am hoping to coin for those MLEs that seem to do everything from the data cleaning to the modeling to the scaling.
I interviewed with a startup a while back and the role required me to work on everything as I would've been the only MLE on the team.
This seems to be more common in the startup world and in general I would say that the full stack MLE prioritizes making things work.
Making things work is a very software engineer / indie hacker - centric concept but even for the MLE there are times when you just really need to make shit work for your team.
I would say that if the other categories of MLEs go super deep, the full-stack MLE has the widest breadth of knowledge across the full ML stack.
Some readings for you:
I will just leave you with one thing for the weekend that might pique your interest:
The resource above gives a good overview of how broad the end to end ML workflow is.
If you like content like this, check out my MLE newsletter which focuses on real experiences from being a MLE: list.raymondyoo.com