Machine Learning Team - Yousef's Notes
Machine Learning Team

Machine Learning Team

#Approaches

#ML Data Scientists + Software Engineers

  • Specialization of each team member ML scientists devise effective models for the given goal(s); software engineers implement efficient and maintainable code.
  • Well-defined skillsets with a common language. ML scientists know ML theory/techniques and software engineers know ML frameworks and SE principles/patterns/best practices.
  • Everyone understands what is needed but not how it is done.

#ML Software Engineers

  • Scientists are hard to integrate with software engineering teams.
  • Scientists may devise solutions that are impractical and cannot be effectively executed in the production environment.
  • Scientists don’t usually write efficient, well-structured code.
  • Software engineers have to rewrite models into production code, possibly incurring in overheads and additional costs.

#Skills and Roles

#Data Engineers

  • Software engineers that extract, transform, and load (ETL) data.
  • Create an automated pipeline, in which raw data is transformed into analysis-ready data.
  • Design how to structure the data and how to integrate them from various resources.
  • Write on-demand queries or fast application programming interfaces (APIs) to make data easily accessible by analysts and other data consumers.
  • Not expected to know any machine learning.

#Data Labelers

  • Manually or semi-automatically assign labels to unlabeled examples according to the specification provided by data analysts.
  • Building labeling tools.
  • Manage outsourced labelers
  • Validate labeled examples for quality.

#Domain Experts

  • Work closely with scientists and engineers
  • Contribute to decide about model inputs, outputs, features and prediction objectives.
  • Contribute to feature engineering by indicating how data relates to business decisions.
  • Contribute to translate a business problem into a ML by indicating what the clients pay for and what is a deal-breaker for them.

#DevOps Engineers

  • Work closely with machine learning engineers to automate model deployment, loading, monitoring, and occasional or regular model maintenance.
  • Smaller companies and startups: may be part of the ML team, or an ML engineer could be responsible for devops.
  • Big companies: work in larger Devops team and are employed in ml projects.
  • MLOps: deploy machine learning models in production, upgrade those models, and build data processing pipelines involving ml models.