fp.careers

Functional Programming Jobs

ML Core Scala/Spark Engineer: Automated Feature Engineering

dotData

  • scala
  • spark
  • machine-learning
  • concurrency
  • sql

Quick Facts

Company
dotData
Location
San Mateo, CA, US
Employment
Full-time
Type
On-site

Job description

Job overview


dotData is hiring high caliber engineers who are excited to democratize Machine Learning with automation. You will work on dotData’s proprietary core engine components: AI-powered feature engineering and AutoML. There’s a lot of interesting stuff to learn and do in building out our high performance parallel distributed platform whether you’re a distributed systems coder, a Machine Learning expert, or both.



  • Our AI-powered Feature Engineering discovers and evaluates millions of combinations of joins and aggregations of relational data. To explore them efficiently we built our own distributed in-memory engine on top of Apache Spark. Skewed keys, wide tables, exploding number of records after join - we have to take care of all of that.

  • Our advanced AutoML component automatically explores and evaluates state-of-the-art ML models with automated hyper-parameter optimization and model selection. Logical plan is divided into computationally-intensive jobs that are executed in parallel with strict resiliency requirements. We take advantage of industry-standard machine learning libraries, like sklearn, XGBoost, LightGBM, TensorFlow, and Pytorch.


If you’re looking to build amazing products with a smart bunch of colleagues and blow the minds of customers this is the place.

Apply now