- W-2 Open Positions Need to be Filled Immediately. Consultant must be on our company payroll, Corp-to-Corp (C2C) is not allowed.
Job ID:
J36993
Specialized Area:
Python
Job Title:
Python Data engineer
Location:
San Francisco,CA
Duration:
8 Months
Domain Exposure:
Healthcare, Retail, Education, IT/Software
Work Authorization:
US Citizen, Green Card, OPT-EAD, CPT, H-1B,
H4-EAD, L2-EAD, GC-EADClient:
To Be Discussed Later
Employment Type:
W-2 (Consultant must be on our company payroll. C2C is not allowed)
Immediate need for a senior data engineer who can also do some data science work. Experience in Biomedical space is desired. Experience in setting up BigData, Datascience infrastructire on AWS is also desired. Automated setup for EKS, PySpark, AWS Glue, EMR and Sagemaker based environment. Setup security for Redshift, S3, and EC2 instances
Responsibilities
Assemble large, complex data sets in the format fit for each use case
Write generic Python/Pyspark modules for processing data from various data sources (XML, Parquet, CSV, Relational)
Demonstrable experience architecting, developing and optimizing ETL pipelines using Python, Spark, EMR, Docker, Kubernetes and Airflow
Develop and optimize big data pipelines for data scientists (requires a basic understanding of data science concepts and ML)
Research and recommend new innovative methods and systems to manage data for business improvement
Participate in internal governance to drive the data quality business cycle and roadmap Required Skills
Python, Spark, ETL/Data engineering, Docker/Kubernetes, automation/devops related experience in AWS.
Development and management of Airflow based data flows
Bachelor’s or Master’s degree in computer science or software engineering
3+ years of programming experience (including functional programming); must be advanced in Python Experience building and optimizing big data pipelines using Spark Experience with AWS cloud services: S3, EC2, EMR, RDS, Redshift, Glue, Lambda, EKS, Sagemaker
Experience with relational SQL and NoSQL databases, including Postgres Solid understanding of how to design robust data workflows including optimization and user experience
Strong analytical and problem-solving skills Excellent oral and written communication skills Able to work in teams and collaborate with others to clarify requirements
Strong co-ordination and project management skills to handle complex projects Experience developing and working with XML, JSON, and external web services Preferred Qualifications
Clinical drug development domain knowledge Experience working with clinical and biomedical data types (clinical patient data, omics, imaging, etc.)
Apply Now
QUANTUM TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. QUANTUM TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will QUANTUM TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract