About the Role:
We are seeking a skilled and motivated Data Engineer to join our team to work on AI Projects. The ideal candidate will have experience designing, building, and maintaining scalable data pipelines and architectures to support analytics and business intelligence solutions.
Key Responsibilities:
- Design, build, and maintain efficient, scalable, and reliable data pipelines.
- Develop ETL/ELT processes to collect, transform, and store structured and unstructured data from various sources.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
- Ensure data quality, integrity, security, and availability across systems.
- Optimize performance of data platforms and troubleshoot any data-related issues.
- Create and maintain data architecture documentation.
- Work with cloud-based data platforms (GCP) and modern data tools.
- Implement data governance and best practices in metadata management.
- Coach and mentor team members
Required Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field (Master’s preferred).
- 5+ years of experience in data engineering or similar role.
- Proficient in SQL and programming languages such as Python.
- Hands-on experience with ETL frameworks and tools.
- Experience with cloud data platforms such as Google BigQuery and Google Dataflow.
- Familiarity with data modeling concepts and database design (relational and NoSQL).
- Knowledge of big data technologies like Spark, Hadoop, Kafka, Google Cloud Pub/Sub is a plus.
- Experience with CI/CD and version control (e.g., Git).
Preferred Skills:
- Experience with containerization (Docker, Kubernetes).
- Understanding of data privacy regulations (GDPR, HIPAA, etc.).
- Familiarity with data cataloging and lineage tools.
- Strong analytical, problem-solving, and communication skills.
- Exposure to Healthcare data standards including EDI, X12, FHIR, etc.