Senior Data Engineer(Databricks and Azure) job opportunity at Unison Group.

Home Jobs In Malaysia Tag: Job Senior Data Engineer(Databricks and Azure) position at Unison Group


Date2025-12-11T02:58:11.950Z bot
Unison Group Senior Data Engineer(Databricks and Azure)
Experience: 8-years
Pattern: Full-time
Country:
apply Apply Now
Salary:
Status:

Job

Copy Link Report
degreeGeneral
loacation Kuala Lumpur, Malaysia
loacation Kuala Lumpur....Malaysia

Overview We are looking for a Senior Data Engineer with strong expertise in Databricks, PySpark, Delta Lake, and cloud-based data pipelines . The ideal candidate will design and build scalable ETL/ELT solutions, implement Lakehouse/Medallion architectures, and integrate data from multiple internal and external systems. This role requires strong technical leadership and hands-on architecture experience.   Key Responsibilities Design, build, and optimize data ingestion and transformation pipelines using Databricks, PySpark, and Python. Implement Delta Lake and Medallion architecture for scalable enterprise data platforms. Develop ingestion frameworks for data from SFTP, REST APIs, SharePoint/Graph API, AWS, and Azure sources . Automate workflows using Databricks Workflows, ADF, Azure Functions, and CI/CD pipelines . Optimize Spark jobs for performance, reliability, and cost efficiency. Implement data validation, quality checks, and monitoring with automated alerts and retries. Design secure and governed datasets using Unity Catalog and cloud security best practices. Collaborate with analysts, business users, and cross-functional teams to deliver curated datasets for reporting and analytics. Provide technical leadership and guidance to junior team members.   Required Skills 5–8+ years of experience in Data Engineering . Strong hands-on experience with Databricks, PySpark, Delta Lake, SQL, Python . Experience with Azure Data Lake, ADF, Azure Functions , or AWS equivalents (S3, Lambda). Experience integrating data from APIs, SFTP servers, vendor data providers, and cloud storage . Knowledge of ETL/ELT concepts, Lakehouse/Meddalion architecture , and distributed processing. Strong experience with Git, Azure DevOps CI/CD , and YAML pipelines. Ability to optimize Spark workloads (partitioning, caching, Z-ordering, performance tuning).   Good to Have Exposure to Oil & Gas or trading analytics (SPARTA, KPLER, IIR, OPEC). Knowledge of Power BI or data visualization concepts. Familiarity with Terraform , Scala, or PostgreSQL. Experience with SharePoint development or .NET (optional).

Other Ai Matches

Full Stack Developer (Vue JS & Node JS & .Net) Applicants are expected to have a solid experience in handling Job related tasks
Avaloq Business Analyst Applicants are expected to have a solid experience in handling Job related tasks
QA Tester (Web Applications - Arabic & English) Applicants are expected to have a solid experience in handling Job related tasks
Performance tester Applicants are expected to have a solid experience in handling Job related tasks
PMO(Finance) Applicants are expected to have a solid experience in handling Job related tasks
SAP BTP Specialist (Application & Integration) - Remote working Applicants are expected to have a solid experience in handling Job related tasks
APIC Developer Applicants are expected to have a solid experience in handling Job related tasks
SAP Cloud Migration Specialist Applicants are expected to have a solid experience in handling Job related tasks
Business Analyst Applicants are expected to have a solid experience in handling Job related tasks
RPA Automation Engineer (UiPath + Selenium) Applicants are expected to have a solid experience in handling Job related tasks
PowerBuilder Developer Applicants are expected to have a solid experience in handling Job related tasks
System Analyst/ L3 Production Support(Java & Oracle ) - Banking Applicants are expected to have a solid experience in handling Job related tasks
L2 Application Support (Unix Shell Script, Sql) - Banking Applicants are expected to have a solid experience in handling Sql) - Banking related tasks
Project Manager (Enterprise Budgeting & Financial Consolidation) Applicants are expected to have a solid experience in handling Job related tasks
Workday senior Analyst Applicants are expected to have a solid experience in handling Job related tasks
Senior Software Engineer - Core Applications (Life Asia) Applicants are expected to have a solid experience in handling Job related tasks
DC Operator (with Facilities managment experience) Applicants are expected to have a solid experience in handling Job related tasks
L1-L2 Application Support (Unix Shell Script, Sql) Applicants are expected to have a solid experience in handling Sql) related tasks
DevOps Engineer - Trade Finance Applicants are expected to have a solid experience in handling Job related tasks
Manufacturing product Design Engineer (Systems Analyst) Applicants are expected to have a solid experience in handling Job related tasks
MS CRM System Analyst/Business Analyst Applicants are expected to have a solid experience in handling Job related tasks
Site Reliability Engineer (SRE)/Devops Engineer Applicants are expected to have a solid experience in handling Job related tasks
Senior Project Manager - Operational Transformation & Digitization Applicants are expected to have a solid experience in handling Job related tasks