Job Description

Design and implement data ingestion, transformation, and movement using Azure Data Factory (ADF), Azure Synapse, and Data Lake.
Collaborate with business stakeholders, data scientists, and engineers to build robust data solutions.
Develop and manage high-volume, real-time, and batch data ingestion pipelines using Azure Data Factory (ADF).
Implement event-driven architectures for real-time data movement and processing.
Develop large-scale data processing solutions using Azure Databricks and Apache Spark with PySpark, Scala, or Python.
Optimize data partitioning, caching, and indexing for efficient performance.
Manage complex transformations and aggregations for structured and unstructured datasets.
Design and implement high-performance data models in Azure Synapse Analytics using dedicated SQL Pools and Spark Pools.
Optimize query performance, workload management, and cost efficiency in Synapse Analytics.
Implement column store indexes, partitioning strategies, and data caching to enhance performance.
Design and manage secure, scalable data lakes using Azure Data Lake Storage Gen2 (ADLS Gen2).
Implement RBAC (Role-Based Access Control), encryption, and data masking to ensure security.
Implement Azure Monitor, Log Analytics, and Application Insights for data pipeline monitoring and troubleshooting.
Optimize cost management, auto-scaling, and performance tuning across Azure services.

Requirements

8+ years of experience in data engineering, ETL development, and cloud-based data integration,
Strong experience with Azure Data Factory (ADF): ETL/ELT pipeline orchestration and data movement.
Azure Databricks: Large-scale data transformation and big data processing with Apache Spark, PySpark, Scala, or Python.
Azure Synapse Analytics: Data warehousing, SQL Pools, Spark Pools, and performance tuning.
Azure Data Lake Storage Gen2 (ADLS Gen2): Secure, scalable data lake architecture.
SQL Server: Advanced T-SQL, stored procedures, indexing, and query optimization.
Power BI, Tableau, or other BI tools for data visualization.
Financial/Banking experience.

Job Tags

Full time,

Similar Jobs

Arona Home Essentials

Delivery Driver Job at Arona Home Essentials

...Nebraska, Illinois, Michigan, Florida, Texas, Colorado, Missouri, Kentucky, Puerto Rico and Indiana. We are currently looking for a Delivery Specialist . You will have access to a comprehensive benefits package that includes: ~ Paid time off including vacation days...

Activate Talent

Video and Photo Editor Job at Activate Talent

Duties and Responsibilities: Edit Video and Photo Content: Edit video and photo content for various marketing channels, ensuring high-quality output and brand consistency. Collaborate with Teams: Work closely with the creative and marketing teams to understand project...

iFlow Inc

Req:: Junior Project Manager _Remote (Irvine OR Palo Alto Only) _ W2 Only Job at iFlow Inc

...Job ID: 2909 Role- Junior Project Manager Location- This role is 100% remote, but only candidates local to Irvine and Palo Alto CA for any future conversions. Duration- Long Term JOB DESCRIPTION Schedule: M-F 8-6/9-6 PT (with flex depending...

Pro Talent Crafter

UKG Project Manager :: REMOTE Job at Pro Talent Crafter

...End Client: Fashion/Apparel Duration: 12 months Mode of Interview: Video Location: REMOTE UKG Project Manager Role: An experienced project manager with knowledge in UKG methodology and projects and HR systems. Responsibilities Manages...

JBS

PRODUCTION SUPERVISOR 2ND SHIFT Job at JBS

...applicable state and federal requirements are met regarding OSHA, food safety, and fair employment practices in assigned area of... ...more years of previous Supervisory experience.- Poultry or food industryexperiencesa plus.- Previous leadership skills with 20 or more...

Sr Azure Data Engineer Job at Zapcom, Dallas, TX

clRRVUtCdnRpRzRxSnU0Z2UwK2M2TEdSVGc9PQ==