Data Engineer Job Description template and pdf with duties [2024]

Job summary 1

The Data Engineer will support the business intelligence and data science strategies and related initiatives that deliver comprehensive data models for actionable data reporting and analysis for commercial, functional, business leaders, and team members.

Duties & Responsibilities 1

special offer

Download PDF with all 10 Data Engineer Job Descriptions

100Hires
eBook

Free Guide:

The Ultimate guide
to recruiting

Work in collaboration with infrastructure and IT team to design, create and maintain optimal data pipeline architecture and data structures for Commercial Data Lake/Data Warehouse.
Assemble large, complex data sets from business and reporting systems such as SAP, Business Objects, CRM, and Cognos/EDW that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Databricks, and AWS technology.
- Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data analytics needs.
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Streamline existing and introduce enhanced reporting and analysis solutions that leverage complex data sources derived from multiple internal systems.

Requirements & Skills 1

Minimum 2-3 years of experience in a Data Engineer role, who has attained a degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field.
Must have SQL knowledge and experience working with relational databases, query authoring (SQL) as well as familiarity with a variety of databases including Redshift
Must have experience with Python or a similar programming language
Experience with Databricks, Spark, or similar tools
Experience with at least one data pipeline and ETL tools like Databricks, Informatica, AWS Glue, and Talend.
Experience working with AWS cloud services: EC2, S3, RDS, Redshift.
- Experience creating Logical and Physical data models and documenting ERD
- Experience with Data mining, Data Science, and Predictive analytics is a plus.
- Experience with visualization tools like Power BI, Tableau.

Job summary 2

The Data Engineer will be responsible for supporting the development of a data factory pipeline within a cloud environment. The Data Engineer will work closely with Software Engineers, DevOps, and Data Analysts. They must be comfortable working both independently and as part of the larger data factory team in a complex, fluid environment.

Duties & Responsibilities 2

Develop and maintain optimal data pipeline architecture - including development related to data acquisition and monitoring, data quality, integration, normalization, and analytics development.
Adhere to good code practices within an agile environment with a DevOps approach to development and implementation.
Develop/design appropriate orchestration and structures supporting data transformation, metadata, code dependency, and workload management.

Requirements & Skills 2

Job summary 3

In this role you will work cross-functionally to create the data products that internal stakeholders and customers depend on to inform their decision-making. You will be diving deeply into understanding our data and transforming it into clean, efficient data models that power Data Science and Analytics platforms.

Duties & Responsibilities 3

Collaborate with business stakeholders to develop new data models and pipelines
- Become our data subject matter expert
- Drive improvements to data quality across the data pipelines
- Contribute broadly to data engineering development work

Requirements & Skills 3

BS/MS in Computer Science, Engineering, or related degrees
3+ years of experience in Data Engineering or Data Analytics
Large scale database experience; Snowflake, BigQuery, Redshift
Experience writing SQL and ETL pipelines
- Data modeling experience
- Software engineering mindset with the ability to write clean, maintainable code
- Ability to communicate effectively with both technical and business partners

Job summary 4

You will be supporting an internal big data analytics application that will be hosted on a cloud platform. You will have prior experience working in an agile/DevOps environment. You will have the opportunity to work with some of the newest technologies, and transformative projects, and play a strong role in shaping the innovation agenda across the company.

Duties & Responsibilities 4

Provide database design, development, and implementation support
Participate in daily scrum meetings and support the development team
Work with customers and team members in an Agile development environment
- Follow Continuous Integration/Continuous Delivery (CI/CD) best practices for code build and deployments
- Develop PL/SQL, stored procedures, and ETL scripts to support this analytics application
- Document database design and develop optimum data ingest techniques from multiple data sources

Requirements & Skills 4

Minimum Requirements: Bachelor's Degree in Computer Science, IT, or related field & minimum 5-7 years of QA experience
Minimum experience of 5 years with relational databases such as Oracle, SQL Server, Sybase, RedShift
Minimum 3+ years of experience with ETL technologies

Job summary 5

We are seeking a Data Engineer. As a member of the IT department, you will work with big data applications and new developments to meet the firm’s evolving needs. You will convert business requirements into working software that adheres to our architectural standards for multiple business processes.

Duties & Responsibilities 5

Take part in development projects
Partner with business stakeholders and technology partners to translate business requirements into efficient and robust technology solutions
Adhere to existing policies and procedures including SDLC, architecture standards, information security, and compliances
Learn new tools and expand big data functionalities by implementing additional Hadoop and Spark ecosystem components to meet evolving business needs
- Develop training materials and provide training to internal IT and business users
- Evaluate new releases and features of the platform, provide recommendations on upgrade and impact analysis
- Provide after-hour system and application support as needed

Requirements & Skills 5

Bachelor’s degree or equivalent in MIS, Computer Science, Engineering, or related discipline
3+ years of experience as a Cloudera/Hadoop/Spark Developer
Cloudera or Hortonworks Developer Certification (CCP Data Engineer, CCA Spark and Hadoop Developer or Hortonworks Certified Developer)
Knowledge of Linux and Windows servers, bash shell scripting
Knowledge of relational database structures, theories, and practices
Demonstrated experience in data modeling, design, and implementation of adding new datasets into the cluster
- Cloudera Data Platform (CDP), Sqoop, Nifi, Hive, HiveQL, Impala, Ranger, Spark, and Spark SQLCloud development using Microsoft Azure
- One or more languages: Java, Scala, or Python
- CI/CD, Kafka, tokenization, or Machine Learning is a plus

Job summary 6

As a Data Engineer, you will be responsible for all aspects of the software development lifecycle, including design, coding, integration testing, deployment, and documentation. You will create and maintain new data applications relying heavily on experience and judgment to plan and accomplish goals.

Duties & Responsibilities 6

Design & Develop custom data warehouse solutions that enable leadership and general users to make informed data-driven decisions and address the high demand for data available in the enterprise.
Responsible for creating work estimates, technical project plans, and implementation plans
Research emerging development technologies, products, and processes
- Work collaboratively within an agile project team
- Follow best practices and coding standards
- Grow your skillset as well as the skills of the team

Requirements & Skills 6

Job summary 7

We are looking for passionate Data Engineers who are looking to tackle challenges and build solutions.

Duties & Responsibilities 7

Create and maintain optimal data pipeline architecture
- Assemble large, complex data sets that meet functional / non-functional business requirements
- Build the infrastructure required for optimal extraction transformation and loading of data from a wide variety of data sources,
- Work with stakeholders including Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs

Requirements & Skills 7

Experience building and optimizing data pipelines, architectures, and data sets
A successful history of manipulating, processing, and extracting value from large disconnected datasets
Experience in creating reports and dashboards in Tableau

Job summary 8

We are now looking for an experienced Data Engineer to join our Data Engineering team.

Duties & Responsibilities 8

Prototype, build, deploy, and manage data engineering pipelines.
Contribute to the design and creation of high-quality solutions.
Work with other data engineers, business intelligence, and machine learning experts to solve real-life, challenging business problems.

Requirements & Skills 8

Degree in computer science or related.
3+ years of relevant professional experience.
Extensive experience in Python and SQL
Experience with batch and real-time data processing tools and technologies (Databricks, Spark, Kafka)
Knowledge of distributed data solutions, storage systems, and columnar databases.
Knowledge of Cloud Computing on Microsoft Azure or any other public cloud offering
- Familiarity with Continuous Integration/Continuous Deployment, Git.
- Fluent in spoken and written English.
- Knowledge about Agile development methods like Scrum and Kanban.

Job summary 9

We are in search of a Data Engineer to join our team.

Duties & Responsibilities 9

Design, develop, optimize, and maintain data architecture and pipelines
Solve complex data problems to deliver insights that help our business to achieve their goals
- Create data products for analytics and data scientist team members to improve their productivity
- Advise, consult, mentor, and coach other data and analytic professionals on data standards and practices
- Lead the evaluation, implementation, and deployment of emerging tools and processes for analytic data engineering to improve our productivity as a team

Requirements & Skills 9

Job summary 10

As a Data Engineer, you will be implementing solutions for our company.

Duties & Responsibilities 10

Mentor and develop other team members to foster a continuous learning environment
Interface with Clients to interpret their needs
- Asses use cases for numerous teams within the client company
- Develop data models
- Asses strategy within the company

Requirements & Skills 10

Bachelor's degree in Computer science or equivalent
Experience with Cloud ecosystem
Established problem-solving skills and troubleshooting experience
Highly skilled with Java programming
Talented in a wide variety of SQL environments
- Must have experience in real-time streaming technologies (Kafka or Kinesis)
- Understanding of AWS services such as S3, RDS, and EC2
- A major plus if experienced in Kubernetes and/or Docker!

Data Engineer Job Description

Job summary 1

Duties & Responsibilities 1

Requirements & Skills 1

Job summary 2

Duties & Responsibilities 2

Requirements & Skills 2

Job summary 3

Duties & Responsibilities 3

Requirements & Skills 3

Job summary 4

Duties & Responsibilities 4

Requirements & Skills 4

Job summary 5

Duties & Responsibilities 5

Requirements & Skills 5

Job summary 6

Duties & Responsibilities 6

Requirements & Skills 6

Job summary 7

Duties & Responsibilities 7

Requirements & Skills 7

Job summary 8

Duties & Responsibilities 8

Requirements & Skills 8

Job summary 9

Duties & Responsibilities 9

Requirements & Skills 9

Job summary 10

Duties & Responsibilities 10

Requirements & Skills 10

Start Using
100hires for Free

Weekly Webinars

Data Engineer Job Description

Job summary 1

Duties & Responsibilities 1

Requirements & Skills 1

Job summary 2

Duties & Responsibilities 2

Requirements & Skills 2

Job summary 3

Duties & Responsibilities 3

Requirements & Skills 3

Job summary 4

Duties & Responsibilities 4

Requirements & Skills 4

Job summary 5

Duties & Responsibilities 5

Requirements & Skills 5

Job summary 6

Duties & Responsibilities 6

Requirements & Skills 6

Job summary 7

Duties & Responsibilities 7

Requirements & Skills 7

Job summary 8

Duties & Responsibilities 8

Requirements & Skills 8

Job summary 9

Duties & Responsibilities 9

Requirements & Skills 9

Job summary 10

Duties & Responsibilities 10

Requirements & Skills 10

Start Using 100hires for Free

Weekly Webinars

Start Using
100hires for Free