Senior Big Data Engineer
6 days ago
Job Title: Senior Big Data Engineer - Large Scale Data Processing
We are seeking an experienced Senior Big Data Engineer to join our team at Link Group. As a key member of our data engineering team, you will design, implement, and optimize large-scale data processing pipelines using Apache Spark and Python.
About the Role:
- Develop, test, and maintain PySpark-based ETL pipelines to process and analyze large datasets.
- Collaborate with data engineers, data scientists, and business stakeholders to understand data requirements and design optimal solutions.
- Optimize PySpark applications for performance and scalability in distributed computing environments.
- Work with Hadoop-based data platforms and integrate with other tools like Hive, HDFS, or Kafka.
- Ensure data quality and integrity through robust validation and monitoring practices.
- Debug and resolve issues in production and pre-production environments.
- Document technical solutions and best practices.
Requirements:
- Data engineering experience with 5+ years of experience in big data processing and analytics.
- Proficiency in Python programming with experience in libraries commonly used in data processing (e.g., Pandas, NumPy).
- Strong understanding of Apache Spark concepts: Spark Core, Spark SQL, and Spark Streaming.
- Experience with distributed data processing frameworks and working in cloud-based environments (e.g., AWS, Azure, GCP).
- Solid knowledge of big data technologies like Hadoop, Hive, HDFS, Kafka, or Airflow.
- Familiarity with CI/CD pipelines and version control (e.g., Git).
Preferred Qualifications:
- Experience with streaming data using Spark Streaming or Kafka.
- Knowledge of machine learning workflows and integration with big data pipelines.
- Understanding of containerization tools like Docker or orchestration with Kubernetes.
Estimated Salary: $120,000 - $180,000 per annum
Benefits:
- A competitive salary package including bonus and benefits.
- Ongoing professional development and training opportunities.
- A dynamic and collaborative work environment.
-
Senior Big Data Engineer Lead at OpenX
2 months ago
Remote, Kraków, Czech Republic OpenX Full timeJob DescriptionWe are seeking an experienced Senior Big Data Engineer to join our team at OpenX. This role will involve working on all aspects of agile application development, including our enterprise platform that interfaces with multiple services delivering billions of requests daily.About the RoleYou will be responsible for architecting, creating, and...
-
Big Data Analytics Engineer
1 month ago
Remote, Czech Republic Link Group Full timeWe are seeking a highly skilled Big Data Analytics Engineer to join our team at Link Group.As a key member of our data engineering team, you will design, implement, and optimize large-scale data processing pipelines using Apache Spark and Python.Key Responsibilities:Develop, test, and maintain PySpark-based ETL pipelines to process and analyze large...
-
Big Data Engineer: Innovation at the Core
3 days ago
Remote, Białystok, Wrocław, Kraków, Czech Republic Grape Up Full timeJob OverviewWe are seeking a seasoned Big Data Engineer to join our team at Grape Up. As a key member of our data engineering group, you will be responsible for designing and implementing scalable big data systems deployed on-premises.Key ResponsibilitiesDesign, develop, and maintain large-scale big data systems to meet the needs of our clientsDevelop and...
-
DataOps Engineer for Big Data Technology
4 weeks ago
Remote, Czech Republic Adaptiq Full timeAbout the Role:Adaptiq is seeking a skilled DataOps Engineer to join our team. This role is ideal for someone dedicated to building and supporting high-quality data products and processes.The successful candidate will be responsible for maintaining the quality of our data services and knowledge platform across all product data processes. This includes...
-
Chief Big Data Architect
2 weeks ago
Remote, Białystok, Wrocław, Kraków, Czech Republic Grape Up Full timeJob Title: Chief Big Data ArchitectWe are seeking a highly skilled Chief Big Data Architect to join our team at Grape Up.OverviewGrape Up is a software development company that leverages technology to drive innovation. Our mission is to create cutting-edge solutions for our clients, and we're currently building a Data & Analytics platform for a leading...
-
Engineering Manager
2 months ago
Remote, Warsaw, Wrocław, Białystok, Kraków, Gdańsk, Czech Republic Addepto Full timeThis role is ideal for a leader who combines technical expertise with strong leadership skills, ready to drive innovative projects in data science and big data. 🎯 What you’ll need to succeed in this role: Bachelor’s or higher in Computer Science, Mathematics, Physics, or related field. Proven track record in managing technical teams and leading...
-
Senior Big Data Engineer @ OpenX
3 months ago
Remote, Kraków, Czech Republic OpenX Full time8+ years experience in Java, Scala or Python development 3+ years of experience in Big Data systems development Experience building complex data pipelines with data transformation tools like DBT or Dataform and SQL. Expertise using BigQuery or other data warehouses including performance tuning and monitoring. Understands data orchestration tools like...
-
Data Engineer for Cloud and Data Pipelines
2 weeks ago
Remote, Warsaw, Wroclaw, Czech Republic apreel Sp. z o.o. Full timeAbout the RoleWe are seeking a skilled Data Engineer to join our team in Warsaw or Wroclaw, with the option to work 100% remotely. As a Data Engineer at apreel Sp. z o.o., you will be responsible for designing, developing, and maintaining large-scale data pipelines using GCP and Spark technologies.Company OverviewOur client is one of the largest Big Four...
-
Senior Data Engineer @ apreel Sp. z o.o.
2 weeks ago
Remote, Warsaw, Wroclaw, Czech Republic apreel Sp. z o.o. Full timeWhat you need: Technical skills are similar for both, Medior and Senior roles, and 80%-90% of responsibility is data pipelines development Must have: pySpark Managing Spark clusters GCP (other cloud providers are not relevant) Overall implementation of data pipelines is fine, their specific technology is not required Senior profile: might be...
-
Senior Data Engineer @ Grape Up
2 weeks ago
Remote, Białystok, Wrocław, Kraków, Czech Republic Grape Up Full timeRequirements 5+ of professional experience in Data Engineering and Big Data Proven experience in designing and implementing data governance and data management frameworks Expert level knowledge of Python Master’s degree in computer science, Data Science, AI, or related field Strong problem-solving skills and independent work ethic Fluency in...
-
Senior Performance Optimization Engineer
1 month ago
Remote, Czech Republic Digis Full timeAt Digis, we are seeking a highly skilled Senior Performance Optimization Engineer to join our team. This role requires expertise in optimizing software performance across all layers, from low-level compiler development to high-level frameworks like Apache Spark.About the RoleWe are looking for an experienced engineer with strong skills in C++, Java, and x86...
-
Linux Engineer for Big Data Projects
2 months ago
Remote, Wrocław, Czech Republic Comscore (via CC) Full timeWe are seeking a highly skilled Linux Engineer to join our team at Comscore in Poland and around. As a key member of our international team, you will be responsible for ensuring the smooth operation of our big data projects.With 5+ years of strong Linux experience, you will have a deep understanding of Linux processes, thread management, shell scripting,...
-
Project Manager
4 weeks ago
Remote, Warsaw, Wrocław, Białystok, Kraków, Gdańsk, Czech Republic Addepto Full time🎯 What you’ll need to succeed in this role: Bachelor’s or higher in Computer Science, Mathematics, Physics, or related field. Proven track record in managing technical teams and leading end-to-end projects. Experience working with corporate clients. Strong critical thinking and problem-solving abilities. Excellent communication and presentation...
-
Big Data Architect
2 months ago
Remote, Czech Republic Alcor Full timeAbout AlcorAlcor is a leading organization in the field of data engineering, with a strong focus on innovation and excellence.Salary RangeThe estimated salary for this position is between $120,000 to $180,000 per year, depending on experience and qualifications.Job DescriptionWe are seeking an experienced Data Engineer to join our team. As a key member of...
-
Remote, Czech Republic AVENGA Full timeAbout the RoleWe are seeking a Senior Data Engineer to lead our Machine Learning Operations (MLOps) transformation. As part of our team at Avenga, you will be responsible for driving the operationalization of data science projects using cutting-edge technologies and frameworks.
-
Senior Data Engineer @ Sunscrapers
1 month ago
Remote, Warszawa, Czech Republic Sunscrapers Full timeAt least 5 years of professional experience as a data engineer, Undergraduate or graduate degree in Computer Science, Engineering, Mathematics, or similar, Excellent command in spoken and written English, at least C1, Strong professional experience with Python and SQL, Hands on experience on DBT and Snowflake, Experience in building data pipelines with...
-
Senior Data Engineer with Azure Expertise
1 week ago
Remote, Czech Republic Experis Polska Full timeJob Description:We are seeking a Senior Data Engineer with expertise in Azure to join our team at Experis Polska. In this role, you will play a key part in scaling out the usage of logging by normalizing raw logging to a standardized schema.About the Role:As a Senior Data Engineer, you will be responsible for normalizing raw logging data using Microsoft...
-
Remote, Wroclaw, Czech Republic Proxet Full timeData Engineer Job OverviewProxet is seeking a skilled Senior Data Engineer to play a pivotal role in shaping the data infrastructure for our finance-related project. As a key member of our team, you will design and maintain scalable data pipelines that ingest and transform data from external platforms, ensuring high-quality, reliable, and accessible data...
-
Senior Linux Systems Engineer
2 weeks ago
Remote, Wrocław, Czech Republic Comscore (via CC) Full timeWe are seeking an experienced Senior Linux Systems Engineer to join our team at Comscore in Poland and around. This is a fantastic opportunity for you to work on real big data projects, collaborate with industry leaders like Facebook, Disney, and Amazon, and contribute to empowering businesses in the digital era across media, advertising, e-commerce, and...
-
Senior Data Engineer @ Proxet
1 month ago
Remote, Wroclaw, Czech Republic Proxet Full timePossess a Bachelor’s degree in Computer Science or Software Engineering, or demonstrate substantial experience as a seasoned app developer. Demonstrate a minimum of 3 years of proficiency in Python, SQL, data systems development life cycle. Experience using different kinds of databases (for example, PostgreSQL, BigQuery, Redis) including experience with...