Petabyte-Scale Big Data Engineer

5 days ago


Remote Wrocław, Czech Republic Comscore (via CC) Full time

We are seeking a seasoned Petabyte-Scale Big Data Engineer to join our team at Comscore (via CC). As a key member of our international team, you will play a vital role in driving projects forward and improving existing systems.

About the Role

The ideal candidate will have extensive experience with petabyte-scale big data pipelines using Scala, Spark, Kubernetes, and other cutting-edge technologies. They will be responsible for designing, implementing, and maintaining high-performance big data systems, optimizing them for IO/CPU-bound processes, and conducting proof-of-concepts for enhancements.

Key Responsibilities
  • Design, implement, and maintain petabyte-scale big data pipelines using Scala, Spark, and Kubernetes.
  • Optimize big data systems for performance, working closely with cross-functional teams to identify and address bottlenecks.
  • Conduct proof-of-concept studies to enhance existing systems and develop new features.
  • Write high-quality, performant big data code using Scala.
  • Collaborate with other big data teams to ensure seamless integration and knowledge sharing.
Requirements

To succeed in this role, you will need:

  • Extensive experience with Scala, Spark, and Kubernetes.
  • Strong understanding of big data processing and optimization techniques.
  • Excellent problem-solving skills and ability to work independently.
  • Effective communication and collaboration skills.
  • Experience with cloud-based technologies such as AWS and EMR.
What We Offer

We provide a dynamic and supportive work environment, opportunities for growth and development, and a competitive salary package. You will also enjoy benefits such as flexible working hours, remote work options, and access to modern office facilities.



  • Remote, Wrocław, Czech Republic Comscore (via CC) Full time

    Comscore is a global leader in media analytics, revolutionizing insights into consumer behavior, media consumption, and digital engagement.We are looking for a skilled Java/Spark Big Data Developer to join our team in Poland.About the RoleThis is an exciting opportunity to work on real big data projects at petabytes scale. As a Senior Big Data Engineer, you...


  • Remote, Wrocław, Czech Republic Comscore (via CC) Full time

    The candidate must have: 2+ years of experience with Linux Solid knowledge of Linux (bash, threads, IPC, filesystems; being power-user is strongly desired, understanding how OS works so you can benefit from performance optimizations in production but also in daily workflows) 1+ years of experience with Spark,  primarily using Scala for Big data...


  • Remote, Wrocław, Czech Republic Comscore (via CC) Full time

    The candidate must have:Very good understanding of Linux internals (processes/threads, service management, resource allocation)Linux as a daily driver of operations1+ years of commercial experience with JavaProven ability to debug JVM/Java (hopefully in Spark ecosystem)Ability to design high performance processing pipelinesHands-on, proactive...


  • Remote, Wrocław, Czech Republic Comscore (via CC) Full time

    About ComscoreComscore is a global leader in media analytics, revolutionizing insights into consumer behavior, media consumption, and digital engagement. We lead in measuring and analyzing audiences across diverse digital platforms.We thrive on using cutting-edge technology to deliver accurate data to global businesses, collaborating with industry leaders...


  • Remote, Wrocław, Czech Republic Comscore (via CC) Full time

    Comscore, a global leader in media analytics, is looking for an experienced Scala Developer to join their team. As a Scala Developer at Comscore, you will play a vital role in designing and developing solutions that process petabytes of data.Company OverviewWe are a trusted partner delivering accurate data to global businesses, working with industry leaders...


  • Remote, Wrocław, Czech Republic Comscore (via CC) Full time

    The candidate must have:Very good understanding of Linux internals (processes/threads, service management, resource allocation)Linux as a daily driver of operationsGood understanding of Spark internals and mechanicsKnowledge of Iceberg4+ years of commercial experience with JavaProven ability to debug JVM/Java (hopefully in Spark ecosystem)Ability to design...


  • Remote, Czech Republic SoftServe Full time

    IF YOU ARESpecializing in big data engineering with a focus on data pipeline creationAdept at batch and streaming processingProficient in Python and SQLExperienced in developing data solutions on the AWS cloud platformSkilled in utilizing Apache Spark and DatabricksFamiliar with orchestration tools like Apache Airflow or Managed Apache Airflow...


  • Remote, Czech Republic Adaptiq Full time

    At least 5 years of experience in Data Engineering domainAt least 2 years of experience with GoLangProficiency in SQL, NoSQL, Kafka/Pulsar, ELK, Redis and column store databasesExperienced with big data tools such as Spark or Flink to enhance system performance and scalabilityProven experience with Kubernetes (K8S) and familiarity with GTP tools to ensure...

  • Data Engineer

    4 days ago


    Remote, Warsaw, Wrocław, Białystok, Kraków, Gdańsk, Czech Republic Addepto Full time

    Job OverviewAddepto is a leading consulting and technology company specializing in AI and Big Data. We are seeking an experienced Data Engineer - AI & Big Data Specialist to join our team.Key Responsibilities:Design, develop, and maintain high-performance data processing platforms for automotive and aerospace industries.Create scalable and reliable data...


  • Remote, Kraków, Czech Republic Appliscale Full time

    Required: Minimum of 5 years commercial work experienceBachelor's or higher degree in Computer Science, Software Engineering, or a related fieldProficiency in Python, essential for data processing and analysis tasksHands-on experience with DBT (Data Build Tool)Commercial experience using PySpark and DatabricksKnowledge in Infrastructure as Code tooling,...


  • Remote, Czech Republic Big Human Full time

    About the RoleWe are seeking a seasoned Senior Flutter Developer to join our team at Big Human, a digital product studio with a passion for creating innovative online experiences. As a key member of our development team, you will be responsible for designing and delivering high-quality cross-platform mobile applications that exceed user expectations.Key...


  • Remote, Kraków, Czech Republic Appliscale Full time

    Company Overview:The company is one of the largest game studios known for their successful MOBA and FPS franchises.The Data Team collects and uses data to improve the player's experience, with a mission to harness the power of data for player-centric decisions and AI/ML products.As a core contributor, you will build robust data solutions capable of...


  • Remote, Warsaw, Wrocław, Białystok, Kraków, Gdańsk, Czech Republic Addepto Full time

    Addepto is a leading consulting and technology company specializing in AI and Big Data. Our expertise helps clients deliver innovative data projects, partnering with top-tier global enterprises and pioneering startups.Job OverviewThis role involves working as a Senior Data Engineer on challenging projects across various industries, leveraging cutting-edge...


  • Remote, Wrocław, Czech Republic Rite NRG Full time

    About Rite NRGRite NRG is a globally known interactive entertainment and technology company. We strive to push the boundaries of innovative AI-driven solutions.Job OverviewWe are seeking an experienced AI/LLM Data Engineer to join our ongoing project, focusing on developing and fine-tuning language models to support intelligent, multilingual AI features. The...


  • Remote, Warsaw, Wrocław, Białystok, Kraków, Gdańsk, Czech Republic Addepto Full time

    Company OverviewAddepto is a leading consulting and technology company specializing in AI and Big Data, helping clients deliver innovative data projects. We partner with top-tier global enterprises and pioneering startups, including Rolls Royce, Continental, Porsche, ABB, and WGU.We have earned recognition by Forbes as one of the top 10 AI companies due to...


  • Remote, Warszawa, Cracow, Wrocław, Czech Republic N-iX Full time

    4+ years of hands-on experience in data engineering.Strong expertise with Python, Pyspark and Azure Databricks.Solid experience in Cloud-based Big Data integration, specifically within the Azure ecosystem.Strong data engineering mindset with experience building and maintaining production-level pipelines.Solid knowledge of Azure, GitLab, and OOP...


  • Remote, Warsaw, Wrocław, Białystok, Kraków, Gdańsk, Czech Republic Addepto Full time

    Company OverviewAddepto is a leading consulting and technology company specializing in AI and Big Data, helping clients deliver innovative data projects. We partner with top-tier global enterprises and pioneering startups.Job DescriptionDesign and development of a universal data platform for global aerospace companies.Data Platform Transformation for energy...


  • Remote, Czech Republic SoftServe Full time

    Job OverviewSoftServe is a global digital solutions company with a strong presence in the data engineering space. As a Senior Big Data Engineer, you will be part of our team that designs and implements cloud-based data pipelines to drive business value.About UsWe are a dynamic organization that fosters a culture of innovation and continuous learning. Our...


  • Remote, Czech Republic Link Group Full time

    We are seeking an experienced Senior Teradata Engineer to join our dynamic team at Link Group.This role requires a strong focus on designing, developing, and optimizing large-scale data solutions using Teradata.As a key member of our cross-functional team, you will collaborate closely with stakeholders to ensure data integrity, performance, and scalability...


  • Remote, Warsaw, Wrocław, Białystok, Kraków, Gdańsk, Czech Republic Addepto Full time

    Company OverviewAddepto is a leading consulting and technology company specializing in AI and Big Data. We help clients deliver innovative data projects by partnering with top-tier global enterprises and pioneering startups.Job DescriptionWe are seeking a highly skilled Data Architect to join our team. As a Data Architect, you will design and develop...