Hadoop Big Data Administrator/Engineer

2 days ago

Warszawa, Mazovia, Czech Republic Capco Poland Full time
  • 5+ years of hands-on experience in Big Data administration, automation, and DevOps practices, including Hadoop ecosystem tools such as HDFS, YARN, and MapReduce.
  • Proficient in managing and troubleshooting Kafka, HBase, and Spark.
  • Experience with performance tuning, cluster optimization, and high-availability configurations.
  • Strong experience with automation tools such as Ansible, Shell scripting, and Python scripting.
  • Ability to automate deployment, monitoring, and administration tasks to increase operational efficiency
  • Solid understanding of DevOps concepts and practices, including CI/CD, version control, and deployment pipelines.
  • Hands-on experience with automation of infrastructure provisioning and management using tools like Terraform, Docker, and Kubernetes (optional but desirable).
  • Proficiency in at least one programming language, with a preference for Python.
  • Ability to write efficient, maintainable code for automation and integration tasks.
  • Strong debugging skills and experience in resolving complex issues across distributed systems.
  • Ability to think critically and provide practical solutions in high-pressure situations.
  • Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience).

Nice to Have:

  • Experience with cloud platforms (AWS, Azure, GCP) for Big Data solutions.
  • Knowledge of containerization and orchestration tools (Docker, Kubernetes).
  • Familiarity with data pipeline frameworks and tools like Apache NiFi, Airflow, or similar.

*We are looking for Poland based candidate. 

Joining Capco means joining an organisation that is committed to an inclusive working environment where you're encouraged to #BeYourselfAtWork. We celebrate individuality and recognize that diversity and inclusion, in all forms, is critical to success. It's important to us that we recruit and develop as diverse a range of talent as we can and we believe that everyone brings something different to the table – so we'd love to know what makes you different. Such differences may mean we need to make changes to our process to allow you the best possible platform to succeed, and we are happy to cater to any reasonable adjustments you may require. You will find the section to let us know of these at the bottom of your application form or you can mention it directly to your recruiter at any stage and they will be happy to help.

Capco Poland is a global technology and management consultancy specializing in driving digital transformation across the financial services industry. We are passionate about helping our clients succeed in an ever-changing industry.

We also are experts in Java, Python, Spring, Hadoop, Angular, React, Android, Google Cloud, Selenium, SQL, Docker, Kubernetes focused on development, automation, innovation, and long-term projects in financial services. In Capco, you can code, write, create, and live at your maximum capabilities without getting dull, tired, or foggy.

Capco Poland is a leading global technology and management consultancy, dedicated to driving digital transformation across the financial services industry. Our passion lies in helping our clients navigate the complexities of the financial world, and our expertise spans banking and payments, capital markets, wealth, and asset management. We pride ourselves on maintaining a nimble, agile, and entrepreneurial culture, and we are committed to growing our business by hiring top talent.

We are seeking a highly skilled and motivated Big Data Administrator/Engineer with 5+ years of experience in Hadoop administration and expertise in automation tools like Ansible, shell scripting, or Python scripting. The ideal candidate will have strong DevOps skills and proficiency in coding, particularly in Python. This is a dynamic role focused on managing and engineering Big Data solutions across multiple open-source platforms such as Hadoop, Kafka, HBase, and Spark.

You will be responsible for performing critical Big Data administration, troubleshooting, debugging, and ensuring the seamless operation of various data processing frameworks. If you are a hands-on, results-driven individual with a passion for Big Data technologies, this is the role for you.

WHY JOIN CAPCO?

  • Employment contract and/or Business to Business - whichever you prefer
  • Possibility to work remotely
  • Speaking English on daily basis, mainly in contact with foreign stakeholders and peers
  • Multiple employee benefits packages (MyBenefit Cafeteria, private medical care, life-insurance)
  • Access to 3.000+ Business Courses Platform (Udemy)
  • Access to required IT equipment
  • Paid Referral Program
  • Participation in charity events e.g. Szlachetna Paczka
  • Ongoing learning opportunities to help you acquire new skills or deepen existing expertise
  • Being part of the core squad focused on the growth of the Polish business unit
  • A flat, non-hierarchical structure that will enable you to work with senior partners and directly with clients
  • A work culture focused on innovation and creating lasting value for our clients and employees

ONLINE RECRUITMENT PROCESS STEPS

  • Screening call with Recruiter
  • Technical interview
  • Meeting with Hiring Manager
  • Feedback/Offer
,[Big Data Administration:, Administer and manage Hadoop clusters, ensuring high availability, performance, and scalability., Maintain and troubleshoot Hadoop ecosystem components such as HDFS, MapReduce, YARN, and related tools., Ensure Kafka, HBase, and Spark systems are optimized and running smoothly., Implement monitoring and alerting for Big Data infrastructure., Automation and Scripting:, Develop and automate scripts using tools such as Ansible, Shell scripting, or Python to streamline Big Data administration tasks., Create reusable automation frameworks to reduce manual efforts and improve operational efficiency., Work on CI/CD pipelines for deployment automation and system integration., DevOps Practices:, Apply DevOps principles to the Big Data environment, focusing on continuous integration and continuous delivery (CI/CD)., Build and manage automated deployment processes for Big Data clusters and services., Collaborate with development teams to integrate automation in Big Data workflows., Troubleshooting and Debugging:, Identify, troubleshoot, and resolve issues related to Big Data platforms, including system performance, resource utilization, and service failures., Work with logs, monitoring tools, and other debugging techniques to diagnose and resolve complex issues., Collaboration and Support:, Work closely with other teams to support data pipelines, data quality checks, and performance optimizations., Provide ongoing technical support to ensure that Big Data systems are stable, secure, and aligned with business objectives.] Requirements: Big data, DevOps, Hadoop, HDFS, yarn, Kafka, HBase, Spark, Performance tuning, Ansible, Shell, Python, Terraform, Docker, Kubernetes, Degree, Cloud platform, Azure, GCP, Apache, NiFi, Airflow Additionally: Private healthcare, Employee referral bonus, MyBenefit, Udemy for business.

  • Hadoop Big Data Engineering Specialist

    2 days ago

    Warszawa, Mazovia, Czech Republic Capco Poland Full time

    About Capco PolandCapco Poland is a leading global technology and management consultancy dedicated to driving digital transformation across the financial services industry. Our passion lies in helping clients navigate the complexities of the financial world, and our expertise spans banking and payments, capital markets, wealth, and asset management.We pride...

  • Big Data Developer with Spark and Agile Expertise

    6 days ago

    Warszawa, Mazovia, Czech Republic Axiom software solutions Full time

    Axiom software solutions is seeking a seasoned Big Data Senior Developer to join our team.We are looking for an expert in Big Data technologies, specifically in Spark and Scala. With at least 2 years of experience working with these tools, you will be responsible for designing, developing, and maintaining large-scale data processing systems.As a key member...

  • Big Data Architect for Scalable Data Processing

    3 days ago

    Warszawa, Mazovia, Czech Republic Quontex Ltd Full time

    Welcome to Quontex Ltd, a leading programmatic media company specializing in ingesting large volumes of data and offering a range of products and services across Media, Analytics, and Technology.Our team is expanding rapidly, with over 700 employees and 15 global offices spanning four continents. We are seeking an experienced Data Engineer to join our team...

  • Data Engineer @

    3 days ago

    Warszawa, Mazovia, Czech Republic Quontex Ltd Full time

    3+ years of overall experience in Data Warehouse development and database designDeep understanding of distributed computing principlesExperience with AWS cloud platform, and big data platforms like EMR, Databricks, EC2, S3, RedshiftExperience with Scala, Spark, Hive, Yarn/Mesos, etc.Experience in SQL and NoSQL databases, as well as experience with data...

  • Specjalista ds. Danych Big Data

    3 days ago

    Warszawa, Mazovia, Czech Republic 7N Full time

    Opis stanowiskaJesteśmy poszukiwani przez naszego klienta, firmę z sektora bankowego, których oczekuje od nas współpracy w dalszym rozwoju autorskiego systemu do detekcji anomalii za pomocą narzędzi do budowy modeli uczenia maszynowego.Oczekujemy doświadczonego specjalisty ds. danych big data, kandydata na stanowisko Data Scientist / Big Data...

  • Scalable Data Engineer for ETL Processes

    6 days ago

    Warszawa, Mazovia, Czech Republic Axiom software solutions Full time

    At Axiom Software Solutions, we are looking for a skilled Backend Spark developer to join our team.About the JobWe are seeking a highly motivated and experienced professional to develop, test and deploy technical and functional specifications from our Solution Designers / Business Architects / Business Analysts, ensuring correct operability and compliance...

  • Specjalista ds. Inżynierii Inteligencji Maszynowej

    4 days ago

    Warszawa, Mazovia, Czech Republic 7N Full time

    Nasza firma, 7N, poszukuje doświadczonego specjalisty w dziedzinie inżynierii inteligencji maszynowej na stanowisko AI Engineer / Data Scientist. Praca będzie obejmowała rozwój i powiększanie obecnego zespołu naszego klienta - lidera innowacyjnych rozwiązań bankowych.O projekcieW ramach projektu będziemy pracować nad autorskim systemem do detekcji...

  • Data Architect for Scalable Data Platform

    4 days ago

    Warszawa, Mazovia, Czech Republic Pragmile Full time

    At Pragmile, we are on a mission to maximize green energy production and reduce O&M costs through the use of artificial intelligence models.Solar Spy is an analytical platform dedicated to support solar farm operations. It processes various types of PV plant data, including image data from aerial inspections, weather data, data from photovoltaic installation...

  • Data Platforms Technical Lead

    19 hours ago

    Warszawa, Mazovia, Czech Republic T-Mobile Polska Full time

    Extensive experience (4+ years) in designing and delivering cloud or hybrid data platformsStrong background in data engineering and analyticsProficiency in cloud technologies (AWS, GCP)Experience with Data Warehousing, Big Data technologies and NoSQL databasesStrong leadership and communication skillsAbility to translate complex technical concepts for...

  • Senior Software Engineer, Data Infrastructure @

    2 days ago

    Warszawa, Mazovia, Czech Republic Asana Full time

    About youExpertise in programming, distributed systems design, and infrastructureExperience building and operating scalable, reliable, and highly-available services4+ years designing and implementing production code for backend, infrastructure, and/or data systems2+ years mentoring/coaching other team members on design and execution of projectsEagerness to...

  • Senior Data Engineer Leader

    4 days ago

    Warszawa, Mazovia, Czech Republic SambaTV Full time

    Samba TV is revolutionizing the viewing experience with our innovative data and technology. Our mission is to transform the way media companies connect with audiences and advertisers engage with viewers.We are seeking an experienced Tech Lead - Data Engineering to lead the development of scalable, high-performance data pipelines that power our analytics and...

  • Data Platforms Technical Lead

    16 hours ago

    Warszawa, Mazovia, Czech Republic T-Mobile Polska Full time

    About UsT-Mobile Polska is a leading telecommunications company in Poland, providing innovative solutions to customers across the country.Job DescriptionWe are seeking an experienced Data Platforms Technical Lead - Cloud Architect to join our team. As a key member of our data platforms infrastructure team, you will be responsible for designing, developing,...

  • Database Reliability Engineer @ .com

    2 days ago

    Warszawa, Mazovia, Czech Republic monday Full time

    Proven working experience and deep understanding of RDBMS (MySQL / PostgreSQL/ Oracle / SQL Server / etc).Experience of at least 5+ years with performance optimization and scaling out data worldwide.Coding experience and understanding of code and software environment.Experience and understanding of cloud environments and toolsExperience in leading projects,...

  • Senior Software Engineer, Storage Infrastructure @

    2 days ago

    Warszawa, Mazovia, Czech Republic Asana Full time

    About youExpertise in programming and computer science, and strong interest in distributed systems5+ years of software development and infrastructure experience2+ years of experience building and scaling high-volume online data systems (e.g. search infrastructure, data stores), and operating highly available, user-facing production servicesExperience with...

  • Innovative Data Engineer for Scalable Solutions

    5 days ago

    Warszawa, Mazovia, Czech Republic 7N Full time

    Company OverviewWe are 7N, a Swedish fintech company expanding in Poland. Our mission is to revolutionize investment decision-making by leveraging machine learning and strategic data.Job DescriptionWe are seeking an experienced Fullstack Developer to join our team. As a Fullstack Engineer, you will design, develop, and maintain scalable solutions,...

  • Customer-Facing Data Science Solutions Architect

    6 days ago

    Warszawa, Mazovia, Czech Republic dotData Full time

    **Job Overview**We are seeking a customer-facing data scientist to join our team at dotData. This role requires working directly with customers to use dotData products to solve their challenging problems, and collaborating with the product/engineering teams to incorporate customer feedback into our product development process.

  • Database Reliability Specialist

    2 days ago

    Warszawa, Mazovia, Czech Republic monday Full time

    We are seeking an experienced Database Reliability Engineer to join our rapidly growing DBRE team at monday.com. As a key member of our team, you will be responsible for ensuring the reliability and performance of our hundreds of production and pre-production database environments.The ideal candidate will have a deep understanding of RDBMS and experience...

  • Data Analyst for Healthcare Innovation

    5 days ago

    Warszawa, Mazovia, Czech Republic Solvemed Full time

    Company OverviewSolvemed is a dynamic team of experienced innovators, revolutionizing the Intensive Care Unit using cutting-edge AI and advanced smartphone sensors. Our mission is to transform healthcare by leveraging technology and innovation to save patient lives.Job DescriptionWe are seeking an ambitious Data Analyst to join our team in Warsaw or...

  • Cloud Engineering Expert

    5 days ago

    Warszawa, Mazovia, Czech Republic Ework Group Full time

    About Ework GroupWe are a dynamic team of experts in cloud technology, dedicated to delivering innovative solutions for our clients.Job DescriptionAs a Cloud Engineer at Ework Group, you will be responsible for designing, developing, and maintaining cloud-native applications and services on cloud platforms.You will work closely with engineers and other...

  • Fullstack Developer

    5 days ago

    Warszawa, Mazovia, Czech Republic 7N Sp. z o.o. Full time

    As a Fullstack Engineer at 7N, you will be part of a dynamic team responsible for developing a data and analytics platform designed to revolutionize investment decision-making. This project focuses on leveraging machine learning and strategic data to deliver exceptional insights.About the RoleYou will design, develop, and maintain scalable solutions that...