Badge
NielsenIQ logo

Software Engineer - Big Data Engineer (Spark, Databricks, PySpark , Snowflake)

NielsenIQ

Pune (On-site) · Full-time

🔒 Unlocks at Trust Score 80🔒 Unlocks at 80

Job summary

NielsenIQ is a leading consumer intelligence company that helps clients understand consumer behavior and drive growth. As a Big Data Engineer, you will work with technologies like Spark, Databricks, and Snowflake to develop and maintain data engineering applications. Daily tasks include building data pipelines, troubleshooting issues, and collaborating with cross-functional teams. Strong skills in Python, Spark, and cloud technologies, especially Microsoft Azure, are essential. This role is ideal for someone with 2-5 years of experience who enjoys problem-solving and working in an Agile environment. This job unlocks at Trust Score 80 on Badge.

About the job

Our NielsenIQ Technology teams are working on revamping multiple platform, a unified, global, open data ecosystem powered by Microsoft Azure. Our clients around the world rely on NielsenIQ’s data and insights to innovate and grow.

As a Big Data Software Engineer, you’ll be part of a team of smart, highly skilled technologists who are passionate about learning and prototyping cutting-edge technologies. Ingestion Services is working on unifying and standardizing several varied ingestion methodologies and convergence initiatives with latest Big Data services on Spark, Databricks, PySpark, Snowflake and leverage GitHub to support things like CI/CD and integrations and continuing to adopt the best in class cloud-native, cloud-agnostic technologies. Expertise in Python, Spark, Databricks are mandatory. Skills in PySpark, Snowflake, GitHub, AI, Azure are welcome additions. AI readiness and DevSecOps exposure are key additional facets to possess and exhibit. 

  • Develop, test, troubleshoot, debug, and enhance data engineering applications using Spark, PySpark, Pandas, Databricks, Snowflake, Airflow, and SQL as core technologies.
  • Design, build, and optimize scalable data pipelines and ETL/ELT workflows for large-scale data processing.
  • Deploy and maintain application components through CI/CD pipelines, ensuring reliable and efficient releases.
  • Build utilities, frameworks, and monitoring solutions to automate repetitive tasks and improve operational efficiency.
  • Collaborate with Agile cross-functional teams, including Engineering, Operations, Infrastructure, Technical Operations, and external stakeholders, to deliver business solutions.
  • Investigate production issues, perform root cause analysis, and implement timely bug fixes and performance improvements.
  • Ensure data quality, reliability, and adherence to engineering best practices, coding standards, and security guidelines.
  • Participate in code reviews, sprint planning, design discussions, and continuous improvement initiatives

Qualifications

 Must Have -

  • 2-5 years of experience as a Big Data engineer who works on enterprise standard frameworks like Spark, Databricks, PySpark , Snowflake
  • Strong fundamentals with experience in bigdata technologies, Spark, Pyspark, Pandas, Databricks, Snowflake, Airflow, SQL.
  • Must have experience in cloud technologies, preferably Microsoft Azure.
  • Good Understanding on CI/CD Pipeline i.e. GitHub Actions
  • Strong knowledge of version control tools, preferably GitHub
  • Basic Knowledge on Linux/Unix environment (basic commands, shell scripting, etc.) 
  • Demonstrated ability to thrive in an enterprise Agile/SCRUM environment 
  • Demonstrated ability to work as part of a Global Team 
  • Strong troubleshooting and problem-solving skills and excellent English communication skills, with the ability to effectively interface across cross-functional technology teams and the business 
  • Good to have experience with Devops Technologies as GitHub, Kubernetes, Jenkins, Docker.
  • Good to have knowledge of relational databases, preferably PostgreSQL.

Nice to have -

  • Minimum B.S. degree in Computer Science, Computer Engineering or related field 
  • Experience using Collaboration Technologies: GitHub, Jira, Confluence 
  • Active Cloud certification/Databricks certification/Snowflake certification
  • Motivated, high-potential performer, with demonstrated ability to influence and lead. 
  • Strong communicator with excellent interpersonal skills. 
  • Able to solve complex problems and successfully manage ambiguity and unexpected change. 
  • Teachable and embracing of best practices and feedback as a means of continuous improvement. 
  • Consistently high achiever marked by perseverance, humility and a positive outlook in the face of challenges. 

Additional Information

  • Enjoy a flexible and rewarding work environment with peer-to-peer recognition platforms.
  • Recharge and revitalize with help of wellness plans made for you and your family.
  • Plan your future with financial wellness tools.
  • Stay relevant and upskill yourself with career development opportunities.

Our Benefits

  • Flexible working environment
  • Volunteer time off
  • LinkedIn Learning
  • Employee-Assistance-Program (EAP)

NIQ may utilize artificial intelligence (AI) tools at various stages of the recruitment process, including résumé screening, candidate assessments, interview scheduling, job matching, communication support, and certain administrative tasks that help streamline workflows. These tools are intended to improve efficiency and support fair and consistent evaluation based on job-related criteria. All use of AI is governed by NIQ’s principles of fairness, transparency, human oversight, and inclusion. Final hiring decisions are made exclusively by humans. NIQ regularly reviews its AI tools to help mitigate bias and ensure compliance with applicable laws and regulations. If you have questions, require accommodations, or wish to request human review were permitted by law, please contact your local HR representative. For more information, please visit NIQ’s AI Safety Policies and Guiding Principles: https://nielseniq.com/global/en/info/niqs-ai-safety-policies/

About NIQ

NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View™. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population.

For more information, visit NIQ.com

Want to keep up with our latest updates?

Follow us on:  | Instagram |  | 

Our commitment to Diversity, Equity, and Inclusion

At NIQ, we are steadfast in our commitment to fostering an inclusive workplace that mirrors the rich diversity of the communities and markets we serve. We believe that embracing a wide range of perspectives drives innovation and excellence.  All employment decisions at NIQ are made without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, disability, genetic information, marital status, veteran status, or any other characteristic protected by applicable laws. We invite individuals who share our dedication to inclusivity and equity to join us in making a meaningful impact. To learn more about our ongoing efforts in diversity and inclusion, please visit the https://nielseniq.com/global/en/news-center/diversity-inclusion

Free · 2 minutes. On Badge, recruiters see reviews from people you've worked with, not just your CV.

How can Badge help me land interviews?

Recruiters get thousands of applications. Stand out with anonymous reviews from people you've worked with, and build a Trust Score that gets you more interview calls.

  1. 01

    Build your professional reputation

    Give and collect anonymous reviews with colleagues you have worked with. Each review adds to your Trust Score, and it stays yours for life.

  2. 02

    Showcase your Badge

    Put your Badge QR on your CV and the Badge banner on LinkedIn. Recruiters and clients scan it and see your verified reviews.

  3. 03

    Land interviews

    At 70, jobs on Badge open to you. A higher score ranks you above other applicants, so you get more interview calls.

Ask your teammates to review your work on Badge. It is free, and the score goes with you to your next job.

FAQs

Recruiters get thousands of applications, and most CVs look the same. Badge shows them anonymous reviews from people who actually worked with you, plus your Trust Score. On Badge jobs, applicants are ranked by Trust Score, so a higher score puts you at the top of the recruiter's list. That proof is what helps you stand out and get the interview call.

Getting your first review on Badge takes about a minute — you don't have to chase anyone or write awkward "please review me" messages. Here's how it works: 1. Install the Badge app and create your profile. 2. Give your first anonymous peer review to a colleague you've actually worked with — that's what starts your Trust Score. 3. Share your phonebook so Badge can find the colleagues who can review you back, and reach out on your behalf with short, targeted, anonymous review questions. 4. Their answers land directly on your profile and push your Trust Score up — which is what unlocks startup jobs.

Every job on Badge unlocks at a Trust Score. Entry jobs are open to everyone at the base score of 60; better-paying and more senior roles unlock higher — 75, 85, 95. You raise your score by giving reviews and completing the steps the app puts in front of you. Giving a good peer review is the biggest, most repeatable jump. When your score passes a role's threshold, the Apply button unlocks in the app. Your score is live — it's recalculated from your network, so it can move both ways. If it drops below a role's threshold you earn it back the same way: give more reviews, finish your profile.

Every time you change jobs, you start from zero. Your new team doesn't know you delivered that impossible deadline, that you mentored three juniors, or that your last manager called you the most reliable engineer they'd worked with. Years of hard work — invisible. And when layoffs hit, you're just another resume in a pile of AI-generated applications with fake endorsements. Badge fixes this. It collects anonymous peer reviews from colleagues you've actually worked with — your managers, teammates, and clients. Those reviews build your Trust Score. It's like a credit score, but for how you work. Badge is your proof of work — evidence that you actually did what your resume claims. The score travels with you. Change jobs, switch industries, get laid off, become a freelancer — your reputation follows. Because Badge reviews are verified and from real colleagues, your profile stands out against candidates padding their resumes with fake AI-generated reviews. Recruiters on Badge discover candidates by Trust Score, so a higher score means you show up first when opportunities open. You stop competing on resume keywords and start competing on what actually matters: what people who worked with you say about you.

LinkedIn endorsements are public and named — so people only write positive things. Recruiters and hiring managers already know this. Most endorsements come from friends or people returning the favour, not from colleagues giving an honest assessment of your work. Badge reviews are anonymous and specific. Our AI agents ask reviewers targeted questions about how you actually work — your reliability, collaboration, communication, how you handle pressure. Because it's anonymous, people share the real picture, not just polite compliments. It's your proof of work — verified by the people who were actually there. That specificity is what makes Badge valuable. Instead of generic "highly recommended!" endorsements, your profile shows detailed reviews from people who've worked with you, answering questions that actually matter to recruiters. Ten specific anonymous peer reviews carry more weight than fifty one-click LinkedIn endorsements from connections who barely remember working with you.

Your Trust Score is the key to the jobs board. Jobs are gated by score — the higher yours, the more jobs you can apply to — and inside a role, applicants are ranked by Trust Score, so a higher score also puts you at the top of the recruiter's list. Right now, you're invisible. You apply to jobs, your resume goes into an ATS, and a keyword filter decides your fate. Doesn't matter how good you are — if your resume doesn't match the right keywords, nobody sees you. Badge flips this. Recruiters on Badge browse and search candidates by Trust Score. Instead of you chasing jobs, recruiters find you. A higher score means you appear higher in recruiter searches — your verified peer reviews do the selling before you even know the opportunity exists. Here's how it works: recruiters search for candidates by role, industry, or company. Results are ranked by Trust Score. A candidate with 15 anonymous peer reviews and a strong score shows up before someone with a polished resume and zero verified reviews. Your proof of work puts you ahead of the crowd. The more reviews you collect over time, the stronger your score, the more visible you become. Your reputation compounds instead of resetting every time you switch jobs.

This is the first thing everyone worries about — what if someone who didn't like me leaves a bad review? It's a fair concern. The short answer: one bad review won't hurt you. Your Trust Score is built from all your reviews over time. If 14 colleagues say you're reliable, collaborative, and deliver on time, and 1 person says you're difficult — that one person looks like the outlier, not you. Volume protects you. Badge also has safeguards. Abusive or toxic reviews are automatically detected and made private. You can make any review private if you don't want it visible. You can report spam or fake reviews. The system isn't designed to let one person with a grudge define your career. And here's the thing most people don't consider — a few constructive critical reviews actually make your profile more credible. A profile with nothing but glowing praise looks suspicious. A profile that shows you're great at collaboration but still growing as a public speaker? That's a real person. Recruiters trust that more.

Fake reviews are impossible — every reviewer must prove a verified work overlap with you (same company, same time period) before the review is accepted. Reviews from verified colleagues carry more weight in your Trust Score than reviews from loose connections. The review process itself filters out low-quality responses. Our AI agents guide reviewers through targeted questions about your actual work — not generic "rate this person 1-5" prompts. If someone tries to leave a vague "great person!" review, it gets rejected. The system demands specifics. And if someone you've never worked with tries to leave a review? It carries minimal weight because there's no verified work overlap. You can also report reviews that are fake, spammy, or from people you've never interacted with. The system is built to reward genuine reviews, not let strangers or bad actors influence your score.

Yes — you can see every review on your profile. All reviews on Badge are anonymous. The review content is visible to you and recruiters, but the reviewer's identity is always hidden. That's by design. When people know their name won't be attached, they write what's actually true instead of what's socially safe. That honesty is what makes your Badge profile worth more than a LinkedIn page full of polite endorsements from people who didn't want to say no.

More jobs like this