Allstate Big Data Engineer in Irving, Texas

Big Data Engineer

Apply Now at

Job ID :


Location :

Irving, TX US

Level :

Experienced Professional

Job Category :

Information Technology

Description :

Where good people build rewarding careers.

Think that working in the insurance field can’t be exciting, rewarding and challenging? Think again. You’ll help us reinvent protection and retirement to improve customers’ lives. We’ll help you make an impact with our training and mentoring offerings. Here, you’ll have the opportunity to expand and apply your skills in ways you never thought possible. And you’ll have fun doing it. Join a company of individuals with hopes, plans and passions, all using and developing our talents for good, at work and in life.

Job Description

The role is responsible for building new and advanced analytics applications on Big Data platforms like Hadoop, Kafka etc. The Big Data Engineer job family is accountable for end to end engineering of data solutions which includes designing and building systems for data storage and analytics that enable Allstate analysts to make better decisions to achieve Allstate’s goals. This includes the development of new systems for analyzing data; the coding & development of advanced analytics solutions to make/optimize business decisions and processes and discovery of new technical challenges that can be solved with existing and emerging Big Data hardware and software solutions.

Key Responsibilities

  • Uses and learns a wide variety of tools including R, Python, Scala and any other future analytic systems or languages

  • Researches and evaluates new tools and technologies to solve business problems

  • Responsible for designing and building new Big Data applications for analyzing data into actionable insights

  • Drive projects from technical expertise.

  • Partner with technology and the analytics community on big data efforts.

  • Mentor other team members and participate in cross-training.

  • Support highly available environments that are part of critical business processes.

  • Utilize effective project planning techniques to break down highly complex projects into tasks, manage scope of projects, and ensure deadlines are kept.

  • Keep up with current technology trends in the Big Data and Analytic space.

  • Work on an geographically dispersed team embracing Agile and DevOps strategies for themselves and others while driving adoption to enable greater technology and business value

Job Qualifications

  • Experience in Java, Spark, Python and/or Scala

  • Experience with source control solutions (ex git, GitHub)

  • Experience managing and manipulating large datasets.

  • Experience and solid understanding of Bigdata ecosystems such as Hadoop, Spark, Kafka

  • Experience with various data types (e.g. Relational, Unstructured, Hierarchical, Linked “Graph” Data)

  • Experience with database & ETL technologies

  • Experience developing large-scale distributed applications in Cloudera Hadoop, Hive, and/or Impala

  • Good verbal and written communication skills

  • Open to new ideas and technologies with a strong desire to learn

  • Ability to analyze and solve problems in a creative and innovative fashion

  • Ability to research and document project design, approach and architecture

  • Ability to engage subject matter experts and translate business goals into actionable solutions

  • Ability to work effectively with business and technical teams

  • Ability to meet deadlines, goals and objectives

  • Works collaboratively with multiple areas and comfortable leading projects and mentoring others

  • Familiarity and comfort with participating in a high-performing geographically dispersed team

  • Understanding of predictive modeling techniques is a plus

Essential Skills

  • 3+ Years of experience in relevant technologies.

  • Bachelors / Masters degree, preferably in a Computing or Information Sciences field.

  • Intermediate UNIX/Linux skills.

  • Knowledge of one or more of the following: Java, Scala, Python.

  • Fundamental knowledge of SQL.

  • Understanding of JSON, AVRO, Parquet, XML etc.

The candidate(s) offered this position will be required to submit to a background investigation, which includes a drug screen.

Good Work. Good Life. Good Hands®.

As a Fortune 100 company and industry leader, we provide a competitive salary – but that’s just the beginning. Our Total Rewards package also offers benefits like tuition assistance, medical and dental insurance, as well as a robust pension and 401(k). Plus, you’ll have access to a wide variety of programs to help you balance your work and personal life -- including a generous paid time off policy.

Learn more about life at Allstate. Connect with us on Twitter at , Facebook at , Instagram at and LinkedIn at or watch a video at .

Allstate generally does not sponsor individuals for employment-based visas for this position.

Effective July 1, 2014, under Indiana House Enrolled Act (HEA) 1242, it is against public policy of the State of Indiana and a discriminatory practice for an employer to discriminate against a prospective employee on the basis of status as a veteran by refusing to employ an applicant on the basis that they are a veteran of the armed forces of the United States, a member of the Indiana National Guard or a member of a reserve component.

For jobs in San Francisco, please click "here" at for information regarding the San Francisco Fair Chance Ordinance.

For jobs in Los Angeles, please click "here" at for information regarding the Los Angeles Fair Chance Initiative for Hiring Ordinance.

It is the policy of Allstate to employ the best qualified individuals available for all jobs without regard to race, color, religion, sex, age, national origin, sexual orientation, gender identity/gender expression, disability, and citizenship status as a veteran with a disability or veteran of the Vietnam Era.