IBM Big Data Developer/Designer-Spark/Scala, Sr Consultant in Houston, Texas

Job Description

Is data your jam? Do you think BIG Data is better data? Do you think data lakes are more fun than real lakes? Do you enjoy solving business problems with data and communicating your recommendations to others? Do you get excited about learning new tools in the Hadoop ecosystem? Do you like to travel? Are you self-motivated and like working on teams? This role might be for you!

IBM is seeking Big Data Developer/Designer with technical experience in optimizing of management and deriving insights from large, non-structured, non-relational data.

Responsibilities include:

  • Design, develop, and implement a real time data, integration, and cognitive fabric using Big Data technologies such as Hadoop, Spark, HBase and Impala using Scala programming language

  • Architect, implement and test data processing pipelines (e.g. Hadoop and Spark) and data mining / data science algorithms on a variety of hosted settings, such as Cloud (AWS, Azure, GCP)

  • Formulate approaches and gather data to solve business problems, develop conclusions and present solutions through formal deliverables

  • Create Big Data accelerators to help deploy scalable solutions fast

  • Build processes supporting data transformation, data structures, metadata, dependency and workload management

You will be successful in this role if you enjoy problem solving and utilizing consulting skills. Team leadership experience is preferred.

In this dynamic role, you will have the opportunity to interact directly with clients. You will travel to client sites and may be required to travel up to 4 days per week. Preferred candidates will reside within a 50 mile radius of a major airport.


Health Insurance. Paid time off. Corporate Holidays. Sick leave. Family planning. Financial Guidance. Competitive 401K. Training and Learning. We continue to expand our benefits and programs, offering some of the best support, guidance and coverage for a diverse employee population.




Our goal is to be essential to the world, which starts with our people. Company wide we kicked off an internal talent strategy program called Go Organic. At our core, we are committed to believing and investing in our workforce through:

  • Skill development: helping our employees grow their foundational skills

  • Finding the dream job at IBM: navigating our company with the potential for many careers by channeling an employee’s strengths and career aspirations

  • Diversity of people: Diversity of thought driving collective innovation


With an employee population of 375,000 in over 170 countries, amazingly we connect, collaborate, and care. IBMers drive a corporate culture of shared responsibility. We love grand challenges and everyday improvements for our company and for the world. We care about each other, our clients, and the communities we live, work, and play in!



Required Technical and Professional Expertise

  • Minimum 3 yrs exp working in Unix environment

  • Minimum 2 yrs exp in Hadoop, Spark, Hive/SQL

  • Minimum 2 yrs exp in Java and Scala

  • Minimum 1 yr exp w/HBase or other NoSQL Databases

  • Minimum 1 yr exp w/Oozie

  • Minimum 1 yr exp working with programming languages such as Python, experience with cloud & distributed systems principles, including load balancing, networks, scaling, in-memory vs. disk

  • Minimum 1 yr exp working with large-scale, big data methods, such as MapReduce, Hadoop, Spark, Hive, Impala, & HBase

  • Minimum 1 yr exp working in Unix/Linux environment with exp in source code management systems like GIT

Preferred Tech and Prof Experience

  • 4 years on hands-programming experience

  • Good ETL knowledge and prior data warehousing experience

  • Familiarity with Agile methodologies

  • Exposure to Application and Data integration using REST or Thrift or JMS

  • Experience on continuous build and integration using tools such as Gradle and Jenkins

  • Experience in JUnit and JMeter and MRUnit or other testing tools

  • Knowledge of Domain driven design

  • Good understanding of object oriented design & design patterns; Familiarity with agile software dev practices, testing strategies & solid unit testing skills

  • Experience with the following: cloud computing & virtualization, persistence technologies both relational & No-SQL & multi-layered distributed applications

EO Statement

IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.