Jobs for Veterans, Veteran Job Board |

Post Jobs

Job Information

Shell Senior Data Engineer, Energy Platform - San Francisco, CA in San Francisco, California


Job Title:

Senior Data Engineer, Energy Platform - San Francisco, CA

No. of Positions:


Job Description:

If you want to bring your technical expertise, ‘challenge-accepted!’ mentality and passion for building amazing products with real-world impact for hundreds of millions of people come join our group! We have been looking for you! We have an amazing team with deep backgrounds across technology and energy.

Shell New Energies

Shell is leading the transition towards a low-carbon future. We aim to cut the net carbon footprint of our energy products in half by 2050. Our New Energies business, set up in 2016, supports this ambition. New Energies is an emerging opportunity, in which we plan to invest on average $1-2 billion a year until 2020 as we look for commercial investments in new and fast-growing segments of the energy industry.

Shell New Energies focuses on two areas: new fuels and power. New Fuels consists of investments in hydrogen, biofuels, and electric vehicle charging. In Power, we are building up positions across the full electricity value chain, including in renewable generation, retail energy, distributed energy resources, power trading and marketing, and grid services. Within these focus areas, we look for ways to connect customers with new business models for mobility and energy services, enabled by digital technologies and decentralization of energy systems. Development of our IoT platform for Shell New Energy will be paramount in achieving these goals.

Energy Platform Team

The Energy Platform team is a nimble, cross-functional, deeply technical and passionate group that embodies the speed and agility of a startup while embracing the scale of one of the largest companies in the world. Achieving a balance between agility and global scale provides unique opportunities, and the Energy Platform Team borrows from best-in-class product development, continuous delivery, and commercialization techniques while adapting them to the unique global context within Shell.

The Energy Platform team is empowered to coordinate and align Shell’s energy management platform objectives, strategies, and execution approaches across the company, as well as to design, deliver and maintain a mission-critical component of Shell’s ability to deliver differentiated products, offerings,and capabilities across its expanding global footprint.

We are looking for a Senior Data Engineer with the ability to bring their expertise and excitement for solving complex problems while building one of the largest IoT platforms in the world. There will be no shortage of opportunities to lead, eat, drink, and be merry with the most dynamic team ever assembled in the energy industry.

  • Build, on a daily basis, real-time and big data processing pipelines, optimizing for scalability and performance, under multiple datastore concepts (Relational, NoSQL, Graph).

  • Proficient in building large scale ETL jobs, leveraging big data infrastructure (Hadoop, Spark, Kafka) and modern container orchestration environments (Kubernetes).

  • Comfortable working in a fast-paced environment building, running, testing and shipping data pipelines to serve ML/AI workloads under a common API.

  • Willing to work with a cross-functional team of market analysts, data scientists & software developers to translate their data needs into features inside the Energy Platform.

  • Create data tooling that assists data scientists and analysts in building low latency, scalable and resilient pipelines for machine learning and optimization workloads.

  • Enthusiast of data quality, lifecycle and provenance management, helping establish a DataOps centric culture within Engineering teams.

  • Advanced working knowledge of query authoring and tooling for cross source data aggregation: APIs, 3rd party DB, Object Storages, messaging bus.

  • You have a proven history of working on large scale ETL jobs for data wrangling and cleansing of IoT time-series & telemetry data, as well as IIoT unstructured datasets, focusing on serving machine learning orchestrations.

  • You have experience with Analytical Expression Compiling languages, for runtime analytics during SQL querying.

Auto req ID:



IT Data and Analytics, Information

Country of Work Location:

United States

Employment Type:

Full Time

Company Description:

Shell is a global group of energy and petrochemical companies with about 84,000 employees across more than 70 countries. We aim to meet the world’s growing need for more and cleaner energy solutions in ways that are economically, environmentally and socially responsible. We have expertise in exploration, production, refining and marketing of oil and natural gas, and the manufacturing and marketing of chemicals. As a global energy company operating in a challenging world, we set high standards of performance and ethical behaviors. We are judged by how we act and how we live up to our core values of honesty, integrity and respect for people. Our Business Principles are based on these. They promote trust, openness, teamwork and professionalism, as well as pride in what we do and how we conduct business.Building on our core values, we aspire to sustain a diverse and inclusive culture where everyone feels respected and valued, from our employees to our customers and partners. A diverse workforce and an inclusive work environment are vital to our success, leading to greater innovation and better energy solutions.


Please note: We occasionally amend or withdraw Shell jobs and reserve the right to do so at any time, including prior to the advertised closing date.

Before applying, you are advised to read our data protection policy. This policy describes the processing that may be associated with your personal data and informs you that your personal data may be transferred to Royal Dutch/Shell Group companies around the world.

The Shell Group and its approved recruitment consultants will never ask you for a fee to process or consider your application for a career with Shell. Anyone who demands such a fee is not an authorised Shell representative and you are strongly advised to refuse any such demand.

Shell participates in E-Verify.

All qualified applicants will receive consideration for employment without regard to race, color, sex, national origin, age, religion, disability, sexual orientation, gender identity, protected veteran status, citizenship, genetic information or other protected status under federal, state or local laws.

Shell is an Equal Opportunity Employer - Minorities/Females/Veterans/Disability.

Work Location:

California - San Francisco


  • Legal authorization to work in the US on a full-time basis for anyone other than current employer

  • Bachelor's degree in a relevant technical discipline.

  • 8+ years of experience with demonstrable proficiency in one or more DB and data pipeline tooling: mySQL, PostgreSQL, OSI Pi historian, TimescaleDB, Streamsets, Apache Drill, Apache Parquet, Dremio …

  • You are passionate about building scalable, high performing, data pipelines, and analytic catalogs.

  • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc

  • Proficient with Containerized environments and workloads

  • Excellent analytical, problem-solving, and troubleshooting skills.

  • Experience architecting, deploying, and supporting production applications.

  • You care deeply about performance, accessibility and API design.

  • Great communication skills.

  • Work locations may include Houston, TX or San Francisco, CA

City, State (if applicable):

San Francisco, CA