Jobs for Veterans, Veteran Job Board |

Post Jobs

Job Information

Shell Senior Data Engineer, Energy Platform - San Francisco, CA in San Francisco, California


Job Title:

Senior Data Engineer, Energy Platform - San Francisco, CA

No. of Positions:


Job Description:

If you want to bring your technical expertise, ‘challenge-accepted!’ mentality and passion for building amazing products with real-world impact for hundreds of millions of people come join our group! We have been looking for you! We have an amazing team with deep backgrounds across technology and energy.

Where you fit in

Shell is leading the transition towards a low-carbon future. We aim to cut the net carbon footprint of our energy products in half by 2050. Our New Energies business, set up in 2016, supports this ambition. New Energies is an emerging opportunity, in which we plan to invest on average $1-2 billion a year until 2020 as we look for commercial investments in new and fast-growing segments of the energy industry.

Shell New Energies focuses on two areas: new fuels and power. New Fuels consists of investments in hydrogen, biofuels, and electric vehicle charging. In Power, we are building up positions across the full electricity value chain, including in renewable generation, retail energy, distributed energy resources, power trading and marketing, and grid services. Within these focus areas, we look for ways to connect customers with new business models for mobility and energy services, enabled by digital technologies and decentralization of energy systems. Development of our IoT platform for Shell New Energy will be paramount in achieving these goals.

What's the role?

The Energy Platform team is a nimble, cross-functional, deeply technical and passionate group that embodies the speed and agility of a startup while embracing the scale of one of the largest companies in the world. Achieving a balance between agility and global scale provides unique opportunities, and the Energy Platform Team borrows from best-in-class product development, continuous delivery, and commercialization techniques while adapting them to the unique global context within Shell.

The Energy Platform team is empowered to coordinate and align Shell’s energy management platform objectives, strategies, and execution approaches across the company, as well as to design, deliver and maintain a mission-critical component of Shell’s ability to deliver differentiated products, offerings,and capabilities across its expanding global footprint.

We are looking for a Senior Data Engineer with the ability to bring their expertise and excitement for solving complex problems while building one of the largest IoT platforms in the world. There will be no shortage of opportunities to lead, eat, drink, and be merry with the most dynamic team ever assembled in the energy industry.

  • Build, on a daily basis, real-time and big data processing pipelines, optimizing for scalability and performance, under multiple datastore concepts (Relational, NoSQL, Graph).

  • Proficient in building large scale ETL jobs, leveraging big data infrastructure (Hadoop, Spark, Kafka) and modern container orchestration environments (Kubernetes).

  • Comfortable working in a fast-paced environment building, running, testing and shipping data pipelines to serve ML/AI workloads under a common API.

  • Willing to work with a cross-functional team of market analysts, data scientists & software developers to translate their data needs into features inside the Energy Platform.

  • Create data tooling that assists data scientists and analysts in building low latency, scalable and resilient pipelines for machine learning and optimization workloads.

  • Enthusiast of data quality, lifecycle and provenance management, helping establish a DataOps centric culture within Engineering teams.

  • Advanced working knowledge of query authoring and tooling for cross source data aggregation: APIs, 3rd party DB, Object Storages, messaging bus.

  • You have a proven history of working on large scale ETL jobs for data wrangling and cleansing of IoT time-series & telemetry data, as well as IIoT unstructured datasets, focusing on serving machine learning orchestrations.

  • You have experience with Analytical Expression Compiling languages, for runtime analytics during SQL querying.

Auto req ID:



IT Data and Analytics, Information

Country of Work Location:

United States

Employment Type:

Full Time

Company Description:

Shell is a global group of energy and petrochemical companies with about 84,000 employees across more than 70 countries. We aim to meet the world’s growing need for more and cleaner energy solutions in ways that are economically, environmentally and socially responsible. We have expertise in exploration, production, refining and marketing of oil and natural gas, and the manufacturing and marketing of chemicals. As a global energy company operating in a challenging world, we set high standards of performance and ethical behaviors. We are judged by how we act and how we live up to our core values of honesty, integrity and respect for people. Our Business Principles are based on these. They promote trust, openness, teamwork and professionalism, as well as pride in what we do and how we conduct business.Building on our core values, we aspire to sustain a diverse and inclusive culture where everyone feels respected and valued, from our employees to our customers and partners. A diverse workforce and an inclusive work environment are vital to our success, leading to greater innovation and better energy solutions.

An innovative place to work

There’s never been a more exciting time to work at Shell. Everyone here is helping solve one of the biggest challenges facing the world today: bringing the benefits of energy to everyone on the planet, whilst managing the risks of climate change.

Join us and you’ll add your talent and imagination to a business with the power to shape the future – whether by investing in renewables, exploring new ways to store energy or developing technology that helps the world to use energy more efficiently.

An inclusive place to work

To power progress together, we need to attract and develop the brightest minds and make sure every voice is heard. Here are just some of the ways we’re nurturing an inclusive environment – one where you can express your ideas, extend your skills and reach your potential…

  • We’re creating a space where people with disabilities can excel through transparent recruitment process, workplace adjustments and ongoing support in their roles. Feel free to let us know about your circumstances when you apply, and we’ll take it from there.

  • We’re closing the gender gap – whether that’s through action on equal pay or by enabling more women to reach senior roles in engineering and technology.

  • We’re striving to be a pioneer of an inclusive and diverse workplace, promoting equality for employees regardless of sexual orientation or gender identity.

  • We consider ourselves a flexible employer and want to support you finding the right balance. We encourage you to discuss this with us in your application.

A rewarding place to work

Combine our creative, collaborative environment and global operations with an impressive range of benefits and joining Shell becomes an inspired career choice.

We’re huge advocates for career development. We’ll encourage you to try new roles and experience new settings. By pushing people to reach their potential, we frequently help them find skills they never knew they had, or make career moves they never thought possible.


Please note: We occasionally amend or withdraw Shell jobs and reserve the right to do so at any time, including prior to the advertised closing date.

Before applying, you are advised to read our data protection policy. This policy describes the processing that may be associated with your personal data and informs you that your personal data may be transferred to Royal Dutch/Shell Group companies around the world.

The Shell Group and its approved recruitment consultants will never ask you for a fee to process or consider your application for a career with Shell. Anyone who demands such a fee is not an authorised Shell representative and you are strongly advised to refuse any such demand.

Shell participates in E-Verify.

All qualified applicants will receive consideration for employment without regard to race, color, sex, national origin, age, religion, disability, sexual orientation, gender identity, protected veteran status, citizenship, genetic information or other protected status under federal, state or local laws.

Shell is an Equal Opportunity Employer - Minorities/Females/Veterans/Disability.

Work Location:

California - San Francisco


What we need from you

  • Must have legal authorization to work in the US on a full-time basis for anyone other than current employer

  • Bachelor's degree in a relevant technical discipline.

  • 8+ years of experience with demonstrable proficiency in one or more DB and data pipeline tooling: mySQL, PostgreSQL, OSI Pi historian, TimescaleDB, Streamsets, Apache Drill, Apache Parquet, Dremio …

  • You are passionate about building scalable, high performing, data pipelines, and analytic catalogs.

  • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc

  • Proficient with Containerized environments and workloads

  • Excellent analytical, problem-solving, and troubleshooting skills.

  • Experience architecting, deploying, and supporting production applications.

  • You care deeply about performance, accessibility and API design.

  • Great communication skills.

  • Work locations may include Houston, TX or San Francisco, CA

City, State (if applicable):

San Francisco, CA