IBM Big Data Software/API Engineer in SAN FRANCISCO, California

You will be responsible for helping build an enterprise platform that is performant for an enterprise-wide data lake and cognitive services on the cloud. You will work closely with other developers and internal IBM teams using the platform. Success in this role requires the ability to collaborate successfully with cross-functional teams. Given the varied types of people you will work with, both technical and not, the ability to communicate effectively in person, in writing, over the phone, on a whiteboard, or on a video conference is essential. Successful candidates should come from a strong data engineering background and must have significant experience with the backend, APIs, cloud and big data technologies as well as traditional data warehouses. The ideal candidate will have significant experience with much of the following or equivalent:- Experienced with Java development (must have at least 5 years of experience)- Experience with developing and deploying RESTful/RPC-based services (must have at least 2 years of experience)- Experience working with relational databases and solid knowledge of SQL (must have at least 3 years of experience)- Experience with using search tools (e.g. ElasticSearch, Solr) (highly desirable)- Experience working with Big Data tools in the Hadoop ecosystem (highly desirable)- Experience with Enterprise authentication and authorization tools (highly desirable)- 7+ years of big data software/API or backend software experience in reputed companies- Strategic thinker with proven ability to build and manage scalable big data cloud systems using open source tools and big data technologies such as Hadoop, HBase, Spark, etc.- Experience in cloud-based Agile DevOps experience with PaaS & IaaS- Experience with SQL, NoSQL, relational database design and methods for efficiently retrieving data- Experience evangelizing solutions internally and train users on how to best optimize usage of platform- Experience navigating internal processes for data, support and hardware resources- Experience collaborating with highly technical people and comfortable doing so; team leadership is a plus- Ability to communicate effectively in person, in writing, over the phone, on a whiteboard or on a videoconference is essential Specific Technical Skills and Knowledge:- JAVA REST API, RPC- ElasticSearch- Data knowledge, query languages, SQL, BigSQL, NoSQL- Test-driven development methodology, JUnit, JMeter- Hadoop, Oozie, Python- Spark, Scala, Node.js, Flink and other big data analytics technologies- Docker, containers, K8s- Authentication, JWT, OAuth2- Big data ingestion, processing, querying and tools- API, middleware and software development- Cloud development and administration- DevOps and Continuous Integration- Agile Software Development, teamwork and collaboration- Technical leadership >Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

  • 5+ years of related work experience: in software engineering, architecture and design-designing and delivering large scale distributed software systems, preferably in large scale global business, service oriented architecture (SOA) principles and Web services standards and best practices

  • Hands-on experience as a software architect designing and delivering large scale distributed software systems, preferably in large scale global business

  • Capability to architect highly scalable distributed systems using open source tools and big data technologies such as Hadoop, HBase, Spark, Storm, Etc

  • Experience in: cloud-based agile, devops experience with PaaS & IaaS, using Big Data batch and streaming tools,SQL, NoSQL, relational database design, and methods for efficiently retrieving data

  • Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)

  • Programming in Java, Scala, Python, SQL

IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.