# Lead Software Engineer - Pyspark, AWS, Python at JPMorganChase
Source: https://true-hire-ten.vercel.app/jobs/lead-software-engineer-java-python-aws-spark-jpmorganchase-eze27a5t · Company profile: https://true-hire-ten.vercel.app/companies/jp-morgan-chase · Apply on the employer's site: https://jpmc.fa.oraclecloud.com/hcmUI/CandidateExperience/en/sites/jobsearch/job/210786029?utm_source=freehire.me
## Job details
- Location: Pune, Maharashtra, India
- Employment type: Full-time
- Seniority tag: Lead
- Category: Software Engineering
- Countries: India
- Posted: 2026-11-06 (80 days ago, stale, reposted 2×)
- Skills named: accessibility, agile, ai, airflow, analytics, automation, avro, aws, bdd, big-data, ci-cd, cloud, cloudformation, data-engineering, data-governance, data-lake, data-modeling, data-pipelines, data-quality, data-vault, databricks, design-patterns, docker, hadoop, infrastructure-as-code, java, json, kafka, kubernetes, lakehouse, machine-learning, microservices, nosql, parquet, protobuf, python, root-cause-analysis, scala, secure-coding, serverless, snowflake, spark, sql, step-functions, tdd, terraform, test-automation, vault
## About the role
We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.

As a Lead Software Engineer at JPMorganChase within the Consumer and Community Banking, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.

**Job responsibilities**

- Lead the design, development, and maintenance of robust, scalable cloud-based data processing pipelines and infrastructure, ensuring adherence to engineering standards, governance frameworks, and industry best practices.

- Architect and refine data models for large-scale datasets, optimizing for efficient storage, high-performance retrieval, and advanced analytics while upholding data integrity and quality.

- Partner with cross-functional teams to translate complex business requirements into effective, scalable data engineering solutions that drive organizational value.

- Champion a culture of innovation and continuous improvement, proactively identifying and implementing enhancements to data infrastructure, processing workflows, and analytics capabilities.

- Define and execute data strategy, including the development of enterprise data models and the management of end-to-end data infrastructure—from design and construction to installation and ongoing maintenance of large-scale processing systems.

- Drive data quality initiatives, ensure seamless data accessibility for analysts and data scientists, and maintain strict compliance with data governance and regulatory requirements.

- Align data engineering practices with business objectives, ensuring solutions are both technically sound and strategically relevant.

- Author, review, and approve technical requirements and architectural designs, and lead process re-engineering efforts to deliver cost-effective, high-impact business solution

- Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.

- Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.

**Required qualifications, capabilities, and skills**

- Formal training or certification on software engineering concepts and 5+ years applied experience

- Expert in at least one distributed data processing framework (Spark). Expert in at least one cloud data Lakehouse platforms (AWS Data lake services or Databricks, if not Hadoop),

- Expert in at least one scheduling/orchestration tools ( Airflow, alternatively AWS Step Functions or similar) & Expert with relational and NoSQL databases. Expert in data structures, data serialization formats (JSON, AVRO, Protobuf, or similar), and big-data storage formats (Parquet, Iceberg, or similar)

- Hands-on professional experience in one or more programming language(s), including Java or Python, proficiency in Python, SQL, and at least one additional language (e.g. Java or Scala) for data engineering tasks

- Hands-on experience utilizing Apache Spark for large-scale data processing, including developing and optimizing data pipelines, performing real-time and batch analytics, and leveraging Spark’s libraries for machine learning and data transformation to drive actionable business insights.

- Proficiency in microservices architecture, serverless computing and distributed cluster computing tools such as Docker, Kubernetes etc. Experience in one or more data modelling techniques (Dimensional, Data Vault, Kimball, Inmon, etc.)

- Experience with test-driven development (TDD) or behavior-driven development (BDD) practices, as well as working with continuous integration and continuous deployment (CI/CD) tools.

- Experience organizing and leading design workshops, coding sessions, and hackathons to promote a culture of excellence and innovation in data engineering. Expertise in architecting reusable, future-ready design patterns that address diverse use cases across the organization.

- Expertise in working with streaming platforms like Kafka, MQ etc.

- Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.

- Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices

**Preferred qualifications, capabilities, and skills**

- Hands-on experience with Infrastructure as Code (IaC) tools, preferably Terraform; experience with AWS CloudFormation is also valued.

- Proficiency in cloud-based data pipeline technologies such as Spinnaker or similar platforms.

- Strong working knowledge of the Snowflake data platform.

- Experience in budgeting and resource allocation for data engineering projects.

- Proven ability to manage vendor relationships effectively.
© TrueHire. Listing data from freehire; applications go to the employer's own page.