Data Engineer – AWS/ETL/Python

  • Software Engineering
  • Professional

Data Engineer – AWS/ETL/Python

  • Software Engineering
  • Professional

Introduction
At IBM, work is more than a job – it’s a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you’ve never thought possible. Are you ready to lead in this new era of technology and solve some of the world’s most challenging problems? If so, lets talk.

Your Role and Responsibilities
Day-to-day troubleshooting of forecasting systems, mainly working through data anomalies that cause inaccurate forecasts or prevent forecasts’ generation.
Collaborate with the data science team to enhance existing forecasting systems for the trade floors.
Create dynamic object-oriented methods, full stack solutions, and integrations to existing code solutions.
Develop individual Python classes, methods, functions that support the data flow of existing and new projects.
Work on code additions to seamlessly support projects for data flows, including logging and support, with little to no supervision.
Experience in modifying packages, testing, and repository instances to support CI/CD.

MXCON24

Required Technical and Professional Expertise
Design, develop, test, and deploy Python applications on AWS, ensuring high availability, scalability, and security
Develop and maintain technical documentation for Python applications and AWS infrastructure
Optimize application performance, scalability, and reliability using AWS services such as:
+ Serverless technologies (Lambda, API Gateway, Step Functions)
+ S3 for data storage and retrieval
+ Glue for data integration and ETL
+ SQS and SNS for message queuing and notification
+ Cognito for user authentication and authorization
+ CloudWatch for monitoring and logging

Preferred Technical and Professional Expertise
Implement automated testing, deployment, and monitoring using tools like Jenkins, Docker, and CloudWatch
Troubleshoot and resolve technical issues in Python applications and AWS infrastructure
Stay up-to-date with the latest developments in Python and AWS, and apply this knowledge to improve our applications and infrastructure
Design and develop Serverless front-end applications using AWS services such as API Gateway, Lambda, and S3
Implement PDS Proxy to handle data processing and analytics workloads
Integrate AWS services with external systems and APIs using APIs, SDKs, and other integration tools
Implement Row Level Security (RLS) to segregate data and ensure secure access to sensitive information
Design and implement data models and database schemas for DynamoDB and PostgreSQL
Design and develop data ingestion pipelines using AWS Lambda, Glue, AWS Batch and restful API’s
Thorough understanding of AWS cloud concepts and related technologies like AWS VPC, Subnets, AZ’s, SG, IAM policies & Roles, EC2, AWS ALB, API Gateway, ECS, RDS, AWS MSK, Kinesis, SQS, SNS, S3, DynamoDB, Secret Manager, Cloud Watch, Cloud Formation scrips(yaml/json), AWS CDK.
Collaborate with cross-functional teams to identify and prioritize project requirements
Knowledge of API unit & performance testing tools like postman, JMeter, SOAP UI is a plus.
Stay up to date with emerging technologies and trends, evaluating their potential impact on our projects.
Mentor and provide technical guidance to junior developers, fostering their growth and helping them overcome challenges.
Collaborate with cross-functional teams to integrate web applications with existing systems and third-party services.
Ensure compliance with development standards, security guidelines, and best practices.
Conduct code reviews to maintain high code quality, consistency, and adherence to coding standards.

Want to know what it’s like to be an IBMer?


About Business Unit

IBM Consulting is IBM’s consulting and global professional services business, with market leading capabilities in business and technology transformation. With deep expertise in many industries, we offer strategy, experience, technology, and operations services to many of the most innovative and valuable companies in the world. Our people are focused on accelerating our clients’ businesses through the power of collaboration. We believe in the power of technology responsibly used to help people, partners and the planet.


Your Life @ IBM

In a world where technology never stands still, we understand that, dedication to our clients success, innovation that matters, and trust and personal responsibility in all our relationships, lives in what we do as IBMers as we strive to be the catalyst that makes the world work better.

Being an IBMer means you’ll be able to learn and develop yourself and your career, you’ll be encouraged to be courageous and experiment everyday, all whilst having continuous trust and support in an environment where everyone can thrive whatever their personal or professional background.

Our IBMers are growth minded, always staying curious, open to feedback and learning new information and skills to constantly transform themselves and our company. They are trusted to provide on-going feedback to help other IBMers grow, as well as collaborate with colleagues keeping in mind a team focused approach to include different perspectives to drive exceptional outcomes for our customers. The courage our IBMers have to make critical decisions everyday is essential to IBM becoming the catalyst for progress, always embracing challenges with resources they have to hand, a can-do attitude and always striving for an outcome focused approach within everything that they do.

Are you ready to be an IBMer?


About IBM

IBM’s greatest invention is the IBMer. We believe that through the application of intelligence, reason and science, we can improve business, society and the human condition, bringing the power of an open hybrid cloud and AI strategy to life for our clients and partners around the world.

Restlessly reinventing since 1911, we are not only one of the largest corporate organizations in the world, we’re also one of the biggest technology and consulting employers, with many of the Fortune 50 companies relying on the IBM Cloud to run their business.

At IBM, we pride ourselves on being an early adopter of artificial intelligence, quantum computing and blockchain. Now it’s time for you to join us on our journey to being a responsible technology innovator and a force for good in the world.


Location Statement

For additional information about location requirements, please discuss with the recruiter following submission of your application.


Being You @ IBM

IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.

Key Job Details

Don’t see a fit at this time?

Don’t worry. Join our Talent Network and get notified about the latest opportunities.