Introduction
At IBM, work is more than a job – it’s a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you’ve never thought possible. Are you ready to lead in this new era of technology and solve some of the world’s most challenging problems? If so, let’s talk.

Your Role and Responsibilities
We are seeking an experienced Data Engineer to join the Asset Engineering team. In this role, you will be responsible for building innovative AI-powered solutions that integrate with various systems and applications. You will also be working with the broader team to build, analyse and improve the AI solutions.

Required Technical and Professional Expertise

• 7+ years of professional experience; 5+ years of experience in Data Engineer role.
• Proficiency in Python programming languages. Familiarity with Data Analytics Libraries (e.g., Pandas, Spark, Polars etc.) & exposure to RESTful APIs.
• Develop and maintain ETL (Extract, Transform, Load) pipelines to gather, transform and store data in knowledge bases. This includes maintaining data pipelines for both vectorized unstructured data in vector DB and structured data in relational databases.
• Design and implement advanced retrieval strategies to optimize the performance (both speed and accuracy) of data retrieval from the knowledge bases.
• Build data management toolkits for automating CRUD (Create, Read, Update and Delete) operations of the knowledge bases using MySQL or PostgreSQL.

Preferred Technical and Professional Expertise

• Experience in designing and implementing large-scale AI solutions, including data ingestion, storage, processing, and deployment.
• Implement security measures to protect data at rest and in transit. Also ensure that data handling practices comply with relevant regulations and standards.
• Experience with DevOps tools including GitHub Enterprise, CI/CD, containerization using Docker and Kubernetes.