Job Overview

We are seeking a highly skilled Machine Learning Engineer with expertise in Large Language Models (LLMs) and AWS infrastructure to join our team. You will be responsible for building a robust data pipeline that ingests financial documents, performing data transformations, and training and fine-tuning LLMs for retrieval-augmented generation (RAG).

Key Responsibilities:

• Data Ingestion & Processing: Build and maintain a data pipeline to retrieve, clean, and structure data using AWS services (Lambda, S3, Glue, etc.).

• LLM Training & Fine-Tuning: Train, fine-tune, and optimize Large Language Models (LLMs) like LLaMA using AWS SageMaker and implement retrieval-augmented generation (RAG) techniques for improved accuracy.

• AWS Architecture: Design and implement scalable AWS infrastructure for data storage, ETL processes, model training, and inference (S3, SageMaker, Glue, Athena, OpenSearch, etc.).

• API & Frontend Interface Development: Build and integrate REST APIs (using AWS API Gateway and Lambda) that connect the LLMs with a frontend interface (AWS Amplify/CloudFront).

• Search and Retrieval: Implement search functionality using AWS OpenSearch for document indexing and retrieval to support LLM fine-tuning and generate accurate, contextualized responses.

• Optimization: Work on performance tuning and cost optimization across AWS services to ensure high availability, low latency, and efficiency.

Required Qualifications:

• Strong experience with AWS services, including Lambda, S3, SageMaker, Glue, Athena, and OpenSearch.

• Hands-on experience with model training, deployment, and inference in cloud environments (preferably AWS SageMaker).

• Proficiency in Python, including data processing and model training libraries (PyTorch, TensorFlow, Hugging Face, etc.).

• Experience working with financial data and/or document retrieval systems is a plus. Strong understanding of ETL processes and building scalable data pipelines.

• Excellent problem-solving skills, and ability to work independently and collaboratively in a fast-paced environment.

Preferred Qualifications:

• Experience with retrieval-augmented generation (RAG) techniques.

• Familiarity with deploying user interfaces on AWS (Amplify, CloudFront). ● Knowledge of financial markets, SEC filings, and investment data is highly advantageous.