Staff Software Engineer, AAI Research

Google Cloud's software engineers develop next-generation technologies that change how billions of users connect, explore, and interact with information and each other. The Google Cloud AI Research team addresses AI challenges motivated by Google Cloud’s mission of bringing AI to tech, healthcare, finance, retail and many other industries. We work on unique problems focused on research topics that maximize scientific and real-world impact, aiming to push the state-of-the-art in AI and share findings with the broader research community. We also collaborate with product teams to bring innovations to real-world impact that benefits our customers.

• Execute quality iterations on reference agents, identifying potential model or agent framework improvements.
• Source or facilitate creation of datasets for evaluation and training to improve model performance for Cloud customers.
• Design new agentic systems and context engineering algorithms, utilizing existing models and identifying the need for new training/evaluation objectives where necessary.
• Define and implement metrics that correspond to business problems.
• Prototype and iterate on the solution, working closely with customers, product management and business development.

• Bachelor's degree or equivalent practical experience.
• 8 years of experience working with Large Language Models (LLMs) and building agents.
• Solid understanding of essential concepts such as tokens, context, Retrieval-Augmented Generation (RAG), and function calling.
• Solid understanding of core ML concepts, training algorithms, and best practices for evaluation.
• Strong understanding of data and familiarity with basic statistical analysis concepts (e.g., variance, p-value, bias) and general data science principles.
• Familiarity with running quality interactions, involving understanding business goals, defining technical metrics, implementing evaluation frameworks, designing experiments, analyzing results, performing RCAs, formulating hypotheses, and conducting ablation or live experiments.
• Proficiency in Python.

Google Cloud is dedicated to developing technologies that transform how billions of users connect and interact, with a mission to bring AI to various industries including tech, healthcare, finance, and retail.

The agent benchmark engineering team drives the discovery and expansion of next-generation agent capabilities, empowering customers to deploy high-performance AI systems. The team delivers robust, production-ready agentic systems by leading the engineering of the ML quality feedback loop, including hypothesis-driven quality iteration, experimental systems optimization, and data engineering for analysis.