PhD Position (A Foundation Model and Graph Transformer Networks for Big Data Analytics)

Project Description

Global digital transformation increasingly relies on cloud computing for big data analytics, where a growing blend of SQL and Machine Learning (ML) workloads is becoming common. Achieving effective cost-performance optimization in this context requires precise models of SQL and ML workloads. However, a key challenge arises: How can we develop a universal modeling approach that effectively generalizes across millions of cloud users? This challenge is compounded by significant workload diversity (with each user exhibiting a unique workload), data variability (ranging from sparse to extensive telemetry data), and cluster heterogeneity (hundreds of cloud instances with diverse software and hardware configurations). While numerous modeling techniques have been developed, they typically train individual models for each user workload, leveraging tree-based methods or, more flexibly, graph neural networks. However, these models often suffer from high error rates, largely due to insufficient training data.

To tackle this challenge, this PhD project aims to pioneer the development of a foundational model for Relational Algebra, the core representation underpinning SQL queries in big data analytics. Building on our extensive experience with Graph Transformer Networks (GTNs) from the previous ERC project, our research will take a deep dive into GTN architectures, systematically deconstructing them across various design dimensions. For each dimension, we will innovate new methods specifically tailored to the unique requirements of relational algebra. In addition, we will develop self-supervised learning techniques adapted to relational algebra to scale training effectively. We also plan to explore novel optimizations to accelerate training while reducing computational complexity.

The second objective of this project is to extend our modeling solution to encompass non-SQL tasks, such as machine learning workloads. We will introduce a novel representation learning paradigm for non- SQL workloads, designed to capture workload characteristics from incomplete and noisy telemetry data. This paradigm will integrate seamlessly with our SQL-based modeling approach, utilizing a generalized Graph Transformer Network that accommodates both SQL and non-SQL tasks in a unified framework.

The third objective of this project is to fine-tune the foundation model or the generalized graph transformer network for different user workloads and apply them to assist in multi-objective optimization of cloud analytics workloads, thereby enabling the best cost performance for each cloud user.