On “Smart Scheduler, an Intelligence for Resource Optimization”
AIVO recently caught up with Manikya Swathi Vallabhajosyula (Swathi), PhD candidate and Graduate Research Assistant at The Ohio State University (OSU), who specializes in AI for High-Performance Computing (HPC) Resource Provisioning.
Swathi works with the AI institute for Intelligent Cyberinfrastructure with Computational Learning in the Environment (ICICLE). ICICLE is building the next generation of cyberinfrastructure (CI) to render AI more accessible to everyone and drive its further democratization in the larger society.
We talked about her work with ICICLE on the Smart Scheduler, an intelligence for resource optimization. Check out this excerpt from our interview:
AIVO: Swathi, you’re working with ICICLE on the Smart Scheduler, an intelligence for resource optimization. What prompted you to delve into this research area?
Swathi: This quote from the movie 3 Idiots has been my research inspiration: “Anything that reduces human effort is a machine.” Whether it’s a pen designed for space or an AI tool optimizing HPC workloads, machines should simplify and accelerate human tasks. However, humans don’t always know how to use machines effectively.
When I began my PhD, I was paired with Dr. Rajiv Ramnath, who introduced me to a fascinating project. I attended his lab meetings, which included outreach to high school students, where I saw researchers at all levels struggling to use HPC effectively.
I dove into these challenges through a collaboration with researchers from the Department of Evolution, Ecology, and Organismal Biology (EEOB). They were focused on genome sequencing but often faced technical roadblocks when using HPC systems. Many relied on legacy scripts handed down through labs—scripts that frequently broke due to outdated dependencies or system updates. Troubleshooting often took hours or days, pulling researchers away from their core scientific work.
As I continued working on the ICICLE project, I saw similar patterns in other domains. Animal ecologists, for example, spent significant time figuring out how to apply AI models for tasks like detecting human intrusion via camera traps and when to fine-tune those models using HPC resources. My lab mates working with generative AI models faced challenges in selecting the right resources, with single experiments costing upwards of $120–$160. Across disciplines, it became clear: researchers needed intelligent HPC middleware that could recommend optimal resources and simplify job submissions to accelerate their path to discovery.
To address this, I began engaging directly with graduate students to observe how they interacted with supercomputing environments. This helped shape my research focus around key questions like:
- How can we avoid unnecessary full-node allocations?
- How can we better utilize clusters with minimal human oversight?
I also explored solutions for handling jobs that don’t neatly fit existing allocation policies, seeking ways to execute them efficiently through smarter job monitoring systems. A recurring issue was over-allocation, because researchers simply used default scripts, highlighting the need for dynamic resource matching and script optimization.
While CI platforms provide documentation and guides, they can be outdated or overlooked. Researchers copy templates from colleagues, only tweaking them after failures occur. Large Language Models (LLMs) can only offer general help since they are not aware of the specific environment the researcher is working in.
For stable domains like genome sequencing, resource estimation is manageable because workflows and tools remain consistent. But in areas like deep learning, where architectures and parameters change frequently, predicting resource needs becomes far more complex. This calls for adaptive frameworks that can account for evolving neural network designs while providing accurate resource guidance.
Hence, my thesis is centered around developing AI-driven frameworks that intelligently predict and allocate HPC resources, reducing researchers’ “time-to-science” and enabling them to focus on advancing their discoveries.
AIVO: Can you give us some details about your research? What is the goal of your project and/or the practical or real-world use of the Smart Scheduler?
Swathi: My research focuses on developing AI-driven solutions to optimize resource utilization in high-performance computing (HPC) environments, with a strong emphasis on enabling domain scientists to integrate AI into their workflows seamlessly.
I began working with ICICLE at its early stages by integrating my prototyped framework, HARP (HPC Application Resource Predictor), into Deep Neural Network (DNN) workflows. This involved creating training profiles and estimation models to predict training times for backbone neural networks like AlexNet, ResNet, and EfficientNet. I extended this work to domain-specific AI tools such as MegaDetector, which uses a YOLO (You Only Look Once)-based backbone for tasks like intrusion detection and species classification in animal ecology.
Through this translational science approach, I recognized the challenges faced by researchers, especially in fields like ecology, when applying AI without deep expertise in HPC or resource management. This led to the development of the Smart Scheduler, a user-centric framework designed to reduce “time-to-science” by automating resource estimation, job orchestration, and AI model deployment across both central HPC systems and edge devices.
ICICLE, where my work is anchored, aims to inject intelligence into cyberinfrastructure (CI) to support user-inspired domains such as agriculture, animal ecology, and smart foodsheds. Within ICICLE’s AI4CI thrust, my focus is on resource estimation, smart job allocation, and optimizing HPC usage. The Smart Scheduler complements tools like the ML (Machine Learning) Field Planner and ML Workbench, providing middleware that simplifies AI adoption for scientists by answering critical questions like:
- Which AI models are best suited for deployment on resource-constrained edge devices?
- How can I efficiently train or fine-tune models given my hardware limitations and energy constraints?
For example, in collaboration with ecologists through a project led by Dr. Joe Stubbs at the Texas Advanced Computing Center (TACC) as part of the SGX3 Summer Fellowship, I worked on deploying AI models (YOLO family, MegaDetector) for wildlife monitoring using camera traps and drones. These edge devices have strict power and memory constraints. Using the Smart Scheduler and ML Field Planner, we evaluated various AI models, debunking common myths like “newer models are always better.” In fact, older versions of MegaDetector outperformed newer, lightweight versions on domain-specific datasets, emphasizing the need for tools that guide researchers in selecting and tuning models based on both performance and resource efficiency.
The goal of the Smart Scheduler is to act as a bridge between AI capabilities and scientific workflows, making HPC and AI resources more accessible, efficient, and sustainable. It automates complex tasks like:
- Estimating compute, memory, and energy requirements.
- Allocating cost-effective resources.
- Managing end-to-end AI training and inference tasks.
In essence, tools like the Smart Scheduler, ML Field Planner, and ML Edge function as science gateways that simplify access to computational resources, reduce overhead, and accelerate scientific discovery. They empower domain experts—without deep technical backgrounds—to leverage AI effectively, whether in labs, on HPC clusters, or in remote field environments with edge devices.
AIVO: What’s the role of ICICLE in your project? How do you work with their team to move your project forward?
Swathi: ICICLE plays a foundational role in my research by providing both the vision and collaborative ecosystem needed to develop intelligent CI solutions like the Smart Scheduler. As part of the AI4CI thrust within ICICLE, my work directly aligns with its mission to make AI more accessible and to empower domain scientists through smarter, AI-driven resource management across HPC and edge environments.
ICICLE brings together interdisciplinary teams spanning computer science, AI, CI, and domain sciences such as agriculture, animal ecology, and environmental studies. This collaborative environment has been instrumental in shaping my project. By working closely with ICICLE researchers, software engineers, and domain experts, I gain critical insights into real-world challenges faced by scientists when integrating AI into their workflows, whether it’s deploying AI models on resource-constrained edge devices or optimizing HPC job submissions for complex workloads.
Being part of ICICLE’s middleware and tools group, I collaborate with cyberinfrastructure (CI) researchers to build software stacks that sit directly above the core infrastructure. This includes contributing to a suite of tools such as the AI-Aware Adaptive Scheduler, the CKN (Cyberinfrastructure Knowledge Network) for capturing runtime statistics and monitoring tasks across edge and central systems, Patra Model Cards for efficient model metadata management, wall-time predictors, CI-centric queue wait-time estimators, and smart compilers. Together, these tools form an integrated ecosystem designed to streamline AI deployment and optimize resource usage for scientific applications.
Looking ahead, energy efficiency and sustainability are key areas of future work. Collaborating with researchers in fields like animal ecology, who deploy drones and camera traps in situ, presents a valuable set of real-world challenges. These collaborations help us simulate realistic deployment scenarios, enabling the design of more adaptive and energy-aware solutions tailored for field conditions where power and memory constraints are critical.
In summary, ICICLE serves as both the backbone and catalyst for my project, offering the collaborative network, technical infrastructure, and domain-driven focus necessary to advance AI-powered, resource-optimized CI. Working within this dynamic environment allows me to continuously refine solutions that bridge complex CI systems and the needs of domain scientists, accelerating discovery while paving the way for sustainable computing practices.
AIVO: How did you find out about the opportunity to have a project with ICICLE?
Swathi: I found out about the opportunity to work with ICICLE when I started my PhD at The Ohio State University (OSU) under Dr. Rajiv Ramnath, who was a Co-PI on the ICICLE project. Initially, I was involved in an NSF EAGER grant with him, which focused on addressing challenges researchers face when using HPC systems. As my work aligned closely with ICICLE’s mission, I was introduced to the AI4CI thrust, where I could apply my interest in AI-driven resource optimization to a broader CI context.
From there, my involvement grew naturally. I began collaborating with teams across ICICLE, contributing to middleware solutions that enhance HPC usability for domain scientists. This opportunity not only expanded the scope of my research but also connected me with a vibrant, interdisciplinary community focused on advancing intelligent CI.
AIVO: How did you interact with systems, and how did OSU shape your research? What are some ways you plan to use your degree after you graduate?
Swathi: My journey with systems began long before I formally entered the field of computer science. From my early fascination with Microsoft tools and learning the power of automation through formulas, I developed a deep curiosity about how systems operate and how they can simplify human effort. And “Clippy” (the earliest Microsoft Agent) was my pal in that learning journey. This curiosity evolved through my undergraduate years, where I was introduced to core computing concepts like algorithms, system patterns, and performance testing, laying the groundwork for my understanding of computational efficiency.
However, it was at OSU where my interaction with complex systems truly transformed into focused research. When I started my PhD at OSU, I had the opportunity to collaborate with researchers from the Department of Evolution, Ecology, and Organismal Biology (EEOB) and Animal Ecology. Observing genomic researchers interact with the Ohio Supercomputer Center (OSC) opened my eyes to the real-world challenges scientists face when leveraging high-performance computing (HPC). Many relied on outdated scripts and manual interventions, which delayed their scientific progress. This experience was pivotal—it shaped my core research question: “How can we make HPC environments smarter and more accessible for domain scientists?”
At OSU, working on projects like ICICLE provided the perfect environment to explore this intersection of AI, HPC, and user-centric design. Through coursework like Advanced Artificial Intelligence, Neural Networks, and High-Performance Deep Learning (HiDL), as well as collaborations and exposure to cutting-edge projects, I refined my focus on developing AI-driven middleware solutions. Initiatives like HARP (HPC Application Resource Predictor) and the Smart Scheduler emerged from this ecosystem, aiming to reduce “time-to-science” by optimizing resource allocation and job scheduling.
Moreover, OSU’s collaborative culture allowed me to work across disciplines—from genomics to animal ecology—understanding diverse scientific workflows and tailoring solutions to real user needs. Working with ICICLE enabled me to attend several conferences like NuEurips (Neural Information Processing Systems), SC (Super Computing), PEARC (Practice & Experience In Advanced Research Computing), and many workshops like NAIRR (National Artificial Intelligence Research Resource Pilot) and 5NRP (Fifth National Research Platform), which further expanded my perspective on how intelligent cyberinfrastructure (CI) could democratize AI for researchers.
In essence, my interaction with systems evolved from learning how to operate within them to innovating ways to improve them. OSU didn’t just shape my technical skills—it shaped my approach to research by grounding it in practical impact, interdisciplinary collaboration, and a commitment to making advanced computing more intuitive and efficient for the scientific community.
Throughout my journey at OSU, I’ve realized that my passion extends beyond research. I am deeply committed to mentorship, collaboration, and contributing to the growth of future scientists and engineers. During my time at OSU, I have actively embraced opportunities to mentor and lead. I’ve guided 10 master’s students and 4 undergraduates through their research projects, helping them navigate complex topics in AI, HPC, and CI. Additionally, I launched a summer research initiative in India, where I mentored nearly 200 students, introducing them to hands-on computing projects and fostering a culture of curiosity and innovation.
Through my leadership roles, such as serving as the ICICLE NextGen Student Group Manager and leading AI outreach initiatives for K–12 students in collaboration with OHI/O (a program that fosters a tech culture at OSU and its surrounding communities), I have further strengthened my dedication to building inclusive and supportive research communities. I’ve also been actively involved in the broader HPC community as a student volunteer at PEARC, which has provided me with valuable insights into international research collaboration and community engagement.
Looking ahead, I see a postdoctoral fellowship as a critical next step—not only to advance my research in AI-driven CI but also to expand my role as a mentor and emerging research leader. A postdoc will provide the platform to formally mentor graduate students and PhD candidates, fostering their development while continuing to build collaborative, interdisciplinary teams. It will also allow me to gain deeper experience in grant writing, proposal development, and managing the full lifecycle of research projects—from securing funding to delivering impactful outcomes.
Ultimately, my goal is to lead my own research lab, where I can drive innovation at the intersection of AI, HPC, and scientific discovery, while cultivating a strong mentoring culture. The experience I’ve gained at OSU has laid a solid foundation in both technical expertise and leadership, and I believe a postdoctoral role will further equip me with the skills needed to manage end-to-end research programs, balancing groundbreaking research with the responsibility of guiding the next generation of scholars.
Many thanks to Swathi for taking time to talk with us! We’re excited to hear of your accomplishments and wish you a promising career ahead!
About ICICLE
Converging under one virtual roof, ICICLE will foster interdisciplinary communities, advance foundational AI and CI, and transform application domains. Through its innovative approach to training and technology transfer, ICICLE will grow an AI-enabled workforce and incubate innovative companies with sustained capability-building at all levels.
