Automating Custom Loss Functions in Deep Learning
Designing and training deep learning models often requires advanced expertise, particularly when defining loss functions, optimizers, and model architectures. This work explored how large language models could lower those barriers by automating the creation of custom loss functions tailored to specific optimization goals.
The capstone was presented at the Graduate Poster Exhibition during the 2025 SPARK! (Showcase of Projects, Art, Research, and Knowledge), a reimagining of Research Week that highlighted graduate research and creative work across disciplines. Developed within the Rutgers University–Camden Data Science M.S. program, the project was completed by Naimish Sharma and Gaurav Shetty. The abstract below introduces their work, AI-Powered Custom Loss Function Generation and Beyond for Deep Learning Models.
Abstract: AI-Powered Custom Loss Function Generation and Beyond for Deep Learning Models
Training deep learning models requires careful selection and design of loss functions, activation functions, optimizers, and network architectures, often demanding significant expertise. This capstone project aimed to automate the generation of custom loss functions using Llama7B, a large language model, combined with advanced prompt engineering techniques.
The proposed system allows users to upload a code snippet and a query describing their optimization objective. Based on this input, the AI-driven assistant generates a custom loss function tailored to the task, along with a unit test to validate its correctness. The system ensures that the generated function adheres to PyTorch’s interface and performs robustly under testing conditions.
Beyond loss functions, the approach and architecture can be extended to other key aspects of machine learning, including activation functions, optimizer selection, and model architecture decisions such as determining the number of layers in a neural network. By integrating large language models for automated code generation, the project reduces the complexity of deep learning development, making it more accessible to researchers and practitioners. This framework aligns with broader innovations in AI-assisted model training, including meta-learning and automated architecture selection, streamlining workflows while minimizing reliance on manual coding expertise.
Graduate Poster Exhibition at SPARK!
The Graduate Poster Exhibition celebrates the research and creative work of the graduate community, showcasing everything from prose and code to original research and artistic expression. As part of SPARK! (Showcase of Projects, Art, Research, and Knowledge), a reimagining of Research Week, the exhibition highlights the depth, range, and impact of graduate scholarship and invites the campus community to engage with ideas taking shape across disciplines.
Mastering Data: The Data Science Program
The Master of Science in Data Science at Rutgers University–Camden is an interdisciplinary program that equips students with skills in data analysis, machine learning, and statistical modeling. The curriculum integrates tools from mathematics, computer science, public policy, and prevention science, emphasizing practical applications across various industries. Core courses include Foundations of Data Science, Statistical Methods for Data Science, Data Visualization, and Applied Data Mining and Machine Learning. Students also engage in a thesis or capstone project, applying their knowledge to real-world problems.
Spotlights & Stories
Want more stories like this? Spotlights and Stories highlights alumni, faculty, staff, and student experiences through stories, video, and voice. Learn more
Explore Graduate School Programs
Rutgers Graduate School-Camden offers 20+ graduate certificates, master’s, and doctoral programs across various disciplines, including biology, data science, creative writing, and psychology. We take pride in our academic diversity. Learn more
Former Students, Get Involved
Stay connected with Rutgers-Camden! Engage with former students, attend events, and support current students. Your involvement strengthens our community and helps shape future success. Learn more