Deep Learning School 2025

• When?

June 23rd, 2025 to July 4th, 2025

• Where?

Campus SophiaTech, Sophia Antipolis

• Language

English

• Target audience

Engineers, master and doctorate students, researchers

• Pre-requisite 

▸ Master of Science
▸ If you do not have strong background and practice in machine learning, you are strongly advised to register to tutorials, additionally to conferences
▸ Being currently employed or seeking a job

External companies / Individuals

Price per week (tutorials and conferences): 900 €
Price per week (conferences only): 500 €

Register

Partnering companies / Individuals

Price per week (tutorials and conferences): 810 €
Price per week (conferences only): 450 €

Register

External academic staff

Price per week (tutorials and conferences): 630 €
Price per week (conferences only): 350 €

Register

Academic staff from EFELIA Côte d’Azur consortium

Price per week (tutorials and conferences): 220 €
Price per week (conferences only): 220 €

Register

Week 1
Tutorials: Monday and Tuesday
Conferences: Wednesday, Thursday, and Friday

Week 2
Tutorials: Monday and Tuesday
Conferences: Wednesday, Thursday, and Friday

Prices include lectures with labs or tutorials with labs for each day.

Monday, June 23

Tutorial and lab: Build your own LLM from Scratch  

• Morning: Refresher Pytorch, Multi-Layer Perceptron, Recurrent Neural Network, applied to Natural Language Processing
• Afternoon: Tokenizer, Text embedding, Attention

Speakers: Prof. Frederic Precioso & Team EFELIA Côte d'Azur

Tuesday, June 24

Tutorial and lab: Build your own LLM from Scratch

• Morning: Transformer-Encoder, Attention, Multi-Head Attention
• Afternoon: LLM (Encoder) for Text Classification, for Image + Text classification

Speakers: Prof. Frederic Precioso & Team EFELIA Côte d'Azur

Wednesday, June 25

• Morning

Speaker: Elie Bakouch, Co-leader of SmolLM Team at Hugging Face
Topic: SmolLM, how Small Language Models can compete with LLMs

Title: Pre-training smol and large LLMs
Abstract: In this talk, you'll get a clear overview of current best practices for pre-training language models, both small and large. I'll discuss the latest optimizers (AdamW, Muon, Shampoo), how to choose learning rate schedules and batch sizes, stability improvements (normalization, weight decay), recent architecture innovations (linear attention, MoE), and effective methods for extending context lengths (RoPE, chunked attention). Examples from models like DeepSeek and the upcoming Llama 4 will make these techniques concrete and actionable.

• Afternoon

Speaker: Nouamane Tazi, Co-leader of Ultra-scale Playbook Team at Hugging Face
Topic: The Ultra-Scale Playbook - Training efficiently LLMs on GPU Clusters

Title: The Ultra-Scale Talk: Scaling Training to Thousands of GPUs
Abstract: Training large language models (LLMs) efficiently requires scaling across multiple GPUs. This lecture will explore methodologies for expanding training from a single GPU to thousands, covering 5D parallelism techniques. Attendees will gain insights into optimizing throughput, GPU utilization, and training efficiency, with practical examples and benchmarks.
Learn more

Thursday, June 26

• Speaker: Prof. Mihaela van der Schaar, University of Cambridge (UK)
• Topic: Code Agents, Science Agents, System Agents, Network Agents

Title: The Rise of AI Agents: Code, Science, Systems, Networks
Abstract: AI agents are redefining autonomy in artificial intelligence—evolving from passive predictors into dynamic systems that can plan, act, adapt, and collaborate. This talk provides a structured introduction to the emerging field of AI agents, aimed at researchers and practitioners in machine learning, systems, and science.
We will define what constitutes an AI agent, outline a taxonomy of current approaches, and illustrate their capabilities through diverse examples: agents that write and debug code, agents that discover scientific laws, agents that solve complex planning tasks, and agents that power digital twins in healthcare and infrastructure. Throughout, we will highlight key architectural components, modes of interaction, and open research challenges shaping the future of agentic AI.

Friday, June 27

• Speaker: Prof. Yulan He, King’s College London (UK)
• Topic: Self-evolution of large language models (LLMs)

Title: Self-Evolution of Large Language Models
Abstract: This tutorial explores the emerging concept of self-evolution in large language models (LLMs), where models self-evaluate, refine, and improve their reasoning capabilities over time with minimal human intervention. We will start with the technical foundations behind self-improvement, including approaches such as bootstrapped reasoning, synthesising reasoning and acting, verbalised reinforcement learning, and LLM learning via self-play or self-planning. We will then present case studies illustrating LLM self-evolution in various applications, such as question answering, student answer scoring, causal event extraction, and the use of LLM agents in murder mystery games. In addition, we will discuss the challenges of model alignment, control, and safety in the context of LLM self-evolution. Finally, we will conclude the tutorial with an outlook for future research.

Monday, June 30

Tutorial and lab: Build your own LLM from Scratch

• Morning: Transformer-Decoder, Masked Multi-Head Attention, LLM(Decoder) for Text generation
• Afternoon: LLM(Encoder-Decoder), Cross-Attention, for Translation, Summarization

Speakers: Prof. Frederic Precioso & Team EFELIA Côte d'Azur

Tuesday, July 1st

Tutorial and lab: Build your own LLM from Scratch

• Morning: Reinforcement Learning, Reinforcement Learning from Human Feedback (RLHF)
• Afternoon: Use case DeepSeek R1  

Speakers: Prof. Frederic Precioso & Team EFELIA Côte d'Azur

Wednesday, July 2nd

• Speaker: Prof. Elisa Ricci, University of Trento (Italy)
• Topic: Foundation models for multimedia / Computer vision and vision-language models

Title: Beyond Zero-Shot Generalization: Strengthening Vision-Language Models with Adaptation and Personalization
Abstract: Groundbreaking achievements in Vision-Language pretraining have increased the interest in crafting Vision-Language Models (VLMs) that can understand visual content alongside natural language, enabling a new definition of zero-shot classification. Despite huge pretraining databases, VLMs still face limitations, suffering from performance degradation in case of large train-test dissimilarity and requiring the design of highly generalizing textual templates. Test-Time Adaptation (TTA) and Few-Shot Adaptation (FSA) can effectively improve the robustness of VLMs by adapting a given model to online inputs or to a specific downstream task given a few annotated examples. In this talk, I will present an overview of a few recent works from my research group on TTA, FSA and other strategies aimed at improving the robustness of VLMs across various downstream tasks.

Thursday, July 3rd

• Speaker: Prof. Dan Jurafsky, Stanford University (USA)
• Topic: LLMs assessment, Ethics

Title: Social, Interactional, and Multilingual Perspectives on Large Language Models: Problems and Potentials
Abstract: This lecture describes work in our Stanford lab on understanding the properties of large language models, as well as their applications to multilingual domains and social questions.  I discuss failures we uncover in large language model related to social and interactional meaning.  We show that LMs are linguistically overconfident, overly anthropomorphic, lack conversational grounding, confuse language use with language mention and belief with knowledge, and generate language and decisions consistent with racial bias.  I'll discuss possible causes and the implications for users.  I'll also cover other interpretive studies attempting to elucidate the internal structure of language models, and applications to improve processing of low-resource languages.  Finally, I'll cover the use of  language models as a tool in measuring latent social meaning.

Friday, July 4th

• Speaker: Prof. Stuart Russell, University of California at Berkeley (USA)
• Topic: From Reinforcement Learning from Human Feedback for LLMs to Assistance Games 
   

Title: What if we succeed?
Abstract: Many experts claim that recent advances in AI put artificial general intelligence (AGI) within reach.  Is this true? If so, is that a good thing? Alan Turing predicted that AGI would result in the machines taking control. I will argue that Turing was right to express concern but wrong to think that doom is inevitable. Instead, we need to develop a new kind of AI that is provably beneficial to humans. Unfortunately, we are heading in the opposite direction and we need to take steps to correct this. Even so, questions remain about whether human flourishing is compatible with AGI.