9/15/23: HuggingFace lab

 

Machine Learning Study Group

Welcome! We start at 4:00, and go for 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 80th meeting, Sept. 15, 2023.
• Status updates/news/inputs/comments
• Student and project updates? 
• BL will defend his PhD proposal in two weeks, Friday 9/29, at 4:00. The videoconference link is different. It will be at: 
• Google Meet, https://meet.google.com/dib-fyhb-odw.
• Therefore, we will meet next  week as usual, but the following week we will not meet at the usual Zoom link like we are today, but instead everyone is invited to the Google Meet link for BL's proposal defense.
  
• Schedule of activities:
• We are now in "week" 2 of the 3-week quasi-cycle (which consists of reading in week 1, and HuggingFace labs in weeks 2 and 3, interspersed with weeks with other activities, like next week's proposal). This means we will do a Huggingface lab today. Then we will do more on it in week 3, next time.
• Next week we will take a break from the quasi-cycle with a presentation by EG on multithreading.
• The following week will be the ML proposal (at a different videoconference link).
• The week after that - back to week 3 of our quasi-cycle with more on the HuggingFace lab we've been working through.
  

• Status of the threads of activity:
• Here is where we are with the Huggingface lab exercises:
• To continue with the Huggingface lab program, this time we got up to the section "Mask Filling" in chapter 1,  part 3, https://huggingface.co/learn/nlp-course/chapter1/3?fw=tf. We can start with that next time we do it, reading about it briefly and then trying it in the running code window.
  

• Readings: We will discuss the top ranked item from the page of evaluated items next time we do readings.

 

9/8/23: Finish evaluating a list of candidate readings

 

Machine Learning Study Group

Welcome! We start at 4:00, and go for 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 79th meeting, Sept. 8, 2023.
• We are continuing to finish up, from last time, "week" 1 of the 3-week quasi-cycle (which consists of reading in week 1, and HuggingFace labs in weeks 2 and 3, interspersed with weeks with other activities). This means we will read more abstracts and evaluate whether to read the fuller articles.
  
•  Status updates/news/inputs/comments
• Student and project updates? 
• 
  
• Schedule of activities:
• Today: read & evaluate various possibilities for the next reading. We got up t0 #14 last time and will continue with #14 this time.
• Any other updates/news/inputs or comments?

• Status of the threads of activity:
• Here is where we are with the Huggingface lab exercises:
• To continue with the Huggingface lab program, last time we did this we worked on chapter 1,  part 3, https://huggingface.co/learn/nlp-course/chapter1/3?fw=tf. After the first paragraph we went to the collab link where we worked on that, starting to experiment with the code window

from transformers import pipeline

 

generator = pipeline("text-generation", model="distilgpt2")

generator(

    "In this course, we will teach you how to",

    max_length=30,

    num_return_sequences=2,

)

Next time we can experiment a bit more and then go on from there.

• Readings: let us discuss and evaluate the abstracts of the potential readings in the page of reading possibilities.
• We have evaluated items 1-11 and will finish the list next week (instead of going on to week 2 of the 3-week quasi-cycle).
  

• Readings/videos/demos we have finished. 
• 6/30/23: We finished https://openai.com/research/gpt-4 about GPT-4!
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
• We also finished other things from before this list of finished sources was created to keep a record of them.

 

9/1/23: Discussion and reading

 

Machine Learning Study Group

Welcome! We start at 4:00, and go for 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 78th meeting, Sept. 1, 2023.
• We are finishing up, from last time, "week" 1 of the 3-week quasi-cycle (which consists of reading in week 1, and HuggingFace labs in weeks 2 and 3, interspersed with weeks with other activities). This means we will read more abstracts and evaluate whether to read the fuller articles.
  
•  Status updates/news/inputs/comments
• Student and project updates? 
• Congratulations to new Dr. VW!
• Discuss convergence issue in a NN.
  
• Schedule of activities:
• Today: read & evaluate various possibilities for the next reading. We got up t0 #14 and will continue next time.

  

The meeting ended here

• Any other updates/news/inputs or comments?

• Status of the threads of activity:
• Here is where we are with the Huggingface lab exercises:
• To continue with the Huggingface lab program, last time we did this we worked on chapter 1,  part 3, https://huggingface.co/learn/nlp-course/chapter1/3?fw=tf. After the first paragraph we went to the collab link where we worked on that, starting to experiment with the code window

from transformers import pipeline

 

generator = pipeline("text-generation", model="distilgpt2")

generator(

    "In this course, we will teach you how to",

    max_length=30,

    num_return_sequences=2,

)

Next time we can experiment a bit more and then go on from there.

• Readings: let us discuss and evaluate the abstracts of the potential readings in the page of reading possibilities.
• We have evaluated items 1-11 and will finish the list next week (instead of going on to week 2 of the 3-week quasi-cycle).
  

• Readings/videos/demos we have finished. 
• 6/30/23: We finished https://openai.com/research/gpt-4 about GPT-4!
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
• We also finished other things from before this list of finished sources was created to keep a record of them.

 

8/25/23: Continue & finish "week" 1 of 3-week quasi-cycle: Readings (+ evaluate what to read next)

 

Machine Learning Study Group

Welcome! We start at 4:00, and go for 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 77th meeting, Aug. 25, 2023.
• We are finishing up, from last time, "week" 1 of the 3-week quasi-cycle (which consists of reading in week 1, and HuggingFace labs in weeks 2 and 3, interspersed with weeks with other activities).
•  Status updates/news/inputs/comments
• Student and project updates?
• Schedule of activities:
• Today: read & evaluate various possibilities for the next reading
  
• Any other updates/news/inputs or comments?

• Status of the threads of activity:
• Here is where we are with the Huggingface lab exercises:
• To continue with the Huggingface lab program, last time we worked on chapter 1,  part 3, https://huggingface.co/learn/nlp-course/chapter1/3?fw=tf. After the first paragraph we went to the collab link where we worked on that, starting to experiment with the code window

from transformers import pipeline

 

generator = pipeline("text-generation", model="distilgpt2")

generator(

    "In this course, we will teach you how to",

    max_length=30,

    num_return_sequences=2,

)

Next time we can experiment a bit more and then go on from there.

• Readings: let us discuss and evaluate the abstracts of the potential readings in the page of reading possibilities.
• We have evaluated items 1-11 and will finish the list next week (instead of going on to week 2 of the 3-week quasi-cycle).
  

• Readings/videos/demos we have finished. 
• 6/30/23: We finished https://openai.com/research/gpt-4 about GPT-4!
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
• We also finished other things from before this list of finished sources was created to keep a record of them.

 

8/18/23: Background reading (week 1 of 3-week quasi-cycle): discuss what to read next

 

Machine Learning Study Group

Welcome! We start at 4:00, and go for 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 76th meeting, Aug. 18, 2023.
• We are starting a new 3-week quasi-cycle of reading in week 1, and HuggingFace labs in weeks 2 and 3, interspersed with weeks with other activities.
•  Status updates/news/inputs/comments
• Student and project updates?
• Schedule of activities:
• Today: Background reading (week 1 of the 3-week quasi-cycle)
• Any other updates/news/inputs or comments?

• Status of the threads of activity:
• Here is where we are with the Huggingface lab exercises:
• To continue with the Huggingface lab program, last time we worked on chapter 1,  part 3, https://huggingface.co/learn/nlp-course/chapter1/3?fw=tf. After the first paragraph we went to the collab link where we worked on that, starting to experiment with the code window

from transformers import pipeline

 

generator = pipeline("text-generation", model="distilgpt2")

generator(

    "In this course, we will teach you how to",

    max_length=30,

    num_return_sequences=2,

)

Next time we can experiment a bit more and then go on from there.

• Readings: we can discuss and evaluate the abstracts of the potential readings in the page of reading possibilities.
• We evaluated items 1-7 and decided to finish the list next week (instead of going on to week 2 of the 3-week quasi-cycle).
  

• Readings/videos/demos we have finished. 
• 6/30/23: We finished https://openai.com/research/gpt-4 about GPT-4!
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
• We also finished other things from before this list of finished sources was created to keep a record of them.

 

8/11/23: Draft article discussion (by VW)

 

   Machine Learning Study Group

Welcome! We start at 4:00, and go for 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 75th meeting, Aug. 11, 2023.
• Discussion of draft article by VW, who will lead the discussion.
  

• Here are some other things we could choose to read from in the future.
  

• Foundational Models Defining a New Era in Vision: A Survey and Outlook, https://arxiv.org/abs/2307.13721

• VW has developed an annotated bibliography of LLM articles. We could read these like we would abstracts, then decide whether to read the actual articles based on that.
• At some point we can discuss and vote on other sources. See the list on the page of sources. Also Youtube is full of videos about neural nets, transformers, etc.
• MC suggested: Sparks of Artificial General Intelligence: Early experiments with GPT-4 (https://arxiv.org/abs/2303.12712). We read the abstract to decide whether to read the article. The group's evaluation was 3 6/7 out of 5 to read it.

• Readings/videos/demos we have finished. 
• 6/30/23: We finished https://openai.com/research/gpt-4 about GPT-4!
• 2/17/23: "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
  
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
  
• We also finished other things from before this list of finished sources was created to keep a record of them.

8/4/23: EG paper draft discussion

   Machine Learning Study Group

Welcome! We start at 4:00, and go for 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 74th meeting, Aug. 4, 2023.
• Discussion of draft article by EG, who will lead the discussion.
  

• Here are some other things we could choose to read from in the future.
  

• Foundational Models Defining a New Era in Vision: A Survey and Outlook, https://arxiv.org/abs/2307.13721

• VW has developed an annotated bibliography of LLM articles. We could read these like we would abstracts, then decide whether to read the actual articles based on that.
• At some point we can discuss and vote on other sources. See the list on the page of sources. Also Youtube is full of videos about neural nets, transformers, etc.
• MC suggested: Sparks of Artificial General Intelligence: Early experiments with GPT-4 (https://arxiv.org/abs/2303.12712). We read the abstract to decide whether to read the article. The group's evaluation was 3 6/7 out of 5 to read it.

• Readings/videos/demos we have finished. 
• 6/30/23: We finished https://openai.com/research/gpt-4 about GPT-4!
• 2/17/23: "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
  
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
  
• We also finished other things from before this list of finished sources was created to keep a record of them.