5/5/23: Plan and GPT 4 reading

 

   Machine Learning Study Group

Welcome! We start at 4:00 (or as soon as the weekly seminar is over and people have logged in). Zoom automatically ends the meeting after 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Also feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 63rd meeting, May 5, 2023.
• Status updates/news/inputs/comments
• Student and project updates
• ?
  
• Any other updates/news/inputs or comments?
  

• We read https://openai.com/research/gpt-4 about GPT-4 up to the section "Risks & mitigations" We could proceed from there.

• We evaluated https://huggingface.co/learn/nlp-course/chapter0/1?fw=tf. Evaluation was 4 1/5 to continue reading it, which would involve setting up environments since reading it means working through it like a lab.
• Suggestions: 
• One week of preparatory discussion and background reading, followed by a week of lab exercise together working through it. Then repeat. 
• Alternatively, a 3 week cycle of prep/discuss one week, lab the following week, something unrelated the 3rd week, and repeat. What do people think of that?  No one, as no one is really needed to "lead."
  
• We can probably go with option 2 at least for now.

 The meeting ended here. 

•  
  MC suggested: Sparks of Artificial General Intelligence: Early experiments with GPT-4 (https://arxiv.org/abs/2303.12712). We read the abstract to decide whether to read the article. The group's evaluation was 3 6/7 out of 5 to read it.
  
• Here are some other things we could choose to read from in the future from as time allows.
  
• At some point we can discuss and vote on other sources. See the list on the page of sources. Also Youtube is full of videos about neural nets, transformers, etc.
  

• Readings/videos/demos we have finished. 
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
  
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
  
• We also finished other things from before this list of finished sources was created to keep a record of them.

4/28/23: Updates; evaluations of potential readings

   Machine Learning Study Group

Welcome! We start at 4:00 (or as soon as the weekly seminar is over and people have logged in). Zoom automatically ends the meeting after 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Also feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 62nd meeting, April 28, 2023.
• Status updates/news/inputs/comments
• Student and project updates
• PT is working on different window sizes for the LSTM algorithm. Will give us an update on preliminary results.
  
• Superimposed text project (VK). Paper submitted to FTC 2023 on VQA on images with superimposed text. Any updates? We should hear back about the paper some time in mid-May.
  
• DD has finished the tutorial.
• Any other updates/news/inputs or comments?
  

• We read https://openai.com/research/gpt-4 about GPT-4 up to the section "Limitations."
• Next time we can evaluate https://huggingface.co/learn/nlp-course/chapter0/1?fw=tf. Evaluation was 4 1/5 to continue reading it, which would involve setting up environments since reading it means working through it like a lab.
  
• https://huggingface.co/learn/nlp-course/chapter7/3?fw=tf. Evaluation was 4.0 to read more of this article.
• MC suggested: Sparks of Artificial General Intelligence: Early experiments with GPT-4 (https://arxiv.org/abs/2303.12712). We read the abstract to decide whether to read the article. The group's evaluation was 3 6/7 out of 5 to read it.
  
• Here are some other things we could choose to read from in the future from as time allows.
  
• At some point we can discuss and vote on other sources. See the list on the page of sources. Also Youtube is full of videos about neural nets, transformers, etc.
  

• Readings/videos/demos we have finished. 
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
  
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
  
• We also finished other things from before this list of finished sources was created to keep a record of them.

4/21/23: Updates; evaluate another reading

 

  Machine Learning Study Group

Welcome! We start at 4:00 (or as soon as the weekly seminar is over and people have logged in). Zoom automatically ends the meeting after 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Also feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 61st meeting, April 21, 2023.
• Status updates/news/inputs/comments
• Student and project updates
• PT is working on different window sizes for the LSTM algorithm. Will give us an update on preliminary results next time.
  
• K is working on installing Rapids.AI.
  
• Superimposed text project (VK). Paper submitted to FTC 2023 on VQA on images with superimposed text.
• DD has an algorithm that detects ambiguity in words. Would a NN work better? Plan: finish the tutorial on fine tuning a large language model. Might be able to tell us about the key points in it next week.
• Any other updates/news/inputs or comments?
  

• https://huggingface.co/learn/nlp-course/chapter7/3?fw=tf. Evaluation was 4.0 to read more of this article.
• Next time we can evaluate https://huggingface.co/learn/nlp-course/chapter0/1?fw=tf.
  
• In the article "ChatGPT Strategy" by C. Goodall, we got through item 2 in the section "10 WAYS TEACHERS CAN USE CHAT GPT TO REDUCE WORKLOAD." We can start from there. Evaluation was 3.0 to keep reading it.
  
• We read https://openai.com/research/gpt-4 about GPT-4 up to the section "Steerability" but got cut off before we could evaluate it. So we will evaluate it next time.
• MC suggested: Sparks of Artificial General Intelligence: Early experiments with GPT-4 (https://arxiv.org/abs/2303.12712). We read the abstract to decide whether to read the article. The group's evaluation was 3 6/7 out of 5 to read it.
  
• Here are some other things we could choose to read from in the future from as time allows.
  
• ChatGPT related readings, video viewings, etc.
  

• ChatGPT: What Is It & How Can You Use It? (searchenginejournal.com). We already read and discussed up to "The moderators at the coding Q&A website Stack Overflow ..." on 1/6/23.
• ChatGPT Is a Game-Changer. Here’s How to Use it in Your Writing. We already discussed more or less up to "2. Headline generation" on 12/30/22.
  
• ChatGPT Caused 'Code Red' at Google, Report Says - CNET. We already discussed up to "ChatGPT is an AI chatbot" on 12/30/22. 
• 20 Entertaining Uses of ChatGPT You Never Knew Were Possible | by Mark Schaefer | Dec, 2022 | Medium  
• Large Language Models: A New Moore's Law? (huggingface.co)
• The Brilliance and Weirdness of ChatGPT - The New York Times
• AI experts are increasingly afraid of what they’re creating
• Short video: ChatGPT in medicine

• At some point we can discuss and vote on other sources. See the list on the page of sources. Also Youtube is full of videos about neural nets, transformers, etc.
  

• Readings/videos/demos we have finished. 
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
  
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
  
• We also finished other things from before this list of finished sources was created to keep a record of them.

4/14/23: Student updates; evaluate potential readings

 

 

 Machine Learning Study Group

Welcome! We start at 4:00 (or as soon as the weekly seminar is over and people have logged in). Zoom automatically ends the meeting after 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Also feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 60th meeting, April 14, 2023.
• Status updates/news/inputs/comments
• Student and project updates
• EG introduced us to Rapids.AI. It is much faster than its competitors.
  
• PT is working on predicting spacecraft lifetimes using LSTMs to digest the data and make predictions. He is incorporating nominal variables (category bins) into the model. Example: destination, which could be Mars, or Saturn, or Jupiter, etc., which does not have a continuous value.
• K is working on installing Rapids.AI.
  
•  Superimposed text project (VK). Paper underway on VQA on images with superimposed text.
  
• Any other updates/news/inputs or comments?
  

• In the article "ChatGPT Strategy" by C. Goodall, we got through item 2 in the section "10 WAYS TEACHERS CAN USE CHAT GPT TO REDUCE WORKLOAD." We can start from there. Evaluation was 3.0 to keep reading it.
  
• We read https://openai.com/research/gpt-4 about GPT-4 up to the section "Steerability" but got cut off before we could evaluate it. So we will evaluate it next time.
• MC suggested: Sparks of Artificial General Intelligence: Early experiments with GPT-4 (https://arxiv.org/abs/2303.12712). We read the abstract to decide whether to read the article. The group's evaluation was 3 6/7 out of 5 to read it.
  
• Here are some other things we could choose to read from in the future from as time allows.
  
• ChatGPT related readings, video viewings, etc.
  

• ChatGPT: What Is It & How Can You Use It? (searchenginejournal.com). We already read and discussed up to "The moderators at the coding Q&A website Stack Overflow ..." on 1/6/23.
• ChatGPT Is a Game-Changer. Here’s How to Use it in Your Writing. We already discussed more or less up to "2. Headline generation" on 12/30/22.
  
• ChatGPT Caused 'Code Red' at Google, Report Says - CNET. We already discussed up to "ChatGPT is an AI chatbot" on 12/30/22. 
• 20 Entertaining Uses of ChatGPT You Never Knew Were Possible | by Mark Schaefer | Dec, 2022 | Medium  
• Large Language Models: A New Moore's Law? (huggingface.co)
• The Brilliance and Weirdness of ChatGPT - The New York Times
• AI experts are increasingly afraid of what they’re creating
• Short video: ChatGPT in medicine

• At some point we can discuss and vote on other sources. See the list on the page of sources. Also Youtube is full of videos about neural nets, transformers, etc.
  

• Readings/videos/demos we have finished. 
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
  
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
  
• We also finished other things from before this list of finished sources was created to keep a record of them.

4/7/23: Student updates

 

 Machine Learning Study Group

Welcome! We start at 4:00 (or as soon as the weekly seminar is over and people have logged in). Zoom automatically ends the meeting after 40 minutes. New members are always welcome and anyone can join. You may attend any or all sessions. Also feel free to ask to be removed from the invite list at any time, as we have no wish to send unneeded emails which we all certainly get too many of.

 

Agenda & Minutes

• Welcome to the 59th meeting, April 7, 2023.
• Status updates/news/inputs/comments
• Project/student updates? Start next week but a bit this time. We got updates from some students and the general consensus was that this is a good thing to do in these meetings.
  
• Any other updates/news/inputs or comments? There was brief discussion about this.
  

• VK continued stepping us through the article, "ChatGPT Strategy" by C. Goodall. We got through item 1 in the section "10 WAYS TEACHERS CAN USE CHAT GPT TO REDUCE WORKLOAD." We can start there next time.
  
• We did not get to https://openai.com/research/gpt-4 about GPT-4, but we are up to "We preview GPT-4’s performance by evaluating it on a narrow suite of standard academic vision benchmarks." where we can start again next time we do this article.
•  MC suggested: Sparks of Artificial General Intelligence: Early experiments with GPT-4 (https://arxiv.org/abs/2303.12712). We could check the abstract and decide what else if anything in it to read.
  
• Here are some other things we could choose to read from in the future from as time allows.
  
• ChatGPT related readings, video viewings, etc.
  

• ChatGPT: What Is It & How Can You Use It? (searchenginejournal.com). We already read and discussed up to "The moderators at the coding Q&A website Stack Overflow ..." on 1/6/23.
• ChatGPT Is a Game-Changer. Here’s How to Use it in Your Writing. We already discussed more or less up to "2. Headline generation" on 12/30/22.
  
• ChatGPT Caused 'Code Red' at Google, Report Says - CNET. We already discussed up to "ChatGPT is an AI chatbot" on 12/30/22. 
• 20 Entertaining Uses of ChatGPT You Never Knew Were Possible | by Mark Schaefer | Dec, 2022 | Medium  
• Large Language Models: A New Moore's Law? (huggingface.co)
• The Brilliance and Weirdness of ChatGPT - The New York Times
• AI experts are increasingly afraid of what they’re creating
• Short video: ChatGPT in medicine

• At some point we can discuss and vote on other sources. See the list on the page of sources. Also Youtube is full of videos about neural nets, transformers, etc.
  

• Readings/videos/demos we have finished. 
• 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
  
• ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
• 12/16/22: We demoed ChatGPT.
• 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
• Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
• The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
• https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
  
• We also finished other things from before this list of finished sources was created to keep a record of them.