Machine Learning Study Group
- Welcome to the 63rd meeting, May 5, 2023.
- Status updates/news/inputs/comments
- Student and project updates
- ?
- Any other updates/news/inputs or comments?
- We read https://openai.com/research/gpt-4 about GPT-4 up to the section "Risks & mitigations" We could proceed from there.
- We evaluated https://huggingface.co/learn/nlp-course/chapter0/1?fw=tf. Evaluation was 4 1/5 to continue reading it, which would involve setting up environments since reading it means working through it like a lab.
- Suggestions:
- One week of preparatory discussion and background reading, followed by a week of lab exercise together working through it. Then repeat.
- Alternatively, a 3 week cycle of prep/discuss one week, lab the following week, something unrelated the 3rd week, and repeat. What do people think of that? No one, as no one is really needed to "lead."
- We can probably go with option 2 at least for now.
- MC suggested: Sparks of Artificial General Intelligence: Early experiments with GPT-4 (https://arxiv.org/abs/2303.12712). We read the abstract to decide whether to read the article. The group's evaluation was 3 6/7 out of 5 to read it.
- Here are some other things we could choose to read from in the future from as time allows.
- At some point we can discuss and vote on other sources. See
the list on the page of sources. Also Youtube is full of videos about neural nets, transformers, etc.
- Readings/videos/demos we have finished.
- 2/17/23; "Deep Learning for AI" by Bengio, LeCun and Hinton, the Turing
Award Lecture paper for 2018, published in CACM, 2021, https://dl.acm.org/doi/pdf/10.1145/3448250.
- ChatGPT Is a Tipping Point for AI (Harvard Business Review). Finished 1/27/23.
- 12/16/22: We demoed ChatGPT.
- 11/4/22: We viewed and discussed a transformers video: https://www.youtube.com/watch?v=TQQlZhbC5ps first. This seemed to be one of the better videos of the many out there.
- Transformers From Scratch," https://e2eml.school/transformers.html. Finished on 10/28/22.
- The Narrated Transformer Language Model, Jay Alammar, https://www.youtube.com/watch?v=-QH8fRhqFHM, finished 7/22/22.
- https://www.youtube.com/watch?v=F1X4fHzF4mQ. Finished 8/26/22.
- We also finished other things from before this list of finished sources was created to keep a record of them.