10/24/24: Chapter 5 video

   Machine Learning Study Group

Welcome! We meet from 4:00-4:45 p.m. Central Time. Anyone can join. Feel free to attend any or all sessions, or ask to be removed from the invite list as we have no wish to send unneeded emails of which we all certainly get too many.   Contacts: jdberleant@ualr.edu and mgmilanova@ualr.edu Agenda & Minutes  (135th meeting, Oct. 24, 2024) Announcements, updates, questions, etc. MM brought up a free faculty workshop series leading to certification from NVIDIA. See https://events.nvidia.com/faculty-development-virtual-workshops-higher-ed. DB has forwarded a link to this group's members. RM will give a guest presentation for part of the meeting next week (Nov. 1). Topic: ML for antenna design. UALR official AI activities. From an email: "The Provost has a team participating in AAC&U's AI Institute on AI Pedagogy in the Curriculum this year [.... Our ML Study Group member IU] is the representative from DCSTEM. The Provost's office would also like each college to form an ad hoc Committee on AI Pedagogy in the Curriculum that would meet ~monthly with our AI Institute representative." Any updates? Here are the latest on readings and viewings Next we will work through chapter 5: https://www.youtube.com/watch?v=wjZofJX0v4M. We got up 15:50 and can start there next time we work on this video! We can work through chapter 6: https://www.youtube.com/watch?v=eMlx5fFNoYc We can work through chapter 7: https://www.youtube.com/watch?v=9-Jl0dxWQs8 Computer scientists win Nobel prize in physics! Https://www.nobelprize.org/uploads/2024/10/popular-physicsprize2024-2.pdf got a evaluation of 5.0 for a detailed reading. We can evaluate https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10718663 for reading & discussion. Chapter 6 recommends material by Andrej Karpathy, https://www.youtube.com/@AndrejKarpathy/videos for learning more. Chapter 6 recommends material by Chris Olah, https://www.youtube.com/results?search_query=chris+olah Chapter 6 recommended https://www.youtube.com/c/VCubingX for relevant material, in particular https://www.youtube.com/watch?v=1il-s4mgNdI Chapter 6 recommended Art of the Problem, in particular https://www.youtube.com/watch?v=OFS90-FX6pg LLMs and the singularity: https://philpapers.org/go.pl?id=ISHLLM&u=https%3A%2F%2Fphilpapers.org%2Farchive%2FISHLLM.pdf (summarized at: https://poe.com/s/WuYyhuciNwlFuSR0SVEt). 6/7/24: vote was 4 3/7. We read the abstract. We could start it any time. We could even spend some time on this and some time on something else in the same meeting.  Here is the Zoom-generated transcript:   Quick recap The meeting covered various topics related to AI technology and its applications in education and communication. Participants discussed new features on their platform, upcoming workshops and presentations, and technical aspects of screen sharing during meetings. The team also explored the concept of generative pre-trained transformers (GPT), their performance, and potential challenges in AI-generated content detection, as well as the emerging role of prompt engineers in the field. Next steps • MM to provide details on the Free Faculty Workshop Series for interested participants. • RM to give a guest presentation on machine learning for antenna design on November 1st. • DB to contact IU for an update on AI pedagogy initiatives at the university level. Summary Exploring AI Companion Feature D and M. discussed a new feature they found on their platform, which they referred to as AI Companion. M. explained that it's a button at the bottom of the screen with a starry icon, and when clicked, it sends a request to the host to start a meeting summary with AI Companion. D confirmed that he saw the same button and tried it, but it didn't have the same options as M.'s. They decided to start the feature and see how it works. G mentioned that this feature is similar to Co-Pilot in Microsoft Teams, which he uses in his organization. D agreed to try it out and possibly post the results on a blog. Free Faculty Workshop Series Update In the meeting, D discussed the upcoming Free Faculty Workshop Series, which R will be a part of, and encouraged others to sign up. He also mentioned that R will give a guest presentation on machine learning for antenna design on November 1st. D also introduced a new team member, I, who is the representative from their college to the American Association of Colleges and Universities AI Institute on AI Pedagogy. He expressed his intention to ask Ifor an update on the university's AI pedagogy initiatives. Lastly, he reminded the team that they had covered chapters one through four and were now on chapter five of the video series. Screen Sharing and Audio Issues D led a discussion about sharing a screen and audio during a meeting. Initially, there were issues with the audio being too faint, which DD suggested might be due to the audio not being broadcasted. D tried various solutions, including unsharing, advanced sharing options, and optimizing for video clip, which also automatically shared sound. The team was still experiencing some audio issues at the end of the meeting. Explaining Generative Pre-Trained Transformers DB explained the concept of generative pre-trained transformers (GPT), a type of bot that generates new text. He detailed how the model learns from a massive amount of data and can be fine-tuned for specific tasks. He also discussed the core invention underlying the current AI boom, the transformer neural network, and its various applications, including text-to-speech, image generation, and language translation. D also outlined how a prediction model can be used to generate longer pieces of text by appending new samples to the initial text. He encouraged questions and comments, acknowledging that the concept might seem surprising. GPT-2 and GPT-3 Performance Discussion DB discussed the performance of GPT-2 and GPT-3 models, noting that GPT-3, despite being the same basic model as GPT-2 but with more data, generated a more sensible story. The team speculated about the potential for diminishing returns with more data and the possibility of reaching a point where the model is almost as good as a human. DD suggested that the architecture might be the limiting factor, and that overfitting could become a problem if the model is not properly managed. The team also discussed the quality and quantity of data being fed to the models, with M. questioning whether they have reached a point of diminishing returns with the current architecture. AI Training Challenges and Prompts The team discussed the challenges of training AI with content generated by AI itself, as it could lead to a "poisoned well" of information. They also touched on the emerging role of prompt engineers, who are skilled at crafting effective prompts for AI systems. DD and DB shared their observations on the increasing demand for such skills and the potential for high-paying positions. The conversation ended with a reflection on the importance of effective communication with AI systems. AI Language Models and Detection DB and DD discussed the potential for AI language models to provide users with a distribution of possible next words, allowing them to choose their preferred word. They also considered the possibility of detecting AI-generated content by analyzing the likelihood of words used. G mentioned a student's dissertation on detecting fake news or videos, which could be applied to this problem. The team agreed that AI-generated content could be identified by its predictability, but acknowledged the difficulty in proving this definitively.