Machine Learning Study Group
|
Contacts: jdberleant@ualr.edu and mgmilanova@ualr.edu
Agenda & Minutes (142nd meeting, Dec. 13, 2024)
Table of Contents
* Agenda and minutes
* Transcript (when available)
Agenda and minutes
- Announcements, updates, questions, presentations, etc.
- DB has found some masters students interested in the project below starting next semester for their masters degree project requirement. An important qualification for the student is to be able to attend these meetings weekly to update us on progress and get suggestions from all of us!
- Project description: Suppose a generative AI like ChatGPT or Claude.ai was used to write a book about a simply stated task, like "how to scramble an egg," "how to plant and care for a persimmon tree," "how to check and change the oil in your car," or any other question like that. Just ask the AI to provide a step by step guide, then ask it to expand on each step with substeps, then ask it to expand on each substep, continuing until you reached 100,000 words or whatever impressive target one might have.
- LG: "How to bake a cake." Has some thoughts on the hallucination issue that AB mentioned.
- ET: Gardening (veggies, herbs in particular). Specifically, growing vegetables from seeds.
- DD: Better to write about something you know something about.
- YP:
Prompt engineering would require looping, giving feedback, maybe agents
would be useful. Youtube videos are available by subject matter
experts. So you can familiarize with a topic that way.
- MM: We could show examples of prompting using agents.
- JK is focusing on prompt eng. with agents (but is not here today).
- If anyone else has a project they would like to help supervise, let me know!
- JK submitted a video to AAAI 2024: video (https://drive.google.com/file/d/1KJNQQU7IfSywljADxkGHMZuDkchbIc38/view?usp=sharing); call for videos (https://aaai.org/about-aaai/aaai-awards/aaai-educational-ai-videos). See also complex prompts, etc. (https://drive.google.com/drive/u/0/folders/1uuG4P7puw8w2Cm_S5opis2t0_NF6gBCZ).
- NM: Updates/problems/questions on the process of turning the thesis into a publishable document using AI to help, while not having it look like it was AI generated and, more importantly, not having it overly AI generated to the extent that truth is compromised, while still using AI as an assistant as much as possible.
- Here
is a tool the library is providing. Some people here thought it would
be a good idea to try it live during a meeting, so we can do that.
Library trial of AI-driven product Primo Research AssistantThe library is testing use of Primo Research Assistant, a generative AI-powered feature of Primo, the library's search tool. Primo Research Assistant takes natural-language queries and chooses academic resources from the library search to produce a brief answer summary and list of relevant resources. This video provides further detail about how the Assistant works.
You can access Primo Research Assistant directly here, or, if you click "Search" below the search box on the library home page, you will see blue buttons for Research Assistant on the top navigation bar and far right of the Primo page that opens. You will be prompted to log in using your UALR credentials in order to use the Research Assistant.
- The campus has assigned a group to participate in the AAC&U AI Institute's activity "AI Pedagogy in the Curriculum." IU is on it and may be able to provide updates when available, every now and then but not every week.
- Anything else anyone would like to bring up?
- Here are the latest on readings and viewings
- Next
we will continue to work through chapter 5:
https://www.youtube.com/watch?v=wjZofJX0v4M. We got up 15:50 awhile ago
but it was indeed awhile ago so we started from the beginning and went
to 15:50 again.
Next time we do this video, we will go on from there. (When sharing the
screen,
we need to click the option to optimize for sharing a video.)
- We can work through chapter 6: https://www.youtube.com/watch?v=eMlx5fFNoYc.
- We can work through chapter 7: https://www.youtube.com/watch?v=9-Jl0dxWQs8
- Computer scientists win Nobel prize in physics! Https://www.nobelprize.org/uploads/2024/10/popular-physicsprize2024-2.pdf got a evaluation of 5.0 for a detailed reading.
- We can evaluate https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10718663 for reading & discussion.
- Chapter 6 recommends material by Andrej Karpathy, https://www.youtube.com/@AndrejKarpathy/videos for learning more.
- Chapter 6 recommends material by Chris Olah, https://www.youtube.com/results?search_query=chris+olah
- Chapter 6 recommended https://www.youtube.com/c/VCubingX for relevant material, in particular https://www.youtube.com/watch?v=1il-s4mgNdI
- Chapter 6 recommended Art of the Problem, in particular https://www.youtube.com/watch?v=OFS90-FX6pg
- LLMs and the singularity: https://philpapers.org/go.pl?id=ISHLLM&u=https%3A%2F%2Fphilpapers.org%2Farchive%2FISHLLM.pdf (summarized at: https://poe.com/s/WuYyhuciNwlFuSR0SVEt).
6/7/24: vote was 4 3/7. We read the abstract. We could start it any
time. We could even spend some time on this and some time on something
else in the same meeting.
Transcript:
ML discussion groupFri, Dec 13, 2024
7:23 - D. B.
All right. So anyway, if you're new here, we call this the Machine Learning study group or maybe machine learning discussion group. And it's every Friday at four. We're going to even meet next Friday, although the following Friday, probably not. And we talk about machine learning and artificial intelligence. So just to kind of announce or describe agenda item number one. So there's a number of students in our master's program, information science master's program, who would like to do a project of the following type. So they're gonna, you know, the idea is to use the generative AI, like ChatGPT, Cloud.AI, you know, Google Gemini, some other, whatever, okay? Or maybe a book with pictures, they could use a picture generating AI. But anyway, the intent is to write a book about some, you know, with some simple title or simply stated task, like how to scramble an egg, how to cook an egg, how to plant, care for your persimmon tree, how to check and change the oil in your car, how to read the dipstick for the oil in your car. That's not as easy as it sounds for some cars. Any other question like that? And it's just sort of an experimental idea of just what would be the experience of using an AI to a full length book about it, or an equivalent task, not write a book, but create a informational website that's equivalent in content to a book, okay? Since a lot of people check websites instead of reading books nowadays. Ultimately, I would like to see a book like that self-published on Amazon or a website put up for people to, for the world to, make use of and we'll see how that goes as part of the project. And it's basically a prompt engineering problem because you can't just tell the AI to write the book. You've got to step it through and provide some structure to the task and get it to do it. Okay, that's the question. How do you get it to do it? And I think it'll be really interesting to see how a number of different students, what their experiences are in doing that. And And I think other people here would like to see how that goes and provide suggestions and advice along the way. So really, the only real requirement for the students is that you be attending these meetings.
10:20 - A. B.
Because otherwise, we don't know what you're doing. Like how will the, is it on the student then to kind of validate the quality of the output, like, you know, along the way, or is it just to, like, I guess, so what if they, you know, we, you pick a topic and they, you know, you start compiling these different, you know, they're prompting and compiling these different outputs, like, is it on them as the onus on them to then kind of validate the, you know, is the information correct before bringing it back into the folder? Is that part of what we're measuring or just curious? Well, yeah.
11:01 - D. B.
I mean, one of the problems with using AI-generated text is that the AIs, when they don't know something, they'll make it up and it'll sound pretty convincing. That is a problem to which I do not have a solution, but I'm glad you raised it. And it's going to have to be addressed somehow, maybe not successfully, but it's a question that has to, you know, that will hover over the project.
11:28 - Unidentified Speaker
How about that? Yeah.
11:29 - A. B.
Is the AI, you know, how do you tell?
11:32 - D. B.
Because, you know, normally, you know, maybe, you know, many times you don't know the answer. So you're, yeah, so that is a question. And I'm glad you raised it. And we'll have to have to make that part of the project somehow. Or maybe not, we'll just come up with a book that sounds plausible, but has mistakes. Some real books have that too. I don't know, does anyone else have any thoughts on that? Okay, so let's see who the students are here. We'd like to welcome E. T.
12:16 - M. M.
Oh, okay.
12:17 - D. B.
I thought there were like four different, there were like four or five people, but only two people are here, which might make the project a little more manageable. So anyway, thanks for showing up and you're already among the few and the proud and you were the ones to show up. One of the, the question that I sort of sent you early on was to come up with some, some title or question or something that would be the subject of your of your book or website. And I'm going to turn it over to you folks and let you tell us what you came up with.
13:01 - L. G.
It's okay, I'll go first. Sure. With the idea of how to bake a cake, because it seems fairly simple, but it does have quite a few steps. And areas where you could, you know, get the ads, maybe do something different, right? And to Mr. Berry's question, I thought that one of the things we would be studying in the process are the hallucinations. So, you know, it's a part of the book, but it's also something that you can document and you can get some understanding about, or you could either try to do some prompt engineering around it and say, hey, I ran the question x these number of ways. And so, still get a hallucinogenic answer to it.
13:45 - D. B.
So I thought it was that kind of part of the project it could be addressed. Absolutely, yeah.
13:55 - Unidentified Speaker
I was thinking gardening.
13:58 - E. T.
I mean gardening is my hobby and I sort of have some of the knowledge on gardening. So simply gardening, growing vegetables or mostly vegetables or herbs seems easy, especially if you go get your plants, seedlings from the store. It's fairly easier compared to growing from seeds. But I was thinking growing. From seeds. Okay.
14:34 - D. B.
It has some steps and especially some for certain vegetables. Yes, you know, the little, the virtually nothing I know about growing plants from seeds is, you know, a lot of times you have to stratify and scarify, all these things, you have to do the seeds to make them grow. And then sometimes they don't grow. I don't know. It's just, I think that would be a book that I would find useful and interesting to read.
15:08 - E. T.
OK.
15:08 - D. B.
Those both sound really interesting to me. And I hope that we'll all learn a lot by seeing how the process goes and where it works, where it doesn't work, where hopefully people have some ideas to get things going again when you get stuck, if you get stuck. Yeah. All right. And are you okay with meeting all next semester and maybe the semester after that, assuming you're doing a two-semester project on these Fridays at four? Because that's kind of part of the plan.
15:46 - A. B.
I was just curious on what the strategy would be to get, because 100,000 words is a lot of words, right? Right. Yeah.
15:57 - D. B.
Well, you know, a hundred. Yeah. I think that would be a pretty long book, but you know, if it's 70,000, that's, that's book length too.
16:05 - A. B.
Right. Well, but I guess, is it like, would the strategy be something to the effect of, I don't know, like you kind of identify like the larger steps, whatever the process is you pick. And then like, then, you know, as you get those, like keep asking, like, so, you know, the, whatever, you know, get your flower and then like, you know, kind of continually prompt and ask it to elaborate, elaborate on like one of that, you know, get you on the first step and then, and then, you know, go on to the second step. And so for like, I'm just trying to think through how to logistically do that. Right. Cause if you just ask it to, how do you make a cake, it's going to give you like a, probably a pat answer out of, you know, that's going to be at a certain length.
16:50 - D. B.
Right.
16:50 - A. B.
So then how do you, How do you continually get it to draw that out longer and longer, I guess, is the question.
16:59 - D. D.
That's prompt engineering, isn't it?
17:01 - L. G.
I did try something just before volunteering to do this project, because it's kind of outside of what I thought. So let's say, for example, it said, hey, to make a cake, we need to turn on the oven to 375. And then I asked it, how would I turn on in the oven to 375.
17:22 - A. B.
And they need to give you a whole nother list. And my favorite one I tried was like, the second step was like, you would need to grease a pan. And I'm like, how do I grease a pan?
17:34 - L. G.
So the whole process, like you can use butter and you can do this. And I just kept trying to bring it down.
17:41 - A. B.
Or, you know, or if you say, or you could ask a question, like, is there something I could add to this step?
17:48 - L. G.
Like, Is it possible to use, you know, large degrees of paying for your cake? And then it'll give you some more information that you could then add. So I'm just kind of playing around with it from that kind of sequential thing, but I thought eventually you would need to say, okay, you have this, can you make it a paragraph? You know, instead of being a list.
18:07 - Multiple Speakers
That makes a lot of sense. Yeah. Well, and I get what you're saying.
18:11 - A. B.
I was just like, I wasn't, I was thinking that more around where you prompt it one prompt and you get an output then you know like what's the you'd have to do something like you have to ask more specific questions about each like granular step right to draw it out of this what I was trying to get to you know make sense yeah that does sound like a really good strategy that he's got one of the things that
18:35 - D. D.
like if I was going to undertake this project I would definitely write about something that I could already do And that way I could verify its response. So I would be, it would be like me taking, I would take, I'm going to write this book and I'm going to get ChatGPT to help me so I can write it faster. That's the way I would approach the problem.
19:00 - Multiple Speakers
And then I already know what I want.
19:03 - D. D.
And then I would just have the AI write it for me.
19:08 - Y. P.
is saying 100% whether it is machine learning or generative AI feedback loop. Somebody did mention isn't that prompt engineering and part of prompt engineering is better prompts and that's where you know the agents and automated agents are going where the agents know what to ask better automatically and then they continuously do that until, but that feedback loop and feedback loop is only possible when you know the subject matter that you are doing things or prompting about very well. I concur with that. And that's why, you know, I was talking to somebody who's researching And that's what I was trying to say that when you're building an agent, what you are building agent on perhaps is more important because that's how your agents will actually do the right agency, if I can say that. But I agree with what D. was saying. I just wanted to concur on that. J. is kind of the agents guy.
20:24 - D. B.
He's really getting, he's not here today, but he was really getting to this whole agents idea. But I have to admit that I'm a little in the dark about it.
20:39 - M. M.
This is extremely good approach. Like we show some examples of multi-agents and how they can do different tasks, subtasks. And this is how we can approach creating a good book.
20:56 - Y. P.
And in second week or third week of January, my team will demonstrate agents in the area of product development. And in fact, we will have a couple of people on from UL are joining that project from January as well. If things go well, we are kind of interviewing or assessing them. And we will demonstrate some concepts of agents. It's still not at the autonomous stage, that is phase two of the project, but we'll demonstrate some agents in the product development area. And concepts of the way, whether prompt engineering or feedback loop, everybody was saying, sorry, this is my car. Be big, I'm safe. But what I was was that we will try to demonstrate some aspects of it and coming to D.'s and I don't know who was saying about prompt engineering on that having that subject matter being an expert in something that you are building things on or if you are an engineer then having somebody who is an expert with you perhaps will help you to do that.
22:16 - D. B.
Well you know just getting back to the nuts and bolts of these master's projects, it's good to have, you need three people on your committee, three faculty members or similar to, you know, in the role of faculty members. And, you know, it's good to pick someone with some expertise in the area that you're doing your project, whether it's technical expertise or some other expertise. So like, you know, I don't know if there's any faculty members around that are avid, bakers or gardeners but if there were you know that or maybe someone in another department that that you know of that would be a good person to have on your committee um yeah that's right um well um e. or um uh l. do you have any Any other questions you want to ask us about or anything like that? Any other concerns?
23:23 - E. T.
I actually have some questions about how we're going to, I mean, what are the requirements? What is the expectation every week during the meeting? What do I need to get prepared? Yeah, I mean, this is a kind of a general purpose discussion group.
23:40 - Multiple Speakers
We cover a lot of different things, talk a lot about a lot different things at different times.
23:47 - D. B.
So this is not going to take over the meetings entirely, but I'd like to see some sort of a one minute progress update that we can either absorb or comment on or something. In other words, it's not going to be a big time sink for each meeting, but I'd like to see some progress. Of course, if you miss a meeting or you don't have for a week or something, of course you know everybody knows that's that's fine but you know you should be making progress all along steady progress throughout the two semesters
24:26 - E. T.
and and therefore have something brief to tell us about each time yes are you know are you taking a two semester sequence um actually that's that was the only So I need to complete my graduate project to be able to graduate. So I have one course left in my graduate project. I talked to Dr. Pierce at the beginning of the fall semester, and she was like, maybe register for one credit graduate project, and then this spring term, you would register for the two credits so it would add up to three credits and it would fulfill your graduate project requirement. However, the thing is the course I registered along with the one credit graduate project for the fall semester was intense and unfortunately I was not able to do anything toward my graduate project.
25:35 - D. B.
Which is why you're here this project.
25:38 - E. T.
Okay yeah well let me let me just say you know these kinds some some of the sort of advising issues we can discuss um you know outside the meeting but the important you know if if you're going to graduate
25:50 - D. B.
next semester you've got to you got to do the whole six credits or whatever is left to finish the project if you're going to graduate in two more semesters or include the summer then you should do it across across
26:03 - E. T.
two semesters. I see. That's my guideline. Yeah.
26:06 - D. B.
Better to do it across two semesters. But if you're graduating in a hurry, you got to do the whole thing at once.
26:14 - E. T.
So this is for two semesters, spring and summer. Am I right?
26:19 - D. B.
Well, if you're planning on graduating at the end of next summer, then you would do it over spring and summer.
26:27 - E. T.
If you want to graduate in to do the whole thing next semester. OK. Would it be possible to do it during the spring term?
26:37 - D. B.
If that's, yeah, that's something we should really discuss. It's kind of an advising issue. You send me an email, and we'll discuss it.
26:47 - Multiple Speakers
I will.
26:48 - E. T.
Thanks so much.
26:49 - D. B.
L., how about you? What's your situation?
26:52 - L. G.
Oh, OK. So I don't know if more or less complicated than hers. I have three hours remaining of the requirements. I have three hours left for the final project, but the first part of my, I guess we'll talk about the rest later, but the question I had was, I'm open to other questions, but just like, I believe it was Mr., I don't know which person said it, I felt like I wanted something that I would know kind of what, you know, when you get down the path, what the correct answer was. Right. But I am open to other topics if there's some that you guys think may be better topics. I do. I've played around with agents a little bit. And so I'm curious to see what they would do with that. But I think maybe I try to try it out a little bit over the next week to see how that would work.
27:49 - D. B.
I'm going to leave it up to you. I mean, I kind of agree that, you know, if you write about, if you do this about some topic that you know something about, you're gonna be able to catch the hallucinations better. But if you don't, you know, I'm just, you know, again, this is exploratory.
28:10 - L. G.
I'm just curious what's gonna happen, you know?
28:13 - Y. P.
And I'm a big fan of, I'm a big fan of students doing what's interesting to them.
28:19 - D. B.
So if you're really, on writing a book about something you don't know anything about, I would say go for it and let's just see what happens. Yeah. Okay. Any other comments on, yeah, go ahead.
28:37 - Y. P.
The one idea that I have that we are using is sometimes, I'm going to give an example. So for example, we are using Next.js as a framework, which we had never used, very difficult to find subject matter experts. So it's very difficult when you don't have subject matter experts, which I spoke about that knowing something that you are putting agents on is helpful. So what I would, or we did was, which is a trick, is in those cases, you can use YouTube videos, top YouTube videos, which you can, I would say, so for example, if somebody is doing, scrambled eggs or something like that, or gardening, you find the top 10 chefs or whoever who have been scrambled eggs, their videos, take a summary of that, YouTube videos, use Gemini to do that, or there are other tools to create a description. And you can use third parties or third party sources as your subject matter expert also. Just an idea that we use because we didn't have a subject matter expert. Or we had to build our subject matter expertise in Next.js, which is a new framework. But that's another idea that we used third party reliable sources, like for example, for scrambled eggs, it is reliable top chefs, right? If they have scrambled egg recipe, then you can depend on those recipes because they are the top chefs in the world. I just wanted to give idea that sometimes you may be asked to do something where you are not the subject matter experts but you can use these sources as if you have heard about RAC and create that environmental layer on top of it which becomes like a learning module and you can use that as your virtual subject matter expert. Just sharing an idea if somebody has to choose a topic they're not able to find something or somebody who is a subject matter expert.
31:22 - Unidentified Speaker
Thank you. Thank you.
31:25 - D. B.
Any other comments? Well, J is not here. I keep wanting to show this video that he submitted to the AAAI conference. I'd rather he sort of thought it'd be better if he's here, but I guess I could show it when he's not here. And the video stands on its own and it's pretty interesting video. So let me do that. And we can always ask him questions about it if he comes back some other time or whatever. Anyway, so he submitted this This is one of the major academic artificial intelligence conferences. It's been going on for 40 years or more. Anyway, this year, apparently, they have a video track where they're soliciting AI-generated videos, or I don't know what kind of videos exactly, but videos intended for a public audience. And J submitted one. Let's take a look. I'm gonna bring it up here and I may have to, actually, I need probably to unshare my screen and then optimize for video or something. Let me try that. I'm gonna optimize for a video clip. Reshare. Okay, I am sharing, am I not?
33:00 - D. D.
I do not see a video. There is no video.
33:04 - D. B.
Okay, but you see my page, right?
33:06 - Multiple Speakers
Yeah, I see your page.
33:08 - D. B.
Yes, you are currently sharing your page. All right, I'm gonna go to the video. Okay, do you see the purple? We do. Okay, I'm just gonna go ahead and play it, you know. This is from J. J submitted it. He made it. I'm going to go ahead with it.
33:27 - Unidentified Speaker
A lot of people view generative AI as a threat to creativity, that it's going to replace artists. But I think it's going to have the opposite effect. It's going to make us more successful than ever. Artists who learn prompt engineering, which is just the technical term for communicating with AI, can build a team of mentors and collaborators. For example, if you use a prompt like, you are an expert grant writer for independent painters, based on my work, please walk me through the process of filling out this application. You can put that prompt into chat GPT, and because you've told it that it's an expert grant writer, it's gonna be able to act as that expert. Same thing if you say, you're a publicist for photographers, please help me plan my social media strategy. This kind of mentorship used to only be available to the top tier of creators. Now everyone can have a world-class art coach.
34:34 - Y. P.
Generative AI is also multimodal, meaning that it can produce text, images, audio, and video.
34:41 - Unidentified Speaker
So it can complement your work as a collaborator. Let's say you're a musician. You are really passionate producing your music, but now you can team up with AI to generate album art, create music videos, and produce writing that complements your work in a way that you couldn't do before. So AI isn't going to replace what we do best, it's going to amplify it. No matter what your style, medium, income, location is, you now have the opportunity to reach audiences that you wouldn't have otherwise and have a greater shot at producing art successfully full-time. So I guess what I'm saying is, it's not gonna replace us, it's gonna empower us. Oops, trying to catch that.
35:36 - D. B.
Yeah, because it says what methods you used.
35:41 - R. S.
Some of those I haven't heard before.
35:45 - D. B.
Yeah.
35:46 - R. S.
Cling AI, Suno AI, and 11 labs. OK. But I mean, match between the text and images.
35:57 - M. M.
There is no exactly kind of link between the text and images, but otherwise the text is wonderful. The images. OK.
36:13 - D. B.
Images are kind of random. I mean, they're kind of fun, but they're not...
36:19 - M. M.
Not related to the text.
36:22 - D. B.
Yeah, there's a loose connection, I guess.
36:25 - M. M.
Yeah. Well, that is good. I can see V is here and V has a very good design experience. Maybe one moment he will share with us now or later, V?
36:39 - V. W.
I was just enjoying seeing... I enjoyed seeing J's video for the second time because it has an impressionistic quality that I didn't appreciate the first time. I had been in a similar situation where I was trying to give life to some disparate ideas using and when you use runway, you can take an image that's on task for what you're trying to do, and then you can give it a prompt and you get a three or four second video out of it. And then you string those together like a storyboard and you've got content. So I thought it was very creative. You know, J is a science fiction writer, among other things, and I could see that influence in his artwork. So, yeah, this week I signed up for the two hundred dollar a month chat GPT pro, and it is truly the road to perdition because I found it so engaging. And it was like having a superpower king for a day or something like that. And so I just I just chose a task that I thought would be interesting. And I kept making progress I didn't expect to make. And it was really fun and very, very exciting. And it wore me completely out.
37:52 - D. B.
I'm still recovering from it, so. Well, if it wore you out, that means you weren't just sort of lazily letting AI do the work. You were sort of more truly part of an active part of the larger Yeah, it's like having a brainstorm that won't stop.
38:11 - V. W.
It's because whatever you can come up with, it amplifies it. And then the the relationship, as J points out, is. It's it's incredible because you're able to do all these things you always wanted to do that you thought might take a year, you can do them in seconds or moments or, you know, minutes or hours. And it's a very intoxicating adrenalizing type of activity and like what E said you know the AI is changing us and I'm not completely sure. I think it's something we're gonna have to learn to manage.
38:52 - D. B.
So Dr.
38:53 - Y. P.
W., can you share something about what was so different between the $20 and $200 if you with some example or something. Now you've got me excited to sign up for 200 and try something. Right.
39:09 - V. W.
I totally recommend that you do that because you can, well, I was told that you could back out of it if you wanted to, but I shouldn't advertise that as being true because I don't know for a fact that it's true. I just heard or read a report about it and it said so. So, you know, this is a do it your own risk kind of activity, But I would recommend that you work in an area that you're very familiar with so that you can judge the quality. And also, you can take these giant steps. Whereas before in programming, we design top down, but then we implement bottom up a line of code at a time. And I just wasn't used to developing 100 lines of high quality code. Over 100 lines of high quality code an hour is just a thrilling experience. Because it's such a the right it's a right question so it I think it lights up your whole brain and yeah I still I'm still coping with what happened I remember it being very exciting just the whole it was
40:16 - Y. P.
so exciting I continued doing it for 12 hours and so you know I'm not sure that's completely a good idea yeah and most likely that will happen to me too if you are that excited but I'm I'm curious I'm I'm going to use it now I may have some feedback about this my personal because I was like this paid one basic one is helping me so much how much great it would be but it seemed like there is a lot of value I'm trying to see what that value is I kept running I kept running against limits early on when trying to program with it.
41:00 - V. W.
You have to kind of babysit it. You have to figure out how you're going to talk to it so it doesn't forget what you're talking about. And that is to some degree been released. What I found out by sort of hard, the school of hard knocks was that it cannot deliver back to you in one gulp the complete rehash of what you've just talked about. So I was able to go to 20 shots, just getting a monolithic response. But then at 20 shots, we had to break the project up into eight chunks because it couldn't deliver all the content that had been accumulated because it was even past the 200 a month buffer length. But what was nice that I didn't realize before is that it had retained state information that it had simply forgotten before. So it's like you have access to a larger persistent state of consciousness, but you still have to get that back out in bite-sized chunks. It's a little bit like a person who's in a coma and you say blink once for yes and twice for no or something, except the chunk size is much bigger than a blink, but there's still this, there's still a
42:15 - Multiple Speakers
bottleneck on communication that it can't, you know, it can't completely flush out everything you've done every time. Does it have capability to build agents, automated agents or anything like that?
42:26 - Y. P.
No, actually.
42:27 - V. W.
Well, the right answer is I don't know, because I had to build it took me an hour to build the prompt for what I wanted. I gave it a loose description of what I wanted that had all the facets I thought were important. But then I said, please interview me. In each of these areas of graphical user interface and geometry and all the user interface subtleties that make a program usable with low user friction. And then it spent about an hour asking me questions that I answered as completely as I could. And then it created an eight point multi-page summary of what we were going to embark on. And then and only then did we embark on the first step of the eight step steps to try to get a version running one chunk at a time, which we did. Got it. Thank you.
43:24 - D. B.
So I'm wondering, you know, $200 a month sounds like a lot, so I'm glad they offered you a free two weeks or whatever. But, you know, if you're using it for your job, like you're something or, you know, professionally, it doesn't take that much an improvement in productivity to make it worthwhile. I mean, when you think about, like, take a professor, for example, $200 to me sounds like a lot to be spending on something for fun. But, you know, if it could save, you know, if it saved me several hours a month in time preparing classes or something like that, you know, it'd be worthwhile.
44:09 - V. W.
For example, a professional video editor will cost you $400 an hour to sit with them and put together content that's important to you. Or if you want to rent an airplane, it's going to be $120 an hour for the airplane and 50 for the gas. And so that's per hour. And so here we have something for 200 a month. So I asked myself if I work 12 hours a day for 30 days, it only works out to like a couple of dollars an hour. And so since it's like having a heavy machine tool in your shop, you have to pay for having this heavy machine tool there. But if you're always using this giant lathe to make stuff, then it sort of becomes absorbed into the cost of doing business. And this is like, it's so on the center line of exactly what it is we're doing in knowledge engineering and understanding what these tools are capable of, and acting as extensions of ourselves that it's, it's the only thing I'm
45:09 - Unidentified Speaker
missing is more hours in the day to consume it and, or use it. It's a lot of fun.
45:18 - V. W.
Yeah.
45:19 - D. B.
Or if you're, if you're a programmer, if it makes you 10% more productive, 15% more productive, it was probably a boost your professional career.
45:34 - V. W.
or it's more like 10 times more productive, which makes it a little bit addictive because remember you're used to struggling to get, well, you know, they say at JPL, the average programmer built four lines of code a day. And I thought, well, that's silly. Most people I know can do a hundred lines of code a day, but then it was like, well, you can now do a hundred lines of code in an hour. So you're eight to 10 times more productive. So that's allowing you to cover ground that you couldn't cover before intellectually because it's all in front of you.
46:05 - R. S.
Are you using artificial intelligence to generate 100 lines of code an hour? Yes.
46:13 - V. W.
But you have to guide those 100. I mean, you totally have to steer it.
46:20 - D. B.
But here's the thing I was thinking about.
46:24 - V. W.
It was like being able to have a magic wand And instead of saying how I wanted something done, I could say what I wanted done and the how you get for free. So it's like being a manager who can take a magic wand and simply illuminate tasks that you want done and then poof, they exist. And now what are you gonna do? And there were a couple of points where I had to do off task activities to complete the main thread. Like I had to build an airfoil database. And so I said, I don't really feel like doing an Air Force database. So give me a bash script that'll create a directory and populate it with these data files for these airfoils. And so I said, oh, okay, I'll do that. It built me the bash script. I ran the bash script and now I have the thing. It was like, kabam that we, I mean, I could have done it manually, but it would just took all that heavy lifting out of the equation and just allowed me to have the thing I needed right now. And I wondered like, it's like being a man, a total manager instead of just a grunt who's shaving off one piece of code at a time, like you're whittling or something.
47:39 - D. B.
Another question I have is, so J is into this idea of telling the AI, maybe you are too, V, telling the AI, you are an expert in gardening, tell me how to garden, as opposed to just prompt, like, just tell me how to garden. Does telling the AI you are a world-class expert make it do better?
48:05 - V. W.
Well, to me, the task factored a little bit differently. It was like saying, I want a hundred rows of corn now. Like it says, poof, you have a hundred rows of corn, now what? It's like, you already have the garden. That you can conceive of, you are just a few moments away from possessing. So that's an odd sort of superpower to have, and you have to actually learn how to use the superpower. So I spent a lot of time having it ask me to construct things. Then we ran into a weird misunderstanding about a couple hours in, and neither one of us knew where the misunderstanding was. And when we finally found it, it was extremely illuminating and got rid of a bug. And it had to do with the conversion from SI to imperial units, display calculations and internal calculations. And we had to make policies like internally, we always use SI units, but the user can see imperial units if they want, but those are not the internal versions. And we had to make these policies. So we jointly created policies that would simplify the code and make the chance for errors go away. And so then the error went away and it was a very kind of collaboration like you have at a meeting where a project is run behind, people are in trouble, they don't know exactly why things aren't going well, mythical man month, Fred Brooks and all that. And then you just have this deep discussion, knock down, drag out, fight to get at what is the problem here. And then when it's solved, it's incredibly like uplifting. So I got into one of those kinds of troubles. So, and then I got it. But the nice thing was, is instead of being in that trouble, you get into trouble, you recognize the trouble, maybe you've over-constrained the problem description, and that over-constraint is showing up later as having made something you thought was easy impossible, because there's a built-in contradiction in these conflicting goals that you've specified, but you haven't realized that embedded in your goals is this conflict, and then in the process of doing it, you understand the presence of this conflict, you resolve You create a policy to resolve the conflict and that process can usually take weeks or months and often are accompanied with tears and budget overruns and difficulties. And to be able to get through that in moments instead of months is very empowering. It's just, yeah, I was joking with my hand radio friends. This is the road to partition because it's so uplifting and empowering.
50:37 - D. B.
All right, well, we're kind of running out of time. Mentioned something that I'd like to spend a few more minutes on like next week. So N. is here and he's like to turn his master's thesis into a publishable article. And the idea that I sort of propose is to use AI to do it as much as possible, but not have it look like it was AI generated. Because otherwise, you know, publishers aren't gonna like it potentially. But more importantly, you don't want to have it so overly AI generated that its truth is compromised, but yet you want to use AI as much as possible. So that was the sort of task I posed for N., and I think I'd like him to address it next week for a few moments, although we're kind of out of time for this week.
51:31 - V. W.
That's a great task.
51:32 - N. M.
In a short but quick explanation, I did all the work I had all the research. I can explain things in my own way, but a lot of people fail to understand how I'm explaining it. So I put all the research into ChatGPT, point by point, and say, rewrite. Here's my work. Rewrite this to make it more comprehensive. Right. And having it not let...
52:00 - V. W.
I have to often discipline the AI not to take away my voice. And it can use previous interactions that it's had with you. It can use the personal profile that you can build with chat GPT to make sure that it's staying on voice. And often, especially when you use a lesser AI like Grammarly, it'll try to rewrite some words or phrases of emphasis that are not you, and you have to not let it do that. But also AI and the large can do that. And you can almost start reading it when it gets too flowery or too descriptive or too complete, It lacks that human feeling, and people will kind of tune out of it because they can feel it's automated.
52:42 - Multiple Speakers
It's automated. It's generated. It's whatever.
52:45 - D. D.
Exactly. Exactly. Yeah. I mean, I think that my approach would be to try to get the model to teach me how to write better and then write better.
52:59 - Multiple Speakers
That's how I would approach this. I wouldn't have the AI write a single word that I used. Well, that's not true. That's a little over.
53:08 - D. D.
I wouldn't want it writing complete sentences on its own. I think you've identified the crucial thing of chunk size.
53:15 - V. W.
If you write a whole big paper and then you feed it to AI and say, now make a decent paper out of this, the chunk size is so big that you're going to get you overwritten with it. But if you hand it smaller crafted chunks and say, these to reflect very specific lexical and semantic goals that I have. Let's just get that piece tightened up. Then when you make those a brick at a time, accumulate, you get something that reflects your own creativity and not just the AI slop.
53:47 - N. M.
Yes, that's exactly what I did. I didn't feed it the whole paper. If I was talking about the flaws of the H-index, let's say, because that's what my paper was about, I would give it one point. I'd give it the information that I collected, the research I've done about one point of the falls. I'd say, rewrite this and make it more comprehensive. Then I would deal with the next one and the next one and the next one.
54:15 - V. W.
I wouldn't give it the whole thing because I knew it would eventually merge something or make it into a format that I would not like, It'll tear stuff up.
54:26 - Unidentified Speaker
Exactly.
54:26 - N. M.
Exactly. Well, thanks, everybody.
54:28 - D. B.
And I guess we'll go ahead and meet next week. But the following week is going to be kind of whatever. We'll see. But we're going to meet next week. And we'll go from there.
54:43 - V. W.
Awesome.
54:43 - D. B.
So everybody, have a good weekend. And we'll see you back soon.
54:48 - Multiple Speakers
Great meeting, guys.
54:49 - N. M.
Thanks. Thank you. Bye, everyone.
This
hackathon represents a remarkable opportunity to push the limits of
what is possible with LLM Agents technology, connect with a vibrant
community, and potentially win big.