Machine Learning Study Group
|
Contacts: jdberleant@ualr.edu and mgmilanova@ualr.edu
Agenda & Minutes (143rd meeting, Dec. 20, 2024)
Table of Contents
* Agenda and minutes
* Transcript (when available)
Agenda and minutes
- Announcements, updates, questions, presentations, etc.
- Meet next week, 12/27? No, next meeting will be Jan. 3, 2025.
- Recall the masters project that some students are doing and need our suggestions about:
- Suppose a generative AI like ChatGPT or Claude.ai was used to write a book or content-focused website about a simply stated task, like "how to scramble an egg," "how to plant and care for a persimmon tree," "how to check and change the oil in your car," or any other question like that. Interact with an AI to collaboratively write a book or an informationally near-equivalent website about it!
- BI: Maybe something like "Public health policy."
- LG: Change to "How to plan for retirement."
- ET: Gardening (veggies, herbs in particular). Specifically, growing vegetables from seeds.
- JK is focusing on prompt eng. with agents and may have comments if present.
- If anyone else has a project they would like to help supervise, let me know!
- JK proposes complex prompts, etc. (https://drive.google.com/drive/u/0/folders/1uuG4P7puw8w2Cm_S5opis2t0_NF6gBCZ).
- NM project.
- Here
is a tool the library is providing. Some people here thought it would
be a good idea to try it live during a meeting, so we did.
Library trial of AI-driven product Primo Research AssistantThe library is testing use of Primo Research Assistant, a generative AI-powered feature of Primo, the library's search tool. Primo Research Assistant takes natural-language queries and chooses academic resources from the library search to produce a brief answer summary and list of relevant resources. This video provides further detail about how the Assistant works.
You can access Primo Research Assistant directly here, or, if you click "Search" below the search box on the library home page, you will see blue buttons for Research Assistant on the top navigation bar and far right of the Primo page that opens. You will be prompted to log in using your UALR credentials in order to use the Research Assistant.
The meeting ended here.
- The campus has assigned a group to participate in the AAC&U AI Institute's activity "AI Pedagogy in the Curriculum." IU is on it and may be able to provide updates when available, every now and then but not every week.
- Anything else anyone would like to bring up?
- Here are the latest on readings and viewings
- Next
we will continue to work through chapter 5:
https://www.youtube.com/watch?v=wjZofJX0v4M. We got up 15:50 awhile ago
but it was indeed awhile ago so we started from the beginning and went
to 15:50 again.
Next time we do this video, we will go on from there. (When sharing the
screen,
we need to click the option to optimize for sharing a video.)
- We can work through chapter 6: https://www.youtube.com/watch?v=eMlx5fFNoYc.
- We can work through chapter 7: https://www.youtube.com/watch?v=9-Jl0dxWQs8
- Computer scientists win Nobel prize in physics! Https://www.nobelprize.org/uploads/2024/10/popular-physicsprize2024-2.pdf got a evaluation of 5.0 for a detailed reading.
- We can evaluate https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10718663 for reading & discussion.
- Chapter 6 recommends material by Andrej Karpathy, https://www.youtube.com/@AndrejKarpathy/videos for learning more.
- Chapter 6 recommends material by Chris Olah, https://www.youtube.com/results?search_query=chris+olah
- Chapter 6 recommended https://www.youtube.com/c/VCubingX for relevant material, in particular https://www.youtube.com/watch?v=1il-s4mgNdI
- Chapter 6 recommended Art of the Problem, in particular https://www.youtube.com/watch?v=OFS90-FX6pg
- LLMs and the singularity: https://philpapers.org/go.pl?id=ISHLLM&u=https%3A%2F%2Fphilpapers.org%2Farchive%2FISHLLM.pdf (summarized at: https://poe.com/s/WuYyhuciNwlFuSR0SVEt).
6/7/24: vote was 4 3/7. We read the abstract. We could start it any
time. We could even spend some time on this and some time on something
else in the same meeting.
Transcript:
ML discussion groupFri, Dec 20, 2024
1:01 - Unidentified Speaker
Hi, everyone. Good afternoon.
1:06 - Multiple Speakers
Good evening.
1:09 - D. B.
How are you?
1:13 - Multiple Speakers
Good.
1:19 - D. B.
We'll give it another minute. And we'll get going.
1:53 - Unidentified Speaker
Hi, D. H. Hello.
1:55 - Y. P.
How are you? Good.
1:57 - D. B.
How's it going with the all those people that you were?
2:03 - Y. P.
So I spoke to V. today. And I'm going to have a follow up call tomorrow. D. was to send me some updates. And the other guy never So I think only three responded. And I'll send you a summary of this weekend. I'll tell you, give you an update of the people who have reached out and I've been talking to. But it seems that V. is the first person who is likely to start. And then D., I think I'm waiting. He said he wants to update his profile and then get back. So I'm waiting for him. And R., right? There's one person, I don't think I got ever a response, but two candidates I'm in touch with. And I think that's what you're aware of, right? There are three people who reached out.
2:59 - D. B.
Oh, well, there were more than I thought there were like five or six or something, but you know, you expect some people to sort of make an initial inquiry and that's it. And the same things happen with the idea of using AIs to write a book. Several people responded, and only two people were here last week. But B. is here this week. Welcome. Thank you.
3:30 - B. I.
Yeah. So anyway. I'm sorry. I did send you an email regarding the other project. I'm not sure if it's But
3:45 - Y. P.
And who is speaking right now? I'm sorry.
3:48 - B. I.
My name is B. I. OK, I did not see your email. Maybe for some of you, I may have to look at the spam. I'm sorry about it. OK, I'll send another mail.
4:02 - Y. P.
What I'm going to do is in the chat, I'm going to share my phone number. If you don't hear back from me today or tomorrow, just text me or whatever.
4:14 - B. I.
accept me and I'll make sure that somebody will reach out to you. Okay. So let me thank you.
4:30 - D. B.
All right.
4:32 - L. G.
All right, so Y., did you did you put it in the chat?
4:49 - D. B.
I'm going to put in two minutes.
4:57 - Unidentified Speaker
Okay.
4:59 - D. B.
All right, so the next thing, I guess, or the first thing, nominally, do you all want to meet next week? Or should we skip that and go on to, skip on to January? Okay, well, why don't we- I'm good either way.
5:20 - Multiple Speakers
Yeah. I'm okay.
5:27 - N. M.
I won't be here next week.
5:30 - A. B.
I won't be here either next week.
5:33 - D. B.
Yeah. Well, let's go ahead and move. We'll skip next week and meet on January 3.
5:54 - Unidentified Speaker
OK, let's see what else.
5:57 - D. B.
OK, so last time we started talking about this project that a number of people, masters students, would like to do, which is using generative AI to write books or websites that are roughly equivalent to a book. And so we had two people give topics for their books. I proposed tentative topics for the books last time, and B. is here today. Did you get that message about suggesting a topic for your book or website?
6:42 - B. I.
So I'm still thinking, but I was thinking of doing something Maybe I was thinking of doing something in policymaking, so something like public health. But I haven't gone too deep. I'm still working. Well, I think a book on public health policy would be pretty interesting.
7:10 - D. B.
And it would be an interesting counterpoint to how to bake a cake. Or how to grow vegetables from the seeds.
7:21 - L. G.
It's just a different experience. It'd be interesting to see what happens.
7:26 - Unidentified Speaker
Dr.
7:27 - L. G.
B., I would like to change mine. I did a lot of preliminary research and kind of took the feedback that Dr. P. gave and someone. I wanted to change it to how to plan for your retirement. It gave a lot more options for this book. It made it a little bit more interesting because I could bring in some secondary, like you said, bring in YouTube videos as sources that the determinative stuff was correct. It just gave a lot more diversity in what the AI could give us. I thought it would be a little more exciting topic and more readable for average people.
8:10 - D. B.
Definitely. I'm moving towards retirement at some point. All right. Yeah, that's interesting that the two topics would give different kind of experiences as you try to use preliminary investigation of them. And I don't know who it was, but two people had ideas.
8:30 - L. G.
Instead of doing the prompt or the chain of thought prompting and so forth, maybe you could use agents. And then one person I think we're saying, hey, you could use a single agent, or maybe you could create multiple agents that work together. And so I'm still kind of sketching that idea out kind of like on the back of a napkin. Like if you, if there was like a comparison you could do between the three methods, like what is the right amount of human interaction in it maybe, if there's like a comparative nature that could happen there, or if there was something I could, how many agents would we use in the morning? To agent, like so forth like that.
9:13 - D. B.
From what I understand, agents are a big kind of focus currently of, or there's a lot of interest currently on agents as prompts or agents for prompts, something like that. And J., who's not here this week, is sort of focusing on agents. And if he shows up maybe next time or whenever he shows up, I'll just ask him to tell us a little bit about his agent-based process that he's been thinking about. So yeah, agents for prompting is something I'd like to hear more about personally. OK. And E., welcome. Anything that you want to say about your project?
10:07 - E. T.
Hello. I'm not sure if I need to change my idea. I mean, if that's necessary, I don't mind changing the idea.
10:17 - D. B.
Oh, no, no. It's totally up to you. I mean, it's just really, it's not something that we're expecting or not expecting.
10:27 - E. T.
No, I mean, I just didn't know those were some of the options over there. Went one of them, went with one of them. If it needs to be something more related to information science, I can go with something like that. But again, I haven't thought of something different yet. I gave it a start, though, with Chachapiti. And so just general prompt, if I wanted to write a book about gardening from seeds, growing from seeds, it gave some general steps. And then I started diving deeper into like, if I wanna choose, because the first step was choosing the right seeds for the environment, for the climate, the place I live in. And I was like, okay, how do I choose the right seeds? What vegetables would grow in better in my area? And it started giving me more details. It went on for a while, but that was all I have done towards. All right.
11:38 - D. B.
Well, as you work on it, probably questions and issues will come up, and we can talk about them in our meetings.
11:49 - E. T.
All right.
11:49 - D. B.
Anybody else has a project they want students for? I can always send out another solicitation for students, especially after the semester there's probably going to be some looking for projects. This is from J., but he's not here today. So we'll put that off till next time. And N., did you have anything you wanted to mention about your project? No, sir.
12:15 - Unidentified Speaker
Not at the current moment. OK.
12:18 - N. M.
All right.
12:19 - D. B.
So this has been on the agenda for a bit. There's a poll that came up.
12:25 - Multiple Speakers
Dr. B.? Yeah. I have a quick question.
12:28 - B. I.
As I continue to flesh out my projects, can I always update and change stuff? Does it really have to be the same thing that I said today, or can I change as I continue to do more research?
12:45 - Unidentified Speaker
Yeah, you can change. OK. Yeah, no problem.
12:49 - D. B.
OK, so there's this tool that the library has. It's an AI-driven research assistant to help you the library or help you find sources or something. They call it Primo Research Assistant. And people were interested in, and I thought it'd be interesting to go there and just try it out. So why don't we do that? So you go here.
13:20 - Unidentified Speaker
OK, well, I'm going to the library.
13:38 - Unidentified Speaker
Oh, there it is. Let me raise the phone. Well, any anyone have a question they want to ask?
14:00 - D. D.
Yeah, yeah. Information about AI as agent.
14:12 - Unidentified Speaker
Let's see.
14:22 - D. B.
OK?
14:23 - Unidentified Speaker
So it wrote a lot of stuff here. We'll go through it bit by bit.
14:37 - D. B.
All right, why don't we read this paragraph and then and see if there's anything to talk about.
14:54 - A. B.
And then the one reference, I guess that's pointing to a specific source below. I don't know.
15:11 - D. D.
That's what it looks like, but I don't see two anywhere.
15:16 - Unidentified Speaker
It says stream. Yeah.
15:18 - A. B.
Look at that ebook for right there, agents and multi-agent systems.
15:22 - D. D.
Why is it not referenced? Might be.
15:26 - D. B.
Oh, here it is referenced down here. I don't know why they numbered it up to three. All right, anything else on this paragraph? On the reference, did I read that right? It was a video?
15:46 - A. B.
Let's see. Yep, streaming video. So if you click on that, it actually takes you to the video reference.
16:02 - Unidentified Speaker
That's really neat.
16:04 - D. B.
I don't know. Oh, here's a reference.
16:09 - A. B.
Can you scroll down? I'm curious if it had the direct link to it or something. It has more details.
16:29 - D. B.
OK.
16:30 - A. B.
Well, it is a video, but it doesn't seem to have the video here.
16:36 - D. B.
You can find available services, I guess.
16:40 - A. B.
OK, well, it gets you back to the database, essentially. So got it.
16:46 - Unidentified Speaker
OK. All right.
17:05 - Unidentified Speaker
Let's go for this one. Any thoughts or comments?
17:33 - D. B.
I guess I would comment that unless the AI agent has access to real-time sensory input, like camera inputs or microphone inputs or something like that, it just basically lives in the web, which is a kind of virtual environment. That's my only thought.
17:55 - D. D.
Well, the thing is, even in a prompt environment, if you So if you have that kind of information, then you would need some type of a playground or interface API where you can reach the agent. And then you would have to have some other kind of script or algorithm that's running that's grabbing that real-time information and then using it in prompts to prompt the AI. So then it would have access.
18:29 - D. D.
you know, feasible. I mean, it seems like you could do it. I mean, I was able to prompt, prompt AI to the playground, you know, right from my, and get feedback from the AI, right from my, what, what's that visual studio code that the free Python or, well, I guess it's for more than Python, but the IDE.
19:00 - E. G.
Yeah, I was able to do it all inside.
19:13 - Unidentified Speaker
Yeah. All right. All right. Let's do those.
19:26 - Y. P.
Anything?
19:27 - D. B.
Well, to me, an interest in a focus or going in the direction of agents means making AIs more like AGs. Artificial General Intelligence, where they're doing higher level coordinating and managing like you'd currently expect a human to do. You could have a book writing agent and then the master's students would have a very easy time writing a book because all they have to do is call the book writing agent.
20:18 - N. M.
That's true.
20:22 - D. B.
you know, how close we are to having a book writing agent, I don't know, it might be a ways into the future or maybe it's doable now, I don't know.
20:39 - Unidentified Speaker
All right.
20:40 - Y. P.
Yeah, my point of view is, you know, even when you say book writing agent, it could be something that is telling you about how to write a book and flow a book and the language and depending on the type of the book whether it's humor or classic or whatever it would be but then also you could be very focused on the subject matter like a book on somebody was talking about food, cooking, gardening whatever it would be and and there would be agents that you know kind of are cross-functional so an agent that just is focused on writing a book and in a particular manner whether it's a memo or biography autobiography and all that and then there are agents that are very specific to topics so I foresee all the agents multi-agent whatever cross-sectional eventually getting into perfection. And to give example that, you know, the team members that are joining, we'll be focusing on two sets of code, one for user interface, which is CSS, HTML, and JavaScript. And then we will focus on Python. These two, if you remember, Dr. Warren had shared that chart. So we have picked up two and then few dimensions. And a few models. But essentially the goal is, can we get a perfect code without any interference? That will be the end vision or the goal that we have. And there will be agents, autonomous agents. So we are building something. I think somebody spoke about APIs, where it will continuously do reiterate and similar to machine learning and others we'll have feedback and it will improvise and then they'll again have feedback it will improvise to an extent that it becomes perfect so that is where I think all these will go but I think it will also be cross-functional that's what I feel eventually it will go to that there is a technical piece of it and there is like an industry piece of it and and then you are getting these outputs or UI code and rules engine or the backend code or API code coming out. And then this, I wanted to share. Any thoughts on that?
23:22 - D. B.
I think that's sort of J. was also talking about having collections of agents, if I understand where you're going. And I can kind of see that a specific agent is very specialized. So if you're going to have sort of general, generality, you'd need a, like a collection of agents that would have to, then you'd have to have something, an agent that coordinates agents or the human coordinates, the agents or something
23:55 - Unidentified Speaker
like that.
23:56 - Multiple Speakers
Um, uh, M., are you familiar with orchestration engines? Huh?
24:01 - E. G.
Are you guys familiar with orchestration engines?
24:04 - D. B.
No. An orchestration engine.
24:06 - E. G.
Um, Say I've got a series of applications that I'm running. Some have predecessors, some have time constraints, some have validation routines. They have to follow a certain schema. An orchestration engine basically takes all of these little pieces, these little applets, code segments, runs them based on parameters. What orchestrates how the information is going to flow across or what's going to run to do certain things. I use the term information at this point because a lot of time orchestration engines basically pull in data, do stuff with it, and then hand it off at the end. But there's a lot of different things that'll do. Say if you have multiple sources that you're bringing in and you're consolidate it down to one format. A lot of times what you're doing in healthcare with FHIR or HL7 from the EMRs, the orchestration engine basically does that. That's what it sounds like you're doing here, is you're basically having this overarching process that'll go out, run these different things. Say, writing a book, first we're going to start with a topic. So you're going to run against an AI engine that's good at setting up a template for a story. Then you're going to take each section and send it out to another one that can allow it to delve deep. A third one that'll identify references and resources so that way you could apply those links to it. And you have that type of cascading effect, almost like an annealing process with a thought.
26:12 - Multiple Speakers
Well, the word orchestration literally means sort of high level management.
26:18 - E. G.
And isn't this what it's doing?
26:21 - Unidentified Speaker
Yeah.
26:22 - L. G.
I mean, I That's kind of the way I was thinking about how you could use it too. I guess I was looking more kind of like if you had, if you had like a, kind of like a hierarchy and then you have, you know, kind of one checking the other one, checking the other one, and eventually someone kind of doing the sourcing for it, you know, and that you know, kind of one checking the other one, checking the other one, and eventually someone kind of doing the sourcing for it, you know, and that way, instead of like me trying to catch all the hidden stations, you have like one AI that pulls, you know, information to check what the other AI had written to then say, OK, maybe we need to rewrite this paragraph. And with third, AI would check that and go through and input. So it's just kind of like what Mr. I don't know if it's possible, but that's what I was thinking, how you could use a multi-agent environment.
27:09 - D. B.
So like a hallucination checking agent.
27:11 - L. G.
Yeah, yeah, something like that. Or even an editor, like an editor in a way.
27:15 - D. B.
Yeah. Or in real life, they wouldn't call it a hallucination checking agent. They would call it a fact checker.
27:21 - L. G.
OK, a fact checker. Well, I've built validation engines.
27:26 - E. G.
So what I'll do is I'll take the output of one AI, pull it in and run it against another AI to validate whether or not the first is hallucinating.
27:39 - D. B.
What would happen if you took the output of one AI, just plugged it into another, just paste it into the of another AI and said, please fact check the following.
27:55 - N. M.
It's the same thing.
27:58 - E. G.
The only difference at that point is you're automating the process versus manually handing it off.
28:07 - D. B.
Yeah. All right. Any other comments? Not from me at this moment.
28:14 - N. M.
I have a question for A.
28:18 - Y. P.
heard you saying validation engines between the prompts. Is it like a plug and play? Have you used something industry that is available industry wide or did you build it grounds up? I built it ground up.
28:36 - E. G.
I'm building out the prompt and I've got the processes. I'll actually have multiple threads running where I'm taking the output one prompt handing it off to another process would read it as the input into another prompt. This allows for the desegregation. And also, a lot of times, if you keep it in the same process, it has that memory. So it'll hallucinate that, yeah, I'm right. But if you put it into another process, it doesn't have that fresh memory or what's called warmed cash at that point. So it's not going to pull from it. OK.
29:22 - Y. P.
Are you open to having conversation in the next two, three weeks? I don't know how your holiday schedule is about this topic offline. I'm very open to it.
29:34 - E. G.
I love talking about it. OK. I mean, this is, this is my hobby. All right. Let's speak about it.
29:43 - Y. P.
I'll send you a message.
29:45 - E. G.
Got it, sir. Thank you. All right. All right.
29:49 - D. B.
So I'm just kind of scrolling down. And I guess, as I interpret it, so far, this is what you'd expect from any AI. This is Primo Research Assistant, not ChatGPT. But it almost could be ChatGPT, right?
30:04 - N. M.
But now it's giving us this. Since it is a library, it's giving us links to sources.
30:13 - D. B.
What do you all think?
30:16 - E. G.
I'd like to put the same thing into OpenAI or chat GPT. So see what we get for an output. But let's let's manage the prompt a little bit better where we ask for say five references. Yeah.
30:39 - D. B.
I think if you ask chat GPT to give you some references, it probably will. And then see if it gives us the same ones.
30:50 - Unidentified Speaker
Yeah.
30:51 - Multiple Speakers
It's pretty bad at references.
30:53 - L. G.
Chat GPT is pretty bad at giving good references like this. Oh, absolutely.
30:59 - N. M.
It gives some like websites and some like published reports.
31:03 - L. G.
And then and sometimes they're not actually right, the references aren't right.
31:08 - D. B.
It'll just make up the references, yeah.
31:11 - L. G.
Well, it's like, yeah, not completely made up, but it may be the wrong year of the report, or it may be, you know, they got that it's like a secondary report, they reported that someone said it was there, but
31:27 - D. B.
it might not be correct. You know, Gemini, Google Gemini is, you know, it's, I've heard that it doesn't, It's not as good as the other AIs in a lot of ways, but actually it is pretty good at giving search results, because that's what Google does, right? That's their specialty. So it'll give, I mean, it may not give results like this, but it'll give links to websites and web pages, because it's like a certain part of a search engine. Yeah. Well, we can go ahead and try the same query for another, another AI, but let's just scroll down a little bit more and just see what else. So here's an abstract for this.
32:11 - Y. P.
The first link.
32:13 - D. B.
I guess we could, you know, it's linked to that. I don't know.
32:18 - Y. P.
One major difference that I would say between this and the other sources that we're talking about is I'm assuming that when you are doing research, the output that you're receiving is more authenticate. So there will be hopefully close to zero elucidation here. So I mean, that is a major plus point. I believe that a library resource would have versus other resources that we're talking about where, hey, if these are the sources that library is suggesting, I can trust in these sources versus if it comes in chatgpt etc. I mean that is a major difference that what I'm seeing right now versus what I would see otherwise and I think it was L. who mentioned on the referencing so one is chatgpt references sometimes if you ask for a reference that this is your source for this but the bigger problem of these other generative AIs, it does not reference at all what training data was used to create that intelligence to reference that. And that's a bigger thread because you don't know what the presumption, assumptions it has before creating that content. And this library content, I believe, and I don't know the back end of how it is generating all these things. So I would be curious to know what is the intelligence being used to even get this. But I'm assuming that that intelligence training perhaps would be more reliable than the other generative. So two pieces, one is what I'm seeing perhaps is more reliable. And also, I'm assuming before this was or this is off somebody has done that ethical and other checks before even allowing this to be used for library. So two thoughts on that. Is it making sense, D.?
34:36 - Unidentified Speaker
Yeah.
34:37 - E. G.
Y., I think that when you're asking for that type of detail, that's the secret sauce. That's the competitive advantage. Everybody knows how to do it. To train an AI. But the nuances associated to that training is what's going to allow a company to have a competitive advantage. And that's where they start. Right now, we're starting to see things like this, where it's maybe trained more to ensure that we had correct references that truly relate to the topic at hand that's tuned to that particular piece.
35:23 - Unidentified Speaker
Correct.
35:24 - E. G.
But A., there is two things, right?
35:28 - Y. P.
One is what I mentioned was the data. I'm not talking about the precise algorithm, transformers, the code that is being used over the data. And I completely agree with you that that particular intelligence. So there are two pieces. One is the code and the core IP that is being used to train the data. And then you have this data that Google has used for search engine and chat GPT is used for chat GPT or open as used for chat GPT. So the intelligence and the code and transformers and how each company is building those transformers and then agents and everything and the whole ecosystem infrastructure and platform. Yes, very, very, very unique. But perhaps, you know, whether it's Bing or Google for search engine, they are going after the same or similar set of data to make all that core component intelligent. So, and I'm talking about data where the illusion hallucination happens and I'm assuming that the library system has kind of done some background check if I can say that so that we can depend on what we are seeing here more so than other generative AI and we can speak offline I don't want to go into detail but and I might be wrong also by the way but I'm going with assumptions Well, this is showing mostly books and a
37:15 - D. B.
journal article, whereas Gemini would give web pages because that's what Google does.
37:25 - Unidentified Speaker
That gives you some more questions. Click on Additional Questions.
37:32 - D. B.
All right, well, I guess we can go to another AI and ask the same question, see what happens. So you were going to say, give me references, right? All right, so now I'm going to. Figure out how to navigate here.
38:04 - Unidentified Speaker
I'm going to chat GPT. So you're saying, please give references?
38:18 - N. M.
What model are you using? Come with different models?
38:26 - D. B.
Yeah, this is not the good model, you can only use it a couple times in a day, and then it won't let you do it anymore unless you pay. So and I already did it today. So this is not their most advanced model. It's one of the older ones.
38:43 - D. B.
So there's that, but let's see what it does.
38:48 - N. M.
There's also been an issue with it giving references and links to articles where it will give you a link that'll take you to one article, but it'll give you a completely different article once you click on it. All right, well, let's hear the reference.
39:12 - D. B.
All right, this is an old standard AI textbook that's been in print for many years. The older versions probably don't even mention, barely, you know, probably have one chapter on neural networks. Here's another book. I think it's a book. Papers.
39:36 - B. I.
I think with the library AI set, that one kind of points you directly to the specific paragraph or the specific part of the book or article is its reference in which I kind of like more as opposed to Chad
39:59 - D. B.
GBC. Yeah, I mean there's no links here.
40:03 - Unidentified Speaker
It's pointing to chapter two. But yeah, no links. Courses.
40:14 - D. B.
I guess if we went to Gemini, let's just go to Google. Am I right? If I type that query in here, it would access Gemini?
40:40 - B. I.
So you have to go to the nine dots at the corner, close to your name.
40:52 - E. G.
Upright. Upright. Upright corner. Square dots.
40:57 - Multiple Speakers
Here? Yeah. No. D.
40:59 - Y. P.
Next to D. Next to D circle. You see the green circle with D? Yeah.
41:07 - D. B.
Right next to it.
41:10 - Y. P.
Right next to it. Yes.
41:12 - D. B.
That was covered up by my Zoom.
41:16 - B. I.
And then you see Gemini.
41:19 - D. B.
Oh, OK. Gemini right here. OK. All right.
41:23 - B. I.
Alternatively, you can just Google search Gemini too.
41:28 - Unidentified Speaker
Yeah.
41:29 - Y. P.
Is that right? Shall I click it?
41:34 - D. B.
I wonder if you have to specify you need actual references, or will it?
41:43 - A. B.
Well, that gave you a quick answer.
41:47 - E. G.
No references either.
41:49 - D. B.
I thought it was generating something, and then it got away.
41:57 - A. B.
I'll try it.
41:58 - D. B.
So A., you're saying, oh, with academic references or something or references. OK.
42:08 - A. B.
So it started again, it started to do something and then it's like, no, nevermind.
42:17 - D. B.
Hey, I was checking.
42:20 - A. B.
I was, I've been kind of quiet playing with this Primo, it's actually really neat. Because I was curious, I took some of my research question, like the double machine learning and stuff like that, and I was playing around with it. It gives you the first five articles, but it's interesting. If you click on the View More Results, it actually takes whatever question you prompt it with. If you look up in the search header, basically like parses it and all the potential different ways like you see at the top search section there you know the ors and all that so it's it basically like takes whatever question you have and then it converts it into however you know many different combinations that it could potentially show up and puts you know you know spits it out in a bunch of or statements and then that's you know ultimately what pulls back the results and they summarize apparently it looks like they just summarized the five at the top.
43:22 - D. B.
So this is this idea of sort of automatic query augmentation. Exactly. I mean, that's been around for some time.
43:31 - A. B.
I haven't.
43:32 - D. B.
I didn't know it would do it. Yeah.
43:36 - A. B.
So it looks like Primo is just sitting on top, right? So it's basically taking your prompt and whatnot, converting it to a query to this database. Pulling back results, and then it looks like it's summarizing just purely what comes back, at least the top five.
43:56 - D. B.
Someone else asked the same question. I'd be really curious to know whether Primo is based on a back-end engine like the OpenAI, same as one of the major ones that we've heard of, or whether they made their own pre-trained?
44:15 - E. G.
Meta dot AI. Select the 400. And for the record, I think this is really cool.
44:28 - A. B.
I'm definitely gonna use it. Yeah, me too.
44:43 - N. M.
OK.
44:43 - D. B.
So wait, what was meta AI? What was the connection?
44:47 - Multiple Speakers
I used login in the lower left. I logged in with my Facebook account.
44:54 - D. B.
Oh, OK.
44:55 - E. G.
And the reason I did that, because you can select your model type. And at that point, I had selected the Lama 3.14 hundred and 5 billion. And I'm getting some really, really nice Output from that one hmm Answered output for what the same question Well, I haven't plugged in this question, but for other questions I've been And you've got there's another with llama 3.2 Just try the question in there and see what we get out of it.
45:41 - D. B.
Like that?
45:43 - Unidentified Speaker
Yes. Oh. Yeah.
45:46 - Y. P.
D., M. Z. wants to know who you are. So that's when you're born.
45:59 - Unidentified Speaker
OK.
46:00 - Y. P.
I'm 65. You want me to share my screen?
46:06 - E. G.
In a minute. Let's just see if this works.
46:10 - D. B.
Maybe not, but if it doesn't work, then.
46:14 - Unidentified Speaker
Or you want to share your screen? Yeah, we can do that. Here's what it's got.
46:25 - Y. P.
A., did you get better answer just because you're logged in?
46:41 - E. G.
Mine was different. I wouldn't say it's better. But I'm also forcing it to use the $405 billion.
46:56 - D. B.
This is getting the same stuff that that other one gave, right?
47:07 - N. M.
The same references.
47:09 - D. B.
What was the other one that we were using that we tried? ChatGPT. Huh? ChatGPT. ChatGPT. Yeah, OK. So this is giving the same answers as ChatGPT.
47:21 - Y. P.
Yeah, so if you remember, I was telling you the point of uniqueness. This is kind of partially proving what I was trying to say, that whether it's Bing or Google, it's going after the same data. And same thing. Like Meta and Chaijipati is going after the same data and it boils down to how they are going, you know, the process or the methods and infrastructure and other things that they have to get after the same data. It boils down to all that and how they are processing, how they are giving answers and what people in general can do with this on top of this will perhaps set them apart because they are going after the Same
48:07 - E. G.
data well, but let me go ahead and share my screen because I'm getting a different output So here I put in the prompt.
48:36 - D. B.
E., if you can increase the font size a little bit, I think it would help. Yeah, there we go.
48:44 - E. G.
provide sources and reference.
48:52 - Unidentified Speaker
That's pretty good. That's a pretty good list.
49:12 - E. G.
Other than validating all of the references that it has? I'm confident.
49:18 - Multiple Speakers
Now, a lot has to do with prompt engineering, too, by asking the prompt.
49:28 - N. M.
And here it goes through all of the references.
49:57 - D. B.
Makes sense
49:57 - D. B.
Makes sense.
49:58 - A. B.
But then to clarify, though, with this, the Primo application, right? It looks like from what I can see, it's, you know, it's converting your prompt into essentially a query. That hits the the library database. Pulls back, you know, and then parses and all sorts of potential queries and then pulls back the results. And then it looks like it's then summarizing those results and presenting it to you. It's not really trained.
50:29 - D. B.
It's generating, it's taking your question and turning it into a traditional type expanded, automatically expanded query, as opposed to, you know, ChatGPT or these other AIs, which don't do that. They don't convert your question into a query. They use the question as the query and use AI to figure out how to respond.
50:52 - A. B.
Exactly. So I think on the upside of this Primo thing, any new potential research that comes available, you're going to immediately get it as you hit the library database, for example, as opposed to LLM that would maybe need to be refreshed every so often or something like that.
51:13 - D. B.
OK, but Primo is also doing an AI-style summary. So it must have a subroutine call chat GPT or one of the others, I'm thinking.
51:26 - A. B.
Well, that's possible. I was thinking it was probably summarizing the information that's getting back from the queries in the database, but I'm just guessing.
52:34 - Unidentified Speaker
It's still chugging away.
52:39 - E. G.
And I've broken this before a few times. Mark Zuckerberg is going to come after you.
52:49 - Y. P.
Who's using all my processing power?
53:01 - Unidentified Speaker
OK, I'm just logging in.
53:04 - Multiple Speakers
I'm logging in. I used a tar out of it.
All right, well, we're kind of at the end.
53:16 - D. B.
We were successful, though. We broke AI.
53:20 - L. G.
We can start back.
53:22 - Multiple Speakers
If you consider yourself successful with that, just come to my house.
53:29 - E. G.
I do that two or three times a day.
53:33 - Y. P.
Yeah, same here.
53:36 - Multiple Speakers
Sorry, I'm going after OpenAI.
53:39 - Y. P.
I break them left and right. In fact, I was sharing my agents just work while we are sleeping. And that's why, E., I've sent you an email already. I'm trying to build better orchestration or perhaps share what we have built and etc. See, imagine some automated tools continuously doing this. That's what we are doing to open AI until they say or come back to us or stop us from doing it. Auto agents are nothing but doing this. They're continuously going towards perfection. Sorry, did I stop you from saying something?
54:35 - D. B.
No, no, no. I'm just looking at what's in the chat. Oh, yeah, sorry.
54:41 - A. B.
I was just that that was, they had some additional information on it.
54:41 - A. B.
And it does look like that's what it's just kind of an LLM sitting on top of the library database service. So OK. So it's not, I don't think the LLM really is trained on that information. It's just kind of, it appears to be just kind of parsing and creating queries and summarizing results as it gets it back.
55:06 - Multiple Speakers
Think of it like a Google appliance that used to sit on your own database for searching. Yeah.
55:12 - D. B.
Well, that makes an interesting application of where they take an AI, as we're familiar with, and embed it as a component of a different system, right?
55:22 - A. B.
Oh, yeah. No. That's interesting. Makes a lot of sense. All right.
55:29 - D. B.
So I guess we will meet again in a couple of weeks. And I'll look forward to seeing you all then.
55:40 - Y. P.
Happy holidays. Take care.
55:42 - A. B.
Bye, everyone.
55:43 - Multiple Speakers
Happy holidays.
55:44 - L. G.
Bye, everybody.
55:45 - Unidentified Speaker
Bye.
55:46 - Y. P.
you
