Friday, March 21, 2025

3/21/25: DD presentation

Artificial Intelligence Study Group

 Welcome! We meet from 4:00-4:45 p.m. Central Time. Anyone can join. Feel free to attend any or all sessions, or ask to be removed from the invite list as we have no wish to send unneeded emails of which we all certainly get too many. 
Contacts: jdberleant@ualr.edu and mgmilanova@ualr.edu
Agenda & Minutes  (155th meeting, March 21, 2025)

Table of Contents
* Agenda and minutes
* Appendix 1: Syllabus of new proposed 4000/5000 level applied AI course
* Appendix 2: Transcript (when available)

Agenda and minutes
 The meeting ended here. 
    • Fri. April 18 at 3:00 p.m. (an hour earlier than our usual meeting time!) GS PhD defense, Optimizing Small AI Models for Biomedical Tasks Through Efficient Knowledge Transfer from Large Domain Models.
    • Fri. April 25: YP will informally present his draft AI course outline and welcomes comment. See Appendix 1 below.
    • TE is in the informal campus faculty AI discussion group.
    • News: SL writes: "I’m excited to share that I’ve been asked to lead the DCSTEM College AI Ad Hoc Committee. A call for department representatives went out a few months ago, but since so much time has passed, we're going to start fresh. Nick S[...] and I will be co-leading this initiative on behalf of the DCSTEM College.

      This committee will bring together faculty to collaborate on AI initiatives in teaching, learning, and research. Ideally, each department will have 1–2 representatives who are familiar with their department’s instructors and willing to serve as a liaison between them and the committee. These representatives will help share knowledge, facilitate discussions, and explore AI’s impact on our curriculum.

      If you’re interested, please join our first informational/organizational meeting on Tuesday, March 18, at 3:00 p.m. on Zoom. Formal membership can be adjusted after this meeting.

      Sign up here: [Google Form]
      Once you submit the form, you’ll receive a calendar invite with the Zoom link.

      We’ll discuss AI’s role in our curriculum, how to integrate AI literacy into courses, and strategies for guiding students on responsible AI use. Your input is invaluable, and we look forward to working together to shape AI’s role in our college.

      Hope to see you there!

      Sandra & Nick

  • Recall the masters project that some students are doing and need our suggestions about:
    1. Suppose a generative AI like ChatGPT or Claude.ai was used to write a book or content-focused website about a simply stated task, like "how to scramble an egg," "how to plant and care for a persimmon tree," "how to check and change the oil in your car," or any other question like that. Interact with an AI to collaboratively write a book or an informationally near-equivalent website about it!
      • ET: Growing vegetables from seeds. (2/21/25)
        • Working on proposal (+ project)
        • Making AI images for the book (inc. cover)
          • Getting better supported! You can use verbal commands to modify its pictures. Prompts are a challenge.
          • TOC is together. (We're back to the book goal!)
        • Found an online course on prompt engineering
        • Gemini: writes well compared to ChatGPT
  • We finished the Chapter 6 video, https://www.youtube.com/watch?v=eMlx5fFNoYc. We decided to run through this one again.
  • Schedule back burner "when possible" items:
    • If anyone else has a project they would like to help supervise, let me know.
    • (2/14/25) An ad hoc group is forming on campus for people to discuss AI and teaching of diverse subjects by ES. It would be interesting to hear from someone in that group at some point to see what people are thinking and doing regarding AIs and their teaching activities.
    • The campus has assigned a group to participate in the AAC&U AI Institute's activity "AI Pedagogy in the Curriculum." IU is on it and may be able to provide updates now and then.
  • Here is the latest on future readings and viewings

Appendix 1: New proposed 4000/5000 level applied AI course

In today's AI-driven world, professionals across all levels—graduate, undergraduate, and PhD students—must develop a comprehensive understanding of AI technologies, business applications, and governance frameworks to remain competitive. The Applied AI for Functional Leaders course is designed to bridge the gap between AI innovation and responsible implementation, equipping students with technical skills in AI development, strategic business insights, and expertise in governance, compliance, and risk management.

 

With industries increasingly relying on AI for decision-making, automation, and innovation, graduates with AI proficiency are in high demand across finance, healthcare, retail, cybersecurity, and beyond. This course offers hands-on training with real-world AI tools (Azure AI, ChatGPT, LangChain, TensorFlow), enabling students to develop AI solutions while understanding the ethical and regulatory landscape (NIST AI Risk Framework, EU AI Act).

 

Why This Course Matters for Students:

 

■v Future-Proof Career Skills – Gain expertise in AI, ML, and Generative AI to stay relevant in a rapidly evolving job market.

■v Business & Strategy Integration – Learn how to apply AI for business growth, decision- making, and competitive advantage.

v■ Governance & Ethics – Understand AI regulations, ethical AI implementation, and risk management frameworks.

v■ Hands-on Experience – Work on real-world AI projects using top industry tools (Azure AI, ChatGPT, Python, LangChain).

Why UALR Should Adopt This Course Now:

 

v■ Industry Demand – AI-skilled professionals are a necessity across sectors, and universities must adapt their curricula.

v■ Cutting-Edge Curriculum – A balanced mix of technology, business strategy, and governance makes this course unique.

v■ Reputation & Enrollment Growth – Offering a governance-focused AI course positions UALR as a leader in AI education.

v■ Cross-Disciplinary Impact – AI knowledge benefits students in business, healthcare, finance, cybersecurity, and STEM fields.

By implementing this course, UALR can produce graduates ready to lead in the AI era, making them highly sought after by top employers while ensuring AI is developed and used responsibly and ethically in business and society.

Applied AI (6 + 8 Weeks Course, 2 Hours/Week)

5-month Applied Artificial Intelligence course outline tailored for techno-functional, functional or technical leaders, integrating technical foundations, business use cases, and governance frameworks.

 

This can be split in 6 weeks certification plus additional funds for credit course with actual use case.

 

I have also leveraged insights from leading universities such as Purdue’s Applied Generative AI Specialization and UT Austin’s AI & ML Executive Program.


 

Balance: 1/3 Technology | 1/3 Business Use Cases | 1/3 Governance, Compliance & AI Resistance


 

Module 1: Foundations of AI and Business Alignment (Weeks 1-4)

 

v■ Technology: AI fundamentals, Machine Learning, Deep Learning

v■ Business: Industry Use Cases, AI for Competitive Advantage

■v Governance: AI Frameworks, Risk Management, Compliance

 

·         Week 1: Introduction to AI for Business and Leadership

o    Overview of AI capabilities (ML, DL, Generative AI)

o    Business impact: AI-driven innovation in finance, healthcare, and retail

o    Introduction to AI governance frameworks (NIST, EU AI Act)

·         Week 2: AI Lifecycle and Implementation Strategy

o    AI model development, deployment, and monitoring

o    Case study: AI adoption in enterprise settings

o    AI governance structures and risk mitigation strategies

·         Week 3: Key AI Technologies and Tools

o    Supervised vs. Unsupervised Learning

o    Python, Jupyter Notebooks, and cloud-based AI tools (Azure AI Studio, AWS SageMaker)

o    Governance focus: AI compliance and regulatory challenges

·         Week 4: AI for Business Growth and Market Leadership

o    AI-driven automation and decision-making

o    Case study: AI-powered business analysis and forecasting

o    Compliance focus: Ethical AI and responsible AI adoption



 

■v Technology: NLP, Computer Vision, Reinforcement Learning

v■ Business: AI in business functions - Marketing, HR, Finance

■v Governance: Bias Mitigation, Explainability, AI Trust

 

·         Week 5: Natural Language Processing (NLP) & AI in Customer Experience

o    Sentiment analysis, text classification, and chatbots

o    Business case: AI in customer service (chatbots, virtual assistants)

o    Governance focus: Privacy and data security concerns (GDPR, CCPA)

·         Week 6: AI for Operational Efficiency

o    Business use cases: AI for fraud detection, surveillance, manufacturing automation

o    Compliance focus: AI security and adversarial attacks

·         Week 7: Reinforcement Learning & AI in Decision-Making

o    Autonomous systems, robotics, and self-learning models

o    Business case: AI-driven investment strategies and risk assessment

o    Resistance focus: Overcoming corporate fear of AI adoption

·         Week 8: AI in Marketing, HR, and Business Optimization

o    AI-driven personalization, recommendation engines

o    Business case: AI in recruitment, talent management

o    Compliance focus: AI bias mitigation and fairness in hiring


 

Module 3: AI Governance, Compliance & Ethics (Weeks 7-10)

 

v■ Technology: Secure AI Systems, Explainability

■v Business: Regulatory Compliance, AI Risk Management

■v Governance: Responsible AI, Transparency, Algorithm Audits

 

·         Week 9: AI Governance Frameworks & Global Regulations

o    NIST AI Risk Management, ISO/IEC 23894, EU AI Act

o    Industry-specific regulations (HIPAA for healthcare AI, SEC for AI in finance)

o    AI governance tools (audit logs, explainability reports)

·         Week 10: AI Explainability & Bias Management

o    Interpretable AI techniques

o    Case study: Bias in AI hiring systems and credit risk models

o    Business responsibility in AI model transparency

·         Week 11: AI Security, Privacy, and Risk Management

o    Secure AI model deployment strategies

o    Governance: AI trust frameworks (eg: IBM AI Fairness 360)

o    Case study: Managing AI risks in cloud-based solutions

·         Week 12: AI Resistance and Corporate Change Management

o    Strategies for AI adoption in enterprises

o    Business case: AI integration in legacy systems

o    Ethics: Impact of AI on jobs, social responsibility, and legal liabilities


 

Module 4: AI Strategy, Implementation, and Future Trends (Weeks 11-12)

 

■v Technology: AI Product Development

■v Business: AI Implementation, Enterprise AI Strategy

v■ Governance: AI Regulatory Compliance & Future Legislation

 

·         Week 13: Overview of AI Deployment and Scalability

o    Deploying AI models on cloud (Azure AI Studio, AWS, GCP)

o    Business case: Scaling AI solutions in enterprise environments

o    Compliance: AI model monitoring, drift detection

·         Week 14: AI for Competitive Advantage & Industry-Specific Applications

o    AI in industry : e.g.: supply chain, autonomous vehicles, healthcare diagnostics

o    Case study: e.g.: AI-driven drug discovery and logistics optimization

o    Compliance: AI liability and regulatory accountability

·         Week 15: AI Governance and Responsible Innovation

o    Innovating with AI : e.g. financial services (algorithmic trading, fraud detection)

o    Ethics: Ensuring fairness and avoiding discrimination in AI models

o    Risk assessment frameworks for enterprise AI adoption

·         Week 16: The Future of AI: Trends, Risks & Opportunities

o    Generative AI (DALL-E, ChatGPT, LangChain applications)

o    AI and Web3, decentralized AI governance

o    Case study: AI-powered governance in blockchain ecosystems


 

Module 5: Capstone Project & Final Presentations (Weeks 12-14. Process starts in Week 7/8)

 

■v Technology: Hands-on AI Application Development

v■ Business: AI Use Case in Industry

■v Governance: Compliance Strategy & Ethical AI

 

·         Weeks 17-19: AI Capstone Project

o    Develop an AI-driven business solution with governance compliance

o    AI application areas: Business analytics, customer engagement, fraud detection

o    Report: Governance strategy and AI risk mitigation plan

·         Week 20: Final Project Presentations & Certification

o    Peer review and feedback

o    Industry guest panel discussion on AI’s role in future business strategies

o    Course completion certification

Tools & Technologies Covered:

·         AI Development: Python, TensorFlow, PyTorch, Scikit-learn, GenAI models

·         Cloud AI Platforms: Azure AI Studio, AWS AI Services, GCP Vertex AI

·         NLP & Generative AI: ChatGPT, DALL-E, LangChain, BERT, Stable Diffusion

·         AI Governance & Risk: SHAP, LIME, AI fairness toolkits

 

Appendix 2: Transcript

AI Discussion Group

0:16 - M. M.

Oh, D., you're good. I say I'm excited to see the presentation, but I want more people.

0:27 - D. B.

Well, we got another minute.

0:32 - D. D.

Well, it's spring break. You got the important people.

0:37 - D. B.

How about that?

0:38 - Multiple Speakers

I'm not going to draw a crowd.

0:42 - M. M.

Yeah, if you need people, I will invite more.

0:46 - D. D.

But it'll be nice if there's anybody interested in it.

0:52 - M. M.

No, it's very interesting, by the way.

0:55 - Multiple Speakers

I like it. Yeah. And I receive invitation, I am guest editor of one journal that will accept our extended version of the papers that we present for the workshop.

1:11 - M. M.

So I just didn't have time, D., to thank you the invitation. That is a free open access journal. Otherwise, you have to pay $2,000.

1:25 - Unidentified Speaker

Yeah.

1:27 - D. D.

So. Send it.

1:28 - Multiple Speakers

As soon as you can, I'll look at it for sure.

1:34 - M. M.

Yeah, yeah, yeah, yeah. So it's very interesting topic. They select a few of you, not everybody is selected. Yeah. Let's wait a little bit.

1:47 - D. B.

It's 4.01, we can, we'll start at 4.03. How does that sound?

1:57 - D. D.

Now we get some people here.

2:05 - Unidentified Speaker

Yeah. So R. here and J.

2:18 - Unidentified Speaker

More girls are joining.

2:23 - M. M.

This is good. We have smart girls. Did you watch the keynote speech of NVIDIA? The media was very, very interesting.

2:49 - D. D.

I didn't see it.

2:51 - M. M.

No, you have to see the recording.

2:55 - D. D.

Okay.

2:55 - M. M.

They advertise everywhere, so it was very, very, very good.

3:01 - D. B.

I can post the link. I had it.

3:06 - M. M.

You have it? I don't know.

3:09 - D. D.

They're selling graphics cards.

3:11 - Unidentified Speaker

Oh.

3:11 - M. M.

No. Is collaborating with all kinds of companies. Every company is integrated right now. Do you want me to send the link?

3:21 - D. B.

Yeah, if you send the link or put it in the chat, I'll post it in the minutes, if anyone reads the minutes.

3:29 - M. M.

Yeah, yeah, good. I'm going to go ahead and start.

3:33 - D. D.

Yeah, I think it's 4.03.

3:35 - D. B.

I think we've got a decent group I think you have to stop sharing. Let me stop sharing and you can share yours.

4:05 - D. D.

Parsing requirements for automatic prompting of large language models for requirements validation. Dr. M. is my mentor, advisor, I'm not sure, seems to be a little ambiguity in that, but she's my boss at the university. All right, and what is all this? Well, this research's ultimate goal, the whole point that people get into this is because we have this dream that we're going to be able to step on the bridge of the enterprise and say, make me a program that does X, Y, Z and it'll do it. But we want full automation, right? So that we can basically get all of our work done. We just come up with what we want and then the machine gives it to us. But the variables are huge. So everybody has their own different way of writing requirements. They do storytelling, descriptions, you know, people just do use cases and don't even really specify There's different kinds of apps are needed, different kinds of platforms.

5:37 - Unidentified Speaker

Not everybody wants an app that's good everywhere.

5:43 - D. D.

Sometimes they just want it specifically for this version of Linux or just for Apple phones.

5:57 - Unidentified Speaker

Then, you know, we're dealing with English. Most of the time, I guess it is possible to use other languages, but...

6:05 - D. D.

And language is very ambiguous, so there's a lot of things that can be communicated that can be problematic when automatically parsing it, right? And then we need some kind of a way to make sure that it works right and does what it's supposed to do.

6:26 - Unidentified Speaker

So right now we're just going for semi-automatic solutions. And oops, see if I can go back.

6:36 - D. D.

Right, and so in this research that I'm doing, I'm gonna work on class modeling and core requirements validation. So I just drew the line. Going back. It goes back a long way. But people develop natural language toolkits. Just ways to parse text. There's a lot of them out there. Spacey, Stanford's got a few of them. There was Lolita. There's a lot of different you know, natural language toolkits. They started with, you know, basic classifications. They were just learning algorithms where they classify. You would take the text and they would classify it and try to label it with parts of speech or, you know, draw phrases. I think the parts of speech is mostly just tagging it. There's these syntactic trees, the phrase levels, and all these things in these engines. And the last time I looked, and it's been two years ago, neural networks were at the top of the learning models for natural language toolkits. They were the most used for the modern natural language toolkits, they're using some kind of neural network. Things that they would do, they would build classifiers, build rules, you know, so okay, this needs to be a class, this needs to be a function of the class, or a method of the class, And if there's domain, if it was domain specific, you know, you might have domain words that you know have a particular meaning and you would just include that in a glossary and just match it. 2009, plant UML, this is kind of a markup code for UML if you're not familiar with UML that's just pictures. It's just a graphic. Pie and Source, that's what I'm using. It's a free application that draws UML from Python. Now moving ahead, this technology basically got integrated into everything that is machine and text, right? Your IDEs, you know, they look at your words. They parse this in some way. Your Grammarly, document management systems, all this old tech is still out there, right? It's still being used. It got integrated into all kinds of apps, modern day apps. It's everywhere. Spellcheck, you know, these things are all part of these technologies. But they started making bots and bots is kind of a you know what we have now what we call a bot what we used to call a bot a bot really is kind of like an automated script it just kind of runs and takes care of stuff whenever maybe a listener you know just sits quietly and listens and waits for something to pop up that it's supposed to handle or you know just handle routine tasks And then we got large language models that actually understand what we mean. Big, giant leap forward. They're collaborative. You can go back and forth with them. I have had a terrible cold all week, guys, and I will dry out really quick.

11:03 - D. D.

And so these, they're generative, and that can do a lot. But there are some drawbacks. First, can we just trust them to do the job? Can it hallucinate? Will it introduce a bug? And if we trust it too much, are we setting ourselves to have a bug later that's really going to cost us. So there needs to be some checks and balances. And so in this research, what I'm doing is I'm taking the old technology, I'm using it to get started, to generate things, to check, to keep the large language model in check, and then I'm using controlled generated prompts to process these specification documents, just kind of to step further into the future of this type of research. So basically this is just, I'm going to make algorithms generate some classes, generate some scripts, generate the ML. I'm going to use the Stanford, this Stanza by Stanford toolkit. It's free. This particular one is for Python, and that's why I selected it. And Stanford's trusted. They document everything. They write research papers. It's a really good tool. Kit. I'm using ChatGPT, that's kind of the... When I got started it was kind of the leader. I don't know if it still is. And then this pie and source that I'm using, I'm using it because it's fast and easy. I think going more to generating the actual code for the UML would be better, but for this, for this, I just, I got something quick and easy to, to just top it off. So just go over this again. I'm going to take the requirements. I'm going to stick it through natural, or yeah, a natural language processing toolkit. After it gets through with the toolkit, I'm going to stick my, I'm going to run it through the algorithms. I'm going to tag it with prompts and generate some lists. I'm going to prompt a large language model for Python classes. And then I'm going to take those classes, generate prompts. And then I'm going to go back to another, the same large language model, but a different bot, so to speak. With a different system message.

14:11 - D. B.

And I'm going to make these scripts.

14:15 - D. D.

I'm going to check it, unfortunately, manually against my checklist. And then I'm going to run it through PySource, generate the model, and then do a final validation and evaluation. This is graphically it. So anybody that needs to see that graphically, that's what it looks like, It's a little zigzaggy there. Wanted it big.

14:47 - Unidentified Speaker

There's a little bit of pre-processing where I chose to split the sentences and things like that.

14:57 - D. D.

Everybody good? And so here's, This is the specification. It's not a giant monster, but it's a small specification, but it can act like the core system of this library. So I stick it in the natural language toolkit, run it through. I have algorithms meant to deal specifically. See how this is the first sentence. I don't know how well you can see it. It's really good on my screen. But you can see that there's a colon there, and this entire list really matches that first sentence, right? It's really part of that first, it's all associated with that first sentence, all the way down till we get past number five, and then there's kind of a transition, an unknown transition off of the list. So it actually set of problems that are unique to the way this specification was written, which I'm not going to go into that right now. It will bore you to death. But it'll be pages in my dissertation if you are interested. So we get everything back to Toolkit. It looks something like this. I chose this tree structure here from the root down just to kind of show you some of the information comes out of this toolkit, the sentence level, the phrase level, and then the word level. And each word, you know, has its own tag associated. That's a parts of speech tag. And so all this information, when you design rules and algorithms, you might look for patterns in these sentences. In fact, you know, I don't know how many papers have been written about it, but they all use some variation of sifting through this and pulling out the information that they want. And then when this next box here, after it goes to that first algorithm, you'll get information like this. That's just kind of what I sifted out of it. And then I just run it through. Add the prompts, I'm telling this first bot, this is what I want you to make. This is what I want you to do. So this right here goes directly to the large language model. It's not all of it, it's just a snippet of it. It's basically giving it specific instructions. I'm gonna move forward and Okay, so this is bot one, let's just call it that. This bot here, this is the system prompt. This is the only thing that I want this large language model to focus on. And it's just gonna make classes. It's not supposed to do anything spectacular. The settings are published in the paper, the actual settings of the large language model in ChatGPT. So there was a discussion where we talked about its creativity meter. I forgot exactly what we called that. It's been a minute. But this is turned all the way down to zero. And this bot right here is perfectly consistent. It gives the same result 10 out of 10 times. This thing has no variation, not an extra space is given. It is what I call the best bot ever. It's like an algorithm.

18:53 - Multiple Speakers

Oh, temperature. Huh? Temperature. Yeah, the temperature.

18:56 - D. D.

Thank you, E. That's right. It's the temperature set on zero. There's no play here. Now, I get back exactly what I asked for every time. I mean, I don't know if I did it hundred times maybe, I just didn't want to look through a hundred times, so I just did it ten. But I get this, this is just a little snippet of what I get, it's every single possible class, you know, other research would, and they didn't have a large language model to lean on, so they had to try to pick which classes would be a class, which ones would be a property, I don't have to do that because the large model can make sense of it. So this is my second prompt. It's where I say I want to build a validation script. I want you to do everything. To me, it is all high variability. And I'll share with you how I checked it.

20:06 - D. D.

Oh, excuse me, guys.

20:08 - Unidentified Speaker

Same settings, though. I'm choking this thing down. I don't want it to do not even be creative.

20:19 - D. D.

Just do exactly what I want. And that gives me some usable Python code. And sometimes it actually works. It's never been 100% accurate. Not once, but it does work sometimes. Sometimes it actually will run. I've asked it to, you know, I think it had, I think I need to break it down and send it to more bots to get better. Right. I mean, I, I got two bots, but probably I really need 10, you know, and hopefully I'll get to work on that, um, this year before I get, after I get done with the literature review, this, uh, I take that script. After I check, I go through the list, I manually check and do everything. And then I build this, I send that script to pi and source, and I get back a UML. This is what UML looks like, So it's just, this helps a lot.

21:29 - Unidentified Speaker

helps people understand what the application does.

21:32 - D. D.

And the whole point of this semi-automatic process up to this point is not to build any application, but to make sure that when you start building it, you're starting from a good, solid point that's been checked. You know exactly what you want to build, and you can build it. There's different software methods. So my favorite, of course, is the waterfall where you get it right the first time and you go till it's over. But that doesn't always work in real life, right? It'd be nice if it did, but it doesn't always work. So there needs to be some extra checks and balances in there. So what does it mean to validate? I mean, really, what does it mean? You know, E. says it's good, Is it done? I mean, that's how it works.

22:29 - Unidentified Speaker

So we come up with mathematical models, things that we can say.

22:34 - D. D.

Well, not only do I say it, but I've got, I wrote out this beautiful math here that states that it works. And that's called a formal method. Well, this could be considered a formal method. I don't even want to say it's a formal method. But it does, Python is a closed language. It only has so many ways you can write it up. And so in a way it is kind of formal, but it's very informal. The AI can write Python all day long. Most novice coders can understand Python very quickly. And if I'm not mistaken, I think some of the other disciplines are actually giving people a Python class, and letting them learn Python just without any of the computer science. If they don't have any background, they're going to hire me to be their tutor, but a lot of people are able to grasp Python pretty quick. So this works really good. A lot of stuff that's manual, I think, can be automatic. I think if I gave And one afternoon, he could get that AI doing all this stuff automatically. But I'm not him. So there's a paper that I just found the other day. It's very exciting. I mean, there's papers coming out all the time. It's very exciting what they're doing. If you can get the large language model to execute it, and test it and you could get a validation, especially against the checklist that I'm generating off the algorithms. I could pull that together and maybe even validate this thing 100% automatic. But we'll see. So accuracy is kind of a hard thing to pin down. If somebody says it's right, then does that mean it's right? The specification that I got, there was a bunch of people that actually did this called the 12 Papers of the Library Problem, I think. They all did their own thing. They all had different choices. They all made different decisions. You know, are they all right? Who knows, right? So what I want to check was variability. Like I said, my first bot, it has zero variability. When I tell it to do something, it does that every time. It is the best bot ever. The main bot, it has a pretty high variability and usability and variations. So you can see here. The first letter, the first number in the variability column, that's how variable it is on a scale of one to four. So if it's four, it was way different. Now, what I did was I did the first controlled test. I just ran it. This is what it is. Right or wrong, this is what it made, right? And then I did it 10 more times and I compared against that first test and see how much different it was. And then I documented the differences in here, and then I made a judgment call. If it seemed like a lot of variation, as you can look at number five, if you can see that, I said that was highly variable. It made an extra four methods and added two properties. It hallucinated a couple of times. Sometimes it didn't hallucinate. And so that gave us some variability and probabilities and methods. There wasn't serious hallucinations, as you'll see here in a minute. And then the usability here, how easy was it for me to make it work, to get this Python to run in an acceptable manner that I could say that it that I could make this app work. Now, there's some drawbacks. There's some things in the specification that are missing that, you know, there's no unique identifiers for any of the copies of the books. It's littered with ambiguity. So the argument, would it ever validate? It really technically shouldn't be validated without finishing some of the very important structures problems in the library but so I work I think I spent 15 minutes on a couple of these most of them I got working pretty quick and I'm not a complete novice though working with small scripts so you know I would think that a you know someone that was a professional coder that you know that's their nine-to-five I think they could do much faster than me. Some of this came out. If you look at number 10, that was my best one. I told it to add constraints every time, but it just doesn't. It just says I give up, punts it. One time it actually did, which I thought was really nice. So one, that's the highest rate. On usability, that thing came out ready to roll. It wasn't 100%. I still had to make a few changes, but it wasn't anything major. But you can say that, you know, there's some things, there's some work here that still can be done.

28:49 - Unidentified Speaker

So, hallucinations.

28:51 - D. D.

I got a couple of, I want to add a title to the book, but the specification doesn't say there's a title to a book. Now, you and I, everybody knows there's a title to every book. You know, and so does that large language model. And even if I tell it not to put it in there, it will put it in there. So, and the user wanted to give a name. At some point, Some of them gave IDs, but there is no username specified. Just there are two types of users is all that's specified. And so, I mean, are these really bad hallucinations? I mean, I don't know. I mean, any hallucination is a bad hallucination. You want things to do what you tell it to do, but these are certainly understandable hallucinations. If I was working with another person and they added title to the book because they just felt like it had to be there. Could you really blame them? No, you couldn't blame a person for that. Would you blame a person for saying every user has to have a name? No, you couldn't blame them for that. But I had a check against and I was able to catch it with my checklist. And I think the large language model was, you know, it's adequate. It's very good. It's very nice. It's like having a helper, you know, that understands what you want.

30:24 - A. B.

Can we just ask questions along the way?

30:27 - D. D.

I don't care. I mean, I have a place for questions.

30:32 - A. B.

Yeah, we're going to wait. That's fine.

30:34 - D. D.

Okay. So I think it was kind of effective for semi-automation. I got some good class models. They weren't the best. I put the best in the paper, but they all weren't pretty like that. But some of them took some work to get them to look good. And there's some things that could still be done better. Obviously, more automation. The goal is we want to be able to just tell that computer what we want, preferably voice commanded. Like when I talked to my assistant over here, you know, that's what we want, but we're still a ways from that. So are there any questions, A.?

31:21 - A. B.

Yeah, well, no, it's really interesting stuff. I was curious in your results, like as you're evaluating, I guess, would the idea be to, you know, adjust temperature, right? Go up, you know, maybe go up in the temperature, right? See if how that affects your results or even test against different large language models.

31:44 - D. D.

You know, that I would never ever dream of making the temperature go up, but maybe I should try. Yeah, I was just a curious thought, yeah.

31:55 - A. B.

Yeah, and I know Dr.

31:56 - D. D.

M., all the things that it, you know, she's made it clear to me that it needs to be used with any large language model. And so yes, other large language models need to be tested, vigorous testing. And maybe I'd say it, but I think you're right. I think adjusting that temperature, seeing what it does with seeing how, seeing if it can do better with more temperature.

32:24 - A. B.

Yeah, that's kind of what I was thinking. Imagine you have this grid or whatnot usability, but then you tested it amongst different LLMs and then maybe you had different temperature thresholds and whatnot that you tested. And some models don't have temperature settings.

32:43 - D. D.

True, true, yeah. And they're always going to have a temperature of probably one or something, right?

32:51 - E. G.

Anybody else? Hey D., you're old enough to remember 4GLs. Would this be a modern modern take on for GLS.

33:01 - D. D.

I don't understand what you're saying.

33:06 - E. G.

Maybe for for GL fourth generation language back in the 90s. Look old.

33:16 - D. D.

I'm not really old. It was one.

33:22 - E. G.

I think the very the very first sign-on was a 4GL language. What you do is you would, in human terms, describe what you want, and it would actually produce the code. Let me put a link. Yeah. I remember this. Definitely.

33:43 - D. D.

I want to look at that. Yeah. That sounds wonderful.

33:48 - M. M.

I remember this. It was good, actually.

33:51 - D. D.

I might integrate. Something like that.

33:54 - M. M.

It might help me generate prompts. E., what's happened with this language? I remember that I study and I like it.

34:05 - E. G.

The problem was the language was easily readable by the machine, but not by a user to maintain it. So it fell out of favor. It would use, when creating variable names, It would use random letters and numbers that had no sensical relationship to the data it was storing. The functions were the same way.

34:36 - M. M.

So, no embedding. They cannot, yeah.

34:40 - E. G.

Because one of the things about programming, and Dr. B., pipe in here. When you develop a program, you're not developing it just for the requirements or the specifications that you're creating it for. You're developing it so that way somebody coming up behind you understands the process that you're going through.

35:12 - D. B.

There's this term, self-documenting code. Kind of consistent with that.

35:19 - Unidentified Speaker

I remember this. Self-documenting code.

35:23 - M. M.

Well, that's the term extreme programming means that I don't document my code.

35:33 - Unidentified Speaker

The code is the documentation.

35:37 - E. G.

But if anybody's ever written APL and tried to read APL, forget it. Is it easier to read the code?

35:49 - D. D.

Well, no.

35:50 - E. G.

It's easier to rewrite it to a specification and read the code.

35:56 - D. D.

Yeah. I like the idea of being able to write code without documentation. But once you get 1,000 lines or so, you're going to hate your life.

36:11 - E. G.

You're going to want to break that up.

36:14 - D. D.

You're going to want documentation. You're going to need it. And documentation is so important. And when people try to break away from it, it probably works if it's a little bitty small thing.

36:28 - E. G.

I used to say that writing an application to a specification and walking on water are easy. When they're frozen.

36:38 - D. D.

Yeah, they're not moving, right? When they're frozen, that's right.

36:44 - M. M.

I have a question. Because you mentioned bots, you're using bots. People right now, they will like the idea of agents. Do you consider thinking in this way or you will make it more complicated? If you say that actually you're using agents, your bots are doing different job and serving... It is, it is. It is. So people will say why you're not using the concept of agents instead of bots? Or do you think this will complicate a lot of the things?

37:27 - D. D.

I think people can handle it. I think people could handle it. A bot, an agent, a particular system prompt. Yeah, it is an agent.

37:40 - M. M.

It's a- It is. Yeah, it is.

37:44 - D. D.

And today- Bots are really, their first use was really, they're not really a learner. They don't really comprehend any depth. But I think that they start I think you started calling them bots originally on, what's that, P.? Didn't they start calling them bots, build your own bots?

38:10 - M. M.

Is that on P.? No? Maybe the people will ask you the difference, you know, and there is a difference. But I like, today I was, when another student present the talk about automatically generating generate the LaTeX template. So you automatically generate also this, your UML, the same. Yeah, the same. That's nice. Anytime you can save modeling.

38:40 - Multiple Speakers

I mean, I guess some people probably love modeling.

38:45 - D. D.

I mean, I don't want to. Some people probably think that's the best Oh, I should look at models. I don't like to make them.

39:01 - M. M.

They're kind of tedious. I'd rather write code.

39:05 - D. D.

The new papers, can you compare a little bit with your- I don't know if anybody's- I know that there are a lot of papers about generating models. These are text. Excuse me.

39:28 - D. D.

Usually these are, you know, just a formatted text, right? And they will call that a model. It's basically like a script that you can follow when you actually draw the model. That's why I like this plant UML. What this plant UML does, is it actually starts the drawing of the model. It gives you something to work with. Now, in this particular application, you can actually pay and then you can draw a full UML when you're done or whenever you're ready. If you wanted to add stuff, this is just the free, add this script and it gives it back to you. But, uh, yeah.

40:36 - D. B.

Do you remind us what the, what particular UML formalism this is?

40:42 - D. D.

Is that what you're asking me? Yeah.

40:50 - D. D.

Let's see here. I put plant UML in here. Maybe I didn't.

40:56 - M. M.

It's called plant UML. Maybe I neglected.

40:59 - D. D.

It was in there. I saw it. There it is.

41:04 - D. B.

Well, there's pie and source.

41:07 - D. D.

That's the pie and source. Oh, there it is.

41:11 - D. B.

OK, I see.

41:13 - D. D.

Yeah. So plant to UML. Came out in 2009. So are you aware that if you were to take Java, for instance, and go into NetBeans, that you can get your models generated from there, from your code?

41:30 - M. M.

So let's plant UML or work on Java.

41:34 - D. D.

And so the only real difference is this PySource has said, OK, well, if it works in Java, I'm going to make it work. And with Python. So that's the only difference. If I had this, if I did this in Java, it would be much easier to get the UML.

41:56 - A. B.

So I use a, it's called Mermaid Chart, and they have a bunch of different, it's basically like you can do Gantt's or UML or, you know, what you name it. Name it, right? ERDs, all that stuff, right? But I've been using it, so I'll take my code and then get it to, and then ChatGBT or whatnot can output, like Mermaid, the scripting, it's got its own kind of like scripting, you know, language or whatnot, and you can output it and basically get your code and convert it to whatever, you know, format you need it. They actually have some AI things built into it as well, but it's very interesting, because it's kind of like an application of this, where you could take- But I'll share a link to that.

42:38 - Multiple Speakers

Yeah, I'll ping it in the chat.

42:42 - A. B.

Yeah, I haven't heard of it.

42:44 - D. D.

Like if I wanna make a model or something, I go to, I can't even think of what it is right now, but it was the one that we learned about in school. Lucid chart. Yeah, lucid, yeah. Yeah, lucid chart, that's the, That's the one, what's the man's name? He does Cosmo, Dr. B., can anybody think of his name? Yes, he does the Cosmo, sends us a newsletter now, it's pretty nice. He was teaching software engineering and had us do some things a chart I thought was really helpful. You're able to find out. Modeling really helps validate. That's why it's so important. You can see mistakes that you made and just stopping at a class model is not enough. I've read quite a few papers and the theme is most everybody uses the class model but after that it kind of falls off really fast. If you had the bottles and you could generate them, people would use them more. I have a friend, he's an electrical engineer. Oh, this is so hard. He designs circuit boards and then has them manufactured write code for the circuit board. I asked him to mate with me and show me what their engineering process was. He showed me that their diagram was all excuse me just a second please they they had a drawing they literally drew their diagram in a like a palette in word I had one drawing that was all their modeling and then I went through and I just made a bunch of like modeling lists you know and that's why that's why the I think the library problem is so powerful because it's that list format it people tend to go to you know these types of writing to kind of format things together it's like well this does this so let's make a list right and so I've been out of process those lists I thought was pretty important and that's why I built my algorithm specifically for this library problem with these specific types of list problems because they're porting out there's been other research that's looked into these lists and done different things but I've I'm reasonably certain I'm the only one that's done this for the library problem and it's Like I was saying earlier during the presentation, the lists are hard. Sometimes you got a list with two sentences in it. It's not just one item, it's two items. How do you compensate for that? How do you know when the list ends? What if there's a three list item in there?

46:34 - Unidentified Speaker

You can go on and look for the next item, but the last list item, is it one sentence?

46:41 - D. D.

Is it two sentences? How do you know? So there's, you know, there's an argument for structured specifications. You know, it comes up almost in, I don't know, I'd say 90% of the papers. You know, it's like, hey, this would be better if we made some rules on how to write specifications. Instead of just saying, hey, write it however you want. I think we're going to have it that way though. I think the large language models and the AIs are going to be able to, they don't care how you write it. They'll figure it out. They can read it faster than we can. Take our fastest reader and stick it up against the AI, I bet they'll lose. They can beat us at chess. Now they can read faster than us. Anybody have any more questions? If not, I can turn this meeting back over to Dr.

47:44 - Unidentified Speaker

B. Really interesting. Thank you.

47:46 - A. B.

I appreciate that.

47:48 - Multiple Speakers

It's really seems hard to get some interest drummed up about this stuff.

47:55 - D. B.

Well, thank you for the presentation. I always tell people who are presenting, feel free to make it as informal as you like, because I don't want to make a high bar. But your presentation was relatively polished, and I appreciate that.

48:15 - D. D.

I'm afraid some of my instructors might show up.

48:19 - D. B.

I don't want to get in trouble. Right. Well, does anyone else? Let me just share my screen. Where was it? You think that you should take a look at this fourth generation programming language.

48:36 - M. M.

I think it makes sense. The one that E. was talking about? Yeah, that E. suggests.

48:46 - D. D.

How old is that? It's old.

48:49 - M. M.

It's 16, 18. Oh, it's a Wikipedia. Yeah, yeah, but 85 is when I was using it.

48:59 - E. G.

I used it all the way through the 90s.

49:03 - M. M.

Yes, exactly.

49:04 - D. D.

I remember I was playing football in 85.

49:08 - M. M.

E., you're baby, you're baby, you're not born. Yes.

49:12 - Multiple Speakers

OK, J.'s taken. Wait a second.

49:15 - A. B.

I wasn't even around then, was I?

49:18 - J. H.

I was born. I was almost around.

49:21 - Unidentified Speaker

Yeah.

49:22 - Multiple Speakers

I used to have dreams of going to the NFL back then.

49:30 - M. M.

Some old stuff really makes sense now, so you can take a look and see. A., did I get this right? Which one? Yeah, it's a YouTube video with Keynote. With the keynote?

49:52 - D. B.

Keynote from NVIDIA, yes.

49:55 - Unidentified Speaker

49:55 - M. M.

That's a two hour video. Yeah, two hours. Yes, it's a two hour. But you gotta be dedicated. Excited.

50:08 - D. D.

Crank that speed up, get some popcorn.

50:13 - M. M.

Speed up, but you will enjoy it. I'm sure, it's very good.

50:20 - D. B.

It's by the president, the CEO?

50:23 - M. M.

Yes, the CEO. What's his name, J.?

50:27 - D. B.

J. Hey, that was a really good job.

50:31 - M. M.

Yeah, entertaining. Definitely a professional. The robot that he show was the same that he show last year, but the rest is new.

50:44 - D. B.

All right. Next week is spring break. We can meet. I'll be here if you want to meet. Anyone have any opinions or you want to meet you don't want to meet anything like that? I like meeting.

50:56 - D. D.

It's kind of my highlight of the week. So if you guys want to come, let's do it.

51:02 - D. B.

All right. I enjoy, you know, running these meetings, too. I enjoy having them. I guess I don't enjoy running them. I enjoy having them.

51:10 - D. D.

Let's put it that way. They watch that video, keep watching the AI video on the transformer, that's pretty good. I don't know what you have planned.

51:22 - D. B.

There's nothing on the agenda specifically for next week, so what we do is just general updates and video readings and video watchings. Okay, well then let's call it a day for today, and thanks Thanks to everyone for being here, and especially to D. for presenting, and we'll see you all next week, hopefully.

finished3-21-25AI Discussion Group Transcript.txt

Displaying finished3-21-25AI Discussion Group Transcript.txt.

 

Displaying jan10FinishedTranscript.txt.

at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)