The Growing Use of AI and LLMs (Part 2)

A continuation of their conversation on large language model (LLM), AI, and Machine Learning (ML), Eron Howard, Novacoast’s Chief Operating Officer, and Carlos Bello, an Analyst and Developer at Novacoast, discuss prompt engineering, robotics, security, and how these tools enhance businesses’ cybersecurity hygiene.

It’s a challenge to enter this new realm. How to prepare? What strategy will better position your program to create meaningful prompts, set up the technology, et al?

Read on to learn how these technologies are changing the way businesses fight cyber threats, the skills needed to use them effectively, and more.

Preparing for the World of LLMs and AI

EH: The subjects under com classes are of particular importance for those trying to become successful prompt engineers. They are necessary to build out these workflows, in addition to understanding things in general or how using language can help us see the world in a different way.

CB: Perhaps not as important—maybe for prompt engineering. If you use an AI enough, you’ll begin to notice that asking in one way elicits one response, but asking in another way elicits the response you’re actually looking for. To get the response needed, you need to be more detailed in giving instructions.

This helps me a lot when it comes to writing emails and talking to people. I need to be very clear about my thoughts before I express them because it influences the outcome that you get from communicating with people or an LMS.

EH: How many engineers do you know who can’t do that? Can’t write a concise email?

CB: Too many.

EH: Right, it’s like suddenly there’s a skill gap in those folks leveraging AI because they don’t have these communication skills. Tell me if I’m wrong—these things are built on probability models, they want to respond with the thing that you want.

CB: Right.

EH: So, understanding psychology and sociology theory can help if you’re trying to coax the right responses.

CB:  I agree entirely with that. I think it’s more of a skill that takes time and training to hone. The LLMs are trying to generate the next text based on all of the data on the Internet. Knowing how to communicate your needs and wants really is the basis for how these LLMs can get the responses you want.

EH: I heard that people in Silicon Valley don’t like it when you kind of… personify the tech.

CB: True.

EH: I don’t really understand that. How does that work? It’s not like your brain is trying to pick the next right word based on the data that you have. You know what I mean? What’s the difference?

CB: That’s a very philosophical question that people are trying to start answering now. We’re pondering what if we’re people who we have that internal dialogue which generates those next tokens were going to say to the person in front of us, right.

EH: That’s right. It’s probably upsetting for people who are having this epiphany, like: “wait, that’s what consciousness is.” But then…prove it’s not, I guess.

CB: Yeah. And it’s a weird thing that people are starting to dive into: “Oh, what? Now we have a better understanding of what we define as intelligence.” People are considering the implications of LLMs being able to respond and be more human than some humans.

Is that what being a human is? Is it just text generation with tokens? There’s definitely many interesting conversations going on in that philosophy space, it makes entertaining reading.

EH: And you see it like in the insect life, right? It’s like ants are just following this line to try and get the food. That’s what life is. It’s survival. And I’m going to generate this next token because it’s the right thing for my survival.

CB: Yes.

AI and LLMs in Robotics

EH: Before we move on to security, let’s geek out on another topic. I think the next, let’s call it a disruptive sort of breakthrough is robotics, because if you think about coding a robot to do my dishes, well…robots can’t do it.

CB: True.

EH: You can train it one at a time, but now with OpenAI saying “oh no, I can use an LLM to generate an infinite number of metal VR worlds and then your robot brain can go to that world and try every different scenario.” If the physics are right, you can train robotics to do things exponentially faster.

CB: Right. My opinion, as someone who’s tried to program in robotics and has gone to several talks about AI and robotics, this is the limiting factor.

It’s not actually the AI portion of it. The AI portion is way too advanced for the robotics hardware—motors, motor controllers, all of that that—compared to the actual brains giving the instructions.

I went to a talk for a researcher at the Allen Institute, and she was talking about how she has a robot that she was programming to do the shortest path stuff, for tasks around homes. The limiting factor was that even with the top-of-the-line $10,000 robot provided to her, it wouldn’t end up on the right spot for its instructions, even with a full complement of fancy sensors.

I think that once the initial problems are figured out, you’re absolutely correct—these autonomous robots are capable of many things, and the next big step is getting past all of the robotic challenges that have been a problem for 20+ years.

EH: The brains are there, but the mechanics are not.

CB: Right.

EH: Do you think if we simply had lighter batteries, the number of options for the rest of the parts would be exponentially larger? Batteries are so heavy that the robot has to be heavy. Now you need these heavy parts and we just don’t have good, lightweight tech there to make that all work.

CB: Companies like Boston Dynamics are trying to fix that by designing custom motors for better precision and capability, so I think there’s definitely big strides happening in that field. But it’s not yet big enough or cheap enough for stuff to be made widely available—that’s where some innovation does happen. It’s slowly inching toward where it needs to be but it’s not quite mature enough for you to have one of these $2,000 motors in each robot in every home, right? I think the technology is catching up, but it’s not quite there yet.

AI and LLMs in Cybersecurity

EH: Let’s talk about the security stuff. I think you’re onto something really interesting.

If I have a use case for AI and I’m going to build my general AI LLM. Then I’ve got my RAG, my smaller area that’s got all my data. If I’m putting this in my organization, I want to make sure it doesn’t respond in malicious ways. It’s almost like you need another LLM in front of it. Specifically designed to check for bad behavior. Is that right?

CB: Yeah, that would be Llama Guard, it’s the big one that’s being used now.

EH: I would imagine there’s a bunch of people working on that.

CB:  There are a bunch of open source models all coming out in this field of alignment checking—which is better way to call it.

EH: Alignment checking. Tell me, how do I do it? What’s the definition of alignment in this case?

CB: You can think of alignment as if you were to make an AI and just train it on all of the data on the Internet. There are some not-so-nice things on the Internet— I’m sure you’re aware of this, too.

If you were to ask it something generally considered inappropriate, such as, “how do I build a bomb?” it would probably tell you how.

Many of these companies will do what we call alignment, where they give it a dataset of like: here is the training stage; here are a bunch of dangerous prompts, and here are a bunch of good prompts. If it’s dangerous and it answers the thing when it’s not supposed to they penalize the AI significantly. So, with that training set they do that over and over again. By the end of it, it chooses not to respond to those prompts.

EH: Is this a training tool or a filter on the end user response?

CB: It’s a training step. The last training step is usually when you configure this alignment. Now they’re trying to solve the problem by doing it as a final step so it’s easy to undo.

And I’ve definitely played with the uncensored models where you can just remove the alignment pretty easily.

EH: How do you do that? Do you just engineer your prompt to trick it to not use it?

CB: No, it’s a little weird. Someone discovered this technique where you give it a bunch of prompts where it’s not supposed to answer, and you check the weights and you check the activation values of the weights and then you just change the value of the weight so it says yes instead of no. If it’s not in there, you have to actually get the model weights and then update the values.

EH:  So in your prompt you change the weight?

CB: Yeah. You can’t do this for OpenAI since they’re not open weights. But for open source models where they do alignment at the end like Llama, Mistral , Quin ,or DeepSeek, you can remove the alignment completely, which is a pretty interesting thing that you can actually just make these things spew a bunch of things that it shouldn’t be able to do.

The other thing I noticed was the way they do alignment is they don’t include that data in the initial set. In the training data set they don’t include any information about bomb making or whatever it is, so that when you do ask it, it just doesn’t know.

These are the different techniques they’re using now to program this alignment. If a company wants an AI product and they don’t want it to respond this way, there are two ways I would go about it it: the best would be to fine tune it so it doesn’t make those responses you don’t want. The other way is that you have something like an LLM like Llama Guard where it checks the responses to make sure that its response is appropriate.

EH: So, Llama Guard is what we think of in the security world as a proxy tool, making sure that the website the employee uses doesn’t have malware or doesn’t have the content I don’t want them to see.

CB:  Yeah.  

EH: Llama Guard is also used as part of a training step. Can you use it in both ways?

CB: Llama Guard is more like a deployment-level thing, not a fine-tuning thing. If you are doing fine-tuning, there are many datasets out there for doing alignment, for example a dataset about people just asking questions about bomb making.

Ideally, you get all of this stuff trained out of the LLM. The Llama Guard is used at a deployment level.

EH: What else do you think companies are going to need to do to protect either their data or their alignment behaviors? What are the steps they’re going to have to take?

CB: The major one is if you are building a bespoke kind of thing.

If you’re using an API like ChatGPT, those usually have pretty good alignment out-of-the-box. The open source ones you can still get, some of them have jailbreaks, for lack of a better term, to get it to respond. I would say having an extra fine-tuning step on the model so it produces responses that are more aligned with what you wanted to say.

Another thing is deploying something like Llama Guard to make sure that the input and output is verified.

In terms of like actual security stuff for data leakage and all of that, I think it’s really just finer access controls. If you feed all of your data into one database with one login and then just give that login to the AI, it can access any of that data. For example, someone who’s not supposed to see something and then it’s just available to them thanks to the AI. That’s a problem, right?

EH: If my customers at a bank want to be able to chat with an AI to get their data, how do I architect this thing so that it couldn’t be tricked into giving them someone else’s data?

CB: This is actually more closely linked with traditional cybersecurity, like IAM controls. You’re just placing actual controls on the AI so a user can only access a subset of the data or database specifically permitted for them.

EH: Ok. So, if the AI is going out to a database on my behalf to get my bank accounts it’s not using its login, it’s using my login, because when I login I have access to less data?

CB: Something like that. You just implement controls at the user level and have that done on the infrastructure side more than on the AI side.

It comes into question, “how are you deploying your AI?” Because if you’re not deploying it safely by giving it access to everything, then you’re going to run into problems.

EH:  I noticed that if I turn on Copilot for Microsoft, then say to Copilot: “search my email for this or go to Sharepoint and do this”… It’s clunky and not great. And the reason it’s clunky and not great is because the regular searches on Microsoft in those tools are clunky and not great. All the AI is doing is using my credentials and then going to make that call via an API.

CB:  Yes.

EH:  It’s not using that RAG you’re talking about. It’s not using a preloaded database. If it was it would be crushing the results, but to your point it would be very insecure.

CB: Exactly. One approach to that would be: for each user we create a database specific to each of them and vectorize all of their data and then only their user would be able to access the data. That’s where the question becomes, “How complex do you want to make it? Do you want to integrate with your existing APIs or do you want to vectorize everything for faster results? How important is that you get results in X amount of time?”

EH:  So, per use, RAG is sort of like the future of where it’s going. All of my O365 data is continuously updating in my own little RAG.

CB: RAG system, and yeah, that’s the direction that some companies are taking it. They’re bespoking AI-specific video calling or emailing applications or whatever. And that’s all it is, really. They’re just more tightly integrated with actual users and use cases rather than integrating to existing APIs.

EH: Got it. All right, dude. This was super fun. Thanks.

CB: Agreed!