Ray Fernando

One of the ways that I can tell is there's this command line called asitop. And it actually shows us all of the resources that it's eating up. Thankfully, I have 128 gigabytes on my machine because I do live streams. I do all this stuff at the same time. And you can kind of see how much RAM it takes up right now with me hosting the stream plus running this model locally.

One of the ways that I can tell is there's this command line called asitop. And it actually shows us all of the resources that it's eating up. Thankfully, I have 128 gigabytes on my machine because I do live streams. I do all this stuff at the same time. And you can kind of see how much RAM it takes up right now with me hosting the stream plus running this model locally.

So yeah, this is actually what it does here. One of the things that we could even do is try to test that prompts that we were using earlier so that we can run this command locally. So earlier, what we did was we were running like a whole analysis on something and it would just fail out.

So yeah, this is actually what it does here. One of the things that we could even do is try to test that prompts that we were using earlier so that we can run this command locally. So earlier, what we did was we were running like a whole analysis on something and it would just fail out.

So yeah, this is actually what it does here. One of the things that we could even do is try to test that prompts that we were using earlier so that we can run this command locally. So earlier, what we did was we were running like a whole analysis on something and it would just fail out.

So this thoughtful analysis that I was showing you, we can try to see if we can run this on a local model and just see the difference as well. So this is basically the transcript that I had earlier, plus the analysis stuff. And if I go to open web UI and then just go ahead and kind of go ahead and go back here and create a new chat. and hit paste, and then hit run.

So this thoughtful analysis that I was showing you, we can try to see if we can run this on a local model and just see the difference as well. So this is basically the transcript that I had earlier, plus the analysis stuff. And if I go to open web UI and then just go ahead and kind of go ahead and go back here and create a new chat. and hit paste, and then hit run.

So this thoughtful analysis that I was showing you, we can try to see if we can run this on a local model and just see the difference as well. So this is basically the transcript that I had earlier, plus the analysis stuff. And if I go to open web UI and then just go ahead and kind of go ahead and go back here and create a new chat. and hit paste, and then hit run.

So this is going to see it's thinking here, and it's using up all the resources on my local machine to run this model. And it's quite a lot of tokens. And it's still fairly impressive what a smaller model can do that's running on my machine. And you'll have different versions that you can use. And so this one is using the 7 billion parameter model. If you get something that's a little bit higher,

So this is going to see it's thinking here, and it's using up all the resources on my local machine to run this model. And it's quite a lot of tokens. And it's still fairly impressive what a smaller model can do that's running on my machine. And you'll have different versions that you can use. And so this one is using the 7 billion parameter model. If you get something that's a little bit higher,

So this is going to see it's thinking here, and it's using up all the resources on my local machine to run this model. And it's quite a lot of tokens. And it's still fairly impressive what a smaller model can do that's running on my machine. And you'll have different versions that you can use. And so this one is using the 7 billion parameter model. If you get something that's a little bit higher,

This is probably going to get you a little bit more detailed response. And I would definitely play around with these things. Another important setting I think that you can tweak, and we can probably run this as a next chat, is while this is going here, there's a control section. So this control section at the very top will show us, let's see what to dismiss this.

This is probably going to get you a little bit more detailed response. And I would definitely play around with these things. Another important setting I think that you can tweak, and we can probably run this as a next chat, is while this is going here, there's a control section. So this control section at the very top will show us, let's see what to dismiss this.

This is probably going to get you a little bit more detailed response. And I would definitely play around with these things. Another important setting I think that you can tweak, and we can probably run this as a next chat, is while this is going here, there's a control section. So this control section at the very top will show us, let's see what to dismiss this.

So the controls, one of the controls that you'll probably want to change around to get different results is the temperature. So it's setting the temperature from like, you know, 0.8, the default to like a lower temperature will actually make it like hallucinate less is kind of what people say. And so it'll tend to follow instructions better and then not kind of veer off into different tangents.

So the controls, one of the controls that you'll probably want to change around to get different results is the temperature. So it's setting the temperature from like, you know, 0.8, the default to like a lower temperature will actually make it like hallucinate less is kind of what people say. And so it'll tend to follow instructions better and then not kind of veer off into different tangents.

So the controls, one of the controls that you'll probably want to change around to get different results is the temperature. So it's setting the temperature from like, you know, 0.8, the default to like a lower temperature will actually make it like hallucinate less is kind of what people say. And so it'll tend to follow instructions better and then not kind of veer off into different tangents.

And then another one, if you go all the way to one, it'll just be extremely creative. So you can think about those as far as maybe if you're doing some creative writing, some non-logical reasoning, that can be really helpful if you want to kind of think out of the box and have it kind of go into La La Land.

And then another one, if you go all the way to one, it'll just be extremely creative. So you can think about those as far as maybe if you're doing some creative writing, some non-logical reasoning, that can be really helpful if you want to kind of think out of the box and have it kind of go into La La Land.

And then another one, if you go all the way to one, it'll just be extremely creative. So you can think about those as far as maybe if you're doing some creative writing, some non-logical reasoning, that can be really helpful if you want to kind of think out of the box and have it kind of go into La La Land.