Ray Fernando
speaker
138 appearances
1 recordings
1 series
first heard Jan 2025
last heard Jan 2025
Ray Fernando’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
Mines is already showing the app is running here, but that's how you already know it's installed. It will require the terminal, but it really won't hurt you too bad if you have it running. So the first command that you will basically run is this one that's listed on their quick start. So the quick start will be listed here and we'll have this available as a guide for those to download.
And all you have to do is follow these two steps, really. So the first step is pull the container. So this is you just copy this and then you put it into the terminal and then it's going to do this little pulling thing and it'll probably download like, you know, several gigabytes of files onto your machine. And then the next step is literally what they call running the container.
So with Docker, the whole app and everything is all contained in one. So that way you don't have to spend a bunch of time doing extra terminal things. This is probably the only two terminal things that you will run. If you're running a PC, you probably want to run, especially for NVIDIA, you'll want to run this command, GPUs-all. So all you have to do is just copy this one if you're running NVIDIA.
And that'll take advantage of your GPU. And it'll run more efficiently when you're running it locally. So the one I like to do is just a single user mode, which doesn't require sign in. That way, if you're the only one that's using it at your house or on your network, that's probably the best way to do it.
So you just copy this command here and then you put it into the terminal and it'll say, hey, you know, great. It's up and running. And then all you have to do now is just go to the website localhost3000.com. And then once you're running on localhost 3000, you're going to be presented with some user interface thing like this. And I have a model that's currently loaded here.
That's kind of why it's showing us here. But you're not going to have any models loaded. So in order for you, the next step here that we have to do is actually have a couple of options. One of the things that I do is you can just download a model locally. And I use a thing called Ollama. And so ollama.com. is something you'll want to download there. And so that way you can run any local model.
And it's simple as just, you know, finding the model and so forth. So once you hit download, it's going to download for your machine and you install it. You'll see like the way that it's downloaded, you'll actually see this little llama guy that's at the very top and a little figure there. And that's how you know it's currently running.
And so once that's currently running there, you'll see the models that are currently listed here in the model section at the very top. And so DeepSeek R1 is going to be the one that we want to use here. So we'll go back to our web UI instance, and then we're going to go ahead and hit where it says user at the bottom. From there, we're going to go to the admin panel.
And from the admin panel, there's like a section, a settings area. And so this settings area has an area of our connections with a little cloud icon. And this is kind of where we're going to connect our other providers here. So let me make this a little bit bigger so that everyone can see. So as you can see, the Ollama API is already configured for us, which is nice.
And this is already going to have the Docker container there, which is great. And so when you hit the little pencil here and you hit plus, change the model, you can type in the model like deep seek. And if you don't see it available, which it may not be there, you'll see this option at the bottom that says pull deep seek from olama.com. And so that'll actually search olama here to get it for you.
So like, for example, if we wanted to download the fee four model, I'll just type that one in just as an example. So you can see fee four. So I have no, I don't have that model currently downloaded. I just can hit here and it's just going to go ahead and find it and it downloads it. So in no time, basically this model would just be downloaded on my machine.
And then I could just type in fee for, and I'll be able to use that in the future going forward. So what we can do is go ahead and type in, you know, we're starting a new chat and we're going to basically select the model deep seek dash R1. And you'll see it'll be colon latest is kind of what it's listed at. And that's actually how you know that that's the one that's currently running locally.
And so once you select that one, you can just say something like explain options training and then go ahead and hit enter. And so what this does is, you know, it's basically, you know, thinking and you can see the thinking tokens of what's going on when it's thinking. And so all of this is actually running on my computer, which is amazing.
One of the ways that I can tell is there's this command line called asitop. And it actually shows us all of the resources that it's eating up. Thankfully, I have 128 gigabytes on my machine because I do live streams. I do all this stuff at the same time. And you can kind of see how much RAM it takes up right now with me hosting the stream plus running this model locally.
So yeah, this is actually what it does here. One of the things that we could even do is try to test that prompts that we were using earlier so that we can run this command locally. So earlier, what we did was we were running like a whole analysis on something and it would just fail out.
So this thoughtful analysis that I was showing you, we can try to see if we can run this on a local model and just see the difference as well. So this is basically the transcript that I had earlier, plus the analysis stuff. And if I go to open web UI and then just go ahead and kind of go ahead and go back here and create a new chat. and hit paste, and then hit run.
So this is going to see it's thinking here, and it's using up all the resources on my local machine to run this model. And it's quite a lot of tokens. And it's still fairly impressive what a smaller model can do that's running on my machine. And you'll have different versions that you can use. And so this one is using the 7 billion parameter model. If you get something that's a little bit higher,
This is probably going to get you a little bit more detailed response. And I would definitely play around with these things. Another important setting I think that you can tweak, and we can probably run this as a next chat, is while this is going here, there's a control section. So this control section at the very top will show us, let's see what to dismiss this.
So the controls, one of the controls that you'll probably want to change around to get different results is the temperature. So it's setting the temperature from like, you know, 0.8, the default to like a lower temperature will actually make it like hallucinate less is kind of what people say. And so it'll tend to follow instructions better and then not kind of veer off into different tangents.
And then another one, if you go all the way to one, it'll just be extremely creative. So you can think about those as far as maybe if you're doing some creative writing, some non-logical reasoning, that can be really helpful if you want to kind of think out of the box and have it kind of go into La La Land.
Showing 61–80 of 138 · page 4 of 7
← Previous
Next →