LLM vs Classical Vision Models for Real-World Object Detection

Speaker 0: Hi, everyone. How's it going? How's your evening? Speaker 1: Let me show you my application. It's a Telegram mini bot. And the idea of this bot came to me before Christmas with all of this Christmas mood. I decided to make another way how to celebrate Secret Santa with my friends and to build an app that, may involve small coffee shops to take part in this app. Because my app actually helping people, and coffee shops, of course, that probably I'll charge in future. Speaker 1: We'll see. So, the main idea of this app, it's to involve users, come to the coffee shop where I'll, prayer prepare, boxes for the game. And, user, once he came, he just should choose the mood. If you want, you can try. It's much easier if you can scan it. Speaker 1: It's much better than 1000 words if you have Telegram. So, once you try l continue. The idea was to involve people to come to the coffee shop to, my application will check the location of this, of of person and will sign, the box that's currently free. And I met a problem that, when people come to the to the coffee shop and probably will put the gift that he should prepare, I cannot really Callan, is it a real gift or what what kind of thing is it might be. So I decided to make an LLM layer that will help me to recognize what thing is it. Speaker 1: And, I'm asking people, I'm asking, the user once he came to this Callan. I'm asking to, to come to the to make a photo of this. And this photo, I'm not storing into my database. I'm just sending it into the, Gemini, 2 and a half flashlight and, with and, with some kind of, like, orchestration. Let me show, probably. Speaker 1: Or I I'll, I'll share all this, and everyone can see the code. So I'll just explain in in words. So, my application sent first of all, it's, zipped 4 to send it to the Gemini just to reduce the cost of, every every every input. And, Gemini returned me JSON text, which I validate later on my app. And, with, like, prayer prayer rules, I can then, understand, I ask Gemini to return, specific specific columns as you can see here. Speaker 1: For example, I am asking to categorize the item that is on the photo and return me, what kind of item, like, what kind of, category, what what item is it. And then, I'm stored first of all, I'm storing every scan into my database just to validate it later and to check, did I have any problems with the validation. And I send it into the admin to, make a final check. If, for example, it's, water because I have problems with Gemini at the moment, for example, I asked Gemini to, not to allow alcohol, but, if if I scan, for example, bottle of wine or some some specific alcohol, it cannot recognize that it's an alcohol. It can just send it that it's a wine, for example. Speaker 1: And, my application will then approve it because, it cannot approve alcohol, but it doesn't know that it Callan approve wine. So it will return that it's approved. And that's why I'm located it only to liquid. And, if it's liquid, it will send it to me. I can I can I can probably okay? Speaker 1: I'll show it to you later. A little bit later. IDENTIFICATION will, return my application, will return if it's, liquid, it will send it to the admin, and admin should make an approval. And until this approval, it will not go through through the app. And, once it's approval, it then can be assigned it to the specific box, and user can put it into the box. Speaker 1: And now the user will come and take it later. And I found it very useful, this approach, because, previously I tried to, use different, auxir scanners and to, like, tisiract, something like that, easy auxir, and for for recognizing of, text on the photo. And, found it not super useful for me because this approach is, bring me lots of noise inside of the the response. And, it was lots of things should be done for like, clear all this noise, in the text. And, later, I tried YOLA, but YOLA, it's a it's a very nice approach, but and it's it can did anyone try YOLA and, disrupt or something like that? Speaker 1: So YOLA, it's very nice for, like, stream recognition, but it can recognize only, like, for example, shape of the, of of the thing that people try to scan. For example, it can recognize that it's a bottle, but it cannot recognize what kind of bottle it is. But this approach can recognize the exact what is this, and it's, at the moment, much cheaper than to, to use, like, a very nice computer with all of these specs, and you should prepare all this data for y'all and so on. So this approach Callan work out of the box, right, just very easy to install. And but in this case of Gemini, you can easily spend too much tokens if some for example, someone will, know your API and it's not secured. Speaker 1: People just Callan very often, like, abuse your API and spend too much tokens. That's why, first of all, I have limited, the, FOTA. And, at the moment, it's I can show it to you. We we can see together how much it is spending for me. So right now, I'm just come into the app from another form. Speaker 1: From the very old phone just to show because it's super very old phone. And I took it just to show how it is how it really can recognize. And it's very nice in my case because I already have future plans, how how to can use it. Just a minute. Let's, for example, scan what? Speaker 1: Let's let's scan this 1. Yeah. Okay. You can try it too. Excellent. Speaker 1: And now you can see yeah. So for this scan, I've spent around around, 621 tokens, and it's super cheap because and I have a calculator inside of my app, and I can't, tell you how much this where is it? So at the moment, it costs around yeah. It's super cheap because, I can, I can scan around 12,000 of photos, for $1 for €1, something like that? And it's it's super cheap for my approach because if I'll involve coffee shops, I'll cover all this, all these scans super easy. Speaker 1: And, with my limitation, in my in my app, I made a limitation of, for example, I didn't include any, like, reasoning for my model because, it's enough for me just to receive my response that I heard the moment. And, I have, like, counter if, for example, model return me a problem from from the model side. I'll just let people let the user try again. If, the user will scan something like if the user will scan and everything is fine, but he will return to the scanner again and will try to scan again, The user will be banned for 5 minutes and the second time for 15 and so on. So just to not to abuse my app and to all the scans, they are, like, they are also assigned with the any data that is coming from Telegram as in, out authorization token. Speaker 1: And that's why it's super difficult to, like, to to replace this in a data. So at the moment, looks like everything is Callan, and it's difficult to abuse. Can we see the bot? Can you see it on the chart? See the chat of what? Speaker 1: The bot. Ah, yes. Sorry. Sorry. Yeah. Speaker 1: Yeah. Here it is. Okay. Then there you go. Just if you have Telegram, just try to scan it.