Structured charting with LLMs

Hi. Hello. My name is Ivo. If you don't know me, I'm a machine learning engineer. I work in cybersecurity. I think I've been coming to AI Tinkers since long as I can remember. Okay. So my presentation, because I do love to yap a lot, it's really simple. Right? As an engineer, especially as a machine learning engineer, the whole what companies usually want to work for is, like, really simple pipelines. Right? Things that just work without, you know, the manager going, like, why is this too complicated? Why is the code base so bloated? Why isn't my mechanical runner not working because the code base is too big? Right? So I kind of had an idea to make something really AI, like, Pydantic for visualizations. Right? So when you first think of how can I get an LM to visualize something for me, right, it's like, oh, I'm going to get to get the code for it, and the LM can then execute the code, and then we have visualization? It's a really simple problem. Right? But having it inside of a pipeline is actually quite crucial because, for example, let's say you're building a database l l m that's gonna, you know, answer questions about your database, but then also the c suite also wants visualizations in the chat at the same time. How do you make it that's, you know, concurrent and accurate? Well, for my answer for this is this thing called Visdantic, which I built here. It's so simple that it's stupid. And when something is stupid, that's a good idea. So the only, the only dependency we have pretty much for it is Pydantic. That's literally the only single dependency that you want. You have optional dependencies, of course, like Plotly, which is the only 1 I support so far. But I try to keep it as tiny and minimal as possible to the finest grain, and I want the lines of code to be as minimal as possible. And its 1 and only true objective is to make visualizations for any type of LOM for 2 calling for inside of your IDE or your terminal. When you run it, it will just make you a visualization of your choice and the styling of your choice consistently every single time. That is the AI thing of it, and it's so basic. This is where it lives. This is the spec. Right? You have your vis spec class, which pretty much every visualization needs a title or an optional legend or an optional, like, back end, for example. And, you know, it might be Plotly or MatLab, etcetera, blah AI blah. Right? And then you have your different types of visualizations. You have your Cartesian specs. So for example, we go here for your Cartesian charts, which is a bar, line, area, or scatter. So on, so on, so on. You can have point charts with your point spec. You have all these specs. Right? So I think I have, like, 20 specs that cover all kinds of specs that you can do for a visualization. And then you pretty much go to your adapter. So the smart thing about an adapter is with AI, you have this VISSPEC adapter is instead of it going through every single thing, whatever the LLM says, whichever spec adapter I wanna use, PyDantic will just grab that. And that's in the big o notation of o 1 AI. So it's, like, extremely quick. It's super simple. Once you Haugh that, you can do validate. You can validate your spec. What did the LLM say? Which spec does it wanna use? Is it valid? Once it's valid, perfect. Amazing. And then you have your schema. And this all it does is it gets your spec that the LLM returned in a JSON, and you can put it into a prompt. So you can just place hold schema into a prompt. So you can prompt it any way you like, any how the way you like, but the LML will only choose from the specs that it can do here, all the visualization types. And that's it. That's it in a nutshell. So I made a little demo for you guys. So how to use this library or this repo is really simple. So for example, in this case, I have a visualization model. Right? So I have my prompt. I load in my prompt from a thing called the AI config, which is pretty great because, for example, you can have your visualization config where you can, you know, have your model, your temperature, your region, if whatever cloud provider you're doing, and this is your prompt. It's a very small prompt. All I said is, hey. You have this dummy data. Hey. This is a schema from the AI file we saw earlier on, and this is the user request. If the user wants, IDENTIFICATION, you give it to them. If they don't, you explain the data. That's it. That's all you need to do, really. You're just like that. And all there is to it then from this side of it, from Visdantec, is you have your schema, your validate, and then your render, which is an optional plugin for Plotly. So for example, when you download Visdantec, you can do it manually, or you can just download pip install or u v install or v u v add Plotly by itself. Right? You don't need to do it. You can make your own visualization if you want to. Spec validate validates what did the LLM say for the spec, then you render the spec, and also you have a team. So this is the 1 of the coolest things Tinkerers you can build your own team, which is pretty much a function. So I chose to go with Plotly. You can have a Google team, for example. Here's your Google colors. Here's your fonts, your title, how it should look like. The LLM does none of that. All the LLM will do is just generate a visualization, but you'll have your team set in a function. And in that sense, all you have to literally do is get your fig equals render to spec and the data, which is your, you know, whatever you're using, your table you're using to render it from, and fig equals Google team fig, fig dot show. And that's it. So simple. Right. Let's have fun and play around with it. So I'm just using a thing called the rich terminal, which kinda makes it look quite nice. I hope I'm still connected to my hot spot. Oh, I'm getting baseline. K. Cool. Okay. There you go. So I have dummy data. I have a quarter revenue expenses and profits. So I'm gonna say, what was the total revenue from okay. I'm actually going to make it better. Is that is that better for you guys? Yeah. Yeah. It's funny how you can still see the screen sharing thing. Oh, yeah. Thank you. Okay. I said waters, but whatever. Okay. So the LM is gonna go on due to from all my waters. So it says, here's your waters, your quarters, and then would you like a visualization? Yeah. And then AI I'm gonna say like this, create a visualization because this thing doesn't have memory. Spoiler alert. I know. You're probably saying everyone has memory when they build these, but no. This is just a thing. Okay. And then you have your IDENTIFICATION, and it's using my Google team pattern. And I can say make any visualization, give me info, make it 3 d. One's gonna be thinking. This is using a clock sonnet or opus, actually, from, AWS Bedrock. K. Interesting. Looking at your data, I can create a version that has numeric value. Okay. It's actually giving me suggestions. Okay. Yeah. So I AI want suggestion. This is a prompting issue, by the way. K. There you go. And now you have your 3 d visualization in your specific team or font of color. So you can do this for as long as you like. You can pick whatever spec you want. You can have a point graph distribution spec, parts, matrix, flow spec, hierarchy, so on and on. So I'm gonna close it at that. But I just wanna say, what is the point of this? Like, why the hell would you build something so basic? Well, the problem is, like, I think the trend now with AI engineering, especially for, like, enterprise, is you have this huge AI, and you're just trying to make things work all the time. But as it grows bigger and bigger, deterministic stuff starts to get less deterministic. So with this, this is all validated before it even hits the LOM. There is no code involved in it. There's no code function execution, really, except the team. All it does is it literally gets a AI JSON back to you. That's the validated, hey. This is my spec, and it chooses a spec for you based on the user's request. And that's it. Yeah. If you AI to give me a star, I'd be very happy. Thank you very much.