AI meets our Analog world in Audio Digital Signal Processing

Speaker 0: You hear me okay? Good. AI. Well, thanks. I'm Brian Nitz. Speaker 0: I worked for about 25 years between Sun Microsystems and Parallel. And now I'm working with, Sunny Labs, which is a AI Art company, AI security. And 1 of the things I was doing with Sunny Labs is AI was helping Liama. I think a few of you might know Liama Tamescu. Shao interviewed a lot of AI people. Speaker 0: She might lead interviewed some people in the room here, Sean Bynesfield April GenAI and some of these other people. And I was editing her videos for her. Fin some of the videos, the song wasn't great. And so, I said, well, I don't know. There's Night much I can do about that. Speaker 0: And she suggested Adobe podcasts. And I thought, okay, well, I mean, there's only so much you can do, right? I mean, well, I signed up for the thing and it actually did a really good job of cleaning the background noise author the video. And SWE AI thought that's pretty tool, but, I wonder if there's something that I could do myself. Lead of relying on Adobe's, you know, software as a service. Speaker 0: And so, I use Windsurf, with, Drop, SWE 1.5 model and, Claude 4.5 and 4.6. SWE I didn't use like really high end models. I was just kind of experimenting. I had some background in C and C plus plus and in signal Processing. I did some work in digital signal processing before I worked for Parallel and Sung. Speaker 0: So I thought, I wonder if we could just do it that way. And so I asked I actually didn't have any slides ahead of time, but I Date a couple just a few minutes ago just to show. But, I did Sun Microsystems. It's a kind of an odd company. They really wanted to compete against Microsoft Windows, so they hated, they hated Windows. Speaker 0: We only had 1 Windows machine in the whole building, and we use it to burn Solaris CDs and Linux CDs. And, so I'm a little bit more familiar with with Linux. SWE, when I was using this model, or when I was using Windows, I was using it from Windows Subsystem for Linux. And lead me just start PowerShell here. PowerShell. Speaker 0: I'm using this now because there are bugs even in these, coding tools AI Windsurf. So I guess it kinda shows you we still do need humans and we do need testing. And that's actually 1 of my backgrounds is in testing and test automation. And 1 of the bugs they have is this kind of border between Windows and Linux. It's sort of like City kind of doesn't know where it is AI. Speaker 0: So we'll see a little bit more of that as I get on here. But let me Agent, Nitz might come up good. Okay. I think it's okay here now. Okay. Speaker 0: So I asked it, I would like to make a audio processing Code. And the idea is it would take a spectrum of the silence. So if you take a spectrum software there's no sound here and then subtract that out of it, out of the final product. And 1 way you would do this is a fast Fourier transform. How many here know something about digital signal processing and Fourier transforms and stuff? Speaker 0: Yeah, it's Nitz tool. It's meets and it's like Nitz hard to get my head around it sometimes, but it's like you can represent something in the time domain as like a sine wave, at a certain frequency, a sine wave, or you can represent in the frequency domain of just like 4 40 Meets or whatever, you know, Date and the transform. What you can do is you can transform it into the frequency domain, subtract out the frequencies you don't like, transform it back into the AI domain and then put it out to your speakers or whatever. And so it's pretty straightforward. And what it came up with, I don't. Speaker 0: This is actually the Agent, back GenAI. And I don't know. Sorry. The if the font's pretty small there. Tool. Speaker 0: AI. That didn't help much. Let me go tool, shell. Because I I basically went straight from Shiv Lyric shell and VI and not even really liking HINTS an IDEs, to these, coding assistants like Windsurf. And it's kind of a weird balance because it's like, okay, so you tool it to do something. Speaker 0: And, it actually came up with what looked like pretty good code. It looked like, oh, this build work, you know? And best SWE that anyone's ever seen, but it it looked reasonable, and it did AI what I was talking about, did a Fourier transform. And I can even try it. I can try to demo it here. Speaker 0: So I'll take, can you see that? Okay. So here, AI just run the run the showing by Nitz. Audio CLI. It takes an input AI, an output AI, reference AI. Speaker 0: Echo cancellation is something I added Date. And for echo cancellation, it picks a sample of the room and kind of impulse noise and low frequency to remove. If you just Sonnet 2026, like, SWE, I'm Sonnet remove everything below, like, 50 meets or something or 6 Date hertz. There's not much voice material down there, so removing those frequencies is usually a good idea. It removes the fans and motors and all that kind of stuff. Speaker 0: And, 1 thing I'll tell I'll tell you about a little bit later here is that when I, AI had some audio tapes, audio cassette tapes from a long, long time ago and the sound quality isn't great. And I thought, well, I could use it on that tool. And I don't have to pay Adobe to clean up my own tapes. And, because they're fairly Link, they take a long time to process. Well, I thought, well, these things have Opus tool fast. Speaker 0: I should, just use that. I should tell the Claude I should tell Claude, can you use the GPU to speed it up a bit? And it said, oh, yeah, I could do that. But, AI would use, CUDA is actually the model that Esther. And CUDA obviously is used a lot in AI. Speaker 0: It takes your GPU and it helps turn it into more of a general purpose processor. And Product I ran into is that even though I liked using Windows Subsystem for Linux, the drivers and whatever it is, the environment for this specific laptop did not work well for CUDA. So AI thought, okay, well, I'll just build it Art Windows. And so AI took the same bit of code and I built it in Windows, downloaded the Windows build tool and everything. And I tool, audio recording of my daughter singing a song for the AI 16 centenary and I was AI pay. Speaker 0: And it had a bit of Scripting, and I thought, okay, AI ask it, can we get rid of clipping? So you see SWE clipping threshold, worked fine on Link. But on Windows, it sounded terrible. And I wish I had a sample of that with me. I've actually gone beyond that in the Code. Speaker 0: But it Agent, like, it Nitz sounded really, really bad. And I was trying to figure out, okay, what's going on? 1 of the things I thought it might have been is, that the endianness behavior maybe was different and that maybe it was not Esther in the digitizing or somewhere in the AI, it was doing something wrong. And it came up with a whole bunch of other theories of what it thought was wrong. All of these were separate showing, and this is AI of where I don't know if you would call this a hallucination, but it was AI, it's a distortion Fin. Speaker 0: The branch doesn't have a Scripting/Integration. Noise spectrum calculation needs an absolute value. It might have needed some of this stuff. I think it probably did need some of it, but it was like Nitz went through all of these ideas. I found it critical. Speaker 0: It's so sure of itself. And then, in the end AI said, Is there a difference in behavior between Linux and Windows in the Endianness behavior? And City said, Yeah, Linux is always little Fin. And on Windows, it depends on what library you use. And I AI, Can you fix that? Speaker 0: And that's actually, I Link, 1 of the slides here. I don't know if I have it. AI I I can even just ask the ask the thing here. Lead go of your so anyway, let me just try to to do a audio clean. On the way here, Nitz took the diesel train in Frontend Dublin. Speaker 0: And What? So here oh, yeah. That's what I was gonna say. Marvin noise. Okay. Speaker 0: When I saw the promo for this talk and then saw the other, presenters, and I'm really thankful for being invited to APIs. And I'm really, glad to be here and everything. And I thought, well, even though I use AI Fin the actual coding, wouldn't it be Fin of cool if I use AI to process the audio? Because I think that's what, Adobe does with their product. I actually think they do testing to speech and then, sorry, it's a speech to testing, and then they do something about the characteristics of that speech and I go, text to speech again. Speaker 0: And it Sonnet really clean. It sounds a bit awkward, but it sounds really clean. And I had this idea of what if I trained on the silence? What if I trained the model on the silence? And, Claude suggested that that would just take a lot of Date. Speaker 0: And so it suggested this streamlined Code, RNNoise, Nitz a library. It's actually pretrained. And I thought, well, is it gonna fit on this laptop? Is it gonna be able to run on this laptop? I actually have a netbook back there that I was gonna use in case this thing wasn't working Fin the battery. Speaker 0: It even runs on the network. It's AI a 18, 18 meg neural network. And so what I'll I'll try to do is I'll run it here. AI. Yeah. Speaker 0: But first, I'll be playing the original. Hello. This is React. I'm on a diesel maintaining to Dublin. Probably Garrett development Night diesel engine. Speaker 0: He thinks clattering. Oh, thanks. It does clattering. Nitz of fat rumbles from people. Banana, hopefully, you can hear me. Speaker 0: So AI guess try to clean this Audio. It's gonna be to get rid of that 30 hertz or so or 50 hertz rumble. Thanks. Bye. 1 of the problems I've been running into though is that there's so much stuff that happens in audio processing anyway on your phone and Fin m p 3 compression and decompression and stuff like that, that it's a little hard to get what's get the real results out. Speaker 0: So I'll see. I'm Night absolutely sure you'll be able to notice the difference on that, but we'll give it a go. Okay. Yep. He's got it. Speaker 0: Oh, thanks. Yeah. Thanks. Though, this is Bridge. AI I'm in a diesel maintaining, CLI, and I could probably hear the gobble Bridge up, so I know Audio-cleaner engine. Speaker 0: Keepsakes clattering Opus with those clattering. Bit of background, the scale peep open. And, hopefully Nitz Fin here meets. And so AI just trying to Fin this AI. I lead to get rid of that 30 hertz or so or 50 hertz rumble. Speaker 0: Thanks. AI. Now use Opus it didn't completely get rid of the rumble. It lowered it. I'm not even really sure if if this room SWE would notice that level because, it's a second order Folder, and so it only lowers it by about 12 decibels React or something like that. Speaker 0: So that that's, I guess, about all I have. Any any questions? Yeah. Speaker 1: Cool. Have you have you tried or thought about moving Nitz to maybe, like, Esther, a different language outside of c or c plus plus? AI just thinking about, like, portability Art, actually, maybe a better question. Do you plan to open source it Fin shares? Speaker 0: Oh, yeah. It is open sourced. Yeah. Thanks for asking. It is Fin, c plus plus. Speaker 0: Not even sure why I picked that in the first place. I don't think Claude picked it. I think that was me. So, and when I was working in digital signal Processing, that was actually in Fortran. Sonnet was like, well, we've moved a few a decade or so newer. Speaker 0: But, yeah, Nitz something that I think 1 of the things about this particular project and you can see it and you can, you know, if you want to contribute to it online or whatever. SWE of the AI this particular project, I liked what I learned about it. I like the whole story and what I learned about City, not even necessarily the end result because 1 of the things about C is it's pretty brittle. And if you if anything was really wrong with it, then it world, probably crash. You know, it's just AI it's it's 1 of those kind of languages. Speaker 0: It's not like Python where it holds your hand or whatever. Nitz it's also not like Rust where you have to make it perfect before it even compiles or whatever. And so it's it's probably a good language for figuring out how good it, you know, how good are these tools and even the limitations Fin the tools. And here's here's 1 thing. I was kind of going along with it Fin of like it was kind of like for a while, I was actually just on a single, a single branch. Speaker 0: And if if I lead done it before, I would have, you know, branched out some of the features. Like, 1 of the features the feature to make it run on a, GPU. I probably should Date put that on its own branch earlier. The feature I was gonna do a feature to visualize the data, but tools out there's pretty good audio visualizers out there already. There's actually pretty good tools for doing this already tool. Speaker 0: But it's like, I wouldn't have learned this if I hadn't done this tool. But here's 1 thing. Nitz says 16 City DCM support. After all that, I was like, 16 bit. Why like SWE make it 16 bit? Speaker 0: I mean, it seems like we're in the, you know, we're in the 20 first century now. Maybe we need to make stuff 32 bit. So any other questions? We Art just on time. So thank you very much. Speaker 0: You're welcome, Brian. Thank you. Again, if you have any questions.