Idea for Frequency Domain FX Transfer Function Plugin

[ewanpettigrew]

I don't think this idea exists in any other music software.

I have been working quite a bit with Xpressive lately, and the possibilities it gives for shaping waves in the time domain are enormous. The fact that anyone with a little bit of time domain knowledge can rapidly create new timbres is amazing.

This got me thinking, if LMMS was able to implement an FX plugin which allowed the user to enter a transfer function in the frequency domain (instead of a time domain formula such as in Xpressive), the possibilities for user defined effects would be greatly expanded. I think this would really set LMMS apart from other music software.

The plugin could perform a Fast Fourier Transform (FFT) on the incoming audio, multiply by the user’s transfer function H(f), and then perform an inverse FFT.

For example, the user could enter something like:

H(f) = 1 / (1 + (f/2000)⁴)

to create a low-pass filter.

I believe that a transfer-function FX plugin could allow rapid creation of many different audio processes defined entirely by the user, similar to how Xpressive has expanded what is possible using time domain formulas.

In the frequency domain this could include things like:
custom EQ curves, spectral gating, spectral warping, phase shaping, pitch/timbre shifting, noise shaping, formants, filters, and compression.
And if memory or previous-frame data is allowed, possibly even reverb or delay-style effects.

I have some very limited C++ knowledge from introductory programming classes at University. However, I could come up with a proof of concept (as there are so many tools these days to assist) maybe in LADSPA if this would be feasible.

I understand that the team probably has a lot on. However, I believe that that this idea could be a game changer for LMMS.

[regulus]

That sounds cool! I also once had an idea that maybe it would be cool to have a plugin where you can place poles and zeros of a transfer function. Where basically the user can drag the points around the 2d complex plane and view the resulting frequency/phase response. That would probably be a bit unintuitive though; maybe for educational purposes it would be neat.

Your idea of doing a real FFT + IFFT sounds more powerful. I am curious how you would deal with windowing/artifacts with it though. Afaik FFTs normally create artifacts if used on short segments of audio if they don't perfectly loop, since there's a discontinuity at the end/start. I know sometimes that's alleviated by adding a smooth windowing function which tapers off the ends. But I'm not an expert in that so idk what the proper way to do it is.

[ewanpettigrew]

Hi Regulus,

Your idea about moving poles and zeros around the complex plane is a great one not just as a tool, but especially as something educational. Even if it isn’t the kind of thing you normally see in a DAW, it would be amazing for people who are learning filter theory.

Over the last couple of days I actually tried to put together a very basic LADSPA plugin on Windows just to check whether my compiler setup was correct, and I ended up giving up after hours of fighting with it. I couldn’t get the required libraries to build, I couldn’t even get all the libraries to install. After that I tried compiling LMMS on Windows so I could just slot a prototype effect directly into the source, but I ran into a ton of issues with Qt. I tried MSYS2/MinGW64, PowerShell and the x64 Native Tools command prompt for VS, and the build still wouldn’t complete.

I then thought about trying to build the most bare bones VST2 plugin instead, but I realised that LMMS only really handles VSTs as instruments, not effects I believe. At that point I had to admit defeat, I’m really not a C++ developer (just someone who has studied a couple of introductory programming courses).

So in hindsight I probably shouldn’t have suggested I’d be able to make a prototype myself. I should have kept things on the conceptual/creative side of the idea.

Thanks for raising the point about artefacts that’s a really important part of the whole thing. Even though I’m not a programmer, I had thought a bit about how to avoid them. I think an STFT overlap add method could avoid the typical FFT windowing discontinuity problem. If the audio was split into overlapping frames, multiplied by a window, and the frames faded in/out, that could keep everything continuous.

Thanks again for the insights.

PS. I did think about installing Linux to build LMMS as this seems just too hard on Windows. However, my computer only has 2 SATA connectors with two drives already, so I have no other options at present.

[ewanpettigrew]

--

[ewanpettigrew]

I’ve started putting a plugin together for the idea above, and I’ve placed it inside a fork of LMMS on GitHub:
https://github.com/ewanp2025/lmms_H-s-/ ... erFunction

It’s compiling, showing up in the list of effects and transforming sound on my Azure VM (Ubuntu) which I can hear through X2Go Client. I had no luck at all trying to compile it on Windows. I still have a long way to go and a lot to learn, but I’ve made some progress so far. I have managed to get rid of most of the clicking by moving to the STFT. However, none of the H(s) are correct as I am still having trouble with implementing the parsing. I am going to try and work out how Xpressive does the parsing and may change from a simple parser to exprtk.

[regulus]

That's cool, I might take a look/try out your fork if I have time.

[ewanpettigrew]

Thanks Regulus. I appreciate if you have any time to look. I really appreciate everything you developers are doing with LMMS in your own time.

I actually believe that I made good progress today until I got stuck with my last point. Previously, the biggest challenge was trying to work out how LMMS native plugins work and I hit so many errors in the early stages where I was going to give up.

I realised that everyone is trailblazing and working out things as they are going. There is no 'guidebook' to development. I plan to put my notes together on the forum to help others like myself.

At this point in time:

• The state of the plugin saves to the song file
• Multiple transfer functions can be entered into the text entry
• STFT implemented with Hann window and overlap add
• Parsing for H(s) and H(f) implemented
• Bode plot added
• Stereo added
• Presets / examples added to dial - focus on filters, but should be able to do delays, echos etc
• Dial option one to manually enter a transfer function added
• Tested, hasn't crashed so far
• Aliasing fixed
• User interface implemented
• Bode plot for preset 1 is calculated and drawn based on what is entered
• Bode plots for presets 2 to 17 are drawn based on secondary copies of the equations stored within TransferFunctionControls.cpp . This means that if TransferFunction.cpp's presets change, these will not match. I know this not the best way yet as it does not follow single source of truth

To fix: I’ve had some problems with high pass filtering using common transfer functions. To remove the low end I basically required a brickwall response (preset 3) unless I increased the filter order (exponent in the equation). The high end removal works fine even with low orders. This might be related to my current hop size or overlap amount or how I normalise the STFT, so any guidance here would be appreciated as i've spent a bit of time trying unsuccessfully to solve this.