Here time turns into space: Does consciousness implement the fractional Fourier transform?

I was lucky enough to be introduced to the Fourier transform when I was relatively young. I was an media nerd as a teenager, and as such I spent a significant amount of time editing digital images and audio using software like Adobe Photoshop and Ableton Live. Lacking restraint, I often had a lot of fun combining the filters available in original ways – layering up excessive colour correction or convolution filters in Photoshop, or perhaps chorus, reverb, and equalisation in Ableton.

My life changed when someone whispered the words signal theory in my direction. This lead me to understand that audio and images were just one and two dimensional digital signals, and that mathematically there wasn’t much difference between the amplitude of a sample and the brightness of a pixel. Additionally, many of the one dimensional transformations I was applying had two dimensional equivalents – and vice versa. An audio low-pass filter is also a Gaussian blur when you apply it to an image! Got it.

The second breakthrough came when someone else nudged me towards reading about the Fourier transform. I gave a brief review of the Fourier transform in my previous post:

I found this revelatory. Suddenly I was no longer restricted to thinking about the media I worked with in the time domain or spatial domain – there was a whole new frequency domain I could transform signals into. It felt like I could now see the world from a vantage point in a dimension orthogonal to spacetime.

I began to notice applications of the Fourier transform all around me; how it pervaded our technological reality and how I could use it to reason about the signals I encountered in everyday life. I now understood how the compression codec in my MP3 player and the pitch shifting algorithm in my guitar’s DigiTech Whammy pedal worked.

I was delighted to discover audio software like MetaSynth and later Adobe Audition, which has a spectrogram view which lets you work inside the short-time Fourier transform directly – and even found a two-dimensional Fourier transform plugin for Adobe Photoshop, which was so powerful that I was surprised that Photoshop did not come bundled with such a feature by default.

Later in life I would often find excuses to work frequency domain processing into various creative coding projects, as I found that I could often derive much more original and surprising effects than what was achievable otherwise.

Even later, I find myself involved with consciousness research. I’ve described it better elsewhere – but in brief, I regard this as a process of reverse engineering, which involves observing phenomenology in both sober and altered states of consciousness, and discussing which mathematical tools are most appropriate for modelling its dynamics.

Frequency domain analysis is incredibly useful, so it would be surprising to me if evolution did not find a way of implementing it too. For example, it can be used in object recognition to recognise frequency domain invariants of a given object independent of translation, rotation, and scale. Perhaps it would be fruitful to consider how a biological analogue of the Fourier transform might be implemented – and if so, might we also recognise its phenomenological signature?

In this post, I’m going to discuss how the brain may implement something like the Fourier transform. Specifically, I believe it may use something called the fractional Fourier transform – a generalisation of the regular Fourier transform which is amenable to implementation using wave dynamics.

Firstly, I’ll explain what the fractional Fourier transform is and show how it can appear naturally in a variety of physical systems. Secondly, I’ll then discuss what kind of phenomenological signature we might expect the fractional Fourier transform to leave behind – namely, the characteristic ringing artifacts which arise in the visual field during psychedelic experiences. Thirdly, I’ll explore how the brain might implement such a transform using travelling waves in cortical structures. Then finally, I’ll discuss why this would be computationally advantageous – it would provide a means of implementing the kind of massively parallel pattern recognition operations which would be prohibitively expensive to implement any other way.

What is the fractional Fourier transform?

I’m looking for a biologically plausible implementation, so I was pleased to discover the fractional Fourier transform exists – a continuous domain transform which can smoothly interpolate from time domain or spatial domain to the frequency domain and back again.

Crucially, the fractional Fourier transform can be implemented in wave-based systems – for example, in optics via Fresnel diffraction, or speculatively cortical travelling waves. We’ll get into how that works in a moment, but first I’m going to explain the fractional Fourier transform. We’ll work in two dimensions, as I think that’s more illustrative. We’ll start by looking at the regular two-dimensional Fourier transform:

Note that the lowest frequencies are at the origin, while the highest frequencies are at the border. In this case, the presence of low frequencies generates the bright central cusp. We also see a pair of diagonal streaks indicating the presence of broadband diagonal spatial frequencies in the original signal.

The fractional Fourier transform can be considered to be a rotation in the time-frequency plane, and as such the relevant free parameter is known as the angle α, which is a value in the range 0 to 2π. Though sometimes we use the order a, which is in the range 0 to 4. This is the fractional Fourier transform at four different orders:

Isn’t that neat. I think the ringing artifacts at a = ⅓ are particularly curious. The integer values of a are useful to understand:

0. The signal
1. The Fourier transformed signal
2. The inverse signal
3. The inverse Fourier transformed signal
4. The signal

The fractional Fourier transform loops back on itself at a = 4:

Perhaps an animated version should be more illustrative:

Isn’t this mysterious. If you have noticed the aesthetic resemblance with diffraction patterns, that’s not a coincidence. The fractional Fourier transform can be implemented using a coherent light source and a pair of lenses – and analog frequency domain filters can even be implemented by inserting a mask at the focal plane:

Quantum systems and fractional revivals

The fractional Fourier transform can also be implemented using the Schrödinger wave equation from quantum mechanics. In fact, the Fourier equation and the Schrödinger equation are very deeply related to one another. The Schrödinger wave function is usually expressed in position space ψ(x, t), but to obtain the wavefunction in momentum space ϕ(p, t) you simply apply the Fourier transform. This also shows why the Heisenberg uncertainty principle is mathematically equivalent to the Fourier uncertainty principle.

Anyway, to compute the time evolution of a Schrödinger wave function, it’s natural to move into momentum space – so this is where the Fourier transform comes in. So, the time evolution of the wave function of a particle in free space is equivalent to the Fresnel transform, which is itself equivalent to the fractional Fourier transform except that it spreads out in space as time goes on. For a particle in a quadratic potential well, the time evolution is exactly the fractional Fourier transform. Other potential wells are more complicated, but may also be approximated locally by similar Fourier-type transformations.

Such systems may return to their original states periodically, despite undergoing complicated spreading and self-interference along the way. The period on which they repeat themselves is known as the revival time, and in some systems, the wave function passes through exact fractional Fourier transforms of the original signal along the way. In other systems, fractional revivals may instead produce multiple smaller copies of the original wave packet rather than a simple Fourier relation.

Revivals are a generic wave phenomenon and are not unique to quantum systems – they show up in classical optical and acoustic systems as well. I found this to be an encouraging prospect – suddenly the class of systems which could implement the fractional Fourier transform became much broader. I began experiencing imaginal visions, of the structure of the universe reflecting and repeating itself all the way down, dispersing and reviving on an eternal loop, at every scale, forever…

Talbot carpets and whispering gallery modes

Another way of viewing the fractional Fourier transform is to think of its order as corresponding to the propagation distance in free-space diffraction. For a plane wave passing through a periodic grating, the field at distance z is given by the fractional Fourier transform of the grating’s transmission function at some order a proportional to z. Plotting the field intensity as a function of position and distance then gives you a pattern known as a Talbot carpet:

These patterns are of real concern when one considers the whispering gallery modes that arise close to the surface of fiber optics:

Mostly I am highlighting Talbot carpets here because they round out our tour of the fractional Fourier transform’s ubiquity. From the quantum revivals in potential wells to the interference patterns in fiber optics, it seems that the fractional Fourier transform is kind of just lying around in nature, waiting to be noticed – not as some exotic mathematical object, but as a fundamental organisational principle underlying the evolution of wave systems in time.

Does the fractional Fourier transform show up in subjective experience?

We think that it’s fairly straightforward to make the case that the plain Fourier transform shows up in subjective experience. I presented an example of frequency domain attentional modulation in my previous post:

We believe this becomes easier to observe in altered states. Have you ever taken a small amount of LSD and found yourself engrossed in a pattern on the wallpaper or carpet? Consider what frequency domain transform that might correspond to – are specific spectral peaks being stabilised or amplified, while spatial precision is reduced? Andrés explored uncertainty principle qualia in great detail on The DemystifySci Podcast last year:

What I have found is that there is something akin to the Heisenberg uncertainty principle going on, having to do with what kind of patterns you can perceive. And essentially there’s a trade-off between having more information about the spatial domain – like the position of things – versus having more information about the frequency domain – like the frequencies and vibrations, and the spectrum, like what’s happening in different time scales.

And so one extreme – this happens in high dose LSD, but it’s also a jhāna effect – on the one extreme you can concentrate all the information into position, and when that happens you collapse into one point. It’s a very strange experience – you just become one point, you’re not a person anymore, it feels as if all of your attention is just one point. That’s a real state of consciousness – the Buddhists talk about it – it’s very peculiar.

But then also, the complete opposite – you can concentrate all of the information, or all of the sampling into frequency. So when you do that, you turn into a vibe. You stop being anywhere and you’re just a vibration. You’re just a wave – it’s very emotional, it’s a very different type of experience.

So, my guess is that these are the extremes. So when you’re hyper-concentrated, you can do this or you can do that – I think normal states of consciousness are a mixture. Like we have several attentional centers, and they’re sampling both spatial and temporal information – both momentum and position – and the precise balance that you choose between them corresponds to your personality and your state of consciousness. In a way, your way of being is intimately related with how you sample the world. There’s always going to be something you’re missing – because if you just focus on frequency you’re gonna lose the spatial information, and vice versa. So I wonder if that is connected – no entity can actually know a space fully – it has to do a trade-off, and there’s no way around it.

However, I am interested in looking for signatures of the fractional Fourier transform, which is a little more specific and complicated. I’ve shown a handful of people the visuals I generated for this post, and in the process of doing so I’ve received some amount of positive feedback.

People have compared these renderings to a variety of unusual phenomena, including k-holes, LSD, meditative cessations, and even the arising and passing of consciousness frames. I find this encouraging, although the states described tend to be quite extreme – only accessible through high doses of drugs or extensive training in meditation. I also don’t think this kind of informal phenomenology is repeatable enough to stake any strong claims on – I’m more interested in finding accessible, low level, simple phenomena which are amenable to study using psychophysics experiments.

Fresnel fringes and ringing artifacts

In the fractional Fourier transform demonstration earlier, did you notice the distinctive ringing artifacts around the edges of objects? These are known as Fresnel fringes. Characteristically, the spacing of these fringes decreases with distance from the edge, such that the fringes become progressively finer. This pattern arises from the quadratic phase terms underlying the Fresnel diffraction formalism, which also underpins optical implementations of the fractional Fourier transform.

These show up in any wave based media in which Fresnel diffraction applies – not just in optics, but also in acoustic and quantum wave systems. Ringing artifacts also show up in my visual field when I’m on a moderate dose of a serotonergic psychedelic – for instance one or two grams of psilocybin mushrooms. You might also call these auras or haloes.

I’ve spent some time looking at ringing artifacts while on psychedelics, and I have found that – at least at low doses – some psychedelics generate coarser or finer ringing artifacts than others. In order from coarse to fine, 2C-B, LSD and psilocybin generate progressively tighter ringing artifacts.

The key aspect is whether or not the fringes become finer with distance from the edge. If this is the case, then I believe this would be a strong argument that dynamics analogous to Fresnel optics are somehow playing out within the brain – which would mean that the brain may provide a viable substrate for analog computations like the fractional Fourier transform. I’ve shown these animations to some friends who have confirmed that the ringing artifacts that they see while on psychedelics resemble the ones in these renderings – and that the fringes do indeed become finer with distance from the edge.

Personally, it’s been some time since I last took a decent dose of psychedelics – and the last time I did so, I wasn’t keeping an eye on the ringing artifacts. I’ll have to set some time aside to take a couple grams of mushrooms and check for myself soon whether they resemble Fresnel fringes. If I find this is the case, then I should be motivated to construct some kind of questionnaire or psychophysics tool which could be used to verify whether other people also see the fingerprint of Fresnel diffraction in their subjective experience.

How might the fractional Fourier transform show up in the brain?

There’s a baseline assumption which so far I’ve neglected to address, which is that representations in the mind are structurally similar to the objects in the world which they represent – sufficiently so at least that a transformation of some kind on one can approximate a transformation on the other.

There have been a number of recent successes in reading video and speech representations from the brain, but generally this involves an intermediary decoding layer where machine learning techniques are used to interpret neuroimaging signals. I might be opinionated here when I claim that these results are simultaneously impressive and unsatisfying. Perhaps interpretability researchers might sympathise when I say I think the assumption that neural representations will be forever inscrutable – requiring essentially a neural network to read a neural network – is an overly pessimistic assumption, and the fact that we haven’t uncovered the true representations yet is more a factor of insufficient neuroimaging fidelity than the result of messy, illegible complexity at every layer in the stack. Why should illegibility be the default assumption, when there are computational benefits to well-structured representations?

Travelling waves and spatiotemporal dynamics

A recent paper from the Kempner Institute makes the case that the spatiotemporal dynamics of traveling waves could provide a reasonable basis for neural representations, as these dynamics can be recruited to encode the symmetries of the world as conserved quantities. From A Spacetime Perspective on Dynamical Computation in Neural Information Processing Systems (Keller et al., 2024):

There is now substantial evidence for traveling waves and other structured spatiotemporal recurrent neural dynamics in cortical structures; but these observations have typically been difficult to reconcile with notions of topographically organized selectivity and feedforward receptive fields. We introduce a new ‘spacetime’ perspective on neural computation in which structured selectivity and dynamics are not contradictory but instead are complimentary. We show that spatiotemporal dynamics may be a mechanism by which natural neural systems encode approximate visual, temporal, and abstract symmetries of the world as conserved quantities, thereby enabling improved generalization and long-term working memory.

Keller argues that rather than simply learning features that are invariant to transformations, the brain may be explicitly learning the symmetries themselves:

To begin to understand the relationship between spatiotemporally structured dynamics and symmetries in neural representations, it is helpful take a step back and understand more generally what makes a ‘good’ representation. Consider a natural image – a full megapixel array representing an image is very high dimensional, but the parts of the image that need to be extracted are generated by a much lower dimensional process. For example, imagine a puppet that is controlled by nine strings (one to each leg, one to each hand, one to each shoulder, one to each ear for head movements, and one to the base of the spine for bowing); the state of the puppet could be transmitted to another location by the time course of nine parameters, which could be reconstructed in another puppet. It is therefore more efficient to represent the world in terms of these lower-dimensional factors in order to be able to reduce the correlated structural redundancies in the very high-dimensional data.

At a high level, one can understand the goal of ‘learning’ with deep neural networks as attempting to construct these useful factors that are abstractions of the high-resolution degrees of freedom in sensory inputs, exactly like inferring the control strings of the universal puppet master. This is similar to the way that the renormalization group procedure in physics compresses irrelevant degrees of freedom by coarse graining to reveal new physical regularities at larger spatiotemporal scales, and is a model for how new laws emerge at different spatial and temporal scales. As agents in the complex natural world, we seek to represent our surroundings in terms of useful abstract concepts that can help us predict and manipulate the world to enhance our survival.

One early idealized view of how the brain might be computing such representations is by learning features that are invariant to a variety of natural transformations. This view was motivated by the clear ability of humans to rapidly recognize objects despite their diverse appearance at the pixel level while undergoing a variety natural geometric transformations, and further by the early findings of Gross et al. (1973) that individual neurons in higher levels of the visual hierarchy responded selectively to specific objects irrespective of their position, size, and orientation.

Another Kempner Institute preprint puts these ideas into practice, demonstrating that travelling waves can be used to discover spectral signatures which are useful for shape classification. From Traveling Waves Integrate Spatial Information Through Time (Jacobs et al., 2025):

Traveling waves of neural activity are widely observed in the brain, but their precise computational function remains unclear. One prominent hypothesis is that they enable the transfer and integration of spatial information across neural populations. However, few computational models have explored how traveling waves might be harnessed to perform such integrative processing.

Drawing inspiration from the famous “Can one hear the shape of a drum?” problem – which highlights how normal modes of wave dynamics encode geometric information – we investigate whether similar principles can be leveraged in artificial neural networks. Specifically, we introduce convolutional recurrent neural networks that learn to produce traveling waves in their hidden states in response to visual stimuli, enabling spatial integration. By then treating these wave-like activation sequences as visual representations themselves, we obtain a powerful representational space that outperforms local feed-forward networks on tasks requiring global spatial context.

As the travelling waves dissipate, each neuron has the opportunity to accumulate spectral information about the shape of the structure it is contained within. Perhaps an animation will be illustrative – from the authors’ GitHub page:

In this manner, every part may accumulate a representation of the whole, and the accumulated spectral qualities can be used to infer what shape the part is contained within:

I recommend checking out Mozes Jacobs’ video presentation, as he has even generated sounds representative of each shape being classified. Incredibly cool:

This is extremely speculative, but I feel that there’s something analogous to a wave based implementation of the fractional Fourier transform going on here – only, what’s being calculated is not so much the spectral qualities of the signal as the spectral qualities of the resonant cavity it is contained within. I currently lack the mathematical background necessary to explore a reconciliation between these two ideas.

Entorhinal cortex and visual cortex

I did also search for other literature presenting evidence for the Fourier transform in neural processes. Orchard et al. (2013) propose that the grid cells in the entorhinal cortex and place cells in the hippocampus reconstruct a spatial map of an animal’s environment using an inverse Fourier transform – but most fascinating to me was a very old electrode study on cats and macaque monkeys proposing that neurons in the visual cortex actually respond to the spectral components of visual stimuli. As always, I have a soft spot for weird old neuroscience papers, so I include it here. From Responses of striate cortex cells to grating and checkerboard patterns (De Valois et al., 1979), various visual stimuli and their two-dimensional Fourier spectra:

Note how the harmonics for the square wave grating are arranged at 0° to the origin, whereas the harmonics for the checkerboard are arranged at 45° to the origin. The authors propose that if orientation selective neurons actually respond to a spectral basis rather than orientation alone, then we would expect neurons which fire when presented with a grating rotated by 0° to also fire when presented with a checkerboard rotated by 45°. Well, that is exactly what they found:

The consensus in modern neuroscience is that neurons in the visual cortex actually respond to Gabor wavelet receptive fields, not spectral components. A Gabor function is constructed by multiplying a Gaussian function by a sine wave – and represents the ideal trade-off between spatial and spectral domain uncertainty. In other words, Gabor wavelet receptive fields are already performing a kind of Fourier analysis, extracting information about both the spatial and frequency domain qualities of visual features simultaneously.

The important difference between the Fourier transform model and the Gabor receptive field model is as follows: In the Fourier model, neurons would respond to frequency components regardless of where they appear in the viusal field, whereas in the Gabor model, neurons would respond to frequency components at a specific location. The grating and checkerboard experiments described above suggest that the visual system may track the global frequency decomposition of a scene, not just local frequency patches. I’m not sure how to reconcile this with mainstream understanding or why the findings of this paper are not more widely known. Did nobody ever repeat this experiment – do all modern studies use gratings alone?

If something like the Fourier transform model does turn out to be the case, I’d be inclined to wonder if other sensory cortices may be performing analogous transforms on incoming sensory information – and maybe the entire brain is composed of an complex hierarchy of such structures? The auditory cortex is the obvious candidate for analysis – but perhaps this could also explain the mysterious gaps found on our representation of the somatosensory homunculus?

What is the computational utility of the fractional Fourier transform?

Alright, so what is all this for? As I have written previously, I think a big part of what consciousness is doing is performing a massively parallel analogue self-convolution on the contents of awareness, scouring sensory information for any patterns it can find in the pursuit of reifying a parsimonious world model in which every part still has an influence on the whole. To understand why this would be expensive to implement, consider what convolution actually does. For every point in a signal, you have to compare it with every other point – which adds up to O(n²) operations in total. For a high resolution visual field with millions of “points”, this quickly becomes intractable.

The convolution theorem tells us that the convolution of two time or spatial domain signals is the same as their product in the frequency domain. So, we first take the Fourier transform our signal, multiply the frequency components together, and then take the inverse Fourier transform to convert it back. The multiplication operation only requires O(n) operations, but the Fourier transforms require O(n log n). This is a sizeable improvement, but it’s still computationally demanding, and evolution would still need to find some way to hard code the point-to-point wiring required to implement the Fourier transform into our genomes. This is why the fractional Fourier transform is so appealing – it can be implemented using simple wave dynamics without requiring any hard-wired connections. The waves just do their thing, and the Fourier transform arises naturally from the physics.

Additionally, if reports from the meditators I know are to be believed, upon investigation it appears that consciousness refreshes itself at a rate of around 40 Hz. This isn’t just idle speculation – experienced meditators across multiple contemplative traditions who have cultivated high sensory clarity often report being able to perceive the discrete, flickering nature of conscious experience arising and passing away, often at a rate of roughly forty times per second.

Is one consciousness frame equivalent to one cycle of the fractional Fourier transform? If this is the case, it would suggest that each moment or “frame” of experience corresponds to a complete rotation through the time-frequency plane and back again. How might we falsify such a hypothesis? Might certain drugs alter the refresh rate in observable or measurable ways?

If this turns out to be true, and if one is willing to put an estimate on the size of the contents of awareness – then we can start putting a lower bound on the number of operations required in order to process the contents of awareness digitally. This said – does it even make sense to talk of computational complexity when discussing analogue computers? In any case, I believe that the fractional Fourier transform offers a plausible mechanism by which this computationally expensive process can be implemented using wave dynamics.

Open questions

As usual, I have a handful of excess tangents which don’t quite fit into the body of the post, but which I’d still like to explore briefly before concluding the post entirely. Here goes:

How would you use the fractional Fourier transform to implement higher order pattern recognition?

Let’s use a musical example. Imagine I take the Fourier transform of a single sine wave – the note A4, which would cause a single spike in the frequency spectrum at 440 Hz. This is all fine if we just want to recognise individual notes, but what if we’d like to recognise intervals on top of that? If I then play a C5, that would cause a spike at 660 Hz, a perfect fifth above the other note – a ratio of 3:2. If we’d like to be able to recognise perfect fifths at any frequency – we’re going to need a secondary transform or convolution operation on top of this one.

I have a weird intuition that in addition to using a hierarchy of such optical elements, either the intermediary orders in the fractional Fourier transform or nonlinear optical effects can be recruited to perform the desired recursive pattern recognition. I still need to explore this idea fully – but helpfully, it seems that such Indra’s net effects get stronger when under the influence of psychedelics, making this amenable to phenomenological study.

If consciousness is implementing the fractional Fourier transform, why do we only experience the initial spatial order, and not the other fractional frequency orders?

Perhaps we normally do only experience the spatial order (a = 0), but it’s possible to dial attention into other stages, or we simply don’t remember them. When this happens is when we experience the weird ringing artifacts – or other, much stranger effects.

As for the upside down spatial order (a = 2), perhaps it’s impossible to tell the difference? Or, as Steven Lehar proposed, nonlinear optical effects could also be used to create phase conjugate mirrors, which would bounce the waves right back to where they came from. This is speculative, but this would mean that we would only ever experience fractional orders in the range 0 to 1.

I’m also not sure whether to expect that we would experience this process cycling – rather, due to the path integral nature of optics, perhaps we’d experience the entire process all at once.

How does nonlinear optics fit into this picture?

For energy to leave such a wave system or for it to have memory, you effectively need some kind of nonlinear effect – the waves need to have some kind of persistent effect on their substrate, for instance modifying the properties of an underlying neuron where they form a high-amplitude cusp.

Nonlinear optical effects can get quite weird, however – this is a topic that is extraordinarily broad and dense. Consider the Kerr self-focusing effect – related to the third nonlinear coefficient, χ⁽³⁾ – where if the amplitude of the radiation increases above a certain value it changes the refractive index of the medium, focusing more radiation into the same location in a feedback loop. I believe this can be a real problem in high-power fiber optics.

I wanted to simulate what happens when I experimented with χ⁽³⁾, as I suspect that this is what 5-HT2A receptor agonists do. The results very quickly went haywire:

Weston Beecroft has published a great nonlinear optical sandbox which can be fun to play around with. Over the coming months, I’d like to spend more time wrapping my head around what is possible using nonlinear optics.