## Wednesday, April 30, 2014

### regularization

"generalizing from finite samples in extremely high dimensions" is an amazing problem. think of life expectancy. let's say men born today expect to live 82 years and women expect to live 84 years (don't know the actual numbers). that's based on a lot of samples of men and women (plus some projections, but let's say for now it was just based on raw statistics of past events). OK. so if you're a man, your best guess for how long you'll live is 82 years, and if a woman, your best guess is 84 years. in other words, if you were doing optimal decision-making or whatever, this is the number you should factor in (well actually the variance could be asymmetric so you ought to integrate over the whole distribution but let's ignore that for now too). OK, *but*, let's suppose the expectancy for white-men is 85 years and for non-white-men is 77 years. maybe for some reason it's reversed for women; white-women 82 years and non-white-women 86 years. NOW, if you know you're a white man, you should use 85 years as your best guess. OK, but what if you factor in height? smoking? educational history? city of birth? environmental toxin exposure? genetics? .... pretty soon, there won't even be a *single sample* of experience to draw on, to make an estimate of the life expectancy in your particular category.

one way around this problem is to assume that each factor operates linearly and independently. then, you solve a big linear regression problem, and linearly interpolate/extrapolate to points in the space where you don't have any samples. note that even under these simplifying assumptions, there are serious problems. for example, when you have many many dimensions, some will by chance have extreme slopes -- which will push your extrapolated prediction into crazy values. but even ignoring those problems, there's the bigger problem that the factors aren't actually linear or independent. they interact very strongly in fact: sometimes one factor might even reverse its direction of effect depending on another factor. so the linear regression is not a good solution.

what can you do? you have to regularize the problem. this is, you have to use some prior knowledge to drastically pare back the number of effective dimensions. for example, you might know some variables are strongly correlated, so you can treat them as one. or you might know that some variables are dominated by others, so you omit the weak variables. or you might know that the underlying function ought to be at least locally smooth, so you impose a smoothness constraint.

but what is the right regularization? brains are somehow really good at this (algorithms are getting better quickly though..)

i was just thinking that the regularizing prior knowledge could be a good place to fit in scale-invariance and holo-stuff. i haven't read enough, but i haven't seen yet much in machine learning on using a single learning framework to accommodate all different kinds/levels of data. i imagine that's what the brain is doing (maybe it's related to the relatively "unitary" consciousness that we seem have subjectively) -- because we're essentially shifting around the focus of more or less the same global model to apply to massively disparate "kinds of things". so we have more samples to draw on for any given problem. we can even turn this global model partially on itself, which perpetually gives us even more samples and might also produce other weird features.

somehow, the circuitry of the cortex must be cleverly set up to do this kind of learning over space and time. (like hawkins' and others' "hierarchical temporal memories" type ideas).

in a friston-type framework, i'm picturing that the structure of the organism at all levels already encodes a deep regularization (for example RNA only interacts with some molecules). since there's no separation between what is "inference" and what is just dynamical structure of the "inferring entity" itself. if every dynamical system can be thought of as doing prediction (which means vastly generalizing from finite samples), then its whole structure from the ground-up is scale-invariantly encoding regularization for all the external states it's exposed to.

## Wednesday, April 23, 2014

### new ideas

Part 1

Some models suggest that the basal ganglia are used to select between actions. Some candidate actions are generated, and the circuitry of the basal ganglia allows one to win and be executed (like Jeff Wickens). I wondered if the basal ganglia are also used to select between "internal actions", like what to think about, what to attend to, etc. This could be connected to habits of mind.

Part 2

I was thinking about Karl Friston's view of sleep. If I understood correctly, sleep is a time of lower top-down precision, when the experience collected during wakefulness can be used to drive big changes to the world-model. It makes sense to isolate your brain from your muscles when you're doing this deep restructuring: when you're awake you need high top-down precision to make your best active predictions.

[As a side note, I wonder how this fits with the data about interstitial volume increasing during sleep to allow "clean up" of metabolic products. Maybe a sleep-state is simply a useful time to do multiple unrelated things. Or maybe the physiological implementation of precision actually has to do with neuron-glia interactions and interstitial space (e.g. glutamate reuptake).]

The most obvious thing that happens during sleep, physiologically, is change in oscillatory activity. High-frequency activity mostly drops away, and low-frequency oscillations become prominent.

Does low-frequency activity correspond to the "subtle" state described by Eastern mystics?

In terms of oscillatory activity, a child's brain looks like a sleeping adult's brain. Roughly, a deeper stage of adult sleep corresponds to an earlier stage of child development. In childhood we also have high plasticity. This is when things seem really meaningful, like they can in dreams.

The adult mind can be pretty locked in place. Old feelings, even from early childhood, are still there somewhere, but maybe don't find much expression. Really deep *meaning* seems to be locked in there somewhere, too. Things can seem a bit abstract until it's in your gut. Truly believing in love isn't something the rigid adult mind is very good at.

What about meditation and the subtle stage of development?

Maybe meditation peels the layers off. Perhaps during development, something happened to hurt that level of the self: suffering (free energy) caused by direct contact with not-self. That then becomes a scar or a rigidity: some loss of dynamic flexibility like Eve Marder's ion channels. Specifically it's a hyper-assertion of the self-pattern at that level.

Meditation brings awareness to the thing that's creating the rigidity. Bringing awareness is the same as nesting the thing in a larger context, allowing it to healthily integrate and discharge some of its exclusivity. In meditation we also see slow oscillatory states.

This suggests that top-down precision is something we should look at in multiple layers. There might be a different kind of precision every time we peel off a layer.

Something like the aPFC or hippocampus might correspond to top-levels of the adult hierarchy. Interestingly, you also grow to a bigger model as you relax levels, because that slice of the self gets integrated with a bigger context. This might be like how unexpected uncertainty could convert to expected uncertainty as the model grows.

A tricky issue is how to reconcile this with the psychedelic state. Psychedelic states have been likened to dream states, and sometimes include a feeling of vast lovingness and meaningfulness. Is this playing the same role as sleep? If so, do psychedelic states show the same oscillatory signature?

Some studies have described increases in low-frequency oscillatory power with psychedelics, but more recent work (Carhart-Harris et al, 2013) showed decreases in oscillatory power across the spectrum (in default mode). Carhart-Harris suggests this corresponds to "destabilization" of brain activity, possibly allowing it to leave attractor states that it's been stuck in.

Part 3

Oscillations, again. Subjectively, people sometimes describe "the present moment". To some extent, every moment seems to be the "same moment". It's always the present moment. Of course in some ways this is vacuously true; it follows from the definition of "now". But I think it might also be describing some aspects of how the brain works.

Clark and I talked about "ontologeny" (he calls it "microgeny"), which is the "waking up" of the universe in each moment. It's possible that it sort of ultra-quickly recapitulates all of the development of the universe, up until now.

Clark wants to extend this to fundamental physics, which I don't necessarily disagree with, but being more conservative for a minute, let's say this is just an attribute of the brain. Within a brain rhythm, you keep doing the same thing over and over again. If this brain rhythm is one signature of the dynamic quasi-stable attractor state you're in, then it's probably repeating the same pattern over and over again. That's "you" -- the stability of that attractor is what makes up the coherence and consistency of your experience. But it's constantly "refreshing", going through all the phase-angles of that space. So in every moment it would feel like you're constantly waking up to that moment as you go through that same cycle. The attractor can gradually (or sometimes quickly) change, both through its own unstable dynamics and through sensory inputs, and there's also long-term plasticity. A wakeful attractor state would have some signature in alpha frequencies in the brain, for example.

This could explain why sometimes it seems like mental events are perpetually happening "before" the present moment of awareness. A simplistic explanation would be that there is some particular phase-angle that actually corresponds to the "waking up" or the present moment (and I suspect that's true in some partial ways), but a more complete explanation probably involves interactions at all the phase-angles. In either explanation, bringing those things into awareness (into the present moment) might involve an attentionally-driven shift in the relative phases of firing of different neural populations.

For me, subjectively, this corresponds to things that I feel were always "there" inside me somewhere. With some attention they can be brought up to "near" awareness, but it sometimes feels like they're still lurking before the edge of the present moment. Sustained attention brings them into the present.

I think this is also a key to the "hierarchical temporal memory" idea from Jeff Hawkins. Microcircuits in the brain must be set up to dynamically mirror aspects of the sensory inputs / the world. But as you go up the hierarchy of the brain, they extract structure. A lot of work has focused on the "spatial" structure they extract (like going from spots to oriented lines to edges to objects to recognizing people, etc), but they are implemented as oscillatory dynamical systems, so it makes sense that the real mirroring and abstraction is deeply spatiotemporal.

[Also, I think it's worth mentioning that we often use the visual system as an example for this type of thing, but a huge area of the brain is devoted to somatosensory, visceral, vestibular, etc processing. Further, I only talked about the sensory side of the hierarchy but the brain can at least as well be thought of as organized hierarchically around action, on the motor side. There are always rhythms in motion, too. Together I think this fits embodied cognition and the body-mind naturally into the framework.]