RESUMEN
With the advancement of technology, signal modulation types are becoming increasingly diverse and complex. The phenomenon of signal time-frequency overlap during transmission poses significant challenges for the classification and recognition of mixed signals, including poor recognition capabilities and low generality. This paper presents a recognition model for the fine-grained analysis of mixed signal characteristics, proposing a Geometry Coordinate Attention mechanism and introducing a low-rank bilinear pooling module to more effectively extract signal features for classification. The model employs a residual neural network as its backbone architecture and utilizes the Geometry Coordinate Attention mechanism for time-frequency weighted analysis based on information geometry theory. This analysis targets multiple-scale features within the architecture, producing time-frequency weighted features of the signal. These weighted features are further analyzed through a low-rank bilinear pooling module, combined with the backbone features, to achieve fine-grained feature fusion. This results in a fused feature vector for mixed signal classification. Experiments were conducted on a simulated dataset comprising 39,600 mixed-signal time-frequency plots. The model was benchmarked against a baseline using a residual neural network. The experimental outcomes demonstrated an improvement of 9% in the exact match ratio and 5% in the Hamming score. These results indicate that the proposed model significantly enhances the recognition capability and generalizability of mixed signal classification.
RESUMEN
There is much interest in the topic of partial information decomposition, both in developing new algorithms and in developing applications. An algorithm, based on standard results from information geometry, was recently proposed by Niu and Quinn (2019). They considered the case of three scalar random variables from an exponential family, including both discrete distributions and a trivariate Gaussian distribution. The purpose of this article is to extend their work to the general case of multivariate Gaussian systems having vector inputs and a vector output. By making use of standard results from information geometry, explicit expressions are derived for the components of the partial information decomposition for this system. These expressions depend on a real-valued parameter which is determined by performing a simple constrained convex optimisation. Furthermore, it is proved that the theoretical properties of non-negativity, self-redundancy, symmetry and monotonicity, which were proposed by Williams and Beer (2010), are valid for the decomposition Iig derived herein. Application of these results to real and simulated data show that the Iig algorithm does produce the results expected when clear expectations are available, although in some scenarios, it can overestimate the level of the synergy and shared information components of the decomposition, and correspondingly underestimate the levels of unique information. Comparisons of the Iig and Idep (Kay and Ince, 2018) methods show that they can both produce very similar results, but interesting differences are provided. The same may be said about comparisons between the Iig and Immi (Barrett, 2015) methods.
RESUMEN
With this follow-up paper, we continue developing a mathematical framework based on information geometry for representing physical objects. The long-term goal is to lay down informational foundations for physics, especially quantum physics. We assume that we can now model information sources as univariate normal probability distributions N (µ, σ0), as before, but with a constant σ0 not necessarily equal to 1. Then, we also relaxed the independence condition when modeling m sources of information. Now, we model m sources with a multivariate normal probability distribution Nm(µ,Σ0) with a constant variance-covariance matrix Σ0 not necessarily diagonal, i.e., with covariance values different to 0, which leads to the concept of modes rather than sources. Invoking Schrödinger's equation, we can still break the information into m quantum harmonic oscillators, one for each mode, and with energy levels independent of the values of σ0, altogether leading to the concept of "intrinsic". Similarly, as in our previous work with the estimator's variance, we found that the expectation of the quadratic Mahalanobis distance to the sample mean equals the energy levels of the quantum harmonic oscillator, being the minimum quadratic Mahalanobis distance at the minimum energy level of the oscillator and reaching the "intrinsic" Cramér-Rao lower bound at the lowest energy level. Also, we demonstrate that the global probability density function of the collective mode of a set of m quantum harmonic oscillators at the lowest energy level still equals the posterior probability distribution calculated using Bayes' theorem from the sources of information for all data values, taking as a prior the Riemannian volume of the informative metric. While these new assumptions certainly add complexity to the mathematical framework, the results proven are invariant under transformations, leading to the concept of "intrinsic" information-theoretic models, which are essential for developing physics.
RESUMEN
We build on the view of the Exact Renormalization Group (ERG) as an instantiation of Optimal Transport described by a functional convection-diffusion equation. We provide a new information-theoretic perspective for understanding the ERG through the intermediary of Bayesian Statistical Inference. This connection is facilitated by the Dynamical Bayesian Inference scheme, which encodes Bayesian inference in the form of a one-parameter family of probability distributions solving an integro-differential equation derived from Bayes' law. In this note, we demonstrate how the Dynamical Bayesian Inference equation is, itself, equivalent to a diffusion equation, which we dub Bayesian Diffusion. By identifying the features that define Bayesian Diffusion and mapping them onto the features that define the ERG, we obtain a dictionary outlining how renormalization can be understood as the inverse of statistical inference.
RESUMEN
We develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training. By examining the underlying high-dimensional probabilistic models, we reveal that the training process explores an effectively low-dimensional manifold. Networks with a wide range of architectures, sizes, trained using different optimization methods, regularization techniques, data augmentation techniques, and weight initializations lie on the same manifold in the prediction space. We study the details of this manifold to find that networks with different architectures follow distinguishable trajectories, but other factors have a minimal influence; larger networks train along a similar manifold as that of smaller networks, just faster; and networks initialized at very different parts of the prediction space converge to the solution along a similar manifold.
RESUMEN
Exponential families are statistical models which are the workhorses in statistics, information theory, and machine learning, among others. An exponential family can either be normalized subtractively by its cumulant or free energy function, or equivalently normalized divisively by its partition function. Both the cumulant and partition functions are strictly convex and smooth functions inducing corresponding pairs of Bregman and Jensen divergences. It is well known that skewed Bhattacharyya distances between the probability densities of an exponential family amount to skewed Jensen divergences induced by the cumulant function between their corresponding natural parameters, and that in limit cases the sided Kullback-Leibler divergences amount to reverse-sided Bregman divergences. In this work, we first show that the α-divergences between non-normalized densities of an exponential family amount to scaled α-skewed Jensen divergences induced by the partition function. We then show how comparative convexity with respect to a pair of quasi-arithmetical means allows both convex functions and their arguments to be deformed, thereby defining dually flat spaces with corresponding divergences when ordinary convexity is preserved.
RESUMEN
In this work, we explore information geometry theoretic measures for characterizing neural information processing from EEG signals simulated by stochastic nonlinear coupled oscillator models for both healthy subjects and Alzheimer's disease (AD) patients with both eyes-closed and eyes-open conditions. In particular, we employ information rates to quantify the time evolution of probability density functions of simulated EEG signals, and employ causal information rates to quantify one signal's instantaneous influence on another signal's information rate. These two measures help us find significant and interesting distinctions between healthy subjects and AD patients when they open or close their eyes. These distinctions may be further related to differences in neural information processing activities of the corresponding brain regions, and to differences in connectivities among these brain regions. Our results show that information rate and causal information rate are superior to their more traditional or established information-theoretic counterparts, i.e., differential entropy and transfer entropy, respectively. Since these novel, information geometry theoretic measures can be applied to experimental EEG signals in a model-free manner, and they are capable of quantifying non-stationary time-varying effects, nonlinearity, and non-Gaussian stochasticity presented in real-world EEG signals, we believe that they can form an important and powerful tool-set for both understanding neural information processing in the brain and the diagnosis of neurological disorders, such as Alzheimer's disease as presented in this work.
RESUMEN
Despite their remarkable performance, deep learning models still lack robustness guarantees, particularly in the presence of adversarial examples. This significant vulnerability raises concerns about their trustworthiness and hinders their deployment in critical domains that require certified levels of robustness. In this paper, we introduce an information geometric framework to establish precise robustness criteria for l2 white-box attacks in a multi-class classification setting. We endow the output space with the Fisher information metric and derive criteria on the input-output Jacobian to ensure robustness. We show that model robustness can be achieved by constraining the model to be partially isometric around the training points. We evaluate our approach using MNIST and CIFAR-10 datasets against adversarial attacks, revealing its substantial improvements over defensive distillation and Jacobian regularization for medium-sized perturbations and its superior robustness performance to adversarial training for large perturbations, all while maintaining the desired accuracy.
RESUMEN
This work addresses J.A. Wheeler's critical idea that all things physical are information-theoretic in origin. In this paper, we introduce a novel mathematical framework based on information geometry, using the Fisher information metric as a particular Riemannian metric, defined in the parameter space of a smooth statistical manifold of normal probability distributions. Following this approach, we study the stationary states with the time-independent Schrödinger's equation to discover that the information could be represented and distributed over a set of quantum harmonic oscillators, one for each independent source of data, whose coordinate for each oscillator is a parameter of the smooth statistical manifold to estimate. We observe that the estimator's variance equals the energy levels of the quantum harmonic oscillator, proving that the estimator's variance is definitively quantized, being the minimum variance at the minimum energy level of the oscillator. Interestingly, we demonstrate that quantum harmonic oscillators reach the Cramér-Rao lower bound on the estimator's variance at the lowest energy level. In parallel, we find that the global probability density function of the collective mode of a set of quantum harmonic oscillators at the lowest energy level equals the posterior probability distribution calculated using Bayes' theorem from the sources of information for all data values, taking as a prior the Riemannian volume of the informative metric. Interestingly, the opposite is also true, as the prior is constant. Altogether, these results suggest that we can break the sources of information into little elements: quantum harmonic oscillators, with the square modulus of the collective mode at the lowest energy representing the most likely reality, supporting A. Zeilinger's recent statement that the world is not broken into physical but informational parts.
RESUMEN
Explainable Artificial Intelligence (XAI) and acceptable artificial intelligence are active topics of research in machine learning. For critical applications, being able to prove or at least to ensure with a high probability the correctness of algorithms is of utmost importance. In practice, however, few theoretical tools are known that can be used for this purpose. Using the Fisher Information Metric (FIM) on the output space yields interesting indicators in both the input and parameter spaces, but the underlying geometry is not yet fully understood. In this work, an approach based on the pullback bundle, a well-known trick for describing bundle morphisms, is introduced and applied to the encoder-decoder block. With constant rank hypothesis on the derivative of the network with respect to its inputs, a description of its behavior is obtained. Further generalization is gained through the introduction of the pullback generalized bundle that takes into account the sensitivity with respect to weights.
RESUMEN
This paper introduces assignment flows for density matrices as state spaces for representation and analysis of data associated with vertices of an underlying weighted graph. Determining an assignment flow by geometric integration of the defining dynamical system causes an interaction of the non-commuting states across the graph, and the assignment of a pure (rank-one) state to each vertex after convergence. Adopting the Riemannian-Bogoliubov-Kubo-Mori metric from information geometry leads to closed-form local expressions that can be computed efficiently and implemented in a fine-grained parallel manner. Restriction to the submanifold of commuting density matrices recovers the assignment flows for categorical probability distributions, which merely assign labels from a finite set to each data point. As shown for these flows in our prior work, the novel class of quantum state assignment flows can also be characterized as Riemannian gradient flows with respect to a non-local, non-convex potential after proper reparameterization and under mild conditions on the underlying weight function. This weight function generates the parameters of the layers of a neural network corresponding to and generated by each step of the geometric integration scheme. Numerical results indicate and illustrate the potential of the novel approach for data representation and analysis, including the representation of correlations of data across the graph by entanglement and tensorization.
RESUMEN
We propose to use a particular case of Kaniadakis' logarithm for the exploratory analysis of compositional data following the Aitchison approach. The affine information geometry derived from Kaniadakis' logarithm provides a consistent setup for the geometric analysis of compositional data. Moreover, the affine setup suggests a rationale for choosing a specific divergence, which we name the Kaniadakis divergence.
RESUMEN
A large number of complex biochemical reaction networks are included in the gene expression, cell development, and cell differentiation of in vivo cells, among other processes. Biochemical reaction-underlying processes are the ones transmitting information from cellular internal or external signaling. However, how this information is measured remains an open question. In this paper, we apply the method of information length, based on the combination of Fisher information and information geometry, to study linear and nonlinear biochemical reaction chains, respectively. Through a lot of random simulations, we find that the amount of information does not always increase with the length of the linear reaction chain; instead, the amount of information varies significantly when this length is not very large. When the length of the linear reaction chain reaches a certain value, the amount of information hardly changes. For nonlinear reaction chains, the amount of information changes not only with the length of this chain, but also with reaction coefficients and rates, and this amount also increases with the length of the nonlinear reaction chain. Our results will help to understand the role of the biochemical reaction networks in cells.
RESUMEN
A geometrical method for assessing stochastic processes in plasma turbulence is investigated in this study. The thermodynamic length methodology allows using a Riemannian metric on the phase space; thus, distances between thermodynamic states can be computed. It constitutes a geometric methodology to understand stochastic processes involved in, e.g., order-disorder transitions, where a sudden increase in distance is expected. We consider gyrokinetic simulations of ion-temperature-gradient (ITG)-mode-driven turbulence in the core region of the stellarator W7-X with realistic quasi-isodynamic topologies. In gyrokinetic plasma turbulence simulations, avalanches, e.g., of heat and particles, are often found, and in this work, a novel method for detection is investigated. This new method combines the singular spectrum analysis algorithm with a hierarchical clustering method such that the time series is decomposed into two parts: useful physical information and noise. The informative component of the time series is used for the calculation of the Hurst exponent, the information length, and the dynamic time. Based on these measures, the physical properties of the time series are revealed.
RESUMEN
A model of spacetime is presented. It has an extension to five dimensions, and in five dimensions the geometry is the dual of the Euclidean geometry w.r.t. an arbitrary positive-definite metric. Dually flat geometries are well-known in the context of information geometry. The present work explores their role in describing the geometry of spacetime. It is shown that the positive-definite metric with its flat 5-d connection can coexist with a pseudometric for which the connection is that of Levi-Civita. The 4-d geodesics are characterized by five conserved quantities, one of which can be chosen freely and is taken equal to zero in the present work. An explicit expression for the parallel transport operators is obtained. It is used to construct a pseudometric for spacetime by choosing an arbitrary possibly degenerate inner product in the tangent space of a reference point, for instance, that of Minkowski. By parallel transport, one obtains a pseudometric for spacetime, the metric connection of which extends to a 5-d connection with vanishing curvature tensor. The de Sitter space is considered as an example.
RESUMEN
We present a simple method to approximate the Fisher-Rao distance between multivariate normal distributions based on discretizing curves joining normal distributions and approximating the Fisher-Rao distances between successive nearby normal distributions on the curves by the square roots of their Jeffreys divergences. We consider experimentally the linear interpolation curves in the ordinary, natural, and expectation parameterizations of the normal distributions, and compare these curves with a curve derived from the Calvo and Oller's isometric embedding of the Fisher-Rao d-variate normal manifold into the cone of (d+1)×(d+1) symmetric positive-definite matrices. We report on our experiments and assess the quality of our approximation technique by comparing the numerical approximations with both lower and upper bounds. Finally, we present several information-geometric properties of Calvo and Oller's isometric embedding.
RESUMEN
We investigate the effects of different stochastic noises on the dynamics of the edge-localised modes (ELMs) in magnetically confined fusion plasmas by using a time-dependent PDF method, path-dependent information geometry (information rate, information length), and entropy-related measures (entropy production, mutual information). The oscillation quenching occurs due to either stochastic particle or magnetic perturbations, although particle perturbation is more effective in this amplitude diminishment compared with magnetic perturbations. On the other hand, magnetic perturbations are more effective at altering the oscillation period; the stochastic noise acts to increase the frequency of explosive oscillations (large ELMs) while decreasing the frequency of more regular oscillations (small ELMs). These stochastic noises significantly reduce power and energy losses caused by ELMs and play a key role in reproducing the observed experimental scaling relation of the ELM power loss with the input power. Furthermore, the maximum power loss is closely linked to the maximum entropy production rate, involving irreversible energy dissipation in non-equilibrium. Notably, over one ELM cycle, the information rate appears to keep almost a constant value, indicative of a geodesic. The information rate is also shown to be useful for characterising the statistical properties of ELMs, such as distinguishing between explosive and regular oscillations and the regulation between the pressure gradient and magnetic fluctuations.
RESUMEN
The recent link discovered between generalized Legendre transforms and non-dually flat statistical manifolds suggests a fundamental reason behind the ubiquity of Rényi's divergence and entropy in a wide range of physical phenomena. However, these early findings still provide little intuition on the nature of this relationship and its implications for physical systems. Here we shed new light on the Legendre transform by revealing the consequences of its deformation via symplectic geometry and complexification. These findings reveal a novel common framework that leads to a principled and unified understanding of physical systems that are not well-described by classic information-theoretic quantities.
RESUMEN
The aim of this paper is to introduce a field of study that has emerged over the last decade, called Bayesian mechanics. Bayesian mechanics is a probabilistic mechanics, comprising tools that enable us to model systems endowed with a particular partition (i.e. into particles), where the internal states (or the trajectories of internal states) of a particular system encode the parameters of beliefs about external states (or their trajectories). These tools allow us to write down mechanical theories for systems that look as if they are estimating posterior probability distributions over the causes of their sensory states. This provides a formal language for modelling the constraints, forces, potentials and other quantities determining the dynamics of such systems, especially as they entail dynamics on a space of beliefs (i.e. on a statistical manifold). Here, we will review the state of the art in the literature on the free energy principle, distinguishing between three ways in which Bayesian mechanics has been applied to particular systems (i.e. path-tracking, mode-tracking and mode-matching). We go on to examine a duality between the free energy principle and the constrained maximum entropy principle, both of which lie at the heart of Bayesian mechanics, and discuss its implications.
RESUMEN
In information geometry, there has been extensive research on the deep connections between differential geometric structures, such as the Fisher metric and the α-connection, and the statistical theory for statistical models satisfying regularity conditions. However, the study of information geometry for non-regular statistical models is insufficient, and a one-sided truncated exponential family (oTEF) is one example of these models. In this paper, based on the asymptotic properties of maximum likelihood estimators, we provide a Riemannian metric for the oTEF. Furthermore, we demonstrate that the oTEF has an α = 1 parallel prior distribution and that the scalar curvature of a certain submodel, including the Pareto family, is a negative constant.