Exploratory Analysis

Exploratory Analysis#

Using a metabolomics dataset (MTBLS13411) as an example, we show how to use the exploratory analysis for the four groups present in the data. The groups are derived based on the sample names, but you could also merge them from available metadata.

  • Principal Component Analysis (PCA)

  • Uniform Manifold Approximation and Projection (UMAP)

  • Coefficient of variation (CoV)

  • Correlation analysis

  • Histogram of the data

  • Summary statistics

The focus of acore is not plotting, but getting the data in the right format for plotting. Nonetheless, we show some examples of how to plot the results. VueCore offers more examples.

%pip install acore

Hide code cell output

 
Requirement already satisfied: acore in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (0.3.0)
Requirement already satisfied: numpy in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (2.4.6)
Requirement already satisfied: pandas in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (2.3.3)
Requirement already satisfied: dsp-pandas in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (0.0.5)
Requirement already satisfied: scipy in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (1.17.1)
Requirement already satisfied: scikit-learn>=1.5 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (1.9.0)
Requirement already satisfied: gseapy!=1.1.5 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (1.2.1)
Requirement already satisfied: pingouin<0.6.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (0.5.5)
Requirement already satisfied: umap-learn in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (0.5.12)
Requirement already satisfied: statsmodels in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (0.14.6)
Requirement already satisfied: combat in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (0.3.3)
Requirement already satisfied: inmoose in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (0.9.1)
Requirement already satisfied: requests in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (2.34.2)
Requirement already satisfied: rarfile in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (4.2)
Requirement already satisfied: pandera in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from acore) (0.31.1)
Requirement already satisfied: matplotlib in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pingouin<0.6.0->acore) (3.11.0)
Requirement already satisfied: pandas-flavor in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pingouin<0.6.0->acore) (0.8.1)
Requirement already satisfied: seaborn in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pingouin<0.6.0->acore) (0.13.2)
Requirement already satisfied: tabulate in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pingouin<0.6.0->acore) (0.10.0)
Requirement already satisfied: contourpy>=1.0.1 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from matplotlib->pingouin<0.6.0->acore) (1.3.3)
Requirement already satisfied: cycler>=0.10 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from matplotlib->pingouin<0.6.0->acore) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from matplotlib->pingouin<0.6.0->acore) (4.63.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from matplotlib->pingouin<0.6.0->acore) (1.5.0)
Requirement already satisfied: packaging>=20.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from matplotlib->pingouin<0.6.0->acore) (26.2)
Requirement already satisfied: pillow>=9 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from matplotlib->pingouin<0.6.0->acore) (12.2.0)
Requirement already satisfied: pyparsing>=3 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from matplotlib->pingouin<0.6.0->acore) (3.3.2)
Requirement already satisfied: python-dateutil>=2.7 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from matplotlib->pingouin<0.6.0->acore) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pandas->acore) (2026.2)
Requirement already satisfied: tzdata>=2022.7 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pandas->acore) (2026.2)
Requirement already satisfied: six>=1.5 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib->pingouin<0.6.0->acore) (1.17.0)
Requirement already satisfied: joblib>=1.4.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from scikit-learn>=1.5->acore) (1.5.3)
Requirement already satisfied: narwhals>=2.0.1 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from scikit-learn>=1.5->acore) (2.22.1)
Requirement already satisfied: threadpoolctl>=3.5.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from scikit-learn>=1.5->acore) (3.6.0)
Requirement already satisfied: mpmath>=1.1.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from combat->acore) (1.4.1)
Requirement already satisfied: patsy>=0.5.1 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from combat->acore) (1.0.2)
Requirement already satisfied: openpyxl in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from dsp-pandas->acore) (3.1.5)
Requirement already satisfied: anndata in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from inmoose->acore) (0.12.17)
Requirement already satisfied: fastcluster in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from inmoose->acore) (1.3.0)
Requirement already satisfied: jinja2 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from inmoose->acore) (3.1.6)
Requirement already satisfied: array-api-compat>=1.7.1 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from anndata->inmoose->acore) (1.15.0)
Requirement already satisfied: h5py>=3.8 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from anndata->inmoose->acore) (3.16.0)
Requirement already satisfied: legacy-api-wrap in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from anndata->inmoose->acore) (1.5)
Requirement already satisfied: natsort in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from anndata->inmoose->acore) (8.4.0)
Requirement already satisfied: scverse-misc>=0.0.3 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from anndata->inmoose->acore) (0.0.3)
Requirement already satisfied: zarr!=3.0.*,>=2.18.7 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from anndata->inmoose->acore) (3.1.6)
Requirement already satisfied: session-info2 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from scverse-misc>=0.0.3->anndata->inmoose->acore) (0.4.1)
Requirement already satisfied: typing-extensions in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from scverse-misc>=0.0.3->anndata->inmoose->acore) (4.15.0)
Requirement already satisfied: donfig>=0.8 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from zarr!=3.0.*,>=2.18.7->anndata->inmoose->acore) (0.8.1.post1)
Requirement already satisfied: google-crc32c>=1.5 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from zarr!=3.0.*,>=2.18.7->anndata->inmoose->acore) (1.8.0)
Requirement already satisfied: numcodecs>=0.14 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from zarr!=3.0.*,>=2.18.7->anndata->inmoose->acore) (0.16.5)
Requirement already satisfied: pyyaml in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from donfig>=0.8->zarr!=3.0.*,>=2.18.7->anndata->inmoose->acore) (6.0.3)
Requirement already satisfied: MarkupSafe>=2.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from jinja2->inmoose->acore) (3.0.3)
Requirement already satisfied: et-xmlfile in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from openpyxl->dsp-pandas->acore) (2.0.0)
Requirement already satisfied: xarray in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pandas-flavor->pingouin<0.6.0->acore) (2026.4.0)
Requirement already satisfied: pydantic in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pandera->acore) (2.13.4)
Requirement already satisfied: typeguard in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pandera->acore) (4.5.2)
Requirement already satisfied: typing_inspect>=0.6.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pandera->acore) (0.9.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from typing_inspect>=0.6.0->pandera->acore) (1.1.0)
Requirement already satisfied: annotated-types>=0.6.0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pydantic->pandera->acore) (0.7.0)
Requirement already satisfied: pydantic-core==2.46.4 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pydantic->pandera->acore) (2.46.4)
Requirement already satisfied: typing-inspection>=0.4.2 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from pydantic->pandera->acore) (0.4.2)
Requirement already satisfied: charset_normalizer<4,>=2 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from requests->acore) (3.4.7)
Requirement already satisfied: idna<4,>=2.5 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from requests->acore) (3.18)
Requirement already satisfied: urllib3<3,>=1.26 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from requests->acore) (2.7.0)
Requirement already satisfied: certifi>=2023.5.7 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from requests->acore) (2026.5.20)
Requirement already satisfied: numba>=0.51.2 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from umap-learn->acore) (0.65.1)
Requirement already satisfied: pynndescent>=0.5 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from umap-learn->acore) (0.6.0)
Requirement already satisfied: tqdm in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from umap-learn->acore) (4.68.2)
Requirement already satisfied: llvmlite<0.48,>=0.47.0dev0 in /home/docs/checkouts/readthedocs.org/user_builds/analytics-core/envs/stable/lib/python3.11/site-packages (from numba>=0.51.2->umap-learn->acore) (0.47.0)
Note: you may need to restart the kernel to use updated packages.

from typing import Optional

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scipy
import seaborn as sns

import acore.correlation_analysis as ca
import acore.exploratory_analysis as ea
from acore.types.exploratory_analysis import (
    AnnotationResult,
    TwoComponentSchema,
    TwoLoadingsSchema,
    TwoVariance,
)

Utility function for plotting

Hide code cell source

 
def make_plot(
    embeddings,
    x: str,
    y: str,
    annotation: Optional[dict[str, str]] = None,
    group: str = "group",
    **kwargs,
):
    """Utility function for static plot of dimensionality reductions."""
    fig, ax = plt.subplots()
    for i, (group, group_df) in enumerate(embeddings.groupby("group")):
        ax = group_df.rename(columns=map_names).plot.scatter(
            x=x,
            y=y,
            label=group,
            c=f"C{i}",
            ax=ax,
        )
    if annotation is not None:
        _ = ax.set(ylabel=annotation.y_title, xlabel=annotation.x_title)
    return fig, ax

Load metabolomics example data#

Hide code cell source

 
data = (
    "https://raw.githubusercontent.com/Multiomics-Analytics-Group/acore/"
    "refs/heads/main/"
    "example_data/MTBLS13311/MTBLS13411_processed_data.csv"
)
data = pd.read_csv(data, index_col=0)
# specific to this data, we shorten some column names for better readability
data.columns = data.columns.str.split("(").str[-1].str.replace(")", "")
data
Uracil Adenine Hypoxanthine Uridine Creatinine Adenosine Cytosine Inosine Guanine Tryptophan ... NADPH GDP S-Adenosylmethionine NADP Oxidized glutathione Ribose 1,5-bisphosphate Fructose 1,6-bisphosphate Arginine Ornithine Lysine
rapZE227Stop-1 26.271 21.874 23.201 19.421 14.929 19.589 20.992 22.907 22.140 21.901 ... 24.807 22.373 22.503 24.580 26.072 21.776 20.164 23.718 25.082 22.330
rapZE227Stop-2 27.293 21.983 23.357 19.394 14.981 19.670 21.713 22.767 22.012 22.124 ... 24.816 22.373 22.492 24.503 26.240 21.527 20.355 23.611 25.016 21.867
rapZE227Stop-3 26.788 21.833 23.167 19.476 15.786 19.686 21.330 22.976 21.025 22.020 ... 24.910 22.422 22.847 24.535 26.333 21.805 20.331 23.834 25.158 21.729
rapZE227Stop-4 27.014 22.120 23.517 19.228 14.820 19.711 21.392 22.757 21.875 21.964 ... 24.877 22.452 21.270 24.451 26.284 22.066 20.545 23.634 24.943 21.525
rapZE227Stop-5 26.364 21.957 23.246 19.615 15.373 19.768 21.045 23.090 20.798 22.135 ... 25.047 22.392 22.773 24.728 26.330 21.861 20.208 23.971 25.206 22.180
rapZE227Stop-6 26.571 21.885 23.121 19.879 15.192 19.763 21.198 22.928 21.358 21.916 ... 25.015 22.285 22.866 24.638 26.544 21.845 20.324 23.917 25.163 21.768
WT-1 26.472 21.826 22.980 18.901 14.931 19.659 21.662 22.816 21.736 22.101 ... 25.341 21.093 23.093 24.838 26.866 21.969 20.368 24.265 25.163 21.638
WT-2 26.325 21.652 22.925 18.373 14.819 19.466 21.739 22.720 21.689 21.958 ... 25.141 20.913 23.283 24.809 26.971 21.971 20.385 24.175 25.260 21.040
WT-3 26.274 21.651 22.885 18.269 14.997 19.440 21.548 22.747 21.548 22.167 ... 25.259 21.097 23.233 24.564 26.899 21.770 20.407 24.244 25.090 21.393
WT-4 26.187 21.780 22.938 18.515 15.014 19.600 21.583 22.734 21.755 21.882 ... 25.144 20.760 23.228 24.807 27.093 21.900 20.605 24.218 25.216 21.399
WT-5 26.244 21.702 22.945 18.708 15.086 19.488 21.509 22.826 21.692 22.064 ... 24.925 20.989 23.347 24.632 26.879 21.768 20.392 24.384 25.164 21.348
WT-6 26.313 21.770 22.851 19.029 15.185 19.635 21.653 22.786 21.751 21.888 ... 25.120 21.147 23.278 24.883 27.165 21.886 20.455 24.136 25.156 21.379
WRP-1 26.301 22.169 23.295 20.328 15.444 19.998 20.972 22.887 20.467 22.309 ... 25.051 22.325 23.264 24.580 27.135 21.850 20.205 23.923 25.111 21.752
WRP-2 25.721 21.546 22.771 19.766 14.972 19.547 20.358 22.564 20.954 22.145 ... 24.798 22.198 23.129 24.395 27.071 21.739 20.523 23.822 24.924 21.126
WRP-3 26.929 21.612 22.696 20.388 15.196 19.593 21.428 22.587 20.916 21.962 ... 24.800 22.561 23.222 24.524 27.231 21.791 20.513 23.828 24.917 21.084
WRP-4 26.265 21.738 22.881 20.278 15.229 19.721 20.672 22.692 21.335 22.245 ... 24.885 22.545 23.085 24.533 27.253 21.864 20.533 23.843 24.988 21.132
WRP-5 26.279 21.596 22.861 19.723 14.893 19.585 20.830 22.656 21.325 22.389 ... 25.069 22.260 23.109 24.499 27.084 21.776 20.555 23.871 25.063 21.508
WRP-6 26.550 21.626 22.849 20.073 15.066 19.692 21.179 22.751 20.995 22.343 ... 24.942 22.204 23.156 24.699 27.126 21.884 20.514 23.927 25.011 21.180
QC-1 26.321 21.572 22.779 19.417 14.920 19.584 21.340 22.854 21.425 22.259 ... 24.901 22.003 23.132 24.550 26.853 21.812 20.339 24.129 25.139 21.544
QC-2 26.289 21.511 22.681 19.252 14.839 19.415 21.459 22.649 21.457 21.987 ... 24.908 22.032 22.991 24.522 26.767 21.821 20.324 24.052 25.049 21.564
QC-3 26.287 21.556 22.732 19.379 14.882 19.459 21.347 22.767 21.383 22.060 ... 24.899 21.984 23.034 24.639 26.870 21.813 20.331 24.066 25.097 21.513

21 rows × 59 columns

We add the group here based on the sample names. Alternatively you could merge it from the avilable metadata.

Hide code cell source

 
data["group"] = data.index.str.split("-").str[0]
data["group"].value_counts()

group
rapZE227Stop   6
WT             6
WRP            6
QC             3
Name: count, dtype: int64

Principal Component Analysis (PCA)#

Show first two principal components of the data.

Hide code cell source

 
# map_names gives the column names for the plot axes (which default to "x" and "y")
map_names = {
    "value": "feature_communiality",
    "x": "PC1",
    "y": "PC2",
}
results_dfs, annotation = ea.run_pca(
    data, drop_cols=[], annotation_cols=[], group="group", components=2, dropna=True
)
pcs, loadings, var_explained = results_dfs

See how much variance is explained by the first two components and validate that they adhere to the expected format:

TwoVariance(pd.Series(var_explained, index=["PC1", "PC2"]))

PC1   0.663
PC2   0.136
dtype: float64

Show the annotation information for plotting and validate that they adhere to the expected format:

annotation = AnnotationResult(**annotation)
annotation

AnnotationResult(x_title='PC1 (0.66)', y_title='PC2 (0.14)', group='group')

Show PCA for the two first components, highlighting the groups

Hide code cell source

 
fig, ax = make_plot(pcs, annotation=annotation, **map_names)
../_images/af7febda5afb752886e7390c9b6228b8468996ecc9947df3fd755c8c2c67d2ef.png

Show what was computed and validate that they adhere to the expected format:

  • first two principal components of the samples

  • loadings for the features on the first two components

We rename the columns for better readability.

TwoComponentSchema(pcs).rename(columns=map_names)
group PC1 PC2
0 rapZE227Stop 2.174 -1.781
1 rapZE227Stop 2.849 -0.288
2 rapZE227Stop 2.029 -2.566
3 rapZE227Stop 1.810 -4.027
4 rapZE227Stop 3.009 -1.492
5 rapZE227Stop 2.428 -1.388
6 WT -4.494 1.236
7 WT -6.055 0.207
8 WT -5.810 2.090
9 WT -6.890 -0.450
10 WT -6.689 -1.228
11 WT -5.945 -0.395
12 WRP 2.769 0.195
13 WRP 2.566 2.377
14 WRP 2.106 -0.492
15 WRP 3.558 3.133
16 WRP 3.268 2.902
17 WRP 2.669 0.961
18 QC 1.601 0.090
19 QC 1.469 0.463
20 QC 1.580 0.453

The feature communality of the loading is the absolute length of the projection. So the features listed first here contribute the most to the two first components, therefore driving the PCA separation.

TwoLoadingsSchema(loadings).rename(columns=map_names)
PC1 PC2 feature_communiality
N-Acetylglucosamine 6-phosphate 0.702 -0.108 0.711
Guanosine -0.149 0.424 0.449
N-Acetylglucosamine 1-phosphate 0.401 -0.070 0.407
UDP-N-acetylglucosamine 0.274 -0.242 0.366
Acetyl-CoA 0.154 0.290 0.328
NADH 0.051 0.283 0.287
D-Ribulose 5-phosphate 0.152 0.233 0.278
Succinyl-CoA 0.097 0.248 0.266
FAD 0.130 0.231 0.265
Glutamine -0.033 -0.224 0.226
3-Phosphoglycerate 0.127 0.165 0.208
Citrulline 0.075 0.189 0.203
S-Adenosylhomocysteine 0.075 0.172 0.188
Oxaloacetate 0.034 -0.174 0.177
Serine 0.030 0.170 0.173
FMN 0.124 0.109 0.165
S-Adenosylmethionine -0.042 0.156 0.162
GDP 0.154 -0.045 0.160
Gamma-Aminobutyric acid -0.015 -0.142 0.143
Dihydroxyacetone phosphate 0.067 0.126 0.142
Oxidized glutathione -0.025 0.139 0.141
Uridine 0.132 0.039 0.138
Threonine -0.054 -0.126 0.137
Glucuronic acid 0.102 0.083 0.131
Nicotinamide riboside -0.081 0.099 0.127
Uridine 5'-diphosphate 0.123 0.005 0.123
ADP-ribose -0.072 -0.099 0.122
Glutaryl-CoA 0.050 0.091 0.104
Uracil 0.021 -0.097 0.100
Cytosine -0.059 -0.074 0.094
UDP-Glucuronate 0.092 0.010 0.093
Lysine 0.026 -0.088 0.091
Glycerol 3-phosphate 0.083 0.026 0.087
ATP -0.036 0.076 0.084
Hypoxanthine 0.012 -0.082 0.082
NAD 0.075 -0.006 0.076
Choline -0.004 -0.071 0.071
Guanine -0.049 -0.050 0.070
Valine 0.013 -0.062 0.063
Adenine 0.007 -0.059 0.059
Phosphoenolpyruvate 0.004 -0.058 0.058
Tryptophan 0.016 0.052 0.055
Arginine -0.043 0.027 0.051
6-Phosphogluconate -0.023 -0.041 0.047
Inosine 0.002 -0.039 0.039
GMP -0.005 0.034 0.035
Creatinine 0.013 -0.032 0.035
NADPH -0.025 0.021 0.033
Alanine 0.027 -0.004 0.028
D-Sedoheptulose 7-phosphate -0.016 0.020 0.025
Fructose 1,6-bisphosphate -0.006 0.022 0.023
NADP -0.022 -0.004 0.023
Adenosine 0.015 -0.017 0.022
Betaine 0.003 -0.018 0.018
Ornithine -0.014 -0.010 0.017
Ribose 1,5-bisphosphate -0.007 -0.014 0.016
Fructose 6-phosphate -0.013 0.004 0.013
Isoleucine -0.004 -0.011 0.012
Leucine -0.001 -0.010 0.010

Uniform Manifold Approximation and Projection (UMAP)#

Visualize UMAP low-dimensional embedding of the data. This uses the umap-learn package, which is documented with examples at umap-learn.readthedocs.io.

# map_names gives the column names for the plot axes (which default to "x" and "y")
map_names = {
    "x": "UMAP1",
    "y": "UMAP2",
}
result, annotation = ea.run_umap(
    data,
    drop_cols=["sample", "subject"],
    group="group",
    n_neighbors=10,
    min_dist=0.3,
    metric="cosine",
    dropna=True,
)

annotation = AnnotationResult(**annotation)
annotation

AnnotationResult(x_title='C1', y_title='C2', group=None)

fig, ax = make_plot(result["umap"], annotation=annotation, **map_names)
TwoComponentSchema(result["umap"]).rename(columns=map_names)
group UMAP1 UMAP2
0 rapZE227Stop 0.485 10.361
1 rapZE227Stop 0.160 9.906
2 rapZE227Stop 1.000 10.926
3 rapZE227Stop 1.508 10.446
4 rapZE227Stop 0.938 9.835
5 rapZE227Stop 1.283 11.499
6 WT 5.501 14.006
7 WT 5.817 15.089
8 WT 4.837 14.213
9 WT 5.032 14.728
10 WT 5.163 15.141
11 WT 5.897 14.514
12 WRP -0.123 12.402
13 WRP -0.976 11.016
14 WRP 0.518 11.941
15 WRP -1.536 10.882
16 WRP -1.218 10.238
17 WRP -0.879 12.299
18 QC -0.270 11.617
19 QC -0.175 11.049
20 QC -0.938 11.720
../_images/b54256d124858823793efc7673bedd4f3649ee07fdc148b4172da993694a2427.png

Make sure to check the parameters and tutorials annotations in the API docs at umap-learn.readthedocs.io.

Coefficient of variation#

Using masspectrometry data, we can compute the coefficient of variation on the non-log transformed intensities. We do this for each group separately. First we undo the log transformation, which is something specific to this dataset.

data_exp = data.drop(columns=["group"]).apply(lambda x: np.exp2(x)).join(data["group"])
data_exp
Uracil Adenine Hypoxanthine Uridine Creatinine Adenosine Cytosine Inosine Guanine Tryptophan ... GDP S-Adenosylmethionine NADP Oxidized glutathione Ribose 1,5-bisphosphate Fructose 1,6-bisphosphate Arginine Ornithine Lysine group
rapZE227Stop-1 80,952,187.500 3,843,312.567 9,639,837.573 702,159.444 31,194.619 788,723.842 2,085,215.024 7,863,918.766 4,622,410.462 3,914,987.928 ... 5,431,389.476 5,945,392.275 25,087,806.090 70,561,829.400 3,590,513.857 1,174,474.922 13,795,026.400 35,518,267.120 5,271,579.497 rapZE227Stop
rapZE227Stop-2 164,478,423.900 4,143,846.199 10,747,288.900 689,125.769 32,349.777 834,340.904 3,437,677.615 7,138,245.050 4,230,561.650 4,570,054.986 ... 5,430,716.328 5,897,285.582 23,781,111.470 79,230,093.470 3,022,185.990 1,341,399.964 12,809,539.150 33,924,648.100 3,825,115.251 rapZE227Stop
rapZE227Stop-3 115,884,911.900 3,734,799.890 9,416,387.265 728,998.396 56,500.262 843,386.050 2,636,152.114 8,249,774.730 2,133,301.638 4,254,206.476 ... 5,619,905.602 7,544,136.071 24,311,986.330 84,510,074.090 3,662,851.862 1,318,970.111 14,953,695.480 37,442,945.060 3,475,926.994 rapZE227Stop
rapZE227Stop-4 135,542,303.000 4,557,393.340 12,001,580.120 613,915.461 28,924.438 858,390.939 2,752,792.659 7,090,580.561 3,846,191.437 4,092,036.363 ... 5,737,716.736 2,528,787.195 22,934,277.310 81,688,642.360 4,390,330.616 1,530,346.180 13,016,662.370 32,245,061.060 3,017,680.456 rapZE227Stop
rapZE227Stop-5 86,347,653.080 4,070,097.561 9,945,944.887 803,114.349 42,435.429 892,767.018 2,163,571.667 8,930,763.375 1,823,486.018 4,606,716.494 ... 5,502,814.841 7,168,629.733 27,790,946.560 84,328,761.540 3,808,766.391 1,211,063.946 16,437,809.080 38,701,049.040 4,752,881.672 rapZE227Stop
rapZE227Stop-6 99,704,509.310 3,872,431.333 9,125,608.729 964,306.497 37,437.599 889,772.831 2,405,813.834 7,980,072.926 2,688,030.496 3,958,299.581 ... 5,111,127.156 7,645,287.481 26,112,961.760 97,848,726.440 3,766,329.210 1,313,026.091 15,842,684.380 37,576,503.460 3,571,209.958 rapZE227Stop
WT-1 93,051,714.760 3,716,750.521 8,273,242.759 489,519.437 31,230.019 827,860.743 3,319,046.432 7,382,191.146 3,493,824.690 4,498,551.491 ... 2,236,473.478 8,945,258.404 30,000,709.120 122,308,419.500 4,104,471.689 1,352,939.592 20,160,609.380 37,566,304.050 3,262,449.579 WT
WT-2 84,051,614.560 3,295,215.467 7,965,279.370 339,510.067 28,905.419 724,332.630 3,499,550.770 6,909,816.359 3,380,209.153 4,072,650.838 ... 1,975,094.707 10,206,077.970 29,393,779.890 131,568,846.100 4,110,323.806 1,369,338.368 18,945,649.600 40,187,314.390 2,155,496.903 WT
WT-3 81,154,213.000 3,293,817.356 7,748,544.536 315,954.339 32,698.961 711,176.623 3,065,853.195 7,040,155.370 3,066,037.925 4,707,810.470 ... 2,242,511.570 9,857,085.263 24,804,758.520 125,106,555.900 3,576,750.174 1,390,585.914 19,869,338.190 35,706,814.510 2,753,085.876 WT
WT-4 76,402,690.380 3,602,270.729 8,038,115.376 374,481.310 33,085.745 794,795.696 3,140,557.467 6,976,765.167 3,539,756.846 3,864,797.908 ... 1,775,945.465 9,825,483.164 29,350,391.000 143,161,110.300 3,914,210.412 1,594,409.312 19,510,109.320 38,983,439.160 2,764,831.328 WT
WT-5 79,450,943.070 3,410,438.972 8,072,124.526 428,124.909 34,781.863 735,452.518 2,984,345.852 7,436,412.508 3,388,908.453 4,385,014.310 ... 2,081,146.471 10,669,764.300 25,999,593.320 123,427,678.200 3,571,788.234 1,375,967.324 21,886,406.910 37,589,180.770 2,668,335.177 WT
WT-6 83,368,573.570 3,575,391.000 7,564,507.881 534,946.875 37,248.094 814,322.462 3,297,181.221 7,230,649.614 3,528,241.515 3,881,246.250 ... 2,322,386.791 10,172,027.980 30,932,199.550 150,477,063.700 3,875,583.194 1,437,376.823 18,440,158.720 37,375,623.800 2,727,264.673 WT
WRP-1 82,668,560.040 4,714,674.559 10,289,099.820 1,315,867.361 44,586.923 1,047,034.012 2,057,222.180 7,757,079.853 1,449,390.442 5,197,438.801 ... 5,254,397.215 10,071,400.760 25,070,806.410 147,373,217.400 3,779,062.284 1,208,918.934 15,906,953.850 36,233,843.160 3,531,952.121 WRP
WRP-2 55,307,183.370 3,062,539.607 7,157,187.140 891,844.310 32,128.365 765,977.502 1,343,468.258 6,198,621.519 2,031,479.307 4,638,883.241 ... 4,810,221.149 9,172,782.457 22,068,530.440 140,990,353.000 3,499,567.365 1,507,091.461 14,833,361.280 31,827,512.020 2,288,196.831 WRP
WRP-3 127,738,834.400 3,206,128.763 6,793,062.697 1,372,167.331 37,533.953 790,819.069 2,820,546.437 6,299,654.632 1,979,137.123 4,084,957.691 ... 6,188,883.352 9,783,901.555 24,117,345.090 157,483,465.400 3,628,996.518 1,496,441.721 14,895,100.220 31,672,065.210 2,223,439.401 WRP
WRP-4 80,635,670.060 3,497,788.537 7,722,114.271 1,271,271.004 38,398.871 864,300.824 1,670,535.855 6,774,187.518 2,644,407.466 4,971,730.147 ... 6,119,572.947 8,896,586.582 24,276,579.520 159,897,210.200 3,818,181.678 1,517,513.578 15,048,299.200 33,270,801.470 2,297,657.057 WRP
WRP-5 81,436,675.040 3,170,561.566 7,616,777.466 865,670.177 30,417.183 786,408.364 1,864,271.668 6,609,797.524 2,626,864.284 5,493,491.682 ... 5,021,112.727 9,044,633.840 23,717,486.290 142,313,217.100 3,591,809.748 1,540,516.377 15,342,849.660 35,045,957.440 2,981,532.011 WRP
WRP-6 98,257,046.250 3,235,960.603 7,554,535.368 1,102,648.601 34,311.891 847,116.727 2,374,013.731 7,058,947.967 2,090,248.659 5,319,036.716 ... 4,832,148.828 9,343,670.921 27,243,017.490 146,425,768.100 3,869,433.985 1,497,132.378 15,953,076.720 33,800,033.510 2,375,761.894 WRP
QC-1 83,852,330.540 3,116,881.472 7,195,269.100 700,008.129 31,009.081 785,880.659 2,654,352.259 7,579,128.487 2,816,013.882 5,017,885.278 ... 4,201,993.824 9,193,694.569 24,562,663.880 121,215,941.200 3,682,587.440 1,326,459.438 18,347,738.540 36,947,962.780 3,057,119.158 QC
QC-2 81,987,268.520 2,988,418.165 6,722,955.085 624,225.582 29,307.199 698,967.545 2,882,163.755 6,578,231.484 2,879,059.497 4,155,488.798 ... 4,289,851.390 8,337,333.646 24,094,529.090 114,177,748.600 3,704,789.464 1,312,212.678 17,387,430.430 34,703,365.420 3,100,319.649 QC
QC-3 81,904,328.570 3,083,034.016 6,964,746.010 681,820.600 30,203.959 720,706.113 2,666,646.428 7,135,178.907 2,734,685.315 4,372,804.103 ... 4,147,556.395 8,589,763.158 26,133,391.670 122,621,793.200 3,685,399.420 1,318,932.902 17,560,549.890 35,896,111.270 2,993,543.673 QC

21 rows × 60 columns

res = ea.get_coefficient_variation(data=data_exp, group="group")
res
name mean_log2 mean coef_of_var group
0 Uracil 26.299 82,581,309.210 1.089 QC
1 Adenine 21.546 3,062,777.884 1.775 QC
2 Hypoxanthine 22.730 6,960,990.065 2.770 QC
3 Uridine 19.349 668,684.770 4.831 QC
4 Creatinine 14.881 30,173.413 2.304 QC
... ... ... ... ... ...
231 Ribose 1,5-bisphosphate 21.813 3,706,829.654 10.821 rapZE227Stop
232 Fructose 1,6-bisphosphate 20.321 1,314,880.202 8.651 rapZE227Stop
233 Arginine 23.781 14,475,902.810 9.487 rapZE227Stop
234 Ornithine 25.095 35,901,412.307 6.274 rapZE227Stop
235 Lysine 21.900 3,985,732.305 19.533 rapZE227Stop

236 rows × 5 columns

res.describe()
mean_log2 mean coef_of_var
count 236.000 236.000 236.000
mean 21.736 14,138,279.703 14.505
std 2.552 33,800,427.066 15.392
min 14.570 24,666.191 0.000
25% 20.088 1,125,714.181 4.336
50% 21.781 3,688,510.176 8.608
75% 23.591 12,759,802.352 19.178
max 28.203 309,494,575.583 92.394 
map_names = {"x": "mean_log2", "y": "coef_of_var", "group": "group"}
fig, ax = make_plot(res, **map_names)
../_images/9dc3630ff83af4dc75bc37234c792d92506069668d5e2f520abb6aa42efdb7d9.png

Correlation analysis#

See acore.correlation_analysis for more functions and details.

The basic functionality is built into pandas, but you need to filter out columns which are not numeric for pearson correlation.

Generally: Ordered categorical values can be used, assuming equal spacing between the categories. Otherwise, continous numeric values are required.

corr = data.drop(columns=["group"]).corr(method="pearson")
corr
Uracil Adenine Hypoxanthine Uridine Creatinine Adenosine Cytosine Inosine Guanine Tryptophan ... NADPH GDP S-Adenosylmethionine NADP Oxidized glutathione Ribose 1,5-bisphosphate Fructose 1,6-bisphosphate Arginine Ornithine Lysine
Uracil 1.000 0.451 0.511 0.154 0.150 0.274 0.469 0.146 0.181 -0.148 ... -0.224 0.354 -0.546 -0.145 -0.451 -0.110 -0.005 -0.489 -0.226 0.192
Adenine 0.451 1.000 0.936 0.128 0.369 0.780 0.055 0.569 0.029 -0.111 ... 0.069 0.229 -0.556 0.044 -0.493 0.167 -0.332 -0.387 0.105 0.559
Hypoxanthine 0.511 0.936 1.000 0.030 0.263 0.625 0.044 0.579 0.150 -0.099 ... -0.044 0.282 -0.709 -0.099 -0.701 0.077 -0.335 -0.486 0.075 0.615
Uridine 0.154 0.128 0.030 1.000 0.398 0.591 -0.645 -0.026 -0.641 0.406 ... -0.544 0.847 -0.045 -0.437 0.132 -0.167 -0.035 -0.625 -0.552 0.050
Creatinine 0.150 0.369 0.263 0.398 1.000 0.590 -0.181 0.531 -0.591 0.064 ... -0.043 0.299 0.166 0.016 -0.058 -0.092 -0.255 -0.135 0.167 0.206
Adenosine 0.274 0.780 0.625 0.591 0.590 1.000 -0.296 0.478 -0.444 0.262 ... 0.007 0.454 -0.214 0.016 -0.096 0.155 -0.203 -0.412 -0.016 0.347
Cytosine 0.469 0.055 0.044 -0.645 -0.181 -0.296 1.000 0.093 0.509 -0.466 ... 0.394 -0.587 -0.002 0.522 -0.126 0.126 -0.050 0.436 0.435 -0.030
Inosine 0.146 0.569 0.579 -0.026 0.531 0.478 0.093 1.000 -0.084 -0.073 ... 0.164 0.092 -0.178 0.301 -0.594 0.083 -0.708 0.040 0.599 0.750
Guanine 0.181 0.029 0.150 -0.641 -0.591 -0.444 0.509 -0.084 1.000 -0.488 ... 0.090 -0.422 -0.345 0.198 -0.366 0.002 0.083 0.092 0.145 0.138
Tryptophan -0.148 -0.111 -0.099 0.406 0.064 0.262 -0.466 -0.073 -0.488 1.000 ... 0.027 0.290 0.231 -0.308 0.327 -0.210 0.073 -0.070 -0.238 -0.115
Leucine -0.070 0.444 0.392 -0.051 0.394 0.392 0.035 0.780 0.011 0.022 ... 0.337 -0.071 0.120 0.390 -0.377 -0.117 -0.708 0.147 0.627 0.752
Guanosine -0.524 -0.532 -0.670 -0.331 -0.400 -0.406 0.111 -0.581 0.132 0.114 ... 0.482 -0.631 0.614 0.330 0.760 0.183 0.427 0.611 0.072 -0.628
Choline 0.271 0.891 0.841 -0.069 0.398 0.666 0.155 0.690 -0.027 -0.067 ... 0.231 -0.007 -0.316 0.240 -0.423 0.141 -0.417 -0.099 0.401 0.551
Isoleucine -0.085 0.397 0.328 -0.105 0.396 0.339 0.110 0.768 0.056 -0.066 ... 0.348 -0.150 0.190 0.466 -0.343 -0.122 -0.704 0.223 0.685 0.710
Betaine 0.173 0.724 0.522 0.348 0.695 0.818 -0.088 0.492 -0.386 0.133 ... 0.160 0.149 0.051 0.181 0.020 0.102 -0.299 -0.090 0.154 0.254
Valine 0.385 0.715 0.804 0.031 0.211 0.446 0.071 0.779 0.117 -0.176 ... -0.130 0.354 -0.667 -0.041 -0.856 0.067 -0.644 -0.371 0.252 0.857
Alanine 0.025 0.306 0.426 0.416 0.131 0.390 -0.500 0.474 -0.245 0.450 ... -0.304 0.641 -0.352 -0.357 -0.509 -0.260 -0.500 -0.499 -0.047 0.668
Threonine 0.315 0.102 -0.009 -0.047 0.238 0.108 0.450 0.222 0.154 -0.371 ... 0.279 -0.197 0.124 0.474 -0.053 0.175 -0.130 0.208 0.306 0.127
Nicotinamide riboside -0.563 -0.539 -0.693 -0.438 -0.202 -0.419 0.290 -0.292 0.029 -0.116 ... 0.521 -0.777 0.739 0.607 0.735 0.284 0.303 0.847 0.440 -0.588
Gamma-Aminobutyric acid 0.413 0.790 0.700 -0.044 0.602 0.549 0.271 0.565 -0.040 -0.418 ... 0.058 -0.000 -0.311 0.173 -0.360 0.203 -0.260 -0.103 0.258 0.309
Glutamine 0.273 0.550 0.568 -0.405 0.193 0.145 0.493 0.749 0.370 -0.577 ... 0.173 -0.186 -0.396 0.356 -0.698 0.212 -0.575 0.115 0.547 0.691
Oxaloacetate 0.557 0.749 0.847 0.042 0.352 0.473 0.137 0.759 0.133 -0.256 ... -0.122 0.383 -0.686 -0.074 -0.881 0.015 -0.546 -0.451 0.201 0.788
Serine -0.069 -0.310 -0.318 0.337 -0.113 0.014 -0.312 -0.417 -0.125 0.515 ... 0.033 0.144 0.267 -0.133 0.410 -0.196 0.395 -0.103 -0.252 -0.249
Glucuronic acid 0.075 -0.001 0.072 0.574 0.070 0.244 -0.511 0.150 -0.293 0.481 ... -0.289 0.698 -0.131 -0.453 -0.282 -0.359 -0.404 -0.509 -0.318 0.399
S-Adenosylhomocysteine -0.112 -0.106 -0.083 0.525 -0.067 0.207 -0.608 -0.181 -0.252 0.719 ... -0.102 0.518 0.027 -0.397 0.108 -0.308 0.017 -0.398 -0.389 0.114
Citrulline -0.220 -0.286 -0.274 0.606 0.010 0.146 -0.672 -0.235 -0.428 0.715 ... -0.166 0.513 0.252 -0.383 0.248 -0.404 0.026 -0.333 -0.369 -0.014
NADH -0.369 -0.459 -0.472 0.353 -0.147 -0.096 -0.501 -0.428 -0.396 0.655 ... -0.046 0.220 0.443 -0.274 0.491 -0.348 0.143 -0.068 -0.311 -0.324
FAD -0.020 -0.166 -0.097 0.620 -0.052 0.161 -0.647 -0.253 -0.319 0.676 ... -0.324 0.643 0.003 -0.532 0.063 -0.453 0.041 -0.539 -0.517 0.046
FMN 0.031 -0.053 -0.033 0.720 0.183 0.319 -0.567 0.106 -0.424 0.479 ... -0.377 0.726 0.006 -0.352 -0.133 -0.358 -0.330 -0.505 -0.335 0.299
Acetyl-CoA 0.046 -0.022 0.069 0.389 -0.066 0.106 -0.512 -0.197 -0.175 0.619 ... -0.189 0.551 -0.152 -0.607 -0.056 -0.463 0.037 -0.509 -0.473 0.132
ADP-ribose -0.008 0.060 0.098 -0.485 -0.224 -0.140 0.338 -0.117 0.303 -0.371 ... 0.260 -0.498 -0.152 0.338 0.130 0.644 0.459 0.273 0.315 -0.402
Glycerol 3-phosphate 0.139 0.337 0.327 0.388 0.270 0.431 -0.296 0.418 -0.208 0.345 ... -0.062 0.556 -0.274 -0.222 -0.340 -0.041 -0.506 -0.391 -0.089 0.493
Dihydroxyacetone phosphate -0.151 0.017 -0.039 0.748 0.164 0.457 -0.754 -0.255 -0.583 0.755 ... -0.229 0.596 0.122 -0.493 0.330 -0.252 0.117 -0.466 -0.495 -0.083
D-Ribulose 5-phosphate -0.043 -0.172 -0.175 0.751 0.142 0.259 -0.646 -0.122 -0.484 0.627 ... -0.309 0.690 0.131 -0.433 0.103 -0.398 -0.113 -0.471 -0.446 0.112
UDP-N-acetylglucosamine 0.439 0.244 0.338 0.704 0.174 0.378 -0.428 0.125 -0.275 0.106 ... -0.725 0.930 -0.617 -0.633 -0.430 -0.082 -0.106 -0.828 -0.523 0.303
N-Acetylglucosamine 1-phosphate 0.256 0.146 0.201 0.847 0.234 0.426 -0.613 0.076 -0.467 0.332 ... -0.696 0.973 -0.380 -0.650 -0.269 -0.239 -0.179 -0.801 -0.556 0.292
NAD 0.016 0.281 0.384 0.207 -0.041 0.227 -0.395 0.126 0.034 0.265 ... -0.203 0.366 -0.276 -0.269 -0.308 -0.327 -0.090 -0.484 -0.091 0.494
Fructose 6-phosphate 0.092 -0.426 -0.454 -0.210 -0.160 -0.452 0.402 -0.142 0.344 -0.334 ... -0.018 -0.193 0.089 0.190 0.096 0.098 0.085 0.286 0.018 -0.277
N-Acetylglucosamine 6-phosphate 0.255 0.143 0.195 0.853 0.211 0.428 -0.608 0.079 -0.461 0.347 ... -0.681 0.969 -0.379 -0.628 -0.267 -0.213 -0.193 -0.786 -0.552 0.297
GMP -0.340 -0.036 0.082 -0.316 -0.141 -0.117 -0.106 0.259 0.274 0.133 ... 0.176 -0.153 -0.103 0.064 -0.262 0.052 -0.227 0.150 0.303 0.231
Phosphoenolpyruvate 0.445 0.305 0.437 -0.243 -0.084 -0.096 0.260 0.228 0.446 -0.179 ... -0.150 0.158 -0.548 -0.294 -0.599 -0.106 -0.277 -0.251 -0.058 0.348
D-Sedoheptulose 7-phosphate 0.112 -0.271 -0.298 -0.317 -0.056 -0.404 0.374 -0.179 0.279 -0.156 ... 0.205 -0.251 0.115 0.035 0.134 0.065 0.090 0.306 0.014 -0.297
3-Phosphoglycerate -0.033 -0.045 -0.029 0.694 0.119 0.325 -0.669 -0.024 -0.409 0.623 ... -0.285 0.722 -0.068 -0.458 -0.006 -0.275 -0.098 -0.525 -0.451 0.173
Glutaryl-CoA 0.045 -0.102 -0.039 0.574 -0.015 0.222 -0.584 -0.229 -0.207 0.659 ... -0.251 0.578 -0.028 -0.451 0.076 -0.388 0.174 -0.521 -0.432 0.048
Succinyl-CoA -0.190 -0.251 -0.302 0.699 0.145 0.235 -0.690 -0.322 -0.557 0.681 ... -0.164 0.533 0.264 -0.411 0.402 -0.255 0.149 -0.326 -0.471 -0.209
Uridine 5'-diphosphate 0.232 0.138 0.180 0.898 0.237 0.466 -0.673 -0.007 -0.509 0.409 ... -0.662 0.961 -0.317 -0.626 -0.156 -0.212 -0.097 -0.817 -0.601 0.183
UDP-Glucuronate 0.168 -0.324 -0.184 0.405 -0.111 -0.207 -0.319 -0.106 -0.043 0.083 ... -0.632 0.668 -0.322 -0.578 -0.276 -0.283 -0.046 -0.500 -0.473 0.104
6-Phosphogluconate 0.014 0.164 0.181 -0.682 -0.016 -0.164 0.652 0.565 0.481 -0.558 ... 0.442 -0.580 -0.027 0.730 -0.383 0.292 -0.377 0.492 0.805 0.359
ATP -0.228 -0.481 -0.572 -0.256 -0.309 -0.376 0.235 -0.363 0.188 -0.078 ... 0.427 -0.480 0.399 0.368 0.446 0.257 0.279 0.530 0.099 -0.434
NADPH -0.224 0.069 -0.044 -0.544 -0.043 0.007 0.394 0.164 0.090 0.027 ... 1.000 -0.700 0.344 0.681 0.265 0.420 0.022 0.661 0.621 -0.048
GDP 0.354 0.229 0.282 0.847 0.299 0.454 -0.587 0.092 -0.422 0.290 ... -0.700 1.000 -0.446 -0.679 -0.320 -0.251 -0.169 -0.858 -0.611 0.308
S-Adenosylmethionine -0.546 -0.556 -0.709 -0.045 0.166 -0.214 -0.002 -0.178 -0.345 0.231 ... 0.344 -0.446 1.000 0.375 0.717 -0.201 0.028 0.646 0.345 -0.404
NADP -0.145 0.044 -0.099 -0.437 0.016 0.016 0.522 0.301 0.198 -0.308 ... 0.681 -0.679 0.375 1.000 0.202 0.438 -0.078 0.624 0.735 -0.012
Oxidized glutathione -0.451 -0.493 -0.701 0.132 -0.058 -0.096 -0.126 -0.594 -0.366 0.327 ... 0.265 -0.320 0.717 0.202 1.000 0.162 0.545 0.437 -0.098 -0.789
Ribose 1,5-bisphosphate -0.110 0.167 0.077 -0.167 -0.092 0.155 0.126 0.083 0.002 -0.210 ... 0.420 -0.251 -0.201 0.438 0.162 1.000 0.199 0.248 0.234 -0.213
Fructose 1,6-bisphosphate -0.005 -0.332 -0.335 -0.035 -0.255 -0.203 -0.050 -0.708 0.083 0.073 ... 0.022 -0.169 0.028 -0.078 0.545 0.199 1.000 0.003 -0.397 -0.753
Arginine -0.489 -0.387 -0.486 -0.625 -0.135 -0.412 0.436 0.040 0.092 -0.070 ... 0.661 -0.858 0.646 0.624 0.437 0.248 0.003 1.000 0.642 -0.288
Ornithine -0.226 0.105 0.075 -0.552 0.167 -0.016 0.435 0.599 0.145 -0.238 ... 0.621 -0.611 0.345 0.735 -0.098 0.234 -0.397 0.642 1.000 0.268
Lysine 0.192 0.559 0.615 0.050 0.206 0.347 -0.030 0.750 0.138 -0.115 ... -0.048 0.308 -0.404 -0.012 -0.789 -0.213 -0.753 -0.288 0.268 1.000

59 rows × 59 columns

Plot the correlation heatmap using seaborn

Hide code cell source

 
plt.rcParams["xtick.labelsize"], plt.rcParams["ytick.labelsize"] = 5, 5
fig, ax = plt.subplots(figsize=(7.1, 6))
heatmap = sns.heatmap(
    corr,
    cmap="vlag",
    center=0,
    square=True,
    linewidths=0.1,
    cbar_kws={"label": "Pearson r"},
    ax=ax,
)
ax.set(title="Correlation Heatmap")
fig.tight_layout()
../_images/3ffd5bee6aa3a55ab2040ddf331379f94cd94db29159c037cf0cfd35854beea7.png 
# If you only want to keep the lower triangle of the correlation matrix to have
# unique values of interst, you can use the utility function:
lower_corr = ca.corr_lower_triangle(data.drop(columns=["group"]), method="pearson")
lower_corr
Uracil Adenine Hypoxanthine Uridine Creatinine Adenosine Cytosine Inosine Guanine Tryptophan ... NADPH GDP S-Adenosylmethionine NADP Oxidized glutathione Ribose 1,5-bisphosphate Fructose 1,6-bisphosphate Arginine Ornithine Lysine
Uracil NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Adenine 0.451 NaN NaN NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Hypoxanthine 0.511 0.936 NaN NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Uridine 0.154 0.128 0.030 NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Creatinine 0.150 0.369 0.263 0.398 NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Adenosine 0.274 0.780 0.625 0.591 0.590 NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Cytosine 0.469 0.055 0.044 -0.645 -0.181 -0.296 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Inosine 0.146 0.569 0.579 -0.026 0.531 0.478 0.093 NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Guanine 0.181 0.029 0.150 -0.641 -0.591 -0.444 0.509 -0.084 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Tryptophan -0.148 -0.111 -0.099 0.406 0.064 0.262 -0.466 -0.073 -0.488 NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Leucine -0.070 0.444 0.392 -0.051 0.394 0.392 0.035 0.780 0.011 0.022 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Guanosine -0.524 -0.532 -0.670 -0.331 -0.400 -0.406 0.111 -0.581 0.132 0.114 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Choline 0.271 0.891 0.841 -0.069 0.398 0.666 0.155 0.690 -0.027 -0.067 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Isoleucine -0.085 0.397 0.328 -0.105 0.396 0.339 0.110 0.768 0.056 -0.066 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Betaine 0.173 0.724 0.522 0.348 0.695 0.818 -0.088 0.492 -0.386 0.133 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Valine 0.385 0.715 0.804 0.031 0.211 0.446 0.071 0.779 0.117 -0.176 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Alanine 0.025 0.306 0.426 0.416 0.131 0.390 -0.500 0.474 -0.245 0.450 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Threonine 0.315 0.102 -0.009 -0.047 0.238 0.108 0.450 0.222 0.154 -0.371 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Nicotinamide riboside -0.563 -0.539 -0.693 -0.438 -0.202 -0.419 0.290 -0.292 0.029 -0.116 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Gamma-Aminobutyric acid 0.413 0.790 0.700 -0.044 0.602 0.549 0.271 0.565 -0.040 -0.418 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Glutamine 0.273 0.550 0.568 -0.405 0.193 0.145 0.493 0.749 0.370 -0.577 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Oxaloacetate 0.557 0.749 0.847 0.042 0.352 0.473 0.137 0.759 0.133 -0.256 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Serine -0.069 -0.310 -0.318 0.337 -0.113 0.014 -0.312 -0.417 -0.125 0.515 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Glucuronic acid 0.075 -0.001 0.072 0.574 0.070 0.244 -0.511 0.150 -0.293 0.481 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
S-Adenosylhomocysteine -0.112 -0.106 -0.083 0.525 -0.067 0.207 -0.608 -0.181 -0.252 0.719 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Citrulline -0.220 -0.286 -0.274 0.606 0.010 0.146 -0.672 -0.235 -0.428 0.715 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NADH -0.369 -0.459 -0.472 0.353 -0.147 -0.096 -0.501 -0.428 -0.396 0.655 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
FAD -0.020 -0.166 -0.097 0.620 -0.052 0.161 -0.647 -0.253 -0.319 0.676 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
FMN 0.031 -0.053 -0.033 0.720 0.183 0.319 -0.567 0.106 -0.424 0.479 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Acetyl-CoA 0.046 -0.022 0.069 0.389 -0.066 0.106 -0.512 -0.197 -0.175 0.619 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
ADP-ribose -0.008 0.060 0.098 -0.485 -0.224 -0.140 0.338 -0.117 0.303 -0.371 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Glycerol 3-phosphate 0.139 0.337 0.327 0.388 0.270 0.431 -0.296 0.418 -0.208 0.345 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Dihydroxyacetone phosphate -0.151 0.017 -0.039 0.748 0.164 0.457 -0.754 -0.255 -0.583 0.755 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
D-Ribulose 5-phosphate -0.043 -0.172 -0.175 0.751 0.142 0.259 -0.646 -0.122 -0.484 0.627 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
UDP-N-acetylglucosamine 0.439 0.244 0.338 0.704 0.174 0.378 -0.428 0.125 -0.275 0.106 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
N-Acetylglucosamine 1-phosphate 0.256 0.146 0.201 0.847 0.234 0.426 -0.613 0.076 -0.467 0.332 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NAD 0.016 0.281 0.384 0.207 -0.041 0.227 -0.395 0.126 0.034 0.265 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Fructose 6-phosphate 0.092 -0.426 -0.454 -0.210 -0.160 -0.452 0.402 -0.142 0.344 -0.334 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
N-Acetylglucosamine 6-phosphate 0.255 0.143 0.195 0.853 0.211 0.428 -0.608 0.079 -0.461 0.347 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
GMP -0.340 -0.036 0.082 -0.316 -0.141 -0.117 -0.106 0.259 0.274 0.133 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Phosphoenolpyruvate 0.445 0.305 0.437 -0.243 -0.084 -0.096 0.260 0.228 0.446 -0.179 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
D-Sedoheptulose 7-phosphate 0.112 -0.271 -0.298 -0.317 -0.056 -0.404 0.374 -0.179 0.279 -0.156 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
3-Phosphoglycerate -0.033 -0.045 -0.029 0.694 0.119 0.325 -0.669 -0.024 -0.409 0.623 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Glutaryl-CoA 0.045 -0.102 -0.039 0.574 -0.015 0.222 -0.584 -0.229 -0.207 0.659 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Succinyl-CoA -0.190 -0.251 -0.302 0.699 0.145 0.235 -0.690 -0.322 -0.557 0.681 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
Uridine 5'-diphosphate 0.232 0.138 0.180 0.898 0.237 0.466 -0.673 -0.007 -0.509 0.409 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
UDP-Glucuronate 0.168 -0.324 -0.184 0.405 -0.111 -0.207 -0.319 -0.106 -0.043 0.083 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
6-Phosphogluconate 0.014 0.164 0.181 -0.682 -0.016 -0.164 0.652 0.565 0.481 -0.558 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
ATP -0.228 -0.481 -0.572 -0.256 -0.309 -0.376 0.235 -0.363 0.188 -0.078 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
NADPH -0.224 0.069 -0.044 -0.544 -0.043 0.007 0.394 0.164 0.090 0.027 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
GDP 0.354 0.229 0.282 0.847 0.299 0.454 -0.587 0.092 -0.422 0.290 ... -0.700 NaN NaN NaN NaN NaN NaN NaN NaN NaN
S-Adenosylmethionine -0.546 -0.556 -0.709 -0.045 0.166 -0.214 -0.002 -0.178 -0.345 0.231 ... 0.344 -0.446 NaN NaN NaN NaN NaN NaN NaN NaN
NADP -0.145 0.044 -0.099 -0.437 0.016 0.016 0.522 0.301 0.198 -0.308 ... 0.681 -0.679 0.375 NaN NaN NaN NaN NaN NaN NaN
Oxidized glutathione -0.451 -0.493 -0.701 0.132 -0.058 -0.096 -0.126 -0.594 -0.366 0.327 ... 0.265 -0.320 0.717 0.202 NaN NaN NaN NaN NaN NaN
Ribose 1,5-bisphosphate -0.110 0.167 0.077 -0.167 -0.092 0.155 0.126 0.083 0.002 -0.210 ... 0.420 -0.251 -0.201 0.438 0.162 NaN NaN NaN NaN NaN
Fructose 1,6-bisphosphate -0.005 -0.332 -0.335 -0.035 -0.255 -0.203 -0.050 -0.708 0.083 0.073 ... 0.022 -0.169 0.028 -0.078 0.545 0.199 NaN NaN NaN NaN
Arginine -0.489 -0.387 -0.486 -0.625 -0.135 -0.412 0.436 0.040 0.092 -0.070 ... 0.661 -0.858 0.646 0.624 0.437 0.248 0.003 NaN NaN NaN
Ornithine -0.226 0.105 0.075 -0.552 0.167 -0.016 0.435 0.599 0.145 -0.238 ... 0.621 -0.611 0.345 0.735 -0.098 0.234 -0.397 0.642 NaN NaN
Lysine 0.192 0.559 0.615 0.050 0.206 0.347 -0.030 0.750 0.138 -0.115 ... -0.048 0.308 -0.404 -0.012 -0.789 -0.213 -0.753 -0.288 0.268 NaN

59 rows × 59 columns

Plot the lower triangle correlations as a histrogram to see the distribution of correlation values

Hide code cell source

 
ax = lower_corr.stack().plot.hist(
    bins=50,
    grid=False,
    figsize=(6, 4),
    title="Distribution of Pearson correlation values",
    xlabel="Pearson r",
    ylabel="Frequency",
    xlim=(-1.02, 1.02),
)
../_images/5cb63ecfd9d7d72408208c8e4b8212618d6e08e71deb249d44dfc907e693c0d2.png

or to find the strongest correlations, which you might want to filter further for uninteresting correlation between redundant features.

Hide code cell source

 
lower_corr_stack = lower_corr.stack()
idx_largerst_corr = lower_corr_stack.abs().sort_values(ascending=False).head(20).index
lower_corr_stack.loc[idx_largerst_corr]

N-Acetylglucosamine 6-phosphate  N-Acetylglucosamine 1-phosphate   0.997
Isoleucine                       Leucine                           0.986
Uridine 5'-diphosphate           N-Acetylglucosamine 6-phosphate   0.974
GDP                              N-Acetylglucosamine 1-phosphate   0.973
Uridine 5'-diphosphate           N-Acetylglucosamine 1-phosphate   0.971
GDP                              N-Acetylglucosamine 6-phosphate   0.969
                                 Uridine 5'-diphosphate            0.961
FAD                              Citrulline                        0.955
D-Ribulose 5-phosphate           Citrulline                        0.947
FAD                              S-Adenosylhomocysteine            0.944
Oxaloacetate                     Valine                            0.941
Glutaryl-CoA                     FAD                               0.941
Citrulline                       S-Adenosylhomocysteine            0.939
Glutaryl-CoA                     S-Adenosylhomocysteine            0.937
D-Ribulose 5-phosphate           FAD                               0.936
Hypoxanthine                     Adenine                           0.936
FMN                              Glucuronic acid                   0.934
Succinyl-CoA                     Citrulline                        0.933
D-Ribulose 5-phosphate           FMN                               0.932
N-Acetylglucosamine 1-phosphate  UDP-N-acetylglucosamine           0.930
dtype: float64

This function can be used to compute multiple correlation methods at once and compare them, here for the first four features.

It only works on numeric values.

Hide code cell source

 
corr = list()
for method in ["pearson", "spearman", "kendall"]:
    _corr = (
        ca.corr_lower_triangle(data.iloc[:, :4], method=method, numeric_only=True)
        .stack()
        .rename(method)
    )
    corr.append(_corr)
corr = pd.concat(corr, axis=1).sort_values(by="pearson", ascending=True)
_ = corr.plot(
    style=".",
    ylim=(-1.05, 1.05),
    alpha=0.5,
    rot=45,
)
../_images/fab16b53182898cfb8f1bd191cc2ddd95d8740a9d0ad63d208cfed3f909105ae.png

Filtering correlations based on p-values with multiple testing correction

  • the p-value depends on the number of samples

  • and the strenght of the correlation

res = ca.calculate_correlations(data.iloc[:, 0], data.iloc[:, 1], method="pearson")
print(res)

CorrelationCoefficient(coefficient=0.4512, pvalue=0.0401)

For the first four features, we would only keep one significant correlation after multiple testing correction with the Benjamini-Hochberg method.

Note that the p-value for the correlation between the same feature is set to one, not zero as in scipy, to not consider these correlation in downstream analysis by accident.

correlation = ca.run_correlation(
    data.iloc[:, :4], alpha=0.05, group="group", method="pearson", correction="fdr_bh"
)
correlation
node1 node2 weight pvalue padj rejected
0 Uracil Uracil 1.000 1.000 1.000 False
1 Adenine Uracil 0.451 0.040 0.134 False
2 Adenine Adenine 1.000 1.000 1.000 False
3 Hypoxanthine Uracil 0.511 0.018 0.089 False
4 Hypoxanthine Adenine 0.936 0.000 0.000 True
5 Hypoxanthine Hypoxanthine 1.000 1.000 1.000 False
6 Uridine Uracil 0.154 0.504 1.000 False
7 Uridine Adenine 0.128 0.579 1.000 False
8 Uridine Hypoxanthine 0.030 0.896 1.000 False
9 Uridine Uridine 1.000 1.000 1.000 False

The efficient correlation calculation can be used to compute the correlation matrix and p-value matrix for larger datasets.

corr, p = ca.run_efficient_correlation(data.iloc[:, :4], method="spearman")
pd.DataFrame(p)
0 1 2 3
0 0.000 NaN NaN NaN
1 0.070 0.000 NaN NaN
2 0.160 0.000 0.000 NaN
3 0.434 0.862 0.703 0.000

you can verify the results against scipy.stats.spearmanr

r, p = scipy.stats.spearmanr(data.iloc[:, :4])
pd.DataFrame(p)
0 1 2 3
0 0.000 0.070 0.160 0.434
1 0.070 0.000 0.000 0.862
2 0.160 0.000 0.000 0.703
3 0.434 0.862 0.703 0.000

same for pearson correlation

r, p = ca.run_efficient_correlation(data.iloc[:, :3], method="pearson")
pd.DataFrame(p)
0 1 2
0 1.000 NaN NaN
1 0.040 1.000 NaN
2 0.018 0.000 1.000 
r_20, p_20 = scipy.stats.pearsonr(data.iloc[:, 0], data.iloc[:, 2])
assert r[2, 0] - r_20 < 1e-8
assert p[2, 0] - p_20 < 1e-8
desc = (
    f"Correlation coefficient (r) in row three and the first column is {r_20:.3f} \n"
    f"and the corresponding p-value is {p_20:.3f}."
)
print(desc)

Correlation coefficient (r) in row three and the first column is 0.511 
and the corresponding p-value is 0.018.

To calculate p-values for the correlation matrix, e.g. using pandas corr you can use:

res = ca.calculate_pvalue_correlation(
    data.iloc[:, :3].corr(method="pearson").values, n_obs=data.shape[0]
)
pd.DataFrame(res)
0 1 2
0 1.000 0.040 0.018
1 0.040 1.000 0.000
2 0.018 0.000 1.000

Histogram of the data#

We often want to plot the distribution of feature values. Sometimes you want to use custom bins, e.g. to align multiple histograms. Here we plot and compute a histogram frequencies with custom bins.

Hide code cell source

 
view = data.select_dtypes(include="number")
bins = np.arange(int(view.min(axis=None)), int(view.max(axis=None)) + 1, step=1)

for col in view.columns[:4]:
    ax = view[col].plot.hist(
        bins=bins,
        alpha=0.5,
        xlabel="Value",
        ylabel="Frequency",
    )
ax.title.set_text("Histogram with custom bins")
_ = ax.legend()
../_images/37e5f5cd6b65d07db8cd054f581276eeea55de77cf50c3683390de8957d8ac50.png

We could easily plot the histogram with custom bins, but if we want the count data of the histogram frequencies in a dataframe, acore has a utility function which can be used to compute the histogram frequencies for a feature.

Hide code cell source

 
hist_series = []
for col in view.columns[:4]:
    s = view[col]
    ret = ea.get_histogram_series(s, bins)
    hist_series.append(ret.rename(col))

hist_df = pd.concat(hist_series, axis=1)
hist_df
Uracil Adenine Hypoxanthine Uridine
bin_number bin_start bin_end
14 15 0 0 0 0 0
15 16 1 0 0 0 0
16 17 2 0 0 0 0
17 18 3 0 0 0 0
18 19 4 0 0 0 5
19 20 5 0 0 0 12
20 21 6 0 0 0 4
21 22 7 0 19 0 0
22 23 8 0 2 14 0
23 24 9 0 0 7 0
24 25 10 0 0 0 0
25 26 11 1 0 0 0
26 27 12 18 0 0 0
27 28 13 2 0 0 0

Summary statistics#

You can use pandas to compute summary statistics for the data, specifying the percentiles you are interested in.

summary_stats = (
    data.select_dtypes(include="number").describe(percentiles=[0.5]).T
).rename(columns={"50%": "median"})
summary_stats
count mean std min median max
Uracil 21.000 26.431 0.339 25.721 26.313 27.293
Adenine 21.000 21.760 0.187 21.511 21.738 22.169
Hypoxanthine 21.000 22.985 0.234 22.681 22.925 23.517
Uridine 21.000 19.401 0.617 18.269 19.417 20.388
Creatinine 21.000 15.074 0.239 14.819 14.997 15.786
Adenosine 21.000 19.622 0.136 19.415 19.600 19.998
Cytosine 21.000 21.283 0.364 20.358 21.347 21.739
Inosine 21.000 22.784 0.128 22.564 22.767 23.090
Guanine 21.000 21.411 0.429 20.467 21.425 22.140
Tryptophan 21.000 22.087 0.155 21.882 22.064 22.389
Leucine 21.000 23.589 0.099 23.428 23.579 23.740
Guanosine 21.000 18.074 1.063 16.131 18.151 19.747
Choline 21.000 19.164 0.241 18.790 19.160 19.718
Isoleucine 21.000 23.011 0.108 22.803 23.006 23.171
Betaine 21.000 20.109 0.168 19.891 20.115 20.623
Valine 21.000 24.240 0.155 23.996 24.225 24.533
Alanine 21.000 23.783 0.147 23.500 23.817 24.080
Threonine 21.000 23.605 0.857 22.361 23.379 26.042
Nicotinamide riboside 21.000 20.900 0.439 20.065 20.957 21.629
Gamma-Aminobutyric acid 21.000 19.794 0.350 19.112 19.810 20.319
Glutamine 21.000 21.349 0.491 20.416 21.380 22.088
Oxaloacetate 21.000 27.604 0.420 26.971 27.467 28.315
Serine 21.000 22.647 0.527 21.980 22.519 24.439
Glucuronic acid 21.000 19.291 0.508 18.181 19.486 19.767
S-Adenosylhomocysteine 21.000 17.475 0.466 16.689 17.516 18.550
Citrulline 21.000 21.386 0.463 20.543 21.446 22.394
NADH 21.000 21.025 0.584 19.863 21.160 22.085
FAD 21.000 20.664 0.685 19.542 20.663 22.018
FMN 21.000 22.713 0.599 21.593 22.909 23.448
Acetyl-CoA 21.000 19.697 0.990 18.145 19.528 21.846
ADP-ribose 21.000 19.704 0.518 18.991 19.538 20.912
Glycerol 3-phosphate 21.000 21.214 0.562 19.987 21.382 22.233
Dihydroxyacetone phosphate 21.000 18.650 0.389 18.143 18.565 19.429
D-Ribulose 5-phosphate 21.000 23.788 0.760 22.458 23.972 24.941
UDP-N-acetylglucosamine 21.000 20.797 1.199 18.684 21.179 22.916
N-Acetylglucosamine 1-phosphate 21.000 23.645 1.593 20.987 24.555 24.861
NAD 21.000 21.778 0.762 20.866 21.458 23.919
Fructose 6-phosphate 21.000 24.995 0.219 24.477 25.013 25.359
N-Acetylglucosamine 6-phosphate 21.000 20.870 2.785 15.962 22.417 22.973
GMP 21.000 20.174 0.455 18.353 20.236 20.554
Phosphoenolpyruvate 21.000 23.614 0.303 22.985 23.613 24.115
D-Sedoheptulose 7-phosphate 21.000 23.743 0.187 23.412 23.767 24.086
3-Phosphoglycerate 21.000 22.264 0.617 20.949 22.442 23.515
Glutaryl-CoA 21.000 18.549 0.293 18.139 18.519 19.331
Succinyl-CoA 21.000 22.261 0.619 21.192 22.257 23.523
Uridine 5'-diphosphate 21.000 24.123 0.497 23.214 24.371 24.577
UDP-Glucuronate 21.000 15.162 0.531 14.199 15.352 15.895
6-Phosphogluconate 21.000 19.198 0.158 18.908 19.243 19.402
ATP 21.000 20.346 0.328 19.856 20.368 21.409
NADPH 21.000 24.984 0.153 24.798 24.925 25.341
GDP 21.000 21.924 0.622 20.760 22.204 22.561
S-Adenosylmethionine 21.000 22.968 0.456 21.270 23.109 23.347
NADP 21.000 24.615 0.134 24.395 24.580 24.883
Oxidized glutathione 21.000 26.813 0.364 26.072 26.879 27.253
Ribose 1,5-bisphosphate 21.000 21.833 0.106 21.527 21.821 22.066
Fructose 1,6-bisphosphate 21.000 20.399 0.124 20.164 20.385 20.605
Arginine 21.000 23.979 0.212 23.611 23.927 24.384
Ornithine 21.000 25.091 0.098 24.917 25.097 25.260
Lysine 21.000 21.524 0.340 21.040 21.513 22.330

Done.