acore.drift_correction.loess_drift_correction module

Functions for metabolomics drift correction.

filter_features_by_qc(df: DataFrame, qc_rows: list, threshold: float = 0.5) → DataFrame

Filter features in a DataFrame based on quality control (QC) completeness.

This function removes columns (features) that do not meet a minimum number of valid (non-missing) QC values. The minimum number of required valid values is computed as ceil(n_qc * (1 - threshold)), where n_qc is the number of QC rows.

Parameters:

• df (pandas.DataFrame) – Input DataFrame with samples as rows and features as columns.

• qc_rows (list) – List of row indices corresponding to QC samples.

• threshold (float) – Fraction (between 0 and 1) indicating the maximum allowed proportion of missing QC values per feature. For example, threshold=0.6 allows up to 60% of QC values to be missing; so a feature with 3 out of 5 QC values present (40% present, 60% missing) would be retained. Defaults to 0.5.

Returns pandas.DataFrame:

Filtered DataFrame containing only columns with sufficient valid QC values.

Raises:

ValueError – If threshold is not between 0 and 1.

Example

import pandas as pd import numpy as np

df = pd.DataFrame({

‘A’: [1.0, np.nan, 2.0], ‘B’: [np.nan, 3.0, np.nan], ‘C’: [4.0, np.nan, 6.0]

}, index=[‘QC1’, ‘QC2’, ‘QC3’])

filtered = filter_features_by_qc(df, qc_rows=[‘QC1’, ‘QC2’, ‘QC3’], threshold=0.5) print(filtered) # Output: columns with at least 2 valid QC values (since ceil(3 * (1 - 0.5)) = 2)

qc_rlsc_loess(x_qc, y_qc, x_all, always_use_default=False, default=0.75, alpha_candidates=array([0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 1.]))

Estimate a QC-based drift curve using LOESS smoothing with leave-one-out cross-validation (LOOCV) to select the optimal smoothing span (alpha).

This function: 1. Tests multiple LOESS spans (alpha values). 2. Fits LOESS to QC points for each candidate span. 3. Performs LOOCV to compute prediction error for each span. 4. Selects the alpha producing the lowest LOOCV error. 5. Fits LOESS once more using the best alpha. 6. Interpolates the LOESS fit to all sample injection orders

using a cubic spline, with clamping outside the QC range.

Parameters:

• x_qc (array-like) – Injection order of QC samples (numeric).

• y_qc (array-like) – Intensity values of QC samples corresponding to x_qc.

• x_all (array-like) – Injection order of all samples (QCs + regular samples) in the same order as data rows.

• always_use_default (bool, optional) – If True, the alpha value 0.75 is used for all values. LOOCV is skipped. This option is less computationally heavy.

• alpha_candidates (list of float, optional) – List of LOESS smoothing parameters (fractions of data used in local regression) to evaluate during optimization. Default is [0.4, 0.6, 0.8, 1.0].

Returns:

• drift_curve (ndarray) – The estimated drift correction curve evaluated at each injection order in x_all. Values outside the QC range are clamped to the nearest in-range LOESS value.

• best_alpha (float) – The alpha value producing the lowest LOOCV error.

Notes

• LOESS fits enforce a minimum fraction of (λ + 1) / n, with λ=1 for linear LOESS.

• CubicSpline is used for interpolation without extrapolation. Out-of-range values are manually clamped.

• Drift curve values are clipped to be strictly positive (minimum 1e-6) to prevent division instability.

run_loess_drift_correction(data, qc_rows, sample_rows, sample_order: DataFrame, filter_percent: float = None, qc_min_threshold: int = 4, always_use_default=False, default=0.75)

Perform QC-based drift correction across multiple features using LOESS regression and spline interpolation.

For each feature: 1. Extract QC intensities and corresponding injection order. 2. Optionally filter features based on QC completeness. 3. Compute QC relative standard deviation (RSD). 4. If sufficient QC points exist, estimate a drift curve using

qc_rlsc_loess, finding the best alpha smoothing span with leave-one-out cross validation (LOOCV).

5. Normalize all intensities by dividing by the drift curve and scaling to the QC median.

6. Record drift parameters and correction metadata.

Parameters:

• data (pandas.DataFrame) – Input intensity matrix with samples as rows and features as columns.

• qc_rows (list of str) – Row indices corresponding to QC injections.

• sample_rows (list of str) – Row indices corresponding to biological samples.

• sample_order (pandas.DataFrame) – Table mapping file names to injection order. Must contain columns “File Name” and “Sample ID”. Sample ID must be numeric.

• filter_percent (float, optional) – Minimum proportion of QC values that must be non-missing for a feature to be retained (e.g., 0.6 means at least 60% of QCs must be present).

• qc_min_threshold (int, optional) – Minimum number of QC values required to perform drift correction. Features with fewer QCs are returned uncorrected.

• always_use_default (bool, optional) – If True, LOOCV is skipped and default is used for the smoothing span. This option is less computationally heavy.

• default (float, optional) – Default alpha to use when always_use_default=True or when LOOCV fails. Defaults to 0.75.

Returns:

• corrected_df (pandas.DataFrame) – Full input DataFrame with corrected values applied to sample_rows + qc_rows. Rows outside those arguments are returned unchanged. Features that fail QC requirements are returned unchanged.

• correction_info (dict) – Dictionary keyed by feature name, containing: - ‘alpha’: selected LOESS alpha (or None if skipped) - ‘drift_curve’: the evaluated drift correction vector - ‘y_qc’: QC intensities used - ‘x_qc’: QC injection orders - ‘rsd_qc’: QC relative standard deviation - ‘median’: QC median intensity (used for scaling) - ‘y_all’: original intensities - ‘status’: “corrected”, “skipped_due_to_few_qcs”, or error note

Notes

• Features with insufficient QC points are not corrected.

• High QC RSD (>20%) is flagged but does not prevent correction.

• Drift correction rescales intensities so that QC medians remain unchanged.

• Sample names in data and sample_order must match exactly.