acore.imputation_analysis package

acore.imputation_analysis package#

imputation_KNN(data: DataFrame, drop_cols: Iterable[str] | None = None, group: str | None = None, cutoff=0.6, alone=True, n_neighbors=3)[source]#

K-Nearest Neighbors imputation for pandas dataframes with missing data. For more information visit fancyimpute.

Parameters:

  • data – pandas dataframe with samples as rows and protein identifiers as columns (with additional columns ‘group’, ‘sample’ and ‘subject’).

  • group (str) – column label containing group identifiers, restricted to be one single column for now.

  • drop_cols (list) – column labels to be dropped. Final dataframe should only have gene/protein/etc identifiers as columns.

  • cutoff (float) – minimum fraction of valid values required to impute a each column.

  • alone (bool) – if True removes all columns with any missing values after initial imputation.

Returns:

Pandas dataframe with samples as rows and protein identifiers as columns.

Example:

result = imputation_KNN(data,
            drop_cols=['group', 'sample', 'subject'],
            group='group', cutoff=0.6, alone=True
)
imputation_mixed_norm_KNN(data: DataFrame, drop_cols: Iterable[str] | None = None, shift: float = 1.8, nstd: float = 0.3, group: str = 'group', cutoff: float = 0.6, random_state: int = 112736, n_neighbors: int = 3)[source]#

Missing values are replaced in two steps:

  1. using k-Nearest Neighbors we impute protein columns with a higher ratio of missing/valid values than the defined cutoff,

  2. the remaining missing values are replaced by random numbers that are drawn from a normal distribution.

Parameters:

  • data – pandas dataframe with samples as rows and protein identifiers as columns (with additional columns ‘group’, ‘sample’ and ‘subject’).

  • group (str) – column label containing group identifiers.

  • drop_cols (list) – list of column labels to be set as dataframe index.

  • shift (float) – specifies the amount by which the distribution used for the random numbers is shifted downwards. This is in units of the standard deviation of the valid data.

  • nstd (float) – defines the width of the Gaussian distribution relative to the standard deviation of measured values. A value of 0.5 would mean that the width of the distribution used for drawing random numbers is half of the standard deviation of the data.

  • cutoff (float) – minimum ratio of missing/valid values required to impute in each column.

Returns:

Pandas dataframe with samples as rows and protein identifiers as columns.

Example:

result = imputation_mixed_norm_KNN(data,
            drop_cols=['group', 'sample', 'subject'],
            shift = 1.8, nstd = 0.3, group='group', cutoff=0.6
)
imputation_normal_distribution(data: DataFrame, drop_cols: Iterable[str] | None = None, shift: float = 1.8, nstd: float = 0.3, random_state: int = 112736)[source]#

Missing values will be replaced by random numbers that are drawn from a normal distribution. The imputation is done for each sample (across all proteins) separately. For more information visit replacemissingfromgaussian in coxdocs from MaxQuant. The basic assumptions is that given normally distributed missing values in a sample are low abundant and are therefore replace with a downshifted minimum value. There is no control on if the drawn replacement values are below the absolute observed minimum of that protein at all, which can lead to false positives or negatives in differential expression analysis that considers imputed values.

Parameters:

  • data – pandas dataframe with samples as rows and protein identifiers as columns (with additional columns ‘group’, ‘sample’ and ‘subject’).

  • drop_cols (list) – list of column labels to be dropped from the imputation.

  • shift (float) – specifies the amount by which the distribution used for the random numbers is shifted downwards. This is in units of the standard deviation of the valid data.

  • nstd (float) – defines the width of the Gaussian distribution relative to the standard deviation of measured values. A value of 0.5 would mean that the width of the distribution used for drawing random numbers is half of the standard deviation of the data.

Returns:

Pandas dataframe with samples as rows and protein identifiers as columns.

Example:

result = imputation_normal_distribution(data,
            drop_cols=['group', 'sample', 'subject'],
            shift = 1.8, nstd = 0.3
)
imputation_zeros(data: DataFrame, on_cols: Iterable[str] | None = None, on_rows: Iterable[str] | None = None, drop_cols: Iterable[str] | None = None)[source]#

Replace missing values with zeros.

Parameters:

  • data – DataFrame with samples as rows and features as columns.

  • on_cols (list) – columns to fill with zeros. If None, all numeric columns are filled. Non-numeric columns in “on_cols” will raise a TypeError.

  • on_rows (list) – row index labels to restrict imputation to. If None, all rows are imputed. Useful for imputing only a subset of samples (e.g. QCs, blanks, controls) while leaving others untouched.

  • drop_cols (list) – columns to permanently drop before imputation. If a column appears in both “on_cols” and “drop_cols” it will be dropped and a warning is emitted.

Returns:

DataFrame with missing values in the target columns replaced by zero.

Example

result = imputation_zeros(data, on_cols=[‘featureA’, ‘featureB’]) result = imputation_zeros(data, on_rows=[‘QC1’, ‘QC2’, ‘blank1’])

imputation_half_minimum(data: DataFrame, on_cols: Iterable[str] | None = None, on_rows: Iterable[str] | None = None, drop_cols: Iterable[str] | None = None)[source]#

Replace missing values with half the per-column minimum of observed values.

Parameters:

  • data – DataFrame with samples as rows and features as columns.

  • on_cols (list) – columns to impute. If None, all numeric columns are used. Non-numeric columns in on_cols will raise a TypeError.

  • on_rows (list) – row index labels to restrict imputation to. If None, all rows are imputed. When provided, the per-column minimum is also computed from only those rows, so each subset gets its own half-minimum (e.g. blanks are imputed with half the blank-minimum).

  • drop_cols (list) – columns to permanently drop before imputation. If a column appears in both on_cols and drop_cols it will be dropped and a warning is emitted.

Returns:

DataFrame with missing values replaced by half the per-column minimum.

Example:

result = imputation_half_minimum(data, on_cols=['featureA', 'featureB'])
result = imputation_half_minimum(data, on_rows=['blank1', 'blank2'])