Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Inplement linting with Ruff #674

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

quinto-clara
Copy link

@quinto-clara quinto-clara commented Dec 20, 2024

Resolves #550

Description :

The Pull Request implements linting on xDEM in order to guarantee the same code quality as demcompare.
The idea was to limit the number of tools and to apply the same rules as demcompare.
That's why we choose to implement Ruff in Xdem, a fast python linter and code formatter gathering a lot of functionalities and replacing pylint, black, flake8 and isort.

Key changes :

  • Ruff configuration :

    • ruff.toml file addded including the following rules :

      • Target version of python : 3.10
      • The formatter will wraps lines at a length of 120 and use 4 spaces per tabulation when a line exceed 120 characters.
      • Ruff will ignore 46 lint rules (for now)
    • Ruff added in makefile

    • Ruff added in CI file

    • Ruff added in setup.cfg file

  • Code correction :

    Ruff returned a lot of errors related to 108 lint rules :

    • 62 of them has been fixed, the remaining is ignored and can be fixed later

@vschaffn
Copy link
Contributor

In the tests, the protected attributes (like _need_vars) were changed to non-protected attributes (for example, need_vars). This is causing issues because the non-protected attributes may not actually exist in the class in the same form, which leads to errors. Protected attributes (with a leading underscore) are intentionally kept as internal members of the class and might not have corresponding non-protected versions.

It seems that these changes could have been made in response to warnings from Ruff or due to the use of the --fix option. However, the removal of the leading underscore breaks the logic, since the non-protected version of these attributes may not exist.

To fix this:

  • The protected attributes should be restored with their leading underscores.
  • We may also want to review the Ruff configuration to ensure it's not suggesting incorrect automatic changes.

@vschaffn
Copy link
Contributor

To apply ruff in pre-commits and avoid conflicts with other linters, you would need to add ruff to .pre-commit-config.yaml and remove isort, pylint, flake8 and black.

@@ -5,6 +5,7 @@
############### GLOBAL VARIABLES ######################
.DEFAULT_GOAL := help
SHELL := /bin/bash
PATH :=.
Copy link
Contributor

@vschaffn vschaffn Dec 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command breaks the Makefile by overwriting the system's executable search paths, preventing essential commands from being found. Since PATH already exists, it's unnecessary to redefine it.

@echo "Applying ruff..."
@echo "================"
@echo
@ruff --fix $(path)
Copy link
Contributor

@vschaffn vschaffn Dec 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check --fix $(PATH)

)

return mask_array


class dDEM(Raster): # type: ignore
class Ddem(Raster): # type: ignore
Copy link
Contributor

@vschaffn vschaffn Dec 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this change is very wise, as dDEM mirrors the typo of the DEM class. But if you change it, you have to change it everywhere it is used

@@ -24,7 +24,7 @@
import numpy as np

# Only for Python >= 3.9
if sys.version_info.minor >= 9:
if sys.version_info.minor >= (3, 9):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes an error, the system version should not be a tuple

@@ -3063,7 +2992,7 @@ def fit(
else:
steps = list(self.procstep.pipeline)
argspec = [inspect.getfullargspec(s.__class__) for s in steps]
sub_meta = [s._meta["inputs"]["random"]["subsample"] for s in steps]
sub_meta = [s.meta["inputs"]["random"]["subsample"] for s in steps]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_meta

# Create placeholder variables names if those don't exist
if list_var_names is None:
list_var_names = ["var" + str(i + 1) for i in range(len(list_var))]

# Get the arrays for proxy values and explanatory variables
list_all_arr, gsd = _preprocess_values_with_mask_to_array(
values=[dvalues] + list_var, include_mask=stable_mask, exclude_mask=unstable_mask, preserve_shape=False
values=[dvalues, list_var], include_mask=stable_mask, exclude_mask=unstable_mask, preserve_shape=False,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This modifications triggered an issue : it creates a nested list where list_var is treated as a single element, resulting in [dvalues, [list_var]].

@@ -1007,7 +988,7 @@ def _aggregate_pdist_empirical_variogram(
# Define subsampling parameters
list_inside_radius = []
list_outside_radius: list[float | None] = []
binned_ranges = [0.0] + pdist_multi_ranges
binned_ranges = [0.0, pdist_multi_ranges]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@@ -3104,7 +3027,7 @@ def plot_variogram(
first_xmin = np.min(df.lags) / 2
else:
first_xmin = 0
xscale_range_split = [first_xmin] + xscale_range_split
xscale_range_split = [first_xmin, xscale_range_split]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@@ -3135,7 +3058,7 @@ def plot_variogram(
ax0.set_xticks([])

# Plot the histogram manually with fill_between
interval_var = [0] + list(df.lags)
interval_var = [0, list(df.lags)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@@ -3160,7 +3083,7 @@ def plot_variogram(
ax1 = ax.inset_axes(grid[3:, xgridmin[k] : xgridmax[k]].get_position(fig).bounds)

# Get the lags bin centers
bins_center = np.subtract(df.lags, np.diff([0] + df.lags.tolist()) / 2)
bins_center = np.subtract(df.lags, np.diff([0, df.lags.tolist()]) / 2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[POC] linting with ruff
2 participants