mllint — Linter for Machine Learning projects

mllint is compiled for Linux, MacOS and Windows and is published to PyPI, so it can be installed using pip install -U mllint Alternatively, use one of the Docker containers at bvobart/mllint

mllint is compiled for Linux, MacOS and Windows and is published to PyPI, so it can be installed using pip install -U mllint Alternatively, use one of the Docker containers at bvobart/mllint

mllint can be configured either using a .mllint.yml file or through the project’s pyproject.toml. This allows you to: selectively disable specific linting rules or categories using their slug define custom linting rules configure specific settings for various linting rules. See the code snippets and commands provided below for examples of such configuration files. Commands To print mllint’s current configuration in YAML format, run (optionally providing the path to the project’s folder):...

This category assesses your project’s code quality by running several static analysis tools on your project. Static analysis tools analyse your code without actually running it, in an attempt to find potential bugs, refactoring opportunities and/or coding style violations. The linter for this category will check whether your project is using the configured set of code quality linters. mllint supports (and by default requires) the following linters: pylint mypy black isort bandit For your project to be considered to be using a linter…...

This category evaluates checks whether your project uses Continuous Integration (CI) and how you are using it. Continuous Integration is the practice of automating the integration (merging) of all changes that multiple developers make to a software project. This is done by running an automated process for every commit to your project’s Git repository. This process then downloads your project’s source code at that commit, builds it, runs the linters configured for the project—we hope you include mllint—and runs the project’s tests against the system....

This category enables you to write your own custom evaluation rules for mllint. Custom rules can be useful for enforcing team, company or organisational practices, as well as implementing checks and analyses for how your proprietary / closed-source tools are being used. Custom rules may also be useful for creating ‘plugins’ to mllint, that implement checks on tools that mllint does not yet have built-in rules for. mllint will pick up these custom rules from your configuration and automatically run their checks during its analysis....

This category assesses your project’s data quality. It is not implemented yet. The idea is that this will contain rules on whether you have proper cleaning scripts and may also include dynamic checks on the data that is currently in the repository (e.g. is it complete (not missing values), are types of each value consistent, that sorta stuff. Perhaps with data-linter or tensorflow-data-validation)

This category deals with how your project manages its dependencies: the Python packages that your project uses to make it work, such as scikit-learn, pandas, tensorflow and pytorch. Proper dependency management, i.e., properly specifying which packages your project uses and which exact versions of those packages are being used, is important for being able to recreate the environment that your project was developed in. This allows other developers, automated deployment systems, or even just yourself, to install exactly those Python packages that you had installed while developing your project....

This category evaluates your project’s ability to be deployed in the real world. It is not yet implemented, but may contain rules about Dockerfiles and configurability, among others. Recommendations: SeldonCore - An open source platform to deploy your machine learning models on Kubernetes at massive scale. Seldon handles scaling to thousands of production machine learning models and provides advanced machine learning capabilities out of the box including Advanced Metrics, Request Logging, Explainers, Outlier Detectors, A/B Tests, Canaries and more....

This category deals with the file and folder structure of your ML project. It is not implemented yet. Examples of rules you might see here in the future: Project keeps its data in the ‘./data’ folder Project maintains documentation in a ‘./docs’ folder. Project’s source code is kept in a ‘./src’ folder, or a folder with the same name as the project / package.