I trained a model. What is next?. This post was written by Vladimir… | by Kaggle Team | Kaggle Blog | Sep, 2020
You can improve the readability of your python code by adding syntax formatters and checkers.
It is not hard and not time-consuming. Checkers and formatters will not transform bad code into good, but the readability will go up. Think about fixing syntax as about basic hygiene. It is like brushing your teeth, but for the code.
I wrote a blog post on the topic called Nine Simple Steps for Better Looking python code. Feel free to check it out.
Step 1: configuration files
Add these files to the root of your repository.
Step 2: requirements
Install the required libraries with
pip install black flake8 mypy
Step 3: black
There are 100500 ways to format the code. Formatters like black or yapf modify the code to satisfy a pre-defined set of rules.
It is easier to read codebase that has some standards. When you work on the code for hours and need to switch a context between different coding styles, it drains “willpower energy” — no need to do it without a good reason.
will reformat all python files to follow the set of rules by black.
Step 4: flake8
will not modify the code, but will check code for syntax issues and output them to the screen.
Step 5: mypy
Python does not have mandatory static typization, but it is recommended to add types to the function arguments and return types.
....def forward(x: torch.Tensor) -> torch.Tensor:
You should add typing to the code.
- It makes it easier to read the code.
- You can use the mypy package to check arguments and function types for consistency.
After updating the code run mypy on the whole repo:
If mypy found issues — fix them.
Step 6: pre-commit hook
Running flake8, black, mypy manually all the time is annoying.
There is a tool called pre-commit hook that addresses the issue.
To enable it — copy this file to your repo: .pre-commit-config.yaml.
You need to install the pre-commit package on your machine with:
pip install pre-commit
And initialize with:
You are good to go.
From now on, on every commit, it will run a set of checks and not allow the commit to pass if something is wrong.
The main difference between the manual running of the black, flake8, mypy is that it does not beg you to fix issues, but forces you to do this. Hence, there is no waste of “willpower energy.”
Step 7: Github Actions
You added checks to the pre-commit hook, and you run them locally. But you need a second line of defense. You need Github to run these checks on every pull request.
Way to do it is to add file .github/workflows/ci.yaml to the repo.
There are lines:
- name: Install dependencies
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install black flake8 mypy
- name: Run black
black --check .
- name: Run flake8
- name: Run Mypy
run: mypy retinaface
that tell GitHub what to check.
I also recommend to give up the practice of pushing the code directly to the master branch.
Create a new branch, modify the code, commit, push to Github, create a pull request, and merge to master.
It is a standard in the industry, but it is exceptionally uncommon in the academy and among Kagglers.
If you are not familiar with these tools, it may take more than 20 minutes to add them and fix errors and warnings.
Remember this time. In the next project, add these checks in the first commit, when no code is written. From that moment, every small commit will be checked, and you will need to fix at most a couple lines of code every time: tiny overhead, excellent habit.
I would also recommend reading a book Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones. It talks about small changes in your behavior that improve productivity and the quality of your life.