I trained a model. What is next?. This post was written by Vladimir… | by Kaggle Team | Kaggle Blog | Sep, 2020

[ad_1]

You can improve the readability of your python code by adding syntax formatters and checkers.

It is not hard and not time-consuming. Checkers and formatters will not transform bad code into good, but the readability will go up. Think about fixing syntax as about basic hygiene. It is like brushing your teeth, but for the code.

I wrote a blog post on the topic called Nine Simple Steps for Better Looking python code. Feel free to check it out.

Step 1: configuration files

Add these files to the root of your repository.

Step 2: requirements

Install the required libraries with

pip install black flake8 mypy

Step 3: black

There are 100500 ways to format the code. Formatters like black or yapf modify the code to satisfy a pre-defined set of rules.

It is easier to read codebase that has some standards. When you work on the code for hours and need to switch a context between different coding styles, it drains “willpower energy” — no need to do it without a good reason.

Running

black .

will reformat all python files to follow the set of rules by black.

Step 4: flake8

Running

flake8

will not modify the code, but will check code for syntax issues and output them to the screen.

Fix them.

Step 5: mypy

Python does not have mandatory static typization, but it is recommended to add types to the function arguments and return types.

For example:

class MyModel(nn.Module):
....
def forward(x: torch.Tensor) -> torch.Tensor:
....
return self.final(x)

You should add typing to the code.

  1. It makes it easier to read the code.
  2. You can use the mypy package to check arguments and function types for consistency.

After updating the code run mypy on the whole repo:

mypy .

If mypy found issues — fix them.

Step 6: pre-commit hook

Running flake8, black, mypy manually all the time is annoying.

There is a tool called pre-commit hook that addresses the issue.

To enable it — copy this file to your repo: .pre-commit-config.yaml.

You need to install the pre-commit package on your machine with:

pip install pre-commit

And initialize with:

pre-commit install

You are good to go.

From now on, on every commit, it will run a set of checks and not allow the commit to pass if something is wrong.

The main difference between the manual running of the black, flake8, mypy is that it does not beg you to fix issues, but forces you to do this. Hence, there is no waste of “willpower energy.”

Step 7: Github Actions

You added checks to the pre-commit hook, and you run them locally. But you need a second line of defense. You need Github to run these checks on every pull request.

Way to do it is to add file .github/workflows/ci.yaml to the repo.

There are lines:

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install black flake8 mypy
- name: Run black
run:
black --check .
- name: Run flake8
run: flake8
- name: Run Mypy
run: mypy retinaface

that tell GitHub what to check.

I also recommend to give up the practice of pushing the code directly to the master branch.

Create a new branch, modify the code, commit, push to Github, create a pull request, and merge to master.

It is a standard in the industry, but it is exceptionally uncommon in the academy and among Kagglers.

If you are not familiar with these tools, it may take more than 20 minutes to add them and fix errors and warnings.

Remember this time. In the next project, add these checks in the first commit, when no code is written. From that moment, every small commit will be checked, and you will need to fix at most a couple lines of code every time: tiny overhead, excellent habit.

I would also recommend reading a book Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones. It talks about small changes in your behavior that improve productivity and the quality of your life.

[ad_2]

Source link

Write a comment