Tracking Monster Jobs with tqdm. TQDM is an easy-to-use, extensible… | by Mark Douthwaite

[ad_1]


TQDM is an easy-to-use, extensible progress bar package for Python. This short post gives you a few useful examples to get you started.

Mark Douthwaite
Volodymyr Hryshchenko on Unsplash
import time 
from tqdm import tqdm
with tqdm(total=100) as progress:
for i in range(100):
time.sleep(0.25)
progress.update(1)
A simple tqdm progress bar! Screenshot by Mark Douthwaite
df = pd.read_csv("weather.csv") 
tqdm.pandas(desc="Applying Transformation")
df.progress_apply(lambda x: x)
An example of a progress bar generated from tqdm‘s integration with pandas. Screenshot by Mark Douthwaite
import time 
from tqdm.contrib.concurrent import process_map
def my_process(_):
time.sleep(0.25)
r = process_map(my_process, range(0, 100), max_workers=2, desc="MyProcess")
import time 
import multiprocessing as mp
from tqdm import tqdm
def my_process(pos):
_process = mp.current_process()
with tqdm(desc=f"Process {pos}", total=100, position=pos) as progress:
for _ in range(100):
time.sleep(0.1)
progress.update(1)
n_cpu = mp.cpu_count()with mp.Pool(processes=n_cpu, initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),)) as pool:
pool.map(my_process, range(n_cpu))
An example of progress bars tracking 8 jobs simultaneously. Screenshot by Mark Douthwaite
Some example code, just for you! 💛
import time
from tqdm.notebook import tnrange, tqdm
for i in tnrange(10):
time.sleep(0.1)
Progress bars in Jupyter! Screenshot by Mark Douthwaite

[ad_2]




Source link

Write a comment