Merge, Join, Append, Concat – Pandas




[ad_1]

“There should be one—and preferably only one—obvious way to do it,” — Zen of Python. I certainly wish that were the case with pandas. In reading the docs it feels like there are a thousand ways to do each operation. And it is hard to tell if they do the exact same thing or which one you should use. That’s why I made An Opinionated Guide to pandas—to present you one consistent (and a bit opinionated) way of doing data science with pandas and cut out all the confusion and cruft.

I’ll talk about which methods I use, why I use them and most importantly tell you the stuff that I’ve never touched in my years of data science practice. If this sounds helpful to you then please watch and provide feedback in your comments.

This series is beginner-friendly but aimed most directly at intermediate users.

“Opinionated Guide–Combining DataFrames” GitHub repo:
https://github.com/knathanieltucker/pandas-tutorial/blob/master/notebooks/Combining%20DataFrames.ipynb

Helpful links:
pandas.DataFrame. merge(), concat(): https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html

SQL joins: https://www.w3schools.com/sql/sql_join.asp

Link to GitHub repo including environment setup for tutorials: https://github.com/knathanieltucker/pandas-tutorial

PEP 20 – The Zen of Python link: https://www.python.org/dev/peps/pep-0020/

Source


[ad_2]

Comment List

  • Data Talks
    December 15, 2020

    is it possible to fully replace concat with merge?

  • Data Talks
    December 15, 2020

    Great video – thank you. You can maybe add 3 way joins as well?

  • Data Talks
    December 15, 2020

    Actually came looking for a walkthrough on merge_ordered. There's no video content on it!

  • Data Talks
    December 15, 2020

    Lecture notes – Merge, Join, Append, Concat
    1. Merge(=join)
    -df.reset_index() – before or after merging -> can turn it into useful format
    -specify the on=['a','b'], then pandas will pick it up
    -inner/outer
    -indicator=True -> show you what type of merge was done
    -columns with the same name -> suffixes=('_left', '_right")
    2. Concat(=append)
    -multiple dataframes / merge(stack) rows together
    -keys=['from1','from2'] : add an extra index(I'd like to know where that data source come from)

  • Data Talks
    December 15, 2020

    Can we write sas macro in python

  • Data Talks
    December 15, 2020

    Thanks for that all the info I've already read and take notes, but the course I'm in thought me ALL THE 7 FUNCTIONS AT "ONCE'". I Was going crazy, pandas.pydata.org is my most visited site. Really thank you for that.

  • Data Talks
    December 15, 2020

    Thank you so much for your video! I've just started learning python for data science as self thought and found your videos by chance . I've watched several python tutorials so far but yours are particularly good. Clear explanation, good quality video, easy to follow, not too long, not too short and you go staringht to the point. Very well done and thanks for sharing! 👏

  • Data Talks
    December 15, 2020

    merge done

  • Data Talks
    December 15, 2020

    Great video! do u know reason pandas created concat and append if they are the same thing?

  • Data Talks
    December 15, 2020

    Cool videos, thanks!

Write a comment