How do I apply a function to a pandas Series or DataFrame?




[ad_1]

Have you ever struggled to figure out the differences between apply, map, and applymap? In this video, I’ll explain when you should use each of these methods and demonstrate a few common use cases. Watch the end of the video for three important announcements!

Subscribe to the Data School email newsletter: http://www.dataschool.io/subscribe/

Join “Data School Insiders” for exclusive rewards: https://www.patreon.com/dataschool

== DATA SCHOOL VIDEO TUTORIALS ==
Data analysis with pandas (30 videos): https://www.youtube.com/playlist?list=PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y
Machine learning with scikit-learn (10 videos): https://www.youtube.com/playlist?list=PL5-da3qGB5ICeMbQuqbbCOQWcS6OYBr5A
Version control with Git and GitHub (11 videos): https://www.youtube.com/playlist?list=PL5-da3qGB5IBLMp7LtN8Nc3Efd4hJq0kD

== PANDAS RESOURCES ==
GitHub repository for the series: https://github.com/justmarkham/pandas-videos
Series “map” documentation: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.map.html
Series “apply” documentation: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.apply.html
DataFrame “apply” documentation: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.apply.html
DataFrame “applymap” documentation: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.applymap.html

== RELATED PANDAS VIDEOS ==
loc, iloc, and ix: https://www.youtube.com/watch?v=xvpNA7bC8cs&list=PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y&index=19
string methods: https://www.youtube.com/watch?v=bofaC0IckHo&list=PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y&index=12

== JOIN THE DATA SCHOOL COMMUNITY ==
Blog: http://www.dataschool.io
Newsletter: http://www.dataschool.io/subscribe/
Twitter: https://twitter.com/justmarkham
Facebook: https://www.facebook.com/DataScienceSchool/
YouTube: https://www.youtube.com/user/dataschool?sub_confirmation=1

Source


[ad_2]

Comment List

  • Data School
    December 2, 2020

    Thank you so much for this! You're so clear and concise!

  • Data School
    December 2, 2020

    Thank you very much. You've been a great help.

  • Data School
    December 2, 2020

    Question… apply(np.argmax, axis=1) is not working for me. it just returns the same value as "apply(max, axis=1) method… It's now 2020, so I imagine the pandas library has changed. Do you know what is the alternative method that can return the max value for each of the row value? Thank you in advance.

  • Data School
    December 2, 2020

    Thank you for sharing! I learn so much from every video!

  • Data School
    December 2, 2020

    @Data School, I did some testing and I have found that if you type the lambda function @10:35 a little differently, you get a more efficient processing time:

    import timeit

    tic = timeit.default_timer()

    train.Name.apply(lambda x: x.split(',')[0]) # Method 1

    toc = timeit.default_timer()

    method1 = toc-tic

    print('Method 1:', method1)

    ticktic = timeit.default_timer()

    train.Name.str.split(',').apply(lambda x: x[0]) # Method 2

    toc = timeit.default_timer()

    method2 = toc-tic

    print('Method 2:', method2)

    if method1 < method2:

    print(f'Method 1 is faster by {method2/method1} times.')

    else:

    print(f'Method 2 is faster by {method1/method2} times.')

  • Data School
    December 2, 2020

    you are the best pandas teacher I've ever seen

  • Data School
    December 2, 2020

    You are a sweetheart bro! Thank you so much, the whole playlist is perfect<3

  • Data School
    December 2, 2020

    Subscribed!

  • Data School
    December 2, 2020

    Omg I really loooove your videos… I'm going to use everything you share, it's amazing. I quit on learning python 2 years ago and I want to give it a try again now. And by seeing your pandas videos I truly think it will woth it. Thank you so so much for your effort… keep going 💪🏽

  • Data School
    December 2, 2020

    Incredible explanation which is very easy to understand. One can give this much of a clear explanation on stuffs only if they are confident with what they are talking about and you are nailed it buddy.

  • Data School
    December 2, 2020

    Though in some cases, APPLY may not be as fast as using other approaches, that difference is usually negligible. Kevin doesn't mention that here, in the interest of time/audience; HOWEVER, he did make this very point in an insightful Stack Overflow post that I recently came across. In fact, there was a subtle advantage to using apply, instead of the other alternative, in that it preserved the index. So, yes: speed and optimization are great and all, but most of the time, writing code that is simple / readable — which Kevin acknowledges as being subjective — will just as effectively get the job done.

    This video series is wonderful because it's not only palatable to beginners, but I find myself coming back to it and picking up certain details of why things work as they do, that may not have stuck the first time around. It can initially feel overwhelming that Pandas gives you so many options on HOW to do things, but I've come to appreciate the value in wielding a proverbial swiss-army knife, and I've started to recognize which situations favor using one instrument over another. Thank you, Kevin, for being a great teacher, and for presenting content — beyond this video series, at that — that appeals to users having a range of experience. <BIG THUMBS UP EMOJI>

  • Data School
    December 2, 2020

    I've watched a lot of tutorial videos in my time, but you my friend, are hands-down the best!

  • Data School
    December 2, 2020

    How would you find the best optimal cut off point for continuous variables having a binary outcome variable?

  • Data School
    December 2, 2020

    oh my god you saved me.

  • Data School
    December 2, 2020

    One of the best Pandas videos ever. If i had saw this a few days ago, it would have saved me a few days of frusteration

  • Data School
    December 2, 2020

    most helpful video man

  • Data School
    December 2, 2020

    thanx, appreciate all your efforts in making ML easy to understand

  • Data School
    December 2, 2020

    Thank you so much for the time and effort you have put into your video series, and the amazingly well documented Jupyter Notebook with everything from the series!

  • Data School
    December 2, 2020

    thank you very much. the best explanation of youtube.

  • Data School
    December 2, 2020

    @10:40 couldn't you also use .str.get(0) to extract first elements

Write a comment