I don't like notebooks.- Joel Grus (Allen Institute for Artificial Intelligence)




[ad_1]

I have been using and teaching Python for many years. I wrote a best-selling book about learning data science. And here’s my confession: I don’t like notebooks. (There are dozens of us!) I’ll explain why I find notebooks difficult, show how they frustrate my preferred pedagogy, demonstrate how I prefer to work, and discuss what Jupyter could do to win me over.

Subscribe to O’Reilly on YouTube: http://goo.gl/n3QSYi

Follow O’Reilly on:
Twitter: http://twitter.com/oreillymedia
Facebook: http://facebook.com/OReilly
Instagram: https://www.instagram.com/oreillymedia
LinkedIn: https://www.linkedin.com/company-beta/8459/

Source


[ad_2]

Comment List

  • O'Reilly
    November 19, 2020

    So long story short, dude was trying to (ab)use a tool for something it's not designed for and then complaining about it. Notebooks are first and foremost meant for exploratory work, when you want to have codes and plots side by side, or prefer to see a tabular dataframe formatted nicely, Excel-esquely. Notebooks are not meant to work as text editors, or to replace one. Neither are they meant to teach a programming language or OOP. So if you want any of the above, use another tool to fill in the gap, it's that simple.

    Though most of the issues he pointed out are valid, the tool itself is not the one to blame here but the users. Why the hell would one run PEP8 in a notebook? And if one's code is already bad in a notebook, how can you expect it to be way better if it had been developed in a text editor? And cell magics are not production ready, but for every cell magic there is always an alternate way of doing it without one, etc. Admittedly the memes are good for a laugh but the comparison is awfully lopsided it's impossible to take this guy seriously.

  • O'Reilly
    November 19, 2020

    humor and messages were on point – great job!

  • O'Reilly
    November 19, 2020

    I admit I was fascinated with notebooks when I first encountered them. BUT, I was not developing code that might need to be broken into methods, tested, or placed in production. I was experimenting. Small stuff. Even in that setting, I have found myself migrating to a tool like VS Code. I'm open on what that tool should be except for me, it has to be free. Joel hits a lot of the things that I found bothering me. Extract methods. Intellisense features. Stuff most people who write code for a living will want and benefit from.

    Nice talk but he does talk fast.

  • O'Reilly
    November 19, 2020

    Very often the datasets we load are quite huge and it's time consuming to just REPL it. This is one advantage of notebooks, where we can annotate intermediate steps with the same dataset

  • O'Reilly
    November 19, 2020

    I love this don't like talk.

  • O'Reilly
    November 19, 2020

    Upvote me if you come here because of Fastai's post.

  • O'Reilly
    November 19, 2020

    Almost all the points apply to an ide as well. Would be better to stick to things that are true.

    Eg
    there’s no reason you cant write modular functions and classes in a notebook, i do it all the time. It’s literally the same as without the notebook

    There’s no reason you can’t do tdd in a notebook, i do it all the time. It’s literally the same as without the notebook

    There’s no reason notebooks are not reproducible. Click the restart and run all button, easy. It’s literally the same as without the notebook

    There’s no reason notebooks are incompatible with requirements.txt. It’s literally the same as without the notebook

    To your credit, there are reasons not to use notebooks, but throwing all these incorrect points in makes your point weaker, not stronger.

    I’m going to share a well kept secret: use each tool for what it’s best at. There is no silver bullet. 😉

  • O'Reilly
    November 19, 2020

    Rare to see such a great coder and presenter!

  • O'Reilly
    November 19, 2020

    I have to complaint about this talk to O'Reilly. We don't want to hear a person's hate. So he doesn't like it, doesn't make it a bad technology or discredit the community. I am appalled by that such derogatory talk is allowed. Sad.

  • O'Reilly
    November 19, 2020

    This is certainly outdated, and useless. Because JupyterLab is going to be a standard IDE (almost it is now).

  • O'Reilly
    November 19, 2020

    be me, start a guide with jupyter, don't like it
    use another guide for idle, like it
    look for more guide for more knowledge about data science pandas numpy… and all that
    the guide uses notebooks.ai
    v
    (not working looking for advice on youtube and finding this video)
    v
    the guide actually says in description to use google colab instead cause notebooks.ai isn't working like it used to (and the guide is kinda new)

    why, a begginer programmer is expected while learning to use so many tools he does not and cannot fully understand when he absolutely do not need them?

    amazing presentation thank you for making me more convinced about learning using simpler tools

  • O'Reilly
    November 19, 2020

    For another perspective from Jeremy Howard(founding researcher at fast.ai, past president of Kaggle) see here: https://youtu.be/9Q6sLbz37gk
    Particularly for those new to Data Science, just use whatever works for you.

    You don't have to stick to one thing for everything.
    You don't have to dislike a tool just because someone convinces you to do so.

    If it helps you be more efficient, use it.

    If it disrupts your workflow, then don't bother with it.

    Particularly for Exploratory Data Analysis(EDA), Notebooks can be great.

    Also for tutorials, documentation, and demonstrations that describes how things work.

    But if you are working on more software engineering-focused and code heavy projects that doesn't require visualisations, then you might choose not to use Notebooks. What they usually say is, "Don't use Notebooks in production." And even that advice doesn't hold up all the time. You just need to be aware of the weaknesses of the tools you are using. Don't buy into the "real programmers don't …" hype. And all well-known Cloud ML services are using Notebooks as their main interface to write code. It's not something you can escape. 😉

  • O'Reilly
    November 19, 2020

    I was intentionally searching for this one. Not just me in the community who thinks it sucks.!!!
    This guy did the job….

  • O'Reilly
    November 19, 2020

    It seems to me that Emacs' org-mode and its noweb capabilities could deal with all the problems mentioned?

  • O'Reilly
    November 19, 2020

    Why should anyone learn Python in Jupyter Notebooks? That's crazy! It's not made for this at all. Every tutorial which says that: buuuuuuuurn

  • O'Reilly
    November 19, 2020

    Why was the speed set at 1.5x before uploading?

  • O'Reilly
    November 19, 2020

    For everyone using VSCode and using iPython (or IDLE), just wait until you realize what you can accomplish with PyCharm. It will blow your mind.

    PyCharm is to (VSCode + iPython/IDLE) what those two are to Jupyter

  • O'Reilly
    November 19, 2020

    For people who don't have good speakers. The text editor that the lady (46:27) and other people (49:07) mentioned was
    repl.it
    Atom extension which allows inline plots and other features:
    Hydrogen

  • O'Reilly
    November 19, 2020

    Joel rocks, that is for sure. This made me quit notebooks about 6 months ago and have absolutely no regrets. I brought my data science to the next level. Thanks, Joel!

  • O'Reilly
    November 19, 2020

    Joel is the closest thing we have to wizardry.

  • O'Reilly
    November 19, 2020

    just here for the memes

  • O'Reilly
    November 19, 2020

    Funniest guy ever. His live coding is really great too (not as in funny, but as in interesting).

  • O'Reilly
    November 19, 2020

    I have a programming background so restarting my kernel every and running things top to bottom was always common sense to me. I do agree though that it can be misleading and people may not make that assumption. I think one flaw with his programming style at the end that notebooks solve is having the history of how you checked your data throughout the process. This is really important for cleaning data sets. Unless you save everything you did in iPython along with your python code, it would be hard to trace. For writing models, pure python is definitely better, but for exploring data sets and fixing issues with the data itself, I don't know how notebooks can be beat.

  • O'Reilly
    November 19, 2020

    I am still quite new to data science and I've been using notebooks for my personal projects but listening to this I agree with him and shift my focus. However, I am a little confuse what if I am doing EDA? should I use a notebook or an IDE like Pycharm? because with EDA I am trying to understand the data and that requires me to plot tables and charts as well as code to aggregate data to understand it better. It seems faster to do that in a notebook than to create modules for them especially when viewing graphs isn't as nice in an IDE.

  • O'Reilly
    November 19, 2020

    this guy's meme game is on point lol

  • O'Reilly
    November 19, 2020

    Wow I'm really glad I learned python in Sublime Text and a command prompt

  • O'Reilly
    November 19, 2020

    I love the memes!

  • O'Reilly
    November 19, 2020

    I haven't touched Jupyter since three weeks ago when I graduated with my Master's. Now that I'm trying to do basic operations in Jupyter again and it makes me so frustrated I want to scream, I do not understand how these things are so popular.

  • O'Reilly
    November 19, 2020

    wait a second. isn't python extension only exsit for windows vscode?

  • O'Reilly
    November 19, 2020

    Google colab also in jupyter format.

Write a comment