Chat Bot With PyTorch – NLP And Deep Learning – Python Tutorial (Part 3)




[ad_1]

In this Python Tutorial we build a simple chatbot using PyTorch and Deep Learning. I will also provide an introduction to some basic Natural Language Processing (NLP) techniques.

1) Theory + NLP concepts (Stemming, Tokenization, bag of words)
2) Create training data
3) PyTorch model and training
4) Save/load model and implement the chat

⭐ Kite is a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I’ve been using Kite for 6 months and I love it! https://www.kite.com/get-kite/?utm_medium=referral&utm_source=youtube&utm_campaign=pythonengineer&utm_content=description-only

🚀🚀 JOIN MY NEWSLETTER 🚀🚀
https://www.python-engineer.com/newsletter/

🚀🚀 Get exclusive content on Patreon: 🚀🚀
https://www.patreon.com/patrickloeber

If you enjoyed this video, please subscribe to the channel!

Article “Contextual Chatbots with Tensorflow”:
https://chatbotsmagazine.com/contextual-chat-bots-with-tensorflow-4391749d0077

NLTK:
https://www.nltk.org

You can find the code on GitHub:
https://github.com/python-engineer/pytorch-chatbot

PyTorch Beginner Course:
https://www.youtube.com/playlist?list=PLqnslRFeH2UrcDBWF5mfPGpqQDSta6VK4

Please checkout my website to see all tutorials:
https://www.python-engineer.com

You can find me here:
Twitter: https://twitter.com/python_engineer
GitHub: https://github.com/python-engineer

Icons:
https://fontawesome.com/icons/comments
https://fontawesome.com/icons/robot

#PyTorch #NLP #DeepLearning

Source


[ad_2]

Comment List

  • Python Engineer
    November 21, 2020

    Hey there!
    first of all: well done! i liked the visualisation of your project!
    im pretty done building my own chatbot.
    i did some test where i auto-py-to-exe my bot, cause i wanna send this bot to a friend of mine who is going throw a rough time!
    not i used pyinstaller and well.. my exe is about 2,5 gb xD
    is there any way to reduce this to a size so i can send it per email?
    i know its "exeing" all modules from python like torch, numpy.. but is there a way to reduce this massiv?
    i was think about to safe all needed data into an extra file (like the data file)
    would be great if someone can help me!

  • Python Engineer
    November 21, 2020

    Superb…. Worth… u made that Himalayan task so simple..

  • Python Engineer
    November 21, 2020

    nice video, but had this error
    RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_addmm

  • Python Engineer
    November 21, 2020

    hey can u please help me understand this error size mismatch, m1: [8 x 8], m2: [245 x 5] , i read comments but didnt quite understood it . thank you & great video

  • Python Engineer
    November 21, 2020

    Amazing tutorial! Many thanks master!

  • Python Engineer
    November 21, 2020

    Thanks a lot for this tutorial! You're awesome!

  • Python Engineer
    November 21, 2020

    RuntimeError: size mismatch, m1: [8 x 8], m2: [54 x 8] at ..atensrcTH/generic/THTensorMath.cpp:41

  • Python Engineer
    November 21, 2020

    I'm getting this error please help me solve this

    54 54
    7 ['delivery', 'funny', 'goodbye', 'greeting', 'items', 'payments', 'thanks']
    54 54
    7 ['delivery', 'funny', 'goodbye', 'greeting', 'items', 'payments', 'thanks']
    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 125, in _main
    prepare(preparation_data)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
    File "C:ProgramDataAnaconda3envspytorchlibrunpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
    File "C:ProgramDataAnaconda3envspytorchlibrunpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
    File "C:ProgramDataAnaconda3envspytorchlibrunpy.py", line 87, in _run_code
    exec(code, run_globals)
    File "D:pytorchChatbot1train.py", line 76, in <module>
    for (words, labels) in train_loader:
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 291, in _iter_
    return _MultiProcessingDataLoaderIter(self)
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 737, in _init_
    w.start()
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingprocess.py", line 121, in start
    self._popen = self._Popen(self)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingcontext.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingcontext.py", line 327, in _Popen
    return Popen(process_obj)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingpopen_spawn_win32.py", line 45, in _init_
    prep_data = spawn.get_preparation_data(process_obj._name)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
    RuntimeError:
    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

    if _name_ == '__main__':
    freeze_support()

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
    54 54
    7 ['delivery', 'funny', 'goodbye', 'greeting', 'items', 'payments', 'thanks']
    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 125, in _main
    prepare(preparation_data)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
    File "C:ProgramDataAnaconda3envspytorchlibrunpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
    File "C:ProgramDataAnaconda3envspytorchlibrunpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
    File "C:ProgramDataAnaconda3envspytorchlibrunpy.py", line 87, in _run_code
    exec(code, run_globals)
    File "D:pytorchChatbot1train.py", line 76, in <module>
    for (words, labels) in train_loader:
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 291, in _iter_
    return _MultiProcessingDataLoaderIter(self)
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 737, in _init_
    w.start()
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingprocess.py", line 121, in start
    self._popen = self._Popen(self)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingcontext.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingcontext.py", line 327, in _Popen
    return Popen(process_obj)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingpopen_spawn_win32.py", line 45, in _init_
    prep_data = spawn.get_preparation_data(process_obj._name)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingspawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
    RuntimeError:
    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

    if _name_ == '__main__':
    freeze_support()

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
    Traceback (most recent call last):
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 779, in _try_get_data
    data = self._data_queue.get(timeout=timeout)
    File "C:ProgramDataAnaconda3envspytorchlibmultiprocessingqueues.py", line 108, in get
    raise Empty
    _queue.Empty

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File "D:/pytorch/Chatbot1/train.py", line 76, in <module>
    for (words, labels) in train_loader:
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 363, in _next_
    data = self._next_data()
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 974, in _next_data
    idx, data = self._get_data()
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 941, in _get_data
    success, data = self._try_get_data()
    File "C:ProgramDataAnaconda3envspytorchlibsite-packagestorchutilsdatadataloader.py", line 792, in _try_get_data
    raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str))
    RuntimeError: DataLoader worker (pid(s) 7660, 8444) exited unexpectedly

    Even I checked the code from GitHub …it's the same

  • Python Engineer
    November 21, 2020

    RuntimeError: expected scalar type Long but found Int . please solve this error

  • Python Engineer
    November 21, 2020

    Hi I keep getting this exception.
    Traceback (most recent call last):

    File "c:UsersjigglDesktopProjectsChatBottrain.py", line 66, in <module>

    train_loader = DataLoader(

    TypeError: __init__() got an unexpected keyword argument 'batchsize'

  • Python Engineer
    November 21, 2020

    I have some problem while running train.py at the end (Got output up to the middle of the video but the second half I see this error)
    it shows:
    Traceback (most recent call last):

    File "train.py", line 67, in <module>

    criterion = nn.CrossEntropyLoss()

    NameError: name 'nn' is not defined

    I would really appreciate it if you assist me to solve this issue

  • Python Engineer
    November 21, 2020

    Now I have another problem, the program doesn't want to end, it print "training complete. File saved to data.pth", but the command promp never comes back. it's like it's stuck. I go through the debug, and it says that all is good. I don't get.

  • Python Engineer
    November 21, 2020

    I get this error "runtimeerror expected object of scalar type float but got scalar type Double for argument #2 'mat1' in call to _th_addmm". Wow, after two hours, finally fix it, I have to add in the model.py, after "def forward(self, x):", I added "x = x.float()", then in train.py in the #forward, I change "labels = labels" to "labels = labels.long()" and it now works. It's so funny how you can copy word for word, and yet get a different result. I thought python running on Mac, Linux or even Windows would the same python, like python is the programing software, it shouldn't matter what operating system it's program on. I know that the version can make a difference, but not running on different operating system.

  • Python Engineer
    November 21, 2020

    Thanks for tutorials
    i am getting below error while executing train.py
    Traceback (most recent call last):

    File "train.py", line 102, in <module>

    loss = criterion(outputs, labels)

    File "C:UsersAmit.Ranjananaconda3libsite-packagestorchnnmodulesmodule.py", line 550, in _call_

    result = self.forward(*input, **kwargs)

    File "C:UsersAmit.Ranjananaconda3libsite-packagestorchnnmodulesloss.py", line 932, in forward

    ignore_index=self.ignore_index, reduction=self.reduction)

    File "C:UsersAmit.Ranjananaconda3libsite-packagestorchnnfunctional.py", line 2317, in cross_entropy

    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)

    File "C:UsersAmit.Ranjananaconda3libsite-packagestorchnnfunctional.py", line 2115, in nll_loss

    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

    RuntimeError: expected scalar type Long but found Int

  • Python Engineer
    November 21, 2020

    RuntimeError: DataLoader worker (pid(s) 6360, 2828) exited unexpectedly
    Please anyone can help?

  • Python Engineer
    November 21, 2020

    Hi! and Thanks for these great tutorials:)
    I just have a question, what is the Pytorch version that you are using? because I'm facing this error:
    'ReLU' object has no attribute 'dim'
    and someone told me its related to my Pytorch version…

  • Python Engineer
    November 21, 2020

    Its really nice work..!!
    I tried to implement this, but I faced a issue " not implemented error" , this occurs when i try to print no of epochs and loss rate. Thanks in advance.

  • Python Engineer
    November 21, 2020

    ERROR: "AttributeError: 'NeuralNet' object has no attribute 'to' ".
    I looked at your code on GitHub, I have what you have but have error.
    my code: model = NeuralNet(input_size, hidden_size, output_size).to(device) // all code up to this point works

  • Python Engineer
    November 21, 2020

    Thanks for putting so much effort to teach us. The visualizations come handy a lot. Although I didn't understand PyTorch part much. but I will check out your PyTorch series.

    Thank you very much. Slapped that like and subscribe button. Keep making such quality videos!!!

  • Python Engineer
    November 21, 2020

    RuntimeError:
    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.
    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:
    if _name_ == '__main__':
    freeze_support()

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

    What is the error in here.. This appear when I executing the train.py

  • Python Engineer
    November 21, 2020

    Any tips, how we can deal with large data, I have 137000 chats and the unique words are 35000. And it is making a large numpy arrays.

  • Python Engineer
    November 21, 2020

    Whether torchtext is equivalent to nltk and spacy?. Word-embeddings can be used? in the place of Bag of words in the chatbot creation

  • Python Engineer
    November 21, 2020

    Hei,
    Very nice work. I highly appreciate.
    I get an error though.

    Expected object of scalar type Long but got scalar type Float for argument #2 ‘target’

    Any suggestions ?

  • Python Engineer
    November 21, 2020

    subscribed. thanks for posting

Write a comment