Real Time Sign Language Detection with Tensorflow Object Detection and Python | Deep Learning SSD




[ad_1]

Language barriers are very much still a real thing.

We can take baby steps to help close that.

Speech to text and translators have made it a heap easier.

But what about for those that maybe don’t speak or can’t hear?

What about them?

Well…you can begin to use Tensorflow Object Detection and Python to help close that gap. And in this video, you’ll learn how to take the first steps to doing just that! In this video, you’ll learn how to build an end-to-end custom object detection model that allows you to translate sign language in real time.

In this video you’ll learn how to:
1. Collect images for deep learning using your webcam and OpenCV
2. Label images for sign language detection using LabelImg
3. Setup Tensorflow Object Detection pipeline configuration
4. Use transfer learning to train a deep learning model
5. Detect sign language in real time using OpenCV

Get the training template here: https://github.com/nicknochnack/RealTimeObjectDetection

Other Links Mentioned in the Video
Face Mask Detection Video: https://youtu.be/IOI0o3Cxv9Q
LabelImg: https://github.com/tzutalin/labelImg
Installing the Tensorflow Object Detection API: https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/install.html

Oh, and don’t forget to connect with me!
LinkedIn: https://www.linkedin.com/in/nicholasrenotte
Facebook: https://www.facebook.com/nickrenotte/
GitHub: https://github.com/nicknochnack

Happy coding!
Nick

P.s. Let me know how you go and drop a comment if you need a hand!

Source


[ad_2]

Comment List

  • Nicholas Renotte
    December 15, 2020

    Hi I really love this video its very helpful for my project. I am making an alphabet hand detector, I followed everything but I got stock at 11.02 when I run the program the frame comes out but it will only capture one image and I have this error
    A subdirectory or file TensorflowworkspaceimagescollectedimagesA already exists.

    Collecting images for A

    —————————————————————————

    error Traceback (most recent call last)

    <ipython-input-5-d41e32a3bb14> in <module>

    7 ret, frame = cap.read()

    8 imgname = os.path.join(IMAGES_PATH, label, label+'.'+'{}.jpg'.format(str(uuid.uuid1())))

    —-> 9 cv2.imwrite(imgname, frame)

    10 cv2.imshow('frame', frame)

    11 time.sleep(2)

    error: OpenCV(4.4.0) C:UsersappveyorAppDataLocalTemp1pip-req-build-95hbg2jtopencvmodulesimgcodecssrcloadsave.cpp:738: error: (-215:Assertion failed) !_img.empty() in function 'cv::imwrite'

    I hope you can help me what did I missed. Thank u so much.

  • Nicholas Renotte
    December 15, 2020

    Traceback (most recent call last):

    File "Tensorflow/scripts/generate_tfrecord.py", line 29, in <module>

    from object_detection.utils import dataset_util, label_map_util

    ModuleNotFoundError: No module named 'object_detection'

    Traceback (most recent call last):

    File "Tensorflow/scripts/generate_tfrecord.py", line 29, in <module>

    from object_detection.utils import dataset_util, label_map_util

    ModuleNotFoundError: No module named 'object_detection'

    I am not able to resolve this issue please answer

  • Nicholas Renotte
    December 15, 2020

    can you please do a motion sign language detection ? please

  • Nicholas Renotte
    December 15, 2020

    Hi Nicolas, It not have any picture in folder .how can I do 🙁

  • Nicholas Renotte
    December 15, 2020

    while collecting images it is showing "The system cannot find the specified path"
    what is the error…?

  • Nicholas Renotte
    December 15, 2020

    Do you have some contact ?

  • Nicholas Renotte
    December 15, 2020

    How do you not have more views? This is amazing content!

  • Nicholas Renotte
    December 15, 2020

    Hi Nicholas! Thank you so much!
    I got this, please help:
    error: OpenCV(4.4.0) /tmp/pip-req-build-dglzv4yn/opencv/modules/imgcodecs/src/loadsave.cpp:738: error: (-215:Assertion failed) !_img.empty() in function 'imwrite'

  • Nicholas Renotte
    December 15, 2020

    On running the 2nd cell in the tutorial file I am getting this error

    Traceback (most recent call last):
    File "Tensorflow/scripts/generate_tfrecord.py", line 29, in <module>
    from object_detection.utils import dataset_util, label_map_util
    ModuleNotFoundError: No module named 'object_detection'
    Traceback (most recent call last):
    File "Tensorflow/scripts/generate_tfrecord.py", line 29, in <module>
    from object_detection.utils import dataset_util, label_map_util
    ModuleNotFoundError: No module named 'object_detection'

    The 6th step in the pinned comment worked perfectly fine. Kindly help.

    Also getting error — ERROR: Failed building wheel for pycocotools
    PS: love your videos

  • Nicholas Renotte
    December 15, 2020

    great job man!Amazing! Somehow I was wondering can I use this system to recognize head and arm pose at the same time? Thank you

  • Nicholas Renotte
    December 15, 2020

    Hi nicholas. Thank you for the amazing tutorial!!
    Im facing a tiny problem and I need help. When I run the cell
    'config = config_util.get_configs_from_pipeline_file(CONFIG_PATH)'
    I get the following error: 'module tensorflow has no attribute gfile'
    I tried replacing every tf.gfile in the config_util with tf.io.gfile and it still doesn't work.
    How can I fix this please?

  • Nicholas Renotte
    December 15, 2020

    Can you make gestures to control (I dont know) spotify or the TV ?

  • Nicholas Renotte
    December 15, 2020

    can we use paycharm instead of jupyter notebook?

  • Nicholas Renotte
    December 15, 2020

    I got this error name 'uuid1' is not defined how to fix it?

  • Nicholas Renotte
    December 15, 2020

    Wow so nice bro Thank you. Can you please help us with some dynamic gestures?

  • Nicholas Renotte
    December 15, 2020

    Hey Nicholas,
    For loading the model from checkpoint in step 7, while running the dependencies, some of the files contain tf.contrib which is removed in Tensorflow 2.0
    Can you suggest any other way of passing its functionalities apart from tf.contrib

    Error-from tensorflow.contrib.image.python.ops import image_ops
    No module called tensorflow.contrib found
    File-"Anaconda3libsite-packagesobject_detectionmatchersbipartite_matcher.py"

  • Nicholas Renotte
    December 15, 2020

    wow…………

  • Nicholas Renotte
    December 15, 2020

    Hey @Nicholas Renotte ,
    Great Tutorial ! Thank you so much!!
    I am stuck at the step when we train the model. When I execute the python command in the command prompt, I am encountering the following error-

    File "Tensorflow/models/research/object_detection/model_main_tf2.py", line 32, in <module>
    from object_detection import model_lib_v2ImportError: cannot import name 'model_lib_v2' from 'object_detection'

    UPDATE- Resolved
    Installing the Tensorflow object detection API again following the docs
    Link-https://readthedocs.org/projects/tensorflow-object-detection-api-tutorial/downloads/pdf/latest

  • Nicholas Renotte
    December 15, 2020

    No Module Named tensorflow.compat
    Python 3.6
    Tensorflow 1.5
    Only CPU

    Please Help When I wanna to Create TFRecord File.

  • Nicholas Renotte
    December 15, 2020

    Thank you so much!!!

  • Nicholas Renotte
    December 15, 2020

    Nice video but half the commands don't work for me especially the addressing ones like what located where. Are you using mac os? I tried it on both Windows and Ubuntu and if you used mac, it should have worked in Ubuntu since it's a Linux machine.

  • Nicholas Renotte
    December 15, 2020

    Hi Nicholas
    In the 2nd step i.e create TF record I got these errors , I am not able to resolve this problem could you please help me with this .

    Traceback (most recent call last):

    File "Tensorflow/scripts/generate_tfrecord.py", line 168, in <module>

    tf.app.run()

    File "D:Anaconda3envstensorflowlibsite-packagestensorflowpythonplatformapp.py", line 40, in run

    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)

    File "D:Anaconda3envstensorflowlibsite-packagesabslapp.py", line 303, in run

    _run_main(main, args)

    File "D:Anaconda3envstensorflowlibsite-packagesabslapp.py", line 251, in _run_main

    sys.exit(main(argv))

    File "Tensorflow/scripts/generate_tfrecord.py", line 158, in main

    tf_example = create_tf_example(group, path)

    File "Tensorflow/scripts/generate_tfrecord.py", line 132, in create_tf_example

    classes.append(class_text_to_int(row['class']))

    File "Tensorflow/scripts/generate_tfrecord.py", line 101, in class_text_to_int

    return label_map_dict[row_label]

    KeyError: 'p'

    Traceback (most recent call last):

    File "Tensorflow/scripts/generate_tfrecord.py", line 168, in <module>

    tf.app.run()

    File "D:Anaconda3envstensorflowlibsite-packagestensorflowpythonplatformapp.py", line 40, in run

    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)

    File "D:Anaconda3envstensorflowlibsite-packagesabslapp.py", line 303, in run

    _run_main(main, args)

    File "D:Anaconda3envstensorflowlibsite-packagesabslapp.py", line 251, in _run_main

    sys.exit(main(argv))

    File "Tensorflow/scripts/generate_tfrecord.py", line 158, in main

    tf_example = create_tf_example(group, path)

    File "Tensorflow/scripts/generate_tfrecord.py", line 132, in create_tf_example

    classes.append(class_text_to_int(row['class']))

    File "Tensorflow/scripts/generate_tfrecord.py", line 101, in class_text_to_int

    return label_map_dict[row_label]

    KeyError: 'p'

  • Nicholas Renotte
    December 15, 2020

    Thanks for this tutorial. So, really helpful.

  • Nicholas Renotte
    December 15, 2020

    ​ @Nicholas Renotte

    when I ran this cell

    """

    cap = cv2.VideoCapture(0)

    width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))

    height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

    """

    webcam got activated, but when I tried to run the cell below that I am getting error

    —————————————————————————

    error Traceback (most recent call last)

    <ipython-input-33-8654a62546c0> in <module>

    28 agnostic_mode=False)

    29

    —> 30 cv2.imshow('object detection', cv2.resize(image_np_with_detections, (800, 600)))

    31

    32 if cv2.waitKey(1) & 0xFF == ord('q'):

    error: OpenCV(4.4.0) C:UsersappveyorAppDataLocalTemp1pip-req-build-jw46nckropencvmoduleshighguisrcwindow.cpp:651: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Cocoa support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvShowImage'

    please help…..

  • Nicholas Renotte
    December 15, 2020

    Hi Nicolas, finally I have been able to run everything. Although my recognition is not that accurate . I guess it would be nice to automate the part of selecting the gesture in every picture 🤣

  • Nicholas Renotte
    December 15, 2020

    Does any body gets an error when running this line at the end of the Tutorial file??
    cv2.imshow('object detection', cv2.resize(image_np_with_detections, (800, 600)))
    — error—-
    error: OpenCV(4.4.0) /tmp/pip-req-build-dglzv4yn/opencv/modules/highgui/src/window.cpp:651: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Cocoa support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvShowImage'

  • Nicholas Renotte
    December 15, 2020

    Can we also detect dynamic input with facial expressions along with hand sign, as facial expressions are also imp part of sign language.

  • Nicholas Renotte
    December 15, 2020

    Does anyone gets an error in Tutorial.ipynb : 7. Load the train model from checkpoint ??
    As soon as it goes over the imports I get this error 3 times:

    ERROR:root:Internal Python error in the inspect module.

    Below is the traceback from this internal error.

  • Nicholas Renotte
    December 15, 2020

    Hi Nicholas I have followed your setup guide the only thing that failed was installing that specific OpenCV version. I proceeded to the tutorial Jupiter notebook and one of the problems is in windows cp is not a command changed for this:

    !copy {PRETRAINED_MODEL_PATH+'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8pipeline.config'} {MODEL_PATH+''+CUSTOM_MODEL_NAME}

    still doesn't work (The system cannot find the file specified.
    )
    – but I managed to just copy and paste the pipeline.config inside my_ssd_mobnet folder!

    – In PART 5 – Update config for transfer learning: I can see that pipeline. config is loaded because is displayed but when changing the number of classes and run over that section there is no error and pipeline. config doesn't get modified like in your video.

    What could be wrong???

  • Nicholas Renotte
    December 15, 2020

    Great Tutorial, but i got an error at the copy model config, i got this -'cp' is not recognized as an internal or external command, operable program or batch file.-. What could i do, or i need something ?

  • Nicholas Renotte
    December 15, 2020

    error in first jupyter notebook to create the pictures: the folder collectedimages gets created and so the folder for the first label with the first picture after that the window from the webcam freezes and the error appears

  • Nicholas Renotte
    December 15, 2020

    HI Nicholas , Im getting this error related to the first jupyter notebook that you create to collect photos. Do you have any idea what could cause the error?
    – Using Anaconda Navigator and windows –
    error: OpenCV(4.0.1) C:ciopencv-suite_1573470242804workmoduleshighguisrcwindow.cpp:352: error: (-215:Assertion failed) size.width>0 && size.height>0 in function 'cv::imshow'

  • Nicholas Renotte
    December 15, 2020

    Hey y'all! Wrote up a couple of steps to setup your environment. I'm planning on doing a video on this but until then, here's a crash course post:
    1. First up install Python 3.7.4. Download and install the package for your OS that has the words 2019.07 in it from here https://repo.anaconda.com/archive/. This should give you 3.7.4 to work with.

    2. Then install Visual Studio C++ 2015 from here: https://go.microsoft.com/fwlink/?LinkId=691126. Tensorflow needs this in order to compile

    3. OPTIONAL IF YOU HAVE A GPU – Install Cuda and Cudnn. Install Cuda first, then install Cudnn.
    – Cuda: 10.1 – https://developer.nvidia.com/cuda-10.1-download-archive-base
    – Cudnn: 7.6.5 – https://developer.nvidia.com/rdp/cudnn-download

    Once Cudnn is installed you need to copy the Cudnn files into their respective folders inside the Cuda directory. I used this as a guide: https://towardsdatascience.com/installing-tensorflow-with-cuda-cudnn-and-gpu-support-on-windows-10-60693e46e781

    4. Then install Protoc 3.13 from here: https://github.com/protocolbuffers/protobuf/releases. For windows, download the repository and then add it to your PATH file.

    5. Then install python packages using the pip command:
    Tensorflow: 2.3.1 – pip install tensorflow==2.3.1
    OpenCV: 4.4.0 – pip install tensorflow==4.4.0

    6. Install the object detection API. To do this, run these commands from a command prompt or terminal:
    git clone https://github.com/tensorflow/models
    cd tensorflow/models/research
    protoc object_detection/protos/*.proto –python_out=.
    cp object_detection/packages/tf2/setup.py .
    python -m pip install .

  • Nicholas Renotte
    December 15, 2020

    What version of python, tensorflow and opencv are you using?

  • Nicholas Renotte
    December 15, 2020

    Finally a video that makes sense !!!

  • Nicholas Renotte
    December 15, 2020

    Error???

    Traceback (most recent call last):

    File "Tensorflow/scripts/generate_tfrecord.py", line 31, in <module>

    from object_detection.utils import dataset_util, label_map_util

    ModuleNotFoundError: No module named 'object_detection'

    Traceback (most recent call last):

    File "Tensorflow/scripts/generate_tfrecord.py", line 31, in <module>

    from object_detection.utils import dataset_util, label_map_util

    ModuleNotFoundError: No module named 'object_detection'

  • Nicholas Renotte
    December 15, 2020

    which version of tensorflow

  • Nicholas Renotte
    December 15, 2020

    Wouaouhhhhh… Man, your amazing 🙂 …
    This is what I was searching for after some months of Machine Learing study

  • Nicholas Renotte
    December 15, 2020

    Awesome!!! Thanks so much for your help 🙂

  • Nicholas Renotte
    December 15, 2020

    Awesome Video..Implemented Succesfully !!

  • Nicholas Renotte
    December 15, 2020

    Hello, Nichola. I really appreciate this tutorial video. But the problem is, can I do traffic sign detection with the same method using Tensorflow?

  • Nicholas Renotte
    December 15, 2020

    Can you do this for Tf.js and React

  • Nicholas Renotte
    December 15, 2020

    When react + tensorflow js will come?

  • Nicholas Renotte
    December 15, 2020

    This is lit 🔥 – keep it coming!

  • Nicholas Renotte
    December 15, 2020

    Will this work if you use your images but allow other person to try it out. Will it detect there hands

  • Nicholas Renotte
    December 15, 2020

    I actually wanted to do something that detects facial features and tells you what it is such as eyes and a brief description of the functions of the eye. Do You have any advice on this?
    Love your videos btw

  • Nicholas Renotte
    December 15, 2020

    Awesome video!

Write a comment