Stop Running Jupyter Notebooks From Your Command Line

[ad_1]

By Ashton Sidhu, Data Scientist

Figure

Photo taken by Justin Jairam from @jusspreme (with permission)

 

Jupyter Notebook gives a fantastic platform to provide human-readable paperwork containing code, equations, evaluation, and their descriptions. Some even think about it a strong improvement when combining it with NBDev. For such an integral software, the out of the field begin up will not be the most effective. Each use requires beginning the Jupyter internet software from the command line and coming into your token or password. The complete internet software depends on that terminal window being open. Some would possibly “daemonize” the method after which use nohup to detach it from their terminal, however that’s not probably the most elegant and maintainable resolution.

Lucky for us, Jupyter has already provide you with an answer to this drawback by popping out with an extension of Jupyter Notebooks that runs as a sustainable internet software and has built-in person authentication. To add a cherry on high, it may be managed and sustained by means of Docker permitting for remoted improvement environments.

By the top of this publish we’ll leverage the ability of JupyterHub to entry a Jupyter Notebook occasion which could be accessed with out a terminal, from a number of units inside your community, and a extra person pleasant authentication methodology.

 

Prerequisites

 
A primary data of Docker and the command line can be helpful in setting this up.

I like to recommend doing this on probably the most highly effective machine you’ve gotten and one that’s turned on for many of the day, ideally all day. One of the advantages of this setup is that it is possible for you to to make use of Jupyter Notebook from any machine in your community, however have all of the computation occur on the machine we configure.

 

What is Jupyter Hub

 
JupyterHub brings the ability of notebooks to teams of customers. The thought behind JupyterHub was to scale out using Jupyter Notebooks to enterprises, school rooms, and huge teams of customers. Jupyter Notebook, nevertheless, is meant to run as a neighborhood occasion, on a single node, by a single developer. Unfortunately, there was no center floor to have the usability and scalability of JupyterHub and the simplicity of operating a neighborhood Jupyter Notebook. That is, till now.

JupyterHub has pre-built Docker photos that we will make the most of to spawn a single pocket book on a whim, with little to no overhead in technical complexity. We are going to make use of the mixture of Docker and JupyterHub to entry Jupyter Notebooks from anytime, anyplace, on the identical URL.

 

Architecture

 
The structure of our JupyterHub server will consist of two companies: JupyterHub and JupyterLab. JupyterHub would be the entry level and can spawn JupyterLab situations for any person. Each of those companies will exist as a Docker container on the host.

Figure

JupyterLab Architecture Diagram (Image by Author)

 

 

Building the Docker Images

 
To construct our at-home JupyterHub server we’ll use the pre-built Docker photos of JupyterHub & JupyterLab.

 

Dockerfiles

 
The JupyterHub Docker picture is easy.

FROM jupyterhub/jupyterhub:1.2# Copy the JupyterHub configuration within the container
COPY jupyterhub_config.py .# Download script to routinely cease idle single-user servers
COPY cull_idle_servers.py .# Install dependencies (for superior authentication and spawning)
RUN pip set up dockerspawner

We use the pre-built JupyterHub Docker Image and add our personal configuration file to cease idle servers, cull_idle_servers.py. Lastly, we set up extra packages to spawn JupyterLab situations through Docker.

 

Docker Compose

 
To convey the whole lot collectively, let’s create a docker-compose.yml file to outline our deployments and configuration.

model: '3'companies:
  # Configuration for Hub+Proxy
  jupyterhub:
    construct: .                # Build the container from this folder.
    container_name: jupyterhub_hub   # The service will use this container identify.
    volumes:                         # Give entry to Docker socket.
      - /var/run/docker.sock:/var/run/docker.sock
      - jupyterhub_data:/srv/jupyterlab
    surroundings:                     # Env variables handed to the Hub course of.
      DOCKER_JUPYTER_IMAGE: jupyter/tensorflow-notebook
      DOCKER_NETWORK_NAME: ${COMPOSE_PROJECT_NAME}_default
      HUB_IP: jupyterhub_hub
    ports:
      - 8000:8000
    restart: unless-stopped  # Configuration for the single-user servers
  jupyterlab:
    picture: jupyter/tensorflow-notebook
    command: echovolumes:
  jupyterhub_data:

The key surroundings variables to notice are DOCKER_JUPYTER_IMAGE and DOCKER_NETWORK_NAME. JupyterHub will create Jupyter Notebooks with the photographs outlined within the surroundings variable.For extra data on choosing Jupyter photos you possibly can go to the next Jupyter documentation.

DOCKER_NETWORK_NAME is the identify of the Docker community utilized by the companies. This community will get an automated identify from Docker Compose, however the Hub must know this identify to attach the Jupyter Notebook servers to it. To management the community identify we use a bit hack: we move an surroundings variable COMPOSE_PROJECT_NAME to Docker Compose, and the community identify is obtained by appending _default to it.

Create a file known as .env in the identical listing because the docker-compose.yml file and add the next contents:

COMPOSE_PROJECT_NAME=jupyter_hub

 

Stopping Idle Servers

 
Since that is our residence setup, we would like to have the ability to cease idle situations to protect reminiscence on our machine. JupyterHub has companies that may run alongside facet it and considered one of them being jupyterhub-idle-culler. This service stops any situations which might be idle for a chronic length.

To add this servive, create a brand new file known as cull_idle_servers.py and replica the contents of jupyterhub-idle-culler mission into it.

Ensure `cull_idle_servers.py` is in the identical folder because the Dockerfile.

To discover out extra about JupyterHub companies, take a look at their official documentation on them.

 

Jupyterhub Config

 
To end off, we have to outline configuration choices such, quantity mounts, Docker photos, companies, authentication, and many others. for our JupyterHub occasion.

Below is a straightforward jupyterhub_config.py configuration file I take advantage of.

import os
import sysc.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'
c.DockerSpawner.picture = os.environ['DOCKER_JUPYTER_IMAGE']
c.DockerSpawner.network_name = os.environ['DOCKER_NETWORK_NAME']
c.JupyterHub.hub_connect_ip = os.environ['HUB_IP']
c.JupyterHub.hub_ip = "0.0.0.0" # Makes it accessible from anyplace in your communityc.JupyterHub.admin_access = Truec.JupyterHub.companies = [
    {
        'name': 'cull_idle',
        'admin': True,
        'command': [sys.executable, 'cull_idle_servers.py', '--timeout=42000']
    },
]c.Spawner.default_url = '/lab'notebook_dir = os.environ.get('DOCKER_NOTEBOOK_DIR') or '/residence/jovyan/work'
c.DockerSpawner.notebook_dir = notebook_dir
c.DockerSpawner.volumes = {
    '/residence/sidhu': '/residence/jovyan/work'
}

Take be aware of the next configuration choices:

  • 'command': [sys.executable, 'cull_idle_servers.py', '--timeout=42000'] : Timeout is the variety of seconds till an idle Jupyter occasion is shut down.
  • c.Spawner.default_url = '/lab': Uses Jupyterlab as an alternative of Jupyter Notebook. Comment out this line to make use of Jupyter Notebook.
  • '/residence/sidhu': '/residence/jovyan/work': I mounted my residence listing to the JupyterLab residence listing to have entry to any tasks and notebooks I’ve on my Desktop. This additionally permits us to attain persistence within the case we create new notebooks, they’re saved to our native machine and won’t get deleted if our Jupyter Notebook Docker container is deleted.

Remove this line if you don’t want to mount your house listing and don’t forget to vary sidhu to your person identify.

 

Start the Server

 
To begin the server, merely run docker-compose up -d, navigate to localhost:8000 in your browser and you need to have the ability to see the JupyterHub touchdown web page.

Figure

JupyterHub Landing Page Screenshot (Image by Author)

 

To entry it on different units in your community such asva laptop computer, an iPad, and many others, establish the IP of the host machine by operating ifconfig on Unix machines & ipconfig on Windows.

Figure

Ipconfig (Image by Author)

 

From your different machine, navigate to the IP you discovered on port 8000: http://IP:8000 and you need to see the JupyterHub touchdown web page!

 

Authenticating

 
That leaves us with the final process of authenticating to the server. Since we didn’t arrange a LDAP server or OAuth, JupyterHub will use PAM (Pluggable Authentication Module) authentication to authenticate customers. This means JupyterHub makes use of the person identify and passwords of the host machine to authenticate.

To make use of this, we must create a person on the JupyterHub Docker container. There are different methods of doing this corresponding to having a script positioned on the container and executed at container begin up however we’ll do it manually as an train. If you tear down or rebuild the container you’ll have to recreate customers.

I don’t advocate exhausting coding person credentials into any script or Dockerfile.

1) Find the JupyterLab container ID: docker ps -a

Figure

JupyterLab Container ID (Image by Author)

 

2) “SSH” into the container: docker exec -it $YOUR_CONTAINER_ID bash
3) Create a person and observe the terminal prompts to create a password: useradd $YOUR_USERNAME
4) Sign in with the credentials and also you’re all set!

You now have a able to go Jupyter Notebook server that may be accessed from any machine, within the palm of your arms! Happy Coding!

 

Feedback

 
I welcome any and all suggestions about any of my posts and tutorials. You can message me on twitter or e-mail me at sidhuashton@gmail.com.

 
Bio: Ashton Sidhu (@ashtonasidhu) is a Data Scientist targeted within the cyber safety space. He is an engineer at coronary heart and have each a Bachelors of Applied Science and Engineering in addition to a Master’s of Engineering targeted in Information Systems in addition to Predictive and Perscriptive analytics.

Original. Reposted with permission.

Related:



[ad_2]

Source hyperlink

Write a comment