Carme (#5)

* changed settings for personal hub.

* htop added to singleuser.

* better singleuser.

* config updated.

* bad merge.

* standard updates.

* updates to builds.

* updated.

* updated path.

* loc

* stable build!

* lab enabled.

* dockerspawner bug fix.

* stable upgrade.

* remove env ending.

* minimal installation script with python 2.

* git enabled.

* env.

* upgraded hub for hartke.

* setting cull idle servers.

* default upgrade now working.

* options.

* tweaks for personal settings.

* personalization.

* update.

* attempting full latex build.

* lab build.

* typo

* separated layers.

* got a fancy environment working

* allspark.

* updated personal hub settings.

* allspark done.

* updates. tflow.

* octave deps.

* rstudio and redundancy cleanup.

* WIP

* everything WORKS!!!

* stable state with updates.

* stuff to improve the experience.

* merge.

* working state.

* settings.

* settings for carme, successful build

* settings tweaked for lab, sudo, debugger

* working on carme

* upgrade to python 3.9 and remove conda install steps

* updates

* major cleanup (#6)

* update for slalom

* disable options and grant sudo to all

* vscode launcher

* default url

* pkgs

* config

* updates

* typo

* update

* dont mount extras

* update info in config

* adding install scripts, cleaning up dockerfile

* refactor class

* clean up logic

* carme

* upgrades

* update

* swap order

* spacing

* updates

* paths

* rearranging

* cleanup

* lsp

* cleanup

* culler service update

* unecessary option

* more unecessary args

* Update .env

* monitoring

Co-authored-by: mathematicalmichael <mpilosov@gmail.com>

.env

@@ -7,19 +7,24 @@
 # are not set in the shell environment.
 
 # To override these values, set the shell environment variables.
-JUPYTERHUB_VERSION=0.9.4
+JUPYTERHUB_VERSION=1.5.0
 
 # Assign a port for the hub to be hosted on.
 # To check the ports that are in use, run `docker ps`. 
 # Generally, picking a random number between 8000-9999 won't be an issue.
-PORT_NUM=8001
+PORT_NUM=8000
-HUB_NAME=math
+HUB_NAME=carme
 
 # Name of Docker network
-#DOCKER_NETWORK_NAME=stathub-network
+DOCKER_NETWORK_NAME=hub-network
 
 # Single-user Jupyter Notebook server container image
-DOCKER_NOTEBOOK_IMAGE=jupyter/datascience-notebook
+# Tags / versioning information can be found at:
+# https://hub.docker.com/r/jupyter/tensorflow-notebook/tags
+#DOCKER_NOTEBOOK_IMAGE=jupyter/tensorflow-notebook:4d9c9bd9ced0
+# d0bfe4b38f78: tensorflow==2.4.1 python==3.9.2 lab==3.0.15 notebook==6.3.0 hub==1.4.1
+# 23ac04200f8f: tensorflow==2.6.0 python==3.9.7 lab==3.2.3 notebook==6.4.5 hub==1.5.0
+DOCKER_NOTEBOOK_IMAGE=jupyter/tensorflow-notebook:hub-1.5.0
 
 # the local image we use, after pinning jupyterhub version
 #LOCAL_NOTEBOOK_IMAGE=jupyterhub-user
@@ -39,5 +44,6 @@ DOCKER_SPAWN_CMD=start-singleuser.sh
 # afterwards, you can visit /hub/login_list (if enabled and you are an admin) to see everyone's passwords.
 # by default, hub-admin will be an admin user (defined in jupyterhub_config.py).
 # you can print the password for this hub-admin user with 'bash show_login.sh'
-USERNAME=hub-admin
+USERNAME=`whoami`
 PASSWORD_LENGTH=6
+HUB_LOC=`pwd`

Dockerfile.jupyterhub

@@ -4,18 +4,18 @@ ARG JUPYTERHUB_VERSION
 FROM jupyterhub/jupyterhub-onbuild:$JUPYTERHUB_VERSION
 
 # Install dockerspawner, oauth, postgres
-RUN /opt/conda/bin/conda install -yq psycopg2=2.7 && \
+RUN pip install psycopg2-binary~=2.7 && \
-    /opt/conda/bin/conda clean -tipsy && \
+    pip install --no-cache-dir \
-    /opt/conda/bin/pip install --no-cache-dir \
         oauthenticator==0.8.*
-RUN apt-get update && \
+#RUN apt-get update && \
-    apt-get install -y --no-install-recommends \
+#	apt-get install -y --no-install-recommends \
-    vim
+#	vim \
+# 	&& \
+#	apt-get clean && \
+#	rm -rf /var/lib/apt/lists/*
 
-RUN apt-get clean && \
+RUN pip install jupyterhub-hashauthenticator dockerspawner~=12.0
-    rm -rf /var/lib/apt/lists/*
+RUN pip install jupyterhub-idle-culler
-
-RUN pip install jupyterhub-hashauthenticator dockerspawner==0.10.0
     
 # Copy TLS certificate and key
 #ENV SSL_CERT /srv/jupyterhub/secrets/jupyterhub.crt
@@ -26,5 +26,3 @@ RUN pip install jupyterhub-hashauthenticator dockerspawner==0.10.0
 #    chmod 600 /srv/jupyterhub/secrets/*
 
 #COPY ./userlist /srv/jupyterhub/userlist
-# Download script to automatically stop idle single-user servers
-RUN wget https://raw.githubusercontent.com/jupyterhub/jupyterhub/0.9.4/examples/cull-idle/cull_idle_servers.py

docker-compose.yml

@@ -36,17 +36,12 @@ services:
       - "data:/data"
       - "./jupyterhub_config.py:/srv/jupyterhub/jupyterhub_config.py"
       - "./userlist:/srv/jupyterhub/userlist"
-      - "/home/math/:/home/"
+      # - "/tmp/.X11-unix:/tmp/.X11-unix"
     ports:
       - "${PORT_NUM}:8000"
     links:
       - hub-db
     environment:
-      #HUB_NAME: ${HUB_NAME}
-      # Notebook directory inside user image
-      #DOCKER_NOTEBOOK_DIR: ${DOCKER_NOTEBOOK_DIR}
-      # Using this run command (optional)
-      #DOCKER_SPAWN_CMD: ${DOCKER_SPAWN_CMD}
       # Postgres db info
       POSTGRES_DB: ${HUB_NAME}
       POSTGRES_HOST: hub-db

jupyterhub_config.py

@@ -4,74 +4,148 @@
 
 # Configuration file for JupyterHub
 import os
+import sys
 from subprocess import check_call
 pwd = os.path.dirname(__file__)
 c = get_config()
-hub_name = os.environ['HUB_NAME']
 
-# Spawner dropdown menu?
+
-enable_options=False
+### Helper scripts
+def create_group_map(filename='userlist') -> 'Dict[str, List[str]]':
+    """
+    Parses text file to assign users to groups and creates
+    a dictionary where keys are usernames and values are the list
+    of group names to which they belong.
+
+    You can use group membership to define policies for shared volumes
+    or use an `admin` group to determine which users get root permissions.
+
+    Note that updates to the userlist require restarts for the jupyerhub to
+    take effect. This can be inconvenient as an interruption to existing users.
+    
+    For this reason, we suggest not using `userlist` to manage
+    shared volumes but rather setting up an external filesystem on the network
+    and managing access through that (give users instructions on how to mount them
+    as folders inside their containerized environments), or perhaps opt for
+    object storage like s3 and distribute user-keys / credentials and rely on
+    CLI or programmatic file access.
+    """
+    group_map = {}
+    # TODO: check if file exists and return empty dictionary if it does not.
+    with open(os.path.join(pwd, filename)) as f:
+        for line in f:
+            if not line:
+                continue
+            # each line of file is user: group_1 group_2 ...
+            parts = line.split()
+            # in case of newline at the end of userlist file
+            if len(parts) == 0:
+                continue
+            user_name = parts[0]
+            group_map[user_name] = []
+
+            for i in range(1,len(parts)):
+                group_name = parts.pop()
+                group_map[user_name].append(group_name)
+    return group_map
+
+
+def create_volume_mount(group_id='group', mode='ro', nb_user='jovyan') -> 'Dict[str, Dict[str, str]]':
+    volumes = {}
+    volume_name = f'shared-{group_id}'
+    volume_config = {
+        'bind': f'/home/{nb_user}/{volume_name}',
+        'mode': mode,
+    }
+    volumes[volume_name] = volume_config
+    return volumes
+
+
 
 # We rely on environment variables to configure JupyterHub so that we
 # avoid having to rebuild the JupyterHub container every time we change a
 # configuration parameter.
+HUB_NAME = os.environ['HUB_NAME']
+DEFAULT_IMAGE = f'{HUB_NAME}-user'
+GROUP_MAP = create_group_map('userlist')
+
+# Allow admin users to log into other single-user servers (e.g. for debugging, testing)?  As a courtesy, you should make sure your users know if admin_access is enabled.
+c.JupyterHub.admin_access = True
+
+## Allow named single-user servers per user
+c.JupyterHub.allow_named_servers = True
+
+# Allow admin to access other users' containers
+c.NotebookApp.allow_remote_access = True
+
+# Optional list of images
+ENABLE_DROPDOWN = True
+IMAGE_WHITELIST= {
+    'default': f"{HUB_NAME}-user",
+    'scipy-notebook': "jupyter/scipy-notebook", 
+    'tensorflow-notebook': "jupyter/tensorflow-notebook",
+    'r-notebook': 'jupyter/r-notebook',
+    'base-notebook': "jupyter/base-notebook",
+}
+
 
 # Spawn single-user servers as Docker containers
-#c.JupyterHub.spawner_class = spawner = 'dockerspawner.DockerSpawner'
 from dockerspawner import DockerSpawner
 class MyDockerSpawner(DockerSpawner):
-    group_map = {}
-    with open(os.path.join(pwd, 'userlist')) as f:
-        for line in f:
-            if not line:
-                continue
-            parts = line.split()
-            # in case of newline at the end of userlist file
-            if len(parts) >= 1:
-                user_name = parts[0]
-                group_map[user_name] = []
-
-                for i in range(1,len(parts)):
-                    group_id = parts.pop()
-                    group_map[user_name].append(group_id)
     def start(self):
-        if self.user.name in self.group_map:
+        group_list = GROUP_MAP.get(self.user.name, [])
-            group_list = self.group_map[self.user.name]
+        self.update_volumes(group_list)
-            # add team volume to volumes
+        # if 'admin' in group_list:
-            for group_id in group_list: # admins in userlist get to write files.
+        #     self.mount_config_files()
-                if group_id != 'admin':
+        #     self.grant_sudo()
-                    if 'admin' in group_list: 
+        # self.limit_resources()
-                        self.volumes['shared-{}'.format(group_id)] = \
+        self.enable_lab()
-                            { 'bind': '/home/jovyan/%s'%(group_id),
+        self.grant_sudo()  # grants sudo to all users!!!
-                                'mode': 'rw' } # or ro for read-only
-                    else: # this "shared-" is part of the naming convention
-                        self.volumes['shared-{}'.format(group_id)] = \
-                            {'bind': '/home/jovyan/%s'%(group_id),
-                                'mode': 'ro' } # or rw for write (can cause conflicts)
-                else: # if admin is one of the groups in userlist, mount the following:
-                    self.volumes['%s/userlist'%(os.environ['HUB_LOC'])] = \
-                        { 'bind': '/home/jovyan/userlist', 'mode': 'rw' }
-                    self.volumes['%s/jupyterhub_config.py'%(os.environ['HUB_LOC'])] = \
-                        { 'bind': '/home/jovyan/jupyterhub_config.py', 'mode': 'rw' }
-        self.environment['JUPYTER_ENABLE_LAB'] = 'yes'
         return super().start()
 
+    def grant_sudo(self):
+        """
+        Grants sudo permission to current user being spawned.
+        """
+        self.environment['GRANT_SUDO'] = "1"
+        self.extra_create_kwargs = {'user': 'root'}
+    
+    def enable_lab(self):
+        """
+        Sets Jupyterlab as the default environment which users see.
+        """
+        self.environment['JUPYTER_ENABLE_LAB'] = 'yes'
+        self.default_url = '/lab'
+
+    def update_volumes(self, group_list):
+        for group_id in group_list:
+            mode = 'rw' if 'admin' in group_list else 'ro'
+            volume = create_volume_mount(group_id, mode, 'jovyan') 
+            self.volumes.update(volume)
+
+    def limit_resources(self, mem_limit='8G', cpu_limit=1.0):
+        self.mem_limit = mem_limit
+        self.cpu_limit = cpu_limit
+
+    def mount_config_files(self, username='jovyan'):
+        """
+        Allows editing of `jupyterhub_config.py` + `userlist` from
+        within the container but relies on using `Shut Down` from
+        the admin panel + docker automatically restarting the hub
+        in order for changes to take effect. If you make a mistake,
+        your hub will become unavailable and you will need to edit
+        it by logging into the server hosting the jupyterhub app.
+        """
+        self.volumes['%s/userlist'%(os.environ['HUB_LOC'])] = \
+            { 'bind': f'/home/{username}/userlist', 'mode': 'rw' }
+        self.volumes['%s/jupyterhub_config.py'%(os.environ['HUB_LOC'])] = \
+            { 'bind': f'/home/{username}/jupyterhub_config.py', 'mode': 'rw' }
+
 c.JupyterHub.spawner_class = MyDockerSpawner
-
+c.DockerSpawner.image = '%s-user'%HUB_NAME
-# define some task to do on startup
+c.DockerSpawner.name_template = '%s-{username}-{servername}-{imagename}'%HUB_NAME
-
+if ENABLE_DROPDOWN:
-# Spawn containers from this image (or a whitelist)
+    c.DockerSpawner.allowed_images = IMAGE_WHITELIST
-#c.DockerSpawner.image = "jupyter/datascience-notebook:7254cdcfa22b"
-c.DockerSpawner.image = '%s-user'%hub_name
-c.DockerSpawner.name_template = '{imagename}-{username}'
-if enable_options:
-    # if whitelist enabled, the .container_image will be ignored in favor of the options below:
-    c.DockerSpawner.image_whitelist = {'default': c.DockerSpawner.image , 
-                                     'scipy-notebook': "jupyter/scipy-notebook", 
-                                     'datascience-notebook': "jupyter/datascience-notebook",
-                                     'r-notebook': 'jupyter/r-notebook',
-                                     'base-notebook': "jupyter/base-notebook",
-                                     'RStudio': 'rstudio'}
 
 # JupyterHub requires a single-user instance of the Notebook server, so we
 # default to using the `start-singleuser.sh` script included in the
@@ -81,12 +155,8 @@ if enable_options:
 spawn_cmd = os.environ.get('DOCKER_SPAWN_CMD', "start-singleuser.sh")
 c.DockerSpawner.extra_create_kwargs.update({ 'command': spawn_cmd })
 
-# Memory limit
-c.Spawner.mem_limit = '4G'  # RAM limit
-c.Spawner.cpu_limit = 0.0001
-
 # Connect containers to this Docker network
-network_name = '%s-network'%hub_name
+network_name = '%s-network'%HUB_NAME
 c.DockerSpawner.use_internal_ip = True
 c.DockerSpawner.network_name = network_name
 # Pass the network name as argument to spawned containers
@@ -104,37 +174,31 @@ c.DockerSpawner.volumes = { 'hub-user-{username}': notebook_dir }
 
 # volume_driver is no longer a keyword argument to create_container()
 # c.DockerSpawner.extra_create_kwargs.update({ 'volume_driver': 'local' })
+
 # Remove containers once they are stopped
-c.DockerSpawner.remove_containers = True
+c.DockerSpawner.remove = True
+
 # For debugging arguments passed to spawned containers
 c.DockerSpawner.debug = True
 
 # User containers will access hub by container name on the Docker network
-c.JupyterHub.hub_ip = hub_name
+c.JupyterHub.hub_ip = HUB_NAME
 # The hub will be hosted at example.com/HUB_NAME/ 
-c.JupyterHub.base_url = u'/%s/'%hub_name
+c.JupyterHub.base_url = u'/%s/'%HUB_NAME
 #c.JupyterHub.hub_port = 8001
 
 ## Authentication 
 # Whitlelist users and admins
-c.Authenticator.whitelist = whitelist = set()
+c.Authenticator.allowed_users = whitelist = set()
 c.Authenticator.admin_users = admin = set()
 
 # add default user so that first-time log in is easy.
 admin.add('hub-admin')
-
+for name in GROUP_MAP:
-with open(os.path.join(pwd, 'userlist')) as f:
+    if 'admin' in GROUP_MAP[name]:
-    for line in f:
+        admin.add(name)
-        if not line:
+    else:
-            continue
+        whitelist.add(name)
-        parts = line.split()
-        # in case of newline at the end of userlist file
-        if len(parts) >= 1:
-            name = parts[0]
-            whitelist.add(name)
-            if len(parts) > 1 and parts[1] == 'admin':
-                admin.add(name)
-
 
 # Authenticate users with GitHub OAuth
 # c.JupyterHub.authenticator_class = 'oauthenticator.GitHubOAuthenticator'
@@ -166,17 +230,18 @@ c.JupyterHub.cookie_secret_file = os.path.join(data_dir,
 c.JupyterHub.db_url = 'postgresql://postgres:{password}@{host}/{db}'.format(
     host=os.environ['POSTGRES_HOST'],
     password=os.environ['POSTGRES_PASSWORD'],
-    db=hub_name,
+    db=HUB_NAME,
 )
 
-# Allow admin users to log into other single-user servers (e.g. for debugging, testing)?  As a courtesy, you should make sure your users know if admin_access is enabled.
+# https://github.com/jupyterhub/jupyterhub-idle-culler
-c.JupyterHub.admin_access = True 
-
-# Run script to automatically stop idle single-user servers as a jupyterhub service.
 c.JupyterHub.services = [
     {
-        'name': 'cull_idle',
+        "name": "jupyterhub-idle-culler-service",
-        'admin': True,
+        "command": [
-        'command': 'python /srv/jupyterhub/cull_idle_servers.py --timeout=3600'.split(),
+            sys.executable,
-    },
+            "-m", "jupyterhub_idle_culler",
+            "--timeout=3600",
+        ],
+        "admin": True,
+    }
 ]

makefile

@@ -56,7 +56,7 @@ notebook_image: pull singleuser/Dockerfile
 		--build-arg DOCKER_NOTEBOOK_IMAGE=$(DOCKER_NOTEBOOK_IMAGE) \
 		singleuser
 
-build: check-files network volumes
+build: check-files network volumes secrets/oauth.env secrets/postgres.env
 	docker-compose build
 
 .PHONY: network volumes check-files pull notebook_image build

show_login.sh

@@ -7,6 +7,6 @@ source secrets/oauth.env
 echo "Password for username: hub-admin is "
 docker run --rm $HUB_NAME hashauthpw --length $PASSWORD_LENGTH hub-admin $HASH_SECRET_KEY
 
-#echo "Password for username: $USERNAME is "
+echo "Password for username: $USERNAME is "
-#docker run --rm $HUB_NAME hashauthpw --length $PASSWORD_LENGTH $USERNAME $HASH_SECRET_KEY
+docker run --rm $HUB_NAME hashauthpw --length $PASSWORD_LENGTH $USERNAME $HASH_SECRET_KEY
 

singleuser/Dockerfile

@@ -1,89 +1,60 @@
 ARG DOCKER_NOTEBOOK_IMAGE
-ARG DISPLAY
 FROM $DOCKER_NOTEBOOK_IMAGE
 ARG JUPYTERHUB_VERSION
-#any additional installations go here.
-RUN export DISPLAY=$DISPLAY
 
 USER root
-
 RUN apt-get update && \
-    apt-get install -y --no-install-recommends \
+apt-get install -y --no-install-recommends \
-    vim \
+	dvipng \
-    htop \
+	fonts-lato \
-    fonts-dejavu \
+	fonts-dejavu \
-    tzdata \
+	gcc \
-    gfortran \
+	gfortran \
-    gcc
+	ghostscript \
+	imagemagick \
+	lmodern \ 
+	&& \
+	apt-get -qq purge && \
+	apt-get -qq clean && \
+	rm -rf /var/lib/apt/lists/*
 
-# finish off with MPI dependencies (only required if not installing fenics)
-#RUN sudo apt-get install mpich libmpich-dev -y
-RUN rm -rf /var/lib/apt/lists/*
-
-USER jovyan
-RUN conda update --all
-RUN conda install fenics
-
-# If you do not need parallelism, delete the following. 
-RUN python3 -m pip install ipyparallel mpi4py
-RUN ipython profile create --parallel --profile=mpi
-RUN ipython profile create --parallel --profile=default
-RUN echo "c.IPClusterEngines.engine_launcher_class = 'MPIEngineSetLauncher'" >> /home/jovyan/.ipython/profile_mpi/ipcluster_config.py
-
-# Python 2 environment
-RUN conda create --quiet --yes -p $CONDA_DIR/envs/python2 python=2.7 ipython ipykernel kernda numpy pandas matplotlib ipywidgets yaml ipyparallel mpi4py scipy pyDOE
-RUN /opt/conda/envs/python2/bin/ipython profile create --parallel --profile=mpi
 
+# Dev tools
 USER root
-# Create a global kernelspec in the image and modify it so that it properly activates
+COPY install_dev_tools.sh /tmp/
-# the python2 conda environment.
+RUN /tmp/install_dev_tools.sh
-RUN $CONDA_DIR/envs/python2/bin/python -m ipykernel install && \
-$CONDA_DIR/envs/python2/bin/kernda -o -y /usr/local/share/jupyter/kernels/python2/kernel.json
 
+# Final cleaning and user customizations
 USER $NB_UID
-# Jupyterhub and memory monitoring
+RUN \
-RUN python3 -m pip install --no-cache jupyterhub==$JUPYTERHUB_VERSION nbresuse && \
-    jupyter labextension install --minimize=False jupyterlab-topbar-extension && \
-    jupyter labextension install --minimize=False jupyterlab-system-monitor && \
-    npm cache clean --force && \
-    rm -rf $CONDA_DIR/share/jupyter/lab/staging && \
-    rm -rf /home/$NB_USER/.cache/yarn && \
-    rm -rf /home/$NB_USER/.node-gyp && \
     fix-permissions $CONDA_DIR && \
     fix-permissions /home/$NB_USER
 
-# R environment 
 USER root
+COPY install_vscode.sh /tmp/
+RUN /tmp/install_vscode.sh
 
-RUN apt-get update && \
+USER $NB_UID
-    apt-get install -y --no-install-recommends \
+# RUN mamba install xeus-python -y
-        libapparmor1 \
+RUN pip install jupyter_server_proxy jupyterlab-miami-nights
-        libedit2 \
+RUN pip install black flake8 autopep8 isort pre-commit
-        lsb-release \
-        psmisc \
-        libssl1.0.0 \
-        ;
 
-# You can use rsession from rstudio's desktop package as well.
+COPY install_jupyter_nbextensions.sh /tmp/
-ENV RSTUDIO_PKG=rstudio-server-1.1.463-amd64.deb
+# RUN /tmp/install_jupyter_nbextensions.sh
 
-RUN wget -q http://download2.rstudio.org/${RSTUDIO_PKG}
+COPY install_language_server.sh /tmp/
-RUN dpkg -i ${RSTUDIO_PKG}
+RUN /tmp/install_language_server.sh
-RUN rm ${RSTUDIO_PKG}
-RUN conda install -c r r-shiny && conda clean -tipsy
-RUN apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
 
-USER $NB_USER
+COPY install_monitoring.sh /tmp/
-# The desktop package uses /usr/lib/rstudio/bin
+# RUN /tmp/install_monitoring.sh
-RUN python3 -m pip install jupyter-rsession-proxy
-RUN python3 -m pip install git+https://github.com/jupyterhub/jupyter-rsession-proxy
-RUN jupyter labextension install jupyterlab-server-proxy
-ENV PATH="${PATH}:/usr/lib/rstudio-server/bin"
-ENV LD_LIBRARY_PATH="/usr/lib/R/lib:/lib:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server:/opt/conda/lib/R/lib"
-#ENV LD_LIBRARY_PATH="/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server:/opt/conda/lib/R/lib"
 
+## CONFIG
+# Copy over config which creates launcher icons in jupyterlab
+COPY jupyter_notebook_config.py /home/jovyan/.jupyter/
 
 # USER SETTINGS
-USER jovyan
+RUN echo "export EDITOR=/usr/bin/vim" >> /home/$NB_USER/.bashrc
-RUN echo "export EDITOR=/usr/bin/vim" >> /home/jovyan/.bashrc
+RUN echo "export PAGER=less" >> /home/$NB_USER/.bashrc
+
+# remember git login credentials
+RUN git config --global credential.helper "cache --timeout 144000"

singleuser/Dockerfile-R

@@ -1,34 +1,116 @@
-ARG DOCKER_NOTEBOOK_IMAGE
+FROM jupyter/r-notebook
-FROM $DOCKER_NOTEBOOK_IMAGE
-ARG JUPYTERHUB_VERSION
 
+###### R ENV #######
 USER root
 
+# R pre-requisites
 RUN apt-get update && \
     apt-get install -y --no-install-recommends \
-        libapparmor1 \
+    fonts-dejavu \
-        libedit2 \
+    unixodbc \
-        lsb-release \
+    unixodbc-dev \
-        psmisc \
+    r-cran-rodbc \
-        libssl1.0.0 \
+    gfortran \
-        ;
+    gcc && \
-
-# You can use rsession from rstudio's desktop package as well.
-ENV RSTUDIO_PKG=rstudio-server-1.0.136-amd64.deb
-
-RUN wget -q http://download2.rstudio.org/${RSTUDIO_PKG}
-RUN dpkg -i ${RSTUDIO_PKG}
-RUN rm ${RSTUDIO_PKG}
-RUN conda install -c r r-shiny
-RUN apt-get clean && \
     rm -rf /var/lib/apt/lists/*
 
-USER $NB_USER
+# Fix for devtools https://github.com/conda-forge/r-devtools-feedstock/issues/4
+RUN ln -s /bin/tar /bin/gtar
 
-RUN pip install git+https://github.com/jupyterhub/jupyter-rsession-proxy \
-  && jupyter labextension install jupyterlab-server-proxy
 
-# The desktop package uses /usr/lib/rstudio/bin
+# R pre-requisites
-ENV PATH="${PATH}:/usr/lib/rstudio-server/bin"
+RUN apt-get update && \
-ENV LD_LIBRARY_PATH="/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server:/opt/conda/lib/R/lib"
+    apt-get install -y --no-install-recommends \
+    fonts-dejavu \
+    unixodbc \
+    unixodbc-dev \
+    r-cran-rodbc \
+    gfortran \
+    gcc && \
+    apt-get clean && rm -rf /var/lib/apt/lists/*
 
+
+USER $NB_UID
+# R packages
+RUN conda install --quiet --yes \
+    'r-base=4.0.3' \
+    'r-caret=6.*' \
+    'r-crayon=1.4*' \
+    'r-devtools=2.3*' \
+    'r-forecast=8.13*' \
+    'r-hexbin=1.28*' \
+    'r-htmltools=0.5*' \
+    'r-htmlwidgets=1.5*' \
+    'r-irkernel=1.1*' \
+    'r-nycflights13=1.0*' \
+    'r-randomforest=4.6*' \
+    'r-rcurl=1.98*' \
+    'r-rmarkdown=2.6*' \
+    'r-rodbc=1.3*' \
+    'r-rsqlite=2.2*' \
+    'r-shiny=1.6*' \
+    'r-tidyverse=1.3*' \
+    'unixodbc=2.3.*' \
+    'r-tidymodels=0.1*'
+RUN \
+    conda clean --all -f -y && \
+    fix-permissions "${CONDA_DIR}" && \
+    fix-permissions "/home/${NB_USER}"
+
+# Install e1071 R package (dependency of the caret R package)
+#RUN conda install --quiet --yes r-e1071
+
+
+# RStudio
+RUN python3 -m pip install jupyter-rsession-proxy
+RUN cd /tmp/ && \
+    git clone --depth 1 https://github.com/jupyterhub/jupyter-server-proxy && \
+    cd jupyter-server-proxy/jupyterlab-server-proxy && \
+    npm install && npm run build && jupyter labextension link . && \
+    npm run build && jupyter lab build
+
+
+# install rstudio-server
+USER root
+RUN apt-get update && \
+    curl --silent -L --fail https://download2.rstudio.org/server/bionic/amd64/rstudio-server-1.4.1103-amd64.deb > /tmp/rstudio.deb && \
+    apt-get install -y /tmp/rstudio.deb && \
+    rm /tmp/rstudio.deb
+
+ENV PATH=$PATH:/usr/lib/rstudio-server/bin
+
+# I do not think these are necessary
+#RUN apt-get update && \
+#	apt-get install -y --no-install-recommends \
+#		libapparmor1 \
+#		libedit2 \
+#		lsb-release \
+#		psmisc \
+#		libssl1.0.0 \
+#		;
+
+#RUN apt-get install gdebi-core
+RUN curl --silent -L --fail https://download3.rstudio.org/ubuntu-14.04/x86_64/shiny-server-1.5.9.923-amd64.deb > /tmp/rshiny.deb && \
+    apt-get install -y /tmp/rshiny.deb && \
+    rm /tmp/rshiny.deb
+
+# Cleanup
+RUN rm -rf $CONDA_DIR/share/jupyter/lab/staging && \
+    rm -rf /home/$NB_USER/.cache/yarn && \
+    apt-get -qq purge && \
+    apt-get -qq clean && \
+    rm -rf /var/lib/apt/lists/*
+
+USER $NB_UID
+
+# Final cleaning
+# build and clean up (for good measure)
+RUN  npm cache clean --force && \
+    rm -rf /home/$NB_USER/.node-gyp
+
+RUN conda clean --all -f -y && \
+    fix-permissions $CONDA_DIR && \
+    fix-permissions /home/$NB_USER
+
+USER $NB_UID
+R

singleuser/install_dev_tools.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+apt-get update && \
+	apt-get upgrade -yq && \
+	apt-get install -y \
+	curl \
+	htop \
+	fzf \
+	less \
+	openssh-client \
+	ripgrep \
+	screen \
+	vim \
+	jq \
+	&& \
+	apt-get -qq purge && \
+	apt-get -qq clean && \
+	rm -rf /var/lib/apt/lists/*

singleuser/install_jupyter_nbextensions.sh

@@ -0,0 +1,14 @@
+
+python3 -m pip install jupyter_contrib_nbextensions && \
+	jupyter contrib nbextension install --user --symlink && \
+	jupyter nbextension enable snippets_menu/main && \
+	jupyter nbextension enable codefolding/main && \
+	jupyter nbextension enable codefolding/edit && \
+	jupyter nbextension enable execute_time/ExecuteTime && \
+	jupyter nbextension enable notify/notify && \
+	jupyter nbextension enable rubberband/main && \
+	jupyter nbextension enable varInspector/main && \
+	jupyter nbextension enable latex_envs/latex_envs && \
+	jupyter nbextension enable load_tex_macros/main && \
+	jupyter nbextension enable toc2/main
+

singleuser/install_language_server.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+# Installs language server and creates symlinks to enable discovery of code sources
+pip install 'jupyterlab~=3.0.0' && \
+	pip install jupyterlab-lsp==3.9.1 jupyter-lsp==1.5.0 && \
+	pip install git+https://github.com/krassowski/python-language-server.git@main && \
+	#pip install 'python-lsp-server[all]' && \
+	mkdir -p ~/.lsp_symlink && \
+	cd ~/.lsp_symlink && \
+	ln -s /home home && \
+	ln -s /opt opt && \
+	# pip install \
+	# pydocstyle \
+	# pyflakes \
+	# pylint \
+	# rope \
+	# yapf \
+	# mccabe \
+	# && \
+	mkdir -p ~/.config && \
+	echo "[pycodestyle]" > ~/.config/pycodestyle && \
+	echo "ignore = E402, E703, W391" >> ~/.config/pycodestyle && \
+	echo "max-line-length = 88" >> ~/.config/pycodestyle && \
+	echo "finished installing language server"

singleuser/install_monitoring.sh

@@ -0,0 +1,10 @@
+#!/bin/sh
+python3 -m pip install --no-cache jupyterhub==$JUPYTERHUB_VERSION nbresuse && \
+	jupyter labextension install --minimize=False jupyterlab-topbar-extension && \
+	jupyter labextension install --minimize=False jupyterlab-system-monitor && \
+	npm cache clean --force && \
+	rm -rf $CONDA_DIR/share/jupyter/lab/staging && \
+	rm -rf /home/$NB_USER/.cache/yarn && \
+	rm -rf /home/$NB_USER/.node-gyp && \
+	fix-permissions $CONDA_DIR && \
+	fix-permissions /home/$NB_USER

singleuser/install_vscode.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+apt-get update && \
+	apt-get install -y --no-install-recommends \
+	curl && \
+	echo "getting latest install script" && \
+	curl -fsSL https://code-server.dev/install.sh > install.sh && \
+	sh install.sh && \
+	rm install.sh && \
+	apt-get -qq purge curl && \
+	apt-get -qq purge && \
+	apt-get -qq clean && \
+	rm -rf /var/lib/apt/lists/*
+

singleuser/jupyter_notebook_config.py

@@ -0,0 +1,20 @@
+c.LatexConfig.latex_command = 'pdflatex'
+
+# Jupyterlab icons can be created here to launch apps
+# that are included in the user's image.
+c.ServerProxy.servers = {
+  'vscode': {
+    'command': [
+		'code-server',
+		'--auth', 'none',
+		'--bind-addr', '0.0.0.0',
+		'--port', '5000'
+	],
+    'port': 5000,
+    'absolute_url': False,
+    'new_browser_tab': True,
+    'launcher_entry': {
+        'title': 'VSCode',
+        },
+  }
+}