Jupyterhub | 2i2c

Protecting our hubs against the CopyFail kernel exploit

Mon, 04 May 2026 00:00:00 +0000

The recently disclosed CopyFail Linux kernel zero-day (CVE-2026-31431) opens up a way for code running inside a container to break out onto the underlying node. We took a close look at our hubs to confirm whether they were exposed, confirmed that our hubs are likely not at risk, and added another layer of protection just in case.

Are 2i2c’s hubs at risk? #

No - based on our testing and mitigation efforts, our hubs are not vulnerable to CopyFail.

Why do we think we’re not at risk? #

We tried to reproduce the exploit on a staging hub by following the public Kubernetes proof-of-concept on both AWS and EKS, and the exploit was unable to break out of the container.
Existing JupyterHub hardening on Kubernetes from jupyterhub/kubespawner#545 (originally added by Yuvi in 2021 in response to a different security issue) had already significantly reduced our risk exposure, and the exposure of anyone else running Z2JH (the standard way to deploy JupyterHub on Kubernetes).
As an extra layer of protection, we deployed copyfail-ebpf-k8s as a daemonset across all of our clusters in 2i2c-org/infrastructure#8227. This runs on every node and covers all of our hubs (including those on non-commercial cloud infrastructure, like JetStream2). It blocks the specific kernel features that CopyFail depends on. See the project’s explanation for how that works.
We’ve upgraded all GKE clusters to use a patched image in 2i2c-org/infrastructure#8230.

What else did we look into #

Deckhouse’s mitigation was too platform-specific for us.
OVHcloud’s modprobe blocking likely won’t work on Amazon Linux 2023, since the relevant module is built into the kernel image.
AL2023 security advisories - no patched AL2023 image is available yet, so we can’t rely on a kernel-level fix from AWS for now.

Acknowledgements #

Huge thanks to Georgiana for the deep dive into the exploit and whether we’re exposed here.
Thanks to Yuvi for the PR that reduces JupyterHub’s exposure to this back in 2021!
Thanks to iwanhae for the eBPF daemonset we deployed in Kubernetes, and to JupyterHub for the upstream kubespawner hardening that lowered our exposure.
Thanks to our collaborators at NASA VEDA for the ongoing conversations about hub security.
Thanks to our collaborators at Pythia for supporting ongoing work around security in JupyterHub and BinderHub, especially on non-commercial cloud like JetStream.

Supporting JupyterHub admins on workshop hubs with shared passwords

Mon, 27 Apr 2026 00:00:00 +0000

To facilitate communities using JupyterHub for a workshop, we introduced the idea of a ‘shared password’ based authentication a few years ago. This lets communities set a single global password that is handed out to all workshop attendees (instead of collecting email addresses or GitHub usernames before the workshop starts). Users can login with their email and this shared global password, and get going immediately. We worked with OpenScapes (which runs a lot of workshops) in developing this method, and it’s been heavily used since among other communities!

However, until now this method required us to restrict admin access for all users, preventing users from using admin features like group management, accessing the JupyterHub Admin dashboard (which lets you see which other users are logged in), and using the shared directory.

To mitigate these, we contributed the SharedPasswordAuthenticator upstream to the JupyterHub organization, allowing hub admins to set set two passwords instead of one. Hub admins can now generate a special password that can be distributed just to admins, and that gives users these extra capabilities without giving up the simplicity of using a single password for all workshop participants!

By contributing this upstream, we made sure this feature is available for everyone to use, rather than just for communities served by 2i2c. We’re rolling this out to our communities now, and many communities have already benefitted from this!

Acknowledgements #

To MinRK for code review and merge of the pull requests!
The OpenScapes community for collaborating with us on building features that benefit themselves and everyone else!
Support from our member communities gives us the capacity to invest in upstream open source engagement and build relationships like this

Upgrading community infrastructure to Kubernetes 1.34 and JupyterHub 4.3.3

Wed, 08 Apr 2026 00:00:00 +0000

We’ve completed a major round of infrastructure upgrades across all 2i2c-managed hubs - every hub is now running Kubernetes 1.34 and Z2JH helm chart 4.3.3.

Running up-to-date versions of both Kubernetes and the JupyterHub helm chart ensures that our communities get the best support and reliability, both in terms of features and security.

A new approach to infrastructure upgrades: upgrading in rounds #

This was the first time we rolled out JupyterHub helm chart upgrades in rounds rather than all at once. By upgrading a subset of hubs at a time, we could identify and fix issues in isolation before they affected the broader network. This made the process safer and more predictable.

We’re planning to perform these kinds of upgrades on a regular schedule for our member communities. Around every 6 months we’ll create an issue to make sure nothing falls through the cracks (here’s example config for creating our reminder issues).

Check out our process docs for multi-hub upgrades for more information.

Learn more #

Check out these pages for what kinds of improvements we’ve brought into our clusters / hubs with these latest updates.

Acknowledgements #

Thanks to Georgiana Dolocan for leading this upgrade effort and establishing the rounds-based approach.
Thanks to Chris Holdgraf for adapting and editing Georgiana’s notes into a blog post.

Jenny Wong joins the JupyterHub team

Mon, 09 Feb 2026 00:00:00 +0000

We’re excited to share that Jenny Wong has been invited to join the JupyterHub team as a contributor and maintainer.

Jenny’s contributions to nbgitpuller and grafana-dashboards, along with her active participation in project meetings and community planning, earned her this recognition from the JupyterHub community. Being invited to a project team means that existing team members have recognized a pattern of high-quality contributions and trust in a person’s ability to steward the project.

We’re particularly excited about this because our mission isn’t just about deploying infrastructure - it’s about being good citizens in the open source communities we depend on. We invest in upstream contributions, participate in community governance, and aim to build the kind of relationships that strengthen the whole ecosystem. When our team members are welcomed into upstream project teams, it’s a signal that we’re doing this well.

Learn more #

JupyterHub team compass - how the team is structured and how new members are added
JupyterHub proposal to add Jenny to the team
nbgitpuller and grafana-dashboards - projects Jenny contributes to

Acknowledgements #

The JupyterHub community for fostering an open and welcoming project culture
Support from our member communities gives us the capacity to invest in upstream open source engagement and build relationships like this

Community learning: Hub config to pass oauth tokens into user environments

Thu, 06 Nov 2025 00:00:00 +0000

One of our favorite things to see: communities learning from and building on each other’s work!

MAAP recently contributed infrastructure configuration inspired by EarthScope’s approach to handling authentication tokens. Both communities need to pass OAuth tokens into user environments so their SDKs can access protected data - and MAAP adapted EarthScope’s pattern to fit their needs.

This is the kind of peer-to-peer knowledge sharing we hope to foster with our open infrastructure model. When infrastructure is open and communities can see each other’s solutions, they can adapt and build on proven approaches rather than starting from scratch.

Learn more #

MAAP’s PR adapting the configuration
EarthScope’s original config
Our infrastructure repository where all community configurations live

Acknowledgments #

MAAP team for adapting and contributing this configuration
EarthScope Consortium for the original implementation

2i2c Supports the Science Platforms Coordination IHDEA Working Group

Thu, 30 Oct 2025 00:00:00 +0000

The Science Platforms Coordination IHDEA working group (which includes our own Jim Colliander) is developing international standard software computing environments for Heliophysics. The working group recently presented their work at two major conferences: ML-Helio in Madrid and DASH/IHDEA in San Antonio.

The DASH/IHDEA 2025 conference brings together the heliophysics community to advance data, analysis, and software standards

When the working group received $2k from NASA SMCE for cloud infrastructure, they were already member organizations of 2i2c. This meant we could quickly stand up a JupyterHub with their Heliophysics-tailored environments for the conferences:

Easy access - Shared password authentication for conference attendees
Persistent storage - Work saved across sessions
Serious compute - Up to 119 GB RAM and 15 CPUs (much more than a typical laptop!)

The team successfully demonstrated how cloud resources can enable computational work that laptops simply can’t handle, and conference attendees responded positively to the presentations.

Why we’re excited about this #

This showcases a key benefit we want to create with 2i2c membership: reducing the accidental complexity of leveraging the cloud. Because the working group was already a member organization, deploying and managing infrastructure for the conferences was straightforward once they secured cloud funding. No lengthy setup, no new contracts - just quick deployment of the tools they needed.

Learn more #

ML-Helio Conference - Machine learning in heliophysics
DASH/IHDEA Conference - Data, analysis, and software in heliophysics

Acknowledgements #

NASA SMCE for providing $2k funding and AWS infrastructure
Shawn Polson for being the community champion leading this effort
The IHDEA working group for their collaborative partnership in advancing Heliophysics research infrastructure

AWI-CIROH showcases cloud infrastructure at their DevCon 2025

Thu, 12 Jun 2025 00:00:00 +0000

CIROH published a post on their DocuHub blog about their cloud infrastructure success at DevCon 2025, highlighting how their JupyterHub deployment supported the conference’s interactive computing needs.

Read the full post: DevCon 2025 Infrastructure

Announcing `jupyterhub-groups-exporter`: monitor usage based on JupyterHub group membership with Prometheus and Grafana

Wed, 11 Jun 2025 00:00:00 +0000

Managing user groups in JupyterHub can be a challenging task, especially in environments with dynamic user bases and complex group structures. This post describes how we can leverage the latest group management features in JupyterHub, along with Prometheus and Grafana, to monitor group-level resource usage effectively.

⭐ Members of 2i2c’s community network can use this feature in their hubs by following our cost attribution documentation.

Motivation #

Hub admins have a strong impetus to monitor usage and costs by user groups because it allows them to advocate for better funding and cost recovery models based on data-driven insights. Group-level resource monitoring can help them to answer questions like:

How many people participated in our workshop group?
How much GPU compute is our power user group using?
Is our resource usage cost-effective for X group persona or Y group persona?

Current methods and workarounds include:

ring-fencing resources for specific user groups personas, e.g. creating a separate hub for a workshop group, or creating a separate Dask cluster for a power user group, which increases the admin burden of managing multiple hub instances
writing custom scripts to aggregate per user metrics, that are already available, into groups – which can be time-consuming and error-prone

JupyterHub and user groups #

Recent key developments upstream in JupyterHub for groups management, such as Authenticator managed group membership, makes this piece of work a prime and timely opportunity to be tackled. For more technical details of these upstream contributions, see GitHub PRs jupyterhub/oauthenticator#735 and jupyterhub/oauthenticator#498.

Users can access JupyterHub using a variety of authentication methods. Authentication providers like GitHub have built-in user management features that allow admins to create and manage user groups. These groups can then be configured in JupyterHub to authorize access to the hub, as well as control access to certain hardware profiles.

Following the key upstream contributions above, we can leverage Authenticator-managed group membership to automatically pass user group memberships from the authentication layer to JupyterHub itself. This allows us to capitalize on JupyterHub’s REST API to retrieve user group memberships from other services, such as exporting them as Prometheus metrics.

Exporting user group memberships to Prometheus #

The jupyterhub-groups-exporter project provides a service that integrates with JupyterHub to export user group memberships as Prometheus metrics. This component is readily deployable as part of any JupyterHub instance, such as a standalone deployment or a Zero to JupyterHub deployment on Kubernetes.

The exporter provides a Gauge metric called jupyterhub_user_group_info, which contain the following labels:

namespace – the Kubernetes namespace where the JupyterHub is deployed
usergroup – the name of the user group
username – the unescaped username of the user
username_escape – the escaped username

Escaped usernames are useful because Kubernetes pods have characterset limits for valid pod label names (this limit does not apply to pod annotations). Storing both types of usernames allows us to join escaped versions with their more human-readable unescaped usernames.

Exposing this metric as an endpoint for Prometheus to scrape allows us to query and join groups data with a range of usage metrics to gain powerful group-level insights. Here is an example PromQL query that retrieves the memory usage by user group:

sum(
 container_memory_working_set_bytes{name!="", pod=~"jupyter-.*", namespace=~"$hub_name"}
 * on (namespace, pod) group_left(annotation_hub_jupyter_org_username, usergroup)
 group(
 kube_pod_annotations{namespace=~"$hub_name", annotation_hub_jupyter_org_username=~".*", pod=~"jupyter-.*"}
 ) by (pod, namespace, annotation_hub_jupyter_org_username)
 * on (namespace, annotation_hub_jupyter_org_username) group_left(usergroup)
 group(
 label_replace(jupyterhub_user_group_info{namespace=~"$hub_name", username=~".*", usergroup=~"$user_group"},
 "annotation_hub_jupyter_org_username", "$1", "username", "(.+)")
 ) by (annotation_hub_jupyter_org_username, usergroup, namespace)
) by (usergroup, namespace)

Visualizing user group resource usage with Grafana #

The PromQL query above is rather long and complex to construct! However, you can benefit from an upstream contribution to the jupyterhub/grafana-dashboards project where we have encapsulated the PromQL queries as Jsonnet code and represented them as Grafana Dashboard visualizations (also known as Grafonnet). If you have a Kubernetes cluster running JupyterHub, try deploying these Grafana Dashboards and let us know what you think!

Our particular PromQL query above is visualized in the Grafana Dashboard User Groups Diagnostics under the Memory Usage panel (see also the corresponding screenshot at the top of this post). This is equivalent to its counterpart User Diagnostics dashboard, but with resource usage visualized on a per-group level rather than a per-user level 🎉

Future work #

We have laid the foundation for joining user group data to usage metrics with Prometheus by extracting memberships from JupyterHub’s database. This unlocks potent ways in which observability systems can be extended to group-level reporting and monitoring.

Future directions for this work include:

visualising cloud cost by user group in Grafana
developing more group-level reporting and monitoring dashboards
introducing group-level resource quotas.

What do you think? How would you like to see JupyterHub’s group management features evolve? Have you tried deploying this yourself? We welcome your feedback and feel free to open GitHub issues or make contributions to the repositories mentioned in this post.

Acknowledgements #

Thanks to the JupyterHub project for their collaboration and review of this work.

2i2c hubs now run JupyterHub 5.0

Fri, 17 Jan 2025 00:00:00 +0000

We are excited to announce that all 2i2c hubs now run JupyterHub 5.0!

This is an upgrade that brings some exciting new features and improvements. Some of the highlights include:

The possibility to enable user-initiated server sharing
Authenticator-managed roles

Also, JupyterHub 5 will enable us to offer per-group shared directories in the future! Tracking Issue.

Checkout the JupyterHub 5.0 migration docs or the changelog for more details.

Improving the logged in home page experience in JupyterHub with `jupyterhub-fancy-profiles`

Mon, 18 Nov 2024 12:55:20 -0800

On most research oriented JupyterHub installations, users would like to customize their server (the environment, resources available, etc) after logging in. In Kubernetes based JupyterHub environments, a profile list provides this functionality.

(Profile List for the NASA VEDA JupyterHub with the default implementation from KubeSpawner)

The profile list is the de-facto “logged in homepage” for these users, as that is what they see after they have logged in.

In collaboration with Development Seed, funded by our earlier grant from GESIS as well as the NASA VEDA project, we have been building the jupyterhub-fancy-profiles project to improve this experience.

(Profile List for the NASA VEDA JupyterHub with jupyterhub-fancy-profiles)

Last week, we rolled this new experience out to all 2i2c managed JupyterHubs! Here’s a quick rundown of what this enables:

Descriptions for choices in the dropdowns, making it much easier for users to know what they are getting with each environment (or resource selection).
Fully backwards compatible with the existing KubeSpawner profile list implementation. In our PR to roll this out to all hubs, you notice that we didn’t have to change the structure of any profile lists! So you can safely roll this out to your hubs too without needing to fundamentally change how your profiles are set up.
It is a modern web app (built with react), just like the JupyterHub admin panel. This allows us to evolve and satisfy user needs much faster, as well as expanding the pool of people who can contribute to the project!
Support for dynamically building images using mybinder.org style repositories! It talks to the binderhub API so users can build reproducible environments as they wish without admin involvement nor needing to fully understand how docker and containers work. Our earlier blog post has more information.

This is just the start, and thanks to ongoing funding from the NASA VEDA project, we are going to continue making improvements to this experience.

Use this in your JupyterHub #

As with everything we build at 2i2c (per our right to replicate policy), this project can be used with any JupyterHub installation that uses Kubernetes. There are instructions in the README. Please try it out on yours and let us know what you think!

Credit #

The project was initiated with funding generously provided by GESIS (see our earlier blog post).
Sanjay Bhangar and Oliver Roick from Development Seed for advocating for this project and contributing heavily to it.
The NASA VEDA project (in particular, Brian Freitag and Alex Mandel), for continued funding (in the form of engineering time) plus being early adopters!

Integrating BinderHub with JupyterHub: Empowering users to manage their own environments

Wed, 03 Jan 2024 16:56:14 -0800

Thanks to Arnim Bleier, Jenny Wong, Georgiana Elena, Damián Avila, Jim Colliander and James Munroe for contributing to this blog post

mybinder.org is a very popular service that allows end users to specify and share the environment (languages, packages, etc) required for their notebooks to run correctly by placing configuration files they are already familiar with (like requirements.txt or environment.yml) along with their notebooks. While not without its own set of challenges, this is extremely powerful because it puts control of the environment in the hands of the people who write the code. They can customize the environment to fit the needs of their code, instead of having to fit their code into the environment that admins have made available.

But, mybinder.org (and the BinderHub software that powers it) is built for sharing your work after you are done with it, not for actively doing work. BinderHubs often do not have persistent storage nor persistent user identity, and UX is centered around ephemeral interactivity that can be shared with others (via a link), rather than persistent interactivity that a single user repeatedly comes back to. JupyterHub is more commonly used for this kinda workflow, but doesn’t currently have the ability for users to easily build their own environments. Admins who are running the JupyterHub can make multiple environments available for users to choose from, but this still puts admins in the critical path for environment customization.

Our collaboration with GESIS, NFDI4DS, and CESSDA, aims to bring this flexibility to JupyterHub directly. We aim to empower users to decide for themselves which applications and dependencies are installed on a per-project basis. Our work enables communities with heterogeneous requirements to share a single Hub. Our approach frees administrators from being overwhelmed by installation requests and transforms the JupyterHub platform into a platform for collaborative computational reproducibility. In this update, we report on our progress and upcoming steps in this project.

What does a BinderHub do, exactly? #

It is helpful to understand that BinderHub primarily has 3 responsibilities:

Present a UI to the end user for them to provide details on what to build (this is what you see when you go to mybinder.org)
Call out to repo2docker in a scalable way to actually build and push an image containing the environment for the given repository, and show the user logs as this build process happens. This also allows users to debug issues with their build more easily.
Talk to a JupyterHub instance to launch a user server with the built docker image, and redirect the user to this.

(2) is really the core feature of BinderHub, and we settled on figuring out how to make that available to JupyterHub users. It was really important to us that this was also done in a way that can be sustainably used by everyone, not just 2i2c. This blog post discusses the various improvements to the broad ecosystem of projects in the Jupyter ecosystem to get this done.

Demo #

But first, a very quick demo of how this looks like right now now!

This is very much a work in progress, but the basic flow can be seen clearly. Users see a Server Options menu after they log into JupyterHub. They can specify the two primary things that determine the server configuration:

The resources allocated (RAM, CPU and maybe GPU)
The environment (container image) used, which can be specified in one of 3 ways:

a. A pre-selected list of environments (container images), provided by the administrators who set up this JupyterHub b. A blank text box where you can enter any publicly available docker image they want c. A mybinder.org style way to specify a GitHub repository, which will be then dynamically built into a docker image for the user!

So what did we need to do to accomplish this, in a way that’s very upstream friendly and usable by everyone (and not just 2i2c)?

A Standalone `binderhub-service` helm chart #

The default upstream BinderHub helm chart includes a JupyterHub as a dependency, and configures itself to be used primarily in a manner similar to mybinder.org. As the person who helped make that choice early on, I can tell you why it was made - for convenience! And it was very convenient, as it allowed us to get mybinder.org going fast. However, it makes it difficult to install a BinderHub service alongside an existing JupyterHub. To this end, we have created a standalone BinderHub helm chart, designed to be installed alongside an existing JupyterHub, so we can use it purely to build images. This allows the BinderHub instance to be used as a JupyterHub Service, which is what we want.

While this helm chart is currently under the 2i2c GitHub org, the hope is that it can eventually migrate to a jupyterhub-contrib organization (once it is created), or it can become the upstream helm chart for BinderHub if enough work can be done in BinderHub to allow it to serve use cases like mybinder.org.

As part of this work, we also added a way for BinderHub to run in API only mode, so we can fully turn off the UI and launching ability of BinderHub. This change decoupled the three responsibilities of BinderHub we discussed previously, allowing us to bring our own UI and JupyterHub. BinderHub could now be used purely for its scalable image building features, which is exactly what we want!

Sustainably extending KubeSpawner’s `profileList` #

We identified KubeSpawner’s profileList feature as the ideal location for UI to dynamically build environments (container images), making it just another ’environment choice’ people can choose, along with picking the resources their server needs. From an end-user perspective, it was also the logical place for them to specify a repository to build into an environment, as they could already choose some pre-built environments from here. They can also select other arbitrary resources they want (such as memory, GPU, etc) from here as well. From a maintainer perspective, it helps with long-term maintenance of the JupyterHub projects.

The implementation of profileList however, was not easy to extend at this point. So this PR improved how easy it was to extend it in more complex ways, without making the implementation in KubeSpawner itself complicated. Even though this had no visible end-user effects, it was an extremely important step in allowing us to experiment with UI in a sustainable way without having to rely on upstream. These kinds of changes can sometimes be hard to sell to stakeholders but are extremely important in ensuring a continuous and sustainable relationship with upstream.

Implementing `unlisted_choice` feature in KubeSpawner #

The profileList feature was built to allow JupyterHub admins to specify an explicit list of container images the end-user can choose from. It did not have a way for any choice that was not pre-approved by the admin to be used. We needed this feature since the BinderHub API will build a new docker image for each environment the user wants, and so this can not be chosen from a pre-approved list. We had to safely add this feature to KubeSpawner in such a way that it was generally useful to everyone. Many other communities had been asking for such a feature anyway - the ability to simply ’type in’ an image and have that be used.

NASA VEDA was one such community, so we partnered with Sanjay Bhangar from Development Seed (an organization that helps run NASA VEDA) to implement this feature. Engineers from 2i2c contributed heavily to this feature as well, and after several PRs ( 1, 2, 3, 4 and 5), this feature is now available for everyone to use!

A key component of doing sustainable upstream work is that every addition needs to be useful by itself for a broad group of people. This change was very helpful for many communities that wanted to allow their users the freedom to pick whatever image they want to use, regardless of wether they wanted to use dynamic image building or not. The broad interest allowed us to build a coalition with other interested parties, and get the change accepted upstream more easily!

`jupyterhub-fancy-profiles` #

Once we had all these pieces in place, it was time to actually work on the frontend UI that would allow users to build images dynamically and launch them. Since this will replace the ‘profileList’ feature, it should also allow them to select different resources (RAM, CPU, etc) as needed, as well as type in an existing image if they desire. So it was a full re-implementation of the profileList frontend.

This is ongoing now at the jupyterhub-fancy-profiles project. It is a pure frontend web application, using modern frontend tooling ( React, webpack, Babel, etc) and written in JavaScript. It’s gone through a few revisions, but the demo provided earlier in the blog post is in its current state. Because the default profileList implementation is pure HTML / CSS with very minimal JS, it is limited in what kind of UX it could have. jupyterhub-fancy-profiles aims to be very helpful even when dynamic image-building features are not enabled on a JupyterHub. We hope to roll this out to a few JupyterHubs and improve it over time based on feedback.

`jupyterhub/@binderhub-client` npm package #

While building jupyterhub-fancy-profiles, we wanted to use the same javascript code used by BinderHub frontend to interact with the BinderHub API, instead of re-implementing it. However, the existing BinderHub JavaScript code was not easily consumable by external projects. We refactored the code, added tests, migrated to use modern JS practices and published the jupyterhub/@binderhub-client NPM package that can be used not just by jupyerhub-fancy-profiles but any external project for talking to the BinderHub API.

This had to be done in such a way that current BinderHub installations (such as mybinder.org) do not break. That took quite a few pull requests: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15. This refactoring work was very helpful to us, and also appreciated by the broader community.

Defending against cryptojacking with `cryptnono` #

For Open Science to flourish, we need to allow access to resources without login / paywalls wherever possible. A new menace against this has been cryptojacking - where attackers use up any and all available free compute to mine cryptocurrencies. This has affected many folks on the internet, including GitHub Actions and mybinder.org, the primary public BinderHub installation. mybinder.org has some extra protections against cryptojacking that aren’t easily usable elsewhere, and this has unfortunately meant that the demo JupyterHubs we have with these features enabled have been behind a login wall. I personally believe login walls are long term antithetical to open science, and so this was an important problem to solve.

cryptnono is an open source project designed to help fight cryptojacking, and as part of this grant we ported some of this functionality out of mybinder.org specific code into cryptnono, so other deployments may also benefit from it! We also migrated to using the super efficient ebpf Linux Kernel subsystem, allowing for more complex heuristics to catch a much broader range of cryptomining activity. We have been slowly tweaking the config on mybinder.org, and it has proven to be very effective! This will be very helpful for anyone who wants to provide a JupyterHub (or any other computational service) without a login wall. If you are interested in using cryptnono in this fashion, please reach out to us so we can work together!

Explored pathways that were then discarded #

List of things that were tried and then decided as not good pathways:

repo2docker-service, a separate JupyterHub service that could only build images. As we worked on it, we realized that it was replicating a lot of features that BinderHub already has, so we pivoted to working on BinderHub directly instead.
Building off of tljh-repo2docker. While this already had a nice UI, it would be hard to port it to run on a distributed Kubernetes environment without it becoming a ‘hard fork’.

While these did slow down the implementation of the project, it has allowed us to be very confident that the methods we have chosen are long-term sustainable.

Want to try this out? #

We have a demo of this running at imagebuilding-demo.2i2c.cloud, but unfortunately as we are still fine-tuning cryptnono config, at this moment it is not open to the public. Please contact me with your GitHub account if you want access, and promise to not be a cryptominer and you shall be granted access.

Want to set this up on your own JupyterHub? There is some work in progress documentation and more is being worked on. Drop a line in the linked pull request and we’ll be happy to help. The eventual goal is for anyone to be able to simply follow documentation and set this up for themselves.

We also have user facing documentation on using this service on docs.2i2c.org.

Future work #

This is not complete of course, and there is a lot of future work to be done.

mybinder.org also helps you distribute your content, not just the environment for your code to run in. Since JupyterHub usually comes with a persistent home directory for the user, nbgitpuller is commonly used for this purpose instead. We should explore ways to integrate nbgitpuller (and other ways to distribute content) in the future.
More thorough documentation for how you can recreate what is in the demo for yourself in your own JupyterHub installation.
Better UX for specifying images, including figuring out how to ‘save’ them for future reuse.
Better compatibility with mybinder.org, particularly in allowing other sources of environments (not just GitHub, but Zenodo, raw git repositories, etc) and URL compatibility.
Better authentication workflow between the frontend and the BinderHub API.

Credit #

All this work would not be possible without a large group of collaborators!

From 2i2c: Erik Sundell, Georgiana Elena, Yuvi, James Munroe, and Damián Avila.
The persistent BinderHub project was the direct inspiration for all this work, with particular thanks to Kenan Erdogan.
The tljh-repo2docker project, which explores similar ideas in the context of running only on a single node.
The broad JupyterHub and MyBinder.org community, particularly Simon Li and MinRK.
Funding generously provided by GESIS in cooperation with NFDI4DS (project number: 460234259) and CESSDA.
Arnim Bleier from GESIS was instrumental in making this project happen.

Jupyterhub | 2i2c

Protecting our hubs against the CopyFail kernel exploit

Are 2i2c’s hubs at risk? #

Why do we think we’re not at risk? #

What else did we look into #

Acknowledgements #

Supporting JupyterHub admins on workshop hubs with shared passwords

Acknowledgements #

Upgrading community infrastructure to Kubernetes 1.34 and JupyterHub 4.3.3

A new approach to infrastructure upgrades: upgrading in rounds #

Learn more #

Acknowledgements #

Jenny Wong joins the JupyterHub team

Learn more #

Acknowledgements #

Community learning: Hub config to pass oauth tokens into user environments

Learn more #

Acknowledgments #

2i2c Supports the Science Platforms Coordination IHDEA Working Group

Why we’re excited about this #

Learn more #

Acknowledgements #

AWI-CIROH showcases cloud infrastructure at their DevCon 2025

Announcing `jupyterhub-groups-exporter`: monitor usage based on JupyterHub group membership with Prometheus and Grafana

Motivation #

JupyterHub and user groups #

Exporting user group memberships to Prometheus #

Visualizing user group resource usage with Grafana #

Future work #

Acknowledgements #

2i2c hubs now run JupyterHub 5.0

Improving the logged in home page experience in JupyterHub with `jupyterhub-fancy-profiles`

Use this in your JupyterHub #

Credit #

Integrating BinderHub with JupyterHub: Empowering users to manage their own environments

What does a BinderHub do, exactly? #

Demo #

A Standalone binderhub-service helm chart #

Sustainably extending KubeSpawner’s profileList #

Implementing unlisted_choice feature in KubeSpawner #

jupyterhub-fancy-profiles #

jupyterhub/@binderhub-client npm package #

Defending against cryptojacking with cryptnono #

Explored pathways that were then discarded #

Want to try this out? #

Future work #

Credit #

A Standalone `binderhub-service` helm chart #

Sustainably extending KubeSpawner’s `profileList` #

Implementing `unlisted_choice` feature in KubeSpawner #

`jupyterhub-fancy-profiles` #

`jupyterhub/@binderhub-client` npm package #

Defending against cryptojacking with `cryptnono` #