By default, Deepnote projects run in containers on Debian Buster with Python 3.9, and a number of pre-installed packages. It’s meant to "just work" for the majority of projects, and if you’re a beginner, you most likely don’t need to configure anything. However, for advanced users, Deepnote offers a large amount of flexibility.
There are 3 major ways of installing both system-level dependencies and Python packages: installing them ad-hoc, using a start-up script, and by changing the underlying image.
pip install <package_name> or
apt install <library>, either in the terminal or with the
! prefix in a notebook.
If you are in a conda environment, you can also run
conda install <package_name>.
This will immediately install the library for you to use. Note that when the hardware restarts, the environment will revert back to the default state.
If you have packages pulled from a custom registry which needs to have an IP allowlisted, Deepnote can route your pip download through a proxy. Simply add the
proxy argument with our IP
http://126.96.36.199:3128/ to the pip install.
pip install -r requirements.txt --extra-index-url=http://something.com --proxy http://188.8.131.52:3128
Deepnote provides an optional feature to run custom Python initialization code every time your project hardware boots up. To utilize this capability, you will need to include an 'Init' notebook in your project.
To create an 'Init' notebook:
Navigate to your project's Environment section in the project sidebar.
- Click on the settings icon.
- Click on the Create an initialization notebook button.
After creating it, the 'Init' notebook will be conveniently located at the top of your Notebook list. The contents of this notebook will automatically execute each time your project starts up, providing an ideal place to set up your preferred environment.
By default, the 'Init' notebook installs any packages listed in ./requirements.txt. However, you are free to customize this according to your project's needs.
For a more detailed explanation of project initialization, follow this link.
Deepnote maintains a number of supported runtime environments, including a few versions of Python and R. You can simply pick one from the dropdown in the right sidebar as shown below. The hardware will then be restarted to enable the newly selected environment.\
If you need to customize your environment further for example by installing additional libraries or binaries, you have two ways of doing so. You can either (1) use an environment defined by a local Dockerfile or (2) you can point to a hosted Docker image that you would like to use. These are explained in more detail below.
This option will use environment described by the
Dockerfile file in the root of your project. Deepnote takes care of building and hosting the Docker image. To learn more about our Anaconda Docker image, refer to the conda environment docs.
A common use-case is to extend one of the above mentioned supported images by specifying the
FROM directive at the beginning of the file (such as
FROM deepnote/python:3.7) and then adding what you need. For example, a Dockerfile that will install the dependencies needed for OpenCV would look like:
FROM deepnote/python:3.7 RUN apt update && apt install -y ffmpeg libsm6 libxext6
If you have a local Dockerfile you'd like to use, just select Local ./Dockerfile from the Environment dropdown in the right sidebar (shown above). Read more about using a local Docker file in the "Building images" section below.
You can also pull in a Docker image from somewhere else. This is an extremely powerful feature because it allows you to pick from any of the preconfigured images in public repositories such as Dockerhub.
Those on the Pro plan can also use their own private image repositories (we support Docker Hub, Google Container Registry and Amazon ECR). To use a private image repository, create an integration for the respective provider and enter your credentials. Then, connect the integration to any project where you wish to use your private Docker images.
Deepnote supports any Python version >= 3.6. You can pick 3.7, 3.8, 3.9, 3.10 & 3.11 from the dropdown (as shown above), or use a custom Docker image for more flexibility.
In the environment section (see the right sidebar), you can pick from different versions of R; however, we recommended choosing the
deepnote/ir_with_libs image. This will install R 4.2 and many common data science libraries for you (see the image details here). To learn more about using R in Deepnote, click here.
You can run Deepnote on any Docker image, as long as it satisfies a few requirements. Just add a new image from a public repository, optionally give it a name, and it’s ready to use. The initial load might take slightly longer while the image is being downloaded.
- There must be an installation of Python with version >= 3.6, which can be referenced using the
- Must be built for
linux/amd64platform. For example, if you are running M1 mac, you should build the image with
- There must be an installation of
pip, configured to install packages into a location visible to
- The image must contain
If you want to run a different kernel, read more here.
When you add a custom image to a project, you and other workspace members will be able to re-use it in other projects as well.
Sometimes you may wish to make modifications to the default source image. You can build in these changes directly in Deepnote, just pick the Local ./Dockerfile from the Environment dropdown in the right sidebar.
Even though the definition file is called "Dockerfile", it does not support the full syntax you might be used to. In order to stay compatible with Deepnote, you can only use the following statements:
- FROM clause. You can start from any public base image, or pick one of the Deepnote environments as a base.
- RUN clause. This allows you to run arbitrary commands, for example, install libraries or tools you are using in your project. The commands run as root, so you don't need to use sudo.
- ENV clause. You can use this to set environment variable or to define the default jupyter kernel for creating new notebook files.
The time-out for builds is 10 minutes for free projects and 60 minutes for the paid ones.
You can also use images built locally or from a different platform, like your CI/CD system. Note, the local builds allow for faster iteration thanks to layer caching. You will still need to host the image on an online registry.
Check out this guide for a quickstart on building images locally and hosting them on Docker Hub.
The duplicates of a project with a custom environment will retain the custom environment configuration in these cases:
- When the project is using one of the default Deepnote environments
- When the project is using the local Dockerfile built within Deepnote
- (Coming soon) When the project gets duplicated within the team space