Dependencies and cross-device Compatability

“but it was working on my other device…”

Hi everyone

Yesterday I tried to clone and tinker with an old project I had made on another device. I had only begun importing the libraries when I encountered an age-old problem.

This was working on my Mac!

These libraries had been working great on my other device(Mac) but as soon as I tried to import them using this device(PC) they weren’t available.

So today I’m going to give you a definition of what ‘dependencies’ are and explore some options about how to manage dependencies cross-device using Conda, PIP and Docker.

Dependencies are like dominoes

What are dependencies?

A data dependency in computer science is situation in which a program statement (Instruction) refers to the data of a preceding statement.
- Wikipedia

- Dependency is a broad software engineering term used to refer when a piece of software relies on another one
- Askubuntu

I have a dependency issue because the libraries which are needed for the project are not installed on this new device and therefore parts of the project that depend on those libraries will not work.

These kinds of issues come up when transferring projects/apps/code from one device to another since often the environment on one device is not identical to the other.

Photo by JOSHUA COLEMAN on Unsplash

How to deal with Dependency issues across multiple devices

Let’s identify some workarounds and solutions for dependencies across different devices. We will look at

  • A solution using a USB stick
  • A solution using your terminal
  • A solution with Docker
Photo by Sara Kurfeß on Unsplash

StackOverflow user Schlamar suggests a solution that works by synchronising package environments between two devices as opposed to cloning everything (see below). The following method only works for packages installed with python Package installer (pip).

  1. Set up the pip cache to a network storage / USB stick which is accessible from both PCs (see this link for instructions)
  2. Freeze your current package environment from the source PC into a requirements file: $ pip freeze > req.txt
  3. Copy the req file to the target PC and install the packages:
    $ pip install -r req.txt

If you put the req.txt under a version control system (e.g. Git) you can automate and synchronize this process very smoothly.

Photo by Goran Ivos on Unsplash

This solution involves cloning an environment and assumes you have Anaconda installed.

Use the terminal or an Anaconda Prompt for the following steps:

  1. You can make an exact copy of an environment by creating a clone of it:

Note — Replace myclone with the name of the new environment. Replace myenv with the name of the existing environment that you want to copy.

To verify that the copy was made:

Photo by Todd Cravens on Unsplash

I won’t go into how to get a docker image running however, once you have it is very easy to save the packages in a local environment to the docker image:
1. Like in the first solution we pip freeze our requirements to a text file

2. Then we tell our docker container we want to copy the local requirements.txt file into the container and run a pip install command passing it the requirements file. This installs all the dependencies into the container.

There’s more to docker than I have written about so see the detailed guide on how to get a docker image going… and if you are completely new to Docker or need a reminder you can read my article What the Hell is Docker?

Sources

  1. Version Control Systems — link
  2. Dependencies —link
  3. Setting up links to your pip cache — link
  4. Transfer Python Setups across different devices — link
  5. Setting up a docker container for package dependency management — link
  6. What the Hell is docker? — link

Practicing Data Scientist. Interested in Games, Gamification, Ocean Sciences, Music, Biology.