Vak prep crashes because of package version requirement

Q&A
1

I created a new environment in a machine that served many successful runs with vak and tweetynet.
This time I got an error when running vak prep on the tutorial data.

I tried creating the environment in two ways:

  1. conda create --name tweety-env python=3.8 vak -c pytorch -c conda-forge
  2. conda create --name tweety-env vak -c conda-forge

I then installed tweetynet:
conda install tweetynet -c conda-forge

The error I get is that the package ‘protobuf’ is of the wrong version.
It is a requirement in vak.
@nicholdav, Can this be changed? Or, does it need changing in pypi or conda-forge?

(tweety-env) yardenc@USGPUbeast:/media/yardenc/New Volume/YardenBirds/BF/gy6or6$ vak prep gy6or6_train.toml
Traceback (most recent call last):
File “/home/yardenc/miniconda3/envs/tweety-env/bin/vak”, line 10, in
sys.exit(main())
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/vak/main.py”, line 45, in main
cli.cli(command=args.command, config_file=args.configfile)
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/vak/cli/cli.py”, line 30, in cli
COMMAND_FUNCTION_MAPcommand
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/vak/cli/prep.py”, line 87, in prep
purpose = purpose_from_toml(config_toml, toml_path)
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/vak/cli/prep.py”, line 27, in purpose_from_toml
are_sections_valid(config_toml, toml_path=toml_path)
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/vak/config/validators.py”, line 94, in are_sections_valid
MODEL_NAMES = [model_name for model_name, model_builder in models.find()]
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/vak/config/validators.py”, line 94, in
MODEL_NAMES = [model_name for model_name, model_builder in models.find()]
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/vak/models/models.py”, line 32, in find
yield entrypoint.name, entrypoint.load()
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/pkg_resources/init.py”, line 2470, in load
self.require(*args, **kwargs)
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/pkg_resources/init.py”, line 2493, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File “/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages/pkg_resources/init.py”, line 800, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (protobuf 4.21.9 (/home/yardenc/miniconda3/envs/tweety-env/lib/python3.8/site-packages), Requirement.parse(‘protobuf<=3.20.1’), {‘vak’})

In option 2 it’s the same error, just a different version of python (3.10)

2

Hi @yardenc this is definitely a dependency issue we need to fix, thank you for catching it

workaround

Can you see if you can specify protobuf<=3.20.1’ as a workaround? Like so:

conda create --name tweety-env python=3.8 vak protobuf<=3.20.1 -c pytorch -c conda-forge

longer-term fix

There’s two things going on here:

  1. putting a temporary upper cap on this dependency to avoid an error
    which I did here: DEV: Upper bound protobuf in pyproject.toml to avoid error · vocalpy/vak@d558ae3 · GitHub
    to deal with this issue: [Python] Release 4.21.0 broke multiple Google Cloud client libraries ("TypeError: Descriptors cannot not be created directly.") · Issue #10051 · protocolbuffers/protobuf · GitHub
  2. not specifying that upper cap in the conda-forge version
    which I think might be why it gets installed for you, not sure yet.

So

Can this be changed? Or, does it need changing in pypi or conda-forge?

from that issue above it looks like release 3.20.2 might fix this issue. If so I will update that in the pyproject.toml, release a new vak, and then rebuild for conda-forge. Will get to this by the end of the week

1 Like
3

The workaround worked straight away.

1 Like
4

Great, glad to hear it.

I removed the upper cap: DEV: Remove upper cap on protobuf [skip ci] · vocalpy/vak@5c446fd · GitHub

re-installed into the test environment, and everything seemed to run fine when protobuf=20.1.3 got installed (it’s a transitive dependency, we don’t use it directly, I only specified it with that cap to avoid the issue).

I want to address your issue about spaces in paths but after that I’ll make a new release that should fix things for people installing via conda too

5

Please if you can install a dev version and test this pull request when merged – I think it should fix the spaces issue.

Once you confirm, I’ll release a new version with the upper cap removed.

6
7

Excellent, thank you @yardenc for testing that with the tutorial–good idea.

To address the original issue in this topic, I will release a new version later this week.

8

Just noting this was released, announcement here: Vak version 0.7.0 released!