Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade pyyaml version to make it compatible with python 3.12.2 #247

Conversation

Faraz32123
Copy link

@Faraz32123 Faraz32123 commented Mar 15, 2024

pyyaml==6.0 is not compatible with python 3.12.
pyyaml==6.0.1 is compatible with python 3.8.0 to 3.12.2.
celery=4.x is also not compatible with 3.12.2. It throws below error

2024-04-15 12:36:32 Traceback (most recent call last):
2024-04-15 12:36:32   File "/openedx/venv/bin/celery", line 5, in <module>
2024-04-15 12:36:32     from celery.__main__ import main
2024-04-15 12:36:32   File "/openedx/venv/lib/python3.12/site-packages/celery/__init__.py", line 19, in <module>
2024-04-15 12:36:32     from . import local  # noqa
2024-04-15 12:36:32     ^^^^^^^^^^^^^^^^^^^
2024-04-15 12:36:32   File "/openedx/venv/lib/python3.12/site-packages/celery/local.py", line 17, in <module>
2024-04-15 12:36:32     from .five import PY3, bytes_if_py2, items, string, string_t
2024-04-15 12:36:32   File "/openedx/venv/lib/python3.12/site-packages/celery/five.py", line 7, in <module>
2024-04-15 12:36:32     import vine.five
2024-04-15 12:36:32   File "/openedx/venv/lib/python3.12/site-packages/vine/__init__.py", line 8, in <module>
2024-04-15 12:36:32     from .abstract import Thenable
2024-04-15 12:36:32   File "/openedx/venv/lib/python3.12/site-packages/vine/abstract.py", line 6, in <module>
2024-04-15 12:36:32     from .five import with_metaclass, Callable
2024-04-15 12:36:32   File "/openedx/venv/lib/python3.12/site-packages/vine/five.py", line 364, in <module>
2024-04-15 12:36:32     from inspect import formatargspec, getargspec as _getargspec  # noqa
2024-04-15 12:36:32     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-15 12:36:32 ImportError: cannot import name 'formatargspec' from 'inspect' (/opt/pyenv/versions/3.12.2/lib/python3.12/inspect.py). Did you mean: 'formatargvalues'?

backports-zoneinfo comes by-default in newer versions of python
it gives error while building wheel with python>=3.9

100.5 Downloading zope.interface-6.3-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (255 kB)
101.6    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 255.4/255.4 kB 223.8 kB/s eta 0:00:00
102.2 Building wheels for collected packages: backports-zoneinfo
102.2   Building wheel for backports-zoneinfo (pyproject.toml): started
102.8   Building wheel for backports-zoneinfo (pyproject.toml): finished with status 'error'
102.8   error: subprocess-exited-with-error
102.8
102.8   × Building wheel for backports-zoneinfo (pyproject.toml) did not run successfully.
102.8   │ exit code: 1
102.8   ╰─> [41 lines of output]
102.8       running bdist_wheel
102.8       running build
102.8       running build_py
102.8       creating build
102.8       creating build/lib.linux-x86_64-cpython-312
102.8       creating build/lib.linux-x86_64-cpython-312/backports
102.8       copying src/backports/__init__.py -> build/lib.linux-x86_64-cpython-312/backports
102.8       creating build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/_common.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/_version.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/_zoneinfo.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/__init__.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/_tzpath.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       running egg_info
102.8       writing src/backports.zoneinfo.egg-info/PKG-INFO
102.8       writing dependency_links to src/backports.zoneinfo.egg-info/dependency_links.txt
102.8       writing requirements to src/backports.zoneinfo.egg-info/requires.txt
102.8       writing top-level names to src/backports.zoneinfo.egg-info/top_level.txt
102.8       reading manifest file 'src/backports.zoneinfo.egg-info/SOURCES.txt'
102.8       reading manifest template 'MANIFEST.in'
102.8       warning: no files found matching '*.png' under directory 'docs'
102.8       warning: no files found matching '*.svg' under directory 'docs'
102.8       no previously-included directories found matching 'docs/_build'
102.8       no previously-included directories found matching 'docs/_output'
102.8       adding license file 'LICENSE'
102.8       adding license file 'licenses/LICENSE_APACHE'
102.8       writing manifest file 'src/backports.zoneinfo.egg-info/SOURCES.txt'
102.8       copying src/backports/zoneinfo/__init__.pyi -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/py.typed -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       running build_ext
102.8       building 'backports.zoneinfo._czoneinfo' extension
102.8       creating build/temp.linux-x86_64-cpython-312
102.8       creating build/temp.linux-x86_64-cpython-312/lib
102.8       gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/openedx/venv/include -I/opt/pyenv/versions/3.12.2/include/python3.12 -c lib/zoneinfo_module.c -o build/temp.linux-x86_64-cpython-312/lib/zoneinfo_module.o -std=c99
102.8       lib/zoneinfo_module.c: In function ‘zoneinfo_fromutc’:
102.8       lib/zoneinfo_module.c:600:19: error: ‘_PyLong_One’ undeclared (first use in this function); did you mean ‘_PyLong_New’?
102.8         600 |             one = _PyLong_One;
102.8             |                   ^~~~~~~~~~~
102.8             |                   _PyLong_New
102.8       lib/zoneinfo_module.c:600:19: note: each undeclared identifier is reported only once for each function it appears in
102.8       error: command '/usr/bin/gcc' failed with exit code 1
102.8       [end of output]
102.8
102.8   note: This error originates from a subprocess, and is likely not a problem with pip.
102.8   ERROR: Failed building wheel for backports-zoneinfo
102.8 Failed to build backports-zoneinfo
102.8 ERROR: Could not build wheels for backports-zoneinfo, which is required to install pyproject.toml-based projects
------
Dockerfile:83
--------------------
  81 |
  82 |     # python requirements
  83 | >>> RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared,uid=${APP_USER_ID} pip install -r requirements.txt
  84 |     # https://pypi.org/project/uWSGI/
  85 |     RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared,uid=${APP_USER_ID} pip install uwsgi==2.0.24
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install -r requirements.txt" did not complete successfully: exit code: 1

Upgraded Celery in this PR through constraints.txt which also resolves other dependencies conflicts with python 3.12.2.
This change is necessary to be merged to close this issue in tutor-ecommerce.

@Faraz32123
Copy link
Author

closing this PR in favour of #246

requirements/base.txt Outdated Show resolved Hide resolved
@Faraz32123 Faraz32123 force-pushed the upgrade_versions_for_python_v3.12.2_compatibility branch from f432ae4 to c735e92 Compare April 19, 2024 06:50
@Faraz32123 Faraz32123 force-pushed the upgrade_versions_for_python_v3.12.2_compatibility branch from c735e92 to 085084e Compare April 19, 2024 07:01
@DawoudSheraz
Copy link

@feanil Hi, need your assistance in getting this PR reviewed. Thanks

Also, I see codecov checks are stuck for other PRs as well. We can ignore it for now.

@Faraz32123 Faraz32123 force-pushed the upgrade_versions_for_python_v3.12.2_compatibility branch from ff536b8 to 085084e Compare April 19, 2024 11:49
backports-zoneinfo comes by-default in newer versions of python
it gives error while building wheel with python>=3.9

100.5 Downloading zope.interface-6.3-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (255 kB)
101.6    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 255.4/255.4 kB 223.8 kB/s eta 0:00:00
102.2 Building wheels for collected packages: backports-zoneinfo
102.2   Building wheel for backports-zoneinfo (pyproject.toml): started
102.8   Building wheel for backports-zoneinfo (pyproject.toml): finished with status 'error'
102.8   error: subprocess-exited-with-error
102.8
102.8   × Building wheel for backports-zoneinfo (pyproject.toml) did not run successfully.
102.8   │ exit code: 1
102.8   ╰─> [41 lines of output]
102.8       running bdist_wheel
102.8       running build
102.8       running build_py
102.8       creating build
102.8       creating build/lib.linux-x86_64-cpython-312
102.8       creating build/lib.linux-x86_64-cpython-312/backports
102.8       copying src/backports/__init__.py -> build/lib.linux-x86_64-cpython-312/backports
102.8       creating build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/_common.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/_version.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/_zoneinfo.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/__init__.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/_tzpath.py -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       running egg_info
102.8       writing src/backports.zoneinfo.egg-info/PKG-INFO
102.8       writing dependency_links to src/backports.zoneinfo.egg-info/dependency_links.txt
102.8       writing requirements to src/backports.zoneinfo.egg-info/requires.txt
102.8       writing top-level names to src/backports.zoneinfo.egg-info/top_level.txt
102.8       reading manifest file 'src/backports.zoneinfo.egg-info/SOURCES.txt'
102.8       reading manifest template 'MANIFEST.in'
102.8       warning: no files found matching '*.png' under directory 'docs'
102.8       warning: no files found matching '*.svg' under directory 'docs'
102.8       no previously-included directories found matching 'docs/_build'
102.8       no previously-included directories found matching 'docs/_output'
102.8       adding license file 'LICENSE'
102.8       adding license file 'licenses/LICENSE_APACHE'
102.8       writing manifest file 'src/backports.zoneinfo.egg-info/SOURCES.txt'
102.8       copying src/backports/zoneinfo/__init__.pyi -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       copying src/backports/zoneinfo/py.typed -> build/lib.linux-x86_64-cpython-312/backports/zoneinfo
102.8       running build_ext
102.8       building 'backports.zoneinfo._czoneinfo' extension
102.8       creating build/temp.linux-x86_64-cpython-312
102.8       creating build/temp.linux-x86_64-cpython-312/lib
102.8       gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/openedx/venv/include -I/opt/pyenv/versions/3.12.2/include/python3.12 -c lib/zoneinfo_module.c -o build/temp.linux-x86_64-cpython-312/lib/zoneinfo_module.o -std=c99
102.8       lib/zoneinfo_module.c: In function ‘zoneinfo_fromutc’:
102.8       lib/zoneinfo_module.c:600:19: error: ‘_PyLong_One’ undeclared (first use in this function); did you mean ‘_PyLong_New’?
102.8         600 |             one = _PyLong_One;
102.8             |                   ^~~~~~~~~~~
102.8             |                   _PyLong_New
102.8       lib/zoneinfo_module.c:600:19: note: each undeclared identifier is reported only once for each function it appears in
102.8       error: command '/usr/bin/gcc' failed with exit code 1
102.8       [end of output]
102.8
102.8   note: This error originates from a subprocess, and is likely not a problem with pip.
102.8   ERROR: Failed building wheel for backports-zoneinfo
102.8 Failed to build backports-zoneinfo
102.8 ERROR: Could not build wheels for backports-zoneinfo, which is required to install pyproject.toml-based projects
------
Dockerfile:83
--------------------
  81 |
  82 |     # python requirements
  83 | >>> RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared,uid=${APP_USER_ID} pip install -r requirements.txt
  84 |     # https://pypi.org/project/uWSGI/
  85 |     RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared,uid=${APP_USER_ID} pip install uwsgi==2.0.24
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install -r requirements.txt" did not complete successfully: exit code: 1
@andrey-canon
Copy link

Hi @DawoudSheraz, @feanil I will make a review of this next week :D

@johanseto
Copy link

johanseto commented Apr 29, 2024

Hi everyone I was testing this PR, and I found some problems related to the tutor image creation. In this PR you are increasing Celery and that makes incompatible this line in the tutor ecommerce-worker image.
https://github.com/overhangio/tutor-ecommerce/blob/f145bab7f3923cbcd765100c7e80becef5dbae2a/tutorecommerce/templates/ecommerce/build/ecommerce-worker/Dockerfile#L29
2024-04-29_17-34
2024-04-29_18-08
2024-04-29_18-07

After I changed the command I could use the celery v5.4.0 in the worker.
2024-04-29_17-58
2024-04-29_17-57

image

Finally, I think this PR solves the problem.
https://github.com/overhangio/tutor-ecommerce/pull/72/files#diff-774fa1744f2df2072567ed4736a42e704fd687c9b15f001d93c52b97756a5f0aR42

johanseto added a commit to eduNEXT/tutor-ecommerce that referenced this pull request Apr 29, 2024
This is a proposal in order to allow ecommerce-worker increase celery to v5.4.7.

This upgrade of celery in the following PR: openedx/ecommerce-worker#247
johanseto added a commit to eduNEXT/tutor-ecommerce that referenced this pull request Apr 29, 2024
This is a proposal in order to allow ecommerce-worker increase celery to v5.4.0

This upgrade of celery in the following PR: openedx/ecommerce-worker#247
johanseto added a commit to eduNEXT/tutor-ecommerce that referenced this pull request Apr 29, 2024
This is a proposal in order to allow ecommerce-worker increase celery to v5.4.0

This upgrade of celery in the following PR: openedx/ecommerce-worker#247
@DawoudSheraz
Copy link

@johanseto Hi. Yes, the PR to upgrade tutor-ecommerce plugin has been up for a while. It would be merged once this and openedx/ecommerce related PRs are merged.

@feanil feanil merged commit bcdc416 into openedx:master Apr 30, 2024
4 checks passed
@feanil
Copy link
Contributor

feanil commented Apr 30, 2024

I updated the repo to not require the codecov checks and removed the rev squad as a required reviewer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants