-
Notifications
You must be signed in to change notification settings - Fork 183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cleaning up unused py2 tools ; restoring tensorflow as part of python_tools.spec #7019
Conversation
A new Pull Request was created by @davidlange6 (David Lange) for branch IB/CMSSW_12_0_X/devel. @cmsbuild, @smuzaffar, @mrodozov, @iarspider can you please review it and eventually sign? Thanks. |
test parameters:
|
please test |
-1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ab08a/15916/summary.html External BuildI found compilation warning when building: See details on the summary page. |
Seems that I forgot to commit a file worth of changes. Will fix this afternoon
On Jun 12, 2021 9:42 AM, cmsbuild ***@***.***> wrote:
…-1
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ab08a/15916/summary.html
COMMIT: 2d7cd22<2d7cd22>
CMSSW: CMSSW_12_0_DEVEL_X_2021-06-11-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7019/15916/install.sh to create a dev area with all the needed externals and cmssw changes.
External Build
I found compilation warning when building: See details on the summary page.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#7019 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABGPFQ6RF5YHETHXK6SHEODTSMFSZANCNFSM46QSA5UQ>.
|
Pull request #7019 was updated. |
please test |
-1 Failed Tests: UnitTests The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: Unit TestsI found errors in the following unit tests: ---> test test_PrepareInputDb had ERRORS ---> test test_CreateFileLists had ERRORS ---> test test_MpsWorkFlow had ERRORS ---> test EcalDAQ_O2O_test had ERRORS and more ... Comparison SummarySummary:
|
Pull request #7019 was updated. |
please test |
please test with cms-sw/cmssw#34106 |
please test with cms-sw/cmssw#34106 for CMSSW_12_0_DEVEL_X |
-1 Failed Tests: UnitTests The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: Unit TestsI found errors in the following unit tests: ---> test GeometryDOMCount had ERRORS ---> test testTauEmbeddingProducers had ERRORS Comparison SummarySummary:
|
-1 Failed Tests: UnitTests The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: Unit TestsI found errors in the following unit tests: ---> test TestFWCorePythonFrameworkDriver had ERRORS ---> test GeometryDOMCount had ERRORS ---> test materialBudgetHGCalPlots had ERRORS ---> test materialBudgetTrackerPlots had ERRORS and more ... Comparison SummarySummary:
|
I believe these failures are all in the mainstream IB or expected in DEVEL - @smuzaffar ?
… On Jun 15, 2021, at 12:12 AM, cmsbuild ***@***.***> wrote:
-1
Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ab08a/15939/summary.html
COMMIT: 07e8776
CMSSW: CMSSW_12_0_DEVEL_X_2021-06-13-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7019/15939/install.sh to create a dev area with all the needed externals and cmssw changes.
The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
• #7021 @cms-sw: [BuildRules] use python3 in scripts
• #7025 @cms-sw: updated DetectorDescription-Schema=V02-03-00
• #7024 @belforte: move tested dev to prod
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ab08a/15939/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ab08a/15939/git-merge-result
Unit Tests
I found errors in the following unit tests:
---> test TestFWCorePythonFrameworkDriver had ERRORS
---> test GeometryDOMCount had ERRORS
---> test materialBudgetHGCalPlots had ERRORS
---> test materialBudgetTrackerPlots had ERRORS
and more ...
Comparison Summary
Summary:
• No significant changes to the logs found
• Reco comparison results: 5123 differences found in the comparisons
• DQMHistoTests: Total files compared: 38
• DQMHistoTests: Total histograms compared: 2862520
• DQMHistoTests: Total failures: 7
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2862491
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
• Checked 160 log files, 37 edm output root files, 38 DQM output files
• TriggerResults: no differences found
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
that is correct @davidlange6. What about comparison differences, do you think those are due to python3 used by cmsRun? |
The 7 dqm histogram failures are all message logger - so “normal”.
I’m puzzled how to have reco comparison differences without dqm histogram comparison differences. So, ok, something we have no DQM monitoring of - slimmedJetsPuppi..
The test done with the mainstream build did not have the comparison differences. How can I find what the baseline was for the reco comparisons for the build starting from the DEVEL branch? Was it not a recent DEVEL IB?
… On Jun 15, 2021, at 8:27 AM, Malik Shahzad Muzaffar ***@***.***> wrote:
that is correct @davidlange6. What about comparison differences, do you think those are due to python3 used by cmsRun?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
you can click on the "Details" link for |
Ok, thanks - so that is presumably a python3 difference (feature or bug - to be understood..). I will investigate that. But thats independent of this PR.
… On Jun 15, 2021, at 8:47 AM, Malik Shahzad Muzaffar ***@***.***> wrote:
How can I find what the baseline was for the reco comparisons for the build starting from the DEVEL branch
you can click on the "Details" link for cms/7019/DEVEL/slc7_amd64_gcc900/comparison status which should take you to comparison jenkins job ( in this case https://cmssdt.cern.ch/jenkins/job/compare-root-files-short-matrix/43455/ ) . As we ony generate baseline for 12.0.X so comparison is always done with normal 12.0.X IB.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
the puppi differences are due to different order of things the python config. Eg, 245924c245933
|
lets get this in for DEVEL |
@davidlange6 , 3 of DEVEL IBs unit tests can be fixed if they use
should we just fix this in CMSSW_DEVEL branch or define soem env e.g CMSRUN_PYTHON= |
I think the two in Validation can be changed also in mainstream. The one in FWCore needs to be consistent with the fwk python version- I can make a PR for the two in Validation.
… On Jun 16, 2021, at 9:23 AM, Malik Shahzad Muzaffar ***@***.***> wrote:
@davidlange6 , 3 of DEVEL IBs unit tests can be fixed if they use python3 e.g.
• https://github.com/cms-sw/cmssw/blob/master/FWCore/PythonFramework/test/runPythonTests.sh#L10-L15
• https://github.com/cms-sw/cmssw/blob/master/Validation/Geometry/test/genHGCalPlots.sh#L12
• https://github.com/cms-sw/cmssw/blob/master/Validation/Geometry/test/genTrackerPlots.sh#L11-L16
should we just fix this in CMSSW_DEVEL branch or define soem env e.g CMSRUN_PYTHON=python|python3 and use this in these scripts?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
Sorry, I’m wrong. The two in Validation use the FWCore/PythonFramework so have the same issue that the python version needs to make the python version of the fwk. (Which I don’t think we have an easy way to discover)
… On Jun 16, 2021, at 9:27 AM, David Lange ***@***.***> wrote:
I think the two in Validation can be changed also in mainstream. The one in FWCore needs to be consistent with the fwk python version- I can make a PR for the two in Validation.
> On Jun 16, 2021, at 9:23 AM, Malik Shahzad Muzaffar ***@***.***> wrote:
>
>
> @davidlange6 , 3 of DEVEL IBs unit tests can be fixed if they use python3 e.g.
>
> • https://github.com/cms-sw/cmssw/blob/master/FWCore/PythonFramework/test/runPythonTests.sh#L10-L15
> • https://github.com/cms-sw/cmssw/blob/master/Validation/Geometry/test/genHGCalPlots.sh#L12
> • https://github.com/cms-sw/cmssw/blob/master/Validation/Geometry/test/genTrackerPlots.sh#L11-L16
> should we just fix this in CMSSW_DEVEL branch or define soem env e.g CMSRUN_PYTHON=python|python3 and use this in these scripts?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.
>
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
you can try but I think the tests in Validation should also use same python as used by FWK |
can we do something like the following to find the FWK python?
or add a helper script cms-python which just runs the above and return either |
Yes, I think so - now is the limited lifetime of this script worthwhile to introduce? (Or maybe there is a longer term use case?)
But, eg, this works in a bash script - we could add that to the three unit tests
```
#!/bin/bash
{
python3 -c "from FWCore.PythonFramework.CmsRun import CmsRun" 2>/dev/null &&echo "using python3"
}||
{
python -c "from FWCore.PythonFramework.CmsRun import CmsRun" 2>/dev/null && echo "using python2"
}
```
… On Jun 16, 2021, at 9:38 AM, Malik Shahzad Muzaffar ***@***.***> wrote:
can we do something like the following to find the FWK python?
ldd `which cmsRun` | grep libpython
or add a helper script cms-python which just runs the above and return either python2 or python3
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
Ido not see any long term use case, so fixing these 3 sciprts should be enough. By the way, you want to use |
Ok, lets see if I can fix all three tests
… On Jun 16, 2021, at 10:21 AM, Malik Shahzad Muzaffar ***@***.***> wrote:
Ido not see any long term use case, so fixing these 3 sciprts should be enough. By the way, you want to use python2 instead of python asin PY3 IBs, python is python3 :-)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
(For DEVEL)