Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Big performance difference when notebooks are run by pytest+nbval versus nbconvert #80

Open
tlvu opened this issue Jul 7, 2021 · 1 comment
Open

Big performance difference when notebooks are run by pytest+nbval versus nbconvert #80

tlvu opened this issue Jul 7, 2021 · 1 comment

Comments

@tlvu
Copy link
Contributor

tlvu commented Jul 7, 2021

When the entire testsuite is run by pytest+nbval, it completes in 6m07sec:

00:34:28  ============================= test session starts ==============================
00:34:28  platform linux -- Python 3.7.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
00:34:28  rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
00:34:28  plugins: anyio-3.1.0, dash-1.20.0, nbval-0.9.6, tornasync-0.6.0.post2
00:34:28  collected 172 items
00:34:28  
00:34:31  notebooks/hummingbird.ipynb ............                                 [  6%]
00:35:20  pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 11%]
00:35:26  pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 14%]
00:35:34  pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 19%]
00:35:36  pavics-sdi-master/docs/source/notebooks/WPS_example.ipynb ..........     [ 25%]
00:35:51  pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .                 [ 25%]
00:35:52  pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 26%]
00:35:55  pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 30%]
00:36:00  pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 33%]
00:36:19  pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 41%]
00:37:12  ...............                                                          [ 50%]
00:37:16  pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 52%]
00:37:23  pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 56%]
00:37:36  finch-master/docs/source/notebooks/dap_subset.ipynb ..........           [ 62%]
00:37:45  finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 65%]
00:38:25  finch-master/docs/source/notebooks/subset.ipynb ....................     [ 77%]
00:38:26  PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 77%]
00:38:29  ......                                                                   [ 81%]
00:38:47  PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
00:38:58  .............                                                            [ 89%]
00:39:07  PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 90%]
00:39:43  ....s.                                                                   [ 93%]
00:39:49  PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 94%]
00:39:55  ...                                                                      [ 95%]
00:40:07  PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 96%]
00:40:35  ......                                                                   [100%]
00:40:35  
00:40:35  ================== 171 passed, 1 skipped in 367.14s (0:06:07) ==================

But when re-run under nbconvert, one notebook hits the 4 minutes timeout with some Not authorized to access this resource. Missing user authentication error preceding the timeout, not clear if they are related.

00:45:02  + jupyter nbconvert --to notebook --execute --ExecutePreprocessor.timeout=240 --allow-errors --output-dir buildout --output PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.output.ipynb PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb
00:45:02  [NbConvertApp] Converting notebook PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb to notebook
00:45:02  [NbConvertApp] Executing notebook with kernel: birdy
00:45:34  syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
00:45:34  context: <?xml^ version="1.0" encoding="utf-8"?><ExceptionReport version="1.0.0" xmlns="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd"> <Exception exceptionCode="NoApplicableCode" locator="AccessForbidden"> <ExceptionText>Not authorized to access this resource. Missing user authentication.</ExceptionText> </Exception></ExceptionReport>
00:45:34  syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
00:45:34  context: <?xml^ version="1.0" encoding="utf-8"?><ExceptionReport version="1.0.0" xmlns="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd"> <Exception exceptionCode="NoApplicableCode" locator="AccessForbidden"> <ExceptionText>Not authorized to access this resource. Missing user authentication.</ExceptionText> </Exception></ExceptionReport>
00:49:26  syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
00:49:26  context: <?xml^ version="1.0" encoding="utf-8"?><ExceptionReport version="1.0.0" xmlns="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd"> <Exception exceptionCode="NoApplicableCode" locator="AccessForbidden"> <ExceptionText>Not authorized to access this resource. Missing user authentication.</ExceptionText> </Exception></ExceptionReport>
00:49:26  syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
00:49:26  context: <?xml^ version="1.0" encoding="utf-8"?><ExceptionReport version="1.0.0" xmlns="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd"> <Exception exceptionCode="NoApplicableCode" locator="AccessForbidden"> <ExceptionText>Not authorized to access this resource. Missing user authentication.</ExceptionText> </Exception></ExceptionReport>
00:49:48  [NbConvertApp] ERROR | Timeout waiting for execute reply (240s).
00:49:51  /opt/conda/envs/birdy/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
00:49:51    len(cache))
00:49:51  /opt/conda/envs/birdy/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
00:49:51    len(cache))
00:49:51  /opt/conda/envs/birdy/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
00:49:51    len(cache))
00:49:51  Traceback (most recent call last):
00:49:51    File "/opt/conda/envs/birdy/bin/jupyter-nbconvert", line 8, in <module>
00:49:51      sys.exit(main())
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/jupyter_core/application.py", line 254, in launch_instance
00:49:51      return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/traitlets/config/application.py", line 845, in launch_instance
00:49:51      app.start()
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 340, in start
00:49:51      self.convert_notebooks()
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 510, in convert_notebooks
00:49:51      self.convert_single_notebook(notebook_filename)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 481, in convert_single_notebook
00:49:51      output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 410, in export_single_notebook
00:49:51      output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 179, in from_filename
00:49:51      return self.from_file(f, resources=resources, **kw)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 197, in from_file
00:49:51      return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/exporters/notebook.py", line 32, in from_notebook_node
00:49:51      nb_copy, resources = super(NotebookExporter, self).from_notebook_node(nb, resources, **kw)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 139, in from_notebook_node
00:49:51      nb_copy, resources = self._preprocess(nb_copy, resources)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 316, in _preprocess
00:49:51      nbc, resc = preprocessor(nbc, resc)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 47, in __call__
00:49:51      return self.preprocess(nb, resources)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 405, in preprocess
00:49:51      nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
00:49:51      nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 438, in preprocess_cell
00:49:51      reply, outputs = self.run_cell(cell, cell_index, store_history)
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 571, in run_cell
00:49:51      if self._passed_deadline(deadline):
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 541, in _passed_deadline
00:49:51      self._handle_timeout()
00:49:51    File "/opt/conda/envs/birdy/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 504, in _handle_timeout
00:49:51      raise TimeoutError("Cell execution timed out")
00:49:51  TimeoutError: Cell execution timed out

See http://jenkins.ouranos.ca/job/PAVICS-e2e-workflow-tests/job/master/1128/console
@tlvu
Copy link
Contributor Author

tlvu commented Jul 7, 2021

@matprov FYI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@tlvu