Fix Automatic Travis-CI PyPi Deployment

[kb1lqc]

Per https://travis-ci.org/FaradayRF/faradayio/builds/336006288 the automatic Travis-CI pypi deployment is not working.

Deploying application
running sdist
running check
warning: sdist: standard file not found: should have one of README, README.txt
reading manifest template 'MANIFEST.in'
warning: no files found matching 'LICENSE.txt'
writing manifest file 'MANIFEST'
creating faradayio-0.0.1
creating faradayio-0.0.1/docs
creating faradayio-0.0.1/docs/images
creating faradayio-0.0.1/faradayio
making hard links in faradayio-0.0.1...
hard linking setup.py -> faradayio-0.0.1
hard linking docs/Makefile -> faradayio-0.0.1/docs
hard linking docs/code.rst -> faradayio-0.0.1/docs
hard linking docs/conf.py -> faradayio-0.0.1/docs
hard linking docs/index.rst -> faradayio-0.0.1/docs
hard linking docs/make.bat -> faradayio-0.0.1/docs
hard linking docs/images/Faraday_Logo_Square.png -> faradayio-0.0.1/docs/images
hard linking docs/images/Faraday_Logo_Square_50px.jpg -> faradayio-0.0.1/docs/images
hard linking faradayio/__init__.py -> faradayio-0.0.1/faradayio
hard linking faradayio/faraday.py -> faradayio-0.0.1/faradayio
creating dist
Creating tar archive
removing 'faradayio-0.0.1' (and everything under it)
Uploading distributions to https://upload.pypi.org/legacy/
Uploading faradayio-0.0.1.tar.gz
HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://upload.pypi.org/legacy/
faradayio/__pycache__/__init__.cpython-35.pyc already exists, no checkout
faradayio/__pycache__/faraday.cpython-35.pyc already exists, no checkout
tests/__pycache__/__init__.cpython-35.pyc already exists, no checkout
tests/__pycache__/serialtestclass.cpython-35.pyc already exists, no checkout
tests/__pycache__/test_serial.cpython-35-PYTEST.pyc already exists, no checkout
tests/__pycache__/test_tun.cpython-35-PYTEST.pyc already exists, no checkout
Could not restore untracked files from stash entry
PyPI upload failed.
failed to deploy

[kb1lqc]

Per travis-ci/dpl#377 it appears that special characters may cause the automatic build to fail. Let's test this.

[kb1lqc]

Hmm it's right there on https://docs.travis-ci.com/user/deployment/pypi/:

[kb1lqc]

Closer but still failed:
https://travis-ci.org/kb1lqc/faradayio/builds/337116230

invalid option "--password="
failed to deploy

[kb1lqc]

Man I am stuck. I can't figure this out even with the help of:

• pypi deployment error: invalid option "--password=" travis-ci/dpl#327
• Add encrypted password of PyPI jazzband to sorl-thumbnail jazzband/help#69

Tried to add the --org option and that didn't even work

Current travis.yml

sudo: required
dist: trusty
language: python
python:
- '3.5'
install:
- "~/virtualenv/python3.5/bin/pip3 install -r requirements.txt"
script:
- "~/virtualenv/python3.5/bin/flake8 --exclude docs/ ."
- sudo ~/virtualenv/python3.5/bin/pytest tests/
- sudo ~/virtualenv/python3.5/bin/python3 -m coverage run ~/virtualenv/python3.5/bin/pytest
- coveralls
deploy:
  provider: pypi
  user: KB1LQC
  on:
    branch: issue35
  PASSWORD:
    secure: "Y8OAStuhW4t4+31eAs0ua7qfVxsB+AeWBhziOiW6Ypv7+usuVFk8ShK9GywrAjNSdcz3Zxb+5CUyaxOw5pieQT3Pxepha531naYVGyDQThKRKQkdL2UKMen0HMvDPSn9AJRzHObA/ZjGs4gfpuFoHTK1joPjMBbiMiIPi5h2+nZv1Hqt+avLU96Tc7O0mvgZggReo1T72nX38O08BafqnXADHnGBrKdKhv/g7n6w+JzycuTVUg2bDprSrPHmFSQhLdc5h5073TS8pGPG6RwCYuhwFIgBKhsJ7eKLsoNLNc5yEJVtJD10nmmNNE5ooNkdBeDzYwmIAqQZV7Y62MEsrQJ3dzHOANq3ceHo4cWb3HxA+hksRu9UPHEH/tmqhxcSOGCQbVtkkELIuEvq0mb1y7FoUsKrhQDty/1/wUhrehk6/PjX9C1Vu8tb7wIDhkIuWSrdLvfGID0HrcxMN63wigUKpRhlpMFbuoOrExj404X6JAv1tLZRLuVKtI+h/+ZYuo10KsDupP7ZU9oc5872A3C8o43WscdJJehpHJ+HcgHczVBLupbqjf+BLbxikpKIY1iQqP9/ylI1nwerNRQAGrssQJkWUbm+QjC0X+NvIlZnAiDDlo9tF9+C1Ndw17irHGNBSEY2H83pc897APItqBIQDn+m0TQ9ClhYXW5Q/kI="

Any idea @avelis?

[avelis]

@kb1lqc This is likely a TravisCI config yaml issue. I suggest you contact their support via email and link your travis.yml file. Without seeing any error output it's hard to tell. If I had to guess, try lower casing PASSWORD to password but I really suggest you contact TravisCI support.

[kb1lqc]

Thanks @avelis I have contacted Travis-CI support via email. I appreciate the help!

[kb1lqc]

Received a response from Travis-CI. Here is the excerpt that matters:

To confirm this further, could you try to re-encrypt your password using the following recipe:

0) travis logout --org
1) travis login --org
2) cd path/to/FaradayRF/faradayio
3) travis encrypt <your_pypi_password>

Finally, take note that the value must be encrypted by someone having access to the FaradayRF/faradayio repo itself and must not be coming from a fork. The reason is that both don't use the same encryption keys.

Let's give this a shot.

[kb1lqc]

Checked out FaradayRF/faradayio branch develop directly to make sure I am not using a fork. This is my understanding of the final note from Travis CI support.

[kb1lqc]

Created a brand new virtual environment and installed the requirements.txt file with pip3 from the virtual environment directly.

[kb1lqc]

Performed

(.venv) bryce@bryce-ubuntu:~/Documents/git/FaradayRFfaradayio$ travis logout --org
Successfully logged out!

[kb1lqc]

Signed into Travis CI with my GitHub account via command line using Two-Factor Authentication (2FA) because everyone should!

(.venv) bryce@bryce-ubuntu:~/Documents/git/FaradayRFfaradayio$ travis login --org
We need your GitHub login to identify you.
This information will not be sent to Travis CI, only to api.github.com.
The password will not be displayed.

Try running with --github-token or --auto if you don't want to enter your password anyway.

Username: KB1LQC
Password for KB1LQC: **********
Two-factor authentication code for KB1LQC: 772718
Successfully logged in as kb1lqc!

[kb1lqc]

I'm already in the FaradayRF/faradayio repository folder

(.venv) bryce@bryce-ubuntu:~/Documents/git/FaradayRFfaradayio$ pwd
/home/bryce/Documents/git/FaradayRFfaradayio
(.venv) bryce@bryce-ubuntu:~/Documents/git/FaradayRFfaradayio$ git status
On branch develop
Your branch is up-to-date with 'origin/develop'.
nothing to commit, working directory clean

[kb1lqc]

I followed the Travis CI support instructions exactly except I added the --add command so no copy-paste errors occured:

(.venv) bryce@bryce-ubuntu:~/Documents/git/FaradayRFfaradayio$ travis encrypt <redacted> --add
Environment variables in env.global should be formatted as FOO=bar

It appears that the Travis CI support instructions may have forgot to include password= when encrypting?

[kb1lqc]

OK trying exactly as told:

sudo: required
dist: trusty
language: python
python:
- '3.5'
install:
- "~/virtualenv/python3.5/bin/pip3 install -r requirements.txt"
script:
- "~/virtualenv/python3.5/bin/flake8 --exclude docs/ ."
- sudo ~/virtualenv/python3.5/bin/pytest tests/
- sudo ~/virtualenv/python3.5/bin/python3 -m coverage run ~/virtualenv/python3.5/bin/pytest
- coveralls
deploy:
  on:
    branch: develop
  provider: pypi
  user: KB1LQC
env:
  global:
    secure: dVCpE8uHXR6MG3ldaBzHdIXBATuSeXJuj8dzp9XkqwOgK7T3W/w4JaLGyyise6FOA4TwuN/i3aIfuHZMw5LE9oCETKNUUKclgvRtYAglWMv1Xv77JZtOl/woXt6pJoI2rCTXdWWyx0gSG8NsjoowB9hqrC8hsSSELsigUUbyRZ5ufjyLl8YTE/RI+aXd9lPGjjte+duuQJNAbqye8tfyaKFHF8CZeoEIXmaLVLEQ2PxArYXemcADaR9fwGAXRB1XgVJJmbIV0bdXz/Wrcn2B1HC9xHub/kYhNo+IjryrMm+CCaPWXBlZaXrm3gI5IArXyXr6KjZNR3SKC9vS6wLR3Kx/lJ4jTX7MMXoZoe/XGImKZiU5AwJ4+AdI+rhJNgoU480m0CoxAOnugGES5IPqL2NVcSjcVAVHtPkOcS5TtkxGKesoYS4M0ST3DOH8x05zCwTvJd4ELj0NMDkFpfjn/P80rtw2r0pScCyw/Z39sJDuGHEaGdZUH0RTpHkxADjgruObxu0vW9hrMiLpKGadnweD+EYcGUaVMP3XA9vZLwebmCtud9BQN7+IBdIuakBPa4x1G3rKBschaRh1VpC6EH6N0wn53+HXcm/nTjmbN2+UifXDzJoB29EIQT8xFQ9lx2DYXMHx8nD5LW/pMH2tEp+eDZw1oclvsn6gY6WFMIo=

OK after adding the deploy branch to the travis.yml file build https://travis-ci.org/FaradayRF/faradayio/builds/341300537 failed.

$ rvm $(travis_internal_ruby) --fuzzy do ruby -S gem install dpl
Fetching: dpl-1.8.47.gem (100%)
Successfully installed dpl-1.8.47
1 gem installed
dpl.1
Installing deploy dependencies
--2018-02-14 06:28:44--  https://bootstrap.pypa.io/get-pip.py
Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 151.101.0.175, 151.101.64.175, 151.101.128.175, ...
Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|151.101.0.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1595408 (1.5M) [text/x-python]
Saving to: ‘STDOUT’
100%[======================================>] 1,595,408   9.56MB/s   in 0.2s   
2018-02-14 06:28:45 (9.56 MB/s) - written to stdout [1595408/1595408]
Requirement already up-to-date: pip in /home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages
Requirement already up-to-date: setuptools in /home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages
Collecting twine
  Downloading twine-1.9.1-py2.py3-none-any.whl
Requirement already up-to-date: wheel in /home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages
Collecting tqdm>=4.11 (from twine)
  Downloading tqdm-4.19.5-py2.py3-none-any.whl (51kB)
    100% |████████████████████████████████| 61kB 4.9MB/s 
Collecting pkginfo>=1.0 (from twine)
  Downloading pkginfo-1.4.1-py2.py3-none-any.whl
Collecting requests-toolbelt>=0.8.0 (from twine)
  Downloading requests_toolbelt-0.8.0-py2.py3-none-any.whl (54kB)
    100% |████████████████████████████████| 61kB 5.0MB/s 
Requirement already up-to-date: requests!=2.15,!=2.16,>=2.5.0 in /home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages (from twine)
Requirement already up-to-date: urllib3<1.23,>=1.21.1 in /home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages (from requests!=2.15,!=2.16,>=2.5.0->twine)
Requirement already up-to-date: certifi>=2017.4.17 in /home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages (from requests!=2.15,!=2.16,>=2.5.0->twine)
Requirement already up-to-date: chardet<3.1.0,>=3.0.2 in /home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages (from requests!=2.15,!=2.16,>=2.5.0->twine)
Requirement already up-to-date: idna<2.7,>=2.5 in /home/travis/virtualenv/python3.5.5/lib/python3.5/site-packages (from requests!=2.15,!=2.16,>=2.5.0->twine)
Installing collected packages: tqdm, pkginfo, requests-toolbelt, twine
Successfully installed pkginfo-1.4.1 requests-toolbelt-0.8.0 tqdm-4.19.5 twine-1.9.1
dpl.2
Preparing deploy
No stash entries found.
missing PyPI password
failed to deploy

[kb1lqc]

Trying again with PASSWORD=password instead.

(.venv) bryce@bryce-ubuntu:~/Documents/git/FaradayRFfaradayio$ travis encrypt PASSWORD=<redacted> --add

The resulting travis.yml is

sudo: required
dist: trusty
language: python
python:
- '3.5'
install:
- "~/virtualenv/python3.5/bin/pip3 install -r requirements.txt"
script:
- "~/virtualenv/python3.5/bin/flake8 --exclude docs/ ."
- sudo ~/virtualenv/python3.5/bin/pytest tests/
- sudo ~/virtualenv/python3.5/bin/python3 -m coverage run ~/virtualenv/python3.5/bin/pytest
- coveralls
deploy:
  on:
    branch: develop
  provider: pypi
  user: KB1LQC
env:
  global:
    secure: XoYt36eXJIPAz6rD7lkURnOVEbHN315SGwZetxyUkfGurEfoXH0vxSOuOZM/agve1h/uyExpQA6QiiPiXD4lYEOyheHCR+zvDwsZQdzV2YvpwahSntZRVx8gAqQKd4HhoQ2i3T24twYVhk5qdjFGSnebqZhKJNzAbsdhVOuMMVfLtu5h7TL7ZoWq8cypPXaKJxTd3QRj5Xoztt7s75Nip4XPKqhxwDyPmLSuAv8/wdpzyz0I21HHEJhEG4RqLeuzDuiJ7NL5b2eE7Ha4T0GG8tvloFGEd6/nAwfWYX9ZzcxP3/aYvYvihHjxLy4j6VRN6OrqYc4sEAQS+rBd1WKNXEpcCEMbNElbIMXHZqoUdD7H0N6UQ0KetoL5ygj2h2XD8w8IsGG48nyccuwdkasR5lGHk8XCgicyldBKEP0pHQV4mmLsPxdAFg+1CYn0I0rR0ycHi7yqrjb+P2w/luZ7GIeCENeMEFLdYChS+jMMzdSZ4xjCABGpQO4b9TiheYrt9YnP9wCMIvkec2Clys6EN3Q7WRK+LRQ3aZ9rDcDTXfUYAB+2eZQuOEKKXmo6cyWuIEZeDjP0da/oEMpsA4EzJktrMoFxdzPthFEOZqU0EvOwyugERGPkqBsTYDKoZePycJ3ykdJs3L7g2GcgVaVDi+u01Dvo2DK9FxE6hQViDck=

This resulted in https://travis-ci.org/FaradayRF/faradayio/builds/341302134 which failed to deploy

No stash entries found.
missing PyPI password
failed to deploy

[kb1lqc]

I just changed my password to a similar but different password. For reference, here is the password that I have been using to prove it contains no special characters per travis-ci/dpl#327

b852e6561e89fd0017351af89f0f788f

So far no luck 😦

[lqdev]

So this might sound dumb, but could it have something to do with the indentations? I've run into issues where indentations in yml completely crash a program/process.

[kb1lqc]

That's actually not a crazy idea haha 😉

…

---- On Wed, 14 Feb 2018 05:23:02 -0800 [email protected] wrote ---- So this might sound dumb, but could it have something to do with the indentations? I've run into issues where indentations in yml completely crash a program/process. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[avelis]

@kb1lqc Not to ask a naive question but should the secure setting sit outside the deploy area in the config? Is that what TravisCI suggested to do?

[kb1lqc]

Good point! -----Original Message----- From: Andrew Velis <[email protected]> Reply-to: FaradayRF/faradayio <reply+00050f14ce78b7cf29574bd4738f504501 851d7676884b8392cf00000001169c3ace92a169ce117dc0f8@reply.github.com> To: FaradayRF/faradayio <[email protected]> Cc: Bryce Salmi <[email protected]>, Mention <[email protected]. com> Subject: Re: [FaradayRF/faradayio] Fix Automatic Travis-CI PyPi Deployment (#35) Date: Wed, 14 Feb 2018 17:58:40 +0000 (UTC) @kb1lqc Not to ask a naive question but should the secure setting sit outside the deploy area in the config? Is that what TravisCI suggested to do? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[kb1lqc]

@avelis @lqdev I attempted a few iterations of indents, I also ensured the secure key was indented per documentation. This did not work.

Last I attempted to use the following travis.yml with an --env variable per https://docs.travis-ci.com/user/encryption-keys#Usage:

sudo: required
dist: trusty
language: python
python:
- '3.5'
install:
- "~/virtualenv/python3.5/bin/pip3 install -r requirements.txt"
script:
- "~/virtualenv/python3.5/bin/flake8 --exclude docs/ ."
- sudo ~/virtualenv/python3.5/bin/pytest tests/
- sudo ~/virtualenv/python3.5/bin/python3 -m coverage run ~/virtualenv/python3.5/bin/pytest
- coveralls
deploy:
  on:
    branch: develop
  provider: pypi
  user: KB1LQC
  env:
    secure: q7xbp1N2ZhfAdHw76ztzmOE7e2/ttFtngT9IUe5BF+VJGPql9u5O+DyxnTcc2bWtU7a3Xgda8itK/hjdAg5LZySFxAbSy6+BEBKefbk9YUgQcDqVl54zTE7pO9l8lcCc3Fqn6Pkrd3KsyEtQLE3pRAPolxZSDJn8sWC7x+RYxC2z8SKau/PP5cn5RA2t9t5Ip0JDujQczuK0sljdyD5WUjPPImBOPxKnwdfpD1R0g9uYJmLaw7uBMhfdDhbaWZBLT79iMACGbMycmPpkhUgIJXcCk/dm15SksYNF/OLDZV8/MdkwoSq4Vw4b2kpv6PPH/VrSGp6JS3Yx+VGibWKKTIlLi0bo2oCjPZ0IIIxRH9NsIZJGd5DLxmamCIUF740+lZ+k6K/H/7jHmnD6orzLFc5t1JdURBOq/aMzGg594oXjrREZxWlN0hXJHl780TXHt2gjMdS3DMAiHdEXXP/5HNZWi2kwH9ZH05loePJoXsuOZQ+gjYvrrz35nR1iwMilDObPMtfmX8kYSlY1Kt6eedDluDwADk5R9iO2Nq9o29rYrDe7igEmKHyE38JK5EuIazunCqlQlTvmNdr9unj5U0vqrMN7jxMGEyQNTZYED62sf6Yth5EgwPkkI+qHYZJzphL7OuQ2T9p3dtgIVbRhu2llCqdWjmCiKcsvFTKAjCo=

Note the section of the #Usage text

Please note that the name of the environment variable and its value are both encoded in the string produced by “travis encrypt.” You must add the entry to your .travis.yml with key “secure” (underneath the “env” key). This makes the environment variable SOMEVAR with value “secretvalue” available to your program.

https://travis-ci.org/FaradayRF/faradayio/builds/341770932

This build is not working, it failed with:

invalid option "--env="

I'm pretty darn stuck on this... travis-ci/dpl#377 and travis-ci/dpl#327 do not appear to be relevant here since my password contains only letters and numbers (lowercase, uppercase, numerical)...

[cotsog]

@kb1lqc (Dominic from Travis CI Support Team here) Sorry for the confusion. In the recipe I sent you, I forgot to mention that the generated encrypted variable should be copy/pasted under password: in your deploy: section i.e.

deploy:
  on:
    branch: develop
  provider: pypi
  user: KB1LQC
  password:
    secure: q7xbp1N2...

You can have more details in our PyPI deployment documentation.

Please let me know if this helps!

[kb1lqc]

Removed from the https://github.com/FaradayRF/faradayio/milestone/2 milestone since this is a persistent issue and should not hold back the next version of faradayio.

[kb1lqc]

Ahh that master repo reference might be key here in

debbiemarkslab/EVcouplings#95 (comment)

[kb1lqc]

Per https://travis-ci.org/kb1lqc/faradayio/builds/346682269 this is now building without deploying. When merged with master this will now show a successful Travis CI build.

I'm going to figure this out later. I need to move on for the time being. I think @lqdev and @b-schubert were onto something. Likely the Travis encrypt from a non-forked repo as my password is completely alphanumeric.