New release: 0.0.45 → 0.0.46

* refactor to make the code clearer in some places
* don't call bin/qmk anymore
* ask the user if they want to install python modules when they're missing

.github/workflows/cli_setup.yml

@@ -14,7 +14,7 @@ jobs:
     strategy:
       matrix:
         os: [macos-latest, ubuntu-latest]
-        python-version: [3.6, 3.7, 3.8]
+        python-version: [3.7, 3.8, 3.9]
 
     steps:
     - uses: actions/checkout@v1
@@ -53,10 +53,9 @@ jobs:
         update: true
         install: git mingw-w64-x86_64-toolchain mingw-w64-x86_64-python-pip
 
-    - name: (MSYS2) Upgrade pip and setuptools and install wheel
+    - name: (MSYS2) Install Python dependencies
       shell: msys2 {0}
       run: |
-        python3 -m pip install --upgrade pip setuptools
         python3 -m pip install wheel
 
     - name: (MSYS2) Install QMK CLI from source

.github/workflows/python-publish.yml

@@ -0,0 +1,47 @@
+# This workflow will upload a Python Package using Twine when a release is created
+# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
+
+name: Upload Python Package
+
+on:
+  workflow_dispatch:
+    inputs:
+      version_part:
+        description: 'Which part of the version to increment (patch, minor, major)'
+        required: true
+        default: 'patch'
+
+jobs:
+  deploy:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.7'
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install setuptools wheel 
+        pip install -r requirements-dev.txt
+    - name: Bump version
+      run: |
+        git config --local user.email "hello@qmk.fm"
+        git config --local user.name "QMK Bot"
+        bumpversion ${{ github.event.inputs.version_part }}
+    - name: Push changes
+      uses: ad-m/github-push-action@master
+      with:
+        github_token: ${{ secrets.GITHUB_TOKEN }}
+        branch: master
+        tags: true
+    - name: Build and publish
+      env:
+        TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
+        TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+      run: |
+        python setup.py sdist bdist_wheel
+        twine upload dist/*

CODE_OF_CONDUCT.md

@@ -1,15 +0,0 @@
-# Code Of Conduct
-
-QMK strives to be an inclusive, tolerant, and welcoming community. We encourage participation from anyone regardless of age, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, political belief, race, religion, or sexual identity and orientation.
-
-> “A gentle word turns away wrath, but a harsh word stirs up anger."
-
-Our users, contributors, and collaborators are expected to treat each other with kindness and respect, to assume good intentions, and to gently correct, where possible, rather than react with escalation. While our goal is to be as accurate as possible, kindness and understanding are more valuable than correctness. Some examples of behavior we will not tolerate include, but is not limited to:
-
-* The use of sexualized language or imagery
-* Unwelcome advances, sexual or otherwise
-* Insults or derogatory comments, or personal or political attacks
-* Publishing others’ private information without explicit permission
-* Other conduct which could reasonably be considered inappropriate in a professional setting
-
-If someone is violating this Code of Conduct you may email hello@qmk.fm to bring your concern to the Members. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. 

README.md

@@ -1,5 +1,6 @@
 # QMK CLI
 [![CLI Setup](https://github.com/qmk/qmk_cli/workflows/CLI%20Setup/badge.svg)](https://github.com/qmk/qmk_cli/actions?query=workflow%3A%22CLI+Setup%22)  
+
 A program to help users work with [QMK Firmware](https://qmk.fm/).
 
 # Features
@@ -8,10 +9,17 @@ A program to help users work with [QMK Firmware](https://qmk.fm/).
 * Use `qmk clone` to pull down anyone's `qmk_firmware` fork
 * Setup your build environment with `qmk setup`
 * Check that your environment is correctly setup with `qmk doctor`
-* Dispatches to `qmk_firmware/bin/qmk` for additional functionality:
-    * `qmk compile-json`
-    * More to come
+* Integrates with your qmk_firmware for additional functionality:
+    * `qmk c2json`
+    * `qmk compile`
+    * `qmk flash`
+    * `qmk json2c`
+    * `qmk lint`
+    * `qmk new-keymap`
 
 # Quickstart
 
-To be written.
+* `python3 -m pip install qmk`
+* `qmk setup`
+
+Full documentation: <https://docs.qmk.fm/#/tutorial>

qmk_cli/__init__.py

@@ -1,3 +1,3 @@
 """A program to help you work with qmk_firmware."""
 
-__version__ = '0.0.38'
+__version__ = '0.0.46'

qmk_cli/git.py

@@ -16,17 +16,25 @@ def git_clone(url, destination, branch):
         '--recurse-submodules',
         '--branch=' + branch,
         url,
-        destination,
+        str(destination),
     ]
     cli.log.debug('Git clone command: %s', git_clone)
 
-    with subprocess.Popen(git_clone, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True, encoding='utf-8') as p:
-        for line in p.stdout:
-            print(line, end='')
+    try:
+        with subprocess.Popen(git_clone, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True, encoding='utf-8') as p:
+            for line in p.stdout:
+                print(line, end='')
+
+    except Exception as e:
+        git_cmd = ' '.join([s.replace(' ', r'\ ') for s in git_clone])
+
+        cli.log.error("Could not run '%s': %s: %s", git_cmd, e.__class__.__name__, e)
+        return False
 
     if p.returncode == 0:
         cli.log.info('Successfully cloned %s to %s!', url, destination)
         return True
+
     else:
         cli.log.error('git clone exited %d', p.returncode)
         return False

qmk_cli/helpers.py

@@ -8,6 +8,43 @@ from pathlib import Path
 from milc import cli
 
 
+def is_qmk_firmware(qmk_firmware):
+    """Returns True if the given Path() is a qmk_firmware clone.
+    """
+    paths = [
+        qmk_firmware,
+        qmk_firmware / 'quantum',
+        qmk_firmware / 'requirements.txt',
+        qmk_firmware / 'requirements-dev.txt',
+        qmk_firmware / 'lib/python/qmk/cli/doctor.py'
+    ]
+
+    for path in paths:
+        if not path.exists():
+            return False
+
+    return True
+
+
+def broken_module_imports():
+    """Make sure we can import all the python modules.
+    """
+    broken_modules = find_broken_requirements('requirements.txt')
+    broken_dev_modules = find_broken_requirements('requirements-dev.txt') if cli.config.user.developer else []
+    to_return = [False, False]
+
+    if broken_modules:
+        to_return[0] = True
+
+    if broken_dev_modules:
+        to_return[1] = True
+
+    for module in broken_modules + broken_dev_modules:
+        print('Could not find module %s!' % module)
+
+    return to_return
+
+
 def find_broken_requirements(requirements):
     """ Check if the modules in the given requirements.txt are available.
 
@@ -64,15 +101,12 @@ def find_qmk_firmware():
     return Path.home() / 'qmk_firmware'
 
 
-
 def in_qmk_firmware():
     """Returns the path to the qmk_firmware we are currently in, or None if we are not inside qmk_firmware.
     """
     cur_dir = Path.cwd()
     while len(cur_dir.parents) > 0:
-        found_lib = cur_dir / 'lib/python/qmk/cli/__init__.py'
-        found_quantum = cur_dir / 'quantum'
-        if found_lib.is_file() and found_quantum.is_dir():
+        if is_qmk_firmware(cur_dir):
             return cur_dir
 
         # Move up a directory before the next iteration

qmk_cli/script_qmk.py

@@ -4,6 +4,7 @@
 This program can be run from anywhere, with or without a qmk_firmware repository. If a qmk_firmware repository can be located we will use that to augment our available subcommands.
 """
 import os
+import shlex
 import subprocess
 import sys
 from platform import platform
@@ -11,8 +12,9 @@ from traceback import print_exc
 
 import milc
 import milc.subcommand.config  # noqa
+from milc.questions import yesno
 
-from .helpers import find_broken_requirements, find_qmk_firmware, in_qmk_firmware
+from .helpers import broken_module_imports, find_qmk_firmware, is_qmk_firmware
 
 milc.EMOJI_LOGLEVELS['INFO'] = '{fg_blue}Ψ{style_reset_all}'
 
@@ -24,14 +26,25 @@ def qmk_main(cli):
     cli.print_help()
 
 
+def run_cmd(*command):
+    """Run a command in a subshell.
+    """
+    if 'windows' in milc.cli.platform.lower():
+        safecmd = map(shlex.quote, command)
+        safecmd = ' '.join(safecmd)
+        command = [os.environ['SHELL'], '-c', safecmd]
+
+    return subprocess.run(command)
+
+
 # Python setuptools entrypoint
 def main():
     """Setup the environment before dispatching to the entrypoint.
     """
     # Warn if they use an outdated python version
-    if sys.version_info < (3, 6):
+    if sys.version_info < (3, 7):
         print('Warning: Your Python version is out of date! Some subcommands may not work!')
-        print('Please upgrade to Python 3.6 or later.')
+        print('Please upgrade to Python 3.7 or later.')
 
     if 'windows' in platform().lower():
         msystem = os.environ.get('MSYSTEM', '')
@@ -52,44 +65,46 @@ def main():
     import qmk_cli.subcommands
 
     # Check out and initialize the qmk_firmware environment
-    if qmk_firmware.exists():
+    if is_qmk_firmware(qmk_firmware):
         # All subcommands are run relative to the qmk_firmware root to make it easier to use the right copy of qmk_firmware.
         os.chdir(str(qmk_firmware))
 
         # Check to make sure we have all the requirements
-        broken_modules = find_broken_requirements('requirements.txt')
-        broken_dev_modules = find_broken_requirements('requirements-dev.txt') if milc.cli.config.user.developer else []
+        broken_modules, broken_dev_modules = broken_module_imports()
+        msg_install = 'Please run `python3 -m pip install -r %s` to install required python dependencies.'
 
-        if broken_modules or broken_dev_modules:
-            for module in broken_modules + broken_dev_modules:
-                print('Could not find module %s!' % module)
-
-            msg_install = 'Please run `python3 -m pip install -r %s` to install required python dependencies.'
-            if broken_modules:
+        if broken_modules:
+            if yesno('Would you like to install the required Python modules?'):
+                run_cmd(sys.executable, '-m', 'pip', 'install', '-r', 'requirements.txt')
+            else:
                 print()
-                print(msg_install % (qmk_firmware / 'requirements.txt',))
-
-            if broken_dev_modules:
+                print(msg_install % (os.environ['QMK_HOME'] + '/requirements.txt',))
                 print()
-                print(msg_install % (qmk_firmware / 'requirements-dev.txt',))
+                exit(1)
+
+        if broken_dev_modules:
+            if yesno('Would you like to install the required developer Python modules?'):
+                run_cmd(sys.executable, '-m', 'pip', 'install', '-r', 'requirements.txt')
+            else:
+                print()
+                print(msg_install % (os.environ['QMK_HOME'] + '/requirements-dev.txt',))
                 print('You can also turn off developer mode: qmk config user.developer=None')
+                print()
+                exit(1)
 
-            print()
+        # Environment looks good, include the qmk_firmware subcommands
+        sys.path.append(str(qmk_firmware / 'lib/python'))
 
-        else:
-            # Environment looks good, include the qmk_firmware subcommands
-            sys.path.append(str(qmk_firmware / 'lib/python'))
+        try:
+            import qmk.cli  # noqa
 
-            try:
-                import qmk.cli
+        except ImportError as e:
+            if qmk_firmware.name != 'qmk_firmware':
+                print('Warning: %s does not end in "qmk_firmware". Do you need to set QMK_HOME to "%s/qmk_firmware"?' % (qmk_firmware, qmk_firmware))
 
-            except ImportError as e:
-                if qmk_firmware.name != 'qmk_firmware':
-                    print('Warning: %s does not end in "qmk_firmware". Do you need to set QMK_HOME to "%s/qmk_firmware"?' % (qmk_firmware, qmk_firmware))
-
-                print('Error: %s: %s', (e.__class__.__name__, e))
-                print_exc()
-                sys.exit(1)
+            print('Error: %s: %s', (e.__class__.__name__, e))
+            print_exc()
+            sys.exit(1)
 
     # Call the entrypoint
     return_code = milc.cli()

qmk_cli/subcommands/__init__.py

@@ -2,5 +2,6 @@
 
 We list each subcommand here explicitly because all the reliable ways of searching for modules are slow and delay startup.
 """
-from . import clone
-from . import setup
+from . import clone  # noqa
+from . import env  # noqa
+from . import setup  # noqa

qmk_cli/subcommands/env.py

@@ -0,0 +1,30 @@
+"""Prints environment information.
+"""
+import os
+from pathlib import Path
+
+from milc import cli
+from qmk_cli.helpers import is_qmk_firmware
+
+
+@cli.argument('var', default=None, nargs='?', help='Optional variable to query')
+@cli.subcommand('Prints environment information.')
+def env(cli):
+    home = os.environ.get('QMK_HOME', "")
+    data = {
+        'QMK_HOME': home,
+        'QMK_FIRMWARE': home if is_qmk_firmware(Path(home)) else ""
+    }
+
+    # Now munge the current cli config
+    for key, val in cli.config.general.items():
+        converted_key = 'QMK_' + key.upper()
+        data[converted_key] = val
+
+    if cli.args.var:
+        # dump out requested arg
+        print(data[cli.args.var])
+    else:
+        # dump out everything
+        for key, val in data.items():
+            print(f'{key}="{val}"')

qmk_cli/subcommands/setup.py

@@ -9,6 +9,7 @@ from pathlib import Path
 from milc import cli
 from milc.questions import yesno
 from qmk_cli.git import git_clone
+from qmk_cli.helpers import is_qmk_firmware
 
 default_base = 'https://github.com'
 default_repo = 'qmk_firmware'
@@ -61,8 +62,18 @@ def setup(cli):
         exit(1)
 
     # Check on qmk_firmware, and if it doesn't exist offer to check it out.
-    if (cli.args.home / 'Makefile').exists():
+    if is_qmk_firmware(cli.args.home):
         cli.log.info('Found qmk_firmware at %s.', str(cli.args.home))
+
+    elif cli.args.home.exists():
+        path_str = str(cli.args.home)
+
+        if cli.args.home.name != 'qmk_firmware':
+            cli.log.warning('Warning: %s does not end in "qmk_firmware". Did you mean to use "--home %s/qmk_firmware"?' % (path_str, path_str))
+
+        cli.log.error("Path '%s' exists but is not a qmk_firmware clone!", path_str)
+        exit(1)
+
     else:
         cli.log.error('Could not find qmk_firmware!')
         if yesno(clone_prompt):
@@ -74,18 +85,17 @@ def setup(cli):
                 exit(1)
 
     # Offer to set `user.qmk_home` for them.
-    if cli.args.home != Path(os.environ['QMK_HOME']) and yesno(home_prompt):
+    if str(cli.args.home) != os.environ['QMK_HOME'] and yesno(home_prompt):
         cli.config['user']['qmk_home'] = str(cli.args.home.absolute())
         cli.write_config_option('user', 'qmk_home')
 
-    # Run `qmk_firmware/bin/qmk doctor` to check the rest of the environment out
-    qmk_bin = cli.args.home / 'bin/qmk'
-    doctor_cmd = [sys.executable, qmk_bin.as_posix(), 'doctor']
-
-    if cli.args.yes:
-        doctor_cmd.append('--yes')
+    # Run `qmk doctor` to check the rest of the environment out
+    doctor_command = [sys.argv[0], 'doctor']
 
     if cli.args.no:
-        doctor_cmd.append('--no')
+        doctor_command.append('--no')
 
-    subprocess.run(doctor_cmd)
+    if cli.args.yes:
+        doctor_command.append('--yes')
+
+    cli.run(doctor_command, capture_output=False)

release

@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/bash
 # Increment the verison number and release a new version of qmk_cli.
 #
 # Required packages: pip3 install bumpversion twine
@@ -27,9 +27,6 @@ rm -f dist/*
 bumpversion patch
 git push origin master --tags
 
-# For now we don't use flit
-#flit publish
-
-# Use setuptools until flit works with homebrew
+# Build and upload
 python3 setup.py sdist bdist_wheel
 twine upload dist/qmk-*

setup.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.0.38
+current_version = 0.0.46
 commit = True
 tag = True
 tag_name = {new_version}
@@ -23,7 +23,7 @@ dist-name = qmk
 license_file = LICENSE
 module = qmk_cli
 home-page = https://github.com/qmk/qmk_cli
-requires-python = >=3.5
+requires-python = >=3.7
 
 [entry_points]
 qmk = qmk_cli.script_qmk:main

setup.py

@@ -7,12 +7,12 @@ metadata = setup_cfg['metadata']
 
 if __name__ == "__main__":
     with open('README.md', encoding='utf-8') as readme_file:
-        long_description=readme_file.read()
+        long_description = readme_file.read()
     setup(
         name=metadata['dist-name'],
         description='A program to help users work with QMK Firmware.',
         entry_points={
-            'console_scripts': ['%s = %s' % l for l in setup_cfg['entry_points'].items()],
+            'console_scripts': ['%s = %s' % i for i in setup_cfg['entry_points'].items()],
         },
         license='MIT License',
         url=metadata['home-page'],
@@ -24,7 +24,7 @@ if __name__ == "__main__":
         long_description=long_description,
         long_description_content_type="text/markdown",
         packages=find_packages(),
-        py_modules = ['milc'],
+        py_modules=['qmk_cli'],
         classifiers=[
             'Development Status :: 3 - Alpha',
             'Environment :: Console',
@@ -38,16 +38,20 @@ if __name__ == "__main__":
             'Topic :: Software Development',
             'Topic :: Utilities',
         ],
-        requires_python=metadata['requires-python'],
+        python_requires=metadata['requires-python'],
         install_requires=[
             "appdirs",
             "argcomplete",
             "colorama",
+            "dotty-dict",
             "flake8",
+            "hid",
             "hjson",
-            "milc>=1.0.8",
+            "jsonschema>=3",
+            "milc>=1.3.0",
             "nose2",
             "pygments",
+            "pyusb",
             "yapf"
-        ],
+        ]
     )